#launchpad-dev 2009-08-24
 * wgrant awards lazr.restful the award for Most Useless Error Message.
<jml> I don't know, one of the Launchpad forms has "Invalid value"
<jml> is it worse than that?
<wgrant> Well, that at least tells you which field is wrong.
<wgrant> THis one doesn't.
<jml> oooh.
<wgrant> It's during the WADL generation, so the traceback is useless. It just tells you that there needs to be precisely one IEntry defined for None.
<wgrant> It actually means that the interface specified in a Reference isn't exported as an entry of its own.
<wgrant> I'd forgotten that this particular one wasn't, as it has no implementations that weren't already exported.
<spm> "bus error. core dump." <== what more do you need in an error message?
<wgrant> I have a branch that is going to exceed the 800 line limit. About 300 lines of it is repetitive addition of a new interface to lots of other interfaces, which could be reasonably split into a separate branch. Should I split it?
<lifeless> will it be easier to review?
<lifeless> [also, can you reduce the duplication somehow]
<wgrant> lifeless: It'll be easier to review, but a little pointless.
<wgrant> The duplication is irreducible.
<lifeless> thats a shame. why?
<wgrant> It's actually not adding a new interface, it's just shifting how the classes declare that they implement it.
<wgrant> Because lazr.restful has a One Interface Policy.
<wgrant> (it's IStructuralSubscriptionTarget, implemented by IProject, IProduct, IDistribution, IDistroSeries, IProductSeries, IDistributionSourcePackage)
<jml> wgrant, I think I've filed a bug about that error message.
<wgrant> jml: Ah, good.
<adeuring> good morning
<noodles775> Morning adeuring
<adeuring> hi noodles775!
<gmb> Morning, Launchpeeps.
<mrevell> Morning!
<bigjools> morning mrevell
<mrevell> hey bigjools
<wgrant> Morning mrevell, bigjools, gmb.
<bigjools> how's the bairn?
<bigjools> good evening wgrant
<wgrant> gmb: Can you have a look at my two questions on bug #297458?
<mup> Bug #297458: setting nominations on a bug via the API <api> <launchpadlib :Invalid> <Launchpad Bugs:In Progress by wgrant> <https://launchpad.net/bugs/297458>
<gmb> Sure
 * gmb looks
<gmb> wgrant: Maybe there's a reason not to do this that I can't think of at sub-10am on a Monday but: Why does ISeriesBugTarget need to have no attributes? Couldn't it just descend from IBugTarget?
<wgrant> gmb: An excellent idea.
<gmb> Damn, that's my excellent idea for the day.
<gmb> I was hoping to save it for the binary-blob processing bug. Oh well.
<wgrant> Heh.
<wgrant> That's an oldish one.
<gmb> wgrant: Yeah. Unfortunately, kernel crashes are now getting uploaded to Launchpad, which makes it happen more often.
<wgrant> gmb: Do you agree with my solution for the second?
<gmb> wgrant: I can't think of any reason not to do it, so go for it.
<wgrant> gmb: Thanks muchly.
<gmb> wgrant: np. I'll reply to the bug for the sake of everyone knowing what's going on.
<wgrant> gmb: I also noticed an oddity in IBug.addNomination: nomination for an obsolete distroseries is forbidden, but permitted for an obsolete productseries.
<wgrant> Any idea why that is?
<gmb> wgrant: Hmm. Not off the top of my head.
<gmb> wgrant: One of the registry people might know, but they won't be around for another few hours.
<wgrant> gmb: Thanks.
<wgrant> gmb: Does Registry own structural subscriptions? They still live in canonical.launchpad.
<wgrant> (and I want to export them)
<gmb> wgrant: I don't know, actually. But I remember intellectronica doing a lot of work on structural subs, so if you have any questions he might be able to help you.
 * bigjools basks is Ashes glory today 
<bigjools> s/is/in/
 * wgrant wondered when that would come up.
<bigjools> now I wonder if there's an Aussie around who I can wind up.... ah wgrant!
<wgrant> I'm so dedicated to cricket, I only heard the news through identi.ca this morning.
<noodles775> heh... what news?
<noodles775> ;)
<bigjools> don't shout, I have a hangover
<wgrant> bigjools: So, I cut up the librarian log parser on Saturday afternoon, and ended up with a working but broken in some cases PPA package download counter.
<wgrant> I'm not sure how to handle the ambiguous cases.
<danilos> stub: if I ask LOSAs to QA the modified langpack export script for long transactions, how could they tell if it's using long transactions or not?
<stub> danilos: There is a graph of the longest running transaction on the system. They can monitor that, or that can query pg_stat_activity (select current_timestamp - xact_start as age from pg_stat_activity where usename='langpack')
<danilos> stub: cool, thanks (I'd be doing it on staging before asking for a CP, so this helps)
<bigjools> wgrant: great!  I don't really have time to look at it right now, but maybe later this week.
<wgrant> bigjools: Sure.
<huangjs> Is it possible to make Launchpad work with GIT?
<bigjools> huangjs: it can import git branches
<huangjs> bigjools: i see. is there a way to do the importing periodically?
<bigjools> huangjs: yes
<bigjools> https://help.launchpad.net/Code/UploadingABranch#Mirroring%20a%20branch%20that%27s%20hosted%20elsewhere
<bigjools> and https://help.launchpad.net/Code/Imports
<huangjs> bigjools: thanks!
<bigjools> welcome
<thumper> night lp hackers
<danilos> sinzui: hey, around? I'd like to talk some more about the IHasLogo generation; I had to do a similar fix to what thumper-zzz did, but in a different method
<noodles775> danilos: can I see your MP? Interested to know why you couldn't use the IRootContext that thumper added? (in browser/watermark.py)
<noodles775> or was it not related to the watermark logo?
<danilos> noodles775: hum, I am using IRootContext
<danilos> noodles775: read through "Implementation details" on https://code.edge.launchpad.net/~danilo/launchpad/translator-templates/+merge/10546
<noodles775> danilos: Thanks!
<danilos> noodles775: it's mostly about very similar methods, which is a bit weird
<noodles775> oh danilos, btw, the bzr issue I (thought) I had the other day was just because I was using pqm-submit before the branch had actually been scanned (hence not recognising the format).
<danilos> noodles775: ah, right, nice to know
<rockstar> sinzui, ping
<sinzui> Hi rockstar
<beuno> noodles775, salgado, what did we agree for the breadcrumbs UI?
<noodles775> beuno: that we don't need to keep the existing formatting, just add the new formatting (using a template rather than inline)...
<beuno> noodles775, and is that on your plate now?
<noodles775> unconverted pages will just have the new breadcrumbs in the wrong place.
<noodles775> beuno: if that's ok with bigjools, I'm happy to do it.
 * beuno looks at bigjools with a sad face
<noodles775> lol
<beuno> pleeeeeeeeeeeeeeeeease
 * bigjools is on the phone
<bigjools> will read back in a bit
<beuno> bigjools, just type "yes"
<beuno> (and hit enter)
<beuno> no need to think
<bigjools> I iz back
<bigjools> beuno, noodles775: how much work?
<beuno> I estimate about 10 minutes
<beuno> but lets see what noodles775 has to say
<noodles775> bigjools: it *should* be very straight-forward...
<bigjools> "yes"
 * bigjools hits enter
<noodles775> heh, not 10 minutes, but yes, small. :)
 * beuno has a rare dyslexia that doesn't see quotes
<beuno> YAY
<beuno> thank you bigjools, noodles775
<noodles775> np!
<beuno> with this, my evil Launchpad 3.0 navigation plan is all in motion
<bigjools> beuno: I think Curtis' team will need some evil navigation help with +related-software
<beuno> bigjools, sinzui has a 24/7 private phone line directly to me
<sinzui> bigjools: We wont know that until we work out what that page and all its many incoming links do
<bigjools> beuno: like I said, you have a big red phone, and I shall call you Commissioner Gordon
<beuno> bigjools, I've been called worst
<bigjools> sinzui!  I knew you'd still be online on your vacation
<bigjools> :D
<beuno> I thought sinzui was just taking a vacation so he could do *more* UI
<beuno> :)
<bigjools> he said his self-esteem was proportional to the number of branches he lands
<sinzui> I'm actually trying to fix a bug in my editor's plugins
<beuno> sinzui, gedit mis-behaving?
 * sinzui is tempted to hack on the milestone page because there are secret features he wants to add.
<sinzui> My plugin does not build on jaunty.
<sinzui> I want a PPA that provides the project, bzr, and lp tools I use.
<beuno> sinzui, ah, the gedit-bzr plugin
<beuno> it's pretty unmaintained, no?
<sinzui> Mine uses bzr-gtk. I did not want to reinvent other people's work
<bigjools> I'd really like kdevelop to support Bzr and Python  better
<sinzui> bigjools: I thought kdevelop had great support for plugins/extensions
<bigjools> I don't remember, I last used it 3 years ago for C++
<rockstar> abentley, it's intentional that bzr add-pipe --before and bzr add-pipe --after are different, right?
<rockstar> Er, that the pipe that's created is different.
<abentley> rockstar: possibly.  The pipe that's created by add-pipe --before FOO should be the same revision as the pipe that's created by add-pipe --after FOO.
<abentley> But that may be too clever.  Maybe those options should only control position, not revision.
<rockstar> abentley, if you do --before, it actually creates with the same revno the :next, and --after has the same revno as :prev
<beuno> intellectronica, noodles775, rockstar, EdwinGrubbs, barry, jtv, mrevell, ajax call in 11'!
<rockstar> beuno, 11 feet?
<abentley> rockstar: Right, like I said above.
<beuno> rockstar, yes. We are meeting above your house
<beuno> did you now get the email?  :)
 * rockstar gets the ladder.
<barry> beuno: right! thanks
<abentley> rockstar: https://code.edge.launchpad.net/launchpad/+activereviews seems to have become a lot less useful.  Do you know what happened to "reviews I can do", etc?
<rockstar> abentley, I know thumper was talking about doing something, but I don't know if he changed it.
<rockstar> abentley, actually, it looks like there aren't any you CAN do.
<abentley> rockstar: That gets the "other" header, though.
<rockstar> abentley, the OTHER is for stuff that you could have done, but someone has already gotten to it.
<abentley> rockstar: Roughly, yes.  Where is it?
<sinzui> bac: can you update or unassign bug 175448 and bug 187312
<mup> Bug #175448: RSS icon in FF url bar although no announcement is published yet <project-announcements> <Launchpad Registry:Triaged by bac> <https://launchpad.net/bugs/175448>
<mup> Bug #187312: Standalone global announcements page doesn't seem useful <project-announcements> <Launchpad Registry:Triaged by bac> <https://launchpad.net/bugs/187312>
<bac> yep
<barry> beuno: am i on the right c/c #?
<sinzui> barry: https://edge.launchpad.net/distros looks great
<intellectronica> barry: if you're not hearing muzak then you're not
<barry> sinzui: thanks!
<barry> intellectronica: that's all i'm hearing.  i am rocking to it, but still :)
<beuno> barry, kiko's
<barry> beuno: bueno!
<sinzui> barry: what is the criteria for the * read-only on the distros page?
<barry> sinzui: otp
<rockstar> abentley, do you not see "Other reviews" ?  I do.
<rockstar> abentley, also, sorry for the delay; I'm on a phone call.
<abentley> I don't see it.
<rockstar> EdwinGrubbs, https://devpad.canonical.com/~mars/conversions.html
<rockstar> abentley, that's odd.
<barry> abentley: enable google.com in noscript
<abentley> barry: I have all sites enabled in noscript.
<barry> abentley: dunno. that wfm ;)
<rockstar> abentley, and I assume you're logged in.
<abentley> barry: I disabled the noscript extension, and still no workie.
<abentley> rockstar: I am logged in.
<beuno> rockstar, what are you doing about the major pages?
<beuno> converting first and then designing?
<beuno> there's no dsign for MP, for example
<beuno> I can
<beuno> cool
<beuno> question for everyone:
<beuno> how's the UI reviewing stgoing?
<beuno> with all the new people?
<beuno> yes, I have a bunch of tasks from our last call
<beuno> sorry barry  :)
<beuno> I've been sprinting last week
<abentley> gary_poster: I have a branch that adds bzr 1.18rc1 to bzr+ssh://bazaar.launchpad.net/%7Elaunchpad/lp-source-dependencies/trunk/.  What's the procedure to get that merged?
<gary_poster> abentley: in the abstract, usually I like to test that the distribution does in fact work with launchpad (ec2test now has some flags to help).  then you just add it.  as long as you are not overwriting anything, everything is AOK.
<abentley> gary_poster: I thought there was no risk to adding 1.18rc1 if versions.cfg does not reference it.
<gary_poster> abentley: no there's not, I just am trying to be clean (since everybody has to download what's in the cache).  So that's my only consideration.
<abentley> gary_poster: I have a branch that updates Launchpad so it works with 1.18rc1, but I want to land that after landing the download_cache change, because it's not compatible with 1.17.
<gary_poster> abentley: +1 .
<abentley> gary_poster: cool.
<barry> beuno: i got kicked off
<beuno> barry, the conference calls can only handle a certain level of awesomeness
<abentley> gary_poster: lp:launchpad/lp-source-dependencies/trunk is currently in pack-0.92 format.  We should look at switching it sometime.
<gary_poster> abentley: ack.  will that be smooth if people have a check out rather than a branch, and then do bzr up after the lp repo has switched?
<abentley> gary_poster: No.  It would only be smooth if they had a lightweight checkout.
<gary_poster> abentley: ok.  I'll make an lp bug with the info.  maybe we can make things simpler using rocketfuel-get.  can you bzr upgrade a checkout after you did it for the branch, or is there some other relatively simple fix other than "wipe out the directory and get it again"?
<abentley> gary_poster: You can bzr upgrade the checkout before or after the lp branch is upgraded.
<abentley> gary_poster: (but if you do it before, you won't be able to commit)
<gary_poster> abentley: ah ok, awesome.  thanks
<abentley> gary_poster: np
<danilos> salgado: hey
<salgado> hi danilos
<danilos> salgado: I am trying to add breadcrumbs for Translation groups (translations.launchpad.dev/+groups) page, but it seems not to respect canonical_url() and returns just launchpad.dev/+groups
<danilos> salgado: patch at http://pastebin.ubuntu.com/258759/
<danilos> salgado: (and thanks for getting rid of the limitation that '+something' can't have a breadcrumb, I was hitting that on Friday ;)
<salgado> danilos, you need to set rootsite='translations' on your custom breadcrumb class
<danilos> salgado: ah, ok, I thought having rootsite in zcml was enough
<danilos> salgado: I am wondering what's the reason though? basically, we have no rootsite set on most of our browser:urls in zcml, but we should; is this a work-around to fix such cases, or intentional?
<salgado> danilos, it should be enough. can you file a bug about that?
<danilos> salgado: certainly
<salgado> danilos, it's not really a workaround. what happens is that we need to use canonical_url(object, rootsite='mainsite') when getting the URL for the breadcrumbs, or else all URLs will be under the vhost we're currently on
<salgado> (it is a workaround but for another issue)
<danilos> salgado: except those specified in zcml, right?
<danilos> salgado: I might be mistaken about how canonical_url works though
<salgado> but for objects which already specify the rootsite in the zcml, that should be used instead of 'mainsite'
<danilos> salgado: ah, right, this is for 'point people at the mainsite whenever possible' workaround
<salgado> yes
<danilos> salgado-lunch: fyi, bug 418214
<mup> Bug #418214: Breadcrumbs code doesn't respect rootsite specified in zcml <Launchpad Foundations:New> <https://launchpad.net/bugs/418214>
<rockstar> barry, I just got spam from Barry Warshaw.
<rockstar> abentley, am I being lied to when I push a branch and it says "Source branch format does not support stacking" even though I'm in Branch format 7?
<abentley> rockstar: yes, Branch format 7 supports stacking.
<rockstar> abentley, okay, so it's a bug in codehosting then? I remember there being a bug, but I thought that it had been fixed.
<abentley> rockstar: It's a bug in bzr.
<rockstar> abentley, otay.
<mrevell> okay guys, see you tomorrow
<leonardr> barry, can you help me with some tricky python?
<barry> leonardr: sure
<leonardr> barry: https://pastebin.canonical.com/21438/
<leonardr> when i try to use this code i get TypeError: 'classmethod' object is not callable
<barry> leonardr: line 43 looks weird; a return outside of a function?
<leonardr> it is in a function, the WebServiceWSGIApplicationFactory function
<barry> oh!  i see that's a def
<barry> i don't think you can use @classmethod outside of a class definition
<barry> well, maybe you can, i've just never seen it done
<leonardr> oh, i think i know what it is. i forgot to add configure_server to the dict
<leonardr> no, no change
<leonardr> barry: i was able to use @property outside of a class definition
<barry> leonardr: where do you get the type error?
<leonardr> barry: got it. i should call cls.configure_server instead of configure_server
<barry> leonardr: yep.  @classmethod takes a class as the first arg.
<rockstar> sinzui, yo
<cprov> barry: ping
<barry> cprov: otp, sec...
<cprov> barry: okay, ping when you are done. Is there a way to get `ulimit -n` value elegantly in python ?
<maxb> cprov: python -c "import resource; print resource.getrlimit(resource.RLIMIT_NOFILE)"
<barry> maxb: you beat me to it :)
<cprov> maxb: yup, just found the resource module too :)
<cprov> however, I think i need to get the number of currently open files. Anyone ?
<cprov> btw, resource module is gold!
<salgado> abentley, the links under the "Branches related to" section of https://code.edge.launchpad.net/~sinzui point to lp.net rather than code.lp.net.  shall I file a bug for that?
<abentley> salgado: For me, they point to code.edge.launchpad.net
<salgado> abentley, the links to the teams at the bottom of the page?
<abentley> salgado: Oh, that.
<abentley> salgado: I dunno, I like person links to take me to the person's homepage.  I don't know about team links.
<beuno> all team/people/project/distro/etc links should take you to their overview page
<salgado> abentley, that's how they should work, for people and teams, but the heading there implies that following the links will show me the branches related to those teams
<salgado> beuno, ^
<beuno> sinzui did that change
<abentley> salgado: I don't think that going to the code view for, e.g. "Launchpad API Hackers" would be especially useful.
<salgado> even less so if you go to the team's home page, as there will certainly be no branches related to the team there
<beuno> salgado, I agree that's an odd case, so maybe that listing needs to have a different format
<beuno> that part of the page needs thought and re-design
<salgado> ok, I'll file a bug
<abentley> salgado: I'm not even sure what "Branches related to" is meant to mean.  I think it means that sinzui has created branches related to Launchpad API Hackers, for example.
<beuno> the behavior is correct though
<abentley> salgado: So showing all the branches for Launchpad API Hackers wouldn't connect.
<abentley> salgado: You might argue that it should show all the branches sinzui created which were related to Launchpad API Hackers.
<abentley> salgado: Is there any reason to keep the import of canonical_url in hierarchial-menu.txt?
<salgado> abentley, no, I've removed it after I sent that MP
<abentley> salgado: Okay.
<abentley> salgado: I think Heirarchy.items would be simpler if you used enumerate.  Do you agree?
<salgado> abentley, indeed, good catch
<abentley> salgado: Other than that, looks great.  r=me.
<salgado> cool, thanks
<sinzui> salgado: We are redesigning the distroseries +index page, we should not work on that without also working on the productseries +index page. They should be very similar
<salgado> sinzui, where do I see the list of pages that we are redesigning?
<sinzui> I cannot find it. beuno put it in a funny place
<sinzui> https://dev.launchpad.net/UI/ThreeDotOPages
<sinzui> ^ I am going to link it to something
<sinzui> salgado: I updated https://dev.launchpad.net/UI/ThreeDotOPage to make it clearer
<salgado> thanks sinzui
<salgado> sinzui, so, just to clarify, should I work on another item and wait until you're back to take distroseries?
<sinzui> salgado: I would prefer to do the series myself. I have a secret agenda.
<salgado> ok, it's all yours then. :)
<sinzui> salgado: I use the series and milestones pages as a tool to do my job. I have a lot of small changes I what to make to make them rock
<barry> can i scrounge up a ui review and a code review from anybody?
<beuno> sinzui, if you're around, where can I poke at the 3.0 template?
<beuno> the footer, specifically
<sinzui> beuno: lp/app/template/base-layout-macros.pt
<beuno> sinzui, thanks
<sinzui> beuno: or lp/app/template/base-layout.pt if something else needs fixing
 * beuno updates it to 2.2.8
<barry> beuno: wanna do some screen shot ui reviews?
<beuno> barry, sure
<barry> beuno: excellent.  coming up
<barry> beuno: the following urls have screenshots.  note that the +all pages are the new ones
<barry> http://devpad.canonical.com/~barry/projectgroups-index.png
<barry> http://devpad.canonical.com/~barry/projectgroups-all.png
<barry> http://devpad.canonical.com/~barry/projects-index.png
<barry> http://devpad.canonical.com/~barry/projects-all.png
<barry> http://devpad.canonical.com/~barry/sprints-index.png
<barry> http://devpad.canonical.com/~barry/sprints-all.png
 * beuno loos
<barry> beuno: EOT
<beuno> matsubara, ping
<matsubara> hi beuno
<beuno> matsubara, hi
<matsubara> how can I help you?
<beuno> I wanted to ask you about the impact of doing something on every page load
<beuno> config.launchpad.beta_testers_redirection_host is not None and self.isBetaUser)
<matsubara> beuno, I'm not sure about the impact of those. I think that check is quite cheap
<matsubara> beuno, what do you have in mind?
<beuno> matsubara, I want to show the edge redirect message on the footer
<matsubara> beuno, you mean the one shown in launchpad.net?
<beuno> barry, the only comment I have is that maybe the sprint dates could say "$fromdate to $todate"
<matsubara> beuno, I did something similar for the oops timeout error page which is javascript only
<Ursinha> rockstar, I think buildbot didn't like your branch
<beuno> barry, istead of $from - $to
<Ursinha> rockstar, but it seems a bzr hiccup
<beuno> matsubara, shown for betatesters, yes
<barry> beuno: i could try to clean that up, but i don't know where that's set (i.e. not in code i touched).  thanks, and let's see if i can fix that
<beuno> barry, ui=beuno
<barry> thx!
<matsubara> beuno, cool. see launchpad-requestexpired.pt for the js only I used in the timeout oops template
<beuno> matsubara, it doesn't have a @cachedproperty around it
<beuno> matsubara, this is not so much about the js, as it is on adding that call to every page load
<barry> beuno: easy peasy.  done.
<beuno> barry, bring that count down then!
<barry> beuno: i need a code review which looks like it'll have to wait until tomorrow :(
<beuno> matsubara, so, would you say it's OK to do the change?
<beuno> I don't want to add 200ms to every page load  :)
<matsubara> beuno, you can always remove it if it does :-)
<beuno> matsubara, how do I find out if I did?
<matsubara> beuno, but I don't understand why it would  be useful to have that on all page loads? do people want to leave the beta site as often as that?
<beuno> matsubara, well, yes and no
<beuno> we have a request from a super special stakeholder to move that to the footer on the homepage
<matsubara> perhaps we could have a way to disable/enable edge's redirect for longer than 2h
<beuno> my thought is, if I'm going to do that, I may as well be consistent
<matsubara> I see
<matsubara> beuno, I think that people already complain that LP is slow and adding more stuff to all page loads, that won't really benefit everyone is not a such a good idea, IMHO.
<beuno> matsubara, I feel the same
<barry> okay, i'm off to give my lazr.restful talk for real this time!
<matsubara> beuno, and the best person to discuss this is mars as he was working on profiling LP pages, IIRC
<beuno> :/
<beuno> matsubara, kiko seems to think we already do that check
<beuno> is there anyway we can measure this?
<beuno> can you help me?  :)
<matsubara> beuno, I'll look into it and get back to you tomorrow. I don't know from the top of my head if/how to measure that
<matsubara> hmmm
<matsubara> actually there's a way
<matsubara> the ++oops++ page tim implemented a few cycles ago
<matsubara> anyway, thinking out loud. I'll investigate and get back to you
<jml> good morning
<kfogel> jml: hey!
<jml> kfogel, hello :)
<lifeless> jml: hi hai
<rockstar> Ursinha, has someone testfixed?
<Ursinha> rockstar, nope, but gary is guiding me to do that
<Ursinha> I need to learn :)
<rockstar> Ursinha, okay, great.
<rockstar> Ursinha, why do you think it was me specifically?
<Ursinha> rockstar, it failed when trying to bzr co your branch
<Ursinha> it seems
<gary_poster> yeah, that's it.  the cosmic rays fell on your name this time, rockstar
<rockstar> gary_poster, yeah, I always land through ec2 just to be safe, so I was getting ready to throw a shit fit.  :)
<rockstar> gary_poster, as long as it's something spurious with failbot, I'm okay.
<Ursinha> rockstar, it seems bzr co timed out
<rockstar> Ursinha, it looks like it timed out checking out devel.
<gary_poster> lol
<rockstar> https://lpbuildbot.canonical.com/builders/lp/builds/41/steps/bzr/logs/stdio
<rockstar> Ursinha, so it looks like it had nothing to do with any of our branches.  :)
<Ursinha> rockstar, *whew*
<Ursinha> :)
<Ursinha> rockstar, yeah, I saw that
<rockstar> Ursinha, someone needs to get buildbot some depends.  It craps itself far too often.
<rockstar> I guess that's what mwhudson signed up for.  :)
<mwhudson> hmmmmmmmmmmmm?
<thumper> mwhudson: welcome back
<mwhudson> thumper: hi
<rockstar> mwhudson, I was just commenting on the fact that buildbot timed out trying to branch devel, and Ursinha has been blaming me.
<mwhudson> rockstar: i see
 * mwhudson will ignore all this until the email mountain is a little bit under control
<Ursinha> lol
<Ursinha> hey mwhudson :)
<Ursinha> rockstar, actually I was needing someone to kick buildbot :)
<Ursinha> but doing that now
<rockstar> mwhudson, by the way, welcome back newlywed.
<mwhudson> rockstar: thanks
<jml> mwhudson, hello!
<jml> mwhudson, are you back?
<mwhudson> jml: i am!
<mwhudson> jml: it seems you are too
<jml> mwhudson, welcome back!
<mwhudson> jml: did you just get back today?
<jml> mwhudson, yesterday
<mwhudson> jml: welcome back to you too
<jml> :D
<mwhudson> jml: where have you been in the interim?
<jml> mwhudson, Dublin, Prague, Berlin, Paris
<jml> and Heathrow :P
<mwhudson> jml: cool
<rockstar> thumper, stand up?
<thumper> yes
<lifeless> jml: hows the lag?
<jml> lifeless, ok ish
<jml> lifeless, I woke up at 5:30am this morning :)
 * mwhudson woke up at 4:50 am this morning
<lifeless> jml: heh. The wind woke me at 4
<lifeless> jml: OTOH its nearly lunchtime :)
<jml> heh
<wgrant> beuno: Are you touching requestexpired.pt, or just all of the other notices?
<wgrant> 'cause there's a bit of inverted logic in that template.
<beuno> wgrant, root-index and base-layour-macros
<wgrant> beuno: Ah, damn. (bug #403863)
<mup> Bug #403863: Timeout edge redirect notice logic inverted <Launchpad Foundations:New> <https://launchpad.net/bugs/403863>
<beuno> wgrant, I'm way in too deep with this branch
<beuno> ain't touchin that  :)
<rockstar> thumper, can you check and see if mars conversion table thing for code is reporting the work you've done recently?
<Ursinha> hey beuno
<beuno> hey Ursinha
<Ursinha> hi beuno
<Ursinha> I'm migrating the template of language pages, but not sure which layout to choose
<Ursinha> beuno, like this one, https://translations.edge.launchpad.net/+languages/sr
<Ursinha> beuno, danilo showed me a picture of a whiteboard, he said yours, https://devpad.canonical.com/~danilo/pages/language.jpg
<wgrant> sinzui: So, I'm exporting structural subscriptions. Do you have time to talk a bit about that?
<sinzui> I do
<wgrant> sinzui: Great.
<wgrant> sinzui: I'm thinking that for now the bug/blueprint levels shouldn't be exposed, since only one value is used and the rest will probably change before any others come into use.
 * sinzui nods
#launchpad-dev 2009-08-25
<thumper> rockstar: it is
<wgrant> sinzui: It looks like it should also be split into three branches: 1) Making all of the target interfaces descend from IStructuralSubscriptionTarget, rather than the model classes implements()ing it directly. This is because lazr.restful will only export one interface for each entry.
<wgrant> 2) URL/traversal for ISS
<wgrant> 3) The actual export.
<rockstar> thumper, awesome.
<wgrant> The first is 300ish lines, the second about 200.
<sinzui> wgrant: 1) agreed. This is a common practice to export
<sinzui> 2) barry had some trouble with that last month he landed a change to lazr.restful to make some kinds of object that are not traversable work. The behaviour depends on where the collection has an implied URL.
<sinzui> 2) barry's work may not be relevant, Do you imagine something like
<sinzui>     /ubuntu/karmic/mozilla-firefox/+subscriptions
<wgrant> sinzui: I've used +subscription, rather than +subscriptions. There's inconsistency throughout LP as to whether the plural is used or not.
<maxb> !?!?!?!??! wgrant: One of those mystery test failures of mine no longer fails (the package-diff.txt one)
<wgrant> maxb: Launchpad must like you now that you've added Karmic to the ~launchpad PPA.
<maxb> heh
<wgrant> sinzui: <target>/+subscription/<person> is the scheme.
<sinzui> that is good
<sinzui> wgrant: We have not moved structural subscriptions. were to you think they belong? lp/app or lp/registry?
<wgrant> sinzui: Registry.
<sinzui> good
<wgrant> The only other non-obvious changes are making ISST.removeBugSubscription take an unsubscribed_by, and implementing access control in that and addBugSubscription.
<wgrant> sinzui: Should I move them?
<sinzui> wgrant: you can if you like. There are tools to do it. I think exporting is more valuable than moving
<wgrant> sinzui: I'll just export for now, then. Thanks.
<sinzui> wgrant: I want to move it to registry and add answers and code. I think the move can wait since I am pushing to emphasis subscription and notification in 4.0
<wgrant> maxb: What about the other mysteeeeerious failurs?
<wgrant> sinzui: XMPP! XMPP!
 * sinzui high five's wgrant
<wgrant> Difficult, but rather useful and impressive.
<mwhudson> would be awesome
<maxb> wgrant: the mystery soyuz DONE != ACCEPTED one persists, unfortunately
<wgrant> maxb: Huh.
<maxb> My thoughts exactly :-)
<wgrant> sinzui: How would I best test the traversability of ISS? There are no views for it and no webservice export in this branch.
<sinzui> canonical_url to verify you get the expected url
<sinzui> wgrant:I think you want to test Navigation
 * sinzui think about how we test that
<wgrant> sinzui: Do I add the tests to canonical_url_examples.txt, or is there somewhere better?
<sinzui> I think the right place for this kind of test is a unit test in browser.test.
 * sinzui looks for a traversal test
<wgrant> There's always notfound-traversal.txt!
 * wgrant runs.
<wgrant> Bug c.l.browser.tests.test_branchtraversal looks like a reasonable example.
<wgrant> s/Bug/But/
<sinzui> That looks like the best example. There is some traversal testing in potemplate-views.txt too
<jml> thumper, ping
<thumper> jml: heading to class, back in a couple of hours
<jml> thumper, ok. please let me know when you return.
<thumper> akc
<jml> thanks.
<lifeless> jml: you might enjoy https://code.edge.launchpad.net/~lifeless/bzr/test-speed/+merge/10633
 * jml looks
<jml> hey, result decorators
<jml> I know them
<lifeless> so this is a bit of a bug in the composite, that you can't cleanly intercept code around a test
<lifeless> I'm accumulating a list of such defects
<jml> cool
<jml> I think that's a good idea
<lifeless> 9 so far:)
<jml> lifeless, have you followed much the recent compatibility work in py.test?
<lifeless> not at the code level
<jml> I've only scanned the emails.
<jml> lifeless, that patch does look quite useful.
<lifeless> jml: yes :)
<lifeless> bzr's selftest selftests take 5 seconds now
<lifeless> I have plans to make that lower
<wgrant> Can somebody please ec2test and land https://code.edge.launchpad.net/~wgrant/launchpad/descend-from-istructuralsubscriptiontarget/+merge/10636?
<wgrant> (it's already approved)
<jml> wgrant, sure
<jml> wgrant, commit message?
<jml> mwhudson, ping
<wgrant> jml: Never been asked for one before! "Inherit interfaces from IStructuralSubscriptionTarget, rather than declaring implementation in model classes.", I suppose.
<jml> wgrant, thanks. I've kicked off the run, will let you know when it goes headless
<wgrant> jml: Thanks. Will I be emailed when it's done?
<jml> wgrant, yep
<wgrant> jml: Even better.
 * wgrant finishes writing tests for the other two bits.
<mwhudson> jml: hello
<jml> mwhudson, would you like to talk about things?
<mwhudson> jml: sure, is in about 5 minutes time about right?
<jml> mwhudson, that'll be fine
<maxb> Where were launchpad-dependencies packages kept between the time they were removed from dapper, and when they started being built in the ~launchpad PPA?
<maxb> I've not had any response to my question about rescuing a branch, so maybe if those source packages are stored somewhere, fishing it out from there would be best
<jml> maxb, I can't remember, but I'm guessing on some Canonical-internal server
<jml> maxb, what's your 'rescuing a branch' question?
<maxb> "[Launchpad-dev] A bzr branch for launchpad-dependencies?"
<maxb> https://lists.launchpad.net/launchpad-dev/msg00562.html
<jml> maxb, ta
<maxb> I could just ignore that period and import what's already public, but it seems silly not to ask first
<wgrant> They were on lpdebs.canonical.com, which is devpad.
<wgrant> There are versions available back to 2008 there, but probably no branch.
<maxb> The versions in dapper actually had a branch in the source packages
<mwhudson> jml: ready now?  (that was a cook islands 5 minutes, btw)
<jml> mwhudson, sure
<jml> wgrant, it's gone headless.
<wgrant> jml: What does that mean? It started, checked out OK, and is now actually testing?
<jml> wgrant, exactly
<jml> wgrant, specifically, it's out of my hands now :)
<wgrant> jml: Thanks.
<rockstar> Agh!  Test failures...  *grumble*
<thumper> jml: back
<jml> thumper, h
<jml> i
<jml> thumper, up for a quick call
<thumper> jml: on with mwhudson right now
<jml> thumper, ok. please let me know when you're done.
<thumper> ok
<jml> brb
<thumper> jml: ping
<jml> thumper, hi
<jml> thumper, I'll put on my cloak and wizard hat
<thumper> jml: ready when you are
 * thumper doesn't get it
<jml> skype won't start :(
<jml> thumper, I'll have to reboot. please bear with me.
<thumper> ok
<jml> :(
<mwhudson> thumper: oh, i meant to ask, do we have any imminent sprints/other trips planned?
<thumper> mwhudson: not yet
<mwhudson> k
<thumper> mwhudson: I'm thinking of one perhaps after the TL one
<thumper> so not until at least October
<mwhudson> ok
<thumper> jml: still not working?
<jml> thumper, skype appears to be broken in karmic at present.
<thumper> :(
<thumper> jml: it was working yesterday
<jml> thumper, yesterday I hadn't updated in 2 weeks :)
<thumper> :(
<mwhudson> i'm busy on the 3rd of october ( http://www.organicbeer.co.nz/events.html :) ) but i guess you wouldn't want it _straight_ after the TL sprint anyway
<thumper> no
<thumper> I'm flying then anyway
<mwhudson> does karmic+1 have a 1) name or 2) a uds location/time yet?
<wgrant> 1) No. 2) Mid-November.
<wgrant> The exact answer to 2) is on the Karmic and Launchpad release schedules.
<mwhudson> wgrant: do you know anything about a location ?
<wgrant> mwhudson: No.
<wgrant> They were very quiet about it this time.
<wgrant> We knew about Barcelona in Mountain View.
<elmo> it'll be in the US somewhere
<elmo> we haven't actually got a final location, which is why it hasn't been announced
<wgrant> Ahaha.
 * wgrant checks and rechecks the clock.
 * mwhudson is slightly confused by the dates on https://wiki.ubuntu.com/KarmicReleaseSchedule
<mwhudson> || 30 || November 19th || ||  || Karmic+1 Developer Summit ||
<mwhudson> the 19th is a thursday
<wgrant> Right.
<wgrant> Ubuntu releases are on Thursdays.
<mwhudson> ah
<mwhudson> i guess the uds will (probably) be the week containing that thursday then
<wgrant> Yes.
<mwhudson> mind you, no idea if i'd be going
<thumper> mwhudson: want to?
<mwhudson> thumper: depends on the dates :)
<thumper> :)
<wgrant> What is happening with branch statuses?
<jml> so
<wgrant> It seems to me like that column in the listings would be much better filled with the MP status.
<jml> wgrant, not much. there's a general trend toward killing them.
 * jml returns to his other point
<thumper> well...
<thumper> we tried
<thumper> but people complained
<jml> I'd like a desktop application that:
<jml>   a) registers me a launchpad account
<jml>   b) sets up my GPG key
<jml>   c) sets up an SSH key
<jml> and by 'desktop', I might actually mean 'command line bzr plugin'
<jml> but I might also mean 'GTK thing with fat buttons'
<thumper> jml: what about a "quickly" application :)
<jml> I'm not sure, and I want to make this happen somehow...
<jml> thumper, not a bad idea
 * wgrant would prefer 'bzr lp-register'
<jml> thumper, have we loaded 'Mature' with any behavioural consequences?
<thumper> nope
<thumper> the only consequence is marking branches merged in the scanner
<thumper> and changing merged branches to development on new revisions
<thumper> we don't change "special" branches, those linked to things
<thumper> as in series
<wgrant> This could do with documentation...
<thumper> but we don't have any special meaning with experimental or mature
<wgrant> The second bit in particular is unobvious.
<thumper> wgrant: we could do with much more documentation on lots of areas
<wgrant> thumper: But there aren't too many areas where there is a script silently and secretly doing magic behind the scenes.
<wgrant> Magic that hides and unhides my objects.
<jml> wgrant, I agree.
<jml> it's kind of a misfeature in a way
<wgrant> Particularly as there's no email notification.
<jml> I mean, we shouldn't mark series branches as merged
<thumper> jml: we don't
<thumper> wgrant: there will be soon
<jml> but we probably shouldn't make linking a branch to a series the only way of stopping it from being automatically marked as merged
<wgrant> thumper: Ah, excellent.
<wgrant> thumper: I'm not even emailed when my merge proposal is automatically closed...
<thumper> wgrant: that is the main email that will be happening soon
<thumper> wgrant: we should also do the branch one
 * thumper goes to chop food, bbl
 * mwhudson eods
<jml> mwhudson, ciao
<wgrant> jml: Did something go wrong, or is PQM just slow?
<jml> wgrant, not sure.
<jml> wgrant, pqm hasn't emailed me...
<jml> but it says it's not doing anything
<wgrant> Hmmm.
<jml> where do I file bugs about mailing lists?
<lifeless> bugs  - launchpad
<lifeless> operation issues, answers
<jml> thanks.
<wgrant> launchpad-registry
 * jml turns his attention back to pqm
<jml> wgrant, I don't know what went wrong. I'll submit directly to PQM and see what happens
<wgrant> jml: Thanks.
<jml> hmm. I can't figure out how to do this without fetching the branch to local disk
<wgrant> This often seems to be a problem.
<wgrant> Some manage to do it remotely. Some push it up under their own user on LP. It's all a mystery to me.
<lifeless> jml: patch time
<lifeless> the way others do it is to submit --dry run then manually create the job
<jml> ohhhhhhhh
<lifeless> the next time I need to do it I'll patch the submit tool to do it for me
<jml> wgrant, what happened was that I changed my local config to make 'stable' the submit branch (for more interesting diffing), but that makes ec2test & pqm-submit both try to land the change onto stable
<jml> wgrant, which doesn't work
<wgrant> jml: Aha, I see!
<wgrant> jml: Easy fix, then.
<jml> wgrant, right. your branch is going through pqm now.
<wgrant> jml: Thanks.
<jml> done.
<jml> (waaaay too slow)
<wgrant> Great.
<jtv1> Why are so many pagetests failing on devel?
<jml> g'night all
<wgrant> Night jml.
<wgrant> Thanks.
<adeuring> good morning
<wgrant> Is there an idiom for running the same test class over several target model classes?
<wgrant> In particular, I need to test a new traversal mixin for all the pillars and more.
<wgrant> Or can I just get away with testing it once?
<mwhudson> wgrant: there's bzrlib.tests.multiply_tests
<wgrant> mwhudson: Is there?
 * wgrant can't see it.
<mwhudson> something like that
<mwhudson> 'pydoc bzrlib.tests.multiply_tests'
<wgrant> mwhudson: Oh, in __init__.py, I see.
<wgrant> mwhudson: But do I need to do it? None of the code tested differs between the targets -- the only possible way it could fail is if somebody removes the mixin from one of its descendants.
<wgrant> mwhudson: And if I do need to, I probably can't use it straight from bzrlib, can I?
<mwhudson> wgrant: i don't know about need to, but yes, you can just use it
<mwhudson> for a testcase-style test, of course, it would be a bit awkward (and not really a good idea) for a doctest
<wgrant> mwhudson: I actually just found another related test which does the same thing for a doctest.
<noodles775> wgrant: another example (but of multiple implementations of an interface) is lib/lp/soyuz/tests/test_hasbuildrecords.py
<wgrant> noodles775: That makes four existing rather different ways of doing it, I think...
<mrevell> Morning
<neurocid> good day, im having problems setting up launchpad instance. I have following setup: nginx proxy (public-ip) - intranet server running apache/launchpad (private-ip)
<neurocid> for now LP works great if i run firefox on nginx-server and go to https://launchpad.dev
<neurocid> LP is running on intranet server on two ip-aliases .50 and .51, setup using HOWTO from https://dev.launchpad.net/Running/RemoteAccess
<neurocid> any pointers how to do some rewrite magic on public ip server so i could access intranet LP from outside
<neurocid> i guess i have to do some mod_rewrite magic to parse https://public-ip/launchpad url to be proxyed intranet server as target launchpad.dev?
<neurocid> as it seems that its not possible to access LP using just ip-addresses?
<wgrant> LP makes extensive use of vhosts. You are probably not going to have too much luck rewriting it from under a subdirectory.
<neurocid> ok, is there any pointers then how is could start progressing. I guess someone must tried using nginx as a frontend proxy for LP?
<wgrant> I doubt it.
<wgrant> I know of four proxied LP instances. They all use apache and pound, and are run by Canonical.
<wgrant> What are you trying to do?
<neurocid> basically im trying to setup system with single public ip and multiple intranet servers to serve different kind of stuff all proxied by nginx, for example i have 2 trac instances running on intra servers, and now tried to setup LP also to sit in intra
<neurocid> im not actually sure if my approach is compelety sane :D
<wgrant> What are your plans for the Launchpad instance once you get it working?
<neurocid> to be quite honest, dont actually know, im just man in the middle trying to get it "working"
<wgrant> That really affects how it should be set up.
<neurocid> uh, nice
<neurocid> this is kind of, "try to get it running" task
<neurocid> maybe i should then slow down and do littlebit of background investigation (rtfm that is) any pointers on documentation of dependecy between set up procedure and actual purpose how LP will be used
<wgrant> The only documentation is for setting up local development instances.
<neurocid> ok, i guess what we want is local development instance, but with remote connectivity on top
<wgrant> That's what /Running/RemoteAccess is for.
<wgrant> But proxying is way out of that page's scope.
<bigjools> neurocid: what are you going to be using it for?
<neurocid> bigjools: i guess most intereting parts are code hosting and ppa:s
<bigjools> neurocid: but what are you going to use it for?
<neurocid> umm, im sorry, what you mean
<wgrant> There are licensing restrictions on the default Launchpad images. You cannot use them on a non-development, non-testing instance.
<bigjools> neurocid: are you setting up a development host so you can improve the code, or are you running your own service?
<neurocid> running own service
<bigjools> neurocid: then you won't get much help here.  You are also breaking the law if you don't replace all the trademarked graphics.
<neurocid> bigjools: ok, that is absolutely not my purpose. i really have to check the licensing terms
<wgrant> neurocid: Why do you want to run your own service, rather than using launchpad.net?
<neurocid> wgrant: well as i said im just "man in the middle" with task "see what LP can do/try to get it running"
<maxb> neurocid: You must have some idea of what it's being tried for
<maxb> If all you wanted was to see what it can do... well just wander over to http://launchpad.net
<neurocid> i guess purpose is to codehosting&trac integration
<maxb> i.e. for use, not development of launchpad itself
<maxb> I'd dearly like to run a couple of private instances myself, but the license says "NO!"
<bigjools> neurocid: one of the major features of Launchpad is that there's only one of it, which makes it much better at collaboration, which is one of its aims
<wgrant> There are two very different types of private instances.
<wgrant> One I consider acceptable, one less so.
<wgrant> But both are impossible without replacing the images. :/
<neurocid> maxb: as i said, i wasnt aware of those licensing terms, and now that i am, i have to reconsider all of this
<maxb> And there are so many images, that it's an effective prohibition
<wgrant> That is most likely the point.
<wgrant> Although it does also serve to protect the brand, I suppose.
<neurocid> ok, but i thank you all for kind answers, just re-read Getting page and actually saw the note about image/icons licensing
<neurocid> by no means any licence violation was not my purpose, this just plain ignorance, good that you set things straight
<ddaa> Hey
<ddaa> https://dev.launchpad.net/Getting recommends installing in a virtual environment
<ddaa> I have not stayed on top of virtualization stuff
<ddaa> what is the current latest, greatest, easiest way to set up a virtualized ubuntu guest on an ubuntu host?
<bigjools> ddaa: vmbuilder
<ddaa> cool, never heard of this one :)
<wgrant> It is awesome.
<bigjools> there's some good help pages
<wgrant> Takes a couple of minutes on a good host.
<bigjools> and apt-proxy :)
<wgrant> ddaa: Though vmbuilder only builds the VM image -- does your CPU support KVM?
<bigjools> and soren is really helpful!
<ddaa> wgrant: presumably, it's reasonably recent thinkpad
<thumper> ddaa: I've not set it up in a virtual env
<wgrant> ddaa: Make sure it's enabled in the BIOS; even my three-month-old T400 came with it disabled.
<thumper> ddaa: long time no talk :)
<ddaa> hello thumper
<thumper> ddaa: how's tricks?
<ddaa> currently CTO of a business without income, funding, or a team
<ddaa> but working on learning my trade
<thumper> ddaa: good luck
<ddaa> intending to draw inspiration from launchpad for Windmill integration
 * bigjools repeats the good luck
<ddaa> thanks luck would be useful
<ddaa> but at the moment, we need funding more :)
<ddaa> Meeting investors in a couple of weeks for a few 1e5â¬
<maxb> I don't think it's at all necessary to install lp in a virtual environment. You just have to pay attention to what it's doing to your apache and really really heed the warnings attached to launchpad-database-setup
<ddaa> I was a lp coder for years.
<ddaa> I know the kind of stuff it does in the system.
<maxb> then it's even less necessary to use a virtual environment :-)
<ddaa> I do have a dep on a specific apache config
<ddaa> I'd like to keep lp stuff in a box if I can. To save me some trouble.
<maxb> Fair enough. I've not used the fancy virtualization stuff much, so I tend to regard it as overkill if there is a non-virtual simple solution
<bigjools> ddaa: I got it working in a vm
<bigjools> it's easy, the trick is to specify the right sources.list entries to vmbuilder, then you're good to go
<ddaa> bigjools: can you be more specific about "the trick"?
<wgrant> I initially ran it in a chroot, but once I worked out what evils it did I moved it out.
<deryck> Morning, all.
<wgrant> Morning deryck.
<maxb> yeah, the chroot approach gets a bit insane. Especially if you have a postgres inside and outside the chroot
<wgrant> I had a postgres outside the chroot, and inside another two apart from LP.
<wgrant> It's sometimes a tad confusing when things stop working.
<maxb> * wgrant demonstrates a masterful command of understatement
<bigjools> heh
<wgrant> It's even more confusing when you install postgres in a chroot with another postgres already running in another.
<wgrant> Because it works, but creates the cluster on 5433 instead.
<henninge> danilos: How do I return an empty ResultSet?
<danilos> henninge: use the condition along the lines of 1=0 or something
<henninge> danilos: does that get optimized somewhere to not execute the query?
<danilos> henninge: not sure, but postgres would be very quick with that query, so unless you have that done a big number of times (when you can use the same one), I'd just go with it
<henninge> danilos: thanks
<danilos> henninge: you can look through the Storm code to see if there's something for it, and if not, file a bug (and/or fix it in there :)
<henninge> storm code ....
<henninge> terra incognita ...
<henninge> danilos: err, not now ;)
<danilos> mrevell: ping
<mrevell> hello there danilos
<danilos> mrevell: I need you to come up with nice one paragraph introduction for translation groups on translations.launchpad.net/+groups
 * mrevell looks
<bigjools> henninge: there's an EmptyResultSet
<bigjools> and danilos, FYI ^
<mrevell> danilos: To replace the existing two paragraphs, right?
<danilos> mrevell: I've got http://pastebin.ubuntu.com/259233/ so far (mostly what it was)
 * mrevell looks
<danilos> bigjools: yep, seen it, thanks
<henninge> bigjools: cool
<henninge> thanks
<danilos> mrevell: you'll be free to provide pop-up help later on, but I just want to get our 3.0 template migration off the zero now :)
<mrevell> :)
<jtv> hi danilos!
<jtv> danilos: I thought your migration branch was already landing yesterday...  Did I get that wrong in my notes?
<danilos> jtv: which one? that one was for 'Translator' forms, this one is for translation group listing, and I've got another one coming up for translation group overview (they have both gotten UI reviews yesterday, if that counts :)
<jtv> danilos: there was some confusion in the UI call last night because no conversions showed up in the overview yet.
<danilos> jtv: haven't you seen the total number of the templates go below 50?
<jtv> danilos: I thought it was 51 last I checked, though I haven't made a point of remembering...
<danilos> jtv: so, we've had 5 template removals (replaced by generic-edit.pt) so far, which is 10% of the numbers... my branch in review moves another one to generic-edit and migrates one template
<danilos> jtv: we had 52 at least at one point
<jtv> danilos: cool, thanks
 * jtv scribbles
<jtv> danilos: I thought you converted two TranslationGroup templates in the same branch, so I thought I'd recognize the removals as a piece of blue bar in the graph.
<danilos> jtv: nope, that would have resulted in ~2000 line diff, which is why it's all split up
<jtv> danilos: I don't know what's wrong with reviewers nowadays... they've gone so _soft_...
<jtv> :-P
<danilos> jtv: heh, right, I see you still push them with >1k branches ;)
<jtv> *cough* *cough*
<jtv> Around 600 I thought, "this is headed towards the branch limit, better split off what I have now."  First question I get on -reviews is "can't you split it up?"
<danilos> jtv: also, the simple insertion of transaction.commit() with every potemplate helped reduce the langpack exporter transactions; stub mentioned how after that is CPed (waiting now), we should again start killing all transactions that are longer than 3h
<danilos> jtv: will that be a problem for export-to-branch?
<jtv> danilos: not at all, we have a similar thing cherrypicked there. :)  Thanks for adding them to langpack; I've been yearning to get that behind us.
<danilos> jtv: ok, great to hear that
<jtv> danilos: migration of +translations-to-review is being reviewed; it was very easy because the page was simple, and I hope it opened the door for the rest of our Person pages.
<danilos> jtv: it might, depends on what you decided to do with navigation menu items
<jtv> danilos: "related pages."
<danilos> jtv: ok, we can do the same on all the rest person pages, except the main one :)
<jtv> Looks a bit ugly, but consistent with what we have elsewhere.  I also added the +translations-to-review page to the nav menu.
 * henninge lunches
<danilos> jtv: you can also move it to the right side menu
<danilos> henninge: be quick
<jtv> danilos: in this case I couldn't justify a sidebar.
<henninge> danilos: just have to walk to the kitchen today
<jtv> danilos: stop distracting henninge so he can eat :-P
<wgrant> Ooh.
 * wgrant likes and loathes devel r9224.
<danilos> wgrant: I am just scared ;)
<mrevell> danilos: Look  ok? http://pastebin.ubuntu.com/259245/ Will make that translation group article live after lunch, which I'm taking now.
<danilos> mrevellunch: sounds good, thanks
<wgrant> cprov: I see the email discussion about that issue is rather dead, even though it's blocking uploads of various packages.
<cprov> wgrant: we have to resurrect it, I guess
<wgrant> It's interesting that it's happening so frequently.
<wgrant> I suppose in most cases people have logged into Launchpad once, and done nothing at all, so their Person was removed a couple of weeks ago.
<wgrant> But that seems unlikely!
<deryck> beuno, ping.
<danilos> salgado: hehe, I just renamed the breadcrumbbuilder to breadcrumb in my branch :)
<danilos> salgado: and then I see the email ;)
<salgado> heh
<salgado> danilos, lp:~salgado/launchpad/real-breadcrumbs has a fix for the bug you reported and is on ec2 now. if all tests pass it'll reach PQM in 3h
<danilos> salgado: cool, I'll try it out and see if it works, and just get my branch landed with no rootsite set knowing that once your branch lands it'll be fixed :)
<salgado> danilos, I guess that means you won't be writing any tests for your changes, right? tsc, tsc
<danilos> salgado: bread crumbs? nope, I am just lazy like that... fwiw, I have tests for canonical_urls, but I don't want to test that breadcrumbs are broken or not broken when they should be generically tested
<beuno> deryck, hi
<danilos> salgado: i.e. it'd be like testing that launchpadform works everywhere properly
<salgado> danilos, IMO, it's as necessary as your tests for canonical_url
<danilos> salgado: perhaps, I'll consider it then
<salgado> danilos, it's very easy to write them. for an example, see lib/lp/bugs/browser/tests/test_breadcrumbs.py
<danilos> salgado: heh, I know it's easy, but we are still at 0 translation pages migrated, which is what worries me ;)
<cprov> salgado: what's the next step regarding the Account/Person/EmailAddress relationship issues ? (was: [Launchpad-dev] EmailAddressAlreadyTaken: account and person split)
<salgado> cprov, we need a DB patch + code fix for that. I suggest you bring that up with gary_poster to see if they'll be able to fix it this cycle.  (I don't think I'll be able to)
<cprov> salgado: right, is there an alternative route to fix production tasks failing on this ?
<gary_poster> (I need more context for an opinion, I'm afraid; can talk later today if that helps)
<cprov> salgado: 'quick & dirty hack' possibly fits as 'alternative route' because debian imports are blocked on this.
<danilos> cprov, salgado: I assume we can create them ourselves directly in the DB, and let the code pick those up; though, that will work only if the number of cases we are seeing is small
<salgado> cprov, yes, convince flacoste to approve manual creation of the missing Person entries
<danilos> cprov: perhaps create appropriate Person records for those few entries you need?
<cprov> danilos: yes, that could work temporarily.
<cprov> danilos: did you do that for translations ?
<danilos> cprov: no
<danilos> cprov: I did nothing for translations (we had one occurrence so far, I believe)
<Knut-HB> Hi, I was here last week with the problem of not able to be uploading files to Launchpad. This is the output which we get when trying to upload:http://paste.ubuntu.com/256915/ anyone knows how to handle this?
<cprov> salgado: given an email address, can you write a recipe so the LOSAs can create the missing person ?
<henninge> danilos: ^^
<salgado> cprov, better to ask permission first
<salgado> but yes, I can
<danilos> henninge: ?
<cprov> salgado: sure, thank you.
<henninge> danilos: Knut-HB's problem http://paste.ubuntu.com/256915/
<danilos> Knut-HB: what is it that you are exactly trying to do? have you tried doing make schema and retrying?
<Knut-HB> danilos: yes, I tried. This happens when I want to upload a template.pot file
<danilos> Knut-HB: it tries to put stuff into librarian, which stores data in /var/tmp/fatsam* so make sure you have access there
<danilos> Knut-HB: it's most likely that librarian is not properly starting up for you
<danilos> Knut-HB: do you have stale librarian processes around? ("ps aux |grep python2.4" might tell you)
<Knut-HB> danilos: ok, let me check that
<kfogel> maxb: I'm pinging Matt Zimmerman here to see if he saw your question about the launchpad-dependencies package and its bazaar branch.
<maxb> Thanks. I've just discovered some frankly bizarre delays in a couple of launchpad-dev mails where my mail provider's internal systems have apparently sat on a message for 3 days, so I can't say for certain whether he replied or not
<Knut-HB> danilos: this is the output from "ps aux |grep python2.4" -> http://paste.ubuntu.com/259298/ and rerunning schema and setting chmod 777 on /var/tmp/fatsam/ for testing didn't solve the problem
<mdz> kfogel: hi
<Knut-HB> danilos: still getting the same errors
<kfogel> mdz, maxb: consider yourselves introduced.  maxb, mdz is in a meeting right now so he might appear to be at the end of a slow link, but hopefully he can answer your historical question about that bazaar branch.
<maxb> mdz: Hi. If you saw my mail on launchpad-dev@ cc-ed to you that says everything - if not, let me know and I'll summarize
<mdz> kfogel: oh, was this about launchpad-dependencies?
<maxb> yes
<mdz> I did see that but have been too busy to respond
<danilos> Knut-HB: what do you get when you try to access http://launchpad.dev:58080/
<mdz> maxb: I've replied now.  I don't have anything new to add, I'm afraid
<mdz> I would be happy for someone else to run with it
<Knut-HB> danilos: http://paste.ubuntu.com/259303/ this is the output from launchpad.dev:58080
<maxb> ok. I guess I next ask whether someone wishes to dredge up the source packages for the versions that are currently private and give them to me to synthesize a full history
<Knut-HB> danilos: and i just uploaded the pot-file
<Knut-HB> danilos: don't ask me what I did, I just chmod 777 /var/tmp/fatsam/
<danilos> Knut-HB: right, something weird might have happened, "make schema" tries to reset it all, but I guess it fails sometimes
<kfogel> maxb: I would post that to the list, but personally my answer is 'Not worth it'.  We need it to work now; we don't need it to work as of two years ago.  History is not *so* sacred.
<kfogel> mrevell: where are we on our planet?  I feel I somehow let it drop off my radar screen, but http://castrojo.wordpress.com/2009/08/21/making-daily-builds/ made me think of it again.
<Knut-HB> danilos: ok, seemed to solve the problem... I'll be back if we have more errors ;P Thanks for the help :)
<danilos> Knut-HB: np
<mrevell> kfogel: We're hoping to find someone who can help us theme it. I think beuno was waiting to hear from Matt Nuzum.
<kfogel> mrevell: I'll go poke.
<kfogel> can anyone else reach lists.launchpad.net right now?
<kfogel> Uh, ibarry: going to http://lists.launchpad.net redirects me to https://help.launchpad.net/ListHelp, and going to https://lists.launchpad.net/launchpad-dev hangs forever ATM.
<kfogel> whups
<kfogel> barry: ^^
<kfogel> (mistyped your nick as "ibarry", as though you were an apple product)
<barry> kfogel: otp, will respond soon
<kfogel> barry: thx
<barry> kfogel: still otp, but, the redirect is correct and i was able to get to https://lists.launchpad.net/haibunku just fine
<kfogel> barry: did you try /launchpad-dev like I did?
<kfogel> barry: and why on earth do we do that redirect?  Because there's no page in Launchpad showing all lists, or offering list search?
<barry> kfogel: just tried it, and it came right back
<barry> kfogel: yes, that's exactly why
<EdwinGrubbs> beuno: ping
<kfogel> barry: "came right back" means success for you?
<kfogel> barry: so, assuming infinite time and resources, you too would be in agreement that we should have a real top-level mailing list landing page?
<beuno> EdwinGrubbs, on the phone
<barry> kfogel: off the phone now.  yep, success. quick too
<barry> kfogel: i do think so.  it's currently difficult to know what mailing lists exist on lp. that would be a good page to expose/redirect list.launchpad.net
<kfogel> barry: ok, will file a bug.  as soon as I read the other 1000 emails, and have this phone call, and floss my cell phone, and stuff.  You know, the usual.
<barry> kfogel: why so light a schedule before lunch? :)
<kfogel> barry: oh, just taking it easy before next week's triathlon -- I'm doing the Silicon Slam: twenty miles of XML parsing, followed by the sudden-death editor macro competition, and ending with a flame war about top-posting among the remaining contestants.
<kfogel> I made it to round three last year, but so weakened that I didn't last long in the thread.
<kfogel>  
<kfogel> Ah, what a pity that barry quit in the middle of that :-).
<kfogel> barry: you quit in the middle of my answer to your question "why so light a schedule before lunch?".  Now I'm going to have to repeat my answer:
<kfogel> barry: oh, just taking it easy before next week's triathlon -- I'm doing the Silicon Slam: twenty miles of XML parsing, followed by the sudden-death editor macro competition, and ending with a flame war about top-posting among the remaining contestants.
<kfogel> I made it to round three last year, but so weakened that I didn't last long in the thread.
<barry> kfogel: is that like the ididerhack?
<barry> salgado, bac i have not yet been successful in figuring out why those links disappear
<bac> barry: that is unacceptable.  try harder!
 * barry clenches
<bac> barry: there you go
<salgado> bac, do you have an example page where the links disappear?
<bac> salgado: no, barry does.  my links don't disappear but they don't become inactive either
<barry> salgado: https://launchpad.dev/sprints/+all in lp:~barry/launchpad/273209-toplev
<barry> salgado: if you hit https://launchpad.dev/sprints you see the link, but the +all page hides it
 * salgado tries
 * kfogel is away: lunch + an errand
<leonardr> sinzui2: i've got a lazr.config change to land, but the current trunk is owned by pqm. gary and i think that's pointless since pqm won't test anything. can you confirm? is it okay to change the trunk of the project (i ask you because you're listed as the project driver)
<sinzui2> leonardr: there is a doctest
<sinzui> pqm wont test the tests that are in the tree?
<gary_poster> leonardr, sinzui: sorry, I have a branch that moves us to a lazr.config release (the lazr.* branch leonardr) but it hasn't landed yet
<leonardr> the reason (according to gary) it won't test anything is that launchpad doesn't use lazr.config trunk, it uses a release
<gary_poster> so I misinformed leonardr, but I will hopefully be correct later this week
<salgado> barry, bac, it's because of NavigationMenu._is_menulink_for_view() (in webapp/menu.py)
<gary_poster> sinzui: PQM will not test usually the tests that are in our distributions (though we do run launchpadlib tests that way)
<leonardr> all right, i will run the branch throuh pqm
<gary_poster> actually no
<gary_poster> we do it on buildbot
<barry> salgado: that's explains bac's problem, but does it explain mine?
<gary_poster> leonardr: unless sinzui objects, I would prefer it if you made a ~launchpad branch, and made that the trunk
<sinzui> gary_poster: leonardr: +1
<gary_poster> because that's the pattern for its siblings
<gary_poster> thanks sinzui
<bac> salgado: is that the culprit for barry's issue where the link goes away or my issue where it stays linkified?
<barry> bac, salgado looking at the code, it seems to set link.linked not link.enabled
<barry> bac, salgado i.e. bac's problem not mine
<leonardr> gary_poster, sinzui: i'm not sure which siblings you're talking about. trunk of lazr.restful is ~lazr-developers/lazr.restful/trunk, not ~launchpad
<salgado> that's correct, but bac's link is left linkified
<salgado> this doesn't make sense
<sinzui> gary_poster: leonardr is right (from my experience)
<gary_poster> leonardr: then that's what I mean
<gary_poster> sorry
<gary_poster> I was guessing
<sinzui> I am happy for the team to own the branch
<gary_poster> cool
<leonardr> ok
<bac> has anyone asked sinzui about this mysterious link problem?
<beuno> barry, ping
<sinzui> No, not unless they asked in a mysterious and cryptic way
<barry> sinzui: why does it not work?
<barry> beuno: pong
<beuno> barry, do you feel like talking mailing lists?
<barry> beuno: sure!
<barry> beuno: skype?
<sinzui> barry: is that in the tree? Do I need a paste bin to see it?
<beuno> barry, yes
<beuno> ready when you are
<barry> sinzui: you do not need a pastebin.  hit https://launchpad.dev/sprints and sprints/+all in lp:~barry/launchpad/273209-toplev
<barry> sinzui: well, that's for my selfish problem not bac's :)  going otp w/beuno now
<rockstar> sinzui, the tests! the tests!
<sinzui> rockstar: is unicode breaking answers doctests again?
<rockstar> sinzui, there are some rather hard to work with tests in answers.  I getting through them, I just want you to know how hard it is.  :)
<salgado> barry, try https://launchpad.dev/sprints/+all/++debug++source for a nice surprise
<sinzui> barry: <p/> is illegal HTML Wrap your content, I suspect the you need a div with class="top-portlet" to get margins and padding right.
<salgado> bac, the culprit for your problem could be templates/launchpad-inline-link.pt, which doesn't care about linked/unlinked links
<bac> salgado: ah, i'll investigate.  thanks.
<salgado> bac, or launchpad-inline-icon-link.pt
 * salgado needs food
<rockstar> abentley, just a heads up: You may see some branches that I haven't landed that you approved yesterday.  One prereq branch had spectacular test failures, so it may be a bit before it lands.
<abentley> rockstar: ack
<intellectronica> where do i find the adapters that implement fmt:whataever for stuff?
<intellectronica> sinzui: maybe you know? ^^^
<intellectronica> nm, found it
 * barry is off the phone
<barry> salgado-lunch: re: ++debug++source confirms to me that the problem is that something is asking "are we viewing the page for this link, and if so, hide the link".  i just havn't found out what yet
<barry> sinzui: div +1
 * barry needs food and will continue with this after lunch
<rockstar> sinzui, we have an answers test that is testing that <html lang="pt-BR">  Is that really important?
<sinzui> rockstar: yes, but we want to verify that differently now
<sinzui> rockstar: in base-layout we want to ensure that just the block that id pt-BR is labeled that. I think this is the <title>, listing <td>, comment <div>.
<sinzui> rockstar: so I think we want to change that test to verify that the div that represents the question (and maybe the other comments to be pt-BR
<sinzui> rockstar: I think we need to ensure that question-index.pt creates a title with the right lang, base-layout may need to change.
<rockstar> sinzui, okay.  I won't worry about base-layout for now.
<sinzui> rockstar: right, file a bug. I knew about this when I created base-layout. I assumed I would be the person updating Answers
 * sinzui simplified base layout to make the rest of the conversion simple.
<rockstar> sinzui, understandable.
<sinzui> rockstar: the 'dir' attr is important on the question in the listing table and question. I added an Arabic question to sample data so that you can verify the layout of the text
<abentley> rockstar: we are in testfix mode due to failure to initialize the DB.  I want to submit an empty "testfix".  Can I rs you?
<rockstar> abentley, I was just about to ask you if you could do a testfix.  rs=me
<rockstar> How do I clear the mailqueue so it stops trying to send mail?
<rockstar> It keeps hiding my logs and stuff because it's giving me failure messages in my terminal.
<intellectronica> rockstar: just get rid of /var/tmp/launchpad_mailqueue
<mrevell> night all!
<EdwinGrubbs> beuno-afk: ping
<barry> salgado: so i don't think my problem is that the sprints/+all link is being disabled.  if i actually remove the 'condition="context/enabled"' from launchpad-inline-link.pt i /still/ don't get the sprints/+all link
<salgado> barry, looks like templates/navigationmenu-related-pages.pt doesn't render links that have linked=False
<salgado> barry, is this menu rendered using that template (i.e. view/++@related-pages)
<salgado> @@+related-pages, that is
<barry> salgado: it is, and i see that!
<barry> salgado: that doesn't seem right, does it?
<salgado> it doesn't.  everywhere else we just omit the <a> tag for unlinked links
<barry> salgado: agreed.  that should probably be done in launchpad-inline-link.pt which is what actually renders the link
<barry> salgado: yep, removing that tal:condition brings back the link
<salgado> barry, maybe we can even remove the conditional on link/enabled, if it's also done by launchpad-inline-link.pt
<barry> salgado: i think so.  l-i-l.pt doesn't remove the href yet.  i wonder how hard that will be
<barry> salgado: is it worth it?
<barry> salgado: disabling the href i mean
<salgado> barry, I guess you'll need to place the condition="link/linked" in the <a> itself and add another tag with condition="not: link/linked" to show the text without a link/icon
<barry> salgado: yep
<salgado> is that what you asked?
<barry> salgado: well, i'm actually asking whether it's worth disabling the link when you're on that link's page.  personally, that behavior always bugs the hell out of me where ever i encounter it :)
<barry> salgado: i guess that's a question for beuno
<salgado> agreed
<salgado> I know he's planning to do that for breadcrumbs at least
<barry> or for beuno-afk even
<barry> salgado: so at least consistent annoyance is better? :)  okay, ping beuno-afk for the pronouncement
<kfogel> danilos: thanks for your keeping an eye out on ubuntu-translators@ for replies in the "three wishes for Launchpad" thread
 * rockstar lunches
<kiko> hey
<kiko> somebody needs to ping jamalta to go over a branch he has to fix bug 127489
<mup> Bug #127489: Answers column is in the wrong order in person's "Most active in" table <trivial> <ui> <Launchpad Foundations:Triaged by jamalta> <https://launchpad.net/bugs/127489>
<kiko> https://code.edge.launchpad.net/~jamalta/launchpad/bug-127489/+merge/9564
<kiko> any reviewers?
<statik> hi launchpad-dev, i have a questions about stacking. i desperately need to push a 2a format branch to a project where the dev focus branch is in an older format. is there a way I can force a branch not to stack?
<statik> oops, wrong channel
<mwhudson> good morning
<statik> mwhudson, just the man who can save my life
<mwhudson> statik: bzr init lp:~whatever/whatever/whatever
<mwhudson> bzr reconfigure --unstacked lp:~whatever/whatever/whatever
<mwhudson> bzr upgrade --2a lp:~whatever/whatever/whatever
<mwhudson> bzr push -d ~/yourbranch lp:~whatever/whatever/whatever
<mwhudson> (there might be some easier way, too early in the morning for that)
<statik> mwhudson, you are my hero
<gary_poster> maxb: ping
<maxb> pong
<gary_poster> maxb: hey.  bug 413335 : do you want me to go upstream (I am upstream and can make releases)?  or have you done it, or do you want to?
<mup> Bug #413335: Figure out what to do about zope.sendmail incompatibility with Python >= 2.5.1 <python-upgrade> <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/413335>
<maxb> gary_poster: Ah, right. The reason I've not going upstream with that is that I'm not sure what the correct action is
<rockstar> sinzui, I tire of answers tests.  Can I delete the ones that are redundant?
<gary_poster> maxb: sure, understood, me either. we don't have to have an answer necessarily; we can brainstorm with upstream and see if someone has ideas.  This is a problem for 2.6 as well?  I'm not familiar with the threading changes.  I saw your workaround.
<maxb> I don't think it's changed any further in 2.6
<gary_poster> maxb: ok. btw, I landed the "all of zope from zope distributions" about 8 hours ago.  I have another branch I hope to land tomorrow that brings us up to the most recent releases of our lazr.* stuff (and launchpadlib and so on) tomorrow--tests appear to all be passing.
<maxb> The problem is that in 2.5.0 and below you could use sys.exitfunc as a notification that the main/default thread had exited. In 2.5.1 and above, there is no such mechanism (other than the one that's private between the interpreter and the threading module, which I therefore monkeypatch)
<maxb> Neat!
<maxb> I was starting to teach myself about doctests and trying to understand REnormalizing
<maxb> doctests are great, except they suck at supporting variable output
<gary_poster> yeah
<gary_poster> I agree.  others wouldn't go so far as "are great" ;-)
<gary_poster> RENormalizing helps, ellipses help.  Maybe we'll switch to manuel later, which gives us a lot more flexibility http://packages.python.org/manuel/
<rockstar> maxb, don't go into the light...  :)
<gary_poster> lol
<maxb> I need to decide whether to shift my python2.5 efforts to be based on db-devel, then, I guess
<gary_poster> maxb: ok, for zope.sendmail, I'll write something upstream, and cc you.  If you want to join in (and I hope you do) feel free to join the mailing list, or I'll forward notes you write.
<gary_poster> maxb: why db-devel?
<maxb> I thought your zbuildout branch was targetted at db-devel?
<gary_poster> no, devel
<maxb> oh, weird, the merge proposal said db-devel when I looked at it
<gary_poster> urg, I bet I screwed up.
<maxb> Yay, all the better
<gary_poster> heh
<maxb> Which is the mailing list in question?
<gary_poster> maxb: http://mail.zope.org/mailman/listinfo/zope-dev
<maxb> eek, all of zope on one list, that'll be interesting
<maxb> s/interesting/high-traffic/
<gary_poster> heh
<maxb> Right, looks like I can probably mark lp:~maxb/zope3/launchpad-3.4-py2.5 as Abandoned at this point
<maxb> I'll let my desktop chew on the tests tonight
<thumper> danilos: a translation based breaking on lp builder
<barry> yowza!  look all all the new log spew
 * barry will bet gary_poster had something to do with that :)
<gary_poster> lol
<barry> gary_poster: so what does that buy us?
<gary_poster> barry: (sorry was on call) all of zope is now from distributions, rather than sourcecode.  I have a branch that brings us to trunk (and distributions) for all of our own packages (lazr.* and friends) that I hope to get reveiewed and land tomorrow (I believe tests are now passing entirely).  Then we need to upgrade to the zope versions that upstream have identified as 2.5/2.6 friendly and interoperating, and deal with the fal
<gary_poster> (which will probably be significant, we'll see).
<gary_poster> Then we fix the remaining launchpad bits, and switch to Py 2.5/2.6 immediately after the 3.0 release, is my intent.
 * barry <3 gary_poster 
<gary_poster> :-) heh
<barry> 2.6 please!
<gary_poster> +1 from me ;-)  we'll see
<jml> hey guys
<gary_poster> hiya
<barry> gary_poster: well, we have to before karmic!
<barry> jml: hi!
 * jml reads the good news in the backlog!
<gary_poster> actually, thanks to maxb, we don't: he's made the necessary debs already, AIUI from flacoste.
<gary_poster> but that doesn't mean we don't want to ;-)
<barry> gary_poster: right. is there any reason why we wouldn't switch to 2.6 if our test suite passes?
<gary_poster> barry: losa-type reasons
<maxb> ~launchpad/ppa is complete for karmic/2.4 other than launchpad-dependencies itself, which I'm waiting for someone to sync from my ppa
<gary_poster> (which is to say, none that I know of, but who knows)
<gary_poster> maxb: I thought I saw flacoste did that in the past 24 hours, but maybe I imagined it
<maxb> My instinct based on the kinds of changes is that any and all work done migrating 2.4->2.5 would also need to be done for 2.4->2.6, hence 2.5 as a comfortable stepping stone
<gary_poster> agree
<maxb> And I took a quick look at versions.cfg in devel just now - I think the versions there are clear for 2.5 already, though not for 2.6
<gary_poster> you might be right
<gary_poster> i know 2.6 has some issues: zope.publisher has some problems that I need t record
<gary_poster> need to run
<gary_poster> bye all
<jml> gary_poster, ciao
<jml> dammit.
<jml> missed him by *that* much
<sinzui> rockstar: please delete redundant tests. I have deleted a lot in 2.2.8
<rockstar> sinzui, roger roger.  I've been doing my best.  I'm surprised how many tests broke in various, and sundry, ways with just a few changes.
<barry> gary_poster: i guess i'm sayin': if we can/are going to move to 2.5
<barry>         and all our tests pass on 2.6 we should jump right to 2.6 and ignore
<barry>         old pythons that aren't being maintained by upstream any more <wink>
<sinzui> rockstar: the tests verify the formatting instead of the content. I was young and naive.
<sinzui> rockstar: And I I wrote most of them again, I would be testing the view, not the template.
<sinzui> s/I/if/
<rockstar> sinzui, yeah, had I known it was going to take me this long to fix 35 (!) failing tests, I would have just re-written them that way.
<sinzui> rockstar: I have written all view tests, then watch the story failures. I then add the fix to my view test and delete the page test. I do the 75% of the time.
<rockstar> sinzui, that's a good idea.  I'll keep that in mind.
<jml> thumper, skype is still broken for me
<thumper> jml: :(
<thumper> ok
 * jml restarts, just in case.
<maxb> Anyone have any thoughts on breaking compile up into compile-lp compile-mailman compile-lazr-js ?
<jml> maxb, on the surface, it seems like a good idea
<maxb> Hmm, I need a bit of buildout help...
<maxb> Error: There is a version conflict.
<maxb> We already have: lazr.uri 1.0
<maxb> where do I start looking?
<wgrant> Is your sourcedeps branch up to date?
<maxb> the download-cache branch?
<maxb> yes
<wgrant> Hm.
<wgrant> Tried cleaning and rebuilding?
#launchpad-dev 2009-08-26
<maxb> I've cleaned everything but the eggs cache
<maxb> gah, what?
<maxb> devel works, my python2.5 branch does not
<maxb> But there's nothing there that goes anywhere near lazr anything
<maxb> Getting distribution for 'lazr.uri==1.0.1'.
<maxb> Installing lazr.uri 1.0.1
<maxb> Caused installation of a distribution:
<maxb> lazr.uri 1.0
<maxb> with a different version.
<maxb> Got None.
<maxb> While:
<maxb>   Installing.
<maxb>   Getting section filetemplates.
<maxb>   Initializing part filetemplates.
<maxb> Error: There is a version conflict.
<maxb> We already have: lazr.uri 1.0
<maxb> What on earth is buildout attempting to tell me?
<maxb> oohkay, purging python-lazr-uri .deb made it work...
<maxb> isn't buildout supposed to be insulating you from system python versions?
<maxb> /home/maxb/launchpad/lp-branches/python2.5/lib/canonical/config/__init__.py:19: UserWarning: Module lazr was already imported from None, but /home/maxb/launchpad/lp-branches/python2.5/lib is being added to sys.path
<maxb> !?
<james_w> maxb: there was a fix to lazr.config today I believe
 * maxb observes that the zope sourcedep still isn't completely obsolete, if the symlinks are to be believed
<maxb> OK after cleaning absolutely everything, now I can't "make" devel eitehr
<maxb> either
<thumper> :(
<thumper> my make is failing too in mailman
<maxb> heh, at least you got that far
<thumper> mwhudson: hey, build engineer
<mwhudson> thumper: as of monday! :)
<mwhudson> *next* monday
<thumper> :)
<thumper> does your make work?
<mwhudson> dunno
<mwhudson> running pull now
<mwhudson> i just read something about make in mailman failing from last week, francis said he'd fixed it though
<jml> mwhudson, hello
<jml> thumper, I was just looking at https://bugs.edge.launchpad.net/launchpad-code/+bug/418290
<mup> Bug #418290: Person's code facet has unclear section linking to the teams the person is a member of <Launchpad Bazaar Integration:New> <https://launchpad.net/bugs/418290>
<thumper> jml: yeah
<mwhudson> jml: hello
<thumper> File "/home/tim/src/lp/devel/eggs/zope.proxy-3.5.0-py2.4-linux-i686.egg/zope/proxy/_zope_proxy_proxy.py", line 6, in __bootstrap__
<thumper>     imp.load_dynamic(__name__,__file__)
<thumper> ImportError: /home/tim/src/lp/devel/eggs/zope.proxy-3.5.0-py2.4-linux-i686.egg/zope/proxy/_zope_proxy_proxy.so: cannot open shared object file: No such file or directory
<jml> thumper, can we simply fix the links to link to the code listing?
<thumper> jml: there was a "fix" recently that changed the fmt:link to always go to the root site
<thumper> jml: we may be able to get it to go to the code site
<thumper> jml: I've not yet investigated
<thumper> jml: I think it is just team/fmt:link/code
<thumper> (or branches)
<maxb> So, what is buildout *supposed* to do if I have lazr.uri 1.0 installed as  a .deb but versions.cfg says 1.0.1 ?
<thumper> it is a bit of a mixup
<jml> thumper, yeah, I think the 'fix' is a good idea
<mwhudson> maxb: look in your download-cache
<jml> thumper, but it's different from forbidding links to particular facet pages :)
<jml> thumper, and it seems that linking to the team branch listings actually is desirable
<maxb> mwhudson: Hmm. it appears to be throwing its hands up in horror and bleating incomprehensible error messages instead
<thumper> it is there
<mwhudson> maxb: i'm not completely surprised
<jml> right.
<thumper> jml: the bug was that normally a link to a person or team should take you to the overivew
<thumper> jml: but in this case we should link to listings, and probably give it a better title too
<thumper> s/title/heading
<jml> thumper, I was about to file a bug about the heading, actually :)
<thumper> jml: file away and assign to me
<jml> thumper, will do.
<thumper> grrr
<thumper> how can I code if I can't run the tests
<thumper> a rhetorical question
 * jml thinks about the new project page
<jml> leonardr, hello?
<jml> leonardr, I have three simple questions for you, if you have a moment.
<jml> is spm back today?
<spm> aye
<jml> spm, welcome back :)
<jml> spm, feeling better?
<spm> jml: yeah muchly. The joy of children and bringing home colds from school....
<jml> heh
<jml> spm, my todo list says I have a couple of things to ask you about
<jml> spm, one of them being bazaar.edge.launchpad.net
<thumper> maxb: I blew away the contents of the /eggs dir, and make clean build worked ok
 * mwhudson lunches
<leonardr> jml: since i forgot to quit irc, you may ask your questions :)
<jml> leonardr, heh, thanks :)
<jml> leonardr, first, what should I call the constant for the production API service root. I've picked LPNET_SERVICE_ROOT for now, but STABLE_ and PROD_ might also fit.
<jml> leonardr, second, what should the URL be? I've got https://api.launchpad.net/beta/ now, but the 'beta' troubles me a little.
<leonardr> jml: i would go with PRODUCTION or STABLE
<thumper> lunch time
<jml> leonardr, third, when I authorize an app for the production service, I am taken to an edge web page (probably because of the beta user redirect). Is this a bug?
<leonardr> look at the other constants and do what they do. i believe they also have /beta/
<jml> leonardr, they also have beta.
<jml> leonardr, so I should follow what they do?
<leonardr> yes, the constants are going to behave like "HEAD" or "trunk" in a version control system
<leonardr> they will point to the most current version of the web service
<leonardr> which right now is /beta/
 * jml prefers 'PRODUCTION' to 'STABLE', since the latter might have API stability connotations.
<leonardr> sounds good
<leonardr> #3 is a bug -- you're in the launchpad beta testers team, so your requests to lpnet redirect you to beta
<leonardr> flacoste filed this bug a couple weeks back
<leonardr> s/beta/edge/
<jml> cool.
<jml> leonardr, thanks for that. I'll land the patch with the new constant then.
<leonardr> great
<wgrant> Is the beta team restriction in place on the lpnet API vhost?
<leonardr> wgrant: yes, that's the bug
<wgrant> leonardr: Not the same bug... I mean, api.edge.launchpad.net is restricted to beta testers. Is api.launchpad.net?
<leonardr> ah
<leonardr> i don't know. are you sure that restriction is still in place? i thought we got rid of it
<wgrant> I don't know.
<leonardr> but if it's still in place on edge, it should still be in place on lpnet
<wgrant> It is certainly in place on edge.
<wgrant> I saw somebody complaining about it on identi.ca earlier in the week.
<leonardr> maybe that's another thing we should change
<wgrant> Restricting it on edge might be reasonable.
<maxb> It's not restricted on edge. I used the api long before I joined the beta-testers team
<wgrant> Hm.
<wgrant> Then why was it crashing for a non-beta-tester, I wonder..
<wgrant> Huh, you're right. It's not restricted on edge (or at least wasn't 1.5 months ago).
<maxb> huh. So the testsuite no longer runs unit tests first?
<wgrant> It did for me yesterday, but that was without the new zope.testing.
<jml> oh right.
<jml> new zope.testing.
 * jml has work to do!
<wgrant> jml: What are you doing?
<jml> wgrant, enabling parallelization of the Launchpad test suite
<wgrant> Ooh. Excellent.
<jml> but also, paging the new zope.testing version into my brain
<jml> so I can work on making sure it's stdlib unittest compatible
<jml> (mostly so that lifeless's unit testing work can be applied to it)
<wgrant> subunit?
<jml> wgrant, subunit and testresources
<jml> and ideally, much of the work in bzrlib.tests and bzr-ec2test, but let's not get ahead of ourselves
<wgrant> Argh. 844 line diff :(
<jml> wgrant, most reviewers can be bribed to review diffs over the allowed size
<lifeless> particularly with code that is an improvement
<wgrant> Almost half of it is new tests.
<jml> thumper, beuno: Is https://bugs.edge.launchpad.net/launchpad-code/+bug/236442 much closer to being fixed now that we have spark lines?
<mup> Bug #236442: Show how active I've been <feature> <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/236442>
<wgrant> Well, you don't have sparklines any more...
<thumper> jml: I killed them
<thumper> jml: I intend to bring them back, but they need fixing
<jml> thumper, that seems perfectly fair to me.
<jml> thumper, I think that the bug I mentioned doesn't benefit from sparklines so much as actual graphs.
<thumper> yeah
 * thumper primal screams
<jml> mwhudson, just re-discovered https://bugs.edge.launchpad.net/launchpad-code/+bug/85326
<mup> Bug #85326: Codehosting server should initiate a pull attempt <branch-puller> <codehosting-ssh> <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/85326>
<jml> thumper, wassup?
<thumper> the query for person active reviews is an arse
<thumper> we need two different base collections
<thumper> and different queries
<thumper> unioned
<mwhudson> jml: "hmm"
<jml> mwhudson, it's apparently my oldest open bug on launchpad-code
<mwhudson> jml: that's quite a surprise
<jml> mwhudson, yes.
<jml> bloop
<jml> thumper, when you've got time, I'd like to talk about Monty Taylor's post to launchpad-users
<thumper> yeah, I'm actually trying to get work done right now :(
<jml> thumper, understood.
<mwhudson> jml: i think, probably, 85326 should be wontfix/subsumed by our eventual message queue stuff
<mwhudson> jml: in the fullness of time, the puller might not even run on the same machine as the ssh server
<jml> mwhudson, indeed.
<jml> mwhudson, but it could also be done before-hand.
<mwhudson> jml: you mean as a temporary improvement?
<jml> mwhudson, yeah
<mwhudson> well, yes it could
<jml> mwhudson, I don't think it would hinder any of the message queue work
<jml> and it wouldn't be too difficult a change, I think
<mwhudson> jml: there's quite a lot of stuff in vaguely similar areas i'd rather do first, i think
<mwhudson> (combine puller and scanner, maybe pre-exec-ing bzr lp-server processes)
<wgrant> maxb: I think you want http://lpdebs.canonical.com/{dapper,jaunty}/
<jml> mwhudson, I'd combine the puller and scanner first for sure.
<wgrant> maxb: That has 0.11 to 0.35.
<jml> mwhudson, I think I'd probably trigger a pull before working on pre-exec, since I think waiting for a pull is a worse kind of waiting than waiting for a bzr lp-serve process to spawn
<jml> (also because I know how to trigger a pull :))
<mwhudson> jml: i'm interested to see how my/our puller rewrite works in practice
<jml> probably the memory issue trumps both of those
<jml> mwhudson, me too!
<jml> mwhudson, I guess it's waiting for a prod rollout?
<mwhudson> it will reduce the latency before a pull starts to 10s in some circumstances
<mwhudson> right
<jml> I have to confess I'd forgotten about that patch.
<wgrant> maxb: By 'jaunty' I of course mean 'hardy'
<spm> mwhudson: (mwhudsondoyle?) ref bug 411250. Ha! it seemed to go away on it's own - perhaps thumper can enlighten? it "felt" like a -ve cache issue....
<mup> Bug #411250: rewriter seems to negative cache for a long time <Launchpad Bazaar Integration:Incomplete> <https://launchpad.net/bugs/411250>
<mwhudson> spm: there's enough moving parts for things to be able to get pretty confusing
<mwhudson> spm: but unless it happens again, i intend to basically not think about this again :)
<spm> heh. and fair enough too.
<spm> fwiw, nothing quite like nagios generated sms's on a regular basis to encourage bug-reporting. :-D
<thumper> I need a better title for: 'Reviews requested I can do'
<thumper> because I can now review anything
<thumper> what it really means is:
<thumper> a team I'm a member of has been asked to review this proposal
<thumper> but that's kinda wordy
<jml> hmm
<thumper> I'm about to wander off and go and make dinner
<thumper> coming back on line when the kids are in bed
<stub> thumper-cooking: The key word in the existing title is 'requested'.
<thumper> stub: I have a different question for you
<stub> thumper-cooking: The existing title seems fine (although I think 'Requested reviews I can do' is better)
<thumper> stub: we have constantly issues with the security.cfg
<thumper> stub: I want to have some group defined for "bzr-identity" that has the select rights on all the right tables
<thumper> stub: and have that group used in our other db users
<thumper> stub: is there an example of how to set this up?
<stub> Check out garbo, garbo-hourly and garbo-daily - it will show you how inheritance works in that file.
<thumper> stub: ok, ta
 * thumper really leaves to cook now
<mwhudson> well i've now read all the mail that had arrived by the time i got back on tuesday morning
<mwhudson> but not all the mail that's arrived since then :)
<jml> mwhudson, heh
<noodles775> Morning
<jml> hi
<maxb> wgrant: oh great, that gets me all but 0.7 through 0.10
<wgrant> yep.
<maxb> lifeless: Care to cast your mind back to 2006 and speculate where launchpad-dependencies (0.8) dapper might have been kept?
<lifeless> in dapper
<lifeless> [yes, it was actually in the distro at one point]
<wgrant> 0.6 is the latest in the distro.
<wgrant> I located 0.11 to 0.35 this morning.
<wgrant> And >= 0.36 are in the PPA.
<lifeless> sorry, nothing springs to mind then
<wgrant> It also looks like 0.2 and 0.3 only ever lived in pre-Soyuz Dapper.
<maxb> ah well, 4 missing versions is a lot better than 30ish
<jml> maxb, can you give me a file name that I could search for?
<maxb> launchpad-dependencies_0.7.dsc, 0.8, 0.9, 0.10
<jml> no joy.
<maxb> It's not critical, there's info in debian/changelog
<jml> cool.
<jml> danilos, good morning
<wgrant> maxb: I will grab a recent Karmic live CD in the next couple of days and see if I can reproduce your failures in a really clean environment.
<maxb> meh, I have a couple of new failures, even on 2.4
<maxb> https://dev.launchpad.net/LaunchpadOnKarmic
<wgrant> maxb: How's the 2.5 run looking?
<maxb> see the <tests still in progres...> marker :-)
<wgrant> maxb: Ah, I see.
<jml> maxb, so few?
<maxb> 2 consecutive test runs take longer than overnight :-(
<maxb> jml: hmm? which?
<jml> maxb, after <tests in progress>, there looks to be very few tests that are actually failing
<jml> I'm pleasantly surprised :)
<maxb> ah, well after the marker, those are still the results from before zbuildout landed
<wgrant> But presumably zbuildout should make it *better*.
<jml> ohh, I see.
<adeuring> good morning
<maxb> Oh, and the page doesn't take account of the fact I had to cheekily "pycentral pkgremove python-lazr-uri" to make the buildout work at all
<jml> has the lazr packaging mess been sorted out yet?
<jml> (I guess your comment means "no", maxb)
<wgrant> buildout is the solution to everything *handwave*
<maxb> Well, it certainly hadn't been sorted when I was attempting to get these test runs going before sleeping last night
 * wgrant just up'ed devel and is about to try.
<maxb> wgrant: dpkg -l python-lazr-uri ?
<wgrant> maxb: Not installed, thankfully. I must have an old launchpadlib.
<jml> test with stable
<wgrant> That is a strange package name.
<maxb> lucky you! :-)
<jml> devel might well have test failures (who can say!)
<maxb> jml: good point, well made :-)  Though it's the buildout flat out failing to do what its raison d'etre is that's really annoying me.
 * wgrant throws broken eggs at SHHH
<wgrant> Aha, it works. But zc.tracelog is being a bit loud.
<maxb> Next meeting I intend to ask the peanut gallery to express opions on SHHH
<maxb> and opinions
<jml> lifeless, push-pop-progress, right?
<gmb> wgrant: I've just herded your structural subs branch off to ec2. I started a run last night but the instance crashed and burned some time this morning.
<wgrant> gmb: Thanks.
<lifeless> jml: yes
<jml> lifeless, done
<jml> g'night
<noodles775> Enjoy your evening :)
<bigjools> seeya jml, and congrats BTW
<lifeless> bigjools: on?
<thumper> lifeless: staying alive?
<lifeless> oo oo oo oo
<thumper> staying alive
<thumper> staying alive
<lifeless> oo oo oo oo
<thumper> :)
<mrevell> morning
<deryck> morning, all.
<noodles775> hi deryck !
<wgrant> People seem to think that packaging is trivial and automatic...
<bigjools> wgrant: it ought to be
<bigjools> Quickly will be a step in the right direction
<wgrant> bigjools: For simple apps, sure.
<wgrant> bigjools: But libraries, not so much.
<bigjools> .deb is way over-engineered for most stuff IMO
<noodles775> bigjools: what, in particular, do you think is over-engineered for packaging, say, a gnome game?
<bigjools> nothing in particular
<wgrant> There's nothing wrong with .deb for that.
<wgrant> The build system, perhaps.
<noodles775> bigjools: so what's "most stuff" then?
<bigjools> did you see jono's session at All Hands when he learned to package something?
<noodles775> yep
<wgrant> So you don't mean .deb. You mean the source package format.
<bigjools> probabl
<bigjools> y
<bigjools> and the number of people in the audience who argued over how to do something?
<wgrant> For setuptools or autotools applications, it's very easy with CDBS or dh7.
<bigjools> but then I don't have a great deal of experience packging stuff, I am probably talking out of my arse
<wgrant> Heh.
<bigjools> however, I think it could be easier
<wgrant> Before DH7 and CDBS were big, it was pretty awkward.
<noodles775> bigjools: yeah, so the process could be a lot more helpful - or *feel* a lot more intuitive with the right tools, ,but the biggest difficulty is that the problem-domain is complex
<noodles775> yep.
<bigjools> I think it's only as complex as you want to make it
<james_w> bigjools: that was partly due to the nature of some of the people in the room :-)
<bigjools> james_w: no doubt :)
<bigjools> james_w: btw I owe you an apology, I was confusing you with someone else when I was talking about lpnet vs edge API usage, sorry :(
<james_w> np
<james_w> I'm just happy it's there :-)
<bigjools> yeah, I was surprised it wasn't from the start
<danilos> bigjools: it doesn't count unless it's in an email (an apology that is :)
<danilos> bigjools: btw, I was surprised it wasn't either, flacoste did it basically a day after we've decided to go with it, but it never went out
<bigjools> danilos: shaddap and do your job :)
 * danilos crawls back into his corner
<danilos> mars: btw, is conversions.html based on db-stable or something?  if I run it over my local copy of devel, I get much better stats for translations :)
<wgrant> (also, conversions.html doesn't belong on devpad)
<bigjools> danilos: the report does not lie, you're slackers!
<danilos> bigjools: heh, that much is true as well, but I've got the code from mars $HOME and tweaked it a bit so it displays good results for translations
<bigjools> lol
<bigjools> if project=translations: done=100%
<danilos> wgrant: that was exactly what I was thinking while I was typing this, but Launchpad team is not used to having publicly accessible accounts
<danilos> wgrant: I'll raise it with the team
<danilos> bigjools: heh, exactly :)
<bigjools> geez, what is this zc.tracelog spam in my "make run" output now ...
<wgrant> bigjools: I wondered that...
<danilos> bigjools: that's what wgrant loved yesterday, 9124 or 9224 or something :)
<bigjools> should be DEBUG not INFO
<wgrant> danilos: Note I also loathed it, though.
<wgrant> 'cause it's all eggs.
<danilos> wgrant: heh, I know, I know
<danilos> wgrant: I've got about the same feelings about eggs/buildout combination
<danilos> adeuring: hey
<adeuring> hi danilos
<danilos> adeuring: did you by any chance ask for some hwdb parsing runs on staging yesterday?
<adeuring> danilos: no. what's the problem?
<danilos> adeuring: if you did, I got some logs in my email, but didn't get the logs I actually asked for :)
<danilos> adeuring: ah, never mind then, it seems Chex ran the wrong script for me then
<beuno> EdwinGrubbs, ping
<henninge> beuno: ping
<beuno> henninge, hi
<henninge> beuno: did you notice this bug I assigned to you? bug 418610
<mup> Bug #418610: New translate page needs new icons <Launchpad Translations:New for beuno> <https://launchpad.net/bugs/418610>
<beuno> henninge, I did not
<henninge> beuno: do you think you can get something done with that before you're gone?
<beuno> henninge, could you please attach a ascreenshot of where those icons would go?
<henninge> beuno: sure
<beuno> henninge, I will try
<henninge> beuno: cool, thanks
<gmb> wgrant: Your branch passed its tests and is now in the PQM queue.
<wgrant> gmb: Thanks.
<gmb> np
 * wgrant shall prepare the other two pieces tomorrow.
<henninge> beuno: I attached the screen shot.
<beuno> henninge, thanks
<deryck> barry, ping
<barry> deryck: pong
<barry> beuno: ping
<beuno> barry, pong
<barry> beuno: hi, two things.  first, can you come to the ameu reviewer's meeting in 6m?
<barry> beuno: second, i'd like to talk about what to do about link display on the page the link points to
<barry> reviewers, lurkers and beuno -> #launchpad-meeting in 3m
<beuno> sure
<beuno> barry, lets see about the second after the meeting
<barry> beuno: sounds good
<barry> reviewers, lurkers and beuno -> #launchpad-meeting
<beuno> barry, sorry I wasn't very useful
<beuno> was on a call  :)
<barry> beuno: no, it was cool :)
<mrevell> kfogel: ping, not urgent
<barry> beuno: so, about menu links.  currently, navmenus (and only those i think) hide links for the page that you are viewing.  i don't like this :).  i've heard that in breadcrumbs the link to the page your viewing is still shown, but it is dimmed and not clickable.  i personally don't like that either (i'd rather the link be active but otherwise visually indicative).  we're looking for some ui pronouncement on
<barry> and fix the tests
<kfogel> mrevell: pong placeholder -- on phone, but will ping when off
<beuno> barry, could you tell me more about why you want them clickable?
<barry> beuno: i'm sure it's not a good reason, but i very often use it as a cheap reload.  usually the active link is way closer to my cursor than that leeeettlleee teenie browser reload button way up there in the upper left
<beuno> barry, I understand
<beuno> the problem is, it may misslead you to think it will take you somewhere else
<beuno> or "different"
<barry> beuno: do you think we could do something visual to indicate it's the page your one, but leave the link active?  like grey it down a bunch or something?
<beuno> barry, I'm thinking
<beuno> I'm trying to weigh in the cost/benefit
<beuno> "make barry happy" weighs quite a bit, but, OTOH, we have like 500k users...
<barry> beuno: :)  okay, so here's the scenario...
<barry> beuno: i just clicked on "View all sprints" and i'm transported to /sprints/+all.  now i kind of remember i'm on that page because i clicked it recently, but maybe someone pinged me in irc that they just added two new meetings.  i need to reload the page.  what are my options?  (note this happens fairly often with merge proposals and bug comments)
<beuno> that's a great scenario to address
<beuno> now
<beuno> barry, do you use gmail?
<barry> beuno: no
<beuno> ok, what gmail does
<beuno> as a web application
<barry> well, i have a gmail account, but... ;)
<beuno> is that if something changes while your on the page
<beuno> it tells you
<beuno> or it just updates it
<beuno> depending on the scenario
<beuno> *that* is the proper solution
<beuno> of course, that's a month's worth of work right there
<barry> beuno: agreed!  that would be awesome, and yep, it's a lot of work
<barry> beuno: i have another use case, but it's very launchpad developer centric, so i don't expect accommodation for it ;)
<beuno> barry, I still think the potential confusion out-weighs the benefits
 * beuno nudges mpt 
<beuno> have any thoughts? ^
<barry> beuno: so, do you agree that the link should still be displayed (i.e. not hidden)?
<barry> beuno: it feels like a bug when the link disappears
<beuno> barry, this is the las breadrumb?
<barry> beuno: my current concern is the navmenus
<mpt> beuno, barry: I don't think pages should contain a link to reload themselves unless it's probable that the page contents will have changed since you loaded it. (E.g. Gmail's link to reload the Inbox, or Twitter's "121 more results since you started searching. Refresh to see them.")
<beuno> barry, what's a navmenu on the UI these days?  :)
<mpt> beuno, barry: And where that's true, it should be more prominent than any global navigation will ever be. :-)
<barry> beuno: it's what i'm calling the menu on the top level collections and their +all pages
<beuno> barry, ah, top right?
<barry> beuno: on the right portlet
<beuno> barry, I see
<beuno> the link should not go away, no
<barry> mpt: yes. i (mostly) like the way e.g. facebook does it
<barry> beuno: cool.  so it sounds like you and mpt want the link to deactivate (i.e. not be clickable)
<beuno> barry, yes, I think that's the right thing to do
<barry> beuno: okay.  so, if the link has an icon, and you're sitting on its page, the icon stays and we put unclickable link text instead of the <a href>.  sound about right?
<beuno> barry, it does
<barry> beuno, mpt fab!  thanks very much, i'll make that happen
<barry> bac: ^^ (summary: page you're on will show an unclickable link in the navmenu)
<bac> barry: that sounds great.
<adeuring> where should a right side portlet go in the 3.0 layout that is not an action, or an notification or anything else mentioned on the conversion page? My specific problem: the "Current details" portlet of pages like this: https://code.staging.launchpad.net/~bjornt/launchpad/dont-flash-overlay-on-bug-page/+bug/107247/+delete
<mup> Bug #107247: Launchpad bug pages trigger caret browsing in Firefox and other Gecko browsers <mt-postupstream> <Mozilla Firefox:Confirmed> <Launchpad Bugs:Fix Committed by bjornt> <firefox (Ubuntu):Won't Fix by asac> <firefox-3.0 (Ubuntu):Triaged> <firefox-3.5 (Ubuntu):Triaged> <https://launchpad.net/bugs/107247>
<mpt> adeuring, if any of that information is relevant to your decision whether to delete it, move it to the main body. Otherwise, nuke it. That sound right, beuno?
<mpt> Actually it's rather alarming that that page isn't telling you *what* branch you're about to unlink from *what* bug...
<adeuring> mpt: deleting the portlet is too simple and straightforward ;) I think it does not contain anything interesting
 * beuno reads back
<beuno> correct
<beuno> if it's relevant information, it should go in the main body
<beuno> as mpt says, that page should *really* tell you what's being deleted
<adeuring> beuno: right; I added explicit links to the bug and the brnach to make that absolutely clear ;)
<beuno> adeuring, so the information about the branch  (ie "Merged") is not interesting, I agree
<beuno> the portlet should die
<beuno> just be sure that we don't loose important information
<jtv> intellectronica, got a moment to help with a structural-subscriptions.txt failure?  If it turns out to be hard, we can drop it but I'd at least like to shake this tree and see if anything falls out.
<jtv> structural-subscription-target.txt, rather
<jtv> by the way, why do we have anything in lib/canonical/launchpad/interfaces/ftests?  That directory should be not once but twice gone.
<intellectronica> jtv: sure (sorry, was on the phone until now so didn't notice your ping)
<jtv> intellectronica: cool, thanks.  In wgrant's branch, the very last line of that test fails.
<jtv> intellectronica: the test files a bug, and then compares the set of indirect subscribers to the set of structural subscribers.
<jtv> It expects the difference between those sets to be empty, but now suddenly no-priv is in that set difference.
<intellectronica> i'm not sure i'm looking at the same test as you. can you please paste me the full path again, maybe even the failure?
<jtv> intellectronica: File "lib/canonical/launchpad/interfaces/ftests/structural-subscription-target.txt", line 131, in structural-subscription-target.txt
<jtv> Differences (ndiff with -expected +actual):
<jtv>     - set([])
<jtv>     + set([u'no-priv'])
<intellectronica> jtv: it's important to remember that this test is run several times, with target being different each time
<jtv> intellectronica: remember?  I don't think I knew it in the first place.  :)
<intellectronica> jtv: but you will remember it from now on, right?
<intellectronica> jtv: see lib/canonical/launchpad/interfaces/ftests/test_structuralsubscriptiontarget.py
<jtv> intellectronica: I'll have dreams about it for the rest of my life.  :)
 * jtv sees lib/canonical/launchpad/interfaces/ftests/test_structuralsubscriptiontarget.py
<jtv> intellectronica: can approval of a bug nomination cause someone to become structurally subscribed to the bug?
<intellectronica> definitely not
<intellectronica> but maybe it makes the approver indirectly subscribed? i don't remember
<jtv> I'm asking since my prime suspect here is that wgrant's branch moves the automatic approval of bug nominations from the model code to the view.
<intellectronica> jtv: that doesn't make much sense. is this branch the one for exposing the functionality via the api?
<jtv> intellectronica: yup
<intellectronica> if yes, then i think we want to retain that behaviour
<jtv> The behaviour is still present, but explicitly in the view.  I think it was pre-imped with gmb.
<intellectronica> oh ok, maybe the tought of something i didn't, then
<gmb> intellectronica, jtv: Hang on, let me try to remember. There was a reason for doing it.
<intellectronica> anyway, to debug that failure, start with determining which target type it's failing on
 * jtv starts determining that
<gmb> intellectronica, jtv: So, we decided to make so that the API required explicit approval due to bug https://bugs.edge.launchpad.net/malone/+bug/253597 (the idea being that the UI should eventually change to require this, too).
<mup> Bug #253597: Not possible to nominate a bug if you have permission to approve it <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/253597>
<jtv> gmb: yes, it made sense to me as a Bugs outsider at least.  But something in the branch is breaking this test.  May be a forgotten .approve(...) somewhere.
<jtv> intellectronica: wouldn't you know it, it's the very last one: Hoary.
<jtv> intellectronica: so a new bug for Hoary seems to have lost an indirect subscriber
<intellectronica> who did you expect will have been indirectly subscribed?
<intellectronica> also, is it a private bug? those don't have indirect subscribers?
<jtv> intellectronica: I don't see anything that'd make it private.  The test doesn't say why it expects that particular result to be empty.
<jtv> intellectronica: but the unexpected subscriber seems to be the person filing the bug
<jtv> What kind of subscription do you get to bugs you file?
<intellectronica> yeah, it's a bit of a mystery. i wonder who's the idiot who wrote this
<jtv> intellectronica: well bzr only shows your name, but the code looks older than you.  :)
<intellectronica> jtv: direct
<intellectronica> jtv: ill take that as a compliment
<jtv> fine by me
<intellectronica> i mean, the test makes sense, i just don't remember why i added it
<intellectronica> i think i'll need to look a the branch to be able to help. what's the url?
<jtv> Actually, "structural_subscribers" is just the name of the variable.  It's really bug_target.bug_subscriptions.
<jtv> https://code.edge.launchpad.net/~wgrant/launchpad/export-bug-nominations/+merge/10715
<intellectronica> jtv: well, bug_subscriptions is just a shortcut for structural subscriptions to bug mail
<intellectronica> (in theory, structural subscriptions can be for mail from other modules, though none of them implement this)
<jtv> intellectronica: at one point I knew a bit about structural subscriptions, but it's been a while...  It's basically being subscribed all bugs in something (say, a product) that gets bugs attached, right?
<intellectronica> yes
<rockstar> Is canonical_url_examples.txt failing for anyone else?  I don't see how my branch could have broken it. https://pastebin.canonical.com/21514/
<jtv> rockstar: danilo ran into that one as well... there was a mailing list thread about it.
<rockstar> jtv, oh, I was searching for buildbot emails.
<jtv> rockstar: also, I see the vaunted sample-data change in there; see earlier thread, probably from karl
<rockstar> jtv, you would think that ec2 would use the sample data from the branch that provides the sample data, so I wouldn't have that problem.
<jtv> you know, someone's irc nick was to be removed from the sample data to avoid making his irc client play The Entertainer using a single tone
<jtv> (Not saying who, or it'll beep again :)
<rockstar> jtv, yeah, I know.
<jtv> intellectronica: I've got it: in the Hoary case, the test creates a bug nomination without approving it.
<intellectronica> jtv: right, and that causes the bug reporter to be indirectly subscribed?
<jtv> intellectronica: I think it was what previously caused the bug reporter to be directly subscribedâand therefore indirectly as well.
<jtv> Now that doesn't happen any more, but the test also subscribes that same person structurally, and assumes that the indirect subscriptions are a superset of the structural ones.
<intellectronica> ah, i see
<jtv> intellectronica: that's it for me tonight.  Thanks for walking me through this!
<intellectronica> no problem. and if you ever find who wrote that silly test slap him on my behalf, please
<james_w> how does LP feel about the OAuth session fixation attack that it is vulnerable to?
<james_w> We need to fix python-oauth to allow fixed clients to be written, and this requires API changes
<james_w> so I'll want to fix that in lazr.restfulclient, but I'm not sure how that fits with lp's build system
<james_w> would you need to land a fixed python-oauth at the same time?
<beuno> barry, https://dev.launchpad.net/ReviewerSchedule?action=diff&rev2=46&rev1=45
<beuno> does that complete my task?
<barry> beuno: fantastically so!  thanks.  i'll add a line for you as the current sole ui mentor
 * beuno is alone
<EdwinGrubbs> salgado: ping
<beuno> EdwinGrubbs, hi
<EdwinGrubbs> beuno: hello
<salgado> hi EdwinGrubbs
<beuno> EdwinGrubbs, about your email, I think that we should only highlight joining the team, not leaving it
<beuno> I can put together the bg graphic for joining
<EdwinGrubbs> salgado: I was wondering if you thought it was sane to move canonical.launchpad.browser.branding to lp.registry.browser.branding. The only non-registry objects that use that are sprints.
<salgado> EdwinGrubbs, sounds ok to me
<EdwinGrubbs> beuno: the graphic would be helpful. How long do you think that will be and will I be holding off on landing until then?
<beuno> EdwinGrubbs, maybe 30 minutes, if you tell me that it's easy to just show that format when you haven't joined, and the
<beuno> "leave team" link can go in the actions portlet
<EdwinGrubbs> beuno: yes, it will be easy.
<beuno> EdwinGrubbs, ON IT
<EdwinGrubbs> beuno: thanks
<james_w> ok, the API change doesn't affect lplib's usage, so there's no need to change it
<barry> beuno: when you have a chance; re: navmenu links to the page you're viewing.  i have two screenshots for your approval: https://devpad.canonical.com/~barry/sprints-index.png and https://devpad.canonical.com/~barry/sprints-all.png
<beuno> barry, ui=me
<barry> beuno: awesome!  the unlinked text has a class so we can tweak the style later if we want
<beuno> EdwinGrubbs, take this baby out for a spin: http://people.canonical.com/~beuno/bg-action-add.png
<EdwinGrubbs> beuno: awesome.
<dobey> hola!
<dobey> anyone around to talk about oauth?
<beuno> wgrant, http://people.canonical.com/~beuno/conversions.html
<james_w> dobey: the launchpad-dev might be the best place for this
<dobey> james_w: i thought that's where this is
<james_w>                    ^ mailing list
<james_w> sorry
<dobey> perhaps. was hoping to get a more immediate response though :)
<beuno> dobey, leonardr is your man
<leonardr> dobey, what's your question? (salgado knows more about oauth than i do)
<dobey> leonardr: so oauth.py has a lot of problems (which have become ever apparent over the last 1.5 months since i started trying to fix some of them)... and i was wondering what launchpad's opinion is on having a fork of it
<leonardr> are the original maintainers unresponsive?
<dobey> yes
<dobey> it takes forever to get any response out of upstream
<dobey> and aside from the code being ugly, the 'fixes' that did go in for 1.0a are still incomplete
<leonardr> dobey: i don't think there'd be a problem with creating a launchpad project that keeps a code import of the official code. then you could keep your own branches there. i think that's a pretty common pattern
<leonardr> but i don't really know anything about how launchpad is used in such situations, i could be wrong
<dobey> well i was more interested in whether launchpad would want to switch to using the fork for oauth, instead of continuing to use the broken/undermaintained oauth.py
<dobey> as i understand it, launchpadlib and the server use it, for doing things via the API
<leonardr> launchpadlib uses it, i don't think the server does
<dobey> leonardr: i guess salgado would be the person to answer that more concretely?
<leonardr> salgado could answer about the server side, but i don't think oauth.py even deals with the server side
<leonardr> no, i guess that's not true. anyway, i dunno about the server
<leonardr> but for the client, the package needs to be as easy to install as python-oauth is now, so that people can actually use it
<leonardr> you'll either need to convince whoever currently maintains python-oauth, or we'll need to include the fork as code inside launchpadlib
<leonardr> that would be james_w
<james_w> I couldn't find oauth.py used on the server
<salgado> Launchpad itself only uses OAuthRequest._split_header()
<leonardr> there you go
<salgado> which is a trivial 5-line method, so we should probably roll our own version of that and get rid of contrib/oauth.py
<leonardr> james_w, how would you handle it if oauth forked and we wanted to use the new fork? what would dobey have to do to convince you?
 * salgado files a bug
<james_w> well, python-oauth is only there to satisfy these two projects currently
<james_w> if they switch to something else then we package that instead
<leonardr> which two projects? launchpadlib and what?
<james_w> I'd just rather they do if *soon* if they are going to
<leonardr> i don't think i can promise it soon, so maybe in the next release?
<james_w> if it happens soon I'd even write the patch myself
<leonardr> well, i believe the workflow changed somewhat for oauth 1.0a, so it would be a fairly big production in which we coordinated client and server. i don't think we can do that on your timeframe and i doubt that's a patch you'd want to write
<leonardr> (salgado definitely knows more about this part)
<dobey> well if launchpadlib just uses one 5-line method, that sounds like an easy fix
<leonardr> dobey: *launchpad* uses one 5-line method
<leonardr> launchpadlib uses a lot more
<dobey> oh
<dobey> what does launchpadlib use?
<leonardr> all the request signing stuff
<leonardr> if there's no change in the workflow then if you put up your branch now and salgado and james_w and i understand it, then maybe james_w could write the patch and i could update launchpadlib and we could get everything into karmic, but i don't know what the deadline is
<leonardr> and i question whether a recent version of launchpadlib is going to be in karmic anyway
<james_w> why's that?
<leonardr> because there's an old version in there now
<james_w> and I've been working to fix that for months now
<leonardr> and upgrading to a new version would require adding a whole new package (lazr.restfulclient)
<james_w> that's why I've been complaining on bugs about lazr issues and the like
<james_w> lazr.restfulclient is sat in NEW
<james_w> I'm about to upload the new launchpadlib
<dobey> hmm
<leonardr> james_w, are there any outstanding bugs that are causing you problems?
<james_w> no, we should be good to go now
<james_w> famous last words
<leonardr> ok
<dobey> so i don't have a branch yet. but i can definitely spend all day tomorrow/friday on it if we can do it for karmic
<dobey> though hopefully i can get it all done in one day
<leonardr> dobey: how different is it going to be, what will the benefit be, and if part of the benefit is "1.0a", are there workflow changes?
<dobey> 1.0a requires changes to the server also
<leonardr> so there are workflow changes
<dobey> yes. since the security issue in 1.0 was "the workflow allows you to steal auth" they changed the workflow to fix it :)
<dobey> but i can have the client pieces continue working with 1.0 for launchpadlib, until we can get that changed
<james_w> it's not too important for most current uses of LPs API
<james_w> but I suggest you fix it anyway
<leonardr> well, we're gonna fix it, but until 10 minutes ago it wasn't considered important enough to get a fix in for karmic
<leonardr> i'm trying to figure out if it is now that someone else is doing some of the work
<leonardr> dobey: the server's going to continue to support the 1.0 workflow just so old clients don't break
<leonardr> launchpad doesn't use python-oauth for the server side, so any improvements you make to the server-side code we can't use
<dobey> hrmm, so unless the server will bail on 1.0a requests, it's probably fine for the client to send the 1.0a stuff anyway
<dobey> because i'm guessing it will just get ignored on the server, until the server supports 1.0a
<leonardr> dobey: probably. what's the difference between a 1.0 request and a 1.0a request?
<james_w> that depends on whether the client allows the server to only talk 1.0
<leonardr> without wanting to step on salgado's toes, i can say with a fair degree of certainty that launchpad will not support 1.0a by the karmic deadline
<dobey> 1.0a sends a callback in the request token request, expects oauth_callback_confirmed in that response, and expects an oauth_verifier unique id from authorization that is sent back in the access token request
<leonardr> so the karmic launchpadlib will have to allow a server to talk 1.0
<leonardr> if we can implement 1.0a on the server side and have our karmic installed base start talking 1.0a immediately, that'd be a win
<leonardr> and it sounds like implementing 1.0a wouldn't require any change to launchpadlib itself, only to the oauth library
<dobey> either way, launchpadlib isn't using a lot of the API, and should be an easy patch, and it's easy to test
<leonardr> does that sound right?
<dobey> yeah, launchpadlib probably doesn't need any changes to do 1.0a
<dobey> i have to read over the code entirely and see
<leonardr> dobey: ok, go ahead and make the change, give it to james_w who will evaluate it on its own terms as a patch to python-oauth
<leonardr> once it's released, i can make a change to launchpadlib as long as the change is no more complicated than telling oauth "use 1.0a if it's available, but don't fail if it's not."
<dobey> i'm looking at the lplib code... it might be that the only change necessary is "change the import lines"
<james_w> it will want to have a way for the caller to specify a callback won't it?
<dobey> yes but lplib doesn't use callbacks
<dobey> i'll evaluate what lplib is doing with the API and take that into consideration, and if it's necessary to do any extra changes to it, i'll make them as well
<dobey> sound good?
<james_w> leonardr: launchpadlib 1.5.1 is now in karmic
<leonardr> dobey: if lplib doesn't use callbacks, and we need to specify one to implement 1.0a, i don't see the point of rushing to get this into karmic
<dobey> leonardr: hrmm?
<leonardr> dobey: you said lplib doesn't use callbacks, and you also dais that 1.0a sends a callback in the token request
<leonardr> are those the same callback?
<leonardr> if they are the same callback, i don't see how implementing 1.0 on the server side will make launchpadlib start working just because we applied your patch to oauth
<dobey> leonardr: 1.0a specifies the value to be either an http callback which the server will use later (after the authorization request i think), or to be oob, if you aren't using callbacks
<leonardr> so you can implement 1.0a without using a callback, and it's secure? or are you really just falling back to 1.0 behavior?
<dobey> the callback itself doesn't make the communication secure
<leonardr> but can it be secure if there's no callback?
<dobey> and yeah, for lplib, i think it would just be the 1.0 behavior until the server starts doing 1.0a
<james_w> there's a new "verifier" which also plays a role
<leonardr> i'm really getting the impression that there are 3 steps here. 1) apply dobey's patch to python-oauth. 2) implement the server side. 3) change launchpadlib to accommodate the new workflow with its callbacks and verifier
<dobey> leonardr: it depends on what your definition of secure is i guess
<leonardr> each of those being a fair amount of work
<leonardr> if #3 was trivial, i could see an argument for rushing to get #1 into karmic--so that everything would start working once we put #2 in place
<leonardr> but if i have to do a lot of work to implement #3, that's not going to make it into karmic anyway
<dobey> #3 should be trivial
<leonardr> ok, take care of #1 and ping me once you're done. we'll figure out #3
<dobey> leonardr: heh, i just said you wouldn't have to do any work for lplib to do this. :)
<leonardr> i'm ok with doing a small amount of work
<dobey> ok
<rockstar> beuno, do you know how often the conversion page updates?  It seems out of date.
<dobey> leonardr: ok, i'll bug you tomorrow, thanks
<beuno> rockstar, I'm setting one up on my people.c.c account
<beuno> and I'll get it to update every hour or so
<beuno> http://people.canonical.com/~beuno/conversions.html
<beuno> that's db-devel
<beuno> up tod ate
<beuno> rockstar, I can run against devel
<rockstar> beuno, okay, that'd be much better.
 * beuno branches devel
<rockstar> I'm hoping to get the last answers ones reviewed tonight.
<beuno> awesomeness
<jml> thumper, skype is up, but other difficulties compel a short outage. I'll join the call a little late
<thumper> jml: ok
<beuno> rockstar, http://people.canonical.com/~beuno/conversions.html
<beuno> that's based on devel
<thumper> jml: ping when you're ready
<wgrant> jtv: Damn.
<lifeless> jml: https://code.edge.launchpad.net/~lifeless/testtools/startTestRun/+merge/10773
<lifeless> jml: (also see #bzr re subscribing to your own branches ;P)
<jml> lifeless, I've already reviewed it.
<lifeless> oh, cool
<lifeless> totally my bad
<jml> heh
<lifeless> jml: you own trunk
<lifeless> so I can't make it so
<jml> oh right.
<jml> I'll merge it in then
<lifeless> thanks
<lifeless> jml: you might want to ping whoever packages it too
<lifeless> FF is about-to-or-just-has happened
<jml> lifeless, ahh, good call.
<wgrant> intellectronica, jtv: Thanks for looking at that. Fixed.
<Ursinha> rockstar, could you give me a little help on template migration, pretty please?
<rockstar> Ursinha, sure, what's up?
<Ursinha> rockstar, I'm changing the template of a page that has an admin link on it, but when changing to the main_side template, as described here -> https://dev.launchpad.net/VersionThreeDotO/UI/Conversion#Mechanical%20changes, the link is gone even if my user has the right permissions
<Ursinha> rockstar, what can I be possibly missing?
<rockstar> thumper, on the phone with Ursinha right now, gimme a sec.
<thumper> k
<mwhudson> Ursinha: are you turning into a developer!?
<mwhudson> (if so, commiserations!)
<barry> mwhudson, thumper, jml, rockstar sorry, had some stuff to deal with.  do you guys want to have a meeting?
<rockstar> barry, yessir
<Ursinha> rockstar, thank you thank you thank you
<Ursinha> mwhudson, it seems so :)
<mwhudson> barry: i think it would be a good idea
<rockstar> thumper, chat after the reviewer's meeting?
<barry> rockstar, thumper, mwhudson, jml -> #launchpad-meeting in 2m
<intellectronica> wgrant: np. thanks for fixing this bug. i know for sure that it will make many people very happy!
<wgrant> intellectronica: We can hope.
<wgrant> I thought I had run the whole lp.bugs suite over it, but I must have been confused by the other Bugs branches I was working on at the time...
<wgrant> jtv: /win 4
<wgrant> Gah.
<pochu> I'm getting timeouts when reporting bugs in Ubuntu, can somebody look at what's going on?
<pochu> e.g. OOPS-1334H2895
<pochu> (both in edge and production)
<wgrant> jtv: All lp.bugs tests now pass with r9238.
<mwhudson> pochu: this sort of thing is better talked about in #launchpad btw
<mwhudson> pochu: i imagine it's the dup search query
<wgrant> Easy way to work around it is use fewer terms initially, then correct the summary on the second page.
<pochu> mwhudson: oh okay, it's been a long time since I come here :)
#launchpad-dev 2009-08-27
<mwhudson> woohoooo inbox zero
<jml> mwhudson, grats
<lifeless> is there any chance we could get rid of the 'no new merge proposal while one is outstanding' limit of code reviews?
<lifeless> I'm having to lie to the system at the moment
<jml> lifeless, perhaps...
<lifeless> so my work flow is
<jml> lifeless, what are you trying to do?
<lifeless> hack on a branch a lot
<lifeless> send of bits to be reviewed when I stop
<lifeless> like this morning, did an hour on the test suite
<lifeless> sent it off
<lifeless> code-reviews complained at me that there was a MP already open
<lifeless> so I go to that MP https://code.edge.launchpad.net/~lifeless/bzr/test-speed/+merge/10720
<lifeless> tell it its merged (its not, just approved)
<lifeless> http://pqm.bazaar-vcs.org/ is busy merging it
<lifeless> then send in my new mp
<lifeless> https://code.edge.launchpad.net/~lifeless/bzr/test-speed/+merge/10775
<jml> I see.
<jml> sounds not entirely dissimilar to a race condition
<lifeless> with BB we just said 'send -r -2..'
<jml> hmm.
<jml> putting the constraint on branches, rather than on, say, revision ids, seems to be the mistake here.
<lifeless> I think its fluid
<lifeless> but yes, thats part of the thing driving the issue
<lifeless> If I were to put it as a use case, it would be: 'a cherry pick merge submission sent while an earlier submission on the same branch is active should be accepted and shown separately'
<lifeless> hmm
<lifeless> two scenarios actually
<lifeless> the second is
<lifeless> 'a merge subission sent while an earlier submission on the same branch is active adds to that (or resubmits automatically?) rather than being rejected'
<lifeless> jml: ^ what to do with this
<jml> lifeless, sorry, the phone rang...
<jml> brain occupied at present :)
<spiv> jml: yes, I agree, I mentally think of merge proposals as being for revisions not branches (as branches can change, but the diffs (and reviews!) don't automatically change with them).
<wgrant> Diffs will start changing soon, won't they?
<thumper> wgrant: yes
 * mwhudson lunches
<lifeless> spiv: the followon changes can either be an extension (fixups) or new (different change)
<spiv> jml: and I've also wanted what I think lifeless is asking for here, to be able to file a series of merge proposals off one branch.
<lifeless> I'd be wary of keying too closely into revision id
<spiv> lifeless: agreed.
<lifeless> spiv: uick call?
<spiv> lifeless: sure
<spiv> I'd be happy for LP to make it obvious that there are more revs since the diff was generated, for instance.
<spiv> And for LP to make it easy to refile for the new revs.
<binarymutant> is this the right irc channel for mailing list administration?
<jml> it's close enough :)
<binarymutant> jml, my loco is having problems with it's list :(
<binarymutant> we can't access it
<jml> what's actually happening?
<wgrant> binarymutant: Aren't LoCo mailing lists on lists.ubuntu.com?
<wgrant> Launchpad has nothing to do with lists.ubuntu.com.
<binarymutant> ty for the info wgrant :)
<wgrant> Maybe #canonical-sysadmin, but I'm not sure.
<wgrant> Or #ubuntu-locoteams.
<binarymutant> thank you
 * mwhudson reappears
<mwhudson> spm: hello
<spm> mwhudson: heyo
<mwhudson> spm: can i get you to run some queries?
<spm> mwhudson: only if you use the magic word
<mwhudson> spm: select count(*) from codeimport where git_repo_url like '%sourceforge%';
<mwhudson> spm: please
<spm> bzt. wrong. word is "NOW!!!!" :-)
<mwhudson> :)
<jml> sinzui, you around?
<sinzui> I am
<wgrant> Does anybody else find 3.0 text to be very squished? It's all a lot closer together and less readable than 2.0.
<jml> wgrant, it's hard for me to say, since Firefox 3.5 seems to have different font behaviour to Firefox 3
<jml> got a couple of URLs to compare and contrast?
<wgrant> jml: Look at the comment at https://code.edge.launchpad.net/~wgrant/launchpad/structural-subscription-security/+merge/10776/comments/28159/+reply. But it affects non-monospaced text as well.
 * wgrant finds a good example.
<wgrant> (does anybody want to land that branch for me, since gary_poster's concerns are unfounded and he's gone?)
<mwhudson> wgrant: sure
<wgrant> mwhudson: Thanks.
<jml> mwhudson, oh, and not having the actual failures in the buildbot failure emails :)
<jml> kfogel, hello
<kfogel> jml: hey
<mwhudson> and in other news, if people could stop pushing over non lefthand parents to lp-dev-utils trunk...
<kfogel> mwhudson: ?
<wgrant> mwhudson: I noticed that going on in lazr.restful a lot.
<mwhudson> kfogel: revno's of lp:lp-dev-utils are a bit useless
<mwhudson> because people merge their branches by merging from trunk, then pushing over trunk
<wgrant> It also results in "Merge from trunk" commits down the LHS of trunk.
<mwhudson> right
<kfogel> mwhudson: it's hard for most people to remain that conscious of the ins and outs of their vc system, I think.
<jml> mwhudson, I don't recall doing that, but my sincere apologies if I did
<mwhudson> jml: i don't think it was you
 * mwhudson lays down the law on the mailing list
<thumper> mwhudson: did you get the query run to see how many git imports from sourceforge we have?
<mwhudson> thumper: yes
<mwhudson> thumper: there were 11, i think i fixed all of them
<thumper> mwhudson: cool
<mwhudson> well, we had some where someone had requested the new form already
<mwhudson> fscking bzrtools
<jml> phew. all read.
<jml> now I need to incorporate the "Build Branch to Archive" thread into the spec
<mwhudson> wgrant: seeing as you were asking earlier: https://dev.launchpad.net/BuildEngineer
<jtv> wgrant: right ho, I'm on it.
<wgrant> jtv: Thanks.
<jtv> wgrant: good thing we're in similar timezones!
<wgrant> jtv: Indeed!
<mwhudson> jtv: oh i meant to ask, have you been using staging codehosting over the last two weeks?
<jtv> mwhudson: no
<jtv> why?
<jtv> (It can't be much longer than two weeks, but I'm sure it's been two)
<mwhudson> jtv: because you were trying to use it when i left, and i was trying to fix it
<mwhudson> wgrant: btw, do you have a commit message for https://code.edge.launchpad.net/~wgrant/launchpad/structural-subscription-security/+merge/10776? ?
<mwhudson> (sorry got horribly distracted)
<jtv> mwhudson: thanks for asking... there were some more problems, but those were clear from production logs so we could fix, cherrypick, and see that the problem was gone.
<wgrant> mwhudson: I don't. Feel free to make something up, or convince me to.
<mwhudson> jtv: the answer i was hoping for was "yes, we used staging codehosting a lot and it all worked fine"
<jtv> wgrant: does that latest revision fix both failures?
<jtv> mwhudson: sorry to disappoint!
<wgrant> jtv: It does.
<jtv> wgrant: great, I'll send it back in
<jtv> nice little three-way series of conversations here...
<mwhudson> wgrant: "move structural subscription security checking into the model code" ?
<jml> mwhudson, wgrant: a thing I'd like (and might do myself), is write a simple script to make an ec2test submission given only the merge proposal URL.
<wgrant> mwhudson: Sounds fine!
<wgrant> jtv: Thanks.
<wgrant> jml: That would be good.
<wgrant> And public ec2test even better.
<wgrant> mwhudson: Thanks for making that page public.
<jml> mwhudson, do you want me to file bugs for any of the things I mentioned earlier re pain points?
<mwhudson> jml: yes please
<wgrant> jtv, mwhudson: Are both my branches happily ec2testing?
 * jtv checks
 * wgrant is about to vanish for a couple of hours.
<jtv> wgrant: not detached yet, but still working
<jml> heh "
<jml> Moving to launchpad since lp-devel-infra project is deprecated.
<jml> "
<jml> https://bugs.edge.launchpad.net/launchpad-foundations/+bug/106936
<mup> Bug #106936: daily review mails <email> <infrastructure> <Launchpad Foundations:Confirmed> <https://launchpad.net/bugs/106936>
<wgrant> jml: Bah. Sounds like it needs reviving.
<mwhudson> wgrant: the one i'm doing is
<wgrant> jtv, mwhudson: Great, thanks.
<mwhudson> wgrant: it's not detached yet, but nearly there
<jtv> wgrant: mine just detached
<jtv> mwhudson: race you!
<jml> mwhudson, I've updated the BE wiki page with links to tags, and have moved a few bugs to be b-i tagged
<mwhudson> jml: thank you
<jml> mwhudson, I've probably let some junk through
<jml> mwhudson, and now I've filed a large number of b-i bugs
<mwhudson> jml: if only you could subscribe to a tag
 * mwhudson is reading buildbot code, please send chocolate
 * thumper sends mwhudson chocolate
<jtv> wgrant: Jesus loves you, but He doesn't seem to care for your branch much.  Got the same failures. Did you push?  The test run was on 9235, not 9238, and that was right off the LP branch not off my copy.
<wgrant> jtv: Huh.
 * wgrant checks LP.
<wgrant> jtv: I see 9238 on LP.
<wgrant> jtv: Maybe you caught it before it was mirrored.
<jtv> wgrant: hm, maybe...  Was some time after you pinged me though.  :(
<wgrant> Strange.
<jtv> I'll just try again.
<wgrant> Thanks.
<jtv> wgrant: ha!  That was an old email.  The new one is still running.
<wgrant> jtv: Ahhh. That makes more sense.
<thumper> \o/
<thumper> finally!!!
<thumper> got person-product active reviews working
 * thumper goes back to the old failing pagetest
<thumper> some reviewer is going to hate me
 * thumper wanks the 2000 line review limit back
<thumper> oops
<thumper> typo
<wgrant> hahaha.
<thumper> damn muscle memory
<thumper> my branch is getting bigish
 * wgrant just got a 1400 line refactor branch down to 800ish.
<wgrant> Meant replaying the revisions doing bzr mvs slightly differently, though :(
<thumper> this branch fixes activereviews for people, and adds project, source package, distro source package and person product active reviews
<wgrant> Impressive.
<thumper> well
<thumper> most of the new ones were pretty simple
<thumper> most of the work is done in branch collection
<thumper> one of jml's great ideas
<jml> mwhudson, have you eod'd?
<lifeless> thekorn: grats
<lifeless> meh
<lifeless> thumper: grats
<mwhudson> jml: more or less
<jml> mwhudson, ok.
<thumper> lifeless: for
<thumper> ?
<lifeless> 15:48 < thumper> this branch fixes activereviews for people, and adds project, source package, distro source package and person product active reviews
<jml> anyone here use dchroot a lot?
<thumper> lifeless: ah, ok :)
<jml> I have this problem where I constantly have to manually update my /etc/resolv.conf
<jml> hmm
<lifeless> jml: I have been known to
<lifeless> jml: the one in the chroot?
<jml> lifeless, right.
<lifeless> what updates your external resolv.conf
<lifeless> whatever that is, hook into that and write to your chroots
<jml> NetworkManager
<jml> hmm.
<lifeless> jml: welcome to the house of fun
<lifeless> la-de-la-de-da
<jml> :)
<adeuring> good morning
<jml> hi
<jml> adeuring, I've just started getting errors like these whenever I run any tests:
<jml> There were 1 imports of names not appearing in the __all__.
<jml> You should not import export_destructor_operation from lazr.restful.declarations:
<jml>     canonical.launchpad.interfaces.hwdb
<jml> adeuring, know anything about it?
<lifeless> jml: version mismatch?
<lifeless> jml: I'm thinking a change to lazr.restful that you don't have yet, to make that public.
<jml> lifeless, lazr.restful is managed in buildout, and my buildout is up-to-date.
<jml> which doesn't preclude a mismatch in some version controlled config file, of course.
<adeuring> jml: as lifeless said: you are using an lod version of lazr.restful. I I landed a lazr.restful branch  that includes export_destrutcor_operation in __all__
<adeuring> s/lod/old/
<maxb> I've just seen lp-dev-utils mentioned for the first time on list. Is that something intrinsically Canonical-internal, or should it be public but isn't?
<jml> ok, so it's _not_ in buildout
<jml> adeuring, thanks
<jml> maxb, it's in the process of being made public
<jml> maxb, some stuff will be kept private
<jml> bigjools, hi
<bigjools> morning!
<gmb> Morning peoples.
<jml> gmb, good morning
<jml> bigjools, I've got a bunch of stuff I'd like to chat with you about
<bigjools> jml: sure thing
<jml> bigjools, but I'm not sure I have brain left today
<bigjools> jml: what time will you be back in your morning?
<henninge> staging does not let me do QA ... ;(
<henninge> it times out
<jml> bigjools, UTC 21:30
 * henninge is amazed what a little complaining can achieve ... ;)
<bigjools> jml: heh, not sure *I'll* have brain left then.  Tell you what, I'll see how I feel tonight else I can get up early tomorrow
<mrevell> Morning
 * jml tries anyway
<jml> I have a note that says ProductSeries.sourcepackages is buggy.
<jml> I now want to slap myself for making notes like that.
<bigjools> :)
 * bigjools looks
<jml> mrevell, g'day
<bigjools> jml: my eyes are bleeding looking at it, and it's only 8 lines
<jml> it's not that bad :)
<jml> actually that reminds me.
<jml> I have a branch that extracts out verify_acl's logic that works, modulo some easily-fixed error message differences
<jml> there's still some more stuff I need to do on top of that, but it's a nice milestone.
<bigjools> fab
<jml> and it turns out that eyeblood stains wash out of carpet really easily
<jml> bigjools, you said I should ask you on IRC why branches are associated with distro and not sourcepackage in breadcrumbs
<bigjools> jml: indeed, and if only I could remember what I was thinking when I wrote that
<jml> bigjools, ok. you can get back to me then :)
<jml> bigjools, I was looking at the package navigation thread today (first time since I got back from leave) -- how's all that going?
<bigjools> jml: somewhat underwhelmingly
<bigjools> some pages I wanted to trash were apparently important to some people
<jml> right, I saw you mention that you wanted to get a longer list of use-cases.
<bigjools> right now I am going for the tactic of improving the pages we said we would and then coming back later with the removal questions
<jml> bigjools, good plan.
<bigjools> jml: so for the breadcrumbs, what are the relative pros and cons
<jml> bigjools, sourcepackage is more consistent with the way product branches work
<jml> bigjools, and it's more consistent with the code
<jml> bigjools, but I don't find either of these to be compelling
<gmb> Hmm. So, maybe pasting a huuuuuge list into an iharness session wasn't the best idea I've ever had.
<jml> bigjools, you can also navigate from the sourcepackage to distro, distroseries, and distributionsourcepackage more easily
<mac_v> hi... when a branch is created how do i make the branch private?
<jml> (btw, lets rename SourcePackage)
<bigjools> fuck yes
<bigjools> jml: so, the breadcrumbs would look like -> Ubuntu/Karmic/source/branch ?
<jml> bigjools, visually? I would have guessed "Ubuntu" -> "source in karmic" -> branch
<bigjools> yeah
<bigjools> one page that is under my heavy removal scrutiny is the SourcePackage page
<jml> bigjools, currently it's Ubuntu -> 9.10 -> "hotwire" package
<bigjools> and I don't want that to block on anything branch-related if it comes to it
<bigjools> oh yeah there's a little debate about what to write for a distroseries title
<bigjools> in my DSP page I use series.title.  it doesn't include the vesion, maybe it should
<jml> bigjools, removing the SourcePackage page would probably not be a good thing for branches
<bigjools> jml: it's a useless page :(  go look at one
<jml> bigjools, https://code.edge.launchpad.net/ubuntu/karmic/+source/hotwire
<bigjools> jml: how would one navigate to that page usually?
<jml> bigjools, it's reached by the Code tab from a source package branch
<bigjools> I wonder if I can add links to it in the new DSP page?  (example here: https://dogfood.launchpad.net/ubuntu/+source/amarok)
<jml> bigjools, have you seen https://code.edge.launchpad.net/ubuntu/+source/hotwire
<bigjools> yes
<jml> good.
<jml> bigjools, one of the rationales for linking branches to SourcePackages instead of DistributionSourcePackages was so that there'd be a relevant listing of branches in each cycle of Ubuntu development
<jml> bigjools, maybe we can achieve this without a SourcePackage page
<bigjools> yep
<jml> bigjools, but it needs at least a little thought.
<bigjools> one of the problems with the package navigation is that there's waaaay too many pages with info spread all over them
<bigjools> it does need more thought
<bigjools> we should get together with Curtis
<jml> bigjools, indeed
<bigjools> jml: coming to Europe soon?  you must be due, you haven't been here for a week :)
<jml> bigjools, haha
 * jml has to go now
<jml> g'night all
<bigjools> jml: g'night, and I think +1 for your suggestion on breadcrumbs
<jml> cool.
<jtv> wgrant: Jesus feels a lot better about your branch now than I thought earlier.  It's landed.
<wgrant> jtv: Did you have some trigger for when I became unaway?
<wgrant> Because I got that message about a second after my screen reattached.
<jtv> wgrant: once Jesus gets involved, all bets are off.
<jtv> (Have I pissed off enough people today yet?)
<wgrant> jtv: Heh.
<wgrant> jtv: Thanks for the landing.
<bigjools> mrevell: will you be able to do this? https://bugs.edge.launchpad.net/soyuz/+bug/392239
<mup> Bug #392239: [PPA] "Read about uploading" link doesn't go to the anchor bookmark #Uploading <ppa> <Soyuz:Triaged by matthew.revell> <https://launchpad.net/bugs/392239>
 * mrevell looks
<mrevell> bigjools: yes, no problem
<bigjools> mrevell: super thanks
<jtv> wgrant: no worries... the next task is Q/A.
<wgrant> mwhudson: What happened to my other branch?
<jtv> gmb, can I get you to Q/A wgrant's export-bug-nominations branch?
<gmb> jtv: Sure; I can take a look in about half an hour if that's okay.
<jtv> gmb: that'd be swell, thanks.
<wgrant> jtv: We don't wait for edge?
<jtv> wgrant: for actually performing Q/A, yes, you.  But gmb is having his first look in half an hour.
<wgrant> jtv: OK.
<gmb> jtv, wgrant: So, which branch is it I need to look at?
<gmb> well, branch / commit / bug/ item on the QA page
<jtv> gmb: lp:~wgrant/launchpad/export-bug-nominations will need QA when it shows up on edge.
<gmb> Ok.
<jtv> gmb: thanks.
<deryck> Morning, all.
<bigjools> hey deryck
<gary_poster> wgrant: did you get someone to merge your structural-subscription-security branch?
<wgrant> gary_poster: mwhudson started an ec2test of it several hours ago, and another of my branches started around the same time landed 2.5 hours ago, so I think something went wrong.
<gary_poster> wgrant: ok.  yeah, I see your other one.  probably should wait for feedback to you from Michael then.
<wgrant> gary_poster: Yep, I'll hopefully find something out tomorrow.
<gary_poster> ok cool
<gary_poster> I'll note that Michael tried to land the branch on the MP so anyone following along there knows we aren't ignoring you ;-)
<wgrant> Thanks.
<bigjools> I am going to shoot the next person who uses regular expressions in a test >:(
<wgrant> bigjools: You just wait until you start using renormalizer to get traceback tests working on 2.5...
<leonardr> barry, can you help me with a python problem?
<leonardr> i'm getting a metaclass conflict even though i'm pretty sure all my classes have the same metaclass
<leonardr> here's the code that's causing the problem
<leonardr> https://pastebin.canonical.com/21549/
<leonardr> i pass the result of make_configuration_superclass into make_configuration and get the error when it tries to use the superclass as a superclass of a new class
<leonardr> but i'm using type() everywhere, and __metaclass__ = type in all affected files
 * barry looks
<barry> leonardr: do you have a full traceback as well as the offending code?
<leonardr> barry, sure
<barry> leonardr: or maybe better, a branch
<leonardr> barry, i'll push the branch
<leonardr> give me just a minute to clean up
<barry> no worries, i'll be here :)
<leonardr> actually, maybe i can just make it simpler and destroy the problem that way
<matsubara> stub, Chex, gary_poster, rockstar, bigjools, henninge_, sinzui, intellectronica, Ursinha-afk: meeting in 10 min @ #launchpad-meeting
<matsubara> s/meeting/LP production meeting/
<sinzui> barry:  can you stand in for me in the LP product meeting ^
<intellectronica> matsubara: thanks for the reminder
<barry> matsubara, sinzui sure
<matsubara> henninge_, meeting
<matsubara> actually now danilos attends the production meeting, right?
<henninge> matsubara: at least today, I am about to leave
<matsubara> henninge, ok. danilos showed up. thanks!
<abentley> danilos: Any idea why this would fail this way?  Is order important? http://pastebin.ubuntu.com/260413/
<barry> leonardr: good to see you figured it out
<danilos> abentley: it's a recent commit by jtv, I'd have to take a closer look
<leonardr> barry, thanks for your potential help
<barry> leonardr: i'm always potentially helpful.  sadly, the converse is often true too :)
<danilos> abentley: can you fix the problem using assertContentEquals and that should be fine (if you are submitting something to pqm)
<abentley> danilos: checking it now...
<danilos> abentley: every other test works with only one so it's not a problem right now, but if this works for you, please email jtv@canonical.com (CCing me) mentioning what you had to do, and then the two of us can figure out what's the best way forward
<abentley> danilos: self.assertContentEquals isn't defined, and I'm not sure what you mean.
<abentley> danilos: Found it: assertContentEqual
<abentley> danilos: This does work for me.  I'll include it in the branch I land.
<danilos> abentley: thanks, please do still email jtv@ and danilo@ with this; thanks and sorry for it causing a problem like this!
<bac> hi bigjools
<mrevell> night all
<intellectronica> are people seeing failures in lib/lp/app/browser/tests/menu.txt ?
<intellectronica> i just got failures from ec2 and i don't think it has anything to do with my changes
<salgado> intellectronica, I just saw a landing that changed something there
<bac> intellectronica: barry has a testfix branch running now
<intellectronica> bac: cool, thanks
<barry> beuno: ping
<barry> intellectronica, bac that should already be landed in devel
<beuno> barry, hi
<barry> beuno: i have a quick u/i-ish question for you related to bug 51735
<intellectronica> barry: it has indeed. i got the failure from a branch i sent to ec2 (number of hours it takes to run the full test suite) earlier
<mup> Bug #51735: top level distros page could be improved <ui> <Launchpad Registry:Triaged by barry> <https://launchpad.net/bugs/51735>
<barry> intellectronica: the joys of a long test suite
<beuno> barry, sure, what's up?
<barry> beuno: at the bottom of that bug report sinzui suggests changing the way the read-only asterisk is displayed, but that also (i think) subtly changes the semantics of that star
<barry> beuno: right now the ui says about that footnote: * Read-only distribution. Can be browsed in Launchpad, but not altered or managed, and its information might be out of date.
<barry> beuno: but with sinzui's suggested change it really means "does not use Launchpad in any way"
<sinzui> barry: it does change it.  if you read the code, we have a hack in it
<barry> sinzui: yep, i've seen that
 * beuno looks at the bug
<beuno> so, has anyone tried LP on karmic?
<beuno> will I be shooting myself in the foot (again) if I upgrade my work laptop as well
<beuno> ?
<barry> beuno: not yet, though i did install karmic on a vm last night.  haven't tried building lp on it yet
* beuno changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 2.2.8 | https://dev.launchpad.net/ |  Please use #launchpad for support. | https://launchpad.net/~launchpad-dev | Get it: https://dev.launchpad.net/Getting | http://people.canonical.com/~herb/ | http://paste.ubuntu.com/ | 3.0 goal is to convert all templates: http://people.canonical.com/~beuno/conversions.html
<beuno> barry, I think we don't have the deps in the PPA for karmic
<beuno> actually, it seems we do
<beuno> I'll bite
<mwhudson> wgrant: that ec2test run hung, it seems :(
<thumper> sinzui: are you around?
<sinzui> I am
<thumper> ok, just checking
<thumper> 1 hour
<sinzui> thumper: I am on vacation
<thumper> sinzui: then why are you around?
<sinzui> Do you want to talk about pygtk hacking?
<thumper> sinzui: so our call is off?
<thumper> not really
<thumper> sorry
<thumper> I'll grab rockstar then :)
<rockstar> thumper, hi
<mwhudson> gary_poster: hello
<gary_poster> mwhudson: hiya :-)
<mwhudson> gary_poster: so do you remember what the problem was with the force build button when you turned it on?
<gary_poster> mwhudson: I'm pretty sure that it simply did nothing.
<gary_poster> It *possibly* was worse than nothing--causing some kind of glitch, but I lean towards nothing.
<mwhudson> ok
 * mwhudson is reading buildbot source code
<mwhudson> and as usually, i'm getting a little dizzy
<gary_poster> heh
<gary_poster> kfogel: did you happen to propose to flacoste that we put lp-dev-tools in the lp tree?  if not, would you mind sending a note to the list proposing that, pointing out that they use launchpadlib and we don't have anything else that uses launchpadlib in the tree right now?  I don't care personally, and like the idea of our dev tools being in the tree, so I'd like to see if anyone objects.
<kfogel> gary_poster: will do right now
<gary_poster> thx!
<kfogel> gary_poster: why the nick change?
<kfogel> gary_poster: note that the lp-dev-tools should then get whatever license the other scripts in there have.  If it's AGPLv3, then so be it.
<gary_poster> kfogel: somebody claimed "gary" on freenode a long time ago
<kfogel> gary_poster: oh, I didn't notice we're on freenode
<gary_poster> I'm still gary on canonical.com
<gary_poster> ("gary" will still ping me in my client though, just in case)
<kfogel> barry: is 'bip' an irc client?
<barry> kfogel: it is
<barry> kfogel: it runs in the background and connects to multiple irc servers.  then i use erc (in emacs 'natch) to connect to localhost's irc server from bip's daemon
<barry> kfogel: it generally works well, except for some warts in both bip and erc that have not yet been worth hacking on yet
<kfogel> barry: someday I should watch you use ERC.
<kfogel> I tried to use some Emacs IRC client (maybe it was erc, not sure) a while back and just wasn't happy, so went to XChat.
<barry> kfogel: modulo warts, it's changed my life :)
<kfogel> But I still long for Emacs editing in IRC.
<barry> kfogel: xchat irsii etc all drove me nuts
<kfogel> xchat does drive me nuts, yes :-)
<barry> kfogel: erc+bip has not yet driving me nuts
<barry> kfogel: there are two irc clients for emacs iiuc.  erc seems to work pretty well, though there are some changes i'd make if i had the swine flu
<kfogel> beuno: ping
<beuno> kfogel, hi
<kfogel> beuno: privchat, one sec
<gary_poster> maxb: I saw your reply to bug 413335.  It would be awesome to have a fuller description and an example.  Will you have a chance to do either of those soonish--like by early next week?  If so, I'll wait for you.
<mup> Bug #413335: Figure out what to do about zope.sendmail incompatibility with Python >= 2.5.1 <python-upgrade> <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/413335>
<maxb> gary_poster: yes, I'll definitely sort it over the weekend if I don't before
<gary_poster> maxb: awesome.  ok, thanks
<thumper> rockstar: skype?
<rockstar> thumper, sure.  Early 1-1 then?
<thumper> rockstar: I actually wanted to talk to you about a review :)
<rockstar> Are you sure you can take that much rockstar in one day?
<barry> kfogel: btw, irc + dabbrev ftw!
<rockstar> thumper, 2 min and I'll be ready
<thumper> ok
<kfogel> barry: I totally believe it
<rockstar> thumper, ready
<wgrant> mwhudson: Are you trying again?
<mwhudson> wgrant: not yet
<mwhudson> wgrant: i think i may be being hated by my router, let's get someone else to run it
<mwhudson> thumper: can you run "~/canonical/repos/lp-dev-utils/trunk/ec2test.py --headless -s "[r=gary][ui=none] move structural subscription security checking into the model code" lp:~wgrant/launchpad/structural-subscription-security --email=me@williamgrant.id.au" please?
<thumper> mwhudson: ok
<wgrant> Huh. Lots of people going on leave and leaving and changing position right now.
<mwhudson> hey wouldn't it be nice if i could click force build on buildbot now
<mwhudson> spm: there?
<mwhudson> or mthaddon
<mthaddon> mwhudson: 0/
<mwhudson> mthaddon: can you edit line 267 of buildbot's master.cfg to pass allowForce=True and bounce buildbot pls?
<mthaddon> mwhudson: what does that do?
<mwhudson> mthaddon: it makes the force build button appear on the ui
<mwhudson> mthaddon: which doesn't work, but i'd like to fix that, so debugging info would be useful
<mthaddon> mwhudson: is currently: c['status'].append(html.WebStatus(http_port=8010)) # , allowForce=True))
<mthaddon> mwhudson: so what should that be?
<mwhudson> mthaddon: also, are the buildbot twistd.log files copied anywhere currently?
<mthaddon> mwhudson: they aren't currently no - can you RT that?
<mwhudson> mthaddon: c['status'].append(html.WebStatus(http_port=8010), allowForce=True)
<mwhudson> mthaddon: sure
<wgrant> mwhudson: Misplaced paren.
<mwhudson> ah yes
<mwhudson> c['status'].append(html.WebStatus(http_port=8010, allowForce=True))
<mthaddon> mwhudson: 	exceptions.SyntaxError: invalid syntax (master.cfg, line 267)
<mthaddon> mwhudson: nm, paste fail
<mwhudson> :)
<mwhudson> mthaddon: rt #35484 btw
<mup> Bug #35484: Won't load X  <Ubuntu:Invalid by ubuntu-x-swat> <https://launchpad.net/bugs/35484>
<mwhudson> mup: shut it
<mthaddon> thx
<mwhudson> mthaddon: is the patched buildbot running?
<mthaddon> mwhudson: I've made that config change and restarted buildbot, yep
<mwhudson> hmm
<mwhudson> oh right
<mwhudson> latent buildslaves appear to be disconnected the whole time
<mwhudson> i guess we'll be needing to patch buildbot a bit then
<mwhudson> gary_poster: is there a buildbot irc channel or anything?
<mwhudson> ah, the obvious answer works :)
<gary_poster> :-)
<jml> hello all
<spm> hey jml
<jml> spm, hello :)
<jml> beuno, hi
<beuno> jml, hai
<jml> beuno, I'm working through your email now
<beuno> jml, it's starting to feel more like a wiki than an email   ;)
<jml> beuno, yeah, I know.
<jml> beuno, one more round from me :)
<spm> mwhudson: am now. but I gather tom's assisted?
<mwhudson> spm: yes, thanks
<spm> cool
<beuno> jml, give me your best shot, it's fun
<jml> beuno, I was thinking just that -- it's quite fun :)
<beuno> jml, I think that this is what google wave will be all about
<beuno> real multiplayer email
<wgrant> I think the alignment of the main branding heading is a bit strange. Would it look better if it was aligned with the LHS of the Overview button, rather than the text, I wonder?
<rockstar> barry, do we have a standard where our pagetests now need to be restructured text?
<barry> rockstar: yes
<barry> rockstar: w/conversion as you go
<rockstar> barry, thanks.
<intellectronica> i'm getting lots of errors in test_breadcrumbs. is that a known problem?
<jml> mwhudson, https://bugs.edge.launchpad.net/launchpad-foundations/+bug/420198
<mup> Bug #420198: ec2test --headless takes way too long to detach <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/420198>
<lifeless> jml: hi
<lifeless> jml: I'm guessing you make test bzr servers in the codehosting tests
<lifeless> you may find http://bazaar.launchpad.net/~lifeless/bzr/test-speed/revision/4661 relevant
<jml> lifeless, a little.
 * jml is reminded of other
<jml> code
<lifeless> spiv: I wonder if ChrootServer should take a server on its setUp rather than a transport on its __init__ (we could make a server that just claims to be at an arbitrary url)
<lifeless> spiv: that would be more consistent with other Servers
<mwhudson> yeah, we do that sort of thing a fair bit
<jml> lifeless, we have a function in our base TestCase that does foo.setUp(); self.addCleanup(foo.tearDown)
<mwhudson> TestCase.installFixture woo
<lifeless> jml: thats close to this indeed
<jml> but I don't think we use backing servers in quite the same way
<jml> there's also a base class called Fixture
<lifeless> jml: this is preparation for another change that will deny all access to urls that aren't registered with the test case
<jml> which has its own addCleanup implementation
<lifeless> jml: so the reason I'm headsuping you and mwhudson et al
<jml> is you're going to break our test suite?
<lifeless> is that when you start grabbing from the post 2.0 releases, its going to be brutal
<lifeless> assuming it gets through review
<lifeless> I'm changing from a 'setup things so that bad code is contained' to 'contain bad code by interception'
<lifeless> model for the test isolation code
<lifeless> less work++
<jml> lifeless, http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/annotate/head%3A/lib/lp/codehosting/scanner/fixture.py
<lifeless> nice
<lifeless> uhm, might want to split the zope stuff from the support stuff
<lifeless> just a thought
<jml> yeah
<jml> it's totally in the wrong place
<jml> but it's a whole lot better than thinking about it how good it would be to have something like that :)
<jml> man, filing all of these bugs for the build system has made me want to do something about them
<mwhudson> oop
<mwhudson> s
#launchpad-dev 2009-08-28
<cprov> wgrant: ping
<mwhudson> firefox is annoying the crap out of me today
 * mwhudson lunches
<wgrant> cprov: Hi.
<wgrant> cprov: Sorry, have to run off to a meeting for a while.
<thumper> I'm trying to decide between an 80gig and 160gig intel SSD for my new laptop
<thumper> $300 vs $600 USD
<thumper> is it worth spending the extra for more space?
<mwhudson> 80 gigs is pretty big unless you expect to have heaps of photos/video/music on there
<spm> ... or a launchpad dev environment. ? :-P
<mwhudson> mind you, i'm using 95% of 48 gig currently....
<mwhudson> spm: :)
<mwhudson> 2a really is a lot better on that front
<spm> until bzr can make me a sandwich, it's not good enough
<thumper> mwhudson: yeah, I have 45gig allocated on this laptop right now
<thumper> mwhudson: if I had all 80 assigned to linux, I'll have almost twice the size
<thumper> it is just tempting to get the biggest
<thumper> If I just have a portable pocked HD with eSATA, perhpas that'll work for big backups and video store
<spm> thumper: do you get tax deals around laptops similar to here in oz? if so, get the biggest most expensive; is almost as cheap in the longer run.
<thumper> spm: no idea actually
<thumper> Yay, my redirect view works
<spm> ahh. worth chasing then. in our case, you can claim ~ 50% (subject to bracket) back almost immediately as a work tool; and then depreciate the rest. total cost is so close to zero as makes little diff.
<spiv> spm: you can't do that anymore, actually
<spm> spiv: really!?!?! depression.
<spiv> they made it either/or at the last budget, IIRC.
<spm> gah
<spm> spiv: industry specific; or cross the board for all work tools?
<spiv> laptop/computer specific, IIRC.
<spm> eg tradies and their plumber et al specific tools?
<spm> typical
<mwhudson> spm: you can't do that here either, i think
<mwhudson> you need to depreciate assets
<mwhudson> and i don't think there are many special categories like that
<mwhudson> (for all that au and nz are similar in many ways, the tax system in au seems way more complicated than that here, which means we get fewer funny tricks to pull, aiui)
<thumper> mwhudson: do you know the rate of deprecation for laptops?
<mwhudson> i think i looked it up
<mwhudson> 33%?
<thumper> Aargg
<spiv> spm: FWIW: http://www.theage.com.au/cgi-bin/common/popupPrintArticle.pl?path=/articles/2008/05/26/1211653939561.html "Under the old rules, employees could use salary sacrifice to get these items exempt from FBT and have also been able to claim depreciation on them - providing a double benefit. The Government has cut out depreciation deductions for any FBT-exempt work-related items."
<thumper> enable_only for facets uses "branches"
<thumper> canonical_url rootsite uses "code"
<thumper> grrs
<spm> spiv: that it is depressing. no matter how fair it is in closing an abusive loophole. :-)
<mwhudson> consistency is the hobgoblin of the smallminded!
<spm> heh
<spiv> I suppose http://www.ato.gov.au/taxprofessionals/content.asp?doc=/content/00143392.htm would be slightly more authoratative :)
<spiv> spm: :)
<jtv> morning folks
<spm> hey jtv! you're in early today
<jtv> yup!
<mwhudson> no, 30% i think
<mwhudson> oh, 50% by diminishing value
 * mwhudson finally found the right bit of the document
<thumper> mwhudson: hmm.. I wonder when one can write off the last few dollars :)
<thumper> so...
<thumper> to 80 or 160? that is the question
<jml> back
<mwhudson> test failures on lp!
<mwhudson> looks like abel
<thumper> :(
 * thumper is trying to find the coloured nicks in the new quassel
<jml> heh heh
<Ursinha> 0
<Ursinha> errr
<Ursinha> :/
 * thumper on kid watch for a small while
<jml> noodles775, hello
<noodles775> ah, jml - I was just about to ask you if you could review a 1-line test-fix branch :)
<jml> noodles775, sure thing
<noodles775> (or is the reviewer schedule out of date?)
<noodles775> Great.
<jml> noodles775, it's only a little out of date, and I'm always happy to review 1-line patches :)
<noodles775> jml: http://pastebin.ubuntu.com/260692/
<jml> noodles775, r=jml
<noodles775> Thanks.
<wgrant> thumper: How'd that ec2test go?
<thumper> wgrant: you should have an email
<thumper> wgrant: it failed
<thumper> wgrant: although I'm not entirely sure it was your fault
<thumper> wgrant: there was another failure on trunk around the same time
<wgrant> thumper: I didn't get an email, AFAICT.
<thumper> arse
<thumper> neither did I
<thumper> and I've had to reboot since then :(
<wgrant> But it probably did fail.
 * jml is on the trail
<jml> back again!
<jml> bloody free wireless connections
<jml> anyone around who understands package upload permissions?
<mwhudson> jml: ha ha!
 * wgrant knows how most of that stuff works.
<jml> let me rephrase
<jml> anyone around who uploads packages regularly and has informed opinions about it?
<wgrant> I *used* to upload regularly.
<jml> hooray!
<jml> One of the error messages you can get wrt permissions is: "Signer is not permitted to upload to the component 'universe' of file 'bar_1.0-2.dsc'."
<jml> just how useful is the "file 'bar_1.0-2.dsc'" bit of that?
<wgrant> Not.
<wgrant> I never saw the point.
<wgrant> There's precisely one file in each upload which can produce that error.
<jml> so if I just say "not permitted to upload to the component 'universe'." it would be just as useful?
<jml> (I'm refactoring code I don't understand, so I want to change the behaviour as little as possible)
<wgrant> jml: Yes, that would be fine. It removes no meaning.
<jml> wgrant, thanks.
<jml> moment of truth... all the unit tests pass, and I think they're complete. will the integration tests pass?
<wgrant> jml: Is this your branch to extract ACL checks from their currently very, very integrated spot in the upload processor?
<jml> wgrant, yes, it is.
<thumper> oh no
<thumper> update didn't fix kontact
<wgrant> Karmic?
<thumper> wgrant: no
<wgrant> thumper: It broke in an update?
<thumper> wgrant: maybe, but it started crashing yesterday
<thumper> wgrant: and has crashed regularly today too
<thumper> wgrant: it is normally really solid
<wgrant> thumper: It always loved crashing with IMAP for me.
<wgrant> But that's a bit strange.
<thumper> :(
 * jml writes a massive cover letter
 * mwhudson eods
<jml> mwhudson, have a good weekend.
<mwhudson> jml: you too
 * jml will try
<jml> not that I expect to have to try too hard :)
<jml> thumper, found an odd UI bug in code reviews
<jml> thumper, look at https://code.edge.launchpad.net/~jml/launchpad/package-permission-love/+merge/10830
<thumper> jml: yes?
<jml> thumper, it's got the commit message twice.
<thumper> haha
<thumper> I'm sure I raised that before
<jml> ok cool
<jml> as long as its known
<thumper> jml: we're going to be messing around with the page anyway :)
<jml> thumper, I figured as much :)
<thumper> jml: so things will be changing shortly
<jml> thumper, want to review a branch?
<thumper> jml: after 6pm on a Friday?
<thumper> jml: is it big?
<jml> thumper, it's probably too big for after 6 on a Friday :)
<jml> I'm moving a class from one module to another
<jml> is there a way to update all the imports of it semi-automatically?
<mwhudson> jml: sed!
<jml> mwhudson, it's really quite tricky.
 * jml files a couple more BE bugs :P
<spm> jml: perl!
<spm> and on that note EOW. :-)
<jml> spm, g'night
<thumper> gah
<thumper> how do I add a bug task to an external bug tracker?
<thumper> found it
<Ursinha> *ahem*
<Ursinha> I have one failing test here, and not able to find out why... translations/doc/request_country.txt fails complaining about the lack of geoip db, and asks me to install launchpad-dependencies, which is installed
<wgrant> Ursinha: Is geoip-data-city-lite installed?
<Ursinha> wgrant, don't know, lemme check
<Ursinha> hmm, no!
<wgrant> Huh.
<wgrant> It should be depended upon.
<wgrant> Ahh.
<wgrant> Maybe only by launchpad-developer-dependencies
<wgrant> Are you missing that one too?
<Ursinha> nope, both are properly installed
<Ursinha> lp-dev-dependencies and lp-dependencies
<Ursinha> "properly"
<wgrant> Ursinha: You must have geoip-data installed, then?
<wgrant> That or -city-lite must be installed for launchpad-dependencies to be happy.
<Ursinha> wgrant, geoip-data is installed, yes
<Ursinha> I see lp-dependencies wants one or another, so it should be fine...
<wgrant> You would think.
<Ursinha> hehe
<Ursinha> let's see
<wgrant> I presume geoip-data is some proprietary thing.
<Ursinha> the $*#@*&^ seems to be working now
<Ursinha> running more complete tests to be sure
 * jml smiles
<Ursinha> so I'll write an email asking people why oh why it depends on -lite even the package saying one or another should be fine...
<Ursinha> ...if tests pass now
<Ursinha> oh well, it passed the failing test
 * Ursinha bangs head on the table
<Ursinha> jtv, it seems the problem was that ^^^ then...
<Ursinha> wgrant, how come you know that?
<wgrant> Ursinha: What do I know?
<Ursinha> that the problem was that exact package, did you have this problem before?
<wgrant> No.
<wgrant> But it was obvious enough.
<jtv> wgrant: we're becoming frighteningly dependent on you.  :)
<Ursinha> so it comes that lite package is needed, not optional?
<wgrant> Ursinha: Or your geoip-data is broken.
<Ursinha> wonder how..
<adeuring> good morning
<adeuring> good morning
<noodles775> Moin moin :)
<Ursinha> wgrant, btw, thanks muchly for the tip :)
<wgrant> Ursinha: np
<wgrant> Is mailman on forster really slow, or are my posts to launchpad-users being moderated very quickly?
<wgrant> Hm, no, it seems to hate everybody equally.
<wgrant> 5 or 6 minutes delay incoming, 13 or 14 outgoing.
<jml> g'night all
<wgrant> Night jml.
<noodles775> Enjoy :)
<Ursinha> good night jml
<Ursinha> good morning bigjools :P
<bigjools> morning duderinos
<bigjools> up early Ursinha?
<Ursinha> yes, yes
<mrevell> hi!
<mrevell> jtv: around for a ten minute discussion?
<jtv> mrevell: sure
<jmux> Hi - where can I find some information about lauchpads dbuser mapping? I want to delete an object from sectionselection, but get and "permission denied" error. I know of database/schema/security.cfg, but couldn't find any information, when which users are used.
<deryck> Morning.
<gary_poster> deryck, irt bug 399856 I see that you lowered bug 399853 to a low importance.  May I lower 399856 also? Is it still important enough for a milestone?
<mup> Bug #399856: Get LaunchpadObjectFactory working in Windmill <Launchpad Foundations:Triaged by flacoste> <https://launchpad.net/bugs/399856>
<mup> Bug #399853: Update Windmill test after plus sign in user name branch <Launchpad Bugs:Triaged by deryck> <https://launchpad.net/bugs/399853>
<beuno> noodles775_, hi!  did the new breadcrumbs get rolled out?
<noodles775_> beuno: no...
<deryck> gary_poster, it's not something I'll get back to this cycle, I'm sure.  So it's fine by me to lower 399856.
<gary_poster> deryck: cool thanks
<noodles775_> beuno: but I'm just landing another branch that fixes some display issues in konqueror (as well as getting rid of the extra space left at the top of the page), so it was a good thing.
<noodles775_> beuno: btw, if you have any other 10min branches you want me to do, bigjools says, go jump of a bridge ;)
<beuno> noodles775_, LOL
<beuno> well, time works differently in Argentina
<bigjools> it was long walk off a short bridge
<gary_poster> intellectronica: irt bug 401724, who is qualified to fix this--who can I ask?  This looks like something mars would have fixed.
<mup> Bug #401724: Collapsibles not expanding in IE8 <ui> <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/401724>
<gary_poster> noodles, is bug 411738 still in progress?
<gary_poster> sorry, noodles775_
<bigjools> the seven hundred and seventy fifth noodles
<noodles775_> gary_poster: no, we couldn't reproduce it on dogfood, and I didn't get back to check the logs (nor could another archive admin reproduce it).
<gary_poster> noodles775_: hm...should I mark as incomplete?  or just push it off to another milestone?
<noodles775_> gary_poster: sorry, wrong bug... one tic
<wgrant> Um, are we looking at the same bug?
<wgrant> Aha.
<gary_poster> :-)
<noodles775_> gary_poster: updated. I'll add a comment with the revno
<gary_poster> noodles775_: thank you very much
<gary_poster> beuno, do you happen to know the answer to what I asked intellectronica above?
<gary_poster> eh, I should have been more user-friendly, sorry.  That was, "irt bug 401724, who is qualified to fix this--who can I ask?  This looks like something mars would have fixed." :-)
<mup> Bug #401724: Collapsibles not expanding in IE8 <ui> <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/401724>
<beuno> .me reads
<gary_poster> thx
<beuno> gary_poster, no idea, tbh
<gary_poster> beuno: ok, np, thanks for looking
<gary_poster> flacoste: irt bug 401724, who is qualified to fix this--whom can I ask?  This looks like something mars would have fixed.
<mup> Bug #401724: Collapsibles not expanding in IE8 <ui> <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/401724>
<gary_poster> flacoste: also, so you know, you had assigned yourself bug 399856.  Should I unassign you?
<mup> Bug #399856: Get LaunchpadObjectFactory working in Windmill <Launchpad Foundations:Triaged by flacoste> <https://launchpad.net/bugs/399856>
<flacoste> gary_poster: yes, i didn't start on that
<gary_poster> not "so you know" :-) --the so you know was to tell you that I moved the target.  OK cool
<flacoste> regarding IE8 anybody with JS experience on the team
<flacoste> so it shouldn't have to be Foundations itself
<gary_poster> ok cool, thanks
<dobey> james_w: hey. had to get lunch but i've almost got a branch ready to push up to lp. will keep you posted! :)
<james_w> excellent
<mrevell> bye!
<dobey> hrmm
<dobey> james_w: i just pushed trunk of poauth, but for some reason the branch is private and i don't see how to change that
<james_w> dobey: which team did you make the owner?
<dobey> james_w: ubuntuone-control-tower
<dobey> which shouldn't cause that
<james_w> "Part of Ubuntu One"
<james_w> that might have done it
<dobey> yeah i think that was it
<james_w> any admins around to fix this up?
<dobey> so i just turned that off, deleted the private branch, and re-pushed
<dobey> can tweak that later
<james_w> I'm not sure I like more of the logic being pushed down in to the store implementations
<dobey> james_w: so i think ubuntuone-storage-protocol, ubuntuone-client, and python-launchpadlib can switch to using poauth with just simple import changes
<dobey> hrmm?
<dobey> there isn't more logic in the storage implementation
<james_w> e.g. def authorize_request_token(self, parameters):
<james_w> it has to pick apart parameters and act accordingly
<james_w> the previous storage implementations had a simpler interface
<dobey> james_w: parameters is a dict(), so that we don't have to break API in the event the protocol changes again
<james_w> what was the reason for that change?
<bac> james_w: i can set branch visibility
<bac> james_w: what exactly do you need?
<james_w> bac: thanks, dobey fixed it
<james_w> a new project unexpectedly inherited private-by-default
<bac> james_w: is that the setting on the project group?
<james_w> yeah
<dobey> yeah i just unset project group for now
<bac> james_w: yeah, that is the intended behavior, even if it does surprise sometimes
<bac> dobey: i can set it right, if you want
<dobey> bac: ok, i just added poauth back to the ubuntuone project group. can you make the poauth branch public by default? thanks :)
<bac> dobey, ok poauth code branches are public
<dobey> bac: thanks much
<bac> np.
<james_w> dobey: my concern is that you have created an interface that is easy to implement wrongly
<dobey> james_w: i don't consider it entirely complete yet. i wanted to get something up so that we could perhaps start switching the clients over. i want to make it so that one has to do as little implementation as possible
<james_w> ok
<james_w> it seems like you started down the opposite direction though
<dobey> i don't think so... but we'll work that out
<james_w> have you seen the python-openid store API?
<james_w> I think that would be good for something to work towards
<james_w> it is certainly well documented as to what the requirements of the store are, and leaves you just to write a shim that stores the info in whatever you are using
<dobey> i haven't, but i'll look at that
<dobey> james_w: the rest looks reasonable then?
<james_w> the client side looks ok at a glance
<dobey> should i make a release and get a package branch together then?
<dobey> james_w: well, i am about to head off toward the weekend. let me know what else i need to do, so we can get this in use and working well. :)
#launchpad-dev 2009-08-29
<lifeless> who is vikram
<lifeless> and why are they marking malone bugs 'in progress'
<lifeless> bug 420799 and bug 420807
<mup> Bug #420799: bug lists - undecided complete bugs should sort above critical <Launchpad Bugs:In Progress> <https://launchpad.net/bugs/420799>
<mup> Bug #420807: in bug lists, fix committed bugs should sort higher <Launchpad Bugs:In Progress by dhillon-v10> <https://launchpad.net/bugs/420807>
<lifeless> abentley: evening
 * wgrant wishes for an "undo that user's actions" button in malone.
<lifeless> wgrant: oh?
<wgrant> lifeless: Some triagers like to do bad things.
<wgrant> Like removing bugwatches and invalidating all three tasks on a needs-packaging bug, just because a five-year-old version was removed two years ago.
<wgrant> (even though that bug had a comment a week ago saying that the new package was almost done)
<lifeless> ugh
<lifeless> wgrant: have you seen this dhillon-v10 around?
<lifeless> 09:01 < lifeless> who is vikram
<lifeless> 09:01 < lifeless> and why are they marking malone bugs 'in progress'
<lifeless> 09:01 < lifeless> bug 420799 and bug 420807
<mup> Bug #420799: bug lists - undecided complete bugs should sort above critical <Launchpad Bugs:In Progress> <https://launchpad.net/bugs/420799>
<mup> Bug #420807: in bug lists, fix committed bugs should sort higher <Launchpad Bugs:In Progress by dhillon-v10> <https://launchpad.net/bugs/420807>
<wgrant> lifeless: I wondered that myself.
<wgrant> All I've seen is those two.
<wgrant> Makes no more sense to me than it does to you.
<wgrant> Ah, there's a post on lp-users.
<lifeless> he sure is enthusiastic
<lifeless> [just catching up on bug mail :P]
 * wgrant ventures into the buildd code.
<jml> wgrant, good luck!
<wgrant> 'ubounty'. Nice.
<lifeless> its a bit like ubuty
<wgrant> I'm disappointed that there were only two different spellings.
<wgrant> Not creative enough.
<jml> :)
<jml> hmm. that reminds me, I should fix up the failing tests in my bounty removal branch.
<lifeless> ubuty would be valid straine though
<wgrant> jml: Kill it!
<wgrant> Why's db-devel so out of date?
<jml> wgrant, the normal answer is conflicts between stable & db-devel
 * jml checks to see if the answer is correct this time
<jml> hooray, both builds are broken.
<wgrant> Yay!
 * jml misses PQM
 * wgrant looks for the 'spin and eat disk for several minutes' statement in security.py
<lifeless> 'sql'
<mwhudson> looks like i killed the db_lp builder yesterday :(
<mwhudson> and the thing that merges stable into db-devel seems to have gone away too ?
<wgrant> mwhudson: Ahah. That would explain it.
<lifeless> anyone else seeing the top of edge pages borked /
<thekorn> yes
<wgrant> lifeless: That's normal.
<wgrant> lifeless: It's the new breadcrumbs interacting badly with the old 2.0 views.
<wgrant> Hopefully they'll all be 3.0 in a couple of weeks.
<lifeless> \o/ hit the bottom the yak shaving stack for today
<lifeless> 'glue subunit to pqm' -> 'update pqm clean bugs prep the work area' -> 'release new config-manager' -> 'update the package cause it was removed from debian' -> 'uploaded!'
<wgrant> Nice.
<lifeless> so, we're down to wrapping up  'glue subunit to pqm' -> 'update pqm clean bugs prep the work area'
<lifeless> then I can finally do what I wanted to do.
<lifeless> I spoke to soon
<lifeless> need to do a subunit snapshot :P
<maxb> I see devel != stable at the moment, what's broken, the buildbot or the tests?
<gary_poster> maxb: for that side, the tests.  devel has 14 failures and 10 errors.  (meanwhile, db-devel had a bzr failure, so on that side the machinery failed rather than the tests)
<gary_poster> for ref, http://pastebin.ubuntu.com/261505/
<gary_poster> (we'd like buildbot to be public too; need some work to do that, particularly upgrading to recent buildbot deb.  That might be in progress.)
<maxb> thanks
<maxb> ooi, do you have plans for severing the remaining dependencies on non-egg zope?
<maxb> (ThreadedAsync and docutils)
<maxb> Is anyone else being bombarded by 'lp-sourcedeps/eggs/zope.viewlet-3.4.2-py2.5.egg/zope/__init__.py:3: UserWarning: Module lazr was already imported from None, but /home/maxb/launchpad/lp-branches/python2.5/lib is being added to sys.path' ?
<gary_poster> maxb: bombarded: no, but I know the general cause of that symptom.  I have a branch that should make that particular example of it go away (by moving lazr.* stuff to zc.buildout).  I actually tried to land it Friday but there are some broken lazr.* distributions (they build by trying to go over the net, and we don't allow that) that I'm fixing now.
<maxb> hmm. I get it in several tests, which causes them to then fail since their output is not as expected
<gary_poster> maxb: docutils is already provided by eggs.  Not removing the docutils symlink was an oversight, probably of mine.
<maxb> easy to fix, then :-)
<gary_poster> maxb: yeah. :-) zope has moved past ThreadedAsync, and it is not packaged.  ISTR we still use it.  That should be investigated, and removed now if easy, soonish if not so easy.
<maxb> grep suggests only used by poppy
<maxb> Well here's a bit of a headscratcher... lib/lp/bugs/tests/../doc/bug-export.txt is broken by splitting a single try: except: block across multiple doctest examples by erroneously using all >>>, instead of ... for the continuation lines.
<maxb> Except it passes when run under the lp testsuite under python2.4
<maxb> But the syntax does NOT parse with plain old python2.4 doctest!
<gary_poster> maxb: broken tests: we already filter that warning out *lots* of places.  if you grep for 'was already imported from' you'll see 'em.  I'm surprised we haven't caught the ones you see, because yeah, they make the tests fail.  Maybe it is a 2.5 thing.  You can at least see how to silence the warnings from the grep results.
<gary_poster> maxb: we use zope's fork of doctest with new features.  perhaps it has the usual problems with a fork.
<maxb> Aha... whereabouts in all of zope should I be looking for that?
<gary_poster> zope.testing.doctest
<gary_poster> maxb: Is the ThreadedAsync/poppy thing a problem for py 2.5/2.6?
<maxb> No - it's just a little crazy to be depending on the zope branch just for 2 files
<gary_poster> maxb: eek!  the fact that it is working for everyone is an accident, then.  Deployment is accidentally fine because we use a different mechanism for sourcecode, and old dev instances are fine because they still have zope hanging around, but if somebody new tried to build lp right now it would break for them because it wouldn't get zope, and poppy needs it apparently.  I removed the code that updates the zope branch.
<gary_poster> thanks for bringing that up.  That's probably my first priority now.
<gary_poster> buildbot and ec2 use the same mechanism as deployment, kinda sorta
<gary_poster> ec2test I mean
<gary_poster> so on the bright side, those symlinks will be gone no later than Tuesday ;-)
<ddaa> Hey there
<ddaa> I'd like to know if it's something wrong here, or if it's a known problem
<ddaa> but the windmill test suite does not appear to pass here
<ddaa> I run it with bin/jstest
<ddaa> And it ends telling me "Failed: 3 test suites failed"
<ddaa> I ran a rocketfuel-get to get the latest code just before starting the test suite
<ddaa> currently launchpad r9264
<ddaa> Are the jstests deprecated?
<ddaa> Or something?
<gary_poster> ddaa: last I heard about this was when someone was talking about starting to fix this, about three weeks ago.  That person has had to go to other plans for now.  I would not be surprised if it fails, but I do not know.  It won't truly be part of our system until the test pass and buildbot (or something else like that) runs them regularly.  It's worth asking Monday when more folks will be around.  If you ask me then I can try
<gary_poster> direct you.
<gary_poster> direct the question I mean
<ddaa> yeah, I figured that it was not run by the pqm because it's not in what "make check" does.
<gary_poster> right
<ddaa> Right now I'm in a sprint with afpy folks (the french python user group) and I'm in a track to evaluate selenium/windmill etc.
<ddaa> I figured I should start by looking at how launchpad did it, knowing how the folks are fanatical about test coverage.
<ddaa> Frankly, I'm very disappointed by what I say.
<ddaa> By what I saw.
<ddaa> To get those failure I needed to fix a very trivial import bug.
<ddaa> http://python.pastebin.com/m34a9f134
<ddaa> It's kinda weak to find a bug like this in launchpad mainline :(
<gary_poster> ddaa: Thank you for highlighting this (and the patch).  We are fairly fanatical about test actually, yes, but the person in charge of that part of the effort has been pulled away from it unavoidably (and temporarily), and we've had some other changes that made us drop that ball.  We'll pick it back up starting Monday.
<gary_poster> meanwhile, must run.
<maxb> LP thinks the development focus series for LP itself is 2.1, someone should probably flip that to 3.1
#launchpad-dev 2009-08-30
<jamalta_> hi, does anyone know how i can get a reference to the default db user?
<wgrant> jamalta: There isn't really a default user. What are you trying to do?
<jamalta> wgrant: i'm writing a test that needs to create fake records to the karmacache table
<jamalta> so i have to switch db users to one that has write access to that tab le
<jamalta> table*
<jamalta> and i need to switch back to the regular user after i'm done
<jamalta> but i can't find how to get a reference to it
<jamalta> wgrant: you can look at the diff here https://code.edge.launchpad.net/~jamalta/launchpad/bug-127489/+merge/10785 to see what i'm doing exactly
<wgrant> jamalta: Ah, I see.
<wgrant> Let me have a look...
<jamalta> wgrant: Thank you :)
<jamalta> I got lost trying to find it
<wgrant> jamalta: Have a look at lp.registry.tests.test_distributionsourcepackage
<wgrant> switchDbUser is the magic method.
<jamalta> wgrant: Oh, ok.. that should work then, thanks :)
<jamalta> I am using config.karmacacheupdater.dbuser, so should I change it to match the way it is done in that test instead?
<wgrant> jamalta: Probably. It probably doesn't really matter, but most tests seem to use a literal.
<jamalta> wgrant: Thank you for your help, I will use that route then
<lifeless> hmmm, I think that makes 5 bugs submitted today
<wgrant> Ah, excellent. It is fairly easy to get dev primary archives set up to overlay on top of a real primary archive. That makes testing muuuuch easier.
<wgrant> lifeless: What's the best way to get lpnet7 kicked?
<lifeless> spm: ^
<lifeless> try a sysadmint hat might be around
<lifeless> failing that I'll escalate
<lifeless> how long as it been down for?
<wgrant> lifeless: Around 1.5 hours, I'd say
<wgrant> Could be more.
<lifeless> is it still in rotation?
<wgrant> It is.
<lifeless> ok
<wgrant> So 502s are more frequent than they should be.
<lifeless> we'll give spm 5-10 then I'll ring the SA's
<wgrant> Great.
<lifeless> escalated
<lifeless> thought I really have to wonder just how much you're using lp that you're noticing :P
<wgrant> lifeless: 1/8 isn't small.
<lifeless> anyhoo I escalated
<wgrant> lifeless: Thanks.
<lifeless> the long suffering elmo is on the case
<wgrant> maxb: I still can't reproduce those failures on a clean Karmic installation.
<maxb> hrm
<maxb> Most puzzling
<maxb> What about the "Unable to find mandatory field 'files' in the changes file." ones?
<wgrant> I installed it by using plain rocketfuel-setup, then adding your PPA and installing launchpad-developer-dependencies.
<wgrant> maxb: Those work fine.
<maxb> This is quite confounding
<wgrant> Incredibly so.
<wgrant> You say you can reproduce on two systems?
<maxb> certainly the DONE != ACCEPTED one
<wgrant> I've run that test on two machines in three configurations (one was reinstalled as another arch yesterday).
<wgrant> It works fine.
<wgrant> How strange.
<maxb> Hm. I wonder if it's some bizarre interaction with something in my user account
<maxb> My entire homedir is under Subversion, so the different machines are somewhat similar
<wgrant> Are things oddly shared?
<wgrant> Ahhh.
<wgrant> That is possibly relevant.
<wgrant> adduser will tell.
 * maxb pondering what to chown, etc.
<maxb> gary_poster: When you fix up the symlinks, don't forget to also fix up sourcecode/Makefile!
<maxb> gahh, I hate shhh.py
<maxb> wgrant: You didn't get bitten by Getting distribution for 'lazr.uri==1.0.1'.
<maxb> Installing lazr.uri 1.0.1
<maxb> Caused installation of a distribution:
<maxb> lazr.uri 1.0
<maxb> with a different version.
<maxb> When doing your clean karmic test
<maxb> Got None.
<maxb>  ?
<wgrant> maxb: No -- I didn't have launchpadlib installed yet.
<wgrant> I've since installed it (which brought along python-lazr-uri), and everything remains happy.
<maxb> huh
<maxb> so now even if I purge python-lazr-uri, a clean make is now impossible
<maxb> ImportError: No module named uri
 * maxb has breakfast instead
<wgrant> maxb: Impressive.
<wgrant> Do you have any other lazr.* installed?
<wgrant> eg. lazr.restfulclient?
<maxb> wgrant: aha, that's it exactly
<maxb> Gah. So now I have to choose whether to attempt to debug buildout or attempt to debug launchpad.
<wgrant> maxb: buildout is scary. Run away.
<maxb> Hrm. I wish launchpad wasn't so messy in /var/tmp/
<wgrant> It's not too bad.
<wgrant> You can blow it all away and it will be happy.
<wgrant> And at least it puts stuff only there.
 * mwhudson isn't completely sure about that
<mwhudson> i think some tests still scribble into /tmp
<maxb> Ideally it would all go under /var/tmp/launchpad-$USER/
<mwhudson> (they should be fixed though)
<maxb> Scribbling into /tmp/ isn't wrong, if it's genuinely temporary
<maxb> WTF!?!?!? wgrant it works in my clean user
<mwhudson> maxb: you are right about /var/tmp/launchpad-$USER/, but thinking about it makes me want to cry a little
<wgrant> maxb: Excellent!
<wgrant> maxb: Now, bonus points for working out what it was.
<wgrant> maxb: My first try would be copying over bashrc and the like.
<gary_poster> maxb, wgrant: what's the problem?
<maxb> gary_poster: Which problem would you like to talk about first? :-)
<wgrant> There are a few.
<gary_poster> maxb: oh I dunno.  Just list em for me/
<maxb> I have a couple of weird test failures that happen only for me, but on all my systems. We're now thinking that it's something in my common .bashrc or similar
<gary_poster> huh
<maxb> And, it's currently impossible to bootstrap launchpad on karmic, if you have python-lazr-* system packages installed. buildout gets confused
<maxb> And, when you fix up the zope-removal properly, don't forget to look at sourcecode/Makefile :-)
<gary_poster> grr.  I spent a fair amount of time trying to make that kind of bootstrap problem go away.
<gary_poster> and did so for all of the instances I knew about.
<gary_poster> zope-removal: ack, I saw your ping.  thanks.
<gary_poster> so maxb, those are the 3 problems: (1) something on your system causes test failures only for you.  (2) buildout's bootstrap is broken on karmic if you have lazr.* Installed.  (3) I missed some things in my zope branch removal.
<maxb> yup
<wgrant> (1) is more specifically something to do with a shared home directory
<maxb> I think it's down to my .bashrc
<maxb> I'm bisecting it now
<gary_poster> maxb: I can dupe 2 and 3 on my own.  Fixing 3 should be relatively easy, and once I've seen 2 hopefully I'll know what's up.  1 is harder.  I see your clean user works fine.  oh, shared home directory...
<gary_poster> ok
<gary_poster> maxb: could you pastebin me the traceback from (2)
<gary_poster> when you get a chance
<maxb> There's two flavours of (2) - if you have python-lazr-uri installed, buildout goes to install lazr.uri==1.0.1, but ends up installing 1.0 instead, catches this mistake, and aborts
<maxb> Then, if you purge python-lazr-uri but still have python-lazr-restfulclient, you get a horrendous traceback with the ImportError I mentioned
<maxb> I'll get pastes for both, but it'll be 15mins or so
<gary_poster> maxb: is this during the bootstrap or the buildout?  It sounds like buildout.
<maxb> um, the bit where it populates eggs/ from download-cache/dist/ ?
<gary_poster> buildout, ok
<gary_poster> maxb: I suspect this is new because we have things packaged for karmic that are not packaged in jaunty; and our lazr packages have had some build issues.  I think I've addressed all of them in branches as of this weekend--need to get some reviewed and landed.  I still had hoped that we would be isolated from things like this, but oh well, I'll try again.
<maxb> :-) thanks
<gary_poster> maxb: IRT the two flavors, the first flavor is more interesting to me.  The second one is kind of corollary of the first.  The first problem is "Gary thought we were protected from site-packages now (we are in Jaunty), but it is still leaking."  (Or lazr.uri packaging is more hosed than I thought--it's possible that it is packaged as 1.0.1 but then declares that it provides 1.0, which would explain the problem)
<maxb> !info python-lazr-uri karmic
<gary_poster> maxb: the second problem is: when you take out a dependency of a package, and you try to use that package, it breaks. :-)  the only way it is related to buildout/launchpad is that I had hoped we wouldn't see those packages
<gary_poster> we don't in jaunty :-/
<gary_poster> at least for the examples I have seen
<maxb> I don't really understand what you mean by "when you take out a dependency of a package, and you try to use that package"
<maxb> And unrelated: wgrant: So the DONE != ACCEPTED failure is apparently caused by me changing my umask from 022 to 002
<gary_poster> maxb: lazr.restful depends on lazr.uri.  you removed lazr.uri, and lazr.restful breaks.  It makes sense in isolation.
<maxb> oh.... but what stopped it from trying again to install lazr.uri first?
<gary_poster> Ah, yes, I vaguely recall having been burned by umask in the past
<gary_poster> I'm sorry you were hit by it.  Would you send an email to the list about it?  Others will have more context on that than I
 * maxb afk 5mins
<gary_poster> why didn't it try to install lazr.uri?  Yeah, good question.
<wgrant> maxb: I spent some hours this morning working out a lp-buildd umask issue...
<gary_poster> I regard it (maybe incorrectly) as a corollary of the first one, like I said
<gary_poster> anyway, I need to run go do family things, but maxb, these things (2 and 3, in particular; someone else may need to look at what seems to be the umask issue, maybe wgrant?) will be top billing for me.
<wgrant> maxb: Let's see if I can repro it.
<wgrant> maxb: That does the DONE != ACCEPTED one, but not the others :(
<maxb> mhm, ok then
<maxb> I wasn't actually running anything but that one test whilst searching for the problem, I'm trying the other two now
<wgrant> maxb: Ahaha.
<wgrant> ERROR: Queue item ignored: Bad umask; expected 022, got 002
<wgrant> So, not an entirely unexpected failure.
<maxb> ah... where did that vital piece of info disappear to?
<wgrant> maxb: The BufferLogger that the previous statement logs to.
<maxb> So did it actually end up anywhere useful in a test run, or did you have to run things manually?
<wgrant> maxb: Had to alter the code manually.
<wgrant> maxb: (print logger.buffer.read())
<maxb> ok, I'm on the trail of what's breaking the uploadprocessor tests for me
<wgrant> Great.
<maxb> riiight
<maxb> so the last two are due to my ~/.devscripts
<wgrant> Aha.
<wgrant> So.
<wgrant> On to 2.5!
 * wgrant shall try to set it up tonight.
<maxb> DEBUILD_DPKG_BUILDPACKAGE_OPTS="-us -uc"   <-- upsets launchpad
<maxb> so I guess I need to find the appropriate place to apply a --no-conf option in the testsuite
<wgrant> maxb: OK, test suite running fairly happily with 2.5. I'll poke at any real failures (eg. soyuz-set-of-uploads.txt, which looks unfortunate...) tomorrow.
<maxb> nice :-)
<maxb> Now if I can just figure out why bzr import-dsc is barfing, I'll try to get my launchpad-dependencies branches created.
<maxb> And then progress to actually preparing submission branches for some of the stuff I've accumulated in ~maxb/launchpad/python2.5
<wgrant> maxb: Did you turn of shhh.py in there?
<wgrant> I noticed the build was fairly noisy.
<maxb> yes
<maxb> accidently at first, but I didn't feel like reverting :-)
<wgrant> Are you getting lots of "/home/wgrant/launchpad2.5/lp-sourcedeps/eggs/pytz-2009j-py2.5.egg/pytz/__init__.py:32: UserWarning: Module lazr was already imported from None, but /home/wgrant/launchpad2.5/lp-branches/python2.5/lib is being added to sys.path\n  from pkg_resources import resource_stream"?
<maxb> yes, and that's recent
<maxb> but I think Gary's aware and has some thoughts on that one
<lifeless> moin
<mwhudson> good mornign
<lifeless> mwhudson: so, liking your current role?
<lifeless> or going whoa, where to start?
<maxb> Gah. gah. gah. /me is constructing a branch for launchpad-dependencies, and has just discovered that it basically forked for dapper vs. feisty, with each branch using the same version numbers for entirely different packages
<mwhudson> lifeless: somewhere in between those two :)
<mwhudson> i'm glad i've got 4 weeks left, let's say
 * mwhudson breaks fast
<lifeless> mwhudson: :P
<lifeless> maxb: whats the goal of this archaeology/
<thumper> morning
<maxb> There is very little point. I'm just being obsessive about producing an accurate branch representing past history, if I do so at all
<thumper> mwhudson: still up for a call at 9?
<mwhudson> thumper: sure
<mwhudson> thumper: having said that, 9:10 or so would be better
<thumper> ok
<wgrant> Gr.
<wgrant> Tests that rely on set() ordering make me sad.
<mwhudson> wgrant: KILL
<mwhudson> thumper: call?
<wgrant> maxb: Dammit. I managed to run 'make check' in the same shell that I had my umask set to 002 from earlier.
<maxb> :-)
 * thumper nods
<maxb> ah well, you know exactly what failure that causes, so no problem :-)
<mwhudson> thumper: skype desn't think you're online, is it wrong?
<thumper> mwhudson: yes, I'm trying to get to you too
<wgrant> maxb: I'm hoping it won't cause any others.
<maxb> it didn't for me
<wgrant> maxb: Not even on 2.5?
<maxb> Well I admit I'm conjecturing a bit, but the chances of a test being disrupted by umask on 2.5 but not being disrupted by umask on 2.4 seem vanishingly small
<thumper> https://answers.edge.launchpad.net/launchpad-code/+question/81385
<jml> hi
<spm> hey jml
<jml> spm, hi :)
<lifeless> hi jml
<jml> lifeless, g'day
<jml> lifeless, I had a look at your subunit filter patch last night. It looked like it deserved more concentration than I had though.
 * jml fires off 4 ec2test runs
<lifeless> jml: thanks
<jml> thumper, hi
<thumper> jml: morning
<jml> thumper, you available to talk today?
<thumper> yes, at some stage
<jml> thumper, cool. when do you think?
<thumper> jml: when did you want to talk?
<thumper> I think all the time :)
<jml> thumper, soon-ish, so I can get on with other stuff
<thumper> jml: ok, give me a few minutes to get a cuppa, then we can tal
<thumper> k#
<jml> thumper, cool.
#launchpad-dev 2010-08-30
<lifeless> erm
<lifeless>   File "/var/launchpad/tmp/eggs/setuptools-0.6c11-py2.6.egg/setuptools/package_index.py", line 475, in fetch_distribution
<lifeless> AttributeError: 'NoneType' object has no attribute 'clone'
<lifeless> in devel ?!
<lifeless> Downloading file:///var/launchpad/test/download-cache/dist/setuptools-0.6c11-py2.6.egg
<lifeless> (on ec2)
<deryck> Hi, all.
<lifeless> hiya
<deryck> Anyone else have make blowing up over not having zc.buildout 1.5.1?
<lifeless> yes
<lifeless> just hit me on ec2
<lifeless> as I was about to land the fix for Bug.attachments timing out all over the place
<deryck> I can't figure out how to fix it.  Rolled back bootstap.py to earlier version in the branch I'm in just to get hacking again.
<lifeless> deryck: while you are here
<deryck> sure
<lifeless> do you know why BugAttachment.message doesn't know its own index ?
<lifeless> IIndexedMessage seems pretty expensive to generate
<deryck> I don't.  Meant to look into that when I saw your earlier email, and never got to it with things happening Thurs/Fri.
<lifeless> so I figure gary must have no landed 1.5.1 into the cache
<lifeless> i wonder if its on lp or pypi
<lifeless> deryck: I suspect messages can be shared between bugs
<deryck> yeah, I wondered that too.  But didn't know if he was rolling his own.
<lifeless> but there must be a BugMessage or something that does the actual join
<lifeless> and thus we could have an index in that
<deryck> yeah, there is.
<lifeless> change bugattachment to go via BugMessage
<lifeless> and get the index from that
<deryck> right
 * deryck is looking at code....
<lifeless> should I file a wishlist bug for this? I have Bug.attachments down to 23 queries - constant -
<lifeless> but, the way it does that is to use self.indexed_messages
<lifeless> deryck: ok, there is  1.5.1 on pypi.org, and it seems to be working.
<lifeless> deryck: I'm committing it to the download cache.
<deryck> awesome!
<mwhudson> something funny is going on
<mwhudson> because versions.cfg still says 1.5.0 for buildout
<deryck> lifeless, yeah, file a bug on that.  I'll see if I can get work on that for my performance Tuesday work.
 * deryck plans to do performance Tuesday this Tuesday again
<lifeless> deryck: I'd leave it for a little while, my stop gap should get us through to the 5sec 99% barrier
<deryck> ah, ok
<deryck> cool then
<lifeless> \o/ about perf on tuesday though
<lifeless> there are plenty of other timeouts in bugs code
<lifeless> mwhudson: well, thats interesting.
<lifeless> mwhudson: still, committed. we'll see what happens.
<mwhudson> oh
<mwhudson> hmm
<mwhudson> bootstrap.py must be connecting to the internet to find out that 1.5.1 exists
<mwhudson> this would explain why codebrowse restarts were taking 5 minutes
<lifeless> deryck: https://bugs.edge.launchpad.net/malone/+bugs?field.tag=timeout
<deryck> lifeless, yeah, we're not getting to put the attention on timeouts that I had hoped.  But after this cycle, I'll make sure we're getting the work in.
<lifeless> deryck: Oh, I wasn't meaning to criticise; I was offering that as a palette
<deryck> yes, I appreciate the tag list!  I didn't take it as criticism.  Was just saying we haven't put the attention to the bugs as I had hoped to.
<mwhudson> i bet this affects edge rollouts too
<lifeless> https://bugs.edge.launchpad.net/malone/+bug/121363 looks pretty shallow
<_mup_> Bug #121363: Order 'most recently closed' on 'Bugtask.id DESC' instead of 'BugTask.id' <timeout> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/121363>
<mwhudson> spm: hi, if you have time, can you tell if the last edge rollout took longer than you'd have expected?
<spm> mwhudson: actually it's looking like it brokee rather nicely. I'm still chasing.
<mwhudson> spm: nice
<lifeless> it probably broke due to this issue
<lifeless> which I have just addressed
<lifeless> in a fairly shallow way
<wgrant> However. it still shouldn't be looking anywhere else... What's changed lately? Just buildout 1.5.0?
<lifeless> bootstrap.py
<mwhudson> i guess this is one for that operatic team lead person
<lifeless> spm: also - bzr: ERROR: Could not acquire lock "LockDir(file:///home/warthogs/archives/rocketfuel-built/launchpad/.bzr/branch/lock)":
<lifeless> spm: is really annoying me :)
<spm> ./ignore lifeless
<spm> gah. no wait wrong ignore.
<lifeless> spm: hahaonlyjoking?
<spm> lifeless: I can't see your comments anymore. so no idea if you asked if I'm joking or not. maybe.
<lifeless> spm: :P
<MTecknology> lifeless: This is what I'm doing now - http://staging.profarius.com/
<lifeless> MTecknology: so the artwork for launchpad is not open
<MTecknology> It just popped into my head that we're going that and I realized I'd better check it out..
<MTecknology> s/going/doing/
<lifeless> I'm just looking for the reference
<lifeless> ah yes, LICENSE in the tree:
<lifeless> The image and icon files in Launchpad are copyright Canonical, and
<lifeless> unlike the source code they are not licensed under the AGPLv3.
<lifeless> Canonical grants you the right to use them for testing and development
<lifeless> purposes only, but not to use them in production (commercially or
<lifeless> non-commercially).
<lifeless>              
<lifeless> The Launchpad name and logo are trademarks of Canonical, and may not
<lifeless> be used without the prior written permission of Canonical.
<MTecknology> So I should definitely remove it..
<spiv> I would think (but IANAL) that if you are using the LP logo specifically to refer to the launchpad.net instance then you're on the right side of trademark law.
<wgrant> spiv: Copyright-wise that's less clear, though.
<spiv> wgrant: indeed
<lifeless> spiv: trademark + copyright are needed though
<ajmitch> can't login.ubuntu.com be used instead of LP?
<lifeless> MTecknology: is it general openid you want to refer to
<MTecknology> wgrant: has there ever been anything clear about legal issues?
<lifeless> MTecknology: or lp specifically ?
<wgrant> MTecknology: Frequently.
<MTecknology> it uses launchpad teams
<wgrant> But not when it comes to LP.
<wgrant> I mean, the tree isn't even distributable...
<lifeless> MTecknology: so, I suggest you email.. damn folk are on leave. Thumper. Email him.
<MTecknology> will do
<lifeless> ask for the ok. He'll bounce to jml, jml is awl for 2 weeks.
<lifeless> so it will queue there
<lifeless> and then get bounced to legal. :P
<lifeless> thumper: ^ :P
<MTecknology> :p
<MTecknology> I'll make a nice pretty email for him to read :)
<MTecknology> I never enjoy legal stuff - I kinda wish the whole world was just under the BSD license :P
<lifeless> that has some appeal
<MTecknology> I don't think society as is could handle that though :(
<MTecknology> I've kinda been fighting the same with with the light-drupal-theme - I want it made for wide distribution - but I need to avoid license and trademark issues
<wgrant> MTecknology: I would use Ubuntu SSO for that.
<wgrant> It's more correct, and the trademark and copyright issues are clearer.
<MTecknology> hm?
<lifeless> wgrant: doesn't have lp team memberships
<MTecknology> wgrant: for the theme - I mean images like this - http://s.ubuntu.ru/header.png
<wgrant> lifeless: Doesn't it?
<lifeless> wgrant: AFAIK, no.
<MTecknology> and specific color schemes
<wgrant> lifeless: They're the same code and DB, but with a different theme.
<wgrant> lifeless, MTecknology: login.ubuntu.com does send team memberships.
<wgrant> I just checked.
<lifeless> cool
<lifeless> I dunno if its /meant/ to - its hardly separate from LP :P
<wgrant> It's meant to.
<wgrant> AFAIK nobody thought about how the split was meant to work...
<wgrant> Because it's still hanging off the LP DB, and probably will always have to.
<lifeless> wgrant: I wasn't part of that work, can't comment at all.
<lifeless> but its really a bit of a bastard child atm
<MTecknology> lol
<wgrant> It tries to be separate from LP.
<wgrant> But everything revolves around it having knowledge of LP team memberships.
<MTecknology> Is there any particular reason it was broken out?
<lifeless> sinzui: hi
<lifeless> MTecknology: to make it usable by U1
<sinzui> hi lifeless
<lifeless> sinzui: I am planning on nagging spm about the maps CP
<MTecknology> U1?
<lifeless> sinzui: he's dealing with falling buildings atm
<sinzui> I was just about to do that
<lifeless> MTecknology: ubuntuone
<MTecknology> oh
<sinzui> lifeless, okay, that saves me pinging him
<lifeless> MTecknology: its an (important) branding exercise
<lifeless> but its not really split out ATM
<thumper> MTecknology: isn't it more ubuntu single signon than a launchpad login?
<MTecknology> thumper: This image has been there since before the ubuntu login
<thumper> hmm
 * thumper shrugs
<MTecknology> lp:drupal-{openid,launchpad,teams}
<lifeless> spm: please ping when I can get another profile
<thumper> testing bug 600000
<_mup_> Bug #600000: missing dependency on Bazaar <hitchhiker (Ubuntu):Fix Released> <hitchhiker (Debian):New> <https://launchpad.net/bugs/600000>
<spm> lifeless: just kicking this imports thing for thumper; then you're next?
<thumper> ta spm
<lifeless> spm: thanks
<spm> lifeless: oki; about to do the magic on staging. brb...
<lifeless> wow https://bugs.launchpad.net/ubuntu/+bug/1/+index is a mess
<_mup_> Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Invalid by compscibuntu-bugs> <EasyPeasy Overview:Invalid by ramvi> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <The Linux OS Project:In Progress> <OpenOffice:In Progress by lh-maviya> <Tabuntu:Invalid by tinarussell> <Tivion:Invalid by shakaran> <Tv-Player:New> <Ubuntu:In Progress by sabdfl> <ubuntu-express (Ubuntu):Invali
<lifeless> spm: so I waiting for you to say 'go'
<spm> lifeless: so am I. KaBoom.
<spm> staging appears to be having FireTruck issues
<lifeless> its going
<lifeless> oh
<spm> An error occurred when trying to install zc.buildout. Look above this message for any errors that were output by easy_install.
<mwhudson> spm: big, red doesn't stop at lights?
<spm> make: *** [bin/buildout] Error 1
<spm> ^^ not my idea of 'going' :-)
<lifeless> spm: >< - update the source code cache
<mwhudson> spm: this'll be the 'connecting to the internet' problem maybe?
<lifeless> I put a new buildout in there this morning to work around it
<spm> ugh. it didn't STOP!!!!! ARGH!
<lifeless> spm: is it currently running with profilig on?
<lifeless> spm: cause I don't care if the code is old
<spm> somethings changed in the recent week or so that stops staging/edge from shutting down. *some* of the time.
<spm> lifeless: doubt it; it's been running since Aug 27.
<lifeless> grah
<lifeless> ok
<spm> ie. we'd have faceplanted on disk space by now.
 * lifeless emotes kill -9 @ spm
<spm> I try the regular kill first. the think piece of silk wrapped around the 25 kg sledgehammer. if that doesn't work I take the slik off, and use the -9.
<spm> s/think/thin/
<spm> right. silk comes off.
<spm> we used to have an incredibly aggressive. Try X, fail, try kill, fail, try kill -9 shutdown sequence; that seems to have been removed :-((((
<spm> mwhudson: I don't think so: Link to http://pypi.python.org/simple/zc.buildout/ ***BLOCKED*** by --allow-hosts
<lifeless> spm: thats the thing mwhudson is referring to
<mwhudson> well no, that sounds better
<lifeless> spm: you need to update the dist cache
<spm> the internet problem was a timeout; we had last week
<mwhudson> that's being blocked at the application level, not by the firewall
<spm> https://pastebin.canonical.com/36438/
<lifeless> spm: 11:05 < lifeless>   File "/var/launchpad/tmp/eggs/setuptools-0.6c11-py2.6.egg/setuptools/package_index.py", line 475, in fetch_distribution
<lifeless> 11:05 < lifeless> AttributeError: 'NoneType' object has no attribute 'clone'
<lifeless> 11:05 < lifeless> Downloading file:///var/launchpad/test/download-cache/dist/setuptools-0.6c11-py2.6.egg
<lifeless> 11:05 < lifeless> in devel ?!
<lifeless> 11:05 < lifeless> (on ec2)
<spm> so I assume tehre's a patch floating around somewhere to ensure we have the lastest hotness?
<lifeless> spm: is what I was asking this morning. Its the same - AttributeError: 'NoneType' object has no attribute 'clone' - is the magic bit to note, where it has blown well up.
<lifeless> spm: yes, *update the dist cache*.
<spm> that sounds... broken somewhere. why is our automatic updates not getting this automatically?
<lifeless> something like cd lp-sourcedeps/download-cache; bzr update
<mwhudson> i would like to understand why it's looking for 1.5.1 though
<lifeless> spm: appears to be a skew between bootstrap.py and versions.cfg
<lifeless> or something like that
<spm> gah. sorry - germanium is trying to eat several hundred Gb of disk asap. bbs... maybe.
<lifeless> whats germanium do
<spm> ppa
<mwhudson> it's ppa.launchpad.net i think
<spm> looks like the leaky /tmp crap buglet we saw a week or so ago. G just has more / space. so took longer to manifest.
<lifeless> ah so a simple rm *
<spm> well.... o; but that's the idea. a little more targetted; but.
<spm> well *no*. typo win.
<lifeless> did you just rm -rf / ? Please say yes....
<spm> I could try; but it won't work well :-)
<spm> doing a verify if /tmp is in fact te problem. it *looks* like it; but I want a little more verification before I start rm'ing away; then I'll prolly start with something like: find /tmp -maxdepth 1 -name 'tmp*' -type d -mtime +7 -delete
<spm> actually no. dir's; so -print0 | xargs -0 rm -rf
<lifeless> -type f
<lifeless> then -type d :P
<spm> ... | xargs -rn rm -r <== better :-)
<spm> -0r, argh
<lifeless> -exec { rm \$1 }; ?
<lifeless> actually, probably wants a -cut too
<wgrant> spm: Not my script again? :P
<wgrant> (killing germanium)
<spm> lifeless: for this many folders, -exec would be horrible. fork hell.
<lifeless> spm: clone ftw!
<spm> wgrant: "yes", even if not your fault; it now is.
<lifeless> spm: (actually its exec hell, the fork is cheap)
<spm> whatever :-)
<spm> ok. 15 mins of trying to get a du summary; I call /tmp/tmp* is the problem and move into rm mode
<spm> whee. some srsly old poppy uplods too.
<lifeless> mwhudson: its odd, bootstrap.py was last change in 11419
<lifeless> on tuesday
<lifeless> oh oop, thats my branch
<lifeless> no, devel agrees
<lifeless> ah, 11452 changes versions.cfg
<lifeless> but that doesn't change... ah I wonder if lazr.restful 0.11.3 wants the newer zc/buildout ?
<mwhudson> oh
<mwhudson> maybe
<lifeless> or launchpadlib
<mwhudson> that should lead to a conflict though i think
<spm> lifeless: (vaguely related aside) - did you ever submit a branch to the prod/staging configs that had the profiler in, but disabled?
<lifeless> spm: pretty sure
<wgrant> lifeless: It isn't just because the buildout 1.5.0 has new logic to update, and 1.5.1 was only just released?
<wgrant> Er, English fail there, but you get the picture.
<mwhudson> wgrant: no, because this happens on machines that can't connect to the internet
<wgrant> mwhudson: And it hasn't been happening since the update?
<mwhudson> to buildout?
<mwhudson> no, we don't think so
<wgrant> Yeah.
<wgrant> :(
<lifeless> spm: ah, its in my branch lp:~lifeless/lp-production-configs/timeouts
<spm> lifeless: hrm. doesn't look like it. not an active proposal; and we've only had one other change in the past month. Can I pester/nag/trouble/bother you to get that sooner than later? ;-)
<spm> snap, ftw.
<spm> merge propose away!
<lifeless> spm: the other change was this branch too, I thought
<lifeless> https://code.edge.launchpad.net/~lifeless/lp-production-configs/timeouts/+merge/34042
<spm> lifeless: no; the other recent change only has librarian oops changes.
<spm> ta
<spm> heya poolie
<poolie> hi there spm!
<lifeless> spm: so, does this mean we're ready to profile ?:)
<spm> lifeless: ha you wish. staging is still very much firetrucked. still getting the ambulance round.
<lifeless> shall we call the cops, go for the trifecta
<spm> well, my awesomely wonderful (and delightfully sarcastic) wife is making me a ham sandwich... so........
<mwhudson> lifeless: doesn't look like anything depends on buildbot 1.5.1 so it's a mystery (tm)
<spm> lifeless: approved; pls submit whenever you're ready.
<lifeless> spm: I've no idea how to do that anymore; it pages out so quickly (and its nuts to use pqm for this)
<lifeless> spm: perhaps you could submit it ?
<spm> :-)
<spm> sure
<lifeless> thanks!
<spm> fwiw, we do via pqm; more as a gatekeeper than anything
<lifeless> I love pqm/tarmac systems (I should shouldn't I :P) but they come with a cost, for benefits. if the benefits aren't worth the cost, it's not useful to use it :)
<lifeless> spm: so, you're having lunch? Should I pop my head back in in 20 minutes or something for the staging profile ?
<spm> lifeless: not really... Ill be eating lunch shortly; not breaking; if you ken the difference.
<lifeless> och aye
<spm> too many eggs on the boil to afk yet
<lifeless> so staging - have you tried updating the download cache?
<spm> not yet; eed to unfudge edge 1st
<poolie> hi lifeless (realize lp's eggs are on fire) did anything change with feature flags?
<poolie> i may go back to it say wednesday
<poolie> haven't read much mail ypet
<lifeless> poolie: sinzui has a patch that uses them to control google maps
<lifeless> which are currently stuffed due to a google side problem
<poolie> sweet
 * poolie spoke too soon
<poolie> but i'm glad to see some takeup
<lifeless> it should get CP'd todayish :P
<lifeless> spm: edge almost certainly has the same problem. have you tried updating the download cache :P)
<spm> lifeless: that would emphatically be Doing It Wrongâ¢
<spm> manual intervention on every server? yuk.
<lifeless> spm: well, doesn't your deploy script do that ?
<spm> hrm. from the same state; the question is why is that state borked.
<lifeless> spm: kick off a new deploy, done :P
<lifeless> spm: so we don't know the root cause, but the package I put in the download cache about 4 hours ago seems to work.
<spm> ahh. I see. edge borked is the failed stop on edge4; it never updated to latest shiny.
<lifeless> right
<lifeless> I strongly suspect, if the pastebin you gave is what the error was, that kicking off a new one, which should as a matter of course grab the latest download cache, should work.
<spm> edge hasn't (yet) exhibited) that error...
<spm> has special errors all of it's very own
<lifeless> show n tell ?
<spm> per above; failed to stop and never got updated; so when I stabbed and started it; it came up as the wrong revno.
<lifeless> ah
<spm> we have a check for that you see :-)
<lifeless> spm: so what do you think is wrong?
<spm> lifeless: with edge in this case? it's a reversion on an old bug faict. have just logged bug#626577
<_mup_> Bug #626577: app servers not shutting down (again) <canonical-losa-lp> <Launchpad Foundations:New> <https://launchpad.net/bugs/626577>
<spm> actually - that's a moderately deadly one for continuous rollouts btw. if we need manual intervention to just stop/start servers; that project is pretty much dead before started.
<lifeless> we will fix
<lifeless> the more we do something the more we'll improve it
<spm> Oh i know that; was just saying :-)
<spm> still the root cause of that bug has never been fixed. what actually causes the daemons to lock like that and not shutdown correctly.
<lifeless> spm: did you get a per-thread stack trace?
<spm> way back when; probably. we spent a fair bit of effort ages ago on it. given we had a (hack) fix; it was deemed not worth the time and effort to keep chasing further.
<spm> argh. so staging full udpates died on a replication fail.
<lifeless> was it the new table again ?
<lifeless> spm: anyhow
<lifeless> spm: I don't care about updates, I just want running + profiling :)
<lifeless> spm: can we do that, and you fiddle with updates after ?
<spm> well, tbh, I'd really rather not. otherwise I'm just duplicating the same work and somewhat wasting time messing around with something that needs fixing at the root anyway
<lifeless> spm: ok
<lifeless> spm: I'll go do some other stuff and check back with you in 20-30
<spm> I'd hazard at least part of an educated guess that the pqm spam is all realted to stagings borked ness
<lifeless> spm: the db-devel update is due to sodium crashing mid-update
<lifeless> its left stale bzr locks
<lifeless> at a WAG
<spm> yeah - but those files get auto synced around. so a partial update gets synced around....
<lifeless> ouchies
<spm> right; and we have a dir that's missing stuff. so I call 1+1 = maybe 2 :-)
<StevenK> 1 + 1 = 1.8 ish?
<persia> closer to e
<spm> 2.1, you could average
<lifeless> spm: so how goes it ?
<spm> well we have successful rf builds again.
<spm> right. next regular staging code update is in ~ 5 mins. see how that goes as an automated thing.
<thumper> :((
 * thumper wishes jelmer lived in NZ or AU
<persia> What do you have against the northern hemisphere?  We've a few active timezones on this side of the globe.
<persia> (some east of some AU timezones)
<lifeless> persia: thats only because .au is silly-wide
 * persia is reminded of Negativland's "Time Zones"
<mwhudson> the name of that ci tool that's written in java *still* trips me up
<persia> In terms of increased highlight count?
<mwhudson> no
<mwhudson> just reading emails
<mwhudson> "meet hudson" <blink>
<persia> Ah, so yes, but with a wider definition of "highlight" to be more a cognitive filter than any technical notifaction system :)
<mwhudson> right
<thumper> persia: it is more that I want jelmer around RIGHT NOW
<persia> I figured, but just felt you were unfairly discriminating against landmasses north of the equator.
 * ajmitch thinks it's a fairly reasonable wish
<lifeless> thumper: what do you want from jelmer ?
<spm> lifeless: mwhudson: https://pastebin.canonical.com/36440/ <== that's getting zotted at source, so to speak. if the staging restore/build can't get those files; we have a bigger problem up front.
<lifeless> spm: its in the download cache now
<lifeless> spm: I don't know how staging updates work
<lifeless> but if they are not updating the download cache, its a fail.
<lifeless> if they , that file is available.
<thumper> lifeless: I've emailed him about a git import failure
<lifeless> spm: can you please confirm that zc.buildout-1.5.1.tar.gz exists on disk on staging
<lifeless> oh joy
<lifeless> ec2 hates my branch, but it works locallyt.
<lifeless> what could it be? ... storm
<spm> lifeless: so. cm.py (rf built) runs on sodium. 'builds' the tree; that gets pulled on sourcherry, which every half hour does a staging 'restore'. pushes that where it needs to go.
<lifeless> spm: ok, so lets check on sodium
<spm> dbupdates are done once 1 week; superset of the normal code only updates
<lifeless> does it have a download-cache there
<lifeless> grah this is nuts
<lifeless> any advice on debugging a 'works locally, fails on ec2' issue ?
<lifeless> I've zapped my storm local bugfixes, running whatever the dist facility made
<mwhudson> lifeless: how is it failing on ec2?
<lifeless> two ways
<lifeless> one
<mwhudson> are we using lucid ec2 images yet?
<lifeless> my queries-are-constant test fails
<lifeless> the check for constant finds the second run did 4 less queries
<lifeless> the second thing is a check in bug.txt that something takes 2 queries fails - it takes 3 (which is probably appropriate)
<lifeless> File "lib/lp/bugs/tests/../doc/bug.txt", line 1133, in bug.txt
<lifeless> Failed example:
<lifeless>    len(CursorWrapper.last_executed_sql) - queries
<lifeless> Differences (ndiff with -expected +actual):
<lifeless>    - 2
<lifeless>    + 3
<mwhudson> lifeless: the only thing that leaps to mind is an isolation failure, some test is leaving something with a __del__ that does queries or something behind
<mwhudson> but that seems a bit mental
<lifeless> so they do definitely both pass in isolation
<mwhudson> lifeless: have you tried running not just your test but a few around that?
<lifeless> I guess I'll run a .. yes
<mwhudson> sounds horrible though
<lifeless> fortunately subunit streams give me a nice log
<mwhudson> yeah
<lifeless> I'm fully merged with devel too
<lifeless> for kicks
<mwhudson> i guess the failure of your new test doesn't include the statement logs?
<lifeless> it does
<lifeless> it doesn't include the statement log for the *first* attempt
<mwhudson> ah
<lifeless> thats on my todo to fix (use a currying approach)
<lifeless> get a matcher which builds a second matcher for you
<mwhudson> is it the same number of statements locally the second time
<lifeless> and then diffs the statements
<mwhudson> >
<mwhudson> ?
<mwhudson> that'd be awesome
<lifeless> yes, locally its the same both times
<mwhudson> so it really is the first time doing fewer queries?
<mwhudson> than locally
<mwhudson> that's very odd
<lifeless> no, the first time is doing more
 * mwhudson rubs eyes, learns to read
<lifeless> but I can't imagine a flush being batched up
<lifeless> sorry, crossed wires
<lifeless> *that* count change is in bug.txt
<lifeless> which I didn't really touch at all.
<lifeless> this is my one:
<lifeless> Matcher: HasQueryCount(Equals(23))
<lifeless> Difference: queries do not match: 23 != 19
<lifeless> the 23 is the count from the first API call
<lifeless>    self.assertThat(collector, HasQueryCount(Equals(with_2_count)))
<mwhudson> and locally both are 23?
<lifeless> is what generates that
<lifeless> I think so. Locally both are < 24.
<lifeless> I'll see if it can go lower, I don't think it can.
<mwhudson> so you have the 19 queries that were executed the second time?
<lifeless> yes
<lifeless> I can pastebin
<mwhudson> i guess you could try diffing that to a local run
<lifeless> true
<lifeless> 23 is the local count
<lifeless> so ec2 is missing
<spm> right. versions on db-stable and sourcherry/sodium now match. re-running code staging restore atm. pray.
 * spm afks for school run & lunch
<lifeless> sorry, not missing
<lifeless> they are different
<lifeless> the 19 we get are the same for the OAuthNonce dance
<lifeless> then we get a
<lifeless> 6, 6, 'launchpad-main-master', 'SELECT 1 FROM Person WHERE Person.id = %s')
<lifeless> which I don't see locally
<lifeless> maybe as many as 8 queries
<lifeless> select 1 from looks like an updated/changed .exists()
<lifeless> mwhudson: I propose to make the constant check a < check
<lifeless> mwhudson: that is, it can get lower, it can't get higher.
<mwhudson> yay for AP falling over
 * mwhudson has to run away anyway
<mwhudson> lifeless: good luck
<lifeless> mwhudson: do I have your ok for minor hit-it-hard-tweaks
<lifeless> like
<lifeless> -        self.assertThat(collector, HasQueryCount(Equals(with_2_count)))
<lifeless> +        self.assertThat(collector, HasQueryCount(MatchesAny(
<lifeless> +            Equals(with_2_count),
<lifeless> +            LessThan(with_2_count))))
<lifeless> I'm pretty sure this is a storm version skew issue or something like that
<lifeless> I've seen the storm cache cause all sorts of stuff (and been filing bugs and mps because of it)
<lifeless> ah yes, its definitely storm.
<lifeless> bug 619017
<_mup_> Bug #619017: __storm_loaded__ called on empty object <Storm:Fix Committed by therve> <https://launchpad.net/bugs/619017>
<lifeless> it causes spurious person queries when initialising cached objects
<lifeless> I think its causing both
<lifeless> because getMessageChunks issues a Person lookup too
<lifeless> can probably rewrite that to do a single query instead, but the bug will still shoot us.
<stub> lifeless: How are you using a different version of Storm?
<lifeless> stub: I was running a version with cache fixes
<lifeless> pending 0.18 being released
<stub> Right. So any reason not to add that to buildout?
<lifeless> yes, the fix provokes another bug
<lifeless> that bug is more severe - it stops things dead if triggered, and gustavo is working on the fix
<lifeless> he gave me a 'give this a shot' version
<lifeless> but untested, I didn't want to land that for all devs, better to wait for 0.18
<stub> Right. And getting exact query counts isn't going to work with version skew.
<lifeless> anyhow, the query count variation is entirely explained by Person._init lookups
<lifeless> stub: the tests I write set an upper bound and then check for consistency, so they should not be very sensitive to storm versions, unless the version has brain damage of some sort
<lifeless> (which, sadly, 0.16 and 0.17 did, in different ways)
<stub> lifeless: Consider putting a key -> query count mapping in an external file. If we get lots of these tests, we might want to update the counts in bulk as part of a Storm update. Maybe YAGNI.
<stub> Hmm... a test fixture that automatically does the count check, unless the magic wand is waved and it records the count check...
<lifeless> stub: that might be nice; need to find a way to say 'this is the bit to track
<lifeless> '
<lifeless> stub: that said
<lifeless> stub: if we have a storm update that wants to raise query counts across the board, I'll just send it back with 'no thanks' written in technicolour across its flanks
<lifeless> stub: something that lowers query counts won't break the tests.
<lifeless> nononono test fix nonononoonono
<lifeless> grah
<lifeless> sinzui: poolie: sadly there is an API skew between devel and production for this
<lifeless> https://lpbuildbot.canonical.com/builders/prod_lp/builds/91/steps/shell_7/logs/stdio
<lifeless> spm: whats the blessed way to rollback an lp-prod change (and how did this get so far before being picked up ?)
<poolie> lifeless: "this" being?
<lifeless> sinzui used the flags API
<spm> eek. probably via a cowboy and/or CP vs actually rolling back. where either would essentially revert the change.
<lifeless> but you changed the meaning of something - look at the backtrace in the link I posted - between the last release and this release, so his CP to use flags to disable gmaps fails horribly
<spm> negative patch if you ken.
<spm> lifeless: so. um. don't try and rollout that CP? I was just about to...
<lifeless> spm: hell no
<lifeless> 83 failures 1 errors
 * spm searches that response for possible subtleties...
<spm> \o/
<lifeless> put it this way; users profile pages would be - more- broken.
<spm> lifeless: can you rm that from the requested CP list? actually - it shouldn't be on the CP list until such time as it passes and lands....
<lifeless> spm: I've marked it as snafued
<lifeless> which sinzui will see
<spm> coolio; ta
<spm> just don't want an accidental rollout by the wrong one
<lifeless> naturally.
<lifeless> actually, we can do this much more quickly.
<lifeless> can I get you to be remote fingers ?
<spm> <ding> yes master, what is your will.
<lifeless> on praseodymium
<spm> prasÃ© to it's mates. fwiw.
<lifeless> firstly, check that the archives/... production-devel and production-stable are up to date with LP - I presume PQM still uses local disk as authoritative ?
<spm> should do...
<lifeless> just a 'bzr missing lp:~launchpad-pqm/launchpad/production-stable' should be fine
<lifeless> (and -devel, in -devel)
<lifeless> spm: then, in production-devel
<lifeless> does it have a working tree - are there files present? If no, do 'bzr checkout .'
<lifeless> finally, 'bzr merge . -r -2..-1'
<spm> ~/archives/rocketfuel/launchpad/production-devel <== Branches are up to date.
<lifeless> this should affect only files in lib/lp/registry
<spm> um. actually. I may have done that wrong. pebkac...
<lifeless> specifically, browser/__init__, person, team, tests/mailinglistviews, tests/person-views, 3x stories and 2xtemplates
<spm> production-stable You are missing 5 revision(s): 9649 - 9653
<spm> prod-devel is fine
<lifeless> -stable doesn't actually matter for this
<lifeless> the failure was on -devel
<spm> oh. and stable is CP's. that only gets updated locally with a full rollout.
<lifeless> I wasn't sure when we started (i was reading just ahead of what I typed)
<spm> :-)
<lifeless> ok, so in -devel
<lifeless> is there a tree of files?
<lifeless> and shit
<spm> yup
<spm> to both.
<lifeless> bzr update
<lifeless> then
<spm> (bzr add shit)
<lifeless> or
<lifeless> bzr st
<lifeless> should be empty
<spm> $ bzr st ==> working tree is out of date, run 'bzr update'
<lifeless> run bzr update
<lifeless> :P
<spm> just ensuring we're all on the same page :-)
<lifeless> its good
<spm> Updated to revision 9655.
<lifeless> bzr st
<lifeless> should be empty
<spm> +1
<lifeless> bzr merge . -r -2..-1
<lifeless> bzr diff -r -2
<lifeless> should be empty
<spm> bzr merge --preview . -r -2..-1 <== nothing. good enough?
<lifeless> #bzr commit -m '[rs=lifeless][rs=spm]
<lifeless> spm: huh, no, merge --preview is doing to be totally confused here.
<lifeless> spm: bad fingers!
<spm> actually - before I make the change; can we step back a bit and get an explanation of what we're doing? reverting a change I gather?
<lifeless> yes
<lifeless> the reverse patch
<lifeless> which is done by 'bzr merge . -r -1..-2', and confirmed by checking that there is no difference against the second commit back.
<spm> Oh I see.
<StevenK> But you said -r -2..-1 earlier?
<lifeless> yes
<lifeless> that was a teset
<lifeless> I failed it.
<lifeless> the diff step would have caught it regardless
<spm> what I meant in this case was to (via pqm) submit a reversal patch; not fiddle around with the core players directly.
<lifeless> spm: I can do that if you prefer, its rather pointless IMO : the last commit back is known-good
<spm> mainly as we have so many moving parts; it's really easy to miss one; whereas going via the normal steps, we achieve bliss.
<lifeless> spm: PQM *wasn't used properly* or we wouldn't have the branch broken.
<spm> heh
<lifeless> spm: if PQM was running the test suite, not running in brain-damaged mode, this wouldn't have happened.
<lifeless> its up to you : we can be in test fix for another few hours
<lifeless> or we can be out in 5 minutes.
<spm> Oh. I think i see. we had a branch submitted directly to prod-devel that hadn't actually passed the testing?
<lifeless> that or this branch passes randomly 50% of the time.
 * spm eods in 55 mins. the former option is looking appealling >:)
<lifeless> I think submitted without testing is what happened.
<spm> fair enough
<lifeless> tested in devel, and in dbdevel and db-stable, but not vs prod-devel in ec2
<spm> right
<spm> oki, making magic fingers.
<lifeless> so,
<lifeless> bzr merge . -r -1..-2
<lifeless> will back out the last commit
<lifeless> bzr diff -r -2
<spm> prod-devel, right.
<lifeless> should be empty
<lifeless> ack
<lifeless> if that diff is empty the tip commit's contents are gone.
<spm> um. hang a sec. bzr merge . -r -2..-1 is what you had earlier; vs -1..-2. does it matter?
<lifeless> #and we can bzr commit -m '[rs=lifeless][rs=spm] Back out
<lifeless> yes, I had it wrong before (and the diff would have show n that)
<lifeless> we want 'from tip to the one before'
<lifeless> rather than 'from the one before to tip', which already exists in the branch.
<lifeless> a --preview here shold in fact work - try if you like
<lifeless>  bzr merge --preview . -r -1..-2
<lifeless> and we can bzr commit -m '[rs=lifeless][rs=spm][rollback=9655] Back out broken Google maps API fix due to feature flags api differences with production-devel vs devel.'
<spm> lifeless: for verification: https://pastebin.canonical.com/36441/
<lifeless> spm: that looks right. Is bzr diff -r -2 empty ?
<spm> yup
<lifeless> bzr commit -m '[rs=lifeless][rs=spm][rollback=9655] Back out broken Google maps API fix due to feature flags api differences with production-devel vs devel.'
<spm> ci'ing...
<lifeless> and bzr push
<spm> Using saved push location: bzr+ssh://bazaar.launchpad.net/~launchpad-pqm/launchpad/production-devel/
<spm> Pushed up to revision 9656.
<spm> Oh I see what you've done. rather then "uncommit/revert" to get back to X-good. We deliberately roll the reverse patch. Nice! Very clean.
<lifeless> thank you _very much_
 * spm adds that nice littel trick to his 'bzr notes'
<lifeless> uncommit *count* run into trouble if some branch somewhere had pulled in the intervening time
<lifeless> and then would need a --overwrite to fix up etc etc
<spm> right
<lifeless> so, two las tthings
<lifeless> production-devel buildbot slave is dead
<lifeless> please shoot, corpse in the river.
<spm> le-sigh. again?
<lifeless> https://lpbuildbot.canonical.com/buildslaves
<poolie> lifeless: i don't think the test_addmember failure is related to anything i did
<lifeless> poolie: I don't think you've done anything wrong, but I am pretty positive that production's feature flag module treats *something* differently to devels
<lifeless> poolie: or sinzui's branch wouldn't have passed buildbot for devel->stable, db-devel->db-stable, but blown up on production-devel
<poolie> that's quite possible
<lifeless> poolie: I think it would be nice to harmonise them, either by cping your further flags work to production, or something.
<poolie> it's a bit opaque which version is running where
<poolie> obviously it is in the bzr branches
<lifeless> lp:~launchpad-pqm/launchpad/production-stable is whats deployed
<lifeless> yes, it would be nice to have this a lot clearer.
<lifeless> spm: secondly, I can has profile ?
<spm> lifeless: prod-lp kicked off a new ec2 instance just a few mins ago. so not so much wedged as 'still instansiating"
<lifeless> spm: ok cool., I thought they were real machines now - mea culpa
<spm> and now running apparently. no, just the lucid ones
<spm> lp, db_lp and prod_lp are still ec2
<lifeless> I'll try to remember. Thanks :)
<lifeless> spm: so that leaves just getting a profile from staging :)
<spm> is it back yet? I checked about an hour ago and staging was still restoring (successfully tho)
<lifeless> front page comes up
<spm> it LIVES!
<spm> so the 'fix' was simply ensure the latest shiny was loaded for sourcherry to try and rollout; other problems around access etc went away. fwiw.
<lifeless> yeah.
<lifeless> fragile sucks
<spm> i think my first test was a little too early and hence a few versions behind.
 * lifeless loads more ammo in the just-one-dep-style-please gun
<lifeless> spm: ok, so enable-profiling-and-restart time? pl-pl-please ?
<spm> lifeless: go for it, live.
<lifeless> thanks
<lifeless> got it
<spm> revert?
<lifeless> please
<lifeless> we'll know once it rsyncs if its good or not, but no point slowing up the system in the mean time
<spm> restarting....
<lifeless> OOPS-1703S298
<spm> oh gah. sodium seems to still be borked.
<poolie> lifeless: so production-stable's tip is changed pretty much atomically with it actually being deployed?
<StevenK> spm: Borked how?
<spm> "hardware"
<StevenK> spm: Bleh, still :-(
<spm> normally it recovers on it's own; but seems to not be coming back.
<spm> heh, yeah. I believe it'shad just about everything replaced and still dies.
<StevenK> Perhaps it's wet, sodium reacts badly to water. :-P
<lifeless> poolie: AIUI
<spm> I made that joke /fosty response at the joke theif
<StevenK> Haha
<lifeless> StevenK: elmo says that they've replaced/reassembled the entire thing
<lifeless> its going to be totally replaced, its queued to do so.
<StevenK> lifeless: IE, new name, new everything?
<lifeless> it'll still be sodium I think :)
<lifeless> but new chassis & guts
<spm> it'll still be devpad; maybe not sodium.
<lifeless> true
<lifeless> I live in hope, its a cool name
<spm> aiui, nafallo gets naming rights on new boxes. so....
<spm> which explains some of the more .... well lets just say: thank $deity for ssh tab completion on names
<wgrant> The original armel builders had nicely obscure names. The new ones aren't so good :(
<StevenK> I thanked Nafallo for those. He took it as a compliment.
<wgrant> Heh.
<poolie> StevenK: your hudson url gives 'connection timed out' for me
<StevenK> poolie: Let me check, I think I'm a muppet
<StevenK> poolie: Should work now
<lifeless> StevenK: are you wearing the muppet hat?
<lifeless> StevenK: also congrats
<poolie> also, does it really need to be private?
<wgrant> Does canonical.com support Unicode subdomains?
<poolie> istm the readonly mode could be public
<wgrant> Nafallo could have looots of fun with that :P
<spm> lifeless: sodium should be back
<lifeless> spm: can you make the rsync magic magically happen ?
<spm> only by magik
<StevenK> poolie: I'm paying for the box currently, so I'd like to limit the number of people that can fiddle for the moment
<spm> lifeless: syncin'....
<lifeless> spm: would that be the magic bus ?
<spm> #42, yes.
<poolie> StevenK: logging in sends me back to the default apache "it works" page
<spm> lifeless: should be there now
<lifeless> thanks
<lifeless> pulling
<spm>  /srv/launchpad.net-logs/staging/asuka/
<poolie> i guess because of going back to http not https y
<StevenK> poolie: Right.
<poolie> you probably just want a redirect there
<lifeless> spm: /profiling/ :P
<StevenK> poolie: I just put one in, you were probably too fast
<spm> yeah, that too.
<poolie> i would have expected to see some history for previous builds?
<poolie> StevenK: istm that allowing anonymous readonly access wouldn't cost you very much?
<poolie> i'm not suggesting allowing people to start new builds
<lifeless> ooh shiny this looks like one I may be able to stop on hard
<poolie> just to see if the previous ones wokred
<lifeless> spm: also you can probably clear out > 3 day old profiling things automatically (rephrase - we need to :P)
<StevenK> poolie: Hmmm, I can do that
<poolie> just an idea
<lifeless> spm: as we're going to have on-demand profiling soonish.
<spm> gah. I thought I'd done something like that; maybe just manually...
<StevenK> poolie: And history should be there, drill down into the jobs
<lifeless> spm: I know you did manually the other week
<wgrant> StevenK: What's this thing?
<spm> rm 2008-09-16_2* <== no.
<StevenK> wgrant: A hudson install
<StevenK> wgrant: Have a look: https://hudson.wedontsleep.org/
<lifeless> StevenK: have you hooked up junitxml test reports ?
<StevenK> lifeless: Yup
<lifeless> awesome
<poolie> StevenK: hm it wasn't there before
<lifeless> yeah, thats shiny
<lifeless> bah, that failure isn'y very useful though :(
<lifeless> https://hudson.wedontsleep.org/job/db-devel/4/testReport/junit/lp.codehosting.puller.tests.test_worker/TestWorkerProgressReporting/test_network/
<StevenK> I don't understand the test failures :_/
<wgrant> StevenK: Shiny.
 * StevenK kicks apache until his redirect works
<poolie> StevenK: is this supposed to eventually supersede buildbot?
<StevenK> poolie: I'd like for that to be the plan
<StevenK> I suspect others would too
<lifeless> hmm, something funky in the test times
<lifeless> EMUSTLOOKATTHATSOMEDAY
<StevenK> It doesn't currently use ec2 to build since I suspect it would bankrupt me within 2 or 3 days
<lifeless> get a UEC account
<poolie> lifeless, the dupefinder on edge seems to be giving me pretty weird results recently
<poolie> is that affected by your search changes or is it no more weird than usual?
<lifeless> my search changes were all about the dupefinder
<lifeless> it used to | all the terms
<StevenK> lifeless: Er, but it isn't really work?
<poolie> shall i file a bug or is this known or wontfix?
<lifeless> this performed terribly
<lifeless> StevenK: its very much work :)
<poolie> yes, i remember
<lifeless> poolie: if you can quantify whats going on, please do file a bug.
<lifeless> poolie: however, until we replace the search engine, I don't think there is much we can do sensibly.
<StevenK> lifeless: It's hosted privately, it's been developed privately during spare time; doesn't sound like work to me
<lifeless> the performance curve is terrible.; perhaps we could estimate the size and use more broad searches when it won't blow up
 * StevenK is sad his mail hasn't had any replies yet
<lifeless> StevenK: thats all true; what I mean is that what you are doing is of benefit to the team; and canonical as a sponsor of the team should - dare I say would - be happy to help
<lifeless> spm: man, MailingListApplication:MailingListAPIView must get hammered
<lifeless> spm: every time, its like 90% of the profiles.
<StevenK> lifeless: So, what do you think the next step should be?
<spm> lifeless: unknown
<lifeless> StevenK: I think getting it to the point that you're fairly confident a failure is genuine is important.
<StevenK> lifeless: Agreed
<StevenK> I've had one db-devel build pass
<lifeless> StevenK: at that point, workflow changes to make it what the pqm buildbot thing does would probably be conceivable (for testfix mode stuff)
<StevenK> lifeless: So it should remain where it is for the time being while we work out kinks?
<lifeless> StevenK: its what I'd do
<lifeless> moving it into prod while its in dev would just add friction to your ability to tweak and fiddle.
<StevenK> Oh, clearly
<StevenK> lifeless: But should we investigate using UEC as executors in parallel to that?
<lifeless> sure
<lifeless> I expect gary and or maris will respond
<lifeless> I will definitely reply tomorrow if noone else has
<StevenK> Yes, I'm looking forward to that
<StevenK> The testsuite didn't like running with only 512MiB of RAM
<poolie> lifeless: so you did change it to matching N-1 terms?
<lifeless> yes
<lifeless> vs 1 :P
<lifeless> poolie: specifically it searches for a match in any of the N-1 subsets, and scores across all of them
<lifeless> so the more detailed you are, the less results you'll get
<lifeless> but, unlike before, you can trim too-many-results by being more detailed
<poolie> that's true
<poolie> but not a very obvious use of this dialog
<lifeless> yes
<lifeless> I'd have liked to have kept it just as it was
<lifeless> and hope to restore that when we overhaul search
<poolie> not complaining
<lifeless> I know - just expounding
<lifeless> hah!
<lifeless> is_empty cheap
<lifeless> 22% of bug one is in that 'cheap' query
<lifeless> jtv: ^
<jtv> 22% of Microsoft's market share?
<lifeless> no
<lifeless> of rendering bug 1
<_mup_> Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Invalid by compscibuntu-bugs> <EasyPeasy Overview:Invalid by ramvi> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <The Linux OS Project:In Progress> <OpenOffice:In Progress by lh-maviya> <Tabuntu:Invalid by tinarussell> <Tivion:Invalid by shakaran> <Tv-Player:New> <Ubuntu:In Progress by sabdfl> <ubuntu-express (Ubuntu):Invali
<poolie> https://bugs.edge.launchpad.net/malone/+bug/626656 fwiw
<jtv> Thank you mup, we had that one perma-cached
<jtv> lifeless: that's astounding
<lifeless> jtv: not to me
<_mup_> Bug #626656: dupefinder now over-tight <Launchpad Bugs:New> <https://launchpad.net/bugs/626656>
<poolie> and fwiw it did in fact fail to find my dupe when used in the usual way
<lifeless> poolie: thanks; an open question is whether the usual way should be changed
<lifeless> certainly for apport bugs I expect the current behaviour to be ok
<poolie> yeah, arguably we should not just restore the old behaviour but instead reconsider the story of bug filing
<poolie> hm
<poolie> apport bug duping seems problematic in different ways
<adeuring> good morning
<poolie> hi abel
<jtv> hi adeuring
<adeuring> hi jtv!
<lifeless> jtv: I think its easy for folk to under-estimate the impact of repeated small queries
<lifeless> jtv: its more work for the db - scales N rather than log(N); its more friction up and down the call stack.
<lifeless> jtv: in short, it adds up - a lot-.
<jtv> lifeless: otp
<lifeless> jtv: de nada, catch you another time.
<mrevell> Morning
* StevenK changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 3 of 10.09 | PQM is OPEN | firefighting: - | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews
<StevenK> ... I think it's week 3
<jtv> hi mrevell, otp but your help bubble is up on edge.
<poolie> lifeless: oh, random thought from somewhere on the highway
<poolie> is it at all possible to emit (perhaps shrunken) versions of sql queries into the html as its rendered
<poolie> in comments, obviously, and of course only for some users
<poolie> or will the template/view layering mean that we don't have a good view of when the query was run, or what caused it?
<lifeless> yes, but it would probably cripple things right now
<poolie> in what sense?
<lifeless> (because template rendering is already a slow spot, and many pages do silly-count numbers of queries.
<poolie> ah right
<lifeless> so we'd be adding a lookup into a slow code path, that would be exercised a lot.
<poolie> sure
<poolie> perhaps it could be turned on by a cookie or query parameter
<poolie> but it might still be too much
<poolie> and the implementation may not be trivial
<jtv> lifeless: off phoneâ¦ about the is_empty queries: sounds like a Foo, count(*) query in disguise.
<poolie> because it can't insert comments just anywhere
<jtv> poolie: you want to be able to trace back queries to the TAL that necessitates them?
<poolie> yes
<poolie> perhaps people already have tools to do this or can almost always guess correctly?
<jtv> I would like that too.  But getting pretty deep into the critical path.  What if we had a way of inserting HTML comments with the current query count, so we could couple them to the oops report?  Not great, but relatively low impact.
<poolie> istm on the way to implementing that, you'd want a thing to tell TAL "emit this comment as soon as is convenient and legal"
<poolie> maybe there is such a thing already
<lifeless> jtv: I haven't checked the code yet, but I'm pretty sure its a if block guarding an 'expensive' query
<lifeless> in a loop
<jtv> lifeless: so a Foo, EXISTS(â¦) query in disguise.
<lifeless> jtv: well, I think it just wants to be a single Foo query
<lifeless> I'm saying the exists checking is totally unnecessary
<lifeless> poolie: I love your idea; please do wishlist it in lp-foundations
<lifeless> poolie: I was merely commenting on the pragmaticness of it today ;)
<jtv> lifeless: it's not blocking anything _except_ an expensive query?  Or is it a case of "this entire piece of UI shouldn't be displayed"?
<mrevell> thanks jtv
<mrevell> jtv, I'll land a branch today updating the help content.
<jtv> mrevell: yup, you can do that now!  Maybe we'll want things like "user hasn't done any translation _recently_" or (a bit harder) "user hasn't done any translation since I last updated this text."
<lifeless> jtv: bug 607935, feel free to dig
<_mup_> Bug #607935: timeout on bugtask:+index <timeout> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/607935>
<lifeless> 149 calls to bug.isSubscribed and bug.isSubscribedToDupes
<jtv> lifeless: I'm a bit sick today, so any effort will be sporadic
<lifeless> 148 to the second
<lifeless> for team in self.user.teams_participated_in:
<lifeless>  ^ warning, this may loop a lot
<poolie> lifeless:  https://bugs.edge.launchpad.net/launchpad-foundations/+bug/626673 fwiw
<lifeless> if bug.isSubscribed(team) or bug.isSubscribedToDupes(team)):
<_mup_> Bug #626673: want sql statements interleaved in html comments <Launchpad Foundations:New> <https://launchpad.net/bugs/626673>
<lifeless> poolie: thanks!
<poolie> maybe i should have a selfimposed wip limit for wishes
<lifeless> jtv: so, this is yet another example of what I've been going on about : looks cheap, but actually, its a good fifth of the page, and a simple set intersection query, done once, can return the info needed
<poolie> btv so do you think that would have ever helped you, had it existed at the time?
<lifeless> maybe
<lifeless> it would provide some hint
<lifeless> but what I've seen a lot of you'd just get many queries smooshed at the top of the page
<lifeless> (and in fact, I want us to head to that: no queries mid-page)
<poolie> right
<poolie> i think being told that is still useful
<lifeless> also, hah! this template does this:
<lifeless> s/template/browser:
<lifeless> self.many_bugtasks = len(self.bugtasks) >= 10
<jtv> that's an old favoriteâ¦
<lifeless> oh nvm that one is actually cheap, I guess I'm tired ;)
<lifeless> I was going to say that count() is about as expensive as the full query
<lifeless> and thus to be avoided like the plague
<wgrant> lifeless: Isn't it often even more expensive?
<wgrant> Given that the full query is batched.
<wgrant> Normally.
<lifeless> wgrant: well you're comparing across layers there
<lifeless> batching is something the API or UI exposes
<lifeless> count() is ~= to getting the last row
<lifeless> but, its worse
<lifeless> batching in the API makes iteration O(N^2)
<lifeless> which reminds me, file a bug to turn it off
<lifeless> https://bugs.edge.launchpad.net/launchpad-foundations/+bug/626680 if you're interested
<_mup_> Bug #626680: iteration in LP API's is O(N^2) due to batching <Launchpad Foundations:New> <https://launchpad.net/bugs/626680>
<jtv> lifeless: is the problem that that subscription line you quoted earlier, the "if bug.isSubscribed" etc. one, should be cached across many bugtasks on a page?
<lifeless> jtv: no, its looking for the intersection of 'which of this bug& its dupes am or or my teams subscribed to'
<lifeless> so someone in a lot of teams (e.g. me) will cause 2 queries per team: one for subscribed to the bug, one for subcribed to a dupe.
<lifeless> its building a list of unsubcribe links
<lifeless> so that when you get mail you can click on the link
<lifeless> and in the bug you see 'unsubcribe <team-that-is-subscribed> from this bug'
<lifeless> we can halve the query count with a trivial helper to query bug.isSubcribed and bug.isSubscribedToDupes in one query
<lifeless> and we can make it constant by doing a query that will return the team objects that are in the users participations && subscribed to the bug or a dupe.
<lifeless> that will make the OOPS report a lot easier to read, for the next iteration (and according to the report, take 1.5 seconds off of the page, for me, *for all bug pages I look at*.
<jtv> lifeless: for cases like this one, perhaps we should get used to exposing methods that compose collections, Storm queries etc.  One of our slowest pages could benefit handsomely from a helper that makes it easy to prejoin the icons for a listof persons or products.
<jtv> If we keep optimizing these things at the call sites, we give up some schema flexibility.
<lifeless> this is why I want a separate layer
<lifeless> I like generic collection
<lifeless> I don't think its enough, nor a fit for this sort of thing (yet)
<lifeless> well, it might fit this
<lifeless> but its a little weird starting with team (the result you want), filtering by your-participations, then by your-bug-subscriptions. Perhaps its not.
 * jtv is too used to that to question it
<lifeless> whats missing from collections is getting multiple types back
<jtv> How is that missing?
<jtv> That's built in.
<jtv> collection.find(Foo, Count(Foo.bar), PrejoinedExtra)
 * jtv steps back outside w/book
<lifeless> jtv: I don't know what PrejoinedExtra is
<lifeless> jtv: but perhaps you could look at Person.all_members_prepopulated and show me how to Collectionise it
<lifeless> *keeping the elegance* of collections.
<jtv> lifeless: PrejoinedExtra is a class that I just made up.
<jtv> lifeless: for each of those left joins, you'd refine the collection using joinOuter.    I'd also keep a list of "things I want from this query" in the collection, and extend that for each item you added.
<jtv> I'm not sure there's a truly elegant solution for the variable result columns because it's not a very elegant way to pose the problem.
<lifeless> jtv: well it also needs to populate the cached attributes
<lifeless> jtv: I'd like to have a reusable solution to this
<jtv> One way to do it might be to pass in a series of classes you want, and use that to base the query construction on, and return that same list.  Bit harsh where you need aliases, of course.  For the counts and exists etc. I think you'd want to stuff the aggregates into the result objects somewhere rather than return them.
<jtv> Sorry, I'm being imprecise.
<lifeless> thats ok
<lifeless> this doesn't have to be solved today
<jtv> You return a result set whose columns match the classes that you passed on
<jtv> passed in
<jtv> The aggregates and other things that aren't easily identified as classes probably shouldn't stand on their own; you could have a delegating "pseudo-model" object, e.g. BugWithTasks, that holds cached data so it's similar to a Bug with lots of @cachedpropertys except the caching goes away at the end of the request.
<lifeless> well
<lifeless> This comes back to separated storage and logic
<lifeless> which I want, I don't think that pushing caching out to a separate place is a sane approach
<lifeless> I think that pushing storage out to a separate place may work
<jtv> lifeless: sorry, I just noticed I'm no shape to go in depth right now
<lifeless> no rush
<lifeless> go take some aspirin and lie down
<jtv> y
<deryck> Morning, all.
<Ursinha> sinzui, hi, should I add qa-bad to bug 624981?
<sinzui> No, the code never arrives
<sinzui> arrived
<sinzui> Ursinha, and the code we are using is not on edge or staging, though I landed it first
<sinzui> Ursinha, so the bug was tagged for testing, but there is no sever to test on
<Ursinha> sinzui, I see
<lifeless> moin moin
 * lifeless wonders when the next edge update is
<deryck> hi lifeless
<lifeless> hi deryck
<lifeless> deryck: you might like rev 11472 of devel/stable
* lifeless changed the topic of #launchpad-dev to: Launchpad Development Channel | Performance Tuesday | Week 3 of 10.09 | PQM is OPEN | firefighting: - | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews
<deryck> lifeless, I saw the commit message, but haven't looked at what's in the rev yet.
<lifeless> deryck: its a -little- ugly due to some lazr.restful limits
<lifeless> deryck: but function, very functional
 * deryck takes a break from test fixing to look
 * benji wonders if anyone has looked at just how much spam gets dumped into http://pastebin.ubuntu.com
<deryck> lifeless, yeah, that is nice.  I really like the DecoratedResultSet pattern that we've got now.
<lifeless> cool
<lifeless> deryck: leonardr: either of you know what pageid api/1.0/bugs?assignee=xxx would have ?
<leonardr> lifeless, probably something like IBugSet:assignee
<leonardr> or IMaloneApplication:assignee
<lifeless> IBugSet has no exported() decorators
<lifeless> leonardr: does that mean its not exported?
<deryck> That translates to searchTasks call.  So wouldn't it be a product.searchTasks or distro
<SpamapS> lifeless: seems like it would be a really really fast query.. unless there's no index on the assignee column for bugs. ;)
<lifeless> SpamapS: lets get data
<lifeless> SpamapS: just figuring out where the code is
<leonardr> lifeless: yeah, it's not exported, IMaloneApplication is /bugs/
<lifeless> so MaloneApplication:+bugs perhaps
<lifeless> which on thursday (last day the ppr ran without sodium trashing it)
<lifeless> 256 hits
<lifeless> 1744 total seconds,
<lifeless> 99% in 16.97 seconds, 6.82 mean, 3.38 stddev
<SpamapS> lifeless: I think one problem is, its actually the global bugs list.. assignee=clint-fewbar isn't the right way, is it?
<lifeless> mean 33 sql statements
<lifeless> looks like its mainly sql time
<lifeless> SpamapS: no reason for that not to be fast
<SpamapS> lifeless: agreed, :)
<lifeless> SpamapS: Ubuntu is ~50% of bugs anyway, so you wouldn't save jack by filtering by it
<lifeless> lets see if it has oopsed
<lifeless> if it has we'll have some nice data, if it hasn't we can get a profile from staging.
<lifeless> it is, for reference, in my hitlist already: https://devpad.canonical.com/~stub/ppr/lpnet/daily_2010-08-25_2010-08-26/timeout-candidates.html
<lifeless> about half way down
<lifeless> elmo: around ?
<lifeless> SpamapS: so now I'm grepping our oopses reports (slow page diagnostics)
<lifeless> which will take a while
<lifeless> leonardr: so, I'd really really love to get oops ids on apis when ++oops++ is used
<lifeless> leonardr: do you have any ideas about a tasteful way to do that
<leonardr> lifeless: can you quickly run down how it works on the website?
<lifeless> leonardr: sure
<lifeless> ++oops++ is triggered by traversal, a match anything adapter
<elmo> lifeless: vageuly
<lifeless> it sets a global variable that says to the oops code
<lifeless> elmo: is there an ETA on sodium - its dying so much that many cron based things we use regularly are not completing
<elmo> lifeless: it's been body swapped already
<lifeless> leonardr: so when ++oops++ is traversed it sets a glad in errorreport.py
<lifeless> elmo: !
<elmo> lifeless: that didn't take - the latest theory is that the disks are fucked in interesting ways that is causing the kernel to crash
<lifeless> elmo: -ah-
<elmo> lifeless: we're going to force an fsck of the disks and if that fails, just give up and replace the box wholesale
<lifeless> elmo: sorry for nagging then.
<lifeless> SpamapS: ok, I've got an OOPS from a few days back
<lifeless> leonardr: s/glad/flag/
<lifeless> leonardr: at the end of the request this causes two things: oops report written to disk, and, the oops number put in the comment region in the main template.
<leonardr> lifeless, this only happens if there's an oops, right?
<lifeless> leonardr: these may actually be the 'same' thing - that is the comment region triggers evaluation of the 'should I write an OOPS code' (when no exception has occured)
<leonardr> no oops, ++oops++ does nothing?
<lifeless> leonardr: no, ++oops++ makes a 'user requested oops' - its generated regardless, so you get get operational info on not-quite-crashing-but-bad pages
<lifeless> there is however no big traceback unless an actual crash did occur (in which case ++oops++ has no effect on what happens)
<elmo> lifeless: no problem
<lifeless> leonardr: so, for API's something equivalent would be to somehow (not necessarily ++oops++) tell the api code that at the end of generating all its stuff, it should generate an oops
<lifeless> the oops goes to disk
<lifeless> and the oops id gets put somewhere. like an http header, or in the outermost json dict, or something
<leonardr> ok
<leonardr> how does it generate an oops? just by raising an exception?
<lifeless> it calls into the errorlog code
<lifeless> look at maybe_record_user_requested_oops
<lifeless> essentially we'd want API's to call that function
<lifeless> and store the oops id it returns *somewhere*
<lifeless> SpamapS: so https://lp-oops.canonical.com/oops.py/?oopsid=1701M1562 is *a search* that timed out on that api
<lifeless> actually no, its a regular webapp request
<lifeless> digging more
<SpamapS> lifeless: its very consistent at 11 seconds..
<SpamapS> lifeless: so I doubt it will time out.
<SpamapS> until something else slows everything down that is
<leonardr> lifeless: so, oops ids are already sent out in the X-Oops-ID header (don't remember the exact name)
<leonardr> X-Lazr-Oopsid
<leonardr> that's handled by the WebServiceExceptionView
<leonardr> it would be pretty easy to make lazr.restful check for an incoming X-Lazr-User-Triggered-Oops header or something, and raise an exception at the end
<lifeless> leonardr: so the point of user triggered oops is that they don't have an exception
<lifeless> you still get the response
<lifeless> and you get the oops
<lifeless> I appreciate this is probably more work :P
<lifeless> leonardr: here is my use case
<leonardr> since maybe_record_user_requested_oops is launchpad code, we either need to move it out of launchpad or put a hook in the configuration object
<lifeless> someone says 'api X is slow', I say 'in your browser, do XXX and look at YYY and tell me the OOPSID'
<lifeless> tell you what, I'll file a bug on lp-foundations about it
<lifeless> its possibly easy, but isn't -trivial-
<lifeless> A hook approach sounds sensible to me (though there is already IRequestEnd which is what errorlog is hooked into; is that perhaps not enough?
<leonardr> i don't know about IRequestEnd. at the very least we need a hook so that lazr.restful can set the magic variable that ++oops++ sets
<lifeless> so thats set at the start
<lifeless> its currently done via a traversal adapter
<leonardr> if you want it to be doable in the browser then we need something like ++oops++ to go in the url
<lifeless> browser would be ideal
<lifeless> its just so much easier for people to adhoc stuff with
<lifeless> ah, I have already
<lifeless> bug 606952
<_mup_> Bug #606952: ++oops++ should work on api urls <performance> <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/606952>
<lifeless> or martin has
<lifeless> SpamapS: so the fallback position is staging.
<lifeless> SpamapS: staging is a) slower b) less resources c) lower timeout.
<lifeless> I guarantee it will break for us
<lifeless> once it comes up after the code update ><
<lifeless> matsubara: ping
<matsubara> hi lifeless
<lifeless> matsubara: thanks for working on https://bugs.edge.launchpad.net/launchpad-foundations/+bug/606184
<_mup_> Bug #606184: API Pageid for collections is 'scopedcollection:collectionresource'  which does not mention the origin page id <qa-needstesting> <Launchpad Foundations:Fix Committed by matsubara> <https://launchpad.net/bugs/606184>
<lifeless> matsubara: did you see my follow up there ? :)
<matsubara> lifeless, np, I saw your comment there and will follow up later
<lifeless> ok kk
<matsubara> lifeless, but I guess, the fix will have what you want because it builds on benji's work to include the named operation for webservice oopses
<matsubara> (unless I misunderstood what you wanted)
<leonardr> lifeless: bottom line, it's a small to medium sized project. unless you just want to be able to tell people to use their browsers, there's also a client-side component
<lifeless> leonardr: just browser would be fine in the first iteration
<lifeless> leonardr: we're very read-heavy, and GETs are fine in the browser
<lifeless> matsubara: well I wanted instead of ScopedCollection:CollectionResource, IMaloneApplication:searchTasks
<lifeless> matsubara: as I understand it you've added in the *type* of the collection, which is nice to have, but doesn't actually help pinning down the code to look at.
<lifeless> matsubara: my goal is that the pageid should be a reliable key for grouping on.
<leonardr> lifeless: in that case the main challenge is figuring out which code goes in lazr.restful and which in launchpad. i would like to do something simple like make the /++oops++/ traversal apply to the api vhost and set this magic variable
<lifeless> leonardr: that would be lovely
<leonardr> so that lazr.restful thinks it's working normally, but the code it's running inside does something different with the data
<matsubara> lifeless, take a look here: https://devpad.canonical.com/~lpqateam/edge-oops.html and you'll see how the pageid looks like now with my fix (search for scopedcollection)
<matsubara> lifeless, my understanding was that you wanted engineers looking at the oops summary to be able to pinpoint if the CollectionResource triggering the error was under their domain and then act to fix it
<matsubara> lifeless, my fix also builds on top of benji's work which includes the named operation (if any) to the page id
<lifeless> matsubara: Thats a necessary condition, but not sufficient :)
<lifeless> matsubara: I think what you've done is a great improvement.
<lifeless> But I'm greedy, I want more.
<matsubara> so, it'd look like: ScopedCollection:CollectionResource:#bug-attachment-resource:searchTasks (of course this is a made up pageid)
<lifeless> matsubara: so this one -  (ScopedCollection:CollectionResource:#bug_attachment-page-resource)
<lifeless> is actually IBug:attachments
<lifeless> if an engineer looks at the IBug interface at the exported 'attachments' collection, they are looking at the right place.
<lifeless> thats what I'd like to achieve.
<lifeless> the scopedcollection:collectionresource stuff is, AFAICT, not relevant
<lifeless> losa ping: is the staging update looking normal - they normally finish by 14 past
<elmo> lifeless: staging got upgraded to lucid today
<lifeless> elmo: \o/
<elmo> (just as a data point)
<elmo> it may be fucking with the updates
<lifeless> elmo: thank you (both for that its upgraded and letting me know)
<lifeless> elmo: it wouldn't surprise me
<matsubara> lifeless, I see. thanks for the feedback. I guess I'll have to make another patch to accomplish that. Would you file another bug please?
<lifeless> elmo: the only signal I have for this is https://staging.launchpad.net/successful-updates.txt AFAIK
<lifeless> matsubara: certainly
<matsubara> thanks lifeless
<mbarnett> lifeless: let me take a look
 * mbarnett checks to see if it finisehd in the mean time as it doesn't seem to be running
<mbarnett> nope
<lifeless> mbarnett: ok, so its still going, thats fine.
<mbarnett> lifeless: it isn't
<lifeless> mbarnett: is it doing a db restore or just code update?
<mbarnett> lifeless: last i see is an error in the logs
<lifeless> mbarnett: oh, its fallen over?
<lifeless> can I see it?
<mbarnett> lifeless: here is the tail of the log...
<mbarnett> https://pastebin.canonical.com/36480/
<mbarnett> let me know if you would like to see more
<lifeless> matsubara: see bug 627027
<_mup_> Bug #627027: further improvements to API collection page ids <Launchpad Foundations:New> <https://launchpad.net/bugs/627027>
<lifeless> ah slony, I love to hate you
<lifeless> mbarnett: AIUI all clients have to be kicked off before the upgrade can happen, but there are staging sso clients connected so it would fail.
<lifeless> mbarnett: those sso appservers may have failed to shutdown as per spm's bug filed yesterday
<matsubara> thanks lifeless
<lifeless> mbarnett: so I'd be inclined to check if thats the case, if so get a thread dump and attach to the bug, then nuke them, then kick the staging restore to go again
<lifeless> s/restore/update/
<lifeless> mbarnett: of course, you probably are already doing all that and I'm just annoying :)
 * lifeless goes for breakfast so he won't be annoying 
<mbarnett> heh, i have not initiated any of it, so i will get on it momentarily
<lifeless> now, where was I
<lifeless> SpamapS: so, we'll look into your api performance question when staging comes up
<lifeless> SpamapS: bugs that would have made it easier to look into have been tickled.
<lifeless> but for now, I'm going to make BugTask:+index faster
<lifeless> it does *300* team subscription lookups on every page for me.
<mbarnett> lifeless: heya, do you have that bug #?
<lifeless> sec, I'll grab it
<mbarnett> i am having parsing problems from my weekend mail!
<mbarnett> thanks
<lifeless> 626577
<lifeless> (I went to launchpad-project and searched for shutdown)
<mbarnett> thx
<mbarnett> hmm, i wonder if this is the same issue..  lp appservers are a bit differerent than the sso client
<lifeless> oh?
<lifeless> I thought the sso account was for the sso appservers, which are a fork of lp ?
<lifeless> Happy to be told I'm wrong :)
<SpamapS> lifeless: doh
<mbarnett> sso clients are running their own package, no longer out of lp trunk
<SpamapS> lifeless: glad I could help tickle some things. :)
<lifeless> SpamapS: :P
<lifeless> mbarnett: ah yes, so it would depend if they have got the same issue merged into them
<SpamapS> Now I just need to figureo ut why jquery/chrome are not happy w/ the response coming back from the API calls..
<lifeless> mbarnett: have they failed to shutdown though ?
<mbarnett> yeah, the sso stuff runs right out of apache, so i think they don't
<lifeless> mbarnett: really? wow, thats very different. And it still talks to the LP db ?
<mbarnett> they have their own sso database
<mbarnett> that part of is replicated back into lp
<lifeless> mbarnett: are teams replicated to their DB ?
<mbarnett> that i am not sure
<lifeless> (because sso.ubuntu.com still serves team data)
<mbarnett> but if you notice
<mbarnett> the sso_staging connections are to the sso_staging database
<lifeless> yeah
<lifeless> slonik takes a lock on all replication sets before doing any schema changes
<lifeless> its a ... feature
<mbarnett> i do see 1 read only thread on the lpmain db though
<lifeless> that would block it
<mbarnett> let me see if that is still there
<mbarnett> if not, i'll fire back off the staging restore
<mbarnett> maybe someone connected at just a very bad moment
<mbarnett> after the kill switch but before the next step..
<lifeless> mbarnett: its just a regular update, not a db restore today though, isn't it ? (with incremental db schema changes done automatically)
<lifeless> SpamapS: can you please file a bug - IMaloneApplication:searchTasks is slow, include the url you used.
<mbarnett> lifeless: the db update failed over the weekend.. stub made some recommendations for a code update then another full update
<lifeless> ah ok
<mbarnett> so, i believe this to be the full update
<lifeless> ok, I shall be patient :)
<mbarnett> ok, i am kiling the read only idle connection
<mbarnett> and fire it back off
<mbarnett> dead
<mbarnett> trying again
<SpamapS> lifeless: Definitely, I have added it to my todo list, I won't get to it for a couple of hours (have to run to an appointment).
<mbarnett> and, we are off.
<mbarnett> lifeless: this will of course take a while.
<lifeless> mbarnett: of course; thank you!
<lifeless> SpamapS: thats ok, we won't fix it for a couple of months.
<rexbron> hey
<rexbron> could any one comment on why the run command is not supported in recipes on launchpad?
<lifeless> because it can run arbitrary code
<lifeless> recipes are evaluated in many contexts, only some of which are secured
<rexbron> I thought that because of this recipes were run in vms, same as builds
<lifeless> source package creation runs in vms
<mwhudson> building the source package from the debianized source tree is done in vms
<rexbron> http://blog.launchpad.net/cool-new-stuff/launchpads-build-farm-improvements
<lifeless> any bugs or registry folk around
<lifeless> sinzui: perhaps you are here?
<rexbron> lifeless: basically, I need to nest and move the debian folder in the the upstream src from the debian branch
<rexbron> merging complains of no common ancestor
<rexbron> and nesting was suggested as a work around
<lifeless> please file a bug with tasks on udd and bzr-builder
<lifeless> with enough details to reproduce
<lifeless> certainly we need to meet your needs, but the run command is not how we'd like to do that
<rexbron> lifeless: basically, In an ideal world, you could merge just a directory, ignoring the rest
<lifeless> spiv has been working on that
<lifeless> I believe you can with the lastest builder; it might not be rolled out yet
<rexbron> lifeless: rolled out to edge?
<lifeless> rexbron: production
<lifeless> there is no 'edge' for anything other than the appservers.
<rexbron> ahh
<lifeless> We're going to get rid of 'edge' for appservers too, but start rolling out much more often (without downtime) in the near future
<lifeless> losa ping - staging update; just want to make sure it hasn't fallen over and died again.
<rexbron> hmm... so what are my options at this point?
<rexbron> lifeless: ^
<lifeless> file the bug as I requested
<lifeless> udd is a high priority for the bzr team
<rexbron> udd? I'm not familar with that term?
<lifeless> launchpad.net/udd
<lifeless> file a bug there
<lifeless> and also add a task to bzr-builder
<lifeless> jkakar: around ?
<rexbron> https://bugs.edge.launchpad.net/udd/+bug/627119
<_mup_> Bug #627119: Can not merge branches that have no common ancestor <Ubuntu Distributed Development:New> <https://launchpad.net/bugs/627119>
<jelmer> thumper: hi
<thumper> jelmer: morning
<jelmer> thumper: did you see my conversation with Peter? It looks like the chicken repo works now.
<thumper> yep saw that
<thumper> awesome
<thumper> thanks for following up
<jelmer> thumper: I was wondering if it would still be possible to do a CP (and perhaps also include the newer bzr-svn ?)
<thumper> jelmer: we are in week 3 now
<thumper> jelmer: and it isn't really critical
<lifeless> thumper: what does that mean
<thumper> lifeless: what are your thoughts on this?
<jelmer> thumper: ok, fair enough
<thumper> lifeless: what I'm saying is that we aren't that far from a real release
<thumper> jelmer: does it need the bzr 2.2 updates too?
<lifeless> thumper: I'm torn
<thumper> lifeless: me too
<lifeless> on the one hand I love getting things out of the way - low kanban limits
<thumper> obviously we want working stuff out ASAP
<lifeless> on the other hand, a release is soon
<lifeless> thumper: flip a coin?
<thumper> is the overhead of chasing the change onto production worth the gain of releasing it 8 or 9 days earlier?
<mwhudson> wait until release, spend the time that would have been spent chasing reducing the overhead?
<lifeless> I like that suggestion
<lifeless> if we have a figure in mind - say we reckon it will take 5 hours to do a CP for all of this.
<lifeless> spend those 5 hours making the next one < 5 hours
<mwhudson> gary_poster: did we find out why bootstrap.py was connecting to the internet?
<jelmer> thumper: I don't know of any people other than Peter are seriously affected by these issues.
<thumper> jelmer: given that he has been waiting months, another 8 days won't kill him :)
<thumper> I know that isn't a great reason
<thumper> but...
 * thumper shrugs
<lifeless> personally, I'd say this.
 * lifeless says it
<lifeless> jelmer: if you want to do the work, I'll +1 a production change for it. But it goes by the book and with due caution.
<jelmer> lifeless: Thanks, I'll pass. :-)
<lifeless> thumper: one thing not covered in our analysis here: doing CP's reduces the size of the 'release'.
<thumper> lifeless: one of things that makes it a little less clear for me is the dependancy on the 2.2 change
<thumper> there was a big change to get that in
<thumper> which caused changes in a lot of tests
<jelmer> lifeless: Thumper and I discussed a CP earlier, but admittedly that was much much earlier in the cycle (first few days of week 1 I believe).
<thumper> if the tests for the new plugin uses these changes, then it won't apply cleanly
<thumper> that is my fear/concern
<lifeless> 2.2 was tricky
<lifeless> yes
<lifeless> I share the concern - thus the due caution comment
 * thumper afk for a few minutes for coffee and planning
<jelmer> hi rexbron
#launchpad-dev 2010-08-31
<mwhudson> lifeless: did you find out what was going on with your query counts yesterday?
<rexbron> hey jelmer
<lifeless> mwhudson: very sure its the storm bug with calling _init early
<lifeless> mwhudson: its a cache-reuse triggered bug
<lifeless> mwhudson: so I just added fuzz and landed
<mwhudson> lifeless: ok
<lifeless> r 11742 if you want to see the comments
<lifeless> is is safe to use with security proxied things
<lifeless> mwhudson: ^
<lifeless> thumper: ^ do you know ?
<thumper> lifeless: ENOTENOUGHCONTEXT
<lifeless> for person_or_team in iterator:
<lifeless>   if person_or_team is self.user
<lifeless> would that be safe? or does is work too deep and be false if person_or_team had a new proxy instance wrapped around it ?
<thumper> I think safer to use ==
<thumper> as that goes through the proxy whereas is doesn't
 * mwhudson too
<lifeless> I wrote it as .id == ....id
<lifeless> because of concerns about is
 * lifeless chalks up another downside of sec proxies
<mwhudson> is there some command line tool you can run to pop up a notification?
<mwhudson> (not a launchpad question!)
<ajmitch> mwhudson: libnotify-bin package looks like it has something
<mwhudson> ajmitch: yeah
<mwhudson> just found that myself
<mwhudson> thanks :)
<ajmitch> not that I've used it :)
<lifeless> spm: losa ping :P
<spm> yo
<lifeless> spm: I wants to know if staging's restore is working
<lifeless> or gone off the rails
<spm> bit of column A, bit of column B
<lifeless> (its been 5 hours I think, now)
<spm> hrm. it's doing a full restore; so ETA is probably around 5-15 hours. I'm not sure why it's down atm tho. should be using the main DB. I suspect something else broke spectacularly and hence...
<lifeless> spm: so it can be brought up ?
<spm> I don't know, I'll try. just need to unpack a few layers of interrupts first.
<lifeless> kk
<lifeless> spm: is it safe fo rme to use the staging ro connection from devpad ?
<spm> yup; the update is into a separate DB
<lifeless> mwhudson: UPDATE CodeImportMachine SET heartbeat=CURRENT_TIMESTAMP AT TIME ZONE $STRING WHERE CodeImportMachine.id = %s: - well on its way to being the top oopser on lpnet
<lifeless> c
<spm> lifeless: just brought up staging app (only), it *seems* happy, but subect to failure without warning. be warned.
<lifeless> hah, thanks
<mwhudson> lifeless: should be an easy fix to only update if the heartbeat is more than 30 secs old or something
<lifeless> wow
<lifeless> 40 queries for a fresh product +assignments page with 10 specs.
<mwhudson> wow high or wow low? :-)
<mwhudson> i would have believed much worse
<lifeless> high
<lifeless> I'm an optimist
<wgrant> That sounds really good to me. Considering it's ancient and... well.. Blueprint.
<lifeless> wgrant: one word
<lifeless> ValidPersonCache
<wgrant> Hah.
<lifeless> bug 618367
<_mup_> Bug #618367: Distribution:+assignments is taking a long time maybe 20% of the time <timeout> <Launchpad Blueprints:Triaged> <https://launchpad.net/bugs/618367>
<mwhudson> lifeless: the good news is that the code is probably so stupid/naive that fixing it will be simple
<lifeless> mwhudson: good god you're an optimist too
<mwhudson> lifeless: well, compared to fixing a +translate timeout or something
<lifeless> check out lib/lp/blueprints/model/specification.py
<lifeless> line 748
<lifeless> yes. Its string-concatenation-to-build-sql time.
<mwhudson> with a side of "oh god can we just delete projects not deactivate them please"
<mwhudson> though at least you can do left outer joins with storm
<lifeless> but not full outer :P
<mwhudson> anyway, it's horrible, but at least that method only generates one query
<lifeless> ah
<mwhudson> lifeless: does the bug require a prepopulation type fix?
<lifeless> but it sets things up to generate 200 others
<lifeless> yes
<mwhudson> i see
<lifeless> is_valid_person
<ajmitch> mm, blueprints - why do I never hear good things about it in here?
<lifeless> give you one guess?
<ajmitch> it's shit?
<lifeless> its unmaintained
<lifeless> and
<lifeless> its extremely close to bugs
<persia> Has been for >4 years too, which only adds to the pain
<ajmitch> which is why wgrant always wishes that it be destroyed
<lifeless> there are variously held opinion about whether specs are even different enough to bugs to warrant the amount of unique code
<wgrant> They're not.
<wgrant> Kill them.
<lifeless> well, if it was that easier :P
<lifeless> self.context is the model object, right ?
<wgrant> In general, yes.
<lifeless> oh god
<persia> wgrant, The one difference I find useful between blueprints and bugs is the ability to link a blueprint to an arbitrary external URL.  This used to be available for bugs, but doesn't seem to be around anymore.
<lifeless> that place I referenced?
<lifeless> not used by this code path.
 * mwhudson gets ready to ship strong drugs to lifeless
<wgrant> persia: And dependencies.
<ajmitch> persia: the other useful thing that I can think of is the dependency graph
<lifeless> Its not that theres duplication, its that there isn't sharing
<wgrant> persia: Bugs don't have dependencies. This is probably a bug.
<ajmitch> sprints, not so useful
<lifeless> urk
 * persia doesn't believe blueprint dependencies are any more meaningful than bug dependencies semantically, and kinda wishes they hadn't been implemented
<mwhudson> wgrant: bug dependencies got added to the "to do, for real this time" pile recently i think
<persia> ajmitch, sprint linking of bugs would be massively useful, assuming we had a sane way to manage sprints.
<lifeless> bugs are getting relationships
<wgrant> mwhudson: As they were at UDS Jaunty.
<lifeless> one relationship will be dependency
<ajmitch> one less reason to keep blueprints
 * lifeless sobs
<persia> lifeless, Why?  There were good and strong arguments about that being semantically bad.
<lifeless> mwhudson: compare person.py specifications() and distribution.py, the same
<lifeless> persia: because vastly more people want it than don't
<mwhudson> lifeless: that's pretty special
<lifeless> persia: FWIW no data was presented in the arguments AFAICR - they were - both sides - purely rhetorical
<persia> It's not that I don't want them, so much as I know they cannot be managed in a sane way, and will lead to a maze of twisty little relationships, all different.
<persia> Oh sure, it's all rhetoric and semantics.  There's no technical reason to do it one way or the other.
<lifeless> persia: a specific case that relationships in general are needed for is this:
<lifeless> some customers with private bugs have complex relationships
<lifeless> where bug tasks expose too much data
<lifeless> e.g. two or three of *their* customers need to collaborate in little partitions on their own slice of the thing, and not know about each other
<persia> Ah, so private -> public dependencies are required for management of certain commerical relationships?
<lifeless> I'm sure there are a variety of uses all in the partial-disclosure space.
<persia> Oh, slices.  hrm.  That raises the spectre of implementing a project management solution.
<persia> (which would be a good thing, and for which there are nice mock-ups available)
<lifeless> man, this code really needs a complete overhaul.
 * lifeless fixes the proximate cause and walks away softly
<ajmitch> it's not the easiest code for someone to walk up to & fix
<mwhudson> when estimating a blueprints change recently i estimated 1 day to get over the nausea and 1 day to implement the feature
<mwhudson> was about right
<lifeless> sanity check
<lifeless> limit=xxx in sqlobject ->
<lifeless> resultset[:xxx] in storm
<wgrant> Yes.
<lifeless> and that returns a resultset to return - it doesn't actually evaluate
<lifeless> ?
<wgrant> It should return a ResultSet, yes.
<wgrant> In fact it probably just calls .config(limit=blah)
<lifeless> this is probably going to be one of these disastrous rabbit holes
<wgrant> More than likely.
<lifeless> and what are you doing this fine performance tuesday?
<wgrant> Uh, trying to wrangle a none-too-useful project team.
<lifeless> I will live in hope for next week then
<lifeless> whats the standard idiom to add another column to return to an existing resultset ?
<lifeless> also
<lifeless> do you have to us 'using' when doing outer joins ?
<lifeless> or can you just put it in the expr list ?
<mwhudson> i don't think you have to use using in that situation
<mwhudson> although i forget when you can
<lifeless> its sad that I want to write static methods to avoid fucking with interfaces.
<lifeless> s/sad/predictable/
<wgrant> lifeless: What's so bad about changing the interface?
<lifeless> wgrant: we have neither the toolchain support of java, nor the ease of use of python.
<lifeless> wgrant: and, AFAICT, not much benefit from it. I may be being slightly harsh.
<lifeless> wgrant: there are, to my knowledge, two uses from most of the interfaces we use:
<lifeless>  - security proxies : a performance *nightmare*.
<lifeless>  - API generation : so far it seems to have a high cost for generating a high performance API itself
<mwhudson> lifeless: how would you do authorization if you could throw away security proxies?
<mwhudson> i'm genuinely curious, the django approach ("explicitly check in all your view methods") seems lame
<lifeless> mwhudson: I'd like to define it in the mapping layer
<lifeless> so that we only get back from the db stuff one can see (covers View)
<lifeless> mm let me rephrase
<lifeless> I think there are several different *sorts* of thing one wants from security
<lifeless> theres disclosure
<lifeless> theres permission (yes, you may add yourself to the team)
<lifeless> maybe others
<persia> accounting: who did what.
<lifeless> anyhow, we do a bit of a mix of disclosure and permission via security proxies
<mwhudson> yeah
<mwhudson> well
<lifeless> persia: not catered for by the tech we're talking about
<mwhudson> it's all squashed into disclosure
<lifeless> persia: for all that it is true that one wants it
<lifeless> so, when I look at our code
<persia> Ah, if you7re concentrating on the middle 'A' of 'AAA', then never mind.
<lifeless> I see adhoc stuff for permission
<mwhudson> lifeless: well, to some extent we do use interfaces for accounting via Snapshot
<lifeless> I have, I think, a reasonable answer for disclosure
<lifeless> mwhudson: I'm not across snapshot yet, I need to do that
<mwhudson> but certainly not proxies
<lifeless> the answer for disclosure is 'dont retrieve what thou dost not want'
<lifeless> which we want in a very definite way for many reasons
<lifeless> performance being a prime one
<mwhudson> and very slowly and ad-hocily we're heading in that direction
<lifeless> I think that making the permission checks be all done via e.g. decorators on methods could be nice
<persia> lifeless, Take a care there: some of the current state of that comes from not checking for authorisation for direct requests to stuff normally not wanted.
<persia> (otherwise known as URL crafting attacks)
<lifeless> I certainly would like to have /one way/ to do 'yes you can do that' checks, rather than some security proxied, and some written in-line, and some via some other way I've temporarily forgotton
<lifeless> persia: huh?
<lifeless> persia: I fail to follow your reasoning; hitting up a url you don't have access too will result in the DB mapping layer returning no record and the attacker seeing a 404, in what I'm proposing.
<persia> Ah, OK.  I just misunderstood then.  I'll stop inserting stuff.
<lifeless> the big question is, what should happen if someone makes a programming mistake
<lifeless> and includes inappropriate results in a mapping output
<mwhudson> lifeless: indeed
<mwhudson> 'fail-closed' is one nice property security proxies have
<lifeless> with our current approach, we bypass the security proxy and lie to it.
<lifeless> mwhudson: we gave it up quite a while back.
<mwhudson> .. when we don't do that
<lifeless> We have three - three! different mechanisms for doing this, that I know of.
<lifeless> mwhudson: I guess I don't have solid answers yet.
<lifeless> right now code cleanliness is at the bottom of my hit-list.
<lifeless> by which I mean this whole discussion :)
<lifeless> persia: your interjections are welcome, but you may need to focus very closely and look at the LP code to see the context and issues being discussed.
<persia> lifeless, Thanks.  More I'm stopping on this discussion, because I'm insufficiently familiar with that bit of code.  I didn't mean that I'd not interject more in the future, but unless I *know* the code, it makes sense to stop at a point, or it wastes everyone else's time.
<thumper> mwhudson: in what way does etags help bash?
<mwhudson> thumper: you can go M-. on a function call and jump to the defintion
<mwhudson> like anything else
<mwhudson> thumper: large parts of lexbuilder are in bash
<thumper> ah
<thumper> ok
<thumper> anyone know how to find a branch that is linked to a translation series?
<thumper> any one would do
<thumper> I'm testing deletion on staging
<lifeless> thumper: can I give you a quick ring; needs a teddy bear
<thumper> ok
<lifeless> select * from specification inner join person on (specification.assignee=person.id or specification.drafter=person.id)
<lifeless> hmm
<lifeless> diff updating is the slow ?
<lifeless> spm: when does edge update?
<spm> lifeless: https://wiki.canonical.com/InformationInfrastructure/OSA/LaunchpadProductionStatus#Edge%20Updates 0800 BST, until lucidified, then 0800UTC
<lifeless> so bah, an hour
 * lifeless wants his fix from yesterday online
<lifeless> and a pony
<spm> shiny pony?
<lifeless> mmm shiny shiny horses
<spm> its a tradeoff. we've had cases where - and almost guaranteed when we're on a sprint - that edge has failed; so the above time is a fair balance to ensuring minimal pain and suffering all round.
<lifeless> oh, I know
<lifeless> and we're making it manual as soon as rt 4whatever is done
<lifeless> but the tradeoff is that we'll then be asking many times a day
<lifeless> :)
<spm> ....
<lifeless> spm: ....
<poolie> lifeless: should i think about why curtis ran in to trouble with flags, or delete it from my mind?
<poolie> it's not obvious to me how i broke him
<lifeless> poolie: he's ported your devel work to production
<lifeless> its been CP'd and is live I believe (spm can confirm)
<poolie> devel meaning db-devel?
<lifeless> poolie: devel and db-devel
<lifeless> all are in sync now I believe.
<lifeless> devel, db-devel, production-devel
<spm> it's not yet been CP'd, about to bee, not yet.
<lifeless> the skew was between (devel, db-devel) and (production-devel)
<spm> too many other things keep breaking and demanding attention.
<lifeless> if you want to mentate on it for your own edification, sure, but the issue is resolved.
<poolie> at this point i only want to know if i need to do something differently next time
<lifeless> uhm
<lifeless> basically if changing something that someone may want to CP a fix which depends on the API, make it clear its changed and they'll need your patch too
<lifeless> we're going to eliminate doing CP's as much as possible though by rolling out stable
<lifeless> so I don't think ou need to alter what you do
 * StevenK looks at buildbot, frowning
<lifeless> StevenK: and b) what don't you know
<lifeless> I mean, what threads have you got to pull on
<StevenK> lifeless: It seems that tests that require other services randomly fail
<StevenK> Or, mostly
<lifeless> other services being librarian et al?
<lifeless> we should have -nothing- needing internet access
<StevenK> lifeless: Things like windmill tests, and tests talking to the xmlrpc service, for example
<lifeless> windmill tests seem to be incredibly fragile
<lifeless> they seem to timeout a lot
<lifeless> maris comes to mind as the go-to guy to understand whats going on and how other folk have debugged it
<StevenK> Maris is difficult to talk to :-)
<StevenK> (Only due to timezones)
<lifeless> I can haz email!
<StevenK> lifeless: Meh
<StevenK> :-)
<lifeless> some things I'd do - run the test in isolation
<lifeless> (via hudson)
<lifeless> does it work ? yes, perhaps resources were an issue
<lifeless> are you testing devel? devel breaks. they might be real breaks.
<lifeless> try testing stable until you've got it robust
<StevenK> lifeless: Right, but the same tests keep coming up time and time
<StevenK> time and again
<lifeless> so, is there enough memory, is the box swapping
<lifeless> etc
<lifeless> brb
<StevenK> It's certainly not OOM'ing, but it will probably swap a little
<StevenK> lifeless: Or we just throw builds at EC2/UEC and see if they stick
<lifeless> you could do that too
<lifeless> spm: is it me, or is the edge update running late?
<spm> or broken...
<spm> blew up. kickin manually
<spm> Oh. no. it can't work. sodium's dead.
<lifeless> hmm, we're getting
<lifeless> Exception exceptions.AttributeError: "'NoneType' object has no attribute 'close'" in <bound method SmartSSHClientMedium.__del__ of SmartSSHClientMedium(bzr+ssh://launchpad-pqm@bazaar.launchpad.net/)> ignored
<lifeless> Exception exceptions.AttributeError: "'NoneType' object has no attribute 'close'" in <function terminate at 0x13b3938> ignored
<lifeless> in cron spam now
<lifeless> poolie: ^
<lifeless> pqm@devpad> $HOME/bin/update-launchpad-built-code.sh
<lifeless> poolie: you should be able to see that, when sodium comes to life
<lifeless> spm: what went wrong ?
<spm> [17:36:42] <spm> Oh. no. it can't work. sodium's dead.
<lifeless> spm: oh, you mean the edge update depends on sodium, the machine with a hernia?
<spm> yup
<lifeless> for some reason I though praÃ© ran it
<lifeless> whats the status of sodium, hyperbole aside
<spm> for hysterical raisins, sodium is still a critical path in the build.
<spm> needs physical visitation to stab
<lifeless> wheres nafallo :P
<lifeless> and I don't mean 'why isn't he in this IRC channel'
<lifeless> we need an rfid chip in his goatee
<lifeless> spm: I thought we had fancy remote-power stuff
<adeuring> good morning
<spm> lifeless: we do, that's not the problem
<lifeless> oh
<poolie> lifeless: that looks like the bug someone reported the ohter week?
<lifeless> spm: what happened
<lifeless> poolie: yes, they were reporting it in 'ec2 land', this is a different script but I suspcet the same root cause; its not harmful, other than a) scary and b) causing cron to mail losas
<poolie> i wonder if there is a bug
<lifeless> spm: did it falter?
<spm> nope; still going
<lifeless> ah
 * lifeless urges it on
<jkakar> lifeless: Here now.
<lifeless> jkakar: Ah, you're back home, of course.
<lifeless> jkakar: I was fighting the good fight with storm
<jkakar> lifeless: :)
<spm> lifeless: give it a whirl now?
<lifeless> ok cool. lets see if the api is better
<lifeless> woo
<lifeless> https://api.edge.launchpad.net/1.0/bugs/414746/attachments
<_mup_> Bug #414746: speakers cannot be muted when using headphones regression (karmic) <apport-bug> <apport-collected> <i386> <regression-release> <pulseaudio (Ubuntu):Confirmed> <https://launchpad.net/bugs/414746>
<lifeless> worked
<lifeless> spm: might get a profile of the bugtask index page again once that patch has percolated to stable. I don't think it has yet.
<adeuring> stub: could you look again at my MP with the schema patch for the "subscribe to search" feature?
<lifeless> stub: hey
<lifeless> stub: so ValidPersonCache bites again in specifications
<lifeless> stub: I'm wondering if we can figure out whats up with it - why it peforms so terribly
<wgrant> DELETE
<lifeless> I mean, we can eliminate all uses of it eventually
<lifeless> wgrant: read much Gordon R Dickson ?
<wgrant> lifeless: None.
<lifeless> jkakar: My perf tuesday mail today, + storm backlog, should cover all the salient points if you're interested in the things I'm rubbing up against.
<jkakar> lifeless: Am very interested, will check it out.
<jkakar> lifeless: I'm not sure what to do about the LeftJoin+aliases issue.  It sounds tricky (I'm still digesting the problem).
<stub> adeuring: done
<adeuring> stub: thanks!
<lifeless> jkakar: if Foo.reference compiled the same as Foo.reference_id, I think it would be fixinationed.
<stub> lifeless: Think of a view like a macro. It gets expanded inline into the query that uses it before things get sent to the query planner.
<lifeless> stub: does pg do materialized views ?
<jkakar> lifeless: Perhaps it's as simple as that, yes.
<stub> lifeless: No. You have to do them yourself using triggers.
<lifeless> stub: the explain for the previous VPC thing we looked at had a 'materialising' line in it, which was -hugely- expensive
<stub> lifeless: Yes, so PG decided it needed to dump that subplan into a temporary table for some reason.
<stub> VPC?
<stub> oh
<lifeless> valid person cache
<lifeless> sorry, long day :)
<stub> headache :-)
<stub> If using a view is slow a) Maybe unnecessary joins or clauses b) The expanded query blows chunks c) the expanded query is just slow.
<stub> It wouldn't be too hard turning VPC into a materialized view if we want to keep it and it looks like being a common problem
<stub> But the whole concept of 'valid' and 'invalid' people is wonky now we have accounts. And politically incorrect.
<stub> ValidPersonCache actually used to be a materialized view, but had to be switched to a real view when we split the database into two replication sets when we where being an OpenId Provider.
<stub> Or maybe I'm making that up. Fuzzy.
<adeuring> stub: patch-2207-08-0.sql is already in use ;)
<stub> adeuring: So we start the filter from 'all tags' or 'no tags' and then adjust that using the BugSubscriptionFilterTag values. Does that mean that the 'include' column there is unnecessary?
<stub> adeuring: Whoops. patch-2207-09-0.sql then
<adeuring> stub: no, we need that one too, for things like "foo -bar".
<adeuring> stub: that patch number is used too ;)
<stub> Really?
<adeuring> 2207-81 seem to be unused
 * stub wonders wtf happened to his tomboy notes
<stub> Argh...
<adeuring> erm, 2207-82
<stub> 2208, not 2207. You need to merge db-devel
<adeuring> stub: arg, thanks!
<stub> adeuring: So patch-2208-08-0.sql
<adeuring> stub: ok
<lifeless> poolie: btw the O(N^2) analysis you did back at the epic on the bug attachments api - spot on
<lifeless> poolie: I have landed a bandaid that avoids the N^2 on attachments but is N on messages, so not ideal, but tolerable
<poolie> oh thanks
<poolie> we just need more transparency, and faster landing of fixes, istm
<poolie> "just"
<jtv> lifeless: you emailed about count(*) not being cheapâ¦  anything specific you had in mind, or just adding general advice?
<lifeless> jtv: general advice
<lifeless> jtv: it was a thing that came up in a few places in the epic & discussions since
<lifeless> uhm, like about apis, and portlets, and so forth.
<lifeless> jtv: only had your name in the header cause thats what reply-all did, wasn't targeted at you per se
<jtv> It may be worth a check that we no longer have count queries in cases where we just want to know "zero or one," or "count up to x."
<jtv> In fact, maybe "count up to a maximum of" could be a useful feature for an ORM.
<jtv> gah head need lie down
<lifeless> go get well
<jtv> thanksâit's not that bad, really, just pacing myself
<deryck> Morning, all.
<lifeless> deryck: heya
<lifeless> deryck: I'm heading to be, but I hope you have a great performance tuesday.
<lifeless> deryck: I also have some good news for you (at the bottom of my perf tuesday mail i sent to the list)
<deryck> ah, good news! :-)  Let me look then....  and good night, too, lifeless  :-)
<deryck> awesome news!  and bryceh will be glad too.
<lifeless> I've mailed leann to let her know to use edge for a couple weeks
<lifeless> deryck: something weird for me though
<lifeless> on launchpad.dev
<lifeless> https://bugs.launchpad.dev/redfish/+bug/15/+index
<_mup_> Bug #15: PO file import errors should be more verbose <feature> <Launchpad Translations:Fix Released by carlos> <https://launchpad.net/bugs/15>
<lifeless> the subscribption widget doesn't show subscribers
<lifeless> is that js based?
<lifeless> local-js seems borkinated for me, or something
<lifeless> yes, definitely that - I see a +, not a pen, running firefox in the vm i s alittle better but still not right
<deryck> hmmmm
<deryck> So the subscriber's list is loaded via an XHR call
<deryck> if this fails, you won't see subscribers.
<lifeless> that would match
<lifeless> this is for bug-607935
<lifeless> where I have a massive improvement in query counts but something broke (which tests caught), I'm trying to see if it looks sane locally.
<lifeless> I'll dig in the morning
<lifeless> we'll cross paths I'm sure
<deryck> the page could be oopsing.  You can load it directly.  Just look via inspector or firebug to see what url the page wants to load.
<deryck> lifeless, ok, sure.  Talk to you then.  Rest well.
<lifeless> (or perhaps you could pull lp:~lifeless/launchpad/bug-607935
<lifeless> :!bin/test -vvt lib/lp/bugs/tests/../stories/bug-privacy
<lifeless> demonstrates the failure
<lifeless> only if you have the time, of course.
<deryck> sure, I don't mind looking.
<lifeless> *really* gone; past 11 :P
<lifeless> gnight
<rexbron> just a thought on how to improve the recipe service: https://bugs.edge.launchpad.net/launchpad-code/+bug/627466
<_mup_> Bug #627466: Use a recipe branch instead of copy paste for source package recipes <Launchpad Bazaar Integration:Opinion> <https://launchpad.net/bugs/627466>
<jelmer> rexbron, I'm not sure I follow - do you mean putting a single recipe file in a branch?
<rexbron> jelmer: basically
<rexbron> I like to keep the recipe files in bzr
<rexbron> but when I commit and push
<jelmer> rexbron: My guess would be that we'd eventually be able to move towards using a single branch with nested trees that get updated, instead of a recipe.
<jelmer> rexbron: Are you updating your recipes particularly often?
<rexbron> jelmer: at least for the first little while
<jelmer> I find that I don't change them after I've put them up.
<rexbron> jelmer: just a thought :)
<noodles775> Originally we'd hoped to make it an option to re-use, or easily-update a recipe for the project when creating a branch: https://dev.launchpad.net/BuildBranchToArchiveUI/UseCaseDailyBuild
<noodles775> but we didn't have the resources to implement it.
<noodles775> rexbron, jelmer ^^
<noodles775> Night all.
<jelmer> g'night noodles!
<mfraz74> What is happening with the Google Earth API change?
<lifeless> morning
<deryck> lifeless, was just about to email you
<deryck> found your test failure
<lifeless> \o/
<deryck> http://pastebin.ubuntu.com/486417/
<deryck> form validation was failing, causing the subscription to not truly be removed
<lifeless> haha
<lifeless> awesome
<lifeless> I wonder
<lifeless> is that logic even Needed on GETS ?
<lifeless> if its only used to process subs ?
<deryck> lifeless, I'm fairly confident it's only used for the +subscribe view, not bugtask index.  So GET for the +subscribe view, yes.  Not for bugtask index.
<deryck> same view for two pages
<lifeless> ah
<deryck> Making it conditional on which view or splitting this into two views would definitely save some work on bugtask index.
<lifeless> I'll file a bug for that
<lifeless> it should be efficient enough now to be out of the way for folk analysing timeouts on +index
<lifeless> thank you -very- much
<deryck> np.  Was a fun curiosity to dig into.
<lifeless> I hope it didn't take up too much time
<deryck> lifeless, nah.  And thank you soooo much for the work you're doing on performance!
<lifeless> my pleasure
<lifeless> bah, my blueprint branch also had a -single- test failure in ec2
<lifeless> hola mpt
<mpt> hi lifeless
<jelmer> 'evening deryck, lifeless
<deryck> hi jelmer
<benji> wgrant: I hear that you have a launchpadlib script that I should take a look at.
<lifeless> benji: its 0432 for him. Good Luck.
<benji> heh
<lifeless> leonardr: how long till your EOD ?
<lifeless> SpamapS has some APIs stuff he wants to ask, about jsonp, and i figure you'll be important for answering :)
<leonardr> lifeless, 2 hours and change
<SpamapS> leonardr: am hoping the public data in launchpad's API can be exposed via JSONP to allow AJAX calls directly to api.launchpad.net
<leonardr> SpamapS, yeah, we can do that
<SpamapS> leonardr: I'd even be interested in doing the dev work on it myself. :)
<leonardr> SpamapS, ok, this would go into lazr.restful. the callback would be passed in with a query string argument called ws.invoke or something
<SpamapS> leonardr: any thoughts on the data security implications? JSONP does sort of discard the XSS controls that would normally keep launchpad's data safe from other nastiness on a page.
<leonardr> well, api.launchpad.net doesn't know who you are, so no private data will show up
<leonardr> launchpad.net knows who you are, but we could decide to only allow jsonp on api.launchpad.net
<cr3> what's the difference between a project group and just a project? and what's the class for just a project?
<lifeless> the class for project is product
<lifeless> the class for project group is project
<lifeless> a project group aggregates projects
<lifeless> its useful for instance for launchpad which has 15 or so separate projects in its umbrella
<cr3> lifeless: gotcha, I didn't make the leap to product
<lifeless> Roughly project -> project group as source-package->distro
<lifeless> this is a piece of _massive_ techdebt
<SpamapS> lifeless: did you have other thoughts on how to get those bug lists btw?
<lifeless> SpamapS: do you have an exmaple url?
<SpamapS> lifeless: if I could iframe something that has the same table content as this: https://bugs.launchpad.net/~clint-fewbar/+assignedbugs  that would be fantastic.
<lifeless> SpamapS: rendered and all ?
<lifeless> there is probably/possibly a template just for it
<SpamapS> lifeless: at the moment, I'm working around the fact that I have zero server side scripting capabilities in the WI tracker.
<lifeless> oh
<lifeless> you could just move it to a better box:)
<SpamapS> Yes, I'm at least trying to get 1 or 2 people to tell me to do that before asking IS for something. :)
<cr3> when adding to long list of base classes in a class definition which is not completely alphabetically ordered, do I just add to the tail of the list? I'm thinking ordering might be important when base classes like IPillar or IServiceUsage are in the list of base clases, maybe those should be near the end as opposed to mine
<lifeless> cr3: I really wish you were doing this adjacent to lp, not in the mix. I think it would go faster and end better.
<cr3> lifeless: it's called hardware certification :)
<lifeless> cr3: that one isn't really integrated in though, is it ?
<lifeless> anyhow, to answer your question; I wouldn't touch the inheritance order. Its semantic, affects the MRO.
<lifeless> sorting is only valid for nonsemantic lists.
<cr3> lifeless: perhaps we have a different perception on how to go about having a results tracker adjacent to lp
 * cr3 wouldn't mind taking a stroll in lifeless's brain for a while
<lifeless> so what does 'integrated' mean
<lifeless> To me it means that the lp UI would show links to result tracker data as appropriate
<lifeless> but that its code wouldn't be responsible for that data
<lifeless> probably use LP APIs as a backend
<lifeless> (which would shake out a bunch of lurking performance issues for leonardr
<cr3> lifeless: so have another web service running elsewhere which would expose a restful API, not so disimilar to LP, to be queried by views in LP and exposed to users that way?
<lifeless> yes
<lifeless> I *thought* that was precisely what we spoke about
<cr3> lifeless: that was indeed precisely what we spoke about, but then I was given the impression by flacoste and jml that it should be in launchpad core because of all the concepts shared between LP and the Results Tracker: users, projects, source packages and maybe even branches
<lifeless> I'm confused about the timeline though
<lifeless> jml was not in montreal after the epic
<lifeless> but you say that this guidance happened in montreal
<cr3> lifeless: yes, I'm confused myself that I proposed to have this separate service in prague. I first pitched this idea to flacoste in berlin, then to him and jml in montreal.
<cr3> lifeless: in prague, I'm very surprised I said this was still the approach agreed upon
<cr3> lifeless: perhaps I said that this was the original design, but I apologize for having introduced this confusion
<cr3> lifeless: on the other hand, I'm pleased to hear you liked my original idea :)
<lifeless> I know that you need to be building a UI
<lifeless> that is something I completely agree on
<lifeless> and was clear in the QA room
<lifeless> but that doesn't, to me, imply same database.
<cr3> lifeless: regarding the UI, do you mean the dashboard rick spencer really wants?
<lifeless> no
<lifeless> whatever UI you build
<cr3> API?
<lifeless> you need to engineer towards the experience
<cr3> lifeless: the experience at this point is mostly geared towards enabling platform folks to persist test results and expose an API so that they can expose their own UI
<cr3> lifeless: until we observe a convergence in UI requirements, only then would it be worthwhile to actually expose one
<cr3> lifeless: at this point, everyone seems to want a different UI so it's not worth investing time building one. just exposing an API would be extremely useful at this point
<lifeless> I got a clear imagine in the QA meeting at the end of the platform sprint, that everyone wanted *a* UI to be built.
<cr3> lifeless: that's probably the dashboard, but that's a placebo to workaround the fragmentation of results scattered all over the place
<cr3> lifeless: in other words, the objective of that UI is to summarize information in a single location which the results tracker aims to do at the source
<lifeless> Perhaps I'm out of the loop.
<lifeless> *I* had the clear understanding that Marjo/Rick/Mdz wanted a use case - including UI - to be built up.
<cr3> lifeless: my understand is that these are separate projects which currently address different concerns, which should eventually converge in due time. however, I'll double check with people which might have a better understanding than me :)
<lifeless> what I want is the following:
<lifeless> organisational:
<lifeless>  - hnour jml's no-new-features-moratorium
<lifeless>  - have all components in Launchpad maintained [unless you have ongoing can-be-interrupted time to fix issues, this would be an unmaintained component]
<lifeless> technical:
<lifeless>  - not add writes to an unscalable system
<lifeless>  - not add stuff to a codebase we're already trying to split up into pieces
<lifeless>  - have the tracker be really flexible and reusable
<abentley> james_w`, with revno 104 of lp:bzr-builder, I get this: http://pastebin.ubuntu.com/486458/
<lifeless>   - get it delivered quickly
<cr3> lifeless: regarding the split, timing sucks unfortunately. if we were 12-18 months from now, that problem would probably be solved so I'd have precedent to work against. however, if I try to solve that problem and introduce the results tracker at the same time, I'm setting myself up for fail.
<lifeless> cr3: not true; we have other things that collaborate with LP in different ways
<lifeless> the Software centre agent
<lifeless> login.ubuntu.com
<lifeless> the UDD package importer
<cr3> lifeless: ah, good to know. so are those examples of ways you envisionned the results tracker integrating with launchpad as a separate web service?
<lifeless> they are all different, and none do quite what you want to do - but what you want to do is unclear to me because you're saying you don't need UI. Or something.
<james_w`> abentley: that's wonky
<james_w`> abentley: I can't diagnose the cause at this distance though
<cr3> lifeless: I haven't committed to delivering a UI, but I'm still double checking
<james_w`> abentley: getting the header line that it is parsing may help
<cr3> lifeless: just got confirmation, the dashboard which will exose a summary of test results will eventually make use of the results tracker api in order to summarize some of its information
<cr3> lifeless: so, the results tracker is still just about exposing an API at this point
<lifeless> ok
<james_w`> abentley: actually, that code looks very odd
<cr3> lifeless: as mentionned earlier though, if we find a convergence in UIs created by people using the API, it might then be worthwhile to learn from that experience and expose a UI at that point
<lifeless> given that, wouldn't it be simplest to just have it be a standalone API server, use API to ask LP things, and it can be queried directly.
<abentley> james_w`, yes, it appears to be raising unconditionally.
<james_w`> abentley: in the merge of your branch some code got deleted
<cr3> lifeless: that was my original proposal, I guess I need to get back to jml about this. I'll hop online tonight to make sure in person.
<abentley> james_w`, the try/except ValueError, right?
<james_w`> yeah
<lifeless> cr3: hes on leave
<lifeless> cr3: for 2 weeks
<cr3> lifeless: awesome!
<lifeless> but
<james_w`> abentley: it appears that I just deleted three lines from the file while merging
<lifeless> if we have to interrupt him that is doable
<abentley> james_w`, Ah.  Well, nobody's prefect.
<cr3> lifeless: if I could have a quick twisted instance running with a restful interface on which I could start hacking, that might work for now.
<james_w`> abentley: I apparently did: http://paste.ubuntu.com/486464/
<james_w`> only the last hunk did I do intentionally
<cr3> lifeless: my objective is to make people happy and to do so, having a prototype is good enough for now
<cr3> lifeless: how would you suggest I proceed knowing I don't need to have anything actually integrated in launchpad before jml returns from his leave?
<cr3> lifeless: also knowing this project might have to be folded into the core when he returns :)
 * cr3 starts hunting for the software centre agent and login.ubuntu.com
<lifeless> gary_poster: do you need me to dig up some help on the storm batching issue ?
<lifeless> cr3: I'd talk to noodles about the csa
<lifeless> and the software stack he used there
<gary_poster> lifeless, oh, are you a storm guy too?  Yeah, that would be fantastic, thank you!
<lifeless> no, but I play one on tv
<gary_poster> :-)
<lifeless> i was actually thinking of just grabbing jkakar in my evening
<lifeless> showing him the bug and going 'pleeeease'
<gary_poster> heh :-)
<lifeless> niemeyer is around now though
<gary_poster> I tired contacting him on privmsg but didn't get a reply
<gary_poster> (jkakar)
<lifeless> what the bug #
<lifeless> (i've forgotten already :P)
<gary_poster> bug 620508
<_mup_> Bug #620508: Slicing a ResultSet breaks subsequent len() calls. <Storm:Triaged> <https://launchpad.net/bugs/620508>
<gary_poster> oh, Jamu triaged it already, yay
<cr3> I thought len always incurred a count(*)
<lifeless> cr3: internal state in the resultset gets corrupted
<james_w`> abentley: 0.5 released with your change and on it's way in to the bzr-builder ppa
<lifeless> cr3: sliced = resultset[0:5]
<abentley> james_w`, thanks!
<lifeless> cr3: resultset.count() -> 5
<james_w`> abentley: thanks for catching that
<lifeless> or something along those lines
<abentley> james_w`, np.
<cr3> lifeless: hm, I can see how that could make sense but probably not what some people would expect :(
<lifeless> cr3: its a bug
<cr3> lifeless: feature? :)
<lifeless> bug
<lifeless> gary_poster: on assertions and the webservice
<lifeless> gary_poster: it is you/benji/leonardr's call
<gary_poster> lifeless: yes!  it is interesting topic
<gary_poster> ok cool
<gary_poster> thanks
<lifeless> but if you want to tease out what I'm thinking/concerned about, I'm delighted to chat about it
<gary_poster> I actually would love to, but I suspect the world would be better served by me trying to finish a review of one of stub's branches for now. :-)  If it arises again, though, I'll take you up on it
<lifeless> kk
<mwhudson> morning
<gary_poster> hey
<mwhudson> gary_poster: hello
<gary_poster> :-)
<mwhudson> gary_poster: have you been complained at about the 'bootstrap talks to the internet' issue?
<gary_poster> mwhudson, no, but I took it upon myself to reply--oh!
<gary_poster> and my reply was insufficient.
<mwhudson> i haven't read my email yet today
<mwhudson> though if it's a bug i might not be subscribed anyway i guess
<gary_poster> It is bug 627159
<_mup_> Bug #627159: codebrowse is taking around 10 minutes to startup <canonical-losa-lp> <Launchpad Bazaar Integration:New> <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/627159>
<gary_poster> mwhudson: codehosting is still edge-only, right?
<gary_poster> eh, not that you'd necessarily know :-)
<mwhudson> gary_poster: code*browse* is
<gary_poster> mwhudson: er, yes, sorry, that's what I meant
<gary_poster> thanks
<mwhudson> gary_poster: what was confusing me was why something was looking for zc.buildout 1.5.1
<gary_poster> mwhudson, if you don't tell the bootstrap what version to get of zc.buildout, it will go out looking for one.  zc.buildout is the only version specification that you have to specify twice, because bootstrap, unsurprisingly, runs before zc.buildout itself starts up to start honoring versions made in versions.cfg.
<mwhudson> gary_poster: ok, but it was looking for 1.5.1 on edge, which doesn't access the internet -- how did it know about it?
<mwhudson> maybe i'm misremembering things
<mwhudson> gary_poster: also is there some way you can tell bootstrap to not do that?
<gary_poster> mwhudson, I think you are remembering what happened on developer machines
<gary_poster> tell bootstrap not to do that: yes, give it a version number, as we used to do, and shall do again
<mwhudson> gary_poster: this was on staging: https://pastebin.canonical.com/36438/
<mwhudson> (asuka)
<gary_poster> mwhudson: ... asuka allows some requests through, or defining a download-base made it find 1.5.1 locally, or there's a code path I don't understand yet.  I'm pretty confident that specifying a --version will fix it though.  ...for the heck of it, I'll disconnect my dev box from the internet and run the branch that I believe fixes it and see if it works...
<mwhudson> gary_poster: cool, it's a shame that such drastic steps are required to test this sort of thign :/
<gary_poster> mwhudson, agreed.  (do you have any other ideas?)
<mwhudson> gary_poster: not really; specifiying --version to bootstrap seems like a good idea
<mwhudson> for testing, i can only think of vms
<lifeless> what would that OS... oh
<gary_poster> worked (though it revealed a Makefile bug)
<cr3> lifeless: an idea I just had is that I could integrate with the launchpad interface without even modifying launchpad by using greasemonkey scripts pointing to the separate instance. for example, if I eventually expose a UI, the greasemonkey script could expose a button: show test results for this project :)
<lifeless> cr3: put down the pipe, exhale, go get some freash air ;)
<gary_poster> lol
<cr3> lifeless: that's the problem, I just went for fresh air. no more of that for me!
<gary_poster> :-)
 * cr3 is sticking to smoking and hard liquor
<cr3> lifeless: for your reassurance, I'm currently exploring a separate standalone service. I'll keep you posted/annoyed on my progress :)
<lifeless> thanks!
<gary_poster> mwhudson, do you have a few minutes for a review of the change pertinent to what we were discussing? https://code.edge.launchpad.net/~gary/launchpad/buildout/+merge/34251
<gary_poster> it's small
 * mwhudson looks
<gary_poster> thanks
<mwhudson> gary_poster: reviewed
<gary_poster> thanks mwhudson
<mwhudson> gary_poster: i wonder if we could block pypi.python.org using iptables or something in our ec2 test images?
<lifeless> mwhudson: you can with security groups, I'm sure.
<gary_poster> mwhudson, lifeless, should be even easier once we have tarmac/PQM back in the picture
<mwhudson> i guess you could just whack '10.1.1.1 pypi.python.org' or something in /etc/hosts
<gary_poster> then the machines will be in the DC
 * gary_poster needs to reply to rockstar's email
<mwhudson> lifeless: security groups only affect incoming connections afaik
<lifeless> oh, hmm
<rockstar> gary_poster, which email?
<rockstar> Oh, tarmac one.
<lifeless> rockstar: tarmac features
<gary_poster> yeah
<gary_poster> rockstar, can do it here if that works for you
<rockstar> gary_poster, it doesn't, unfortunately.  I have to leave for class in 10 minutes.
<gary_poster> rockstar: ok np :-)
<lifeless> gary_poster: hi
<gary_poster> Hey lifeless
<lifeless> gary_poster: I realised another thing i want a few cycles to chat about with you
<lifeless> I want to lower the global timeout more
<gary_poster> ok
<lifeless> but we have same pages which are all of:
<lifeless>  - infrequently used
<lifeless>  - extremely hard to optimise today
<lifeless>  - slow
<lifeless> so I've proposed and I think you were ok, with making the timeout overridable per page
<lifeless> but
<lifeless> this needs a unique-enough key to identify the page in the appserver
<lifeless> I had been thinking to use the page id, but I'm not entirely sure now whether its uniqueenough
<lifeless> or even appropriate
<gary_poster> Yeah, not designed for that, from the get go
 * gary_poster gets into handwaving mode
<gary_poster> ...we could put it on view classes?
<lifeless> :)
<gary_poster> I expect you want it in a config file, but putting it in Python would probably be more expedent
<gary_poster> i
<lifeless> I was actually thinking to lookup (key) as a feature flag scope
<gary_poster> I see...
<lifeless> as long as (key) shows up in oopses, that would be a very low friction means to change this
<lifeless> as in, a sysadmin could just do it, when a page starts blowing up in future, file a critical bug, we fix, rule gets removed returning it to the default
<gary_poster> Yeah, I see how that's a compelling story.
<gary_poster> Thoughts:
<gary_poster> - full class name of view class?  Maybe this would be the start of a new take on page id, but for now we don't have to solve both stories at the same time.
<lifeless> sure, we don't have to conflate the two things
<gary_poster> - Zope does crazy crap with some of the zcml tags we use, making view classes on the fly.  They've been discouraged at ZC for years, for instance.  We use 'em because we have old code.  They would make a simple plane like the first bullet point harder
<gary_poster> simple plan
<lifeless> the key constraints in the short term are: - deterministic key; lookup in features; control per-request timeout if a feature rule is found
<lifeless> + key shown in oops somehow
 * mwhudson can attest to the crazy crap
<gary_poster> :-)
<lifeless> the uniqueness needed is that we should be identifying a code path reasonably precisely
<lifeless> adding stuff to the oops is easy via the request vars dict, we can wedge it in any old how
<gary_poster> - looking them up in features is a short-term requirement?  Seems like a short term very nice to have, but can see either way
<mwhudson> specifying the behaviour in the browser:page directive?
<lifeless> gary_poster: it is because:
<lifeless>  - we don't know -completely reliably- whats going to go *boom* when we change the global timeout
<lifeless>  - when we change it, we'll be flooded with issues, and the config system takes about 1.5 hours to make a change.
<gary_poster> ack, conceded
<lifeless> however, its not a big task - flags are darn easy to use.
<gary_poster> the flags are not my concern; the keys are.  Just deciding on something and trying it wouldn't be the worst approach in the world, I suppose.
<gary_poster> To pull out one thing you've said:
<gary_poster> eh, no it was obvious
<gary_poster> I was thinking about the fact that the OOPS reports are the tool you will use to analyze what needs to be don, not the performance reports, but of course, that's the way it has to be
<lifeless> I would like to use a single key across the system, but thats long term not short term
<gary_poster> agreed
<gary_poster> on both
<gary_poster> OK, lifeless, I have a few minutes before dinner.  Want me to put a bug in on this?
<lifeless> please!
<gary_poster> k, will do
<lifeless> thank you
<gary_poster> np
<thumper> morning people
<lifeless> matsubara: hey, around?
<matsubara> lifeless, yes, about to leave
<lifeless> matsubara: have you followed the thread about oops mails; do you have any thoughts? how hard are the changes discussed to do?
<lifeless> matsubara: you could answer this tomorrow if you like
<lifeless> I have one other question
<lifeless> is it possible to lookup an oops by page id ?
<matsubara> lifeless, I glanced over the thread. there are some changes easy to do (like disabling the weekly summary) others not so much. I can't really make any changes to that in the near future because I'm supposed to be helping the team with the new merge workflow. once the new merge workflow is mostly done, I can resume working on oops-tools again
<lifeless> I have a low volume but consistent oops I want to analyse, its blocking lowering the timeouts
<lifeless> matsubara: thanks for the feedback, yes, we're fully committed workloadwise ;)
<matsubara> lifeless, currently, not it's not possible. but it's doable.
<matsubara> s/not/no/
<lifeless> ok, I'll grep... pity the server :)
<wgrant> http://github.com/blog/712-pull-requests-2-0 is interesting.
<wgrant> It looks... just like LP MPs.
<lifeless> wgrant: indeed, interesting
<gary_poster> lifeless, bug 627701
<_mup_> Bug #627701: Make it possible to use feature flags to override the global timeout for specific pages <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/627701>
<lifeless> gary_poster: thank you!
<gary_poster> sure.  have a good day
<thumper> wgrant: and not a single comment referring to prior art :)
<thumper> or is that :(
<wgrant> thumper: But it's innovative!
<lifeless> well
<lifeless> MP's don't refer to things that came before either
<thumper> I was referring to the reader's comments, not the blog post writer
<lifeless> sure, my bad
<lifeless> ECAFFEINE, though at least the headaches are mostly gone now
<lifeless> anyone run into 'ValueError: Supplied vocabulary values resulted in duplicate term tokens' before ?
#launchpad-dev 2010-09-01
<benji> lifeless: you probably figured it out already, but there has to be a 1:1 mapping from values to tokens (and back)
<lifeless> benji: yeah
<lifeless> I had a query that wasn't DISTINCT
<lifeless> easy fixed
<lifeless> benji: a better error would list the token and titles that were duplicated
<lifeless> wgrant: if you're interested
<lifeless>  54 SELECT DISTINCT DistributionSourcePackageCache.archive, DistributionSourcePackageCache.binpkgdesc ...  DistributionSourcePackageCache.archive IN (%s, %s) AND BinaryPackageName.name = %s ORDER BY name:
<lifeless>    GET: 54 Robots: 2  Local: 40
<lifeless>      54 https://launchpad.net/ubuntu/+search (Distribution:+search)
<lifeless>        OOPS-1704C1791, OOPS-1704C1793, OOPS-1704C554, OOPS-1704C773, OOPS-1704C775
<lifeless> wgrant: e.g. highest oopser yesterday
<rockstar> thumper, I just got your winge email about lazr-js.  What's up?
 * rockstar has no problems
<thumper> rockstar: no worky
<rockstar> thumper, did you do a make?
<thumper> aye
<thumper> I pulled trunk
<thumper> went make
<thumper> said I had bad versions
<thumper> went to download-cache
<thumper> did bzr up
<thumper> went back
<thumper> did make
<thumper> said invalid version for setuptools
<thumper> edited version.cfg
<rockstar> thumper, what about download-cache 'bzr up'
<thumper> did make
<thumper> worked
<thumper> but examples don't work
<rockstar> thumper, do you get any javascript errors?
<thumper> not obviously
<thumper> CSS seems broken
<rockstar> thumper, probably because your make didn't actually work.
<rockstar> thumper, could you file a bug with the output of make and the like?
<rockstar> thumper, I'm concerned that my 'make' works and yours doesn't.
<thumper> rockstar: bit busy right now trying to get ian connected to my network again
<rockstar> thumper, no hurry.  I'm EOD'd, but I'd like to make it work for you tomorrow if I can.
<rockstar> Might be nice for wallyworld to be able to see the demos...
<lifeless> anyone seen this ?
<lifeless>   File "/home/robertc/launchpad/lp-branches/working/lib/canonical/lazr/pidfile.py", line 83, in get_pid
<lifeless>     raise ValueError("Invalid PID %s" % repr(pid))
<lifeless> ValueError: Invalid PID ''
<lifeless> nvm, found the files
<wgrant> noodles775: It looks to me like the distroseriesdifference model doesn't allow for multiple parent series.
<wgrant> That seems like a pretty critical feature.
<wgrant> Is it in the spec?
<lifeless> wgrant: pretty sure its not
<wgrant> Hm.
<wgrant> Odd.
<wgrant> How's that going to work, then?
<wgrant> Surely Linaro's N release is going to want to inherit from Linaro M, and then merge from Ubuntu N.
<lifeless> I suspect it will be done thusly:
<lifeless> L-N lcones from L-M, with differences, and then inherits U-N
<noodles775> wgrant: I've not heard of it before it was mentioned in that email, but I wasn't involved in the initial discussions.
<wgrant> Ah, maybe I should read this thread, then.
 * wgrant looks.
<noodles775> hrm, maybe it was on https://dev.launchpad.net/LEP/DerivativeDistributions/UserTestingRound2 ...
 * noodles775 checks
<noodles775> wgrant: yes, it was mentioned by Loic (see Mock-up 2 for Loic on the above page).
<wgrant> And cjwatson as well.
<wgrant> I should probably read the whole thing.
<noodles775> Yep.
<wgrant> Although LoÃ¯c refers to a parent distribution, not series.
<adeuring> good morning
<wgrant> Hmm.
<StevenK> wgrant?
<wgrant> A friend just ran into a bit of a strange PPA key generator case.
<wgrant> He created his first PPA a while ago, but never uploaded anything.
<wgrant> Then created a new one just now, and uploaded something to it.
<wgrant> Now, of course it had no key preset... so it's now going to get one of its own generated.
<wgrant> How often does the key generator run?
<noodles775> wgrant: currently every 20mins
<wgrant> Yeah, it just appeared now.
<wgrant> Thanks.
<lifeless> any reason that can't run on-demand ?
 * noodles775 remembers cprov requesting hardware entropy.
<noodles775> But not sure.
<lifeless> well, on-demand would have no higher or lower demand on entropy
<lifeless> on average (and if there is enough to backlog after 20 minutes, there is enough to backlog for 20 minutes
<wgrant> I thought it already did have a hardware RNG.
<wgrant> It's particularly bad, since the PPA will be published unsigned.
<wgrant> Then you have to upload something again to get it signed.
<wgrant> lifeless: But yes, the Referer check is stupid.
<wgrant> I proposed it as a quick solution.
<wgrant> I hoped that the user backslash would be enough to convince Foundations to fix it properly.
<wgrant> But apparently not.
<lifeless> thumper: https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1704EB1743
<wgrant> So... user pages have been broken for five days now.
<wgrant> Or 6. One of them.
<lifeless> losa ping
<lifeless> losa ping?
<mthaddon> lifeless: sorry, missed the last one
<mthaddon> lifeless: here now... :)
<lifeless> thats ok :)
<lifeless> I didn't want to interrupt steve if he'd EOD'd.
<lifeless> anyhow
<lifeless> two tings
<lifeless> there is a CP for the home page maps problem in LP
<lifeless> this one actually has the code
<lifeless> secondly, staging - has it successfully restored yet ?
<mthaddon> sweet
<lifeless> I'm a little worried about staging, monday PQM freezes and folk will be qaing like mad... we hope.
<mthaddon> hmm, there was a stale lockfile from the last broken restore, I've removed and it'll kick off very soon
<mthaddon> starting now...
<lifeless> mthaddon: thanks; can you please let uhm, someone, know about the CP ?
<mthaddon> lifeless: I'm doing it now
<lifeless> perhaps get a dev to tickle launchpadstatus on identi.ca or something if it works
<lifeless> mthaddon: cool cool, not rushing you
<lifeless> handing-off, I have to sleep, early wakeup tomorrow
<mthaddon> it's all scripted, so there's no rushing to do :)
<lifeless> \o/
<lifeless> (I fail at sleep)
<lifeless> SpamapS: https://lp-oops.canonical.com/oops.py/?oopsid=1704S20 btw
<lifeless> thats your slow page
<lifeless> for some reason its counting every open bug.
<lifeless> clearly daft.
<lifeless> SpamapS: https://bugs.edge.launchpad.net/malone/+bug/627974
<_mup_> Bug #627974: MaloneApplication:CollectionResource is slow/times out <timeout> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/627974>
<deryck> Morning, all.
<jcsackett> morning, all.
<jcsackett> quick question for anyone who might have an answer: is it unexpected that a test should fail on EC2 but work just fine when you run that test locally?
<salgado> jcsackett, yes, but you might try to run that test by itself on EC2, to find out whether it's test-fails-on-EC2 problem or test-fails-when-running-full-testsuite problem
<salgado> if you're brave enough to run the full testsuite locally and the test passes, then it's likely to be something related to EC2 or one of those intermittent failures we have once in a while
<nigelb> um, anyone around I could talk to about the map issue? Do you folks think its because of the sensor parameter?
<nigelb> It would be nice to have a way to check if its that.
<benji> wgrant: you around?
<Ursinha> nigelb, thanks! :)
<wgrant> benji: Hi.
<nigelb> Ursinha: :)
<benji> wgrant: hey; I hear that you have a launchpadlib script that I should take a look at
<wgrant> benji: It mostly works (albeit very slowly) when running on a lazr.restfulclient with retry support.
<wgrant> But http://qa.ubuntuwire.org/ftbfs/source/ has the code.
<mars> jcsackett, instead of running the entire suite locally you may want to try just running the part of the suite that contains the failing test.  If lp.registry.openid.foo failed, try running all the tests in lp.registry.openid
<mars> jcsackett, it is a fast-running way to see that you did not encounter a test isolation failure
<mars> jcsackett, while you are doing that you can also fire off an EC2 run or two to see if the failure is intermittent:  ec2 test -o '-vv lp.registry.foo.bar'
<jcsackett> mars: i've already run the single test locally; it passed.
<jcsackett> i'm about to fire off EC2
<jcsackett> mars: i believe it may be an isolation issue; if i run the entire bugs set of tests the one in question fails.
<mars> cool
<mars> jcsackett, if it fails locally when running that one suite then I wouldn't bother re-running on EC2
<jcsackett> mars, good to know.
<jcsackett> thanks, mars.
<benji> wgrant: we'd like for you to try you the new lazr.restfulclient (0.10.0) and launchpadlib (1.6.5) and the devel version of the web service; using those versions will be much faster for you and be easier on the servers
<wgrant> benji: Is there a PPA for those two?
<benji> mmm, I don't think so; let me see
<benji> wgrant: sorry, there isn't
<wgrant> benji: OK. It's about to hit midnight, so I'll look at installing those tomorrow.
<wgrant> benji: The script shouldn't need any changes?
<benji> wgrant: it shouldn't
<wgrant> benji: Great. Thanks.
<gmb>  Hurrah for tests that should never have passed and yet somehow did.
<SpamapS> lifeless: :) thanks, I've subscribed to that bug report. :)
<mars> rockstar, ping, have a moment for a question about the bzr pipeline mail you sent to the list last week?
<allenap> Is anyone here an ace with zope.component?
<benji> allenap: what's up?
<allenap> benji: Hi. I'm trying to land lp:~allenap/launchpad/cache-experiment-roll-out, but I keep getting adaption errors when it runs in ec2. I can't replicate them locally.
<allenap> benji: The adapters are registered normally with some zcml, but I've also added registered them with the global site manager at import time.
<allenap> s/added /
<benji> allenap: can you point me at the test results?
<allenap> benji: http://pastebin.ubuntu.com/486797/
<benji> allenap: I hava a suspicion, but it's taking forever for bzr to download the branch so I can confirm the suspicion
<allenap> benji: Shall I send you a bundle?
<benji> allenap: a diff would be good
<allenap> benji: http://paste.ubuntu.com/486805/
<adeuring> gary_poster: can I ask you again for help about this failure: https://pastebin.canonical.com/36340/ for this diff https://pastebin.canonical.com/36336/ ?
 * gary_poster whimpers
<gary_poster> trying to help noodles with an immediate problem while trying to get longer term fix done in background cycles
<benji> allenap: I /suspect/ that the IPropertyCache adapter (get_default_cache perhaps) is returning None, which means "I couldn't adapt"
<gary_poster> but will look adeuring
<adeuring> gary_poster: thanks1
<benji> of course, I don't know why that would happen in EC2 and not in dev
<allenap> benji: Yeah, it's very odd.
<gary_poster> adeuring: help me page back in Robert's comments, please--hat did he say?
<benji> you might add an assertion to get_default_cache that the return value is non-None and run just one of the failing tests in EC2 to see if my suspicion is correct
<adeuring> gary_poster: that we should try a completely different approach for the current issue, ie.e, complete aoid this code, for unrelated reasons.
<adeuring> gary_poster: problem is: his approach -- allow access to the restricted librarian for certain DC machines -- is refused by the admins
<adeuring> due to security concerns
<allenap> benji: Okay, I'll give that a go.
<adeuring> gary_poster: so, to get a fix for bug 620458, i am back to the implementation as sketched by francis
<_mup_> Bug #620458: cannot access attachments of private bugs any more <httplib2:Unknown> <Launchpad Bugs:Triaged by adeuring> <https://launchpad.net/bugs/620458>
<gary_poster> adeuring: oh, LOSAs or James or somebody weighed in and said no?
<adeuring> gary_poster: it was james
<gary_poster> ok
<gary_poster> then here we are, yes, got it. :-(
<gary_poster> Robert's concerns as to load were very valid
<adeuring> gary_poster: sure...
<gary_poster> but we don't have another answer now, I know
<adeuring> right.
<gary_poster> FWIW, adeuring...
<gary_poster> Zope/WSGI allows us to pass file objects back to serve
<adeuring> sounds interesting
<gary_poster> not immediately useful, maybe...but yeah
<gary_poster> if we had the files to serve
<gary_poster> the Zope could just give the open files back as a response (there's probably some incantation)
<adeuring> gary_poster: well, the current implementationn even creates a temp file
<gary_poster> and then the thread would be back open for business.
<gary_poster> right, but it blocks on downloading the temp file, right?
<gary_poster> well s/right/good/ :-)
<gary_poster> eh
<adeuring> gary_poster: yes. But: We have a file-like object there
<gary_poster> I meant
<gary_poster> "good, but it blocks on downloading the temp file, right?"
<gary_poster> fwiw
<gary_poster> ok
<gary_poster> so adeuring, just so I understand, this is an oauth sort of problem, right?
<adeuring> gary_poster: do you my test failures or the whole issue?
<gary_poster> adeuring, the whole issue
<adeuring> gary_poster: yes
<gary_poster> ok
<gary_poster> your test failures, AIUI, are about the fact that something somewhere in the vicinity of canonical_url does not perform as we expect
<adeuring> gary_poster: exactly
<gary_poster> adeuring: does your code work correctly in the real world?
<gary_poster> if you know what I mean?
<adeuring> gary_poster: well, it works launchpad.dev
<gary_poster> right, ok
<gary_poster> adeuring: There, of course, are tons of pieces of context that are not set up, between what you have here in the test, and what you have in live usage.  If this were my problem, I would put a pdb in getUrl, and walk through it once when it worked (launchpad.dev) and once when it didn't (tests).  If you want, give me the branch name, and I'll try to do that comparison for you.
<gary_poster> (To be clear, I'm saying I don't know what context you need.  Sorry. :-/ )
<adeuring> gary_poster: ok, just a second
<adeuring> gary_poster: lp:~adeuring/launchpad/bug-620458-private-bugattachments-api-access
<gary_poster> on it, adeuring.  What are instructions for testing live?
<adeuring> gary_poster: (1) add an attachment to a bug, (2) make that bug private, (3) use the sample code from pitti in bug 620458
<_mup_> Bug #620458: cannot access attachments of private bugs any more <httplib2:Unknown> <Launchpad Bugs:Triaged by adeuring> <https://launchpad.net/bugs/620458>
<adeuring> gary_poster: in short: lp.bugs[1].attachments[0].open()
<gary_poster> k
<lifeless> gary_poster: r375 of storm trunk has the fix we discussed
<gary_poster> lifeless, awesome, thanks for update.  May make a temporary release then , when I get to it.
<lifeless> gary_poster: It should be cherrypickable; and the workaround that is in launchpad/lazr for the sqlobject variant should be backed out
<gary_poster> agreed
<lifeless> I suspect a cherrypick on 0.17 will work better than a snapshot of trunk; there are two bugs in 0.17; one is fixed in trunk, but it exposes the other... the first one resultsin more queries, the second in an internal error.
<lifeless> you're welcome to try however :P just hoping to save you some time.
<lifeless> leonardr: ^
<gary_poster> ack lifeless, thanks
<dobey> is the entire change history of a branch_merge_proposal? like all status changes and such (not votes/comments)
<cr3> lifeless: if you had to choose between running zope or django for the standalone instance of the results tracker, what would you do?
<dobey> leonardr: ping
<dobey> or anyone else i guess
<dobey> should launchpadlib throw HTTPError when it gets a 301 redirect for an api call?
<leonardr> dobey: no. what are you doing? does launchpad.me work?
<dobey> leonardr: getSeries() on a distribution is giving me a redirect and HTTPError
<dobey> http://pastebin.ubuntu.com/486892/
<leonardr> dobey: do you know if this is a recent change, or did you just start using getSeries?
<dobey> i just started using it. i'm trying to get a valid series link so i can pass it in to the source_package_recipe.requestBuild() method, since it doesn't accept u'maverick' as valid
<leonardr> dobey: give me the code you use to invoke getSeries so i can see if the problem has been fixed in a more recent version
<dobey> leonardr: do you need the whole script, or just the launchpad bits after authentication?
<leonardr> dobey: the latter
<dobey> leonardr: http://pastebin.ubuntu.com/486896/
<gary_poster> adeuring: I believe the biggest difference between live QA and automated testing is because the WebserviceTestRequest has a different behavior in getRootURL than the real webservice request.  The real request has the behavior in line 1332 of c/l/w/servers.py, while the test request does not include that method for some reason .
<gary_poster> http://pastebin.ubuntu.com/486893/ gets closer (sorry, it includes your changes because I did this in a not very smart way).  See lib/canonical/launchpad/webapp/servers.py for the most important change, and line 135 .  Can I toss this to you for a bit more digging on your side?
<leonardr> dobey: can you tell me the value of self.series?
<dobey> 'maverick'
<leonardr> dobey: ok, i can reproduce it. will you file a bug against lazr.restfulclient? it might be a problem in httplib2 but we'll start there
<dobey> leonardr: does distribution.series[u'maverick'] give the (ideal) same result?
<leonardr> dobey: well, [x for x in distribution.series if x.name=='maverick'] will work
<leonardr> but the [] operator isn't supported on arbitrary collections right now
<dobey> ok
<lifeless> bryceh: don't put the [] stuff in the launchpad commit message
<lifeless> bryceh: ec2land does it for you
<bryceh> lifeless, why not?  does it hurt anything?
<bryceh> lifeless, from WorkingWithDbDevel it seems to suggest including it in the commit message
<dobey> leonardr: bug #628267
<_mup_> Bug #628267: distribution.getSeries() raises HTTPError on redirect <lazr.restfulclient:New> <https://launchpad.net/bugs/628267>
<cr3> lifeless: did you notice my question above? personally, I'm leaning towards django, not by preference but to keep things lean for now
<lifeless> bryceh: yes, it hurts
<lifeless> bryceh: you'll have them twice
<lifeless> bryceh: they have to be in what pqm receives, which is -not- what goes in the lp commit message; its what ec2 land outputs for you
<lifeless> cr3: uhm, I don't know. Maybe neither?
<cr3> lifeless: what did you have in mind?
<bryceh> lifeless, sheesh
<lifeless> cr3: trolling you successfully
<lifeless> bryceh: please do improve the wiki page to prevent other folk having the same misunderstanding
<benji> cr3: if you feel like being experimental as well as lean, you might like bobo: http://bobo.digicool.com/
<lifeless> rockstar: ^ also tarmac will need to know the logic ec2land uses to do commit messages/approval recording etc; presumably via a plugin (and as folk set this from the commandline, I guess we need extra data when the branch is set to 'Queued'
<lifeless> cr3: our best of breed set is (djanog, zope);
<lifeless> cr3: frankly I think there is a bit of a hole in the middle; djano gives too little support, zope ---to much---
<rockstar> lifeless, maybe I'm missing context there, but that sentence didn't make a lot of sense to me.
<lifeless> rockstar: to use tarmac to land launchpad branches, preserving the [foo=bar] in the commit messages which is used for QA workflow, we'll need to make sure that the 'ec2 land' process still handshakes properly with the thing doing the commits.
<cr3> lifeless: agreed, I'll work towards that
<lifeless> cr3: either django or zope would make sense to me here I guess
<rockstar> lifeless, oh, yeah.  I kinda see what you're saying.  I suspect Tarmac does most of that already (you can get it to format your commit message for you)
<lifeless> rockstar: the tricky bit will be things like ec2 land --incr
<lifeless> rockstar: and --no-qa
<lifeless> rockstar: we may need to change how ec2 land does it for pqm, in advance, if we want to make migration easy
<cr3> lifeless: a combination of both works too, manage.py here, storm there and interfaces everywhere. something like that
<dobey> lifeless: what does ec2land do for approval recording?
<rockstar> lifeless, hm.  I think we just teach ec2 how to set a commit message then.
<lifeless> dobey: generates the [r=foo] prefixes launchpad use by convention (and that PQM checks against a regex)
<lifeless> rockstar: something :)
<lifeless> rockstar: just alerting you is all
<rockstar> lifeless, when we move to Tarmac, we won't want to be setting "Approved" on the branches, and deferring that instead to having ec2 do it on a successful test run.
<lifeless> rockstar: I can't quite parse the nested clauses there.
<lifeless> rockstar: can you rephrase?
<dobey> lifeless: ah. so, tarmac automatically harvests the list of reviewers and shoves them in a revision property on the new revision, called 'reviewers' :)
<rockstar> Tarmac is pretty dumb (by design).  We could have it grow some brains, but I think having our tools know how Tarmac likes it would be better.
<rockstar> dobey, yeah, kinda like what we already do with the commit message formatter.  :)
<lifeless> rockstar: pqm is too :P
<bryceh> does it only generate [r=xxx] or does it also include the [bug=nnn] and [ui=none] stuff?
<lifeless> bryceh: it generates it all
 * bryceh grumbles about crufty wiki pages
<rockstar> bryceh, allenap was the last to touch the Tarmac commit message formatter, but I think it'll do bug= as well.
<bryceh> this is like the 3rd thing WorkingWithDbDevel has misled
<bryceh> er, me on
<lifeless> lets try my oops branch again
<bryceh> I've fixed WorkingWithReviews - it was mentioning that you shouldn't use [r=xx] but only in a footnotey way
<dobey> well, we've moved away from the [r=] bit in Ubuntu One stuff, for branches we're landing with tarmac, since it stores the metadata in revision properties
<lifeless> probably we want a dedicated page for landing-stuff
<lifeless> dobey: sure; the main thing is toolchain friction - need to update a bunch of analysis tools, and in the past folk have said they really like it being visible in trunk with just 'bzr log'
<dobey> yeah, the bugs are currently the only thing visible there
<dobey> but we could (should) fix bzr to make it much easier to see additional revision properties
<adeuring1> gary_poster: thanks!
<lifeless> bryceh: ah so the key thing is 'do not use -s'
<lifeless> I'll make that edit
<lifeless> bryceh: also you don't need to add options, you just target your branch to db-devel and it all Just Works.
<bryceh> lifeless, yeah you'll notice I corrected that already
<lifeless> bryceh: I just used bigger scissors
<bryceh> as a db-devel newbie I found that page full of snakes
<gary_poster> adeuring1: welcome. :-) please feel free to toss it back after you've tackled it a bit more.
<lifeless> I can imagine
<lifeless> bryceh: anyhow, I've just deleted all the stale info around the ectest/land bits
<bryceh> er actually DatabaseSchemaChangesProcess is the one full of snakes
<lifeless> the wiki is full of schnaaaaake
<bryceh> describes a cases that are actually special cases the ordinary newbie doesn't actually need to care about
<bryceh> e.g. all the emphasis about doing sampledata updates, security.cfg changes, etc.
<bryceh> lifeless, yeah that looks better
<lifeless> bryceh: partly this is cruft we've accumulated/rtolerated
<lifeless> bryceh: you do need to do security.py very often
<lifeless> fwiw
<bryceh> lifeless, in my case I did it but it wasn't needed.  The problem with the docs is it is vague about indicating *when* it should/shouldn't be done (perhaps a link to an appropriate doc about what the file is/does?)
<lifeless> good idea :)
<lifeless> the README in the schema is probably appropriate
<lifeless> its visible on bazaar.lp.net/...
<lifeless> bac: is the review meeting today?
<lifeless> bac: if so, how many hours away
<bac> 4:40
<lifeless> thanks
<bac> 0Z
<bryceh> is https://staging.launchpad.net/ supposed to say 'Code Update In Progress' right now?
<lifeless> yes
<lifeless> usually does about this time
<lifeless> however, unless a losa restarted it, its bust
<lifeless> losa ping
<mbarnett> hello lifeless
<lifeless> hi mbarnett
<lifeless> so staging failed overnight
<lifeless> (my overnight)
<lifeless> slony again
<lifeless> stub isn't around for 6-7 hours
<mbarnett> lifeless: i am pretty sure tom kicked it off again a couple horus ago
<mbarnett> let me check
<lifeless> staging is down right now; do we have any way to bring it up or kill-db-clients-harder and start it again.
<lifeless> I saw chatted in lp-code 2 hours back
<lifeless> chatter
<mbarnett> lifeless: hmm, it looks like a db error that we are actually waiting on stub for.
<lifeless> mbarnett: the slony timeout
<lifeless> mbarnett: thats what I said above, isn't it ?
<lifeless> or is it something differnet?
<mbarnett> the slony timeout seems to be the current offender
<lifeless> righto
<lifeless> My understanding, totally limited, is that connected clients break slony.
<lifeless> that *looks* to my limited understanding to be that happening.
<deryck> lifeless, hi.  Some timeout work questions, if you don't mind.
<lifeless> shoot
<lifeless> I'm all eyes
<deryck> so I see 2-3 bugs that all seem to come back to adding notifications.  and I'm trying small steps to improving then breaking that out....
<deryck> trying to just get rid of a flush to start with :-)  See:  http://pastebin.ubuntu.com/486931/
<lifeless> looking
<deryck> now obviously, this has the downside of doing severl inserts, compared to the original way
<lifeless> if its not in an inner loop it shouldn't be an issue
<deryck> ah, so that flush I removed is probably not an issue then?
<lifeless> I don't think it will be
<deryck> ah, ok
<lifeless> flushes have the following overhead:
<lifeless>  - a cache walk to detect dirty objects
<lifeless>  - DB update/insert calls per modified object to clean them up
<lifeless> a flush with no modified objects then has a cachewalk only overhead (costly, but not fatal)
<lifeless> a flush with many modified objects pending behind it is costly, but not because its a flush, because there is a lot of work to do.
<lifeless> note that storm, when you do a select, will often flush automatically anyway.
<deryck> ok, that makes sense and is useful to understand.
<lifeless> the previous single INSERT INTO is probably maximally efficient.
<deryck> right
<lifeless> whats the bug report that got you digging into this code ?
<deryck> and that was my other question, is there a storm way to get that single insert?  i.e. without manually building the sql statement?
<lifeless> lets have a quick poke at expr.py
<lifeless> (storm/expr.py)
<lifeless> see the class Insert
<lifeless> the docstring on it is fairly clear to me:
<deryck> ok, cool.  it wasn't to me.
<lifeless> hmm, Insert isn't good enough
<lifeless> looking more
<lifeless> ah yes it is, it uses itervalues() which confuses me
<lifeless> I wonder how it guarantees iteration order
<lifeless> deryck: https://bugs.edge.launchpad.net/storm/+bug/411556
<_mup_> Bug #411556: Storm should support multi-row inserts <Storm:New> <https://launchpad.net/bugs/411556>
<deryck> ah, looking at the bug....
<lifeless> deryck: so what bug/oopses are you looking at ?
<deryck> let me get numbers, I started with 2 or 3 and just poked and ended up where I was at.
<lifeless> deryck: yeah, I don't doubt that we have a few key inflection points for many issues
<deryck> bug 618403 and bug 611115 and something else that led me here... maybe the one about addComment
<_mup_> Bug #618403: BugTask:+editstatus-page timing out in ~4% of requests <timeout> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/618403>
<lifeless> deryck: I'm just curious ;)
<_mup_> Bug #611115: timeout: bug notifications are calculated in-request <timeout> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/611115>
<lifeless> hah, I recognise the chap that filed those.
<lifeless> terrible fellow
<deryck> heh
<lifeless> lets see if we can get an oops for the first one
<deryck> yeah, todays edge OOPS report had that one
<lifeless> did it, looking
<deryck> trying to get on myself.... slow browser
<deryck> https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1704ED939
<lifeless> so
<lifeless> I think this is fixed
<lifeless> ah no
<deryck> I think the actual timeout on max(bug.heat) is misleading.
<deryck> just before is the call to Bug.addChange, which many of these I looked at had in common
<lifeless> sec
<lifeless> just colating
<lifeless> ok
<lifeless> so I've put my normal stuff-from-an-oops in the tbug
<lifeless> what I've done is:
<lifeless>  - linked the url
<lifeless>  - the query parameters that are needed to make sense of whats happening (if not private)
<lifeless>  - the broad times
<lifeless> and the sql queries that really stand out
<lifeless> note the *300* email lookups - they will be hurting more than 1.6 seconds; I'd estimate 2 or more seconds
<lifeless> as for the 8 second heat lookup, lets see how it does on staging quickly
<lifeless> wheeee its slow
<lifeless> still waiting
<lifeless> Time: 14447.014 ms
<deryck> yeah, i guess it is the max heat select.  I could have sworn the ones I looked at today that was misleading.
<lifeless> deryck: I don't think the 8 second heat lookup is misleading at all :)
<deryck> right :-)
<lifeless> now, its faster the second time, but I could swear we had this the other day
<lifeless> lpmain_staging=> select bug.heat from bug, bugtask where bugtask.bug = bug.id and bugtask.distribution = 1 order by bug.heat desc limit 1;
<lifeless>  heat
<lifeless> -------
<lifeless>  11062
<lifeless> (1 row)
<lifeless> Time: 58.710 ms
<lifeless> that should fix it nicely.
<deryck> nice
<deryck> easy peasy then :-)
<lifeless> should be yes
<deryck> except it makes me nervous.
<lifeless> how so ?
<deryck> I remember us moving to limit 1 before for speed, but then someone changed back to max() saying it was faster.
<deryck> so now limit 1 is suddenly faster again?
<lifeless> so this is db query plan hell
<deryck> yup
<lifeless> lets check whether this exact query went through that transition
<lifeless> annotate time!
<deryck> heh, blame FTW!
<lifeless> bugtarget.py seems to be the file
<deryck> yes
<lifeless> 7675.778.1 is relevant
<lifeless> 10580 perhaps
<lifeless> and 10428
<lifeless> running with -p now
<lifeless> yeah, 06-25 it was switched
<lifeless> r=me in fact
<lifeless> bug 615644
<_mup_> Bug #615644: BugTask:+distrotask timeout on HEAT lookup <qa-ok> <timeout> <Launchpad Bugs:Fix Released by gmb> <https://launchpad.net/bugs/615644>
<lifeless> interestingly
<lifeless> the patch didn't completely change it one way or the other
<deryck> right, I thought I recalled this.
<lifeless> it left projectgroups alone, or they were the current way already or something
<deryck> yeah, in was inconsistent in the first iteration.
<deryck> I imagine oversight, not design :-)
<lifeless> naturally
<lifeless> if we used a query builder for this
<lifeless> rather than complete literals, it would mean only the builder needs changing
<lifeless> rather than multiple similar clauses
<lifeless> ok
<lifeless> so I've looked in the whole history
<lifeless> yes, this has flip flopped
<lifeless> lets look at that old bug
<lifeless> and see what context it was searching on
<lifeless> it may be that distros want one, and products another
<lifeless> ok, so it was a different case stub analysed
<lifeless> and it still runs fast
<deryck> still runs fast meaning the limit runs as fast for the case stub tried as for the case we're at now?
<deryck> rather than the max he suggested?
<lifeless> the current code runs fast for the case that bug 615644 examined
<_mup_> Bug #615644: BugTask:+distrotask timeout on HEAT lookup <qa-ok> <timeout> <Launchpad Bugs:Fix Released by gmb> <https://launchpad.net/bugs/615644>
<lifeless> gmb, quite reasonably changed the other similar queries at the same time.
<lifeless> sadly it looks like the different queries go through the planner differently
<lifeless> here:
<lifeless> lpmain_staging=> SELECT max(Bug.heat) FROM Bug, BugTask, DistroSeries
<lifeless> lpmain_staging-> WHERE BugTask.bug = Bug.id
<lifeless> lpmain_staging->     AND BugTask.distroseries = DistroSeries.id
<lifeless> lpmain_staging->     AND DistroSeries.distribution=3;
<lifeless>  max
<lifeless> -----
<lifeless> (1 row)
<lifeless> Time: 58.259 ms
<lifeless> this is likely related to how many bugs are on distribution 3
<lifeless> 3 is debian
<lifeless> 1 is ubuntu
<cody-somerville> OOPS-1705EB2156
<cody-somerville> hmm... I thought there was a bot that would give a link if I pasted an oops into the channel?
<lifeless> #lp
<lifeless> deryck: so there are two variations here:
<lifeless>  - different query with distro series
<lifeless>  - different distro
<lifeless> lets try changing 1 to 3
<deryck> yeah, I did.  longer but not horrible.  ~200 ms
<deryck> well, not horrible on the order of 8000 :-)
<lifeless> and the limit approach there is 10ms
<lifeless> lets try the limit approach for the qyery from the 06 bug
<lifeless> bingo thats slow
<lifeless> - no rows
<deryck> lifeless, which query is this?
<lifeless> I'm trying permutations
<deryck> I wonder if adding the condition heat not null makes it faster?
<lifeless> we have: MAX or LIMIT
<lifeless> we have distro 1 and distro 3
<lifeless> we have with distroseries and without
<cody-somerville> Does anyone know why calling setOwner on a branch via web API OOPses even though its successful?
<lifeless> cody-somerville: web API ?
<lifeless>   Module canonical.launchpad.webapp.publisher, line 691, in _handle_next_object
<lifeless>     raise NotFound(self.context, name)
<lifeless> cody-somerville: please file a bug
<lifeless> NotFound: Object: <lp.registry.model.personproduct.PersonProduct object at 0x2ab2b0e7dd90>, name: u'omsk'
<lifeless> deryck: no change for me adding not null there
<lifeless> with the MAX case
<deryck> lifeless, I meant for the limit case, sorry.
<lifeless> no
<lifeless> 5seconds
<deryck> hmmm
<lifeless> I'm going to fill in this table:
<lifeless> MAX/LIMIT 1/3 D/DS
<lifeless> if you're on staging, and wanted to do all the LIMIT ones
<lifeless> I'll do all the MAX ones
<lifeless> MAX/LIMIT 1/3 D/DS   TIME
<cody-somerville> lifeless, Filed bug #628352
<deryck> ah.... checking that distroseries is not null seems to speed up the original offending query using limit 1.
<_mup_> Bug #628352: Calling setOwner on a branch via web API OOPses even though operation is successful <oem-services> <Launchpad itself:New> <https://launchpad.net/bugs/628352>
<lifeless> MAX            1      D      2000ms
<lifeless> MAX            3      D      150ms
<lifeless> MAX            1      DS    196ms
<lifeless> MAX            3      DS    2ms
<lifeless> cody-somerville: I don't know what a 'web API' is
<lifeless> deryck: adding that in
<deryck> lifeless, the distro series not null condition?
<cody-somerville> lifeless, http://en.wikipedia.org/wiki/Web_API
<lifeless> deryck: can you paste the query, so I get it right
<lifeless> cody-somerville: ugh
<lifeless> cody-somerville: please say 'LP API'  and you will avoid all confusion
<lifeless> or just "API"
<lifeless> at least round these parts
<deryck> confirming query, lifeless ...
<deryck> lifeless, slow first, fast second:
<deryck> SELECT Bug.heat FROM Bug, Bugtask, DistroSeries
<deryck>     WHERE Bugtask.bug = Bug.id AND
<deryck>     Bugtask.distroseries = DistroSeries.id AND
<deryck>     DistroSeries.distribution = 3 ORDER BY Bug.heat
<deryck>     DESC LIMIT 1;
<deryck> SELECT Bug.heat FROM Bug, Bugtask, DistroSeries
<deryck>     WHERE Bugtask.bug = Bug.id AND
<deryck>     Bugtask.distroseries IS NOT NULL AND
<deryck>     Bugtask.distroseries = DistroSeries.id AND
<deryck>     DistroSeries.distribution = 3 ORDER BY Bug.heat
<deryck>     DESC LIMIT 1;
<lifeless> ugh
<lifeless> (because inner joins imply not null anyhow ..
<lifeless> doing the limit cases in the table
<deryck> lifeless, I really appreciate this help, but I need to leave now.  Passed EOD and kids are arriving home.
<deryck> Sorry if I distracted you from other work
<lifeless> deryck: this sort of stuff is my job :)
<lifeless> MAX/LIMIT 1/3 D/DS/DSNN  TIME
<lifeless> MAX            1      D                 2000ms
<lifeless> MAX            3      D                 150ms
<lifeless> MAX            1      DS               196ms
<lifeless> MAX            3      DS               2ms
<lifeless> LIMIT          3       DSNN         5000ms
<lifeless> LIMIT          3       DSNN         2ms
<lifeless> LIMIT          1       DSNN         200ms
<lifeless> LIMIT          1       DS             2ms
<lifeless> its missing two rows - LIMIT 'D'
<lifeless> adding them in a sec, I'll put this in the bug
<lifeless> catch you later
<deryck> ok, thanks!  I'll build on this work tomorrow, if you don't beat me to it :-)
<deryck> later on
<lifeless> anyone from registry still around ?
<jcsackett> lifeless, yes. what's up?
<lifeless> I had a question
<lifeless> uhm
<lifeless> ah yes
<lifeless> LoginToken:+accountmerge
<lifeless> is that on your kanban board for moving to a job ?
<lifeless> bug 104088
<_mup_> Bug #104088: Time-out problem at merging accounts <merge-deactivate> <timeout> <Launchpad Registry:Triaged> <https://launchpad.net/bugs/104088>
<jcsackett> lifeless, yeah it's on our OOPS backlog.
<lifeless> ok
<lifeless> cool
<lifeless> its the 3rd slowest page in the candidate-timeout-reports
<jcsackett> if you like i can reference it tomorrow at our standup; i'm wrapping up work on some stuff today, and may be able to take a look at it tomorrow.
<lifeless> well, its possibly longwinded; I was just cirious
<lifeless> whatever work item selection process you are following; keep following it :)
<lifeless> oh sodium
<lifeless> thou art the very esssence of a reactive metal
 * rockstar afks
<jcsackett> so if i have a testing situation that is coming up b/c of the way bugs are being cached breaking test isolation, is there a way in a test to clear out the cache?
<lifeless> what sort of cache
<lifeless> memcache? cachedproperty? storm cache?
<lifeless> jcsackett: ^
<cr3> lifeless: in your refactoring of launchpad, are you considering splitting the database into multiple databases per pillar for example?
<lifeless> cr3: yes to splitting the database, ENotDesigned to anything further
<lifeless> and splitting may mean ' one big db using a different tech'
<lifeless> there are all sorts of options
<cr3> lifeless: fun!
<lifeless> I need the breathing space to get into it though
<lifeless> get page performance up and the DB will have a lot more headroom
<lifeless> we can relax a bit and take time to fix stuff up
<jcsackett> lifeless: storm, i believe.
<jcsackett> i'm working off an unclaimed not entirely clear XXX.
<lifeless> gimme details
<lifeless> <- data monster
<jcsackett> lifeless: this is the test case: https://pastebin.canonical.com/36595/
<lifeless> ahha, I wrote this :)
<jcsackett> if i run that test, it's case, or the file it's in by itself, it passes.
<lifeless> ok
<jcsackett> if i run the bugs.tests module or more, it fails, with one more query than expected.
<jcsackett> this is true both locally and on ec2.
<lifeless> let me have a go
<lifeless> so there are two possibilities
<jcsackett> lifeless: i suspect it will pass for you. if i run this on devel there's no problem. i think, given the behavior, it's an isolation problem. but it's one that cropped up b/c of my changes.
 * jcsackett realizes that might not be what you meant by "have a go"
<lifeless> one is that the fuzz I permitting in the test is permitting it to pass when bug 619017 triggers, and that when it doesn't you've caussed a scaled bug
<_mup_> Bug #619017: __storm_loaded__ called on empty object <Storm:Fix Committed by therve> <https://launchpad.net/bugs/619017>
<lifeless> the other possibility is that as you say its purely storm caches getting in the way
<lifeless> so, to reset the storm cache
<lifeless> you can do IMasterStore(Bug).flush(); IMasterStore(Bug).reset()
<lifeless> or something like that
<lifeless> at the top
<lifeless> test_testcase (I think) does this
<lifeless> its a bit of a heavy hammer, but likely needed
<lifeless> 0.18 should fix this, but 0.18 has a different showstopper bug in it
<lifeless> jcsackett: what I'd like you to do though, is to compare the two queries
<lifeless> drop the 24 to 20 or so
<lifeless> and run it in the it-fails manner
<lifeless> that will get you a query list
<lifeless> shove that in a buffer somewhere
<lifeless> and compare it to the query list you get when it fails without modifications
<lifeless> jcsackett: by have a go I meant run bugs.tests
<jcsackett> i can do that, lifeless.
<jcsackett> probably won't have a comparison for a while though.
<lifeless> jcsackett: ok, well I'm going to pop out for a chore or two
<jcsackett> lifeless: i'll be EODing shortly. i'll ping you tomorrow if i have anything informative.
<lifeless> I'll be delighted to help figure out whats going on when I get back. I suspect a diff will be reasonably helpful though.
<lifeless> jcsackett: ok.
<jcsackett> thanks for the help. :-)
<jcsackett> (and who knows, i may still be poking at this)
<lifeless> if it is an isolation issue, we'll definitely want to fix it systematically - I'm adding *lots* of these tests.
<lifeless> and I hope others will too.
<lifeless> its a great way to prevent scaling surprises
<jcsackett> lifeless, i hear that.
<lifeless> one place in particular that may be causing the queries
<lifeless> is Person._init
<lifeless> losa ping
<lifeless> lpoops is telling me
<lifeless> Exception Value:	
<lifeless> [Errno 13] Permission denied: '/x/launchpad.net-logs/production/mizuho/2010-09-01'
<lifeless> on https://lp-oops.canonical.com/oops.py/?oopsid=1705A2026
<mbarnett> let me take a look
<mbarnett> hmm, that's a bad path
 * mbarnett lies
<mbarnett> well, it is.  there is no mizuho directory there
<mbarnett> wow, i am failing this evening
<mbarnett> there is.  checking permissions.
 * lifeless sniggers
<mbarnett> try again
<mbarnett> lifeless: that time i got that it didn't match any oopses..
<lifeless> mbarnett: I got OSError at /oops.py/
<lifeless> [Errno 2] No such file or directory: 'lp_publish'
<mbarnett> hmm, reloading it got me that as well.
<mbarnett> [Wed Sep 01 23:18:25 2010] [error] [client 122.63.10.108] Unauthorized access attempt for 'https://lp-oops.canonical.com/oops.py/?oopsid=1705A2026' by 'https://login.ubuntu.com/+id/kPbPBDC' (teams: [])
<mbarnett> looks like there might be a bug in the lp-oops auth code...
<lifeless> hah
<lifeless> would not surprise me
<mbarnett> i wonder who would handle that..
<lifeless> matsubara, but hes a) afk and b) swamped with merge workflow stuff
<mbarnett> heh, yeah.  i pinged him anywhoo.
<lifeless> I saw :P
 * mbarnett goes to file a bug 
<lifeless> thanks
<lifeless> ok, bbs
<thumper> lifeless: how do I profile a page?
<thumper> lifeless: the new cloud is in place
<thumper> lifeless: but the queries still seem a little slow to me
<thumper> but I don't know which bit is slow
#launchpad-dev 2010-09-02
<lifeless> atm
<lifeless> losa pin
<spm> lifeless: heya
<lifeless> ^
<lifeless> thumper: is it on staging ?
<thumper> lifeless: staging is down
<lifeless> thumper: well, spmdo works miracles
<lifeless> anyhow
<lifeless> the answer is:
<lifeless>  - to profile
<lifeless>  a) get it on staging
<lifeless>  b) losa ping and ask them to enable profiling. hit the page. ask them to disable
<lifeless> and the profile ends up on devpad 3 minutes later
<lifeless> thumper: if its on edge, do you have a user-oops to look at perhaps ?
<thumper> no, but I can create one with ++oops++
<thumper> :)
<lifeless> right, I'd do that - until we have a staging-daily environment its the best you can do until it has percolated to staging
 * thumper nods
<thumper> wgrant: ping
<wgrant> thumper: Hi.
<thumper> wgrant: hi
<thumper> wgrant: IBuildFarmBuildJob says that build can't be None
<thumper> does that fit with your understanding?
<thumper> because the database doesn't have that constraint
<wgrant> thumper: Indeed it doesn't. That's a bug.
<wgrant> But those tables are scheduled for removal as soon as Translations gets their job fixed.
<thumper> ah...
<thumper> what?
<wgrant> Hm?
<thumper> I know there was a migration from some tables to some other tables
<thumper> but I thought that this is the one we migrated to not from
<thumper> because the recipe stuff is using it
<wgrant> The migration is half done.
<thumper> (at least on production)
<thumper> was the recipe stuff landed this cycle?
<wgrant> The remainder cannot be done until Translations does the first half.
<thumper> which might be on db-devel only
<wgrant> Last cycle.
<wgrant> Production has the current build farm schema.
<thumper> if this is a table that needs to go, and recipe stuff should not be using it, and it should have landed last cycle
<thumper> why are we getting oops 1705EB2401
<wgrant> Recipe stuff is using it.
<thumper> but it shouldn't be?
<wgrant> It is part of the old queueing model.
<wgrant> SPRBs and BPBs have been ported to the new one.
<wgrant> But they still use the old one too.
<wgrant> Since TTBJs use the old one.
<lifeless> rockstar: btw, I'd really like us to keep the approval means 'human says ok' and use Queued for tarmac.
<wgrant> Once TTBJs use the new one, we can quickly stop using and remove the old one.
<lifeless> rockstar: really really really like us to do that.
<wgrant> thumper: So, the table is still in use, but is redundant.
<thumper> wgrant: do you know if the TTBJs have been moved?
<wgrant> They haven't.
<wgrant> I was hoping it would happen this cycle.
<wgrant> But apparently not.
 * thumper adds reminder to poke danilos
<lifeless> wgrant: whats DistroSeries:+templates all about ?
<lifeless> reminder to self; nag stub about PPR reliability, staging.
<wgrant> lifeless: Isn't that Translations?
<lifeless> ah, that would make sense
<lifeless> 29.25 - 99% completion time
<wgrant> How is that possible?
<wgrant> How can the 99% be above the timeout?
<lifeless> the magic of science
<lifeless> easily
<mwhudson> hm
<mwhudson> does someone remember where that dead easy graph generator was?
<mwhudson> where you could graph any number on any website?
<poolie> webnumbr?
<mwhudson> poolie: yes, thanks
<lifeless> wgrant: around ?
<lifeless> wgrant: were you going to poke at https://bugs.edge.launchpad.net/soyuz/+bug/618372 or is it my imagination ?
<_mup_> Bug #618372: Distribution:+search slow 50% of requests <timeout> <Soyuz:Triaged> <https://launchpad.net/bugs/618372>
<wgrant> lifeless: I don't think I was doing anything about that.
<wgrant> I did look and cringe at the code and views a few weeks ago, but that was for other reasons.
<lifeless> wgrant: care to be enticed ?
<wgrant> I have four projects due tomorrow... so no, sorry.
<lifeless> wgrant: meep. get on them :)
<wgrant> Heh, maybe.
<lifeless> wgrant: Did you see this one - 19997.01launchpad-main-masterSELECT COUNT(*) FROM Bug WHERE Bug.private = FALSE
<lifeless> bah
<lifeless> thats 1 9997
<lifeless> on https://api.staging.launchpad.net/1.0/bugs?assignee=xxx
<wgrant> Hah.
<wgrant> Awesome.
<wgrant> Any idea what's causing that?
<lifeless> not yet.
<lifeless> but I LoLed
<lifeless> wgrant: also, https://dev.launchpad.net/ArchitectureGuide may be interesting for you; as always I seek feedback on these things.
<wgrant> Hmm.
<lifeless> wgrant: as I say in the presentation, the metrics are noddy
<lifeless> but until we try, we won't have any
<wgrant> Launchpad's never really done the whole metric thing.
<wgrant> So it's a good start.
<lifeless> wgrant: so, distro:+search
<lifeless> it was the top timeout yesterday
<wgrant> Hm, that's good, considering how many other timeouts there used to be.
<lifeless> 57 /   13  Distribution:+search
<lifeless> 44 /  114  CodeImportSchedulerApplication:CodeImportSchedulerAPI
<lifeless> anyhow
<lifeless> you said it was terrible and should use <<>>>
<lifeless> did you have some advice for what it should use ?
<lifeless> wgrant: yes, we're getting the first rung of omg out of the way
<lifeless> wgrant: I'm nearly ready to lower the timeout again
<lifeless> in fact, I'll throw that up now
<wgrant> What are they sitting at now?
<lifeless> lpnet is 17
<lifeless> I'll leave edge for now
<lifeless> hmm edge had 29 timeouts
<lifeless> drop both
<wgrant> Yup.
<wgrant> It would be nice to see how much the two cache tables actually win us.
<wgrant> Because the query there isn't really using them.
<wgrant> I can't see what's generating that query.
<lifeless> I'll poke some
<lifeless> perhaps do-from-scratch
<wgrant> Oh, right, there.
<wgrant> Distribution.searchBinaryPackages, with exact_match=True.
<wgrant> At that point it's really not using the only beneficial part of the cache.
<wgrant> (binpkgnames)
<wgrant> IIRC that search is just about the only user of the tables... so if you can make it work without them we can delete lots.
<lifeless> thumper: ping
<lifeless> spm: ping
<spm> lifeless: heyo
<lifeless> hey
<lifeless> that config change didn't merge
<lifeless> or something
<lifeless> and I have more now
<lifeless> also the diff updater is slow?
<lifeless> by which I mean, please check it doth not need shooting.
<spm> yeah. :-) started looking then got an alert that calamansi has gone awol - just chsing that down atm. I've decied I'm going to relearn forth. the stack based thing is more in tune with my reality/
<lifeless> you could go the musical route
<lifeless> learn fifth
 * lifeless boom tishes
 * spm stares blankly at the screen and conceeds the point to lifeless
<lifeless> spm: I'm pretty sure the MP daemon is ded
<spm> 10 mins and 5 deep in interrupts. le sigh :-)
<lifeless> :<
<lifeless> are they all LP ?
<spm> alas no
<spm> yeah. looks wedged. about 30 mins. killing...
<spm> 20 mins. not 30.
<wgrant> lifeless: What are your performance targets?
<wgrant> For timeout and 99%.
<lifeless> wgrant: the same, and both were in the vision document ;)
<lifeless> wgrant: the immediate target is 5 seconds for 99% of requests
<lifeless> the long term one is 1 second, with hard timeout still at 5 seconds
<lifeless> (so right now they are the same, once we get below 5 seconds across the board it will be different)
<wgrant> lifeless: I didn't read the presentation.
<lifeless> wgrant: :P
<wgrant> Right, I was wanting the long term one.
<lifeless> wgrant: not even 2 months back ?
<wgrant> (I read the presentation just after the Epic)
<wgrant> Yeah.
<lifeless> yeah
<spm> lifeless: well you'll be pleased to know that it was either you or maxb that caused this problem <== science-less accusation
<lifeless> spm: ^^
<spm> but those mp's should now be processed :-)
<lifeless> thanks
<lifeless> spm: ok, https://code.edge.launchpad.net/~lifeless/lp-production-configs/timeouts/+merge/34042 is updated; can you please (I know you're busy) eyeball the updated diff, so i can start the 1 hour lead-in to make the change happen ?
<spm> sure sure
<spm> lifeless: +1'd
<spm> lifeless: actually - I assumed that we have already dropped to equivalent values on staging as a trial?
<lifeless> staging is set to 10 seconds
<lifeless> the data I am going off is the last few days of oops reports for lpnet and edge
<spm> sweet; re-confirming the +1. ta.
<lifeless> because not enough people hammer staging for it to mean anything
<spm> :-)
<spm> I have a selection of hammers here I'm happy to lend?
<spm> small sledge, ball pein, couple of reular claws; japanese chisel (my personal fave); few others....
<lifeless> rubber ducky, you're the one
<thumper> lifeless: pong
<lifeless> thumper: I want your +1 on lowering the timeouts
<lifeless> thumper: since the new policy seems to say noone can JFDI even with sysadmin agreement
<lifeless> thumper: the branch which will need to be deployed is running through now (its not an LP branch)
<thumper> lifeless: lowering which timeouts to what?
<lifeless> lpnet and edge hard timeouts, by 1 second each
<thumper> do it
<thumper> +1
<lifeless> thank you
<lifeless> spm: that is in pqm now, all going well in ~ 60 minutes I'll be asking for a deployment to all appservers.
<lifeless> spm: if that is convenient
 * spm consults the diary
<lifeless> spm: I can use the magic words if you want
<spm> "NOW"?
<lifeless> spm: NOW, Knave!
<spm> Knave. I like it.
<lifeless> but actually, NOW+60 :P
<spm> ${NOW}+60?
<lifeless> :>
<lifeless> what happened between 1500 and 1600 graph time yesterday ?
<lifeless> https://lpstats.canonical.com/graphs/OopsEdgeHourly/
<lifeless> spm: help
<lifeless> zope.configuration.xmlconfig.ZopeXMLConfigurationError: File "/home/pqm/pqm-workdir/home/---trunk/launchpad/script.zcml", line 7.4-7.35
<lifeless>    ZopeXMLConfigurationError: File "/home/pqm/pqm-workdir/home/---trunk/launchpad/lib/canonical/configure.zcml", line 80.4-86.10
<lifeless>    ImportError: No module named debian
<lifeless> I got that merging the config change
<lifeless> make: *** [lib/canonical/launchpad/apidoc/index.html] Error 1
<spm> blink
<lifeless> It means the pqm chroot's packages are out of date
 * spm holds head in hands and cries a little. on the inside at any rate.
<lifeless> spm: can you smack or arrange a smack, of apt-get update; apt-get-upgrade in it ?
<spm> yah, one sec
<lifeless> spm: hi
<lifeless> spm: https://bugs.edge.launchpad.net/soyuz/+bug/618372/comments/6
<_mup_> Bug #618372: Distribution:+search slow 50% of requests <timeout> <Soyuz:Triaged> <https://launchpad.net/bugs/618372>
<lifeless> can you please rnu the query in that link on a prod slave
<lifeless> I've checked staging already, its 10x slower than stub found
<lifeless> I just need the \timing for it
<lifeless> db tuning: funrollloops for adults
<lifeless> wgrant: whould would searchBinaryPackages use the dspc ?
<lifeless> wgrant: also Distribution.searchBinaryPackages interface docstring lies!
<wgrant> lifeless: Why wouldn't it?
<wgrant> Ah.
<wgrant> It returns DSPCs, rather than DSPs?
<lifeless> yes
<lifeless> so its not as simple as 'make a DSP returning function'
<spm> lifeless: ew. fun.
<lifeless> spm: pweese
<lifeless> wgrant: distroarchseries searchBinaryPackages doesn't, it does something different
<lifeless> wgrant: I don't understand why they have or need different code
<lifeless> wgrant: though you may hate das - it uses BPPH, or is that ok ?
<wgrant> lifeless: Well, it might be that simple if you change the callsites to not be insane.
<wgrant> "may hate das"?
<wgrant> Oh.
<wgrant> DAS
<wgrant> Right.
<wgrant> lifeless: If DAS's implementation is fast, it's fine.
<lifeless> its not on the top candidates page
<lifeless> I'll shove it into D
<wgrant> DSPC is basically just a precalculated version of that join.
<spm> lifeless: prod1. Time: 173.932 ms
<lifeless> spm: grahfuck
<spm> repeats are 50ms...
<lifeless> spm: what pg version is staging running ?
<spm> 8.3
<lifeless> ok
<lifeless> so there is something really quite different there ><
<lifeless> spm: that was the one in my comment, comment 6 ?
<spm> yup
<lifeless> thanks, appreciated.
 * lifeless subscribes stub again
<lifeless> wgrant: also, DAS has no code reuse at all. sob.
<mwhudson> lifeless: is it worse that foo.specifications ?
<mwhudson> *than
<lifeless> yes
<lifeless> specs at least has a mix in I could pull code into
<mwhudson> impressive
<wgrant> DAS was stuck between Soyuz and Registry for a while.
<wgrant> It's not well-loved.
<lifeless> it appears positively hated
<lifeless> but fast (or unused)
<lifeless> https://edge.launchpad.net/ubuntu/+search?text=mplayer is still boom on prod
<lifeless> oh and thats interesting
<lifeless> distroseries search is totally differnet
<lifeless> \o/
 * lifeless needs a drink
<lifeless> wgrant: is that a bug?
<lifeless> anyhow, there would be a different if I drop DSPC - rather than one row per source package it would be N rows - one per binary
<lifeless> so, I'm going to assume thats actually the desired win for now
<lifeless> and try and ask bigjools about this later
<wgrant> lifeless: Can't you just get distinct DSPs?
<lifeless> wgrant: *think*
<wgrant> lifeless: Hm?
<lifeless> wgrant: oh, hmm.
<lifeless> yes, should be doable
<lifeless> DAS.sBP returns BinaryPackageRelease,
<lifeless> wgrant: you asked about 99% stuff and timeouts before
<lifeless> wgrant: I forgot to answer
<lifeless> the time at which 99% of requests complete can be over the timeout if either:
<lifeless>  - enough requests fail (e.g. 2%)
<lifeless>  - the requests are completing with high durations as soft time outs due to bugs in the timeout trapping code (or no queries happening after the timeout)
<lifeless> stub: hi!
<lifeless> stub: got a few minutes?
<wgrant> lifeless: I don't see how it's possible unless the timeout stuff is buggy.
<wgrant> Since nothing can finish after the timeout -- it would have timed out.
<lifeless> wgrant: the timeout code works by checking for a timeout at predefined points.
<lifeless> currently that predefined point is 'sql queries within publication'
<stub> yo
<wgrant> lifeless: Ah, so an SQL query won't be terminated?
<lifeless> wgrant: it will, because we tell the server the remaining time
<wgrant> Ah.
<lifeless> wgrant: but there are caveats - pg doesn't cancel queries until you get the locks yoiu're waiting on
<lifeless> wgrant: and if stop querying and go into python, or librarian, or email time, you won't time out.
<lifeless> these are bugs. they will be fixed.
<wgrant> Ew.
<lifeless> stub: so, I have a bug you looked at that puzzles me on staging; https://bugs.edge.launchpad.net/soyuz/+bug/618372 and...
<_mup_> Bug #618372: Distribution:+search slow 50% of requests <dba> <timeout> <Soyuz:Triaged> <https://launchpad.net/bugs/618372>
<lifeless> stub: PPR is failing very often - it seems to run sodium into swap, which triggers a meltdown
<lifeless> and
<lifeless> stub: staging is failing waiting for a slony timeout, so its now not updated since sunday.
<lifeless> stub: I have no  idea how these slot into your priorities
<stub> I got the PPR email. I haven't seen it go that big. We can fix that but it will be slower generating reports (how much? Dunno until it is done)
<lifeless> wgrant: so its not hugely ew; unless we raise a signal right on the timeout its always going to be a little fuzzy
<stub> Staging probably means the slon daemons died. Restarting the process should sort things, but I was going to have a look now.
<lifeless> wgrant: (and doing that has *serious* hair on it)
<lifeless> stub: its died the same way three times in a row
<lifeless> wgrant: so if its approximate, even without massive-overruns, all you need is 2% timing out and the 99% point will be outside the timeout.
<stub> ok. no idea what happened then - the email I had wasn't enough to diagnose.
<lifeless> stub: I say this because I've seen the same pastebin contents +- the list of exact what is attached, on tuesd morn, wed morn & today :)
<stub> That is odd, because the only log on the system is from the pastebin I got emailed yesterday.
<stub> So the restore has been attempted once and it failed, or the logs are disappearing or this was from manual runs
<lifeless> is https://pastebin.canonical.com/36551/ what you got yesteday ?
<lifeless> that happened overnight from my perspective
<stub> oic. This is a code only update, and the slon daemons have died for whatever reason so it can't apply new patches to the existing database.
<stub> So the code only update needs to bouce the replication daemons to ensure they are running
<lifeless> jtv: bug 618393 - you say it would be hard to do for individual translators - could you expand why? it might be like the one I did for Bug.userCanView (which grants EDIT)
<_mup_> Bug #618393: TranslationGroup:+index slow 1-2% of requests timing out <timeout> <Launchpad Translations:Triaged> <https://launchpad.net/bugs/618393>
<lifeless> brb
<stub> spm: /srv/staging.launchpad.net/staging/launchpad/database/replication; LPCONFIG=staging ./slon_ctl.py stop; LPCONFIG=staging ./slon_ctl.py start
<stub> spm: I've run that so the code update should work now.
<spm> stub: context? that needs to be in the staging restore script?
<spm> oh right. ta. I'll rm the lock
<stub> spm: So we must not have a nagios check on the staging replication lag
<spm> cat /etc/nagios/nrpe.cfg :-)
<spm> nope
<stub> spm: Do you know if there is movement on the PG 8.4 on staging request?
<spm> stub: that's stuck on me atm, and no is the short answer; I plan on going into an irc hole of /ignore * tomorrow to make it happen - get the packages ready as in.
<stub> Ta
<spm> ... not helped by codebounce wanting all the swap it can get it's grubby mitts on
<stub> spm, lifeless: One of the things that is improved every PG release is the query planner. Some of the issues we are looking at the moment might just evaporate.
<spm> ^^^ boundless optimism on display from the DBA
<spm> (Oooo! it rhymes!)
<stub> If I can keep 'that should be fixed in the next release' up until retirement, I'll be sweet.
<spm> stub + sweet. the mind rebels.
<adeuring> good morning
<lifeless> spm: have you EOD'd ?
<lifeless> I'm still getting    ZopeXMLConfigurationError: File "/home/pqm/pqm-workdir/home/---trunk/launchpad/lib/canonical/configure.zcml", line 80.4-86.10
<lifeless>    ImportError: No module named debian
<lifeless> landing the config change
<lifeless> losa ping ^
<wgrant> '/home/---trunk/'
<wgrant> ---?
<lifeless> pqm glue
<wgrant> Yay.
<lifeless> wgrant: do you recall the cause of that error ?
<lifeless> wgrant: its stale package versions right ?
<lifeless> mthaddon: ping
<mthaddon> one sec - in a meeting
<lifeless> mthaddon: I'll write here, take it up when you get free :)
<lifeless> the above error is occuring trying to land a change to launchpad-production-config; AIUI landing those configs does a full buildout, so the remaining cause is a stale apt package for (IIRC) python-apt, or something like that.
<lifeless> There should be new packages in CAT; spm had updated this earlier today, we thought.
<wgrant> lifeless: Or stale sourcedeps.
<lifeless> I would deeply deeply appreciate it if we can fix this so that the production-configs change can land
<wgrant> Hm, actually, python-debian is a package now.
<wgrant> So, yeah, old version of the package.
<lifeless> so it may be that PQM is pulling in an old sourcedep
<lifeless> e.g. if for some reason its using a config-manager config rather than buildout
<lifeless> (I suspect it may be)
<lifeless> then, that config may be stale and need to be updated to use the python-debian package rather than the sourcedep.
<spm> lifeless: sorta - was on the weekly with tom
<lifeless> I will pop back later to offer what assistance I can
<lifeless> spm: mthaddon: sorry for interrupting the meeting
<mthaddon> so what update to launchpad-dependencies do we think will fix this?
<spm> oh right. so we updated the packages in the chroot for pqm on prasÃ© that should have pulled in the latest hotness of everything
<lifeless> so, its not the packages
<lifeless> its probably the manner in which it sets up 'sourcecode' ?
<spm> mthaddon: https://pastebin.canonical.com/36602/ is the list of what was updated
<lifeless> I don't see python-debian there
<mthaddon> (pqm-hardy)pqm@praseodymium:~$ dpkg -l | grep python-apt
<mthaddon> ii  python-apt                        0.7.4ubuntu7.5                                             Python interface to libapt-pkg
<mthaddon> er...
<lifeless> no
<lifeless> python-debian
<mthaddon> sorry, wrong package
<mthaddon> ii  python-debian                     0.1.9                                                      python modules to work with Debian-related d
<lifeless> whats that launchpad-dependencies package version ?
<mthaddon> 0.72~0.IS.8.04
<lifeless> current is 0.81
<mthaddon> that's not been installed on any of the servers yet
<lifeless> but nothing in the changelog looks relevant
<lifeless> and if this works on the servers its clearly not the issue
<lifeless> mthaddon: can you pastebin me the pqm stanza for production-configs ?
<spm> hrm. that bb amis rt I'm working on may be relevant to that version....
<mthaddon> yeah, just finding it now
<spm> hrm. no. 72 should be fine.
<lifeless> wgrant: can you check your sourcecode dir
<lifeless> tell me, do you have a python-debian subdir ?
<wgrant> lifeless: I don'
<wgrant> t
<lifeless> wgrant: ah, you're on a recent release
<lifeless> wgrant: look in lib, it has deb822 and debian symlinks
<wgrant> lifeless: Yeah, I'm not sure they're used any more
<lifeless> they are new
<lifeless> rev 11324
<wgrant> Ah.
<lifeless> mthaddon: spm: Thanks for the help.
<mthaddon> sure
<lifeless> stub: when is pg8.4 realistically happening; I don't want to wait another cycle to fix the batch of issues.
<lifeless> stub: but I could; I *really* don't want to wait another 2 cycles.
<stub> lifeless: It will take 1 week after we are happy with staging - swap over one box per day.
<wgrant> stub: Rather than waiting for the query planner to magically be fixed, shouldn't we be reporting problems?
<stub> lifeless: It is on a separate cycle to rollouts
<spm> lifeless: np
<lifeless> stub: ok, thats good to know.
<stub> wgrant: I haven't actually seen problems with the query planner. I have seen issues where it is not smart enough.
<lifeless> wgrant: if the planner isn't smart enough in 8.4 they would reasonably say 'try 9'
<lifeless> wgrant: we should look at bringing up a trunk pg box to do such evaluations on
<lifeless> but not just yet
<wgrant> stub: Well, one could argue that a completely dumb planner is just not smart enough.
<lifeless> ok, pqm is now trying that -again-
<lifeless> we'll know in ~ an hour
<stub> wgrant: Yes. But it isn't like upstream is doing nothing - I mentioned PG 8.4 before *because* of the insane speed upstream moves at.
<stub> wgrant: Its been rare to trip over a PG issue that hasn't already been fixed.
<wgrant> stub: Ah, I see.
<lifeless> ... twitter: When you click on these links from Twitter.com or a Twitter application, Twitter will log that click. We hope to use this data to provide better and more relevant content to you over time. ...
<lifeless> not evil. no, not at all.
<stub> I think the issue with GIN indexes is one that hasn't been dealt with that will help us
<stub> And there is an open bug on that.
<wgrant> lifeless: Doesn't just about everybody do that now?
<wgrant> Google, Facebook...
<stub> lifeless: Which is pretty stupid, as every twitter client I have seen goes through a URL shortening service.
<wgrant> noodles775: Bug 628427 looks relevant.
<_mup_> Bug #628427: oops on /builders - LocationError(SourcePackageRecipeBuildJob, 'build') - BuilderSet:+index <oops> <recipe> <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/628427>
<wgrant> (morning)
 * noodles775 looks. Hi :)
<lifeless> stub: they are going to unwrap it
<lifeless> stub: store it, and rewrap it, AFAICT
<noodles775> wgrant: Indeed, I've updated 628239.
<stub> lifeless: Assuming URL shortening services play ball.
<lifeless> stub: indeed, but they just announced this so I assume their ducks are lined up
<stub> I guess difficult not too
<stub> I heard some time ago they were planning this?
<lifeless> yeah
<lifeless> I mean they've done their 'mail all users' anouncement
<stub> I recall discussion on using this to bypass character limits, so http://someservice.com/lets/have/our/tweet/in/a/url would... erm... I forget why.
<wgrant> noodles775: The expandy bit for PPA source packages lists binaries (without links), then later links to all of the files. Is there any particular reason that it doesn't instead have "(i386) (amd64) (lpia)" links next to each binary package?
<wgrant> It would save space and probably be more understandable.
<allenap> bigjools: Do you know much about lp.archivepublisher, specifically its test suite?
<bigjools> allenap: lots.  But I am OTP, I can talk in 30m
<allenap> bigjools: Cool, thanks :)
<bigjools> allenap: or you can grab one of my esteemed team
<allenap> bigjools: Sure, anyone in particular for lp.ap? My questions are about test layers and possible weird interactions with zope.component.
<allenap> jelmer: Do you have time to help me with a problem in lp.archiveuploader?
<jelmer> allenap, sure
<allenap> jelmer: In 4 tests of test_pool.TestPool, I get "Could not adapt" errors for some new cachedproperty code I've done.
<allenap> jelmer: http://pastebin.ubuntu.com/487147/
<wgrant> It's a plain TestCase.
<wgrant> It has no layer.
<wgrant> So no Zope.
<allenap> wgrant: Yeah, but the adapters are also registered with the global site manager when propertycache is imported.
<wgrant> allenap: Is there a global site manager?
<allenap> wgrant: Always.
<allenap> wgrant: Well, zope.component.getGlobalSiteManager() will always return something :)
<wgrant> Hmm.
<allenap> wgrant, jelmer: One oddity is that these tests run fine locally. On EC2 they fail. I think there's something weird going on when the full suite is run.
<lifeless> allenap: you could consider nuking the zope aspect of this
<lifeless> allenap: I don't think it would make any difference to the spelling
<allenap> lifeless: Of propertycache? Yeah, that's on my mind :-/ I'd like to try and understand why it's not working if I can.
<allenap> lifeless: Yeah, agreed. It's just very frustrating and confusing :)
<jelmer> allenap: I'm at a loss so far, my thought would also be that it's related to layers but then it shouldn't work locally either...
<allenap> jelmer: Yeah, bloody odd. Okay, I'll email the list to see if anyone has inspiration, and remove the Zope-ish bits so this can land. Thank you :)
<wgrant> Graaaaah.
<wgrant> archiveuploader, your lack of test coverage and corresponding missed brokenness distress me.
<wgrant> The relevant test elides the important bit.
<wgrant> Gah.
<allenap> wgrant: Gah indeed :)
<lifeless> wgrant: 'gahk'
<lifeless> \o/
<wgrant> lifeless: That too, if it's the sound of choking on doctests.
<lifeless> https://bugs.edge.launchpad.net/ubuntu/+bug/1 didn't time out
<_mup_> Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Invalid by compscibuntu-bugs> <EasyPeasy Overview:Invalid by ramvi> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <The Linux OS Project:In Progress> <OpenOffice:In Progress by lh-maviya> <Tabuntu:Invalid by tinarussell> <Tivion:Invalid by shakaran> <Tv-Player:New> <Ubuntu:In Progress by sabdfl> <ubuntu-express (Ubuntu):Invali
<lifeless> showing all 1327 pages did
<wgrant> Which pages?
<lifeless> comments
<lifeless> blah
<lifeless> I knew what I meant
<wgrant> Ah.
<wgrant> They should be paginated :(
<lifeless> lets see what the query count is down to
<lifeless> hmm
<lifeless> 1327 comments in the summary
<lifeless> comment 1350 is the end one shown
<lifeless> methinks there is a bug
<lifeless> anyhow, at least that is down to 408 queries :P
<lifeless> hmm, assignments still is right on the edge
<bigjools> allenap: hi, did someone answer your question?
<lifeless> At least 76 queries issued in 13.06 seconds
<allenap> bigjools: Yeah, jelmer helped out. It baffled him too, so I'm taking lifeless's advice, which is to remove the Zopeish bits for now.
<lifeless> ooh thats nice, bug index in 1.17 seconds now
<bigjools> allenap: I didn't really pay attention to what you were talking about, so that doesn't mean much to me :)
<lifeless> for a more regular bug
<allenap> lifeless: There may be "deleted" comments... I /think/ they're still counted so that urls are stable.
<lifeless> allenap: ahh
<lifeless> anyone else feel bug pages are a bit snappier today ?
<lifeless> on edge, since it deployed
<allenap> bigjools: I'll explain if you're interested? Part of my plan was also to write to the list so you can wait for that if you want.
<bigjools> allenap: oh ok I'll wait, don't explain twice
<allenap> bigjools: Cool :)
<lifeless> all of bug 1's comments can be retrieved in 2.5 seconds
<_mup_> Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Invalid by compscibuntu-bugs> <EasyPeasy Overview:Invalid by ramvi> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <The Linux OS Project:In Progress> <OpenOffice:In Progress by lh-maviya> <Tabuntu:Invalid by tinarussell> <Tivion:Invalid by shakaran> <Tv-Player:New> <Ubuntu:In Progress by sabdfl> <ubuntu-express (Ubuntu):Invali
<lifeless> wgrant: ^
<wgrant> lifeless: Wow!
<lifeless> wgrant: theres not reason to paginate, we should be able to spit it out easily
<lifeless> SQL time: 2550 ms
<lifeless> Non-sql time: 14016 ms
<lifeless> Total time: 16566 ms
<lifeless> Statement Count: 408
<wgrant> Impressive.
<lifeless> with some more tuning
<lifeless> I reckon we can take that under 2 seconds sql
<lifeless> next step is a profile on staging, which will be tomorrow - need the patch in db-stable
<lifeless> it also does some daft stuff - grabs all the official bug tags for all related things
<lifeless> rather than joining and getting the relevant ones
<lifeless> thats trivial, but still linear vs log
<lifeless> mthaddon: we run the default threads per appserver right?
<lifeless> mthaddon: what would be in the impact on you & deployment if I asked for a change to 1 thread per appserver instance and 4 times the appserver instances.
<lifeless> mthaddon: same hardware.
<mthaddon> lifeless: it'd be a real maintenance headache (4 times as many initscripts), but it'd be doable
<lifeless> mthaddon: there is mounting evidence that we're having cross thread interaction
<lifeless> not bugs, just python being python
<mthaddon> you're saying python can't do threads?
<lifeless> python can't do threads
<lifeless> well known fact
 * mthaddon gets a poster made up
<wgrant> Well, CPython can't do threads.
<lifeless> if we were primarily waiting on db, it wouldn't matter
<lifeless> but reality is we're getting that sorted fairly well
<noodles775> wgrant: I'm not sure I'm visualising what you mean (re. the build/package links on the package details expander)
<lifeless> i expect db load to go down over the next few months
<wgrant> noodles775: I want to remove the binaries from the file list, replacing them with links in the 'Built packages' section.
<lifeless> wgrant: yes, thats true. But its what everyone means when they don't say 'python the specification'
<lifeless> except perhaps 4 or 5 people
<wgrant> Heh.
<elmo> lifeless: we benched this in montreal as part of the splitit project, and certainly at the time, there was no evidence that we were being hurt by gil contention
<wgrant> That will hopefully change soon.
<lifeless> elmo: I didn't say anything about gil contention
<elmo> while, I'm sure a lot's changed since then, this is not the kind of change I want to run in without some evidence not only that it won't hurt but that it'll help
<lifeless> elmo: I'm talkin about gil serialisation which is the primary gil issue
<elmo> lifeless: then s/contention/problems with python and threads/
<lifeless> elmo: naturally. I'd want an oops-per-instance report; then try one server with the different config and see if the oops rate changes more than due to the splitting of the instances
<lifeless> plus PPR stats of the same
<elmo> what I mean is, specifically, we messed around with number of app servers, number of threads per app server quite a lot
<lifeless> elmo: what application
<lifeless> elmo: because if it was the SSO the results are meaningless for this
<elmo> it was SSO and bugs
<lifeless> bugs?
<elmo> bugs are problems in software, but that's not important right now
<lifeless> ok
<elmo> lifeless: no, seriously, I mean malone/bugs.launchpad.net - whatever you want to call it
<lifeless> so the reason the SSO stuff isn't relevant is that it isn't pushing 15K responses around
<lifeless> elmo: I didn't know you benched malone
<lifeless> elmo: do you have the data -raw or massaged- for me to look at?
<lifeless> elmo: when a request takes 5 seconds to render, most of that time is pure bytecode interpretation
<lifeless> which is primarily serialised
<lifeless> so a 7 second request with 2 seconds db and 5 seconds render [unloaded]
<lifeless> means, effectively, ~4 seconds during which the other threads can do nothing - but its not in one big block because the GIL gets released
<elmo> lifeless: I'm sorry, i don't have any of the data to hand - what there is may be on the wiki if you search for 'splitit'
<elmo> right
<elmo> so wait, how is this not about contention then?
<lifeless> if the timeout is near the requests actual render time then, two of these at once will time each other out.
<elmo> i.e. what are you trying to solve by having one app server?
<lifeless> parallelism
<lifeless> responsiveness more precisely
<elmo> so, the problem with all of this is that app servers are not exactly slim and svelte
<elmo>  8056 launchpa  20   0  814m 510m 9512 S   19  8.5 729:46.99 /usr/bin/python -S bin/run -i lpnet7
<lifeless> elmo: much of the footprint is per-thread, in theory.
<elmo> we can't actually have 16 of those
<lifeless> elmo: of course not.
<elmo> well we'd want 16?
<lifeless> if you run up a single thread appserver and put it under load, you can see how big it will be
<lifeless> we don't want 16 of *those*, we want N of *those/N* where N is the current threadcount (4 for launchpad)
<lifeless> they won't drop quite linearly in size
<lifeless> elmo: https://wiki.canonical.com/TaskForce/SplitIt/PerformanceSprint doesn't talk about bugs at all, only shipit and sso :(
<lifeless> elmo: a key finding: but the average response time increase as the number of concurrent users increase.
<noodles775> wgrant: I don't see how you could move all the "Package files" (ie. 3 per build) next to the one built package? (can you scribble on a screenshot? it might make this easier. I've confidence that what you want will make sense ;) ).
<lifeless> elmo: thats exactly what I'm worried is strongly affecting LP
<wgrant> noodles775: My though was to stick '(i386) (amd64) (lpia)' links next to each package name.
<noodles775> wgrant: OK, what about when the package files are no longer published (ie. we'd want the builds to still be linked)?
<lifeless> elmo: ok thanks I read the wrapup
<lifeless> elmo: what I read correlates with the theory I'm putting forward
<lifeless> elmo: if I'm right, even halving the thread count and doubling the appserver count would be an improvement
<lifeless> elmo: there is no suggestion, in the benchmarks, that you tried 1 or 2 threads per appserver.
<wgrant> noodles775: http://williamgrant.id.au/f/1/2010/listing-archive-extra.png
<wgrant> noodles775: They're not the build links.
<noodles775> wgrant: Yes, I find that more intuitive (thinking as a user after the binary package).
<wgrant> noodles775: Right. I'm looking at this since a friend complained how confusing it was.
<noodles775> wgrant: as a user, I'd want those links (to the debs) to be available from the main PPA page. There's certainly room for a column with the built debs (when available) at https://edge.launchpad.net/~wgrant/+archive/experimental
<wgrant> noodles775: Well, PPA +index needs a bit of a rethink.
<noodles775> wgrant: indeed... there was some thought put into it about 6 months ago, but never the time to actualise it :/
<wgrant> Can I change PPA key generation to always use 'Launchpad PPA for <person.displayname>'? At the moment it will be named after the display name of your first PPA and then shared with the rest, which is almost always not what you want.
<bigjools> +1
<lifeless> bigjools: briefly, before I crash
<lifeless> bigjools: distro/+search
<bigjools> yarp?
<lifeless> distro/series/+search
<lifeless> distro/series/arch/+search
<lifeless> all return different objects
<lifeless> the /arch/ one I can understand
<lifeless> distro/+search is performing very badly
<lifeless> near-top timeout
<lifeless> by volume, yesterday
<lifeless> I'm wondering if you want them harmonised, and if so which objet - DSP or DSPCache is appropriate (assuming that the performance is acceptable either way)
<bigjools> lifeless: I'd need to think about it.  I don't recall the use cases for /arch and /series and it might be useful to talk to Ubuntu folks first.
<wgrant> Do we log who uses pages?
<lifeless> bigjools: so the thing I really care about is DSP/DSPCache, and AFAICT they are meant to be approx equivalent in UI
<bigjools> lifeless: I think DSPCache was invented for performance reasons, if we can do without it I'd be ecstatic since the script that generates the cache only runs daily.
<lifeless> wgrant: effectively, yes.
<lifeless> bigjools: ok, I will have a fiddle on staging tomorrow
<lifeless> bigjools: also I have a favour to ask
<bigjools> lifeless: can I breathe yet? :)
<lifeless> bigjools: the favour is this: have a look at
<lifeless> https://lpstats.canonical.com/graphs/OopsEdgeHourly/
<lifeless> and
<lifeless> https://lpstats.canonical.com/graphs/OopsLpnetHourly
<lifeless> about 70 minutes after mthaddon finishes the timeout CP
<lifeless> I was going to monitor this myself, but an orthogonal issue prevented the CP until now.
<bigjools> what are you CPing?
<lifeless> if it has gone ballistic, roll it back
<lifeless> bigjools: 1 second timeout drop
<bigjools>  /o\
<bigjools> ok
<bigjools> on soft or hard or both?
<lifeless> hard
<lifeless> soft gets tweaked a little to match other reporting tools
<lifeless> lpnet and edge
<lifeless> I don't expect a disaster, or I wouldn't be doing it.
<lifeless> but if it is terrible, we have the tools to undo it rapidly.
<lifeless> mthaddon: can assist rolling back the timeouts *if* needed.
<lifeless> bigjools: are you up for this ?
<bigjools> lifeless: I'll monitor it for an hour or so then I have to go out
<bigjools> I can pass the baton to someone else
<lifeless> bigjools: oh, and as an expectation - 50 hard oopses an hour is tolerable
<lifeless> 200 hard oopses an hour - rollback
<bigjools> good data point, thanks
<lifeless> thank you!
 * lifeless crashes
<bigjools> sleep well
<wgrant> Night lifeless.
<deryck> Morning, all.
<allenap> Morning deryck :)
<jtv> gmb: I have a faint recollection that ProductWithLicenses was one of yours.  Is that right?
<gmb> jtv, No, mot I.
<gmb> *not
<jtv> I'll ask bzr then..
 * gmb ;unches
 * gmb learns to type
<jtv> ah, 'twas EdwinGrubbs
<jtv> EdwinGrubbs: very minor bug in ProductWithLicenses.
<wgrant> bigjools: http://bazaar.launchpad.net/~wgrant/launchpad/ppa-key-name/revision/11486
<bigjools> heh ""Not Celso Providelo""
<wgrant> bigjools: So, does it look reasonable?
<bigjools> I can't think of a reason why it should not land
<wgrant> OK, I'll propose it then.
<wgrant> Thanks.
<bigjools> cheers
 * bigjools -> lunch
<benji> wgrant: I got your script working in a virtualenv with the new launchpadlib; I did have to make some small changes: http://pastebin.ubuntu.com/487228/
<benji> I'm going to see if the changes I made indicate bugs in the latest launchpadlib or if they were intentional changes
<wgrant> Ah, great.
<benji> speed-wise, in unscientific testing I got no meaningful difference in total run time
<stub> Is sourcepackage one word or two? I keep forgetting what we prefer?
<wgrant> It's capitalised as two.
<wgrant> There have been a few debates around this.
<deryck> adeuring, almost done on your review.  Some questions, just to be clear, though....
<deryck> adeuring, there is no public api change here, right?
<adeuring> deryck: yes, the api did not change
<deryck> adeuring, and concerning the bits of the webservice test you added, the main reason for that test is to verify the app server is serving the librarian file?
<adeuring> deryck: right
<deryck> adeuring, so the bits that would fail if this were not true are the X-Powered-By header line?  The rest is just setup?
<adeuring> deryck: yes; thoguh the X-powered-by header is not really important; the point is that "websevice.get(...)" returns the file data
<adeuring> deryck: a corresponding test some lines up in that test connect to the Librarian in a completely different way to read a publci file
<deryck> gotcha
<deryck> adeuring, so what is the final webservice.patch for in the diff?
<adeuring> deryck: some other tests later in that file need a public bug
<deryck> adeuring, ah, because it's sample data based.  So I think we could expand the doc portion of this doctest to make this clear.
<adeuring> deryck: i'll add a note
<deryck> I do realize the webservice tests are kind of a weird page/doc test amalgam, but it wasn't obvious to me.
<deryck> adeuring, so lifeless concern about this was that by serving files from the app server, we will most certainly have timeout issues now with private attachments via the API?
<deryck> that was a question, i.e. is that ^^ correct?
<adeuring> deryck: it could be. But I am not 100% sure: IIRC, the timeout exceptions are raised somewhere "near" the storm layer -- but once we have the LFA ready, the request does not touch DB objects anymore
<deryck> adeuring, so this wasn't lifeless concern then?
<deryck> what was?
<adeuring> deryck: no, as i understood him, these timeouts were his concern
<adeuring> ..perhaps aside from additional load for the app servers
<deryck> adeuring, and you're suggesting that you're not sure you agree that timeouts will be an issue?
<adeuring> deryck: well, we should test that ;) Should be easy: We just need to upload a sufficiemtly large file to the staging librarian and then to download it again via a sufficiently slow connection
<deryck> adeuring, at any rate, we really don't have another option, short of the "enable retracers in dc" option that we cannot do, right?
<adeuring> right. and regarind testing. there is even this linux kernel feature to throttle connections, about which I always forget how to use it
<deryck> adeuring, ok.  I'm about ready to r=me this.  I asked gary_poster to take a look over my shoulder, just to make sure I'm not missing something.
<adeuring> deryck: ok, thanks!
<deryck> sorry for paranoia, but I want this to go in without further problems.
<deryck> adeuring, you could go ahead and ping losa to cowboy the patch to staging and performance test it, since that is a primary concern from lifeless.
<adeuring> deryck: ok, though I think testing it tomorrow should fine
<deryck> adeuring, well, ubuntu beta is today and they need the retracers running.  I'd like to end today telling them to point the retracers at edge and go for it, but I want to have confidence this will work.
<adeuring> deryck: ok
<adeuring> deryck: could you please add your review to to mp?
<deryck> adeuring, yes, sorry.
<adeuring> npmp ;)
<deryck> adeuring, you're ready to go now.  See my comments/reminder about the staging test.
<adeuring> deryck: thanks!
<deryck> np!
<abentley> rockstar, here are the permission changes that are giving me grief: http://pastebin.ubuntu.com/487320/
<abentley> rockstar, or lp:~abentley/launchpad/recipe-interfaces
* gary_poster changed the topic of #launchpad-dev to: Code hosting offline 8.00-9.30 UTC on Friday 3rd September for unexpected hardware maintenance. http://is.gd/eRMxF | Launchpad Development Channel | Performance Tuesday | Week 3 of 10.09 | PQM is OPEN | firefighting: - | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews
<jcsackett> lifeless, you around today yet?
<lifeless> jcsackett: hi
<jcsackett> lifeless: heya.
<jcsackett> so, EdwinGrubbs and I have looked into what was plaguing me yesterday. Edwin may have found a fix for the test: http://pastebin.ubuntu.com/487390/
<jcsackett> so, we may now be good. when i pinged you earlier, still no luck.
<jcsackett> i do have the one extra query isolated, if you're interested.
<lifeless> what is it
<lifeless> looking at your patch
<lifeless> the flush+reset would need an explanation at a minimum
<jcsackett> https://pastebin.canonical.com/36645/
<jcsackett> lifeless ^
<jcsackett> and yeah, this is without comments; just seeing if it passes in the full suite.
<lifeless> that looks like _init to me
<lifeless> what happens is that _init - a storm hook - is called *before* the variables are assigned
<lifeless> Person._init checks self.teamownerID
<lifeless> and storm does an on-demand query to populate the attributes
 * jcsackett nods.
<jcsackett> so this problem is already fix-committed, right?
<lifeless> the storm bug is yes
<lifeless> fixing it though, uncovered a nastier one
<lifeless> https://bugs.edge.launchpad.net/storm/+bug/620615
<_mup_> Bug #620615: please contact the developers <Storm:In Progress by niemeyer> <https://launchpad.net/bugs/620615>
<lifeless> bug 628762
<_mup_> Bug #628762: propertycache adaption failures in test suite <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/628762>
<lifeless> allenap: jfdi ;)
<lifeless> oops graphs are looking good
<jcsackett> lifeless: that does look unpleasant.
<lifeless> slight increase on prod, edge seems unaffected \o/
<lifeless> james_w`: ping
<james_w`> hi lifeless
<lifeless> hiya
<lifeless> james_w`: so, swallowing eh
<james_w`> yeah, I don't want to trigger exceptions
<lifeless> I was starting to reply
<lifeless> I figured I'd chat
<james_w`> usually means that something has gone wrong, and so I don't want my tests to pass
<lifeless> cleanUp is used from two locations
<lifeless> as a cleanUp
<lifeless> and from reset
<lifeless> now, cleanUp isn't nestable
<lifeless> I mean TestCase.addCleanUp -
<lifeless> cleanups are regular work-or-throw functions
<lifeless> they can't represent 2 exceptions
<lifeless> what you have to do is to have two cleanups
<lifeless> so, if TestCase.addCleanUp could in some way handle functions that want to blow up many times
<lifeless> we could change the fixture protocol to match
<lifeless> hmm
<james_w`> why not mandate that cleanups can be run multiple times?
<lifeless> lets say we make TestCase.addCleanUpExt
<lifeless> it takes things that yield exceptions
<lifeless> and has an adapter that is used by addCleanUp
<lifeless> to turn a regular function into a yields-exc-info
<lifeless> james_w`: because for existing trivial cleanups its undefined whether thats safe or not
<lifeless> james_w`: and for many, it won't be, or will raise the same way every time
<lifeless> james_w`: its also less general: it only helps with cleanUps that happen to be a loop-on-a-list
<lifeless> mmm, now what about reset
<lifeless> it will need to signal exceptions in the same way as cleanup
<lifeless> (whether thats raise a MultiException with a list of exc_infos, or return an iterable, I dunno yet)
<lifeless> it also needs to signal 'setUp failed'
<lifeless> probably any different exception would be sufficient for that - e.g. user caused exceptions are fine.
<lifeless> testresources optimisation should stop I guess if it sees reset fail, and skip all the tests with that fixture
<leonardr> lifeless, can you tell me if http://pastebin.ubuntu.com/487402/ is ec2 being flaky or a real problem? i can't duplicate it locally and it seems to have nothing to do with my branch
<lifeless> leonardr: line 18 is that storm bug I was discussing with jcsackett
<lifeless> Person._init being called to early
<leonardr> i see
<lifeless> the fingerprint is the id=%s  limit 1 call
<lifeless> james_w`: what do you think of my sketching ?
<sabdfl> hi folks
<james_w`> lifeless: sounds reasonable to me
<lifeless> hi sabdfl
<sabdfl> does the bug description deliberately specify a different font-face, or is that an bug?
<sabdfl> for example, https://bugs.edge.launchpad.net/indicator-network/+bug/621168
<_mup_> Bug #621168: Connection to encrypted hidden network fails <Connection Manager:Confirmed> <Network Menu:Confirmed> <https://launchpad.net/bugs/621168>
<sabdfl> new font is looking pretty good generally, mono version on the way
<lifeless> deryck: this ones for you I think?
<deryck> ah, yes.
<deryck> sabdfl, it's a bug.  It's falling back to the YUI css for some non-obvious reason.
<deryck> sinzui and I worked on this a bit already.
<beuno> deryck, the YUI CSS is more specific than the one in LP, so it wins
<sabdfl> aha. low importance, i was just curious, but please do file it so it doesn't get lost
<deryck> sabdfl, will do.
<lifeless> jcsackett: with your query count bug
<sabdfl> thanks deryck
<lifeless> jcsackett: please use
<lifeless> Equals(count), Equals(count + 1)
<lifeless> jcsackett: so that we don't have to simulaneously change the tests when we get storm 0.18
<lifeless> leonardr: changing the test you're hitting to be similar to the pastebin changes jcsackett pasted before - with a MatchesAny(Equals(n), Equals(n+1)) and a flush + reset should work around it.
<lifeless> leonardr: I will be very happy when storm 0.18 arrives ;)(
<deryck> beuno, thanks for the "more specific" pointer
<beuno> deryck, also, a different font may break the widget layout. You had worked on making them fluid rather than fixed, may of fixed the font-face problem
<beuno> deryck, we may want to make lazr-js be less specific about CSS, so it doesn't end up being a battle between the projects' CSS and the widget
<deryck> beuno, IIRC, the font had to be specified to make it re-usable.  Changing the font will force changes to the widget, whether it happens in lazr-js on lp's use of the widget.
<beuno> deryck, right. I was thinking of just specifying the font more generically on the DOM elements
<beuno> vaguely enough that it applies by default, but can still be easily overriden
<deryck> ah, so it does win in the specificity battle.
<beuno> right
<beuno> otherwise we end up having to slap !important on
<beuno> or have crazy CSS rules
<beuno> either way, LP still needs to catch-up with YUI 3.1.2
<beuno> so you will probably have to hack it in LP first
<beuno> this was more of a for-the-future thought
<deryck> right
<deryck> thanks, I appreciate it.
<deryck> and now we have bug 629063
<_mup_> Bug #629063: Description editing widget should use the Ubuntu font <javascript> <ui> <Launchpad Bazaar Integration:New> <LAZR Javascript Library:New> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/629063>
<leonardr> lifeless: do you want me to make that change in this branch, or is it ok to land my branch as is? ie. are you just telling me how to work around it for now?
<lifeless> your branch won't land because you're tickling the storm cache enough that you'll see that extra query. I don't know what your branch changes :)
<lifeless> so you need to make that change in your branch, so you can land it. I'll happily review the workaround for you
<leonardr> ok
<leonardr> lifeless: all right, see how http://pastebin.ubuntu.com/487469/ grabs you
<lifeless> leonardr: that should be enough, yes. give it a sping - with XXX around the flush and reset saying 'bug 619017' please
<_mup_> Bug #619017: __storm_loaded__ called on empty object <Storm:Fix Committed by therve> <https://launchpad.net/bugs/619017>
<bdmurray> Is there anyway to change an attribute in a zope macro?
<lifeless> bdmurray: what do you mean?
<bdmurray> lifeless: lib/lp/bugs/templates/bug-portlet-dupe-subscribers-content.pt uses subscriber-row from bug-portlet-subscribers-content and I want to chagne the title attribute in subscriber-row
<lifeless> hmmm
<lifeless> I don't tihnk so
<lifeless> it also feels weird to me
<lifeless> losa ping
<lifeless> https://staging.launchpad.net/successful-updates.txt  claims 9737
<lifeless> but the running instance is 9710
<lifeless> spm: when you start, I know you want to crawl into a hole and hide; I have two small things batched up so that they don't interupt.
<lifeless> spm: one is the staging thing above
<lifeless> spm: the other is can you please put the canonical losa tag whatever it is onto https://bugs.edge.launchpad.net/launchpad-foundations/+bug/629139
<_mup_> Bug #629139: cannot make production config changes <Launchpad Foundations:New> <https://launchpad.net/bugs/629139>
<Ursinha> jcsackett, hello
<Ursinha> jcsackett, just saw your comment on bug 623428 about the bug status change, I'm investigating
<_mup_> Bug #623428: Don't get ZODB 3.10 by installing Zope2.13a <Zope 2:Triaged> <https://launchpad.net/bugs/623428>
<Ursinha> hmmm
<Ursinha> bug 623408
<_mup_> Bug #623408: Offiical_* booleans must be deprecated in favor of usage enums <bridging-the-gap> <Launchpad Registry:In Progress by jcsackett> <https://launchpad.net/bugs/623408>
<Ursinha> there you go :)
<Ursinha> it's not supposed to change the bug status, just tag it as untestable
<Ursinha> because of the new merge workflow
<Ursinha> I'm taking care of it
<jcsackett> Ursinha: thanks.
<mbarnett> lifeless: spm: i have tagged that bug.
<lifeless> mbarnett: oh awesome, thanks
<lifeless> mbarnett: I figured you'd EOD when the ping fell into silence :)
<mbarnett> lifeless: we had an austin meetup this afternoon.. was in transit home
<lifeless> meetup?
<mbarnett> a bunch of canonical employees who live in the area worked from a coffee shop together for a couple hours.
<lifeless> nice
<mbarnett> yeah, i hadn't put a face on a couple people yet, so it was great.
<lifeless> I met the closest canonical staffer to me, in prague :P
<lifeless> (we both live in NZ)
#launchpad-dev 2010-09-03
<michaelh1> Speaking of me,
<michaelh1> is staging up at the moment?  The last entry on successful-updates is from four hours ago
<lifeless> michaelh1: see my question to spm just before the reference to you :P
<michaelh1> Thanks.
 * spm quickly unignores lifeless, reads, adds the ognore back and crawls into a depper hole.
<michaelh1> A different question: is there a way to edit and/or delete bug messages in LP?  If not, can I edit them through the API?
<lifeless> ognore eh?
<spm> and adds extra typos for maximal confusions
<lifeless> michaelh1: no
<lifeless> michaelh1: they can be hidden I believe, in extremis
<michaelh1> No to both the UI and the API?
<lifeless> and the DB
<spm> edited for super awesomely special cases as well; but *really* needs to be special.
<spm> wich is another way of saying I can probably count on one hand the numbers of times it's been done.
<michaelh1> spm: nope, I don't count for that.  We're using messages to add meta data to a bug.  The problem is when the meta data is wrong...  tags are inappropriate.
<wgrant> Why are tags inappropriate?
<michaelh1> wgrant: we're using tickets to also track the upstream status of a bzr revision
<michaelh1> wgrant: the revisions are fully qualified as in lp:gcc-linaro/4.5,revno=1234 and there are up to 20 entries per bug
<wgrant> michaelh1: You could keep them in the description.
<michaelh1> wgrant: I'd like to have a timestamp and author that is easy to review.  Descriptions not a bad idea though...
<lifeless> michaelh1: why are tags inappropriate?
<michaelh1> lifeless: see the reply to wgrant above
<lifeless> michaelh1: I asked this before and you linked to a wiki page which didn't help me understand your design constraints or needs at all
<lifeless> so, I'm open to the idea of building a generic lookaside store, but I want the use cases pinned down very carefully
<lifeless> since there are so many ways such a thing could go wrong
<michaelh1> lifeless: yip.  I'll try and describe it better over the next few weeks
<lifeless> michaelh1: some specifics - please look at https://dev.launchpad.net/ArchitectureGuide for my big picture concerns
<lifeless> michaelh1: and for this thing - do we need:
<lifeless>  - search facilities (e.g. metadata values, modifiers, modification times)
<lifeless>  - logs/journalling
<lifeless>  - display UI in web pages
<lifeless>  - would this be better served by specific integration-information e.g. 'patches to forward upstream'.
<poolie> gary_poster: you could link that lep from the relevant bug
<poolie> hi lifeless
<lifeless> hi poolie
<gary_poster> poolie, I'm trying to link all sub-bugs to LEP, but yeah, you are right, there is that big one, I forgot, thanks
<gary_poster> Currently fighting moin syntax though :-P
<lifeless> spm: so, in all seriousness - what is the status of staging
<spm> gah. sorry distracted. looking...
 * lifeless offers spm some eye for the eyeballs
<spm> it's building an importd atm; so "soon". I'd suggest counted in smller minutes. maybe 15-30?
<lifeless> gary_poster: oh you're still around
<lifeless> excellent. I need help.
<gary_poster> :-P
<lifeless> can you do voice? just so that I can have the code full screen
<gary_poster> I'm here because I didn't have time to get the stuff I needed to get done today, done
<lifeless> L<
<lifeless> :<
<gary_poster> :-)
<lifeless> Let me sketch my issue, and you can decide if mercy is appropriate
<lifeless> I have a patch, lp:~lifeless/launchpad/oops
<gary_poster> ok looking
<lifeless> which replaces the request_statements list with a Timeline object
<lifeless> its failing one test. xx-opstats.txt line 186
<gary_poster> and trunk doesn't, I assume?
<lifeless> right
<gary_poster> ok
<lifeless> I've thrown a bunch of print statements at timeout and soft timeout setting code
<lifeless> and patched SoftTimeoutView in an attempt to understand it
<lifeless> but I'm getting less clue not more
<lifeless> http://pastebin.com/xBnLuiPR has my 'wth' edits
<lifeless> One of the things confusing me is that AFAICT the test should *never have passed*
<lifeless> the page it requests does not DB access, and only DB access can set a hard timeout
<gary_poster> heh, one of those
<lifeless> I added DB access in, but that didn't work
<lifeless> setting the soft time out to 200ms seems to work
<lifeless> 20 doesn't reliably
<gary_poster> have to get boys out of bath, but back in a bit
<lifeless> I'd be ok with a 'does not blow up in the next devs face' tweak
<lifeless> kk
<lifeless> will rabbit on a bit
<lifeless> Ideally I'd actually fix it
<lifeless> one thing i'm sure is happening is this:
<lifeless>  the db code I addedfire, a timeout is raised
<lifeless> but the end request hook sees the request as having no oops id
<lifeless> and the oops code that puts the oops id into the request isn't running until *after* the end request hook has set a soft request oops id
<gary_poster> same request?
<lifeless> pretty sure
<gary_poster> fly by--still putting kids to bed
<lifeless> will check
<lifeless>     + I (<class 'storm.exceptions.TimeoutError'>, TimeoutError(), <traceback object at 0xf1a16c8>)
<lifeless>     + raising request= <canonical.launchpad.webapp.servers.LaunchpadBrowserRequest instance URL=http://localhost> 280053312 (<class 'storm.exceptions.TimeoutError'>, TimeoutError(), <traceback object at 0xf194b48>)
<lifeless>     + G
<lifeless>     +             UPDATE SessionData SET last_accessed = CURRENT_TIMESTAMP
<lifeless>     + end-hook-set-soft <canonical.launchpad.webapp.servers.LaunchpadBrowserRequest instance URL=http://localhost> 280053312
<lifeless>     + raising request= <canonical.launchpad.webapp.servers.LaunchpadBrowserRequest instance URL=http://localhost> 280053312 (<class 'canonical.launchpad.webapp.errorlog.SoftRequestTimeout'>, SoftRequestTimeout(None,), None)
<lifeless>     + save fool
<lifeless>     + raised-soft <canonical.launchpad.webapp.servers.LaunchpadBrowserRequest instance URL=http://localhost> 280053312
<lifeless> save fool is where the oops is set on the reuqest
<lifeless> so - same request
<lifeless> and raising is being called into
<thumper> hmm... lunch is caling
<lifeless> so _makeErrorReport is barfing
<lifeless> and its silently swalloed
<lifeless> gary_poster: thanks
<gary_poster> lifeless, oh solved
<gary_poster> awesome
<lifeless> gary_poster: well not solved
<lifeless> but progress
<gary_poster> I had actually built a branch and was proceeding to start a test
<gary_poster> great
<lifeless> gary_poster: its failing to call sys.exc_info() ><
<gary_poster> heh
<lifeless> s/call/print the exc_info/
<lifeless> I suspect a securityProxy stabbing me
<lifeless> or something
<lifeless>     + 'NoneType' object has no attribute 'microseconds'
<lifeless> *blink*
<lifeless> gary_poster: is there something that one can raise, which *will* escape the publication machinery and give a sensible diagnostic in pagetests?
<gary_poster> do not understand "escape the publication machinery"
<gary_poster> (IOW, the goal)
<gary_poster> lifeless ^^^ (and I need to get off the computer in 4 min for mental health :-) )
<lifeless> gary_poster: I was thinking of ways to make this easier to debug in future
<lifeless> gary_poster: thank you very much for helping me escape my own mental trap
<gary_poster> heh, I'm not sure what I did to help, but I'm glad I did
<lifeless> gary_poster: I think I'm good now. May brainstorm later about how this might have been easier to diagnose.
<lifeless> gary_poster: you were patient; asked good questions, and sympathised.
<lifeless> combination let me take enough of a step back to apply my own awesome skillz :>
<lifeless> now to find this ztrace log thing
<gary_poster> :-) cool.
<gary_poster> have a good day
<spm> lifeless: if you haven't noticed; staging lives1
<lifeless> spm: \i/
<lifeless> spm: erm
<lifeless> spm: something is wrong
<spm> oh?
<lifeless> is https://staging.launchpad.net/successful-updates.txt meant to match with the revision in the footer of staging ?
<spm> shuold
<lifeless> spm: ...
<spm> so I see
<spm> is def revno 9738 in the live tree.
 * lifeless is skeptical
<spm> bzr-version-info.py is 9710 tho. how... curious
<spm> launchpad@asuka:/srv/staging.launchpad.net/staging/launchpad$ bzr revno
<spm> 9738
<spm> ^^ bug in bzr then? :-P
<lifeless> bug in the lp build thingymajig
<lifeless> file a bug with the details please? lp-foundations
<spm> wait a sec... something's not right.
<spm> https://pastebin.canonical.com/36696/ <== build date vs date
<spm> and the rev id for that matter
<lifeless> I agree
<lifeless> something is wrong :)
<lifeless> run generate_version_info
<lifeless> see what happens
<spm> heh, was kinda hoping that was "Oh That's *X*"
<lifeless> gary_poster: a) thanks again; its all sorted and I'm much happier. b) what happened to your mental health :)
<gary_poster> lifeless: (a) yay! (b) it appears to be hosed ;-)
<lifeless> gary_poster: ah, welcome to my world... we can be insane together :)
<gary_poster> lol :-)
<spm> um. find . -name 'generate_version*' <== nothing found. ????
<lifeless> generate_version_info
<lifeless> hmm
<lifeless> grepping
<lifeless> gary_poster: you may like knowing that the fix to my issue also means that every sql statement *attempted* on 'LaunchpadDatabase' will be logged.
<gary_poster> that sounds cool
<lifeless> changing the confusion that existed around attempted-but-not-executed
<gary_poster> right
<lifeless> for now I'm logging them with a duration of 999999
<gary_poster> heh
<gary_poster> that'll stand out, I hope
<lifeless> can dig into the oops stack to see if we can do '-' or something in future
<gary_poster> sure
<lifeless> spm: scripts/update-bzr-version-info.sh
<spm> *blink*
<spm> launchpad@asuka:/srv/staging.launchpad.net/staging/launchpad$ scripts/update-bzr-version-info.sh
<spm> Skipping bzr-version-info.py update; already at revno 9710
<lifeless> bzr 2.2
<lifeless> bet you its a bug
<lifeless> please file; launchpad-foundations + bzr tasks; losa tag, critical (it will really mess us up if that doesn't right itself)
<spm> bzr --version ==> Bazaar (bzr) 2.2.0
<thumper> wallyworld_: best tales docs I've found http://www.owlfish.com/software/simpleTAL/tal-guide.html
<thumper> lifeless: that's that all about?
<lifeless> spiv: can you please assist ^ >
<lifeless> ?
<lifeless> thumper: the thing that updates the revision in the bottom right of staging and edge isn't (updating)
<thumper> ah
<thumper> ok
<lifeless> which suggests, very strongly, a bzr bug
<poolie> this twitter oauth thing strongly reminds me of the "we must have unique per app tokens" launchpad used to have
<poolie> i think we've now seen the light, fortunately
<wgrant> For anonymous access, sure.
<poolie> yes
<poolie> this was proposed for a long time as a reason we couldn't allow anonymous access
<wgrant> Indeed.
<wgrant> But Twitter's situation is different, as it's authenticated.
<spm> lifeless: https://bugs.edge.launchpad.net/launchpad-foundations/+bug/629212
<_mup_> Bug #629212: staging update is showing incorrect version in the html/page footer <canonical-losa-lp> <Launchpad Foundations:New> <https://launchpad.net/bugs/629212>
<lifeless> we can have multiple apps per token, its just a config + trust thing
<lifeless> spiv: if you could look at bug 629212 that would be awesome
<_mup_> Bug #629212: staging update is showing incorrect version in the html/page footer <canonical-losa-lp> <Launchpad Foundations:New> <https://launchpad.net/bugs/629212>
<lifeless> spiv: I realise its an interupt; but we're hitting release-week on Monday, and thats already pretty harsh for lp :(
<poolie> lifeless: he's out atm (or he was)
<lifeless> poolie: oh sure, it can wait for later today
<lifeless> poolie: I just wanted to be clear about it all
<lifeless> ok, thats pretty impressive, I admit it: 2455  OOPS-1706L64    ProductSeries:+bugs
<lifeless> the first number is sql queries
<wgrant> You should have seen Soyuz a year or so ago.
<wgrant> It was better than that :)
<lifeless> wgrant: without timing out ?
<wgrant> lifeless: ... just.
<lifeless> foods time
<mwhudson> lifeless: well, that's why we have such beefy db servers :-)
<lifeless> mwhudson: ><
<lifeless> wtf
<lifeless> we still query AccountPassword?
<wgrant> Yup.
<wgrant> It's still needed for basic auth.
<wgrant> But I don't know why it's queried every time.
<lifeless> we don't support basic auth anymore do we? I mean, I know its *enabled*, but *support* is different
<wgrant> Right.
<wgrant> It's enabled and dangerous.
<wgrant> So should be deleted.
<lifeless> propose a patch
<lifeless> DoIt
<lifeless> make it controlled by a feature flag
<lifeless> if after the rollout folk are screaming
<lifeless> we can add a rule to enable it
<wgrant> Ooh, good idea.
<lifeless> and transition the remaining user(s)
<lifeless> if noone screams by 10.10, we nuke the code completely.
<wgrant> (and the table)
<lifeless> spm: ping
<spm> yo
<lifeless> spm: on staging
<lifeless> I have some evidence suggesting it *is running 9710*
<lifeless> spm: I'd like to dig deeper
<lifeless> here is my evident
<lifeless> my patch 11483 in stable preloads is_valid_person for https://staging.launchpad.net/ubuntu/+assignments
<stub> We still have AccountPassword to support the test OpenId provider the testsuite uses. Dropping it is part of the roadmap I drafted yesterday.
<lifeless> but
<lifeless> stub: awesome
<lifeless> spm: https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1707S19
<lifeless> shows 191 SELECT ValidPersonCache.id FROM ValidPersonCache WHERE ValidPersonCache.id = %s LIMIT 1 calls
<lifeless> spm: on edge, when I checked yesterday, the page didn't have that
<wgrant> stub: I guess it really just needs a place to enter an email address?
<wgrant> And maybe a button to fail auth, to make testing easier.
<stub> wgrant: Yes, and a box to override the OpenId Identity to test some edge cases.
<wgrant> Since that's difficult at the moment.
<lifeless> spm: db-stable 9738 is past 11483
<wgrant> Ah, true.
<lifeless> spm: so, in theory, it should not be issuing those queryies
<lifeless> spm: so I think that staging right now is still useless
<spm> mmmmmmmmmmm
<spm> frown evem
<lifeless> spm: "grep 'def _specification_sort' lib/lp/blueprints/model/specification.py"
<lifeless> spm: if that finds a function, lets cross check that whats showing on the web *is running from where you think its running*
<lifeless> spm: if it doesn't, then what we're seeing on the web is consistent with the source code you're looking at, and we can look at bzr etc
<spm>  was going to ask do you have a way to confirm existance... ta
<spm> lifeless: most curious. zero results found. ??
<lifeless> then we're not running 9738
<lifeless> ok
<spm> orsum
<spm> bzr missing; perhaps?
<lifeless> bzr revision-info --tree
<lifeless> bzr revision-info
<lifeless> run both please
<spm> oki
<lifeless> bug 629212
<_mup_> Bug #629212: staging update is showing incorrect version in the html/page footer <canonical-losa-lp> <Launchpad Foundations:New> <https://launchpad.net/bugs/629212>
<spm> wow
<spm> launchpad@asuka:/srv/staging.launchpad.net/staging/launchpad$ bzr revision-info --tree
<spm> 9710 launchpad@pqm.canonical.com-20100827184922-1f7jszjyrqw4dasv
<spm> launchpad@asuka:/srv/staging.launchpad.net/staging/launchpad$ bzr revision-info
<spm> 9738 launchpad@pqm.canonical.com-20100902161314-dy8vhr3l8epgpo2s
<lifeless> spm: ok, the tree is out of date
<lifeless> I'll update 629212 with that
<lifeless> spm: please run 'bzr update'
<lifeless> and make run, or however you kick things off
<spm> i wonder if this'll break staging rather spectacularly - the DB would have been set via 9710....
<lifeless> spm: almost certainly
<spm> as we just rsync the code around. whee.
<lifeless> well
<lifeless> where does this code start?
<lifeless> whats the source of the rsync
<lifeless> and if you say sodium, el crasho, I'm going to cry.
 * spm hates to see a grown man cry and says nothing
<lifeless> ok, so you probably just need to run 'bzr update' on sodium
<spm> launchpad@asuka:/srv/staging.launchpad.net/staging/launchpad$ bzr update
<spm> bzr: ERROR: This tree contains left-over files from a failed operation.
<spm>     Please examine /srv/staging.launchpad.net/staging/launchpad/.bzr/checkout/limbo to see if it contains any files you wish to
<spm>     keep, and delete it when you are done.
<lifeless> and kick the whole thing off again
 * spm puts head in hands and tears up a little
<lifeless> sodium will be like that too
<lifeless> lets fix at root
<spm> yeah, that's where I'm heading
<spm> damn. sodium is really futzed with that tree.
<lifeless> bug updated
<lifeless> whats the sodium rt ? I cannae see it
<spm> the h/w fail one? not sure if it even has one tbh
<lifeless> it had one
<lifeless> it may have been closed after the chassis was replaced.
<spm> have moved the existing db-stable to *.busticated; doing a brand new branch et al to be sure to be sure
<lifeless> thanks
<lifeless> sorry for the interrupt
<lifeless> spm: I've updated the bug and the bodyswap RT ticket
<lifeless> stub: sodium has less CPU and RAM than it did, its why the PPR may be running into swap when it wasn't before.
<spm> oh no, is fine; and yeah that sounds about right.
<lifeless> hah
<lifeless> rt fail
<lifeless> it mades a new ticket
<lifeless> 41200 ><
<lifeless> spiv: unping
<stub> lifeless: ahh
<lifeless> stub: I only found out reading closed rt tickets looking for the sodium thing
<lifeless> spm: it would be nice to share those things btw, just saying
<lifeless> (and I know it wasn't up to you :>)
<lifeless> http://ayende.com/Blog/archive/2010/08/31/it-really-happened-legacy-programmers-tales.aspx?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed:+AyendeRahien+(Ayende+%40+Rahien)
<stub> Is sourcepackage one word or two in the Launchpad vocabulary? I can never remember. I think I even asked this yesterday...
<lifeless> one I think
<stub> BinaryPackageBuild thinks it is two, pretty much everything else thinks its one. But BPB is new and perhaps the new way forward?
<wgrant> stub: Two.
<wgrant> What says it's one?
<lifeless> spm: Segmentation fault in cron still waorries me
<stub> wgrant: hyphenation of database column names... haven't looked for capitalisation in Python source code.
<stub> but hyphenation can be wonky from when we didn't underscore between words.
<wgrant> stub: Pretty much everything except BPB predates the hyphenation.
<wgrant> Exactly.
<stub> Ta.
<wgrant> It's been capitalised SourcePackage since like 2005.
<spm`> pqm@devpad:/code/rocketfuel-built/db-stable$ bzr revision-info --tree
<spm`> 9739 launchpad@pqm.canonical.com-20100902175034-3o81ksb8qmulx82x
<spm`> pqm@devpad:/code/rocketfuel-built/db-stable$ bzr revision-info
<spm`> 9739 launchpad@pqm.canonical.com-20100902175034-3o81ksb8qmulx82x
<spm`> lifeless: ^^
<lifeless> spm`: \o/
<lifeless> spm`: time for the acid test
<spm`> :-)
<thumper> :(
<thumper> 'lp.services.job.tests.test_runner.TestTwistedJobRunner.test_timeout' spurious test failure
<spm`> lifeless: right. timing worked against us a bit there; have stabbed the (still doing it wrong) restore;  manually ensureed we have the latest pony; and the next restore in .... 4 minutes, should push out 9739
<lifeless> spm`: if my fingers cross any harder they will break
<spm`> :-)
<lifeless> stub: did you remember where our new storm base base class is ?
<stub> lifeless: I don't think anyone has created it yet.
<lifeless> stub: is it ok to use storm.base.Storm ?
<lifeless> stub: on completely new things?
<lifeless> (TimeLimitedToken for the librarian )
<stub> I use storm.locals.Storm, but yet. Better to create a three line subclass in the LP tree though and use it. I don't think anyone knows where it should live though.
<stub> c/yet/yes/
<lifeless> stub: I want to get this branch in shape for Abel
<lifeless> if we're already using Storm directly, anywhere, we'll need to do a mass migration, so its no harder.
<stub> We already use Storm directly elsewhere.
<lifeless> thanks
<lifeless> time to rebuild my session db
<lifeless> stub: will you want a sql patch for the tokens
<lifeless> stub: or just grab the table defn I put in session.sql
<lifeless> spm`: staging mail came through, but staging is down.
<lifeless> spm`: is that normal, monsieur backtick ?
<spm`> yeah. I manually rekicked it.
<lifeless> spm`: just a minute ago ?
<spm`> about 10ish
<spm`> 15 I kicked; it should have restarted abot 10 ago
<spm`> and isn't. sigh.
<lifeless> spm`: I saw mail 6 minutes or so ago
<spm`> yeah. something funkies up
<stub> lifeless: I need to manually apply things, so no need for a database patch. Needs a note on 'unusual deployment' on the rollout page, because things will explode if I forget.
<stub> lifeless: Staging will explode too if it lands without work, so I guess I should make the changes to prod and staging just before you land.
<spm`> wowo. something really blew up.
<spm`> Fri Sep 3 04:22:22 UTC 2010 Applying database updates and permissions to DB
<spm`> Traceback (most recent call last):
<spm`>   File "./upgrade.py", line 13, in <module>
<spm`>     import _pythonpath # Sort PYTHONPATH
<spm`> ImportError: No module named _pythonpath
<spm`> stub: ^^ staging restore just now
<lifeless> buildout
<stub> Yay, it isn't me!
<spm`> although this looks bad too: urllib2.URLError: <urlopen error [Errno 2] No such file or directory: '/srv/staging.launchpad.net/staging/launchpad/download-cache/dist/setuptools-0.6c11-py2.6.egg'>
<spm`> ha
<lifeless> whats the staging librarian url ?
<lifeless>         ZopeXMLConfigurationError: File "/home/robertc/launchpad/lp-branches/working/lib/canonical/shipit/browser/configure.zcml", line 7.4-12.49
<lifeless>         ImportError: No module named cachedproperty
<lifeless> *hate hate hate stab stab stab*
<spm`> lifeless: librarian.staging.launchpad.net ? or did you mean something else...
<wgrant> lifeless: Split split split.
<wgrant> spm`: Er, really?
<wgrant> I really hope not.
<lifeless> spm`: the staging equivalent to launchpadlibrarian.net
<spm`> oh that one. right. staging.launchpadlibrarian.net
<wgrant> (phew)
<spm`> you may begin to see a pattern here... :-)
<lifeless> thumper: can you please put rt 41202 to pri 89
<thumper> lifeless: possibly, trying to login
<lifeless> where do shipit bugs get filed?
<lifeless> like 'we can't rollout until this is fixed' style shipit bugs
<wgrant> shipit
<wgrant> But they're normally fixed by LP...
<lifeless> yeah
<lifeless> I have a sketchy patch
<thumper> lifeless: done
<wgrant> But it can't be broken, can it?
<wgrant> ec2 tests it.
<wgrant> So does buildbot.
<lifeless> wgrant: guess what
<lifeless> wgrant: go on guess
<lifeless> wgrant: I dares you
<wgrant> My guess would be python version differences, but that doesn't seem relevant here.
<wgrant> Or buildbot wasn't update-sourcecoding.
<wgrant> Or prasÃ© isn't.
 * spm` notes the 'prasÃ©' and smiles :-)
<lifeless> wgrant: zcml barfs in includes that don't affect the core test suite don't break it.
<lifeless> wgrant: or something ~= to that
<wgrant> lifeless: O_o
<lifeless> wgrant: anyhow, I dunno.
<lifeless> maybe allenap cheated and didn't ec2land, in which case its going to go boom and we'll be in testfix in, oh, 2.5 hours
<thumper> ec2 giving WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!
<thumper> anyone else seeing this?
<thumper> is there a reason?
<lifeless> wgrant: https://bugs.edge.launchpad.net/shipit/+bug/629259 for your pain
<_mup_> Bug #629259: cachedproperty was imported by shipit <canonical-losa-lp> <ShipIt:Confirmed> <https://launchpad.net/bugs/629259>
<mwhudson> lifeless: have you run utilities/update-sourcecode?
<lifeless> mwhudson: I'm checking that now, but pqm barfed at me with the same error earlier
<mwhudson> ah
<lifeless> mwhudson: on a production config change
<mwhudson> oh there's probably some config-manager branch that needs to change
<mwhudson> maybe
<lifeless> yes
<lifeless> I suspect so now having checked
<lifeless> sourcecode has been udpated
<wgrant> thumper: That just means you're lucky.
<wgrant> thumper: You've got an instance reusing an old IP.
<mwhudson> the config-manager config is in the lp-production-configs branch iirc...
<thumper> :(
<lifeless> mwhudson: yes, had a lovely race condition there last night
<thumper> wgrant: three times in a row
<thumper> must be extra lucky*
<lifeless> actually its a bug
<lifeless> if you're using ec2 land/test
<mwhudson> thumper: rm ~/.ec2/known_hosts
<lifeless> that is meant to zap stuff and set it up for you
<wgrant> Ah, right.
<thumper> lifeless: using ec2 test
<mwhudson> ec2 test should probably use the ssh -RogerMeHarder options to not get it to check keys at all
<mwhudson> if it has them even
<lifeless> mwhudson: every now and then your uk heritage shines through ;)
<thumper> so I can fix this by removing the known_hosts
<thumper> ?
<mwhudson> thumper: yes
<mwhudson> until next time
<mwhudson> lifeless: >:)
 * thumper sighs
<lifeless> spm`: when is the next edge update ?
<lifeless> spm`: can we abort it ? tip of stable is not safe to rollout.
<lifeless> I really dislike this Robert Collins(bugnumnber)...
<lifeless> in emails
<wgrant> As in the fake bug email address?
<lifeless> yes
<wgrant> Yes :(
<lifeless> ok -> airport. bbiab
<spm`> lifeless: about one hour, and yes.
* spm` changed the topic of #launchpad-dev to: Code hosting offline 8.00-9.30 UTC on Friday 3rd September for unexpected hardware maintenance. http://is.gd/eRMxF | Launchpad Development Channel | Performance Tuesday | Week 3 of 10.09 | PQM is OPEN | firefighting: edge rollouts disabled | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews
 * StevenK tries to work out what Zope is telling him
<wgrant> StevenK: What's it complaining about?
<StevenK> wgrant: ComponentLookupError
 * StevenK grumbles at zcml
<lifeless> spm`: can we stop it ?
<spm`> lifeless: it is: https://wiki.canonical.com/InformationInfrastructure/OSA/LaunchpadProductionStatus#Edge%20Updates
<lifeless> spm`: thanks
<spm`> np.
<spm`> I should relaly restart bip to get rid of that '`'
<lifeless> I'll put the rollback into pqm in a few minutes
<spm> coolio
<lifeless> spm: hows staging?
<spm> still unwell I assume, haven't chased tbh.
<spm> lifeless: revno 9741 should be being rolled out in about 5 mins. (fingers crossed etc etc)
<spm> on staging, as in.
<lifeless> spm: great, thanks
<adeuring> good morning
<lifeless> spm: bugger
<lifeless> spm: can you stamp on pqm for me
<spm> wich part?
<lifeless> the job I just mistargeted to prod-devel
<spm> oops
<spm> lifeless: lucky you
<StevenK> Is there some way I can cleanup my lp-sourcedeps/eggs directory?
<spm> Commit message [[rs=deryck][ui=none] Rollback rev 11491.] does not match commit_re
<lifeless> spm: thanks
<lifeless> hopefully this one will get through
<lifeless> adeuring: I have a rollback landing
<adeuring> lifeless: I've seen your mails (but haven't had yet enough coffee)
<lifeless> adeuring: thats cool
<lifeless> adeuring: I"m starting to update bugs and stuff now that I'm back home from the airport
<lifeless> adeuring: simply letting you know so that we don't roll it back twice :P
<adeuring> lifeless: ok. so, how shall we proceed regarding the need to get the retracers working again?
<lifeless> the firewall port has been opened
<lifeless> adeuring: if lp is serving the restricted librarian urls out now, it should be working
<adeuring> lifeless: right
<lifeless> adeuring: have your coffee; catch up on mails
<adeuring> ;)
<lifeless> that will give me time to update bugs to make the plan as clear as I can
<spm> <adeuring> lifeless: I've seen your mails (but haven't had yet enough coffee) <== Man after my own heart. I too find dealing with lifeless and his emails to be much smoother post coffee. ;-)
<allenap> lifeless: Have I broken something?
<lifeless> allenap: I'm not sure
<allenap> :)
<lifeless> allenap: have you landed the shipit change?
<allenap> lifeless: Yeah, that landed over a week ago I think.
<lifeless> allenap: for cachedproperty use?
<stub> Gah. I'm always bitching about other people adding stdout noise to the test suite, and I just notice some from one of my recent landings :-P
<allenap> lifeless: Yep.
<lifeless> allenap: ah, but your lp branch only landed last night?
<allenap> lifeless: Yeah, but versions.cfg only got bumped in my branch too.
<lifeless> allenap: so landing a production config change barfed this morning on shipit zcml
<allenap> lifeless: Sorry, utilities/sourcedeps.conf
<lifeless> allenap: because that doesn't use versions.cfg, we have to land branches that cause incompatibilities in lockstep, not way apart.
<lifeless> allenap: it may be fixed now, I'll toss a test change at it.
<allenap> lifeless: Was shipit tip pulled into production?
<lifeless> allenap: yes, thats how the older stuff works
<allenap> lifeless: Gah, I didn't realise that could happen.
<lifeless> allenap: not live production, but the mechanism by which production config changes are made
<lifeless> allenap: its a hangover, it should be changed.
<allenap> lifeless: Okay. I'm sorry about that. I meant to land the two branches close to one another, then realised there were problems with the Launchpad part, and thought it would be okay - because of sourcedeps.conf - to leave shipit as is.
<lifeless> allenap: no worries, now I know what went on it makes sense. I'm checking now that there isn't a persisting problem
<wgrant> Why doesn't production use sourcedeps.conf?
<allenap> lifeless: Cool, thanks for sorting this out while I slept unaware :)
<lifeless> wgrant: pqm landing production-configs uses config-manager
<lifeless> allenap: I think your branch landing has sorted it
<wgrant> lifeless: Yes, but why?
<lifeless> wgrant: 20:30 < lifeless> allenap: its a hangover, it should be changed.
<wgrant> lifeless: Your clock is wrong.
<wgrant> But, ah.
<StevenK> wgrant: Pedant.
<lifeless> oh foo, I can't land stuff till codehosting is back.
 * lifeless files a bug for redundandcy
<StevenK> A redundant bug?
<jkakar> I think the diff on the merge proposal page would be more readable if it used a fixed-width font.
<wgrant> jkakar: It does.
<wgrant> jkakar: Are you using edge, and also don't have the UbuntuBeta font installed?
<jkakar> wgrant: I'm using edge and I have the UbuntuBeta font installed.
<wgrant> Hmm.
<jkakar> The diff is shown using the UbuntuBeta font.
<wgrant> Not for me :(
<wgrant> Maybe my UbuntuBeta is old.
<wgrant> Hm, no.
<lifeless> thumper filed a bug for this on launchpad-web today, I think.
<lifeless> or someone did.
<allenap> +1 for diffs in cursive.
<StevenK> Argh, no
<StevenK> allenap: Evil!
<wgrant> lifeless: There is a bug for it, yes.
<wgrant> Bug 629181
<_mup_> Bug #629181: code review diffs are in a proportional font on edge <launchpad-web:New> <https://launchpad.net/bugs/629181>
<wgrant> Hence the question about whether the font was installed.
<jkakar> allenap: p:first-letter { font-size: 200% }
<lifeless> wgrant: ah, it was you ;p
<jkakar> Might as well make the diff look like an old manuscript. :)
<allenap> jkakar: Oh, oh, that's so brilliant.
<wgrant> lifeless: It wasn't me.
<wgrant> I just saw it fly past.
<lifeless> oh, someone.
<StevenK> lifeless: mwhudson
<lifeless> ah yes thanks
<lifeless> adeuring: https://bugs.edge.launchpad.net/launchpad-foundations/+bug/395960
<_mup_> Bug #395960: proxying user supplied files via the launchpad appserver domain has security and performance issues <librarian> <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/395960>
<lifeless> adeuring: thats the bug we'll be fixing
<adeuring> right
<lifeless> adeuring: codehosting is back if you want to pull the private-librarian branch
 * adeuring is looking
<lifeless> adeuring: there are instructions on https://code.edge.launchpad.net/~lifeless/launchpad/private-librarian/+merge/31020 about getting going with it
<lifeless> adeuring: hey, so I've updated the bug and the mp and chased a few things around
<adeuring> lifeless: ok
<lifeless> adeuring: so I'm going to work on this monday, tuesday etc
<adeuring> cool
<lifeless> adeuring: if you'd like to collaborate that would be awesome
<lifeless> adeuring: I understand deryck is giving you a slab of time to do this, if you're interested.
<adeuring> lifeless: sure, I am
<lifeless> awesome
<lifeless> so, its heading towards late for me; I'd love to give you a brain scan to get my internal state on it
<lifeless> but failing that, perhaps you can poke around and ask / discuss to get some info before I crash?
<adeuring> lifeless: thanks :) Are the scanners yet internet ready?
<lifeless> adeuring: the scanners?
<lifeless> oh
<adeuring> lifeless: for brain scans
<lifeless> for the scan. No. Thats why I can't
<lifeless> I have them in the doorways at the house; instant backup
<adeuring> lifeless: cool. Anyway, I'm reading the MP, but I'm not that fast ;)
<michaelh1> Hey, is the API up at the moment?  I'm getting strange errors using launchpadlib...
* mthaddon changed the topic of #launchpad-dev to: Launchpad Development Channel | Performance Tuesday | Week 3 of 10.09 | PQM is OPEN | firefighting: edge rollouts disabled | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews
<lifeless> hmm, its not perf tuesday anymore ;P
* lifeless changed the topic of #launchpad-dev to: topic
<lifeless> bah
* lifeless changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 3 of 10.09 | PQM is OPEN | firefighting: edge rollouts disabled | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews
<seb128> hi
<seb128> is there any known bug with untagging bugs with the launchpablib api recently?
<seb128> the retracers are running but seems untagging is not working
<seb128> though the recent retracer crashed on a
<seb128> lazr.restfulclient.errors.HTTPError: HTTP Error 500: Internal Server Error
<seb128> ...
<seb128>       Sorry, you can't upload or download files from Launchpad at the moment,
<seb128>       because we're doing feng shui in the server room.  Normal service will
<seb128>       resume within an hour.
<lifeless> hmm, thats a librarian related error
<seb128> I guess that's different from the untagging
<seb128> it failed to add the stracktrace
<lifeless> mthaddon: sorry for the interrupt; have both librarians been behaving ?
<lifeless> seb128: yeah, thats the upload process
<mthaddon> not aware of any librarian problems
<seb128> ok, I've restarted the retracer
<seb128> let's see if that was a one time issue
<lifeless> adeuring: well I hope it made sense ;)
<adeuring> lifeless: the discussion in the mp? sure
<lifeless> adeuring: gnight
<lifeless> adeuring: I'll probably poke at this in the weekend; please do push up any stuff you do on top of it and mention in the MP or something
<adeuring> lifeless: ok, will do. nice weekend!
<noodles775> Night lifeless
<jtv> Argh.  This familiar to anyone?  It doesn't seem related to anything I did in my branch.
<jtv> ZopeXMLConfigurationError: File "/home/jtv/canonical/lp-branches/bug-618393/ftesting.zcml", line 17.4-17.35
<jtv>     ZopeXMLConfigurationError: File "/home/jtv/canonical/lp-branches/bug-618393/lib/canonical/configure.zcml", line 147.4-148.42
<jtv>     ZopeXMLConfigurationError: File "/home/jtv/canonical/lp-branches/bug-618393/lib/canonical/shipit/configure.zcml", line 7.4-8.28
<jtv>     ZopeXMLConfigurationError: File "/home/jtv/canonical/lp-branches/bug-618393/lib/canonical/shipit/browser/configure.zcml", line 7.4-12.49
<jtv>     ImportError: cannot import name cachedproperty
<deryck> Morning, all.
<noodles775> jtv: bug 629259 ?
<_mup_> Bug #629259: cachedproperty was imported by shipit <canonical-losa-lp> <ShipIt:Fix Released> <https://launchpad.net/bugs/629259>
<jtv> noodles775: ah thanks!
<jtv> hi deryck
<seb128> re
<seb128> retracer crahed again the same way
<seb128> lazr.restfulclient.errors.HTTPError: HTTP Error 500: Internal Server Error
<deryck> hi seb128.
<seb128>       Sorry, you can't upload or download files from Launchpad at the moment,
<seb128>       because we're doing feng shui in the server room.  Normal service will
<seb128>       resume within an hour.
<seb128> hey deryck
<seb128> deryck, seems we progress, the retracers have access to the crashes now
<jtv> noodles775: I wonder why suddenly this started affecting me around the time I renamed my branch directoryâ¦
<seb128> they just get an error when trying to upload the retraced stacktrace
<deryck> seb128, ok, let me check on that.
<jtv> noodles775: ah, a devel merge was also needed.
<mars> beuno, something interesting for you and the design group: http://uxmovement.com/design-articles/faster-with-top-aligned-labels
<beuno> mars, based on research, that's how I defined this: https://wiki.canonical.com/UserExperienceDesign/WebGuidelines/Forms
<beuno> not that I'm allowed to do design anymore (?)
<mars> beuno, that's awesome
<mars> the page I mean :)
<beuno> heh
<beuno> (apologies for all you lurkers not being able to see that page)
<beuno> mars, there's a bunch of stuff there: https://wiki.canonical.com/UserExperienceDesign/WebGuidelines/
<beuno> I don't think anyone has picked up that work
* lifeless changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 3 of 10.09 | PQM is OPEN | firefighting: - | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews
<lifeless> gary_poster: timeline has landed
<gary_poster> great, lifeless
<gary_poster> that will be nice to see
<lifeless> gary_poster: do you know of anything other than (sql, email, memcache) in production that can block atm ?
<gary_poster> do the oops tools need a parallel update?
<lifeless> gary_poster: it would be nice to teach them to do more with the info and look at a format change; for now I worked in the current constraints as expressed in the LP code
<lifeless> so the database column is now category
<gary_poster> fair enough
<lifeless> 'launchpad-master' will be SQL-launchpad-master
<lifeless> so the existing group-and-report-longest should still work
<gary_poster> blocks:I'm told that librarian blocks already, and I do seem to recall this.
<lifeless> ah yes, I knew that one but forgot. Thanks!
<lifeless> I'm looking at doing a tiny patch to add categories and reporting for all of these,
<lifeless> .
<lifeless> and propose it for RC so we get better reporting during the next cycle
<gary_poster> before I forget, lifeless: in regards to your "simplify OOPS emails" email from a while ago, that fits in line with things matsubara and Ursinha had discussed.  In a couple of weeks I'm planning to start focusing them + mars on that.  We'll be in touch soon with our thoughts so we can all be aligned.
<lifeless> gary_poster: great, thanks for letting me know.
<gary_poster> of course
<lifeless> http://queue.acm.org/detail.cfm?id=1854041
<lifeless> slightly tweaked version of his blog post
<mars> heya lifeless, have a sec for a testrepository question?
<lifeless> shoot
<mars> ok
<mars> I was looking at using testrepository with the Zope testrunner
<mars> and thinking about how zope can take test IDs in the form 'bin/test id1 id2 id3'
<thumper> lifeless, any quake damage where you are?
<lifeless> thumper: rangiora; haven't gone around the town looking but not expecting to find much : old riverbed, not reclaimed land
<lifeless> chc turned to jelly after the first shockwave passed through
<thumper> I've seen some pictures
<lifeless> thumper: my place seems fine so far, haven't gone into the weather to audit the outside yet.
<thumper> personally I slept through it
<lifeless> the press has some pretty stunning ones
<thumper> cool
<mars> lifeless, so testrepository has an IDFILE and test_id_option... ok, just thought of something
<lifeless> mars: the zope test runner should support --load-list
<lifeless> mars: jml patched it up
<lifeless> mars: naively, I'd expect the .testr.conf for launchpad to work with it unmodified
<mars> lifeless, hasn't landed in upstream.  I have the latest
<lifeless> ah
<lifeless> maybe zope.testing is ahead of zope itself?
<mars> zope.testing + zope.testrunner
<mars> (now)
<mars> actually
<mars> zope.testing + zope.testrunner + zc.recipe.testrunner
<lifeless> ok
<lifeless> anyhow, --load-list is terribly simple.
<lifeless> best thing to do would be to get that upstream
<lifeless> it is 'load all the tests. Perform a set intersection.'
<mars> lifeless, this should work: deoesn't though:
<mars> [DEFAULT]
<mars> test_command=./bin/test --subunit $IDOPTION
<mars> test_id_option= $IDLIST
<lifeless> what dos it do ?
<mars> prints the string 'IDLIST'
<mars> $ testr run --failing
<lifeless> there are a couple of testr bugs in this area
<mars> 'IDLIST'
<lifeless> up
<lifeless> uhm
<lifeless> try test_command=./bin/test --subunit $IDLIST
<mars> ok, that sort of works, but it looks like it only accepts one module or test
<mars> bin/test that is
<mars> ok, so upstream is still the best option
#launchpad-dev 2010-09-04
<lifeless> up, there goes another aftershake
<lifeless> sigh, staging hasn't come back up :(
<lifeless> now it has. \o.
<lifeless> bbl
<nigelb> lifeless: family and house ok?
<lifeless> yup
<lifeless> thanks for asking
<nigelb> :)
<lifeless> anyone around up for doing a review ?
<mwhudson> lifeless: maybe
<lifeless> mwhudson: https://code.edge.launchpad.net/~lifeless/launchpad/oops/+merge/34600
<lifeless> mwhudson: adds timeline actions for memcache email & librarian
<lifeless> should make e.g. thumpers merge proposal page timeouts easier to analyse
<lifeless> 300 lines, ~ 50 of that is diff context.
<lifeless> make that 70%
<lifeless> mwhudson: ^
<lifeless> gmb: ping :)
<lifeless> man, its intereseting seeing librarian stuff in the request timeline
<lifeless> 00000-00010@librarian-connection http://localhost:58000/93/hi.txt
<lifeless> 10 ms to connect ><
<lifeless> jelmer: around ?
<jelmer> lifeless: hi
<jelmer> lifeless: somewhat
<lifeless> jelmer: can I get you to eyeball an incremental commit in a reviewed branch ?
<jelmer> lifeless: yes, of course
<lifeless> -very small-
<lifeless> its just pushing to the end of lp:~lifeless/launchpad/oops now
<lifeless> done
 * jelmer looks
<lifeless> basically there was a test that was only luckily passing before
<lifeless> oops were including a unicode string
<lifeless> and the oops parser if there are statements in it does an intern
 * jelmer kicks loggerhead
<lifeless> gmb: nope, it wasn't as easy as you hoped :P https://lpbuildbot.canonical.com/builders/lucid_db_lp/builds/175/steps/compile_2/logs/stdio
<jelmer> lifeless: r=me
<lifeless> thanks
<lifeless> time to send it all via ec2, again. :(
<gmb> lifeless: Yeah, I noticed. Henninge has a fix for my borked fix, though, thankfully.
<gmb> lifeless: Once again, it is shown that the failure state of "clever" is "arsehole"
* gmb changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 3 of 10.09 | PQM is CLOSED | firefighting: - | https:/â/âdev.launchpad.net/â | Get the code: https:/â/âdev.launchpad.net/âGetting | On-call review in irc:/â/âirc.freenode.net/â#launchpad-reviews
<lifeless> gmb: lol
#launchpad-dev 2010-09-05
<mars> lifeless, do you know of anyone who has a .testr.conf for nose?
<mars> lifeless, I know about the TestID plugin for nose, but using testr itself would be nice
<mars> lifeless, turns out to be pretty simple: http://pastebin.ubuntu.com/488515/.  But for some reason it will not pick up the list of failing tests.
<mars> ah ha!
<mars> lifeless, you need to redirect STDERR for it to work with nose.  This /almost/ works as expected: http://pastebin.ubuntu.com/488516/
<lifeless> mars: sweet
<lifeless> no idea why it would be writing subunit to stderr.
<lifeless> still, mine not to wonder why
<lifeless> I've no idea how its getting negatives
<lifeless> finish sets duration to a timedelta of 'NOW - start'
<lifeless> start is set to NOW
<lifeless> NOW - NOW = 0, so the duration should be the change in NOW between the two calls being made.
<lifeless> jtv: &
<lifeless> jtv: ^
<lifeless> I wonder if its an errorlog bad-responsibilities thing
<jtv> lifeless: you're sure only one clock is involved?
<lifeless> yes
<lifeless> the durations go down almost linearly as the offset increases
<mwhudson> context?
<lifeless> so something is substracting duration again
<lifeless> bug 630612
<_mup_> Bug #630612: Complete b0rkage of oops timing info <Launchpad itself:New> <https://launchpad.net/bugs/630612>
<lifeless> e.g. https://lp-oops.canonical.com/oops.py/?oopsid=1708EB636
<jtv> Hmm it's definitely not clock skew.
<lifeless> add offset to the times and its right
<lifeless> I think
<jtv> Are you accidentally throwing the baseline time into the subtraction?
<lifeless> no
<lifeless> definitional issue
<lifeless> putting up a patch
<jtv> As in, finish - start - baseline?
<lifeless> no, its that it doesn't want a duration
<lifeless> the old code wasn't tested
<lifeless> and I misunderstood what the disk format was meant to represent
<lifeless> https://code.edge.launchpad.net/~lifeless/launchpad/oops/+merge/34635
<lifeless> thumper: are you around, perchance ?
<thumper> not really
<thumper> whats up?
<lifeless> I broke oopses on edge; fix is trivial, PQM is closed, needs a release-critical stamp
<lifeless> which is you, or gmb in about 3-4 hours
<thumper> lifeless: that mp up?
<lifeless> https://code.edge.launchpad.net/~lifeless/launchpad/oops/+merge/34635
<lifeless> by broke, they work, but you need to think like a pretzel to analyse them
<thumper> diff not there yet
 * lifeless bets its hung
<lifeless> I've pasted a diff in
<thumper> lifeless: the diff and comment don't really help me understand it
<thumper> but if you are sure it is right
<thumper> I'll rc it
<lifeless> thumper: I can explain it pretty quickly
<thumper> go then
<lifeless> look at this oops
<lifeless> https://lp-oops.canonical.com/oops.py/?oopsid=1708EB636
 * thumper is supposed to be making dinner
<lifeless> specifically the sql log
<lifeless> notice how the durations go negative
<lifeless> the actual duration is the reported duration + the start offset.
<thumper> aye
<lifeless> my patch adds the start offset in the generation code
<thumper> ok
<lifeless> I misunderstood what the disk format was meant to have in it.
<thumper> done
<lifeless> thanks
<lifeless> jtv: care to provide the code review stamp that will be wanted by ec2land ?
<jtv> lifeless: since you're phrasing it like I'll have a meaningful role in the process, sure
<lifeless> This won't hit sundays rollout, but I can nurse it into devel tonight and ask spm to trigger a reroll of edge monday am
<lifeless> sorry about breaking it
<lifeless> jtv: I think you do have a meaningful role - there are three people that have read the entire change here: you, me, mwhudson
<lifeless> jtv: I can't think of anyone better suited to review this (trivial) patch
<jtv> Well, it's reviewed.
<lifeless> thank you
<jtv> The only way I could find to do that was to request a review from myself.
<lifeless> oh
<lifeless> all you have to do is type in the comment box
<jtv> I could've added an Approve vote, but not with a specific review type.
<lifeless> and change the drop-down to approve/needs info etc
<jtv> And you want the "code" review type for "ec2 land."
<lifeless> ah
<lifeless> I think a default type == code, doesn't it ?
<lifeless> jtv: thanks for catching this
<lifeless> I would have when I got a chance to look, but I'm glad to catch it as early as possible
<lifeless> I so want a button I can push.
<lifeless> which will deploy.
 * mwhudson is tempted to say that lifeless almost certainly owns a device which can cause a deployment by pressing lots of buttons in the right order
<mwhudson> lifeless: i've just read set_request_timeline :)
<mwhudson> s/:)/:(/
<lifeless> mwhudson: I would deeply deeply love to have that better..
<lifeless> mwhudson: I'm going to be putting it fairly high up in my hygiene requests for foundations I think; can't build on a shaky base.
<lifeless> DistributionSourcePackage:+addquestion is looking pretty unhealthy
<mwhudson> lifeless: did you do any coding towards bug 623199 ?
<_mup_> Bug #623199: scripts do not establish valid zope partiticipations <Launchpad Foundations:New> <https://launchpad.net/bugs/623199>
<lifeless> mwhudson: nothing reusable
<lifeless> mwhudson: I scrapped it as a learning experience
<mwhudson> i could have a hack now, but i guess an hour is probably extremely optimistic to get something useful done
<mwhudson> lifeless: what did you learn?
<lifeless> I learnt that our scripts code is horribly confused about what they do
<mwhudson> :(
<lifeless> I am more and more of the opinion we want impersonation
<lifeless> we want something to be able to:
<lifeless>  - run async
<lifeless>  - use the API to do shit
<lifeless>  - do it on behalf of a user that originated the work
<lifeless> I think the second line, when written 'use SQL to do shit'
<lifeless> should mean nearly-no-code-changes, ideally.
<lifeless> e.g. scripts should setup a participation of the user who the work is on behalf of
<mwhudson> well i can see that would be good
<lifeless> this is obviously only applicable to deferred-work-scripts
<mwhudson> but can't we make canonical.launchpad.webapp.adapter less terrible without that?
<lifeless> others like the PPA access token updater are conceptually different
<lifeless> mwhudson: sure
<mwhudson> lifeless: and by run async, you mean in the context of appserver requests?
<lifeless> no
<mwhudson> not twisted scripts that somehow process more than one job at once?
<lifeless> badly phrased
<lifeless> 'run out of step with appserver stuff'
<mwhudson> ok, that's what i thought you meant
<lifeless> maybe for a single request, maybe much later, and all things inbetween
<mwhudson> i phrased it badly too
<mwhudson> :)
<lifeless> anyhow
<lifeless> yes, we can make adapter better
<lifeless> I think your plan is a good one:
<lifeless>  - a specific interface which needs the following characteristics:
<lifeless>   - adapter and other code like it (timelines, featureflags, permission checking) can rely on [if missing the error/fail appropriate;y]
<lifeless>  - HttpRequest implements it
<lifeless>  - something for scripts implements it
<lifeless> we may want the ability to push-and-pop the contextual-lookup for these objects
<lifeless> example:
<lifeless>  checkwatches starts up, it needs to (picking one such thing) get the timeline for it as a whole and use that while it queries the DB for watches to update
<lifeless> it may then want to, per watch, push a new context, which will get the db queries, errors, http client times, mail sending times, for a single watch.
<mwhudson> well
<mwhudson> there is newInteraction and restoreInteraction
<lifeless> sure
<mwhudson> so zope already supports this to some extent
<lifeless> I'm trying not to talk impl
<mwhudson> ok
<lifeless> you know whats available much better than I
<mwhudson> i only learnt about restoreInteraction a couple of weeks ago :-)
<lifeless> see
<lifeless> you're a couple of weeks ahead of me :P
<lifeless> thats a big fraction of my time in this job :>
<mwhudson> well, if it took me three+ years and you a few months, that's a good sign all round i think
<lifeless> ha!
<mwhudson> i think launchpad suffered for a time for not having any one who really got zope
<lifeless> I've been listening in the corners all this time
<mwhudson> so we should make sure we hang on to gary and benji :-)
<lifeless> mmm, we *started* with serious zopers.
<lifeless> anyhow, we're going well now
<lifeless> and I can see a path to having headroom to really tackle things.
<mwhudson> yeah
<mwhudson> that's good
<lifeless> so webapp adapter
<lifeless> I can - I have - described what I think we need in broad terms.
<lifeless> building on your description on the list
<lifeless> what it needs now is someone to implement it and migrate a couple of scripts over, such that we can see it has legs.
<lifeless> you could probably do the start in an hour
<lifeless> I dunno about getting into the meat
<lifeless> time for me to put on the war of the worlds and ratchet up the private librarian refactoring
<mwhudson> fair enough
<mwhudson> i think i'm going to think about linaro stuff instead :)
<lifeless> \o/ next rollout will be tracking email/librarian/memcache times
<lifeless> badly
<lifeless> but tracking them
<lifeless> jtv: bug 629921 might entertain you
<_mup_> Bug #629921: Archive:+packages with empty name search does like '%%' search. <timeout> <Soyuz:Triaged> <https://launchpad.net/bugs/629921>
<maxb> What's the likelyhood of finding someone capable of bouncing codebrowse on a Sunday? :-/
<lifeless> maxb: low to middling
<lifeless> hmm, whats the recommended url parsing lib for lp code
<wgrant> lifeless: lazr.uri, probably.
<lifeless> wgrant: do you know of any use for https urls on librarian files?
<lifeless> (the current system, I mean)
<wgrant> lifeless: They're used in the webapp.
<wgrant> To avoid insecure content warnings.
<wgrant> eg. project icons.
<lifeless> blah
 * lifeless rethinks part of this
<lifeless> the docs could at least be a little less daft about how they explain it
<lifeless> man the layers are messy
<lifeless> I think its been refactored and docstrings not changed.
<gmb> lifeless, So, the current build failure is because the checked-in wadl is out of sync with what's generated by LP. I'm regenerating the on-disk wadl and checking it in; that should fix the breakage.
<lifeless> heh
<lifeless> so are we meant to do that always? How do we tell if a change is incompatible?
<lifeless> could you perhaps file a bug asking these things of foundations :)
<lifeless> actually
<lifeless> gmb: can you please disable the test.
<gmb> lifeless, You have a good point. I think I can shed some light on the reason for the test anyway:
<gmb> 1. Before we had checked-in WADL, there were always bzr conflicts because people would accidentally check in the apidoc directory (which would be created if it didn't exist already)
<gmb> 2. Therefore, it was decided to check in the WADL to prevent those conflicts.
<gmb> 3. Trouble was that WADL generation took a long time and unless it's --forced it won't overwrite the extant files.
<gmb> So the test is there to prevent us from having something broken rolled out.
<gmb> lifeless, I don't think we should disable the test unless we're going to stop having the WADL checked-in.
<lifeless> gmb: I've replied in the thread
<lifeless> gmb: but lets check my logic.
<lifeless> we have two branches.
<lifeless> stable, db-devel.
<lifeless> both receive API changes.
<lifeless> whats going to happen in a cycle after both have had -any- API change.
<gmb> lifeless, Your reasoning is sound.
<gmb> lifeless, Okay, I agree; I'll disable the test.
<lifeless> Perhaps revert the merge that added it to restore the old logic, whatever that was.
<wgrant> Checking in WADL seems somewhat... odd.
<lifeless> I don't know enough of the guts to suggest the right thing to do.
<lifeless> I believe there was an additional desire to prevent API regressions by making people thing.
<lifeless> s/thing/think/
<lifeless> On reflection, I don't think the WADL is human readable enough for developers to do that routinely.
<gmb> lifeless, I think that test has been around for a while, actually.
<lifeless> gmb: really ?
<gmb> lifeless, Yes. THough I'm not certain. I'll check now.
<lifeless> I though benji landed it late last weke
<lifeless> \o/ we should be getting librarian stuff in oops now. /me goes to try
<gmb> lifeless, Oh, right. For some reason I thought it was something that had been kicking around for a while. In my head, I was blaming mars ;)
<lifeless>  the motivations are good
<lifeless> needs some more glue to work well
<gmb> lifeless, You're right; it landed on devel the week before last.
<gmb> I'll revert the merge(s).
<lifeless> thanks
<lifeless> my sunday is fading fast
<gmb> lifeless, Okay, no worries. I'll take care of this.
<lifeless> \o/
<lifeless> something slightly screwy
<lifeless> 15ms to connect to librarian in the dc
<lifeless> and 0ms to get the diff down
<lifeless> 15ms is a bit slow
<gmb> I swear PQM keeps adding things to the regex so that my submissions fail.
<lifeless> https://lp-oops.canonical.com/oops.py/?oopsid=1709EB904 for folk that can see
<lifeless> gmb: what did you try
<lifeless> and to what branch; they are funkily different
<gmb> lifeless, Oh, I'm being facetious. For some reason it's asking for [ui=] as well as [testfix][r-c][rs].
<lifeless> \o/
<lifeless> what branch
<gmb> devel
<lifeless> freaky
<gmb> Indeed.
<lifeless> so this cycle
<lifeless> when questions goes beserk, I'll be able to point and laugh at email very very easily :P
 * lifeless bets sending email is not cheap
<thekorn> lifeless: hi, your last comment on bug 620458 is a big surprise too me ;) the code from my last comment did not work a week ago,
<_mup_> Bug #620458: cannot access attachments of private bugs any more <qa-needstesting> <httplib2:Unknown> <Launchpad Bugs:Fix Committed by adeuring> <https://launchpad.net/bugs/620458>
<lifeless> thekorn: are you using production or edge?
<thekorn> and no, I'm not running the code in your datacenter
<thekorn> lifeless: works on both, maybe I was facing a different issue
<lifeless> perhaps
<lifeless> perhaps deryck and abel rolled back the privacy change
<lifeless> I'm working on the long term fix atm
<lifeless> hopefully we'll get it out in this rollout, and it will be faster after that.
<thekorn> lifeless: my way to reproduce this bug was always: "attachments are not accessible for private bugreports not reported by myself"
<lifeless> thekorn: well, I'll talk to deryck tomorrow night
<lifeless> gmb may know stuff now.
<lifeless> gnight y'all
<thekorn> great
<thekorn> good noight
<gmb> thekorn, I'm afraid I don't know much about the private attachment stuff besides that there's still work ongoing. I'll speak to adeuring in the morning.
<thekorn> gmb: no problem, just wanted to give lifeless a quick answer to the question he had in his last comment
<gmb> thekorn, Ah, okay, cool.
<lifeless> moin
<ricotz> please, could someone restart https://launchpad.net/~xorg-edgers/+archive/ppa/+build/1945560 which is stuck
<lifeless> ricotz: I don't think anyone with that access is around yet; you might try asking in #launchpad which is the support channel and has different people in it.
<ricotz> lifeless, thanks
<mwhudson> good morning
<lifeless> whats that thing where you can get a librarian running before the test suite starts and use it ?
<mwhudson> LP_PERSISTENT_TEST_SERVICES=1 ?
<mwhudson> possibly spelt a bit differently
<lifeless> hmm
<lifeless> actually
<lifeless> what I mean is
<lifeless> HTF do I debug the librarian daemon
<mwhudson> oh
<mwhudson> lifeless: appears to be ./bin/start_librarian
<lifeless> heres the scenario
<lifeless> oh thanks
<lifeless> I really need to track down why I get leaked processed 1/3 test runs
<lifeless> mwhudson: so, I guess I need to set a config variable too ?
<lifeless> what I want to do is: run some tests that make the librarian 500; with pdb on the librarian
<mwhudson> lifeless: afaik know, LP_CONFIG defaults to development
<mwhudson> aah
<mwhudson> hm
<mwhudson> s/know/no/ yay for homonym substitution
<mwhudson> lifeless: maybe make start_librarian LP_CONFIG=testrunner
<mwhudson> then run the tests you care about with LP_PERSISTENT_TEST_SERVICES=1 set  ?
<mwhudson> would work mostly by chance i guess, but might work
<lifeless> hah
<lifeless> Daemons cannot log to stdout, exiting
<mwhudson> :(
 * lifeless files a bug
<mwhudson> figure out what start_librarian does, do that but add -n to the twistd arguments
<mwhudson> lifeless: it may be easier to do make run_all and recreate the tests by hand
<mwhudson> (or not, depending on circs)
<lifeless> it was entertaining finding this in the librarian
<lifeless> raise LookupError
<thumper> morning people
<wgrant> mwhudson: Homophone! Not homonym!
<wgrant> But anyway, morning.
<thumper> morning wgrant
<wgrant> How're things in NZ after the earthquake?
<lifeless> wgrant: we're still here.
<lifeless> chc is a bit fucked up
<wgrant> Yeah, so it seems...
<lifeless> we got out of there 2 weeks before the quake; good timing if I do say so myself
<wgrant> Er, yes.
<lifeless> wgrant: private librarian stuff is just about gtg
<lifeless> https://code.edge.launchpad.net/++oops++/~lifeless/launchpad/private-librarian/+merge/31020 if you're interested in it
<wgrant> I might remove the ++oops++ :)
 * thumper goes to make a coffee
<wgrant> lifeless: Oh, going straight to multiple domains?
<wgrant> That's great.
<wgrant> It would be nice if access with an invalid token would redirect to the webapp to get a new token.
<wgrant> But LFAs don't have enough context :(
<wgrant> Ah, I see you've already discussed that.
<wgrant> lifeless:
<wgrant> 480	+When the context file is a restricted `LibraryFileAlias`, traversal causes an
<wgrant> 481	+access token to be allocated and a redirection to https on a unique domain to
<wgrant> 482	+be issued.
<wgrant> In that test, can you unelide the 'i....restricted'?
<wgrant> Otherwise it's not obvious then that anything's different about the URL.
<wgrant> And since it's meant to be documentation, that seems like a bad thing.
<lifeless> that test file is unit tests masquerading as docs
<wgrant> Ah.
<thumper> lifeless: got time to chat?
<lifeless> thats on line 465 for me
<lifeless> thumper: sure
<lifeless> skype?
<thumper> lifeless: yep
<wgrant> lifeless: Ah, I didn't have the latest rev.
<wgrant> lifeless: So it doesn't actually check the domain?
<lifeless> right
<wgrant> Not a huge fan, but I guess it's OK.
<lifeless> wgrant: if there is  ahole we can fix it, but I think its ok
<wgrant> It just lets people make mistakes without noticing.
<wgrant> Hm, actually, it might be dangerous.
<wgrant> Yes, it is.
<wgrant> I think.
<wgrant> I forget the exact cross-window security restrictions....
<wgrant> Why can't this be simple? :(
<mwhudson> wgrant: ah right
<wgrant> I'm also not sure how browsers treat Referer when leaving an HTTPS URL for another HTTPS domain.
<wgrant> The RFC only says that they shouldn't send it when going non-secure.
<wgrant> Not cross-domain.
<lifeless> wgrant: what would the attack be
<lifeless> I've asked kees to review as well
<wgrant> lifeless: Somebody visits a private file. I can come along and send them to a page which lives on that same domain. Now, I'm not entirely sure how to get the other URL, but we've now bypassed cross-domain restrictions, so it needs thought.
<wgrant> Complicated :(
<wgrant> Ah, I know.
<wgrant> I know that somebody has access to a private file. I know its webapp URL, LFA ID and filename.
<wgrant> I get an HTML file into a library file, and send them a URL to it on the target LFA's domain.
<wgrant> That page uses an iframe to go to the webapp URL (thus holding a reference to the window).
<wgrant> Once it gets to the webapp, my nasty page can't access the iframe (because it's on a different domain).
<wgrant> But the webapp will then redirect back to the file, on my domain.
<wgrant> Once it's back on my domain, I can access properties of the window (including its URL).
<wgrant> I think that should work.
<lifeless> so, concretely
<lifeless> you file a private bug that they will look at
<lifeless> it has an attachment which will look up some other thing via the attach you describe above
<lifeless> and to block it we need to make the domains line up
<wgrant> It doesn't need to be a private bug.
<lifeless> s/attach/attack/
<wgrant> I just need to get them to a librarian URL somehow.
<wgrant> But yes, you need to ensure that the domains match.
<lifeless> which means knowing the LFA, which is only shown if you have access
<wgrant> lifeless: Oh really?
<wgrant> It can be guessed.
<wgrant> I've done so on a number of occasions :)
<lifeless> ok
<lifeless> I'm trying to estimate the risk if we:
<lifeless>  - deploy roughly what we have today
<lifeless>  - check the request path to be sure Host is preserved
<lifeless>  - enhance it to enforce domain matching in a future revision
<wgrant> "to be sure Host is preserved"?
<lifeless> the request path for the private librarian is: client -> apache -> squid -> librarianN
<lifeless> I'm not entirely sure the host header will be getting through untouched *right now* because we've never depended on it.
<wgrant> Oh, request path as in path of the request.
<wgrant> Not the path attribute of the request.
<wgrant> Right.
<lifeless> to enforce the domain matches the LFA id on all requests, we need to make sure its preserved
<wgrant> Yep.
<lifeless> wgrant: I'd like to get an iteration of this live on thursday
<lifeless> wgrant: we have essentially 2 days to get all the kinks out, or to defer some stuff.
<wgrant> We should also check out how browsers handle Referer.
<wgrant> I can't find any explicit mention of them behaving sanely :(
<lifeless> I don't want to deploy a badly broken system, but if it is better than what we had, and relatively low risk, it might be ok for a week or two
<lifeless> wgrant: assume insanity
<wgrant> lifeless: I am.
<wgrant> (the issue here is that links from private files to external HTTPS sites may reveal the file to the target site)
<lifeless> wgrant: yes. I don't have any ideas how to do that other than having a cookie setting service.
<lifeless> wgrant: and it would still reveal the existence of the files
<lifeless> wgrant: OTOH a file can only shoot itself in the foot
<wgrant> Yeah, the solution I thought of was to have the tokenised URL set a cookie then redirect to something without the token.
<wgrant> But grrrr.
<wgrant> Stupid web.
<lifeless> wgrant: yeah, 'cookie setting service'. meep sucky.
<lifeless> wgrant: but back to /now/ : is the current thing fatally flawed, or something we could iterate on over a couple weeks.
<lifeless> deploy + iterate, that is.
<lifeless> I'm personally fine with attachments that shoot themselves in the foot.
<wgrant> lifeless: I'm not comfortable making a statement either way.
<wgrant> Right, that's probably OK.
<wallyworld_> morning
<wgrant> The attack I outlined earlier is my concern.
<wgrant> But that *in combination* with the shooting-themselves-in-the-foot is pretty bad.
<wgrant> If Referer is indeed sent.
<thumper> wallyworld_: morning
<wgrant> Difficulties I see with implementing the domain restriction:
<wgrant>  - As you say, the request path may not preserve Host.
<wgrant>  - We might have to make launchpad.dev exempt, since otherwise we need wildcard /etc/hosts....
<lifeless> right, there is a comment in the tests about that ;)
<lifeless> we can actually test the librarian without it
<lifeless> (connect on ip, host header passed to librarian)
<wgrant> Yep.
<lifeless> and we could check that private bug /urls/ are of the right shape, not actually connect.
<lifeless> attachment urls, I mean.
<lifeless> do a non-follow-redirect request
<wgrant> (also, are you going to be able to get a cert in time?)
<lifeless> wgrant: its the top ticket in the LP queue.
<lifeless> wgrant: also on the MP it says 'have a thing to allow it to be enabled post rollout'
<wgrant> Ah, good.
<wgrant> I didn't actually read the MP description.
<wgrant> Just the diff.
<lifeless> specifically I'd like to get the code out there
<lifeless> post rollout hernias addressed.
<lifeless> then get a manually inserted TLT, check with wget it works, then enable the appserver code.
<wgrant> I prefer to read the code first, so my interpretation isn't incorrectly influenced by the description.
<wgrant> Right.
<wgrant> Wait, TLT?
<lifeless> time limited token
<wgrant> Oh, right.
<wgrant> I guess this is what feature flags are for.
<lifeless> yes
<wgrant> Handy.
<lifeless> they're a bit rought still
<lifeless> but will do the job
<lifeless> wgrant: http://www.geonet.org.nz/images/news/2010/Fault_0564.jpg
<lifeless> http://www.geonet.org.nz/news/article-sep-4-2010-christchurch-earthquake.html
<wgrant> Ow.
#launchpad-dev 2011-08-29
 * jelmer will have a look at the merge conflict
<wgrant> Hah.
<wgrant> I was just going to ask if you were still around.
<wgrant> Thanks.
<poolie> hi wgrant
<wgrant> Morning poolie.
<wallyworld> wgrant: with the webservice api if something is exported for version 'beta', can i change it?
<StevenK> Edit the decorator in the interface
<wallyworld> StevenK: you are meant to be away? anyway, you mean edit the decorator to change the export_for_version() ?
<wallyworld> i didn't think that was allowed?
<StevenK> wallyworld: (Where are your multiple underscores?) Just because I'm holidays and not working doesn't mean I can't answer questions.
<wgrant> wallyworld: It depends. What are you changing?
<wallyworld> i need to change the semantics of the private attribute on IBranch
<wgrant> (note that beta is officially EOLed, but 1.0 is still supported, but we will change things still if required)
<StevenK> Changes to API functions need to be considered very carefully.
<wallyworld> it is read/write (via setPrivate) but i need to to be readonly
<wallyworld> and there's a new explicitly_private attribute instead
<wallyworld> i could keep it read/write and add a new mutator method for explicitly_private
<wgrant> I would just break compatibility.
<wgrant> That
<wgrant> 's the sort of thing that nobody is likely to care about, and those that do can fix their code.
<wgrant> 1.0 exists mostly to not break code in Ubuntu releases.
<wallyworld> that's what i was hoping for :-)
<wgrant> None of which uses Branch.private.
<wallyworld> cool. it will still appear the same for r/o access
<wallyworld> but now you write using 'explicitly_private' instead of 'private'
<luke-jr> Is there a git-supporting branch yet?
<wgrant> wallyworld: Right.
<wgrant> luke-jr: No.
<nigelb> Morning!
<nigelb> wgrant: I'm going to take a screenshot of you telling lifeless to go away :P
<wgrant> Heh.
<nigelb> I'm trying to do a html = create_initialized_view(person, '+index')()
<nigelb> the html is turning out empty.
<nigelb> (this is in registry)
<nigelb> what am I doing wrong?
<wgrant> Is person actually a Person?
<nigelb> well, yes. I used factory
<nigelb> This is right, isn't it? person = self.factory.makePerson(time_zone='Asia/Kolkata')
<wgrant> That is indeed correct.
<wgrant> Hmm.
<wgrant> What is returned from create_initialized_view(person, '+index')?
<nigelb> empty html as far as I can tell from the test failure
<nigelb> err, empty string
<nigelb> The user story from the guy earlier is depressing.
<nigelb> Also, slightly troll-ish.
<wgrant> Mmm, it's pretty reasonable.
<wgrant> LP hasn't changed significantly since 2005.
<wgrant> UI-concept-wise, that is.
<nigelb> Sure, but LP has a slightly different concept for branches
<nigelb> Its tied to a project and not a person.
<wgrant> Although all those fields he talks about are hidden when unset, except when you're looking at your own page...
<nigelb> Yeah.
<wgrant> Right, but registering a project and stuff is difficult and awkward.
<nigelb> Though, I'd love to see UI improvements.
<nigelb> Does making private branches need an admin? I've never created one or used one.
<wgrant> let's not go there.
<spm> nigelb: not only is that sequence of lines - wgrant telling lifeless to go away - worthy of being screenshotted. I'm tempted to also print out said shot and place prominently on the wall. for whenever I need giggles.
<nigelb> spm: That's the point of the screenshotting! Lets frame it ;)
<wgrant> nigelb: My theory is that Launchpad's privacy features were designed by GitHub.
<spm> framing might be considered excessive. but not by me. good ideaâ¢
<nigelb> wgrant: Oh. That bad?
<wgrant> GitHub's team infiltrated Launchpad before GitHub was created.
<wgrant> And designed Launchpad's privacy features.
<nigelb> heh
<wgrant> To make GitHub win,.
<wgrant> It is probably the most hopelessly incoherent and useless aspect of Launchpad.
<nigelb> Are UI improvements on the way? Because even bugzilla is getting a facelift soonish.
<wgrant> huwshimi has plans.
<wgrant> But we're without him for a while.
<wgrant> But I don't really think he can do it on his own.
<wgrant> and UI designers never stay with us for long.
<wgrant> So the outlook is not stunning.
<nigelb> Oh.
<LPCIBot> Project db-devel build #827: STILL FAILING in 4 hr 41 min: https://lpci.wedontsleep.org/job/db-devel/827/
<wgrant> Most of the points in that blog entry are reasonable, apart from eg. condemning LP for not offering free private branches.
<nigelb> I wish launchpad were more social.
<wgrant> Ahh, and then he goes into the usual "why can't I check out the branch I created thing".
<wgrant> GitHub and BitBucket have poisoned everyone's minds :(
<nigelb> Wait, you can't checkout a branch you just created on github.
<wgrant> Can't you?
<wgrant> The difference with GitHub is that you have to do a server-side fork.
<wgrant> Rather than just pushing up a new branch.
<wgrant> So people expect that from LP.
<wgrant> Nobody had a problem with just pushing before GitHub came along.
<nigelb> Heh.
<nigelb> The problem is, well, they're good. With good UI.
<nigelb> Launchpad's UI has never been its strenghts.
<nigelb> Code review, however, is.
<wgrant> Eh, not much had good UI in 2005.
<wgrant> It's just every other website in the world has moved on.
<nigelb> Heh, that too!
<nigelb> Anyone tried mergebox yet?
<nigelb> wgrant: Also, any clues with the test I can't get to work?
<wgrant> nigelb: Ah, sorry, got distracted. I was interested in the output of create_initialized_view(person, '+index'), *without* the trailing ().
<wgrant> It should give you a view object.
<wgrant> The trailing () executes the view to turn it into HTML, which I want to avoid.
<wgrant> So we can see what the view is.
<nigelb> Okay, what's the easiest way to output a view object?
<wgrant> Just get the repr for now.
<wgrant> "I want to make a branch and then be able to push it back to the original, through the web interface. This should and has to be easy to do and not require juggling between the LP server and my local PC in order to either branch or merge."
<wgrant> I'm confused.
<wgrant> Doesn't one normally develop on one's local PC, necessitating that juggling?
<nigelb> hm, does github/whatever support merging from the web UI?
<nigelb> I thought not yet.
<wgrant> I'm not sure.
<wgrant> But his thing didn't seem to mention actually merging.
<nigelb> BUt that's what it is right? Create a branch, push back into original.
<nigelb> but why would you create a branch and push to LP if you don't want an MP.
<nigelb> Hm.
<nigelb> AssertionError: 'Asia/Calcutta' not in '<zope.browserpage.metaconfigure.SimpleViewClass from /home/nigelbabu/launchpad/lp-branches/188187-time-zone-offset/lib/lp/registry/browser/../templates/person-index.pt object at 0xe97830c>'
<wgrant> Hmm. So that should render to more than ''...
<nigelb> Yeah, I did try to figure out and then give up.
<nigelb> I seem to be having a complicated relationship with tests :D
<wgrant> Launchpad has that effect on people :)
<nigelb> Heh.
<nigelb> I thought I'd get this done without asking too much help, etc. Bam! Tests!
<nigelb> wgrant: What should be the next move?
<nigelb> I wonder if I can skip tests for this :D
<wgrant> No, this definitely needs tests :)
<wgrant> Let's see...
<wgrant> Almost got my new lucid-lxc-on-oneiric development environment set up adequately.
<nigelb> I still don't see the charm of developing on bleeding edge.
<wgrant> Huh.
<wgrant> It's '' here too.
<nigelb> \o/
<nigelb> I'm not going mad!
<wgrant> Using oneiric is nice.
<wgrant> Get to iron out bugs, whine to thumper about the new unity being crap, fglrx 11.8 is slightly less awful than 11.5...
<nigelb> thumper works on Unity?
<wgrant> He defected to the Unity team earlier this year.
 * nigelb had a vague feeling he used to be LP or bzr.
<wgrant> He was lp-bzr.
<wgrant> Or lp-code, as it was known in its later years.
<nigelb> Ah
<nigelb> Right now, there isn't a subteam working on this component is there? All squads working on bits of everything.
<poolie> hi nigelb
<wgrant> Right.
<nigelb> Hey poolie, Morning!
<nigelb> (well, its probably evening for you by now :P)
<poolie> afternoon
<wgrant> nigelb: Is your test in the same class as other view tests?
<nigelb> Yeah, let me grab that class name
<nigelb> Its inside TestPersonIndexView
<wgrant> I'm not quite sure what's going on here. I guess it's not using the root template that includes the slots.
<wgrant> nigelb: Where are you adding this?
<wgrant> The "Time zone" section under "User information"?
<wgrant> nigelb: If so, try asking for +portlet-contact-details instead of +index.
<wgrant> More specific, and it will probably even work.
<nigelb> wgrant: doing that now
<wgrant> Although nothing seems to render that directly either.
<wgrant> You may just have to give up and test an attribute on the view and not the actual display.
<wgrant> Or resort to a browser test.
<nigelb> browser test?
<wgrant> grep for user_browser
<nigelb> \o/
<wgrant> zope.testbrowser is a minimal web browser that exercises most of the Zope stack.
<nigelb> +portlet-contact-details worked
<wgrant> So you'll be able to see the output of the view that way.
<wgrant> Great.
<wgrant> Aha.
<nigelb> Gah, so I get to use doctests again. Win.
<nigelb> ok, looks like everything works.
 * nigelb forces a failure
<nigelb> aaargh. lint failure.
<nigelb> I should be fixing lint failure that I didn't cause, right?
<nigelb> Ok, fixing this lint failure might lead to breakage.
<nigelb> wgrant: I get a bunch of "assigned to but never used" lint failures.
<wgrant> nigelb: In doctests?
<nigelb> Its unrelatd to code I touched.
<nigelb> But in the files I touched
<wgrant> If it's not a doctest, you should just be able to remove the assignment.
<wgrant> ie, s/a = b/b/
<nigelb> A bunch of these http://pastebin.ubuntu.com/677037/
<wgrant> Those should all be trivially fixable.
<nigelb> liene 308 is interesting.
<nigelb> I think the view is created so that a login token exists.
<wgrant> Yeah, you probably need to continue creating the view, but you don't have to assign it to anywhere.
<wgrant> The return value is not important.
<nigelb> aah
<nigelb> wgrant: what does the last one in that lint failure mean?
<wgrant> nigelb: Normally that there's a = or + without a space on both sides.
<nigelb> wgrant: Could you review https://code.launchpad.net/~nigelbabu/launchpad/188187-time-zone-offset/+merge/73168
<nigelb> stub doesn't seem to be here yet.
<rvba> Morning all.
<thumper> wgrant: not entirely sure that defected is the right word
<wgrant> thumper: Escaped?
<nigelb> heh
<nigelb> thumper: "ran away"
<thumper> how about, "needed a change"
<wgrant> Bah.
<nigelb> A lot Canonical seem to be Launchpad defectors. :P
<wgrant> Heh.
<nigelb> wgrant: You picky picky person :D
<wgrant> I do that :)
<nigelb> when I do tal:foo
<nigelb> foo is a variable?
<wgrant> It's irrelevant.
<wgrant> means nothing at all.
<nigelb> wgrant: Updated!
<nigelb> poolie: ooh, that's a nice next bug :D
<poolie> the team timezones thing?
<poolie> yeah
<poolie> it has limits
<poolie> especially as you know nothing about people's habitual online hours
<nigelb> True
<nigelb> I set my timezone on Launchpad as UTC.
<adeuring> good morning
<poolie> i think thisi s one thing people specifically said they missed when we took out the google maps display
<poolie> hi abel
<nigelb> Why did we take out maps? I remember there was some javascript error and then it wasn't around the next day.
<poolie> i think part of it was about google maps over https being a for-pay service
<wgrant> nigelb: Google wanted to charge a heinous sum for HTTPS maps.
<poolie> and maybe some licencing problem
<wgrant> But I believe that is no longer an issue.
<nigelb> Oh.
<poolie> people complained about it being nonfree
<wgrant> As they have decided that HTTPS should be everywhere now.
<wgrant> But it has been suggested it should be replaced with OSM, yes.
<nigelb> I'm always about what works vs non-free :)
<poolie> they have decided SPDY should be everywhere
<nigelb> We had something like that in Loco directory using google maps.
<nigelb> An OSM developer requested we not use OSM because that's not what its meant for.
<poolie> oh?
<poolie> it's meant for precise positioning?
<wgrant> It's interesting that LP has been HTTPS-only forever, and many people used to complain, and now the rest of the world is finally starting to go HTTPS-only. We were years ahead of the curve on one thing, at least.
<poolie> yes, which is quite cool
<nigelb> wgrant: We're ahead only in one curve :)
<poolie> of course many of these maps are not all that accurate
<nigelb> poolie: Because OSM is not ready to get hit with lots of people loading pages with it or something.
<wgrant> nigelb: Indentation on prejoins has the same indentation issue.
<nigelb> GRAR
<poolie> just showing "... which is on the south-east coast of india" would be enough
<nigelb> wgrant: okay, fixed.
<nigelb> poolie: But still, that doesn't help when you want to figure out when is a good time to set a meeting with someone :)
<stub> poolie: If we wanted to be invasive, we probably could work out an awful lot about peoples habitual online hours :-)
<wgrant> s/invasive/Google/
<nigelb> No, please don't. My hours are horrible.
<nigelb> I slept at 4 am, woke up at 11, and I'm now going to work at 1:30 pm. Fun.
<poolie> perhaps it can be a separate service
<poolie> oh i guess if we had a timeline view
<wgrant> Heh.
<nigelb> timeline of commits? that should be fun.
<nigelb> should also overlap with marking who's on vacation when.
<wgrant> Award karma points if you don't obtain any karma between midnight and 8am in your designated timezone.
<poolie> badges
<poolie> fix one bug every hour for 24h
<nigelb> heh.
<nigelb> Double karma between 10 pm and 8 am your timezone.
<stub> I prefer the first variant to try and stop burnout rather than encourage it :)
<nigelb> Heh
<nigelb> I'll then reset the timezone to the timezone my brain lives in.
<nigelb> "Oh, I'm working in Hawaii time today.  Iceland tomorrow!"
<wgrant> I've filed 50 LP UI bugs in 24 hours, but not fixed 24...
<stub> Yer, need roving to cope with my 25 hour cycle :-/
<nigelb> Laters! Off to lunch and then work.
<nigelb> wgrant: Gah. formatimports. I knew I missed something.
* henninge changed the topic of #launchpad-dev to:  #launchpad-dev ist: https://dev.launchpad.net/ | On call reviewer: henninge | Critical bugs: 237 - 0:[#######=]:256
<wgrant> nigelb: When you fix, I will send it off to ec2.
<wgrant> Although it won't land until jelmer hopefully sorts out the testfix on db-devel.
<nigelb> I'll fix when I get home. In about 5 hours.
<wgrant> nigelb: Thanks!
<nigelb> One more inch closer to 128.
<jelmer> wgrant: whoops, I hadn't seen that yet
<wgrant> jelmer: Ah, wasn't expecting you around yet.
<jelmer> testfix is on the way
<wgrant> jelmer: Thanks!
<jelmer> in other news, bzr code imports seem to work on qastaging :)
<jelmer> https://code.qastaging.launchpad.net/~jelmer/bzr-svn/revprops
<jelmer> wgrant: and it looks like staging was updated?
<wgrant> jelmer: Indeed, spm unbroke it this afternoon :)
<wgrant> jelmer: What UI is there for them? "Import details" doesn't seem to show anything...
<jelmer> wgrant: The UI bit hasn't landed yet, the only way to create them on qastaging is using the API
<wgrant> jelmer: Ah, that would do it.
<jelmer> wgrant: I have another branch up for review that changes the web UI to register "bzr" code imports rather than mirrors
<wgrant> Great.
<jelmer> then someday, perhaps, we can get rid of the branch puller
<wgrant> Well, we'd need to fix the way imports work first.
<wgrant> They still use the puller :(
<jelmer> yeah, that bit I'm still pondering
<wgrant> But that is a significantly easier problem than general branch mirroring.
<jelmer> a related issue is that it would be really useful to do stacking of code import branches, but the current architecture makes that hard (especially if the development focus is not a code import)
<wgrant> Indeed.
<wgrant> I think they should probably just be bzr+ssh clients or so.
<wgrant> Or maybe use the internal branchfs.
<matsubara> adeuring, hey, good morning. could you take another look at that oops-tools branch you reviewed on Friday? https://code.launchpad.net/~matsubara/oops-tools/829460-typeerror-qsmmx/+merge/73062
<adeuring> matsubara: sure, but I'd like to have lunch first :)
<wgrant> Assuming I don't accidentally delete all the hosted areas again.
<matsubara> adeuring, sure.
<jelmer> wgrant: :)
<wgrant> jelmer: I guess we probably don't have to consider stacking on private branches.
<wgrant> Which might make things easier.
<jelmer> wgrant: What about private users who are importing multiple branches?
<wgrant> jelmer: We don't support credentials in URLs, do we?
<wgrant> So private imports are not hugely likely at the moment.
<jelmer> wgrant: We do, at least for bzr-svn/bzr-git/bzr-hg imports (some people have silly requirements like logging in with guest/guest. Not sure about mirror URLs.
<wgrant> Bah.
<wgrant> OK.
<wgrant> So we just have to grant the service user access to dev focus branches sometimes too, I guess.
<jelmer> wgrant: hmm, yeah
<jelmer> wgrant: I guess just supporting stacking on public dev focus branches would be easiest for now, and already be a huge improvement over the current situation.
<wgrant> It's all going to be special-cased anyway, because branches can be owned by anyone.
<wgrant> But only owners can write.
<jelmer> does zope support impersonation ? :)
<wgrant> It can, but LP doesn't use it.
<wgrant> eg. SCA is special-cased in security adapters.
<wgrant> I imagine this would, for now, be something similar.
<jelmer> ah, interesting
<StevenK> We just killed bazaar-experts, now you want to resurrect it?
<jelmer> :) I'm glad that's gone..
<wgrant> A celebrity for the importds is probably best for now, I fear. Unless we want to give them access to the branchfs, which sounds bad.
<StevenK> jelmer: The team still exists, since lifeless is a wimp. But it is no longer a celebrity.
<StevenK> wgrant: That makes me very sad.
* benji changed the topic of #launchpad-dev to: ist: https://dev.launchpad.net/ | On call reviewer: henninge, benji | Critical bugs: 237 - 0:[#######=]:256
<wgrant> StevenK: For now.
<wgrant> StevenK: Until we have a less bad permission system.
<wgrant> "now" in Launchpad time -- that is, at least a decade.
<jelmer> wgrant: I agree, about having a celebrity
<jelmer> wgrant: Is it just my change of perception, or is Launchpad development quicker these days?
<StevenK> benji: O hai -- did Irene spare you her wrath?
<wgrant> jelmer: Hmmm.
<wgrant> jelmer: I think it depends where you're working.
<wgrant> And how you're working.
<wgrant> and what you're doing.
<wgrant> But it may be.
<benji> StevenK: yep, it wasn't too bad here; we just got a good dose of rain
<StevenK> benji: Ah, rain is easily coped with.
<wgrant> jelmer: Code's always been quicker than the rest, I think.
<wgrant> jelmer: Because it's been designed.
<wgrant> At least a bit.
<wgrant> At least I've always found it nice.
<rvba> henninge: benji Hi guys, could any of you have a look at this (Javascript) MP https://code.launchpad.net/~rvb/launchpad/confirmation-overlay-bug-830982/+merge/73078 ?
<benji> rvba: I'm available.
<jelmer> wgrant: hmm
<rvba> benji: Thanks a lot!
<henninge> benji: I'd like to take ;)
<benji> henninge: be my guest
<lifeless> wgrant: can you do me a favour tomorrow? put the performance tuesday tag on in your morning
<wgrant> lifeless: Only if you go away, but sure :)
<nigelb> heh
<nigelb> moar screenshots.
<lifeless> wgrant: given I just drove back from the hospital and am about to crash for 6 hours; I can guarantee I am going away.
<wgrant> :)
<wgrant> Night!
<lifeless> night, thanks!
* wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: henninge, benji | Critical bugs: 244 - 0:[#######=]:256
<wgrant> danilos: Hi.
<wgrant> Hmmm.
<wgrant> This is interesting.
<wgrant> Trying to upload via FTP to a poppy running in LXC, and I have the hanging-on-the-last-kilobyte bug.
<wgrant> bigjools: ^^
<bigjools> nice
<wgrant> Reproducible too.
<bigjools> repeatable for any upload?
 * wgrant debugs.
<wgrant> Does suggest it's odd networking, though.
<bigjools> yes
<bigjools> does the LXC NAT?
<wgrant> I'm using libvirt's bridge.
<bigjools> I'll lay real money on it being a nat type issue
<wgrant> It does NAT, but this shouldn't be using it.
<wgrant> Because it's host-to-guest.
<StevenK> Haha. How much real money?
<bigjools> 2 pence
<wgrant> So should just go straight over the internal bridge without NAT.
<bigjools> it'll be doing something internally still
<StevenK> So given the UK currency that's about 0.5 Australian cents
<wgrant> Let's hope.
<bigjools> StevenK: miaow
<wgrant> Wow.
<wgrant> Wireshark doesn't hang Unity!
<wgrant> A complex application that doesn't hang oneiric's unity is a rarity these days.
<nigelb> well, its moved from simple to complex, that's worth rejoicing.
<wgrant> Ah, there we go, lagging up a bit now.
<wgrant> Hmm.
<wgrant> So it does the gpg verification, then doesn't send anything back.
<StevenK> wgrant: I wonder if an SFTP upload also hangs with LXC involved
<wgrant> I doubt it.
<wgrant> It works.
<bigjools> remember that this happened with the zope server too
 * bigjools -> food
<wgrant> Yeah, that is pretty odd.
<wgrant> So, it turns out that in this case it's the GPG verifier crashing.
<StevenK> Nice!
<StevenK> Do you have a traceback?
<wgrant> But I can't see the exception by default...
<wgrant> Not until I add any extra log entry right before the crash.
<wgrant> once I do that, I get an error traceback.
<wgrant> WTF twisted?
<wgrant> Hmm.
<wgrant> I wonder if logging doesn't get completely initialised at startup.
<wgrant> Writing to the Python log somehow finishes setup, allowing the Twisted logger to log.
<StevenK> Can you paste the traceback?
<wgrant> The problem with it is clear: the "key not registered" error is unicode, while twisted wants a str.
<wgrant> But this doesn't explain all the hangs, sadly.
<wgrant> It may explain why we can't work them out, though.
<StevenK> I wonder if that can tested sanely
<deryck> Morning, all.
<adeuring> matsubara: r=me
<adeuring> morning deryck
<matsubara> thanks adeuring
<nigelb> mm, LP commits visualized. http://www.youtube.com/watch?v=9Pe70fw83Zs
<deryck> adeuring, hey, could you join me in mumble for 2 seconds?  Just need to test I'm working.
<deryck> adeuring, ping for standup.
<adeuring> deryck: ok
<jelmer> henninge, benji: can I add a MP to your queue?
<benji> jelmer: sure
* henninge changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: benji | Critical bugs: 244 - 0:[#######=]:256
<jelmer> benji: thanks
<jelmer> https://code.launchpad.net/~jelmer/launchpad/no-code-import-approval/+merge/73187
<henninge> benji: I won't take another after this one.
<henninge> I mean, the one I am working on.#
<benji> henninge: k
<flacoste> adeuring: hi, thanks for following up on bug 835103
<_mup_> Bug #835103: HWDB submissions since Lucid are marked as Invalid <hwdb> <regression> <Launchpad itself:In Progress by adeuring> < https://launchpad.net/bugs/835103 >
<flacoste> adeuring: do we need to open a task against checkbox because of the missing sysfs-attributes?
<matsubara> anyone have an idea how to solve this: https://code.launchpad.net/~matsubara/oops-tools/829460-typeerror-qsmmx/+merge/73062? it's not finding pytz in the download-cache even though I added it there...
<stub> matsubara: you running with Python2.5 or something like that?
<matsubara> stub, it's the tarmac chroot for oops-tools. I'm not sure what version of python is running there
<matsubara> losas: ping
<matsubara> stub, adding pytz-2011h-py2.5.egg to the download cache would solve the problem, right?
<stub> it would work around the problem
<stub> (if it does, you still have the problem that our code is only known to work with Python 2.6)
<adeuring> flacoste: yes, I think we need a checkbox task too -- at least for the next version of checkbox
<matsubara> that's what's in the LTS release, isn't it? maybe I should request that the chroot to be update to 2.6
<deryck> abentley, just need to reboot after upgrade, and I'll ping for pre-imp call.
<abentley> deryck: cool.
<flacoste> adeuring: do you have an ETA on the fix and the time it will take to re-parse all the invalid submissions?
<adeuring> flacoste: I hope to have a fix tormorrow. Processing one submission need 1 or 2 seconds; multiply that by the number of broken submissions for the total processing time
<adeuring> ...and allow for some pausing
<flacoste> adeuring: how can we re-process the invalid files? we flip the status back to processed and allow the script to do its normal thing?
<adeuring> flacoste: exactly
<flacoste> adeuring: is it DBLoopTunabled yet?
<adeuring> flacoste: yes
<flacoste> awesome!
<flacoste> adeuring: do we record the schema error as an OOPS? maybe user-generated one?
<flacoste> or it was only the logs?
<adeuring> flacoste: I think this is only in the logs
<flacoste> ok, we should probably record oopses
<abentley> danilos: when you commit changes to sourcedeps.conf, please also commit the changes to sourcedeps.cache.
<flacoste> that would have made this error show in our report
<flacoste> i'll file a bug for that
<deryck> flacoste, I suggested an integration test of some sort between hwdb/checkbox as well as lpstats for invalid submissions in the bug.  FWIW.
<adeuring> flacoste: well... people may submit totally bogus data, so this would mean also some potential noise in the OOPS reports
<flacoste> adeuring: you mean checkbox can submit invalid data?
<flacoste> or since it's a user-accessible service, someone could upoad bogus data?
<flacoste> in any cases, we could treat those as "user-generated" error
<adeuring> flacoste: well, we want to record this :) But what if people send deliberatley nonsensical data, by messing with checkbox? Or by simply uploading arbitrary files?
<flacoste> that's not really an issue either way
<flacoste> deryck: the integration tests should probably live in checkbox
<deryck> flacoste, yeah, agreed.
<flacoste> gary_poster, matsubara: where does the QA-SLA document for API scripts live? (for things like apport)
 * deryck has to reboot now
<flacoste> the submission HWDB isn't our regular API, but I think the thoughts there should work
<gary_poster> abentley, jam approved https://code.launchpad.net/~gary/bzr/bug835035/+merge/73124 .  I have three questions now for when you have a moment.
<gary_poster> (1) I have an XXX comment in there that jam did not comment on, even though I highlghted it in the cover letter.  He approved the branch as-is, so should I just not worry about it, or should I...do something else (remove the XXX, remove the comment, ...)
<gary_poster> (2) What should I do to actually get this MP landed on the 2.3 line?
<gary_poster> (3) How do I make an egg for LP to use?  egg_info does not seem to be honored.  Should I change bzrlib.version_info from (2, 3, 5, 'dev', 0) to (2, 3, 5, 'dev', 1) or something, and then run ./setup.py build?
<jam> gary_poster: ask us :)
<gary_poster> jam, hi :-)
<gary_poster> didn't realize you were here
<jam> you proposed it to 2.3, so once I get off my tukus and submit it, it will land in 2.3
<jam> for ~1 more hour
<gary_poster> jam, ok cool :-)
<jam> gary_poster: so a few things now that I think of it
<jam> 1) It should be a per_repository_reference test
<gary_poster> flacoste, not sure; I think it is on wiki.canonical.  looking
<jam> since it only needs to be run on repositories that support fallbacks
<matsubara> flacoste, gary_poster: https://wiki.canonical.com/Launchpad/PolicyandProcess/ApiSupport
<gary_poster> yay matsubara, thanks :-)
<matsubara> np
<jam> 2) Could we not hard-code the target format, but use a generic 'incompatible' fallback format?
<jam> that way if we introduce a new Repository, it also will have to conform to this behavior
<flacoste> matsubara: thanks
<jam> (you could either do it by picking a format that is too old for stacking, or by just saying "if 2a, then 1.9, else 1.9"
<flacoste> matsubara, gary_poster: is the comment at the top of the wiki page still true? waiting on the releasing of the OEM scripts?
<jam> else 2a)
<abentley> gary_poster: I'll leave you two to discuss it, but feel free to ping me.
<jelmer> gary_poster: fwiw bzr 2.4 has landed on db-devel and should land on devel at the end of this week, barring any issues that come up during QA
<jam> jelmer: nice. Though it still doesn't hurt us to target 2.3 to start
<gary_poster> jam, re (1): I'll look for that string and see if I can figure out, cool.  re (2): yeah, I thought of something like that but I wasn't quite sure it would be correct.  I'll try that approach then.  Does your (2) negate your (1) or not?
<jam> gary_poster: you should do both
<gary_poster> jelmer, great, thanks.  I intend to make a 2.4 and trunk MP too.
<jam> if you do (2) without (1) then the target format will be something like WeaveFormat which will fail for different reasons.
<jam> gary_poster: we automatically up-merge
<jam> so if it lands in 2.3 it will auto-merge into 2.4 and trunk
<jam> well "automatically"
<jam> it is still requested by a human, but we always do it eventually (everything in 2.3 is in 2.4, etc.)
<gary_poster> jam, (2) and (1) both: ack, thanks.  I'll try it and get back if I have questions
<gary_poster> jam, automatically: ok, cool.  I was thinking it would need to be manual because this function changed files between 2.3 and 2.4
<matsubara> flacoste, not sure how that conversation ended. I requested approval and I think gary took over back then
<gary_poster> (the function that we are changing/testing)
<jam> gary_poster: well, if it needs updating, then we'll need something when we go to merge 2.3 up to 2.4
<gary_poster> jam, right, so just wait to let you all ask for it, or do it yourself?
<jam> checking
<gary_poster> flacoste, matsubara, I think Jane had an issue with the draft and then I never followed up.  I don't remember details.  matsubara, do you have an email from Jane about it?
<matsubara> gary_poster, yes. I forwarded it to you back then. I'll forward it again to you and flacoste
<gary_poster> matsubara, ack thanks
<flacoste> matsubara, gary_poster: thanks, i'll follow-up from there
<jam> gary_poster: it looks like the function in question just got moved (probably to bzrlib/vf_repository.py), I think we can sort that out once your patch has landed.
<gary_poster> ok cool thanks jam
<jam> The general rule is that we would wait, then base a change on the updated bzr/2.3 to merge it into bzr/2.4
<gary_poster> ok
<bigjools> I wonder if people ever look at the dupe finder list when filing bugs
<jelmer> bigjools: I always assume the bug file thingy will suggest dupes to me, if there are any
<bigjools> jelmer: yes.  But lately I've seen stuff that is so obviously a dupe I wonder if people pay any attention to the list
<gary_poster> bigjools, that would involve reading, you know.
<bigjools> :)
<jelmer> bigjools: the list does often contain a lot of irrelevant stuff. Perhaps it shouldn't show anything if there isn't anything significantly similar.
<bigjools> yeah, could be people fed up with unrelated bugs
<nigelb> benji: Hi!
<nigelb> Could you review https://code.launchpad.net/~nigelbabu/launchpad/188187-time-zone-offset/+merge/73168 :)
<benji> nigelb: sure, I'm a little behind so it might be a minute
<nigelb> benji: Sure, no problem :)
<deryck> abentley, I can meet in mumble now, if you'd still like a pre-imp chat.
<abentley> deryck: sure, let's do it.
<deryck> ok, cool
<abentley> deryck: https://bugs.launchpad.net/launchpad/+bug/828409
<_mup_> Bug #828409: cross-format stacked branches can't be accessed <branch-stacking> <codehosting> <Launchpad itself:Triaged> < https://launchpad.net/bugs/828409 >
* benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 244 - 0:[#######=]:256
<nigelb> benji: hi
<nigelb> benji: still around?
 * nigelb needs someone to land his branch
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #828: FIXED in 4 hr 42 min: https://lpci.wedontsleep.org/job/db-devel/828/
<gary_poster> jam, it turns out there was a test very similar to what I had done in per_repository_reference already.  I adjusted it.  https://code.launchpad.net/~gary/bzr/bug835035/+merge/73124 is updated with the change.  You good with that?
<nigelb> gary_poster: would you mind sending a branch of mine to e2 land?
<gary_poster> nigelb, was just about to ask you :-)
<gary_poster> which is it?
<nigelb> heh
<nigelb> https://code.launchpad.net/~nigelbabu/launchpad/188187-time-zone-offset/+merge/73168
<gary_poster> ok, on it nigelb.  I assume we need ec2 tests run?
<nigelb> yes!
<gary_poster> cool
<nigelb> gary_poster: Thank you! :)
<gary_poster> nigelb, welcome :-) instance is firing up now.
 * gary_poster should have added you to the email recipients
 * gary_poster hopes ec2 land does that anyway
<nigelb> Usually, I get emails
<gary_poster> cool
<nigelb> I assme there's nothing special to do.
<nigelb> *assume
 * nigelb marks the time and counts down to 4 hours
<henninge> rvba: review sent, talk to you tomorrow ;-)
<rvba> henninge: I'll have a look right now ... I bet I have quite a reading to do ;)
<henninge> rvba: yes
<henninge> ... and some fixing ;)
<rvba> henninge: ;)
<henninge> ;-)
<henninge> rvba: I am off now and will be late tomorrow morning.
<rvba> henninge: I'll get back to you when I have fixed it so late tomorrow sounds good.
<gary_poster> nigelb, ec2 land has disconnected from my terminal and doing its thing, so hopefully we'll hear something in four or five hours
<nigelb> gary_poster: Cool, thanks!
<gary_poster> jam, I'm going to lunch.  biab, and https://code.launchpad.net/~gary/bzr/bug835035/+merge/73124 is ready for you
<LPCIBot> Project devel build #1,008: STILL FAILING in 4 hr 35 min: https://lpci.wedontsleep.org/job/devel/1008/
<jam> gary_poster: nice that you could use an existing test. I tweaked it slightly because the actual test was being a bit redundant.
<jam> I'll submit it
<gary_poster> great, thanks jam.
<jtv> Any reviewers in the house?  Got this one: https://code.launchpad.net/~jtv/launchpad/bug-836743/+merge/73267
<jam> gary_poster: my tweak to the test is at: lp:~jameinel/bzr/2.3-gary-bug835035 if you want to look at it. It is in PQM now.
<gary_poster> jam, cool, thanks.  I thought about that but was lazy, I guess, since the code & checks of test_add_fallback_doesnt_leave_fallback_locked subsumes test_add_fallback_repository_rejects_incompatible (maybe unless we asserted in test_add_fallback_repository_rejects_incompatible referring was not locked?); but I can see the point of dividing them.
<jam> gary_poster: well a few things
<jam> 1) We don't need to make the 'test' repository
<jam> just test the actual repository we are going to use :)
<jam> 2) Your new test is strictly a superset, yes
<jam> However, I thought it would be good to have an explicit test of the different behavior
<gary_poster> jam, ah, yes, I hadn't noticed # 1!
<jam> gary_poster: yeah, you didn't write that code
<gary_poster> yeah, like I said, I see where you are coming from with #2
<jam> we *do* have to 'make_repository' because RemoteRepository infers its format over the wire
<jam> it isn't known until we query it
<gary_poster> but once is enough
<jam> right
<abentley> lifeless: So I assumed TwistedJobRunner.runJobInSubprocess always returns a Deferred, but sometimes it returns None.  Is it better style to fix the callsite by calling maybeDeferred, or the method by making it always return a Deferred?
<nigelb> lifeless has gone away.
<nigelb> Or at least wgrant tried to get him to go away.
<bac> hi abentley
<abentley> bac: hi.
<abentley> nigelb: :-)
<bac> abentley: would you have time for a call on a weird OOPS re: recipes?
<abentley> bac: Okay.
<bac> abentley: skype ok?
<abentley> bac: sure.
<bac> abentley: it is about bug 828914
<_mup_> Bug #828914: +request-daily-build oops with an AttributeError: 'NoneType' object has no attribute 'published_archives' <oops> <recipe> <Launchpad itself:In Progress by bac> < https://launchpad.net/bugs/828914 >
<adeuring> flacoste: fancy a review? https://code.launchpad.net/~adeuring/launchpad/test-hwsubm/+merge/73294
<flacoste> adeuring: sure, i'm on it
<flacoste> adeuring: just FYI, Lucid is also affected, we have very few valid Lucid submissions, and I guess they date from before the problematic checkbox changes
<adeuring> flacoste: right; I'll check a few lucid reports too
<flacoste> adeuring: do we have hwdb integration tests? If we do, is it worth adding one for a problematic report?
<adeuring> flacoste: right, I am aware that such a test is missing -- the main point is: I am too tired to write one today...
<flacoste> adeuring: but do we have such tests already?
<adeuring> flacoste: only one or two: for correct data and for "trivially wrong data", IIRC
<flacoste> adeuring: ok
<flacoste> adeuring: r=me with the note that Lucid and later are affected, and with a reference to the bug report
<adeuring> flacoste: ok, thanks
<flacoste> how can we have valid person without a preferred email address ?!?
<thumper> flacoste: we can't?
<thumper> or perhaps better put, we shouldn't
<flacoste> thumper: i thought we couldn't actually
<thumper> I thought we couldn't either
<flacoste> seems that this assumptions still hold
<flacoste> i was mistaken by a bad bug comment
<LPCIBot> Project devel build #1,009: STILL FAILING in 4 hr 50 min: https://lpci.wedontsleep.org/job/devel/1009/
 * wallyworld has to go out and run some errands
#launchpad-dev 2011-08-30
* wgrant changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 244 - 0:[#######=]:256
<LPCIBot> Project devel build #1,010: STILL FAILING in 4 hr 5 min: https://lpci.wedontsleep.org/job/devel/1010/
<nigelb> Just when I thought I had great success in not breaking a truckload of tests with an MP, it fails buildbot.
<wgrant> jelmer: https://lpbuildbot.canonical.com/builders/lucid_lp/builds/1313/steps/shell_6/logs/stdio <- your fix yesterday may be flaky.
<wgrant> Forcing.
 * wgrant vomits
<wgrant> httplib2 has its own certificate store.
<wgrant> What is wrong with being sensible :(
<stub> wiki says wgrant or gmb is review bitch. Any chance of https://code.launchpad.net/~stub/launchpad/pgbouncer-fixture/+merge/73283 ?
<nigelb> heh, review bitch.
<wgrant> stub: Hm, changing to TCP connections?
<wgrant> Ah, I see, you don't want to overwrite the config.
<wgrant> We really need to fix lazr.config to not suck for this.
<wgrant> stub: Why are you overriding PGHOST?
<stub> wgrant: Because otherwise it will connect using UNIX sockets, and fail as the pgbouncer package does not (yet) set unix_socket_dir so they don't work.
<wgrant> stub: But you've set host=localhost in the connection strings.
<wgrant> stub: So setting PGHOST=localhost sounds redundant.
<stub> wgrant: I probably won't bother changing it either, as this better represents production
<stub> wgrant: It might be redundant now, yes.
<wgrant> stub: Also, you can hopefully use self.useFixture instead of self.addCleanup and setUp.
<stub> That sounds good.
<wgrant> I think the base Fixture from fixtures should support that.
<stub> Do you want me to pull the PGHOST bit if it is redundant, or leave it since it might be useful in the future if I do what I don't think I'm going to do?
<wgrant> stub: I think you should pull it out.
<wgrant> We have more than 650KLOC already. Let's not add more.
<stub> pypi has terrible google juice
<wgrant> py* has terrible lots of things.
<wgrant> stub: I don't think this is going to interact stunningly with PgTestSetup.
<wgrant>                 # Stash the name we use in the config if a writable config is
<wgrant>                 # available.
<wgrant>                 # Avoid circular imports
<wgrant>                 section = """[database]
<wgrant> rw_main_master: dbname=%s
<wgrant> rw_main_slave:  dbname=%s
<wgrant> Why yes, that is revolting.
 * stub wonders how it is working atm
<wgrant> Possibly PGHOST :)
<stub> ahh... yes :-)
<wgrant> 213	+ # Database is still working.
<wgrant> 214	+ assert self.is_connected()
<wgrant> s/still working/working again/
<stub> Pushing with those changes.
 * stub wonders why the ec2 image doesn't upgrade its own packages
<stub> Anyone around setup to update the ec2 test image?
<stub> Just need launchpad dependency packages updated
<jtv> stub: I thought it did upgradeâ¦ anyway, I should be able to do that.
<wgrant> It doesn't :(
<wgrant> stub: What needs updating? It was last rebuilt a week ago.
<jtv> Meanwhile, does anybody have any idea what the current crop of codehosting buildbot failures is all about?
<stub> Seems simple enough, but I don't want to run an hour long process right at the moment and might not have perms on ec2 either.
<stub> wgrant: Just pull in the newly build launchpad dependencies
<wgrant> jtv: I poked jelmer about them, as they might be related to the stuff he's landed recently.
<stub> c/build/built
<stub> launchpad-developer-dependencies
<jtv> wgrant: he may have landed something unfinishedâ¦ I also noticed a lot of recent lint in his name.
<wgrant> stub: pgbouncer needs a list of users to let through?
<stub> wgrant: yes
<wgrant> stub: What about *_ro and read?
<wgrant> stub: Or are we just going to hope no test that uses them ever goes near the fixture?
<stub> wgrant: I'll need to add them too.
<jtv> StevenK: your AMI rev. 519 is the current, valid image?
<wgrant> jtv: He's on leave, but yes.
<wgrant> It was created last week.
<jtv> thanks
<stub> read isn't a user though, so don't need that one added.
<jtv> BTW, why do we use a 64-bit AMI?  I can't imagine any of our test processes needing more than 3GB address space, and I thought x86 was likely to be faster than amd64 for python.
<stub> Because we run 64bit on production
<wgrant> Right, production is mostly amd64.
<wgrant> i386 tests are quite a bit faster, because RAM usage is roughly halved.
<wgrant> So you get tonnes of extra cache.
<jtv> wgrant: looking into gina domination a bit more.  Do I understand correctly that we want to mark SPPHs that have been superseded by newer publications of the same package as superseded, even if the package has subsequently "fallen out of" the Debian Sources lists?
<wgrant> jtv: Ideally. But that's probably only a problem for the first time, so we could do it as a one-off.
<jtv> Well it happens to be what the dominator already does.  It's just buried among heaps of other code.
<jtv> It also looks like it could be made much faster if it had to.
<jtv> wgrant: by the way, it looks like you're OCR today.  In which case: https://code.launchpad.net/~jtv/launchpad/bug-836743/+merge/73267 plz?
<wgrant> The archivepublisher dominator should take <5s per context now.
<wgrant> Once we finish denorming SPN onto SPPH it will be trivial time.
<wgrant> jtv: However, we already know what's not published any more, so I think using the dominator in the normal case is pointless.
<wgrant> jtv: Looking at that review now.
<jtv> Except we still need domination doneâmight as well.
<jtv> Thanks.
<wgrant> jtv: Why?
<wgrant> Apart from the initial run, the algorithm is as I outlined in the bug.
<jtv> Because we don't want to keep keeping Debian SPPHs in unjustified active statuses.
<wgrant> The dominator will not match what Debian does.
<jtv> In what way?
<wgrant> Debian does not handle superseded packages quite as Ubuntu does.
<wgrant> Superseded sources are kept around in the indices until they are unreferenced.
<wgrant> By binaries, that is.
<jtv> Yes, but doesn't that mean they're still superseded?
<jtv> There is a difference between "superseded" and "deleted" after all.
<wgrant> That's something that is not clear.
<wgrant> Julian argues that anything in the indices is by definition not superseded.
<LPCIBot> Project devel build #1,011: STILL FAILING in 4 hr 7 min: https://lpci.wedontsleep.org/job/devel/1011/
<wgrant> Soyuz currently works under the assumption that (in the indices == published).
<wgrant> I've argued that that should be changed, to allow us to gain Debian's enhanced supersededness stuff. But I don't think now is the time to be violating or changing that assumption.
<jtv> Isn't that what we have pockets for?  I imagine it's a valid assumption for ubuntu exactly because we don't do it that way.
<wgrant> We want to do it that way.
<wgrant> There's a critical bug for it.
<wgrant> It's unrelated to pockets.
<wgrant> jtv: Branch looks good.
<jtv> Thanks.  So in the future we want to keep multiple SPPHs "published" for the same source package name, distroseries, and pocket?
<wgrant> We want to keep multiple SPPHs with the same context (archive, distroseries, pocket) and name in the indices. It's not clear whether that means we should leave the status as Published.
<wgrant> I've traditionally argued that we shouldn't leave the status as published, but it has been a point of significant contention.
<wgrant> jtv: Does anybody use launchpad_langpack on staging these days?
<wgrant> # - Run Langpack ((re)-create a langpack DB that the
<wgrant> #   translations team can use for testing.
<jtv> That's very very old, and we used to use it even for production tarballs sometimes.  Today, we may still want it for Q/A.
<wgrant> That sounds within tolerances of "no".
<jtv> I don't know because I've been on Soyuz feature "rotation" since two interglacials ago.
<wgrant> Megafeature :)
<jtv> Scope creep.  It's the difference between your burndown rate and your burnout rate.
<wgrant> Well, it also should have been split into at least three separate features from the start :(
<jtv> In a way, it was.  They just kept being piled into the same feature rotation.
<wgrant> Well, yes, that's what I meant.
<jtv> Anyway, about the superseding of source publications: what would determine for Ubuntu whether a package is still in the Sources list?  Where would that information be kept?
<wgrant> datepublished IS NOT NULL AND dateremoved IS NOT NULL
<wgrant> Actually, probably datepublished IS NOT NULL AND datemadepending IS NOT NULL
<wgrant> (datemadepending is the date it was scheduled for deletion, not the date it was made pending, because 2005 hates us)
<adeuring> good morning
<jtv> hi adeuring
<adeuring> hi jtv!
<jtv> wgrant: it sounds to me then as if domination should be fixed to handle that uniformly for debian, ubuntu, and derivatives; and gina should merely be updated to set those date fields.
<wgrant> jtv: Possibly, but that is feature scale work.
<jtv> Possibly, but then I don't see why I should solve that problem for just Debian right now in a short-term branch.
<wgrant> I don't think it makes sense to run a dominator over an imported archive, since it is archive publishing logic. The imported archive has the information we need right now. When we remove the (in indices <=> published) invariant, that may have to change.
<jml> http://flossmole.org/content/everything-you-ever-wanted-know-about-software-forges-code-forges-june-2011
<jml> interesting link
<nigelb> that's a long URL.
<nigelb> (twss, yeah I know)
<nigelb> Ooh, interesting cloud.
<wgrant> LP apparently doesn't have announcements.
<nigelb> It does. Right?
<nigelb> The RSS thing that pops in the right corder of a project.
<wgrant> Yes.
<wgrant> Intriguingly, Trac is included as a feature in the feature matrix.
<jelmer> wgrant: hi
<wgrant> Hm, I thought we still provided DOAP.
<wgrant> Maybe we dropped that.
<mrevell> Hello
<nigelb> Wasn't someone talking about having javascript OOPSes?
<nigelb> I just saw this http://errorception.com
<nigelb> I don't know how it works, but apparently some kind of OOPS system. Paid for though :|
<jml> mrevell: hi
* gmb changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: gmb | Critical bugs: 244 - 0:[#######=]:256
<jelmer> wgrant: it looks like the failing test was one that was flapping earlier, which supposedly should be fixed by the introduction of the newer Twisted
<wgrant> jelmer: Yeah, I thought it could have been that :(
<wgrant> But since you'd touched that area recently, I thought I might as well see if you had any thoughts or fault :)
<bigjools> wgrant: thanks for the utf8 fix
<bigjools> not sure wtf the extra blank lines are coming from but it's not exactly important
<wgrant> bigjools: The iterator probably stops at the start of the next entry.
<wgrant> bigjools: You can probably sensibly just call strip() on the changelog string.
<bigjools> as for the person/date line... that would need a change in python-debian methinks.
<bigjools> yeah, I was considering using strip but it might also be in the template
<wgrant> bigjools: python-debian doesn't give you access to the various components of the entry?
<wgrant> bigjools: gina does it somehow.
<wgrant> Although it has its own parser.
<bigjools> gina has its own parser
<wgrant> Heh.
<bigjools> it's do-able, but the right place is in python-debian
<bigjools> aha
<bigjools> I can do block.changes() I tihnk
<wgrant> bigjools: You could also set _no_trailer on the block.
<wgrant> Then __unicode__ won't include that line.
<wgrant> Hah.
<bigjools> unicode?
<wgrant> Rather than using the utf-8 stuff, use unicode() instead of str()
<wgrant> Its __str__ just calls __unicode__
<bigjools> unicode... what unicode :)
<bigjools> which python-debian do you have?
<wgrant> Ah, I'm on oneiric.
<wgrant> Let's see lucid...
<bigjools> ;)
<wgrant> Package `python-debian' is not installed.
<wgrant> Huh.
<wgrant> Oh.
<wgrant> We still use the one in sourcecode/
<bigjools> :(
<wgrant> We should update that and use proper unicode :)
<wgrant> Although that one does still support _no_trailer.
<bigjools> I love the way that the API expects you to poke private variables to change behaviour
<wgrant> You're not meant to poke them from the outside.
<wgrant> We're just doing unexpected things.
<wgrant> I'm not sure about your use of _raw_version either, but meh.
<bigjools> well, quite
<bigjools> the api is a bit rubbish
<bigjools> looks like I can just set _no_trailer then
<bigjools> wgrant: instead of decoding the utf8 in fetch_information it seems to make more sense to do it in aggregate_changelog
<wgrant> bigjools: Probably.
<wgrant> bigjools: I just did it there to get it unbroken.
<bigjools> np
<henninge> AFICT wer are in a good position for a stable deploy, aren't we?
<henninge> both rollbacks have landed.
<wgrant> henninge: Yes.
 * henninge goes about starting that.
<wgrant> Once danilos' QA is done.
<danilos> wgrant, was it not done?
<wgrant> Not a few hours ago when I last checked the deployment report.
<henninge> danilos, wgrant: all green
<wgrant> So it is.
<nigelb> hmm, I should do QA!
<wgrant> doit
<nigelb> or should I?
 * nigelb didn't didn't see a tag show up.
<wgrant> nigelb: It should be there in 20 minutes or so.
<wgrant> Actually, I'm behind. May have been there 10 minutes ago...
<nigelb> Ahh.
<wgrant> Revision 13818 can not be deployed: needstesting
<wgrant> Assignee: nigelbabu
<nigelb> *whee*
 * wallyworld just got power back on after a 90 minute power outage :-(
 * nigelb QAs
<nigelb> FUUUUU
<nigelb> I wasn't subscribed.
<nigelb> Why does assigning the bug to myself not auto-subscribe me :x
<wgrant> You won't be explicitly subscribed, but you'll still get notifications.
<nigelb> Ha, 13 minutes ago.
<bigjools> wgrant: can you think of any good reason why I should not make makeChangelog return a unicode changelog?
<nigelb> No wonder.
<wgrant> bigjools: No.
<wgrant> bigjools: There's little place for bytestrings in LP.
<bigjools> good answer :)
 * bigjools does the file encoding trick
<wgrant> Hm?
<bigjools> # -*- coding: utf-8 -*-
<wgrant> Why?
<StevenK> That's an emacs hint
<bigjools> no it's not
<nigelb> isn't that a vim hint?
<wgrant> Python hint.
<bigjools> it's a PYthon hint
<wgrant> That's the critical bit.
<wgrant> However, it's probably overkill.
<wgrant> What do you think you are up to?
<nigelb> QA'd
<bigjools> I've had this conversation before
<bigjools> If I want to type human-readable unicode in Python source I need that
<wgrant> bigjools: Oh, this is for a test?
<wgrant> I guess that's OK then.
<bigjools> in factory.py
<bigjools> but yes
<bigjools> see the top of test_ppauploadprocessor.py for example
<lifeless> wgrant: A hint for you with fixtures: useFixture is the bomb.
<wgrant> lifeless: Didn't I just suggest that to stub? :)
<lifeless> I may have missed that
<wgrant> 15:25:55 < wgrant> stub: Also, you can hopefully use self.useFixture instead of self.addCleanup and setUp.
<lifeless> I was going off of the MP thread.
<wgrant> I am not guilty.
<wgrant> Ah.
<wgrant> I felt some of the issues needed discussion, so did it on IRC instead.
<lifeless> something I like doing after IRC discussion is to paste that into the MP
<wgrant> Probably should have, yeah. Will do so in future.
<wgrant> bigjools: You prefer using coding: utf-8 over \N{SNOWMAN}?
<bigjools> hell yes
<wgrant> Bah.
 * jelmer wonders how you would type non-ascii characters on a UK keyboard
<lifeless> â¯
<nigelb> buy a new keyboard!
<nigelb> Actually there's a character map thing somewhere you can use :)
<wgrant> jelmer: Compose keys solve everything.
<wgrant> Except for in oneiric, where I can't work out how to configure one...
<wgrant> GNOME 3 wins.
<jelmer> OTOH, the special keys handling dialog in GNOME2 excelled in being complex and confusing...
<wgrant> Yeah.
<wgrant> But I can't find anywhere to configure keyboard layouts at all any more.
<cjwatson> it *was* originally an Emacs hint.  Python just appropriated it.
<wgrant> In fact, the new gnome-control-center seems to take GNOME policy to the extreme and delete most of the options.
<jelmer> wgrant: yeah
<wgrant> Or perhaps Ubuntu's Language Support button has replaced the GNOME 3 regional settings one or something.
<nigelb> Meh. I should just stop. Not helping.
<jelmer> nigelb: hmm?
<wgrant> #launchpad, I presume.
<nigelb> YEah.
<wgrant> cheater is right, our UI is terrible, and attempting to argue otherwise is probably silly :)
<wgrant> LP was designed in a different era, and was never pulled out of it.
<nigelb> heh, I agree with that. BUt its not /that/ terrible.
<wgrant> It is to people who haven't used it before.
<wgrant> And even to those that have it is cluttered and awkward.
<nigelb> Also, I'm trying to say how the UX is different from other similar services.
<wgrant> And does not provide dashboards or anything else that contemporary social applications have.
<nigelb> I failed at that spectacularly.'
<wgrant> It is a good point, but difficult to argue successfully.
<nigelb> What would it take to build a dashboard like approach?
<jelmer> dashboards would be really nice, especially given we track not just stuff on launchpad but elsewhere as well.
<wgrant> Ahahah
<nigelb> Besides lots of crying.
<nigelb> btw, no StevenK these days?
<wgrant> StevenK is away this week.
<nigelb> Or is he using perl to delete large parts of launchpad in secret.
<wgrant> Well, not away, but not meant to be here.
<nigelb> wgrant: Like you were a few weeks back.
<wgrant> Shh.
<nigelb> Bad metaphors need to be shot down.
<jml> nigelb: shot down from where?
<StevenK> jml: WRT Facebook and Google+, congrats!
<jml> StevenK: thanks :D
<nigelb> jml: oh yeah! Congrats!
<jml> nigelb: cheers :)
<StevenK> jml: Pictures of the ring are *required*
<nigelb> jml: I'm adding my comments to the discussion in the other channel.
<nigelb> StevenK: haha, I just realized, I did pictures of the ring with which you proposed :D
<StevenK> nigelb: You did?
<jml> nigelb: was being witty, "shot down" is itself a metaphor
<nigelb> jml: haha. Damn. I fail at my own joke :)
<jml> or maybe that's what you meant
<nigelb> StevenK: Yeah, #ubuntu-women ^-^
<nigelb> back then.
<bigjools> I need unicode help :(
<bigjools> whenever I have to do unicode in Python, I get frustrated
<bigjools> wgrant: do you know the appropriate runes to get makeChangelog working with a unicode changelog? the librarian addFile keeps blowing chunks on me
<jml> bigjools: Python makes it really hard.
<bigjools> jml: it does :(
<wgrant> bigjools: You'll need to encode the changelog before you pass it to makeLFA
<jml> bigjools: you should see the insane stuff mgz has done to get testtools have proper unicode handling.
<bigjools> wgrant: I am, it still bitches
<wgrant> bigjools: Bitches how?
<bigjools> UnicodeEncodeError: 'ascii' codec can't encode character .... blah
<wgrant> jml: Python 3's sort of much better, though.
<wgrant> bigjools: What is your .encode invocation?
<bigjools> jml: I can well imagine
<jml> wgrant: yeah. it's the cross-Python support that makes it hard. Would be better with actual real honest-to-goodness types, and banning the word "string"
<bigjools> content=changelog.decode("utf-8")
<wgrant> bigjools: encode.
<bigjools> ffs
<wgrant> jml: Yeah, I think they designed stuff like the bytes backports to 2.6 just to make everyone's lives hell.
<wgrant> I use "backports" loosely.
<jml> heh. yeah.
<wgrant> I end up mostly resorting to casting everything to bytearray instead.
<wgrant> Which actually has consistent behaviour...
<bigjools> this is just utterly confusing
<wgrant> bigjools: Oh?
<wgrant> bigjools: You were going completely the wrong way before.
<bigjools> encode/decode/unicode etc
<wgrant> That's not Python, it's just people being stupid and not using Unicode everywhere.
<bigjools> I'm confused about whenI need to use encode and decode
<wgrant> If your data is not binary or UTF-8, you are wrong.
<wgrant> bigjools: Librarian files are bytestreams. You need to encode before you put data into an LFA, and decode once you get it out.
<wgrant> Because Python 2 doesn't do text files.
<wgrant> jml: How's the world of app development, anyway?
<wgrant> Hopefully not too much Python unicode support.
<jml> wgrant: fun! I have a spec :)
<jml> wgrant: https://blueprints.launchpad.net/developer-portal/+spec/automagic-binary-packaging
<jml> unicode support only matters insofar as a testtools release would help me and many others at Canonical get things done faster
<wgrant> This reminds me... I must work out what is up with poppy-sftp's logging tomorrow.
<wgrant> bigjools: Have you touched that at all?
<wgrant> bigjools: It's completely screwed.
<bigjools> wgrant: no, not at all
<nigelb> So, codeswarm for Launchpad was fun watching.
<wgrant> bigjools: Normally doesn't log anything, except when you log something explicitly, then it starts logging everything 50-60 times.
<wgrant> and you can actually see exceptions that occur.
<wgrant> After three recursion depth exceeded errors in the log handler.
<bigjools> I remember that happening when I was adding ftp support, then it went away
<bigjools> never got to the bottom of it
<nigelb> jml: Hows your day job? :)
<StevenK> nigelb: Do you have it for Launchpad?
<wgrant> jml: So we're going for widespread package-all-the-things before we have sandboxing, or are we hoping sandboxing will be practical before this is mature?
<nigelb> StevenK: Yes.
<StevenK> nigelb: Can haz?
<nigelb> StevenK: http://www.youtube.com/watch?v=9Pe70fw83Zs
<jml> wgrant: more the latter
<nigelb> Lots of lifeless floating around ;)
<jml> wgrant: for commercial apps, which is my focus for now, we are already operating without sandboxing in much the same way that we do with partner or the things currently in the software center
<wgrant> jml: Yeah, but I don't think that can scale while maintaining responsiveness.
<jml> wgrant: agreed.
<nigelb> jml: I love your work, except for the bits where its limited to one arch :(
<nigelb> (well, for now)
<jml> nigelb: we're moving up to two!
<nigelb> oooh \o/
 * bigjools weeps at having 4 cores which doesn't help when everything bottlenecks on disk
<jml> wgrant: I think sandboxing is the next step.
<wgrant> jml: It should have been the next step for about 15 years, but yeah :)
<nigelb> heh
<wgrant> bigjools: tmpfs + LXC + 6 parallel test runs makes good use of cores.
<cjwatson> sandboxing's already nearly practical - see arkose
<cjwatson> (I'm sure jml knows about that already)
<wgrant> Yeah, it's getting there.
<jml> yeah, I do.
<bigjools> wgrant: I used to run parallel tests using kvm but really, everything bottlenecks on disk.
<cjwatson> it's good enough for skype, which is pretty damn good
<jml> Although haven't had the play with Arkose that I'd like to.
<cjwatson> (the above is not to be taken to mean "skype is pretty damn good")
<bigjools> I know tmpfs is supposed to help but I still struggled to speed it up
<jml> Might do that today, actually, since most of it's consumed by long weekend email recovery
<nigelb> cjwatson: I can see the tabloids "Ubuntu developer says skype is damn good!"
<wgrant> bigjools: Which is why I implemented the LXC-based system that lifeless and I have tested. Have a single base chroot, then start 6 COW guests using aufs on tmpfs.
<wgrant> bigjools: All writes go to tmpfs, and the chroot cache is shared between all of them.
<wgrant> It was entirely CPU bound when I tried it.
<bigjools> excellent
<nigelb> I need a new machine.
<nigelb> 2 cores is now old.
<bigjools> my 4 core is a Phenom, a bit old
<nigelb> Buddy of mine got an 8-core. Runs gentoo one it and says he has great build times.
<bigjools> the i5 feels quicker, even though it only has 2 (although has the pseudo extra 2)
<nigelb> (I repllied with "I use Ubuntu, no build time on my laptop at all!")
<bigjools> gentoo users are weird people :)
<nigelb> haha
<nigelb> jml: that spec looks scary - so.many.items!
<jml> nigelb: I like having lots of small items.
<jml> nigelb: gives me illusions of control and progress.
<nigelb> Ah, they're small.
<wgrant> Far easier to feel that you're making progress that way.
<nigelb> I have like one item from UDS that's going to take 2 cycles to do.
<nigelb> Hmm, https://myapps.developer.ubuntu.com/dev/ looks slick. Except I had to read about it on Rick's blog.
<nigelb> I wish ISD blogged on the planet.
<G> anyone got a second to talk about a bug, and a possible bug?
<G> err a possible patch that is :)
<nigelb> "patch" - the magic word!
<nigelb> G: Actually, you should just talk about it and someone will respond
<G> nigelb: yeah, well my fingers were a timezone behind my brain :)
<nigelb> I know that feeling well :)
<G> Anyway, I spotted https://bugs.launchpad.net/launchpad/+bug/837290 last night and thought "that looks wrong", I took a look/poke at the Javascript tonight, and think I have isolated the issue
<_mup_> Bug #837290: When there is no "Other bug subscribers" incorrect descriptive text is displayed <Launchpad itself:New> < https://launchpad.net/bugs/837290 >
<G> What I seem to have isolated it to, is in lib/lp/app/javascript/subscribers/subscribers_list.js the function combines a set of config variables passed to it, with defaults if not set elsewhere
<G> (~ line 75 of current devel branch)
<G> but the SubscribersList function/module gets called using the uncombined config parameters
<G> which results in items such as 'subscribers_label' ending up, undefined
<nigelb> That's a real narrow situation. Nice catch :)
<bigjools> wgrant: can you sanity check please: http://pastebin.ubuntu.com/677885/
<G> nigelb: my thought exactly :)
<G> I came up with http://pastebin.ubuntu.com/677887/ as a working patch
<wgrant> bigjools: I guess.
<nigelb> G: I don't know if it works, but if you propose a merge, you can ask the oncall reviewer to look at your MP and review/land.
<G> I'm going by https://dev.launchpad.net/FixBugs which says discuss first
<nigelb> mmm
<nigelb> I forgot about that one.
<G> but yeah, I was planning on going the MP route, but wanted to cehck my logic first
<wgrant> bigjools: I don't really like making all the tests unicode like that, but I guess it's fine :)
<bigjools> <wgrant> If your data is not binary or UTF-8, you are wrong.
<jml> <http://www.joelonsoftware.com/articles/Unicode.html>
<bigjools> "Did you ever get an email from your friends in Bulgaria with the subject line "???? ?????? ??? ????"?"
<bigjools> lol
<wgrant> bigjools: By "making all the tests unicode" I mean "using non-ASCII characters just because, which makes the tests ugly"
<bigjools> if it sweeps out decode errors in production, I don't care!
<G> the other thing I'm wondering is if lines 79 & 81 are actually needed at all because everything seems to be 'config.something' not 'this.something'
<wgrant> bigjools: Still, whenever I see bad Unicode code in LP, I just think to myself "at least it isn't lp.services.encoding.ascii_smash()", and then I feel better.
<G> (actually, ignore that last line)
<wgrant> bigjools: (bug #362957)
<bigjools> ascii_smash is ....
<wgrant> Broken and has no cause to exist.
<bigjools> the bug reporter's name made me laugh.  I think I am a bit evil.
<wgrant> MIME solved Unicode in emails in 1996 :/
<wgrant> Heh.
<wgrant> I haven't seen him around in a while.
<wgrant> Hmm, only archiveuploader and soyuz.adapters.notifications use ascii_smash.
<wgrant> Maybe we can delete it.
<bigjools> it should be easy
<wgrant> def safe_fix_maintainer(content, fieldname): """Wrapper for fix_maintainer() to handle unicode and string argument.
<wgrant> It verifies the content type and transform it in a unicode with guess() before call ascii_smash(). Then we can safely call fix_maintainer(). """
<wgrant> Convert to Unicode just so you can ascii_smash it.
<wgrant> YAY
<cjwatson> I filed a bug about that about four years ago :)
<cjwatson> I mean, about ASCII-smashing on uploader names in upload announcements
<wgrant> Bug #33137
<wgrant> https://bugs.launchpad.net/launchpad/+bug/33137
<cjwatson> it got unfixed
<wgrant> I think for Changes it's still fixed.
<cjwatson> see e.g. https://lists.ubuntu.com/archives/oneiric-changes/2011-August/005861.html
<wgrant> Only for addresses is it still insane.
<bigjools> mails from syncs should be ok
<wgrant> bigjools: lol
<bigjools> ?
<bigjools> there are tests to preserve the From address
<wgrant> bigjools: They still use the ascii_smash codepath.
<bigjools> hmm it must be a weird corner case then
<wgrant> cjwatson: You have to remember that the LP team will only do the absolutely minimal fix.
<wgrant> cjwatson: Not the sensible one.
<wgrant> :)
<bigjools> wgrant: that is not a helpful remark
<wgrant> Probably not.
<wgrant> But it is hard to make helpful remarks when ascii_smash exists.
<wgrant> Because the only conceivable reason for its existence was rendered invalid 8 years before LP was begun.
<cjwatson> I'm sure I would be a lot more annoyed about it if it mangled my own name.  Part of the problem is probably that the Latin-alphabet ratio in Ubuntu development is too high.
<wgrant> Yup.
<wgrant> rvba has been useful.
<wgrant> With his Unicode name breaking new Soyuz features :P
<bigjools> heh
<wgrant> Previously we had to rely on danilos.
<nigelb> I should write my name in my mother tongue just to break LP.
<jml> wow
<jml> working with the LP bzr branch is really different when you are only casually dipping in
<cjwatson> One of our proposed baby names contains an accent; I expect to get more annoyed with broken forms and the like
<nigelb> heh
<bigjools> in composite changelogs, how many blank lines, if any, do we want to keep once the trailer line is removed?
<wgrant> charlieS has been wandering around with a snowman in his LP name for a while now.
<wgrant> https://launchpad.net/~cschluti
<wgrant> bigjools: There should be one blank line between each entry.
<danilos> wgrant, cjwatson :)
<nigelb> wgrant: Excellent!
<poolie_> jml: ha ha
 * bigjools curses python-debian
<wgrant> bigjools: So you probably need to drop the trailing and one of the lines.
<cjwatson> Why do we remove trailer lines in composite changelogs anyway?
<danilos> wgrant, actually, I think charlieS is snowman-ready to test the SSO bug they have for our RT systems login (they are not sending the full name over anymore, but I suppose they'd like to)
<nigelb> jml: Now you know how people like me feel :P
<bigjools> cjwatson: that is an extremely good question
<wgrant> danilos: Ah, but he must have changed it in LP too.
<jml> wgrant: $ bzr grep ascii_smash | wc -l
<jml> 29
<wgrant> danilos: Or maybe even he is confused by login.launchpad.net :(
<wgrant> jml: Mostly tests.
<danilos> wgrant, right, could be :)
<jml> wgrant: 29 is a small number. There art thou happy.
<nigelb> jml: I'm curious though, what bits feel different?
<wgrant> jml: It still does not inspire confidence that such a thing existed in the first place :(
<jml> nigelb: the elapsing time
<cjwatson> jml: you have to spend half an hour every time fixing your test setup?
<bigjools> cjwatson: in fact as someone who looks at the emails from syncs, do you want to see them with or without trailer lines for each revision?
<jml> cjwatson:  yeah, that too.
<jml> Also, caching files into memory
<wgrant> I think I'd prefer them with trailers.
<nigelb> jml: Heh :)
<jml> And bzr taking quite some time to download stuff
<wgrant> But it's unconventional and might make parsers angry.
<nigelb> I think I've accept that I'll take 4 hours to write a patch
<cjwatson> hm, I'm not sure, the counterargument is that dpkg-genchanges -v wouldn't include the trailers
<nigelb> and about 12 hours to write a test.
<wgrant> cjwatson: I think that's the only reason not to include them.
<cjwatson> I think e-mails from syncs should omit the trailer lines but representations like +changelog should include them
<cjwatson> probably
<bigjools> dpkg-genchanges output was the original argument for omitting them
<wgrant> We really should replace +changelog with the real changelog, now that we have it.
<bigjools> yes
<cjwatson> I wouldn't be desperately sad if they were included everywhere, but wgrant may be right that the odd thing might get confused
<wgrant> It's only a few lines to get rid of them.
<wgrant> So let's get rid of them.
 * bigjools gets rid of them
<bigjools> now, htf to omit this blank line too
<wgrant> You could strip the chunks and "\n\n".join()
<wgrant> If python-debian isn't being convenient.
<bigjools> gross, but yes
<wgrant> Or just rstrip and + '\n\n' or so.
<bigjools> today is going to need a lot more caffeine
<poolie_> mrevell, wow, interesting post from cheater
<poolie_> something to get started with :)
<wgrant> poolie_: There was some discussion with him in #launchpad earlier.
<poolie_> yeah i saw the quotes
<mrevell> poolie_, I haven't read that yet ... /me dives in
<nigelb> I find parts of it depressing.
<poolie_> it is a bit
<poolie_> in various ways
<poolie_> some of them being that we saw the problem at the time
<poolie_> on the other hand it's a lot of free moderately good user feedback
<nigelb> heh
<poolie_> it's possibly more efficient than him filing 50 bugs
<nigelb> Well, now we'd file the 50 bugs instead of him :P
<nigelb> Hoping, of course, that some of them get fixed in the near future.
<jml> oh that reminds me
<jml> oneiric feedback :(
<poolie_> heh
<nigelb> oneiric feedback?
<wgrant> Argh.
<wgrant> Why do I ever try to fix Unicode stuff in Python 2.
<wgrant> The implicit str<->unicode conversions suck.
<jml> nigelb: I'm running oneiric. It's critically buggy in about a dozen major ways for me. I should file those bugs.
<nigelb> jml: Ahh.
<wgrant> Unity has only crashed once today!
<wgrant> It didn't much like me opening Character Map.
<nigelb> At some point I should upgrade to something with Unity.
<nigelb> I'm still on Lucid and Maverick.
<wgrant> nigelb: You're running maverick?
<wgrant> Hah.
<mrevell> poolie_, On the positive side, some of the things he raises are pretty easy to fix. I've been reading a book, About Face, which is about goal-oriented design. The issues cheater mentions fall under what the book refers to as "implementation model" design ... i.e. the UI arises from how the thing is engineered, rather than from the user's goals. Such an observation isn't new, but I think we can solve a tonne of cod
<mrevell> e.lp's problems by saying, "Right, person A wants to host code ... what's the least work they need to do to get up on LP" and then making everything else optional, something you can do later if it's necessary.
<wgrant> mrevell: Very, very strongly agree.
<nigelb> Amen.
<nigelb> Its not very apprent how to upload code coming from github.
<wgrant> mrevell: It's a huge shift from the last 7 years of UI. But I think it's the way the world has gone.
<nigelb> That I need a project and then add to the project.
<mrevell> So, perhaps we have a simple, default, path where you push something up and it lands in a renamed +junk (+personal?) and we allow merge proposals on +junk branches ... that'd solve some of what cheater mentions.
<cjwatson> nigelb: well, technically you don't, but then you have to put up with your branch being called +junk.
<wgrant> mrevell: We've long wanted an easy way to promote junk to a project.
<nigelb> cjwatson: Yeah. I don't like LP calling my stuff +junk :)
<nigelb> (GAH, no pun intended)
<wgrant> The modern DVCS hosts tend to get around this by not having projects.
<wgrant> Everything is namespaced under a team/person.
<nigelb> cjwatson: I would like LP to say something like.. "Want to start a project? Go here!"
<nigelb> Or something.
<mrevell> wgrant, I think that'd be throwing the baby out with the bathwater, in our case. Projects are a really strong plus point of LP but maybe, in code's case, they shouldn't be compulsory.
<mrevell> I didn't know what "junk" meant in America when we introduced +junk
<wgrant> mrevell: Right, but it makes LP projects seem heavier-weight.
<wgrant> mrevell: Because they are globally namespaced.
<wgrant> And indeed, they sort of are.
<wgrant> I'm not sure how to fix that without fundamentally changing how LP works, which would imply losing the single project community stuff.
<nigelb> I'm not sure current LP users want that changed.
<mrevell> wgrant, Wouldn't being able to promote a branch from +junk to a project solve it? Or am I missing something?
<wgrant> mrevell: That's still a big leap.
<mrevell> wgrant, It is?
<mrevell> technically or conceptually?
<nigelb> I would also like to see code.lp.net telling me how to statr a project.
<wgrant> Conceptually.
<wgrant> You're turning something that was local to you into something that is global.
<wgrant> And more permanent.
<wgrant> And more imposing.
<jelmer> ... especially as we don't have a way for users to delete a project
<wgrant> In other sites there is no leap there, because everything is equal.
<wgrant> jelmer: Exactly.
<nigelb> Argh. Really?
<jml> also, sharing is a big deal
<nigelb> I've never had to do that yet.
<nigelb> Oh, I can't rename a project either, can I?
<jml> licence, commit privs, contributions, etc.
<mrevell> I haven't thought about the consequences too much but what about personal branches that are associated with a project but still viewed as and controlled by the owner, just like a +junk branch that has an affiliation.
<bigjools> wgrant: do we ever need that trailer or is it always redundant?
<wgrant> bigjools: For the current callsites (all one of them) it is redundant.
<bigjools> particularly on acceptance and rejectio nemails
<bigjools> there is more than one call site
<wgrant> Well, only one conceptual callsite.
<wgrant> Notifications.
<bigjools> yes
<wgrant> And they should be uniform.
<wgrant> While announcements are all that are likely to be machine-parsed, we should keep it consistent, as they have been traditionally.
<wgrant> mrevell: How is that different from what we have now?
<wgrant> mrevell: Branches within a project are still controlled by the owner.
<bigjools> ok thanks
<wgrant> mrevell: It becomes a much smaller leap if we make project registration not scary,.
<mrevell> wgrant, Well, sure, so that's where perhaps I'm missing something what you're saying. I guess my suggestion was as a looser affiliation between branch and project.
<wgrant> mrevell: But it is still a significant leap that the other sites don't have.
<mrevell> wgrant, Hmm ... project registration should, perhaps, be emergent from other "stuff" that's on LP. Maybe.
<wgrant> You go from being a repository for code to launchpad.net/launchpad
<wgrant> From relative simplicity to a bombardment of terms and clutter.
<wgrant> Most of which cannot be disabled.
<wgrant> The "Configuration Progress" thing was a good idea.
<wgrant> But the implementation is very confusing.
<wgrant> And doesn't go far enough.
<mrevell> wgrant, We can make the project registration process, etc, much friendlier without fundamentally changing how LP handles projects.
<wgrant> mrevell: Yes.
<wgrant> But it would be nice to be able to get rid of the leap.
<wgrant> But i don't think we can, without the fundamental change.
<wgrant> Which I really, really don't want.
<wgrant> I don't think anybody does :)
<mrevell> :)
<wgrant> I guess the first step is to make project registration not suck. Then we can think about solutions for going further.
<wgrant> Defaulting LP features to off, with a nice simple initial project page, which walks you through maybe turning on additional features as you need.
<mrevell> If we normalise the idea of personal branches ... so take away the pejorative name, make such branches more like a first-class citizen in LP, that would help.
<wgrant> So it doesn't seem 1000x more complex than a +junk branch.
<mrevell> agreed wgrant
<G> gmb: I noticed you are the current reviewer 'on call' do you have a moment?
<gmb> G: Sure. How can I help?
<G> gmb: basically I spotted an issue in LP and I want to run it by someone in the know so I do it right the first time
<wgrant> mrevell: I think junk's a good name, and other projects have used it for a very similar purpose, but as we've seen it does not seem to be well-received by some :(
<G> gmb: I gave an outline an hour ago, but I'm happy to go over it again if you don't want to scroll back
<wgrant> Not sure what to replace it with.
<mrevell> wgrant, As an aside,  project registration sucks for a reason ... a kinda bad reason ... but to discourage mistaken or frivolous registrations.
<gmb> G: Okay, sure. What's the deal, then?
<wgrant> mrevell: Yeah.
<wgrant> mrevell: Although it's better than it used to be.
<G> gmb: okay, bug in question is: https://bugs.launchpad.net/launchpad/+bug/837290
<_mup_> Bug #837290: When there is no "Other bug subscribers" incorrect descriptive text is displayed <Launchpad itself:New> < https://launchpad.net/bugs/837290 >
<wgrant> mrevell: A bit of a workflow, rather than a 5 page form.
<StevenK> mrevell: Making it easier for registry-admins or LOSAs to delete/hide projects, then?
<mrevell> wgrant, I'd suggest either "+personal" or we go the whole way and simple allow branches at the /~ level, without a workaround project
<StevenK> Sigh. s/Making/Make/
<gmb> G: Okay, I'm with you.
<G> gmb: basically, from what I've managed to find from looking at the javascript, and a bit of fiddling, is that SubscribersLoader in lib/lp/app/javascript/subscribers isn't merging config variables passed to the function with the defaults properly
<mrevell> StevenK, Sure. It's very easy to deactivate a project. Do deactivated projects show up in searches etc?
<StevenK> I have no idea. But if it is easy, then excellent.
<G> as a result, when 'resetNoSubscribers' (approx line 596) runs it can't look up the value of subscirbers_label because it doesn't exist
<poolie_> there's a big difference between someone choosing to call their code junk and us making it
<gmb> G: Argh, right.
<mrevell> I'd also like to fix whatever leads to people mistakenly creating a new project, when really they want something else.
<poolie_> nigelb, as it happens i was just reading some of http://www.bwater.com/Uploads/FileManager/Principles/Bridgewater-Associates-Ray-Dalio-Principles.pdf today about the benefits of facing reality, _especially_ when its painful
<G> gmb: it's because the bugs & answers scripts, aren't setting the value in the config var
<wgrant> mrevell: Making the UI less bad would help there.
<poolie_> mrevell, they really want CDs :)
<wgrant> mrevell: Because people might be able to find their way to what they want.
<mrevell> poolie_, Heh, yeah :)
<G> gmb: it got masked by a unit test, but what I'm thinking is essentially: http://pastebin.ubuntu.com/677887/
<nigelb> poolie_: heh :)
 * gmb looks
<wgrant> I wonder if people have stopped looking for ShipIt yet.
<nigelb> G: If you were using for full-name, we'd have good fun with tabcomplete
<nigelb> *your
<G> nigelb: I have the IRC nick 'Nigel' registered ;)
<mrevell> wgrant, huwshimi has a great deal of work when he gets back from leave :)
<poolie_> that thing (200 page) pdf is well worth reading
<wgrant> mrevell: Oh yes :)
<nigelb> G: Ha! that's you
<nigelb> poolie_: 2 days off, I have soemthing to read :)
<G> I don't have 'nigelj' though, can't be bothered registering it :)
<nigelb> hehe
<G> gmb: because basically what is happening, it's merging everything into 'this' but SubscribersList(config) doesn't get 'this' it gets the unmerged config attrs
<gmb> G: I understand and agree :).
<G> gmb: I tried SubscribersList(this) but that crashed and burnt
<G> gmb: so should I go w/ that and do a Merge Proposal?
<bigjools> wgrant: right, it's all fixed.  Would you care to check the MP...
<gmb> G: Yes, run up an MP and ping me the link. I'll take a look and double check it this afternoon.
 * bigjools is now fairly sick of the sight of changelogs and unicode
<G> gmb: okay, thanks!
<gmb> np
<G> (for what it's worth, the unit test never picked it up, because the unit test was pre-populated w/ a config that specified everything)
<StevenK> G: I think you should fix/write another test
<G> StevenK: I was thinking that myself :)
<G> StevenK: I think what I'll have to do though, is create a second setUpSubscribersList without that information in the unit test
<StevenK> G: A second unittest without the information sounds like an excellent idea.
<G> yep just found how I can do it easy :)
<nigelb> wait, you found unittest easy?
<StevenK> nigelb is jealous.
<nigelb> Of course.
<G> nigelb: I'm basically running the same tests, but with a different starting point :)
<nigelb> I headdesk constantly.
<nigelb> And break them in truckloads
<StevenK> nigelb: Is there a marked depression in your desk?
<nigelb> Yes.
<nigelb> do you have a pillow for those situations?
<nigelb> Or does joiing the LP team give you a Launchpad branded pillow.
<StevenK> No, I was just curious if your constant bashing your head was affecting your desk adversely.
<nigelb> heh
<nigelb> BUt I'm geting better.
<nigelb> I wrote a test + fix with just little help the other day.
<nigelb> StevenK: But is it just me or are tests challening?
<wgrant> bigjools: Looks good.
<StevenK> nigelb: I found tests get easier to write as you get more comfortable with the code base.
<StevenK> nigelb: But it is still difficult if I jump into an area of the codebase that I'm not familar with.
<nigelb> StevenK: I guess not being familiar with any bit of the code base is against me.
<nigelb> I'm also unfamiliar with the different ways to test.
<nigelb> Well, I am now familiar with a few, but I'm guessing there are more.
<jml> nigelb: have you done much TDD before?
<nigelb> jml: LP was one of the first. Now I'm doing more with Mozilla projects.
<StevenK> Depending on what I'm doing -- if I'm playing or trying to understand how something works, I will just change the code. If I'm removing stuff, I will tend to just wield the axe and swing. If I'm fixing a bug or something like that, I will write the test first.
<jml> nigelb: and I guess both are big, complex, old codebases
<nigelb> jml: I touch the Mozilla Webdev, not firefox :)
<nigelb> StevenK: Oh. I should try tyhat.
<jml> nigelb: ah ok
<nigelb> I tend to write tests later. THat's probably wrong.
<StevenK> nigelb: It can be.
<jml> nigelb: I found that getting started w/ TDD took a while, because I wasn't used to being so clear about what I was trying to do. Then, there was a bit of a watershed, and then writing tests was easier before *or* after code
<jml> Although I still find writing tests after code deadly dull
<nigelb> SHould find another bug to work on so I can try with tests first.
<nigelb> 2 days holiday, I have 2 full days to come up with something!
<gary_poster> jelmer, hi.  I have a fix for bzr 2.3 that I'd like in LP.  I know that you (?) have 2.4 coming to us Friday-ish.  John said that my fix would go into 2.4 and trunk "automatically"...do you have a recommendation on me getting my fix in?
<gary_poster> About the best situation I can imagine in the situation is "my fix goes into 2.4 this week and when we switch to 2.4 we get the version with the fix".  Other options might be ppostponing getting my work in till later, but from a Lean/mgmt perspective that's not ideal
<gary_poster> Hm, my connection seems a bit odd...
<abentley> gary_poster: bzr 2.4.0 went gold on August 11.
<gary_poster> abentley, I know, but LP uses our own distros of it frequently.
<abentley> gary_poster: It wasn't clear you were talking about using unreleased versions.  Certainly, we could roll our own.
<abentley> gary_poster: but the one coming out this week will not have the fix.
<gary_poster> abentley, ok.  So, do you have a suggestion on how to proceed?  Once the 2.4 line had the fix, I could roll our own and merge it to db-devel
<gary_poster> Or just wait till Friday and update separately...
<gary_poster> I guess it depends in part on how quickly the 2.4 line gets the fix
<gary_poster> (merged via PQM, that is)
<abentley> gary_poster: to me, it doesn't seem like an urgent fix.  Am I missing something?
<gary_poster> abentley, it was causing ec2 land to fail for me.  It appears to be intermittent.
<G> gmb: thanks for the help, I've just submitted the merge proposal: https://code.launchpad.net/~dev-nigelj/launchpad/bug837290/+merge/73369
<gary_poster> abentley, so it would only be urgent to LP
<gmb> G: Okay, thanks. I'll try to get to it in the next hour or so.
<G> gmb: okay, no hurry
<gary_poster> and then only to people who are hitting the intermittent problem
<nigelb> hehe, I got pinged for the nigel there.
<nigelb> Drat.
<abentley> gary_poster: I've seen that behaviour for months if not years.
<G> nigelb: haha :)
<gary_poster> abentley, huh.  I don't know why it was biting me
<gary_poster> but it was
<G> nigelb: you know, I recently had to write an irssi script to ignore people saying "G+" at the start of a line :)
<gary_poster> heh
<deryck> Morning, all.
<abentley> gary_poster: Anyhow, if it matters to you significantly, then branch 2.4 now, apply the fix, and that'll be good until 2.4.1 comes out.
<nigelb> G: haha! Fun!
<StevenK> G: 'bzr pipes' is from a plugin called bzr-pipeline -- lp:bzr-pipeline
<nigelb> THere's a machine called babune with bzr team.
<nigelb> I got pinged for that quite a bit
<nigelb> (My last name is babu, and I had a hilight for that)
<nigelb> I had change how that hilight worked
<deryck> Morning, adeuring.  How are things with the hwdb fixes?
<adeuring> deryck: mixing results. shall we mumble about it?
<deryck> that's not what I wanted to hear. ;)
<deryck> adeuring, yes. let's mumble.  firing it up now.
<G> StevenK: looks like rocketfuel-setup didn't know about it
<StevenK> G: It's optional, but *awesome*
<nigelb> G: It doesn't, I had to set it up manually as well.
<nigelb> You should also probably install python-lint something
<gary_poster> abentley, ok...I understood the fix would be automatically applied to 2.4 (though I don't think it will apply cleanly because pertinent code has moved around between 2.3 and 2.4), and it is not yet there...so I'll wait to see if it is applied and go from there.  Thanks.
<gary_poster> (since 2.4 is not in tree yet)
<wgrant> 2.4 is in db-devel now.
<wgrant> And since staging is back after a two-week absence, we might be able to QA it.
<wgrant> And get it into devel.
<abentley> gary_poster: Not automatically.  bzr devs periodically merge one series into the next.
<wgrant> jelmer: Any progress?
<gary_poster> wgrant, ah great news about staging!
<gary_poster> abentley, ok.  Can I do anything to help or encourage the merge--I'd be happy to prepare my own merge and make an MP if it helped
<gary_poster> But don't want to be a bother if it won't help
<stub> wgrant: Did you happen to rebuild the EC2 image?
<jelmer> wgrant: on bzr 2.4 QA? Apart from loggerhead I managed to do the QA I wanted to
<abentley> gary_poster: I think it would be best to wait until bzr 2.4 is actually released.
<gary_poster> abentley, ok that's fine.
<gary_poster> thanks
<abentley> gary_poster: So if it really matters to you, you should cut your own release.
<stub> gary_poster: If I update launchpad-dependencies, does a losa need to update the buildbot hosts too?
<gary_poster> oh
<abentley> gary_poster: i.e. if you want to have it this week or next week, make your own tarball.
<gary_poster> stub, that's the deb?  If so, yes, I think so.
<stub> Ta. I'll open an RT.
<StevenK> Bah, I can't re-produce that missing pipes thing with lint
<gary_poster> abentley, ok cool thanks.  I did not see "make a release" instructions; is it just "update bzrlib.version_info to increment the last number, and run ./setup.py build"?
<stub> Anyone want to volunteer to update the ec2 test image with new launchpad-developer-dependencies so I don't have to experience the process myself via by Internet connection in a rain storm?
<abentley> gary_poster: First manually override the version number in setup.py, then run "make dist".
<gary_poster> stub, I or someone on yellow can do it.
<gary_poster> stub, it's ready to go now?
<gary_poster> abentley, got it, thank you very much.
<stub> gary_poster: That would be lovely. Its all ready to go in the PPA. Need anything from me?
<gary_poster> stub, not that I can think of.
<abentley> gary_poster: np
<flacoste> abentley, gary_poster, jelmer, adeuring: i'd like to release 13822 today, please QA your revisions
<abentley> flacoste: roger.
<flacoste> thanks!
<gary_poster> abentley, ack, on it
<gary_poster> sorry abentley, I meant flacoste
<nigelb> flacoste says that quite differently from wgrant.
<nigelb> ;)
<abentley> gary_poster: also, sudo make me a sandwich :-)
<StevenK> nigelb: QA!
<nigelb> StevenK: Yeah, you too :P
<gary_poster> abentley :-)
<jelmer> aye aye!
<mrevell> flacoste, Skype?
<wgrant> nigelb: (for the record, I've only said 'QA!' twice, both times to StevenK after he started doing it, despite him doing it to supposedly emulate me!)
<flacoste> mrevell: yes, please
<nigelb> wgrant: hehe
<deryck> flacoste, I need to chat with you about the state of that hwdb change.  but am on stand up now.
<deryck> flacoste, I assume the qa pressure is for that.
<flacoste> deryck: i'm getting on call with mrevell now
<flacoste> deryck: will ping when i'm free
<deryck> flacoste, ok
<rvba> henninge: Hi
<henninge> rvba: Hi! ;)
<henninge> rvba: just reading your reply.
<henninge> now
<abentley> flacoste: I am having difficulty QAing because https://qastaging.launchpad.net/ is timing out.
<rvba> henninge: Okay, I'm available if you need any clarification.
<henninge> rvba: cool, thanks
<rvba> henninge: BTW thanks for the very nice and instructive review ;)
<flacoste> abentley: is your change the source of the timeout? or it's something else causing timeout?
<abentley> flacoste: it must be something else.
<abentley> flacoste: Now qastaging is giving "Please try again/Sorry, there was a problem connecting to the Launchpad server. ", so maybe it was due to a deployment?
<wgrant> qastaging is not updating, so something is broken.
<wgrant> It successfully updated a few hours ago, but the update log file is being truncated frequently when it never has been before, so something may be up.
<abentley> wgrant: how can you tell it's not updating?
<wgrant> wgrant@carob:~$ rsync -a asuka::qastaging-logs/qastaging-update.log .
<wgrant> wgrant@carob:~$ tail qastaging-update.log
<wgrant> Tue Aug 30 13:18:03 UTC 2011 Current Revno: 13822, New Revno: 13822 - nothing to update
<wgrant> wgrant@carob:~$
<wgrant> The next update will not start for almost another minute.
<wgrant> And the last one found nothing to do.
<gmb> G: Branch approved. I'll land it for you now.
<G> gmb: just saw the e-mail, thanks
<henninge> rvba: before I try to find out your asset problem, let me find something out.
<henninge> deryck: Hi!
<deryck> henninge, hello :)
<henninge> deryck: When you merged lazr-js into lp in Dublin, you kept the directory structure.
<henninge> deryck: There is a seperate directory for each lazrjs class.
<henninge> e.g. overlay, formoverlay, etc.
<henninge> deryck: Do you think that structure should be kept when adding more widgets?
<henninge> deryck: Or wouldn't it be enough to have a file per widget?
<deryck> henninge, hmmmm
<henninge> deryck: I think a directory that holds just one file is a bit of a waste.
<henninge> deryck: but then there are assets and tests but they can go into central directories, too.
<deryck> henninge, I prefer keeping the directory structure so we know where tests live.....
<rvba> henninge: ;)
<deryck> henninge, if we do change to individual files and central tests, I don't mind that.  but I'd prefer a conscious change and move everything....
<deryck> not have done one way, and new stuff done another.
<henninge> deryck: well, the lp stuff does not use directories.
<henninge> so there is lib/lp/app/javascript/client.js
<henninge> and not client/client.js
<henninge> so we already have a mixture.
<deryck> hmmm, good pont.
<deryck> point
<deryck> henninge, I don't really care :)  You and rvba decide. :)
<jcsackett> henninge, deryck: there are some cases where multiple files comprise the components. e.g. in the picker directory we have several files. it's probably worth maintaining folders only if we need to break up the files. :-)
<deryck> yeah, I think do what makes sense for the widget.  I'm just not that passionate about this issue. ;)
<henninge> deryck, jcsackett: although I'd prefer a unified approach.
<henninge> ok, we decide ;-)
<henninge> rvba: how do want to do it? ;-)
<flacoste> deryck: i'm free now
<deryck> flacoste, awesome.  let's mumble.
<rvba> henninge: my really first idea was to put my new widget inside formoverlay.
<henninge> rvba: no, that was not one of the choices ... ;-P
<rvba> henninge: ideally I think we should have a directory called 'overlay' with all the 3Â overlays.
<flacoste> deryck: i'm in Orange 1-1
<deryck> coming, sorry....
<rvba> henninge: so, what do you reckon, back to the individual confirmationoverlay directory?
<henninge> rvba: I like the "overlay" idea, too, but not for this branch.
<rvba> henninge: That's right, not for this branch!
<henninge> rvba: I think going back to the individual directory is fine.
<rvba> henninge: So, let's go back to the confirmationoverlay directory, and I shall file a bug.
<henninge> rvba: cool
<rvba> \o/
<henninge> rvba: your other changes look very good, too. I won't force you to remove all usages of join('')  (though I'd like to ;)
<henninge> rvba: push your changes once you are done and 'll have a last look.
<rvba> henninge: Thanks ;). You have to admit I followed the vast majority of your advices.
<henninge> rvba: yes, thanks
<henninge> rvba: but you also wrote some nice code yourself, where I let you ;-)
<henninge> I am impressed at how nicely it came out.
<rvba> Thanks!
<jelmer> gmb: G'day!
<jelmer> gmb: Can I add another small branch to your queue?
<bigjools> gmb: yo, me too! :)
 * nigelb waves
<nigelb> Need to find another bug to work on.
<abentley> jml: Congratulations!  And, I have a Twisted style question.
<G> gmb: oh hey, I just realised, I need to do that Contributor Agreement thing right?
<nigelb> G: Yeah, you do. Send an email with that stuff to francis
<G> suddenly struck me that the code of conduct thing != contributor agreement
<nigelb> hehe
<G> I'll have to do it in the morning when I have access to a scanner
<nigelb> wait, what?
<nigelb> Has it changed?
<G> I'm looking at: http://www.canonical.com/contributors
<nigelb> I didn't scan for sure.
<nigelb> But there's a new agreement.
<rvba> henninge: Done!
<G> nigelb: it's got the fill & in the blank bits in the PDF, but it says it needs to be printed, signed , scanned and sent
<nigelb> Need to check wwhat I did back then
<nigelb> Ah, i just had to send an email with that as an attachment.
<nigelb> With Harmony, the rules have changed.
<G> gmb: looks like the change needs to hold off for a day so I can send in the form
<henninge> rvba: cool. Looking.
<gmb> G: Ah, blast. I'd forgotten about that.
<gmb> Okay, I'll stop the EC2 run.
<jcsackett> gmb: can i throw https://code.launchpad.net/~jcsackett/launchpad/target-adapters-aplenty/+merge/73248 on to your queue?
<gmb> jcsackett: Sure
<jcsackett> thanks, gmb!
<henninge> rvba: I did not know about ObjectAssert. I did not notice you were using it before.
<henninge> rvba: or did you just add that?
<henninge> rvba: Also, areEqual seems to be undocumented ... ;-)
<rvba> henninge: No I did not.
<henninge> ok, I missed that, then.
<henninge> np
<henninge> rvba: but the test is still failing for me
<henninge> not on the ObjectAssert, though
<rvba> henninge: strange, which test?
<henninge> test_hidden_field_added_on_ok: failed.
<henninge> Unexpected error: Cannot call method 'simulate' of null
<henninge> and this:
<henninge> test_call_submit_on_ok: failed.
<henninge> Unexpected error: Cannot call method 'simulate' of null
<rvba> henninge: They all pass here ...
<henninge> hmmm
<rvba> henninge: last revision is 13784
<henninge> I will have to get a new branch. I had merged to get make lint working.
<henninge> rvba: meanwhile: I *do* think it is important to satisfy the linter otherwise we might miss notice real issues.
<henninge> rvba: so thanks for fixing that
<henninge> s/notice//
<rvba> henninge: Welcome :)
<rvba> BTW I only had lint errors for the css file which was mostly copied from existing code. Hence my remark ;)
<jelmer> gmb: Can I add another small branch to your queue?
<gmb> jelmer: SUre
<gmb> jcsackett: Your branch is approved.
<henninge> rvba: even with a new checkout the test is still failing.
<jelmer> gmb: Thanks! It's at https://code.launchpad.net/~jelmer/launchpad/no-code-import-approval-2/+merge/73388
<jcsackett> gmb: thanks. :-)
<gmb> allenap: What's the story with https://code.launchpad.net/~allenap/launchpad/dd-beta-icon-bug-827508/+merge/72492?
<rvba> henninge: Ok ... let me try with a new checkout...
<rvba> gmb: allenap is off today.
<gmb> rvba: What a slacker. Thanks :)
<matsubara> gmb, hi there, can I add this one https://code.launchpad.net/~matsubara/oops-tools/837468-missing-req-vars/+merge/73399 to your queue?
<rvba> gmb: Back on Thursday. ;)
<gmb> jelmer: I love it when a diff confuses the hell out of me. Approved.
<gmb> rvba: I only care about other people's branches on Tuesdays. Says so on my calendar ;)
<rvba> gmb: IÂ see ;)
<gmb> matsubara: Sure, looking now.
<bigjools> gmb: hello, did you see my request from 16:03?  Looks like you're busy though ...
<gmb> bigjools: I missed it, sorry, was picking up my car  from its service/MOT/owner-gouging. I'll take a look after I've looked at matsubara's. Please send me the link again.
<gmb> matsubara: Approved.
<matsubara> gmb, thanks!
<bigjools> gmb: https://code.launchpad.net/~julian-edwards/launchpad/sync-close-bugs-bug-833736/+merge/73401  thank you sir
<bigjools> gmb: and I hear you about the car....
<flacoste> matsubara: you saw jane approved the examples script open-sourcing? can you make the necessary updates to the TestApiApplications page?
<flacoste> matsubara: we'll revisit the whole SLA wording when lifeless once lifeless comes back, but i think the TestingApiApplications documentation is good enough in its current form
<jelmer> gmb: :) thanks
<matsubara> flacoste, yep, I saw Jane's email. I'll add to my todo list and try to get to it this week. when do you need this done? Are the scripts ready for release or OEM's going to release them?
<henninge> rvba: so, it is failing on the second "simulate" in  test_hidden_field_added_on_ok
<rvba> henninge: It looks like the button on the overlay is not found somehow.
<bigjools> gmb: before you start one mine, the current diff is bong, it's not picked up the pre-req branch
<bigjools> on*
<flacoste> matsubara: i think we should take the responsibility of releasing them
<gmb> bigjools: Oh, joy.
<gmb> Hadn't started yet though.
<bigjools> gmb: ok it has now
<gmb> Hurrah.
<flacoste> matsubara: since it's useful to us as examples more than anything else
<gmb> Will look shortly.
<flacoste> matsubara: and there is no urgency to this, do it as time permit
<bigjools> gmb: yes.  Turns out you need to merge and push more than you think :)
<gmb> :)
<flacoste> matsubara: might be good to blog about testing API applications once it's complete
<matsubara> flacoste, ok.
<flacoste> thanks matsubara
<matsubara> np
<rvba> henninge: all good on a fresh checkout.
<henninge> *#@$%& ???
<nigelb> heh
 * rvba *#@$%& too
<james_w> flacoste, matsubara: http://bazaar.launchpad.net/~work-items-tracker-hackers/launchpad-work-items-tracker/trunk/view/head:/lpworkitems/fake_launchpad.py might pique your interest as an approach with pretty much the opposite set of pros and cons :-)
<rvba> henninge: oh oh ... IÂ bet you're using chrome ..?
 * henninge was just starting FF
<henninge> rvba: yes
<henninge> passes in FF
<rvba> Yep :(
<henninge> rvba: what is the difference?
<rvba> I really don't know.
<rvba> Let's see if it's just the tests.
<rvba> henninge: Gosh, something is really wrong with Chrome ...Â IÂ guess I'm back to the drawing board.
<matsubara> james_w, thanks for the link. I'll take a look when I get to update the wiki page.
<LPCIBot> Project devel build #1,012: STILL FAILING in 4 hr 32 min: https://lpci.wedontsleep.org/job/devel/1012/
<henninge> rvba: that is strange, though.
<henninge> maybe it is just the selector in the test?
 * henninge tries
<rvba> henninge: "<button class="lazr-neg lazr-btn">Cancel</button></div>"
<rvba> That's what chrome sees.
<rvba> No "type=submit"
<rvba> I mean no type=button
<gmb> bigjools: Approved
<bigjools> gmb: thanks.  I am surprised you don't have questions!
<gmb> bigjools: Usually when someone says that it means I might have missed something. What kind of questions did you expect?
<bigjools> gmb: well, it's soyuz ....
<bigjools> terminology mainly
<bigjools> maybe you guys are starting to understand it!
 * bigjools lives in hope
<gmb> bigjools: Yeah, but for Soyuz this is pretty obvious. I don't think I got stuck on any terminology beyond a couple of "what-does-that-tla-mean-again" moments.
<henninge> rvba: Not sure where you see that.
<rvba> henninge: In Chrome's dom inspector.
<gmb> bigjools: I shan't profess to understanding *just* yet, thanks. I'm not that crazy ;)
<bigjools> gmb: tell you what though, I am glad I didn't have to write those regexes ...
<henninge> rvba: I see. I was looking at the dom node in the javascript debugger.
<henninge> rvba: that shows me that both buttons are type=submit.
<henninge> rvba: I think that is the problem.
<henninge> or rather, *may* be the problem
<rvba> """var cancel_button = Y.Node.create('<button>').set('type', 'button')"""
<gmb> bigjools: I can imagine :)
<henninge> ha!
<bigjools> gmb: right, ta
<henninge> no
<rvba> henninge: it works if I set a class on the button and use it to grab the button.
<henninge> yes, I expect so.
<henninge> I still wonder about the type thing
<henninge> rvba: but does the form work in chrome?
<rvba> Testing that right now.
<rvba> It does.
<henninge> rvba: This works, too, strangely enough:
<henninge> Y.Node.create('<button type="submit"></button>')
<henninge> rvba: I gotta run. r=me with some comments. Please have a look. ;-)
<rvba> henninge: ok, I will ... I'm still trying to figure out what's going on.
<henninge> rvba: may be a YUI bug.
<rvba> henninge: Looks like allenap had a reason to do it this way:
<henninge> rvba: but of course the test has to pass in both FF and chrome before you land this ...
<rvba> Y.Node.create( '<button type="button" class="lazr-neg lazr-btn" />').set("text", "Cancel");
<rvba> henninge: Sure.
<henninge> Yes, I guess so.
<henninge> See you tomorrow! ;-)
<jml> abentley: hi
<jml> abentley: sorry, was offline processing email.
<jml> abentley: also, thanks :D
<jml> abentley: how can I help?
<abentley> jml: Currently, I have a method that either returns a Deferred or None, and calling code that assumes it always returns a Deferred.
* gmb changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 244 - 0:[#######=]:256
<abentley> jml: is it more tasteful to call maybeDeferred or to update the method to return twisted.internet.defer.success(None)
<jml> abentley: my taste leans to the latter, but either can be alright. Generally maybeDeferred is best when calling arbitrary, "untrusted" code.
<jml> abentley: or if you want to make things easy for function/method writers
<abentley> jml: okay, cool.
<jml> e.g. twisted.cred allows authentication methods to return a non-Deferred thing synchronously, because that's actually quite nice to do a lot of the time.
<nigelb> Who's a good person to talk to about registry UI?
<nigelb> I'm just wondering why we don't do mailto: for email ID's on the person page
<abentley> nigelb: sinzui is the usual guy, but he's not here at the moment.
<nigelb> abentley: Ah, ok :)
<nigelb> I was looking at bug 82002 and wondering if I could link the jabber ones at least.
<_mup_> Bug #82002: Jabber ids look like email addresses, causing confusion <jabber> <lp-registry> <Launchpad itself:Triaged> < https://launchpad.net/bugs/82002 >
<nigelb> abentley: Is he off today/this week?
<abentley> nigelb: I don't know.
<nigelb> Or am I asking at the wrong time
<abentley> nigelb: he is usually around at this time.
<nigelb> cool, so I can try another day :)
<nigelb> abentley: Thanks!
<abentley> nigelb: np
<abentley> deryck: as we have no OCR, could you please review https://code.launchpad.net/~abentley/launchpad/simplify-twisted-runner-2/+merge/73402 ?
<deryck> abentley, sure.
<deryck> abentley, r=me.
<abentley> deryck: ta
<deryck> np!
<nigelb> I have a conflict on utilities/sourcedeps.cache
<nigelb> how do I fix that?
<nigelb> (this is in devel)
<nigelb> i.e., which is preferred? lpreview vs loggerhead apparently.
<abentley> nigelb: use the version you merged, then run update-sourcecode.
<nigelb> so, whatever is in  MERGE-SOURCE is what I want?
<abentley> nigelb: there should be a complete copy called utilities/sourcedeps.cache.OTHER which what you want.
<nigelb> ah, bzr resolve --take-other sourcedeps.cache did the trick!
<abentley> nigelb: not sure if that does the same thing, but it's probably okay.
<nigelb> :)
<lifeless> flacoste: hi; waving at you via the hwdb bug; bye;)
<flacoste> lifeless: hi! don't you have more important things to do than commenting on LP bugs? ;-)
<lifeless> yes; am on the way to the hospital now. its slow going :
 * lifeless goes
<LPCIBot> Project devel build #1,013: STILL FAILING in 3 hr 56 min: https://lpci.wedontsleep.org/job/devel/1013/
<thumper> imapfilter hates bug mail
<thumper> x-launchpad-bug appears once per task
<thumper> however imapfilter uses server side filtering using the IMAP RFC
<thumper> which only checks the first header
<thumper> so trying to match bugmail for bugs with multiple tasks is full of fail
<thumper> I'd love a header like: x-launchpad-project that lists all effected projects
<thumper> x-launchpad-sourcepackage for the packages ones
<thumper> then the mail filtering would at least work
<thumper> also
<thumper> blueprint email has no x-launchpad-message-rational nor x-launchpad-project ...
<thumper> if y'all can get LP working on oneiric I may provide patches
<jelmer> thumper: it works (as far as I can tell) on oneiric here
<G> gmb: btw, I fired off the CLA document, so I don't know what the process is now.  Sorry I didn't realise I hadn't done it before.
#launchpad-dev 2011-08-31
<LPCIBot> Project devel build #1,014: STILL FAILING in 3 hr 58 min: https://lpci.wedontsleep.org/job/devel/1014/
<mwhudson> jelmer: maybe we don't need new code import mails now?
<wgrant> Heh.
<mwhudson> not that i really mind, it's not like they take long to dispatch
<cody-somerville> mwhudson, they're probably going to give me Carpal Tunnel Syndrome though from all the clicking needed to delete them :P
<nigelb> Good morning!
<wgrant> Evening nigelb.
<nigelb> cody-somerville: heh, automate with selenium!
<nigelb> wgrant: I'm taking a shot at TDD today. Writing tests first. I hope that doesn't mean more headdesking :)
<wgrant> nigelb: It gets better :)
<wgrant> nigelb: Particularly when they aren't doctests.
<nigelb> Yeah, I'm wondering why StevenK's deletion spree doesn't delete doctests yet :D
<wgrant> He spent a week deleting lots of doctests a while ago.
<nigelb> So there is hope.
<nigelb> btw, lifeless is proud daddy yet? :)
<wgrant> How was about last night, so I suspect not, but have not heard either way.
<nigelb> Ah, ok.
* jtv changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jtv | Critical bugs: 244 - 0:[#######=]:256
<nigelb> And I've picked up bug 684548
<_mup_> Bug #684548: Display Edit Patch Flag More Prominently <easy> <lp-bugs> <ui> <Launchpad itself:Triaged by nigelbabu> < https://launchpad.net/bugs/684548 >
<nigelb> Morning jtv! I shall be troubling you in a bit.
 * jtv rushes to change the topic :)
<nigelb> heh
<nigelb> Should. Stop. Looking. At. Ivanka's. Pictures.
<LPCIBot> Project devel build #1,015: STILL FAILING in 3 hr 58 min: https://lpci.wedontsleep.org/job/devel/1015/
<StevenK> Text conflict in test_notification between stable and db-devel. Orsum.
<adeuring> good morning
<adeuring> wgrant: could you help me with qa for r13826? It's too early in the morning for me to even understand the related bug... problem is: I am a bit desperate to deploy a revision >= 13822
<wgrant> adeuring: It's been broken for nearly 18 months; is it really urgent?
<adeuring> wgrant: yes, unfortunately
<wgrant> Intriguing.
<wgrant> Looking.
<wgrant> Ah, need to resolve the conflict first, since DF runs db-devel...
<wgrant> jtv: Are you still using dogfood?
<jtv> wgrant: no, I use a piece of cloth now.  It doesn't absorb as well but it muffles more effectively.
<wgrant> Hah.
<jtv> So no.
<jtv> Meanwhile, I see we have a wonderful merge conflict.
<wgrant> I'm pushing a branch for that.
<jtv> Cool
<wgrant> But db-devel hasn't been merged in a while, so my 256kbps upstream is being painful...
<wgrant> adeuring: Hmm, this revision may be bad. We'll see shortly.
<adeuring> ok...
<wgrant> Well, maybe not so shortly, since mawson is mawson.
<wgrant> jtv: http://librarian.dogfood.launchpad.net/71107310/bRhzSbgpU9MP0WU1BbgvqBNgitQ.txt is worrying.
<wgrant> jtv: The DSDJ runner can't check packageset membership...
<wgrant> I wouldn't think that would be a new thing.
<jtv> That sounds familiar somehow.
<jtv> I would suspect that a cowboyed privilege dropped out of the db without the permanent privilege making it in.
<jtv> Either because security.py wasn't run, or it failed, or it did not contain that required permission.
<wgrant> There is no permanent privilege.
<wgrant> I just reran security.py and checked security.conf :(
<wgrant> granted select on flatpackagesetinclusion and packagesetsources, and it works.
<wgrant> And packagesetgroup, it seems.
<wgrant> Does anybody know how to use Thunderbird's account creation wizard?
<wgrant> They've removed manual setup, and I can't work out how to use an account on a provider that they don't know about.
<nigelb> Spend 5 minutes why launchpad.dev wasn't working. Sigh.
<nigelb> *debugging
<wgrant> Was it running?
<nigelb> Yeah, https
<nigelb> I can do a create_initilized_view for all the portlets too?
<wgrant> In general that should work.
<nigelb> tdd is weird, at last at first.
<wgrant> For some it won't.
<nigelb> *least
<wgrant> adeuring: I think that rev is good.
<wgrant> Which means we can deploy.
<adeuring> wgrant: great, thanks for your checks!
<wgrant> Sorry it took so long. High startup time and lots of checks to do.
<nigelb> so, I create a bug.
<nigelb> I add an attachment
<nigelb> now, how do I get the comment of that attachment?
<nigelb> (I feel more lost writing tests first)
<wgrant> nigelb: BugAttachment.message
<nigelb> ah
<nigelb> need to look at models deeper.
<wgrant> nigelb: I often find it's clearer to check the DB structure.
<wgrant> \d bugattachment
<nigelb> I'm not well versed with postgres enough.'
<nigelb> yet.
<wgrant> psql launchpad_dev
<wgrant> \d bugattachment
<wgrant> Done :)
<nigelb> wow
<nigelb> I did not know about psql.
<nigelb> This, is neat.
<jtv> nigelb: I told you that exact same trick!
<nigelb> jtv-eat: well, I didn't know how to use it correctly :)
<StevenK> psql is love
<nigelb> hehe, so with tests first, I can head desk first
<nigelb> and then when I fix it, there's more joy
<nigelb> Gah, I can't do this apparently.
<nigelb> html = create_initialized_view( self.bugattachment.message, name='+index', )()
<wgrant> nigelb: Why not?
<wgrant> nigelb: It's odd syntax, but it should work.
<wgrant> Oh, hmm.
<wgrant> There's no +index for Message itself.
<nigelb> ah.
<wgrant> You'll need to get the BugMessage.
<wgrant> (Message just represents an email in LP. May be on a bug, merge proposal, answer, mailing list...)
<nigelb> oh.
<nigelb> ForbiddenAttribute: ('BugMessage', <Message at 0xf402b0c id=51>)
<wgrant> getUtility(IBugMessageSet).getByBugAndMessage(some_bug, some_message)
<nigelb> hehe
<nigelb> awesome traceback
<wgrant> Oh?
<nigelb> http://dpaste.com/605694/
<nigelb> wgrant: Thoughts? :)
<wgrant> nigelb: Hm, that's pretty nice.
<nigelb> Haha
<wgrant> nigelb: It looks like you're calling it with a bugattachment, not bugattachment.message.
<nigelb> aaah.
<nigelb> Sigh.
<nigelb> http://dpaste.com/605703/
<nigelb> Why do I end up wasting more time than finding a fix in tests :|
<wgrant> Ah, it looks like you actually want BugComment:+index, not BugMessage:+index. And BugComment creation is not for the faint of heart :/
<wgrant> You're possibly better just rendering BugTask:+index.
<nigelb> Well, the problem is the edit shows up in a portlet as wwell.
<nigelb> I don't watch to get that.
<nigelb> OR, I'll have to count the number of appearances.
<nigelb> Or add a class to the find and use soupmacheers
<wgrant> Or you can use BeautifulSoup/soupmatcher to look only in the comment list.
<wgrant> There should be an existing class/ID you can use.
<nigelb> Ah.
<nigelb> Ah, we deployed.
 * nigelb checks his fix.
<wgrant> We did.
<wgrant> It looks good.
<nigelb> \o/
<LPCIBot> Project devel build #1,016: STILL FAILING in 3 hr 55 min: https://lpci.wedontsleep.org/job/devel/1016/
<nigelb> I have to agree with bigjools, soupmatches is quite rad.
<mrevell> Hello
<nigelb> Hello mrevell! Good morning :)
<mrevell> Hey nigelb!
<rvba> nigelb: I'm sure you will come to love soupmatchers ;)
<nigelb> rvba: Heh, already loving it!
<rvba> nigelb: ;)
<nigelb> heh, I summoned bigjools? :P
<bigjools> I was rubbed
<nigelb> xkcd ruined that reference for me.
<nigelb> Anyway, I was mentioning how I like soupmatchers :)
<rvba> jtv: Could you please have a look at this (tiny) MP when you get a chance: https://code.launchpad.net/~rvb/launchpad/masscreate-bug-835040/+merge/73495Â ?
<nigelb> wgrant: GAH.
<nigelb> I'm doing something wrong, I'm getting hideous tracebacks.
<jtv> wgrant: while you're being distracted, can I interest you in reviewing my DB privilege fix?  https://code.launchpad.net/~jtv/launchpad/bug-837893/+merge/73502
* jtv changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 244 - 0:[#######=]:256
<nigelb> My nightmares with tests continue. Sigh.
<jtv> Or I could ask danilos, who is OCR'ing today.
<danilos> jtv, oh, he is? :)
<jtv> Apparently.
<jtv> At least on Gregorian Wednesday.
* danilos changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: danilos | Critical bugs: 244 - 0:[#######=]:256
<jtv> :-)
<nigelb> jtv: You're marked under Europe? HA
<jtv> I'm sort of inbetween.
<danilos> jtv, r=me
<jtv> thanks danilos!
<danilos> jtv, which one of us?
<jtv> wgrant: it's doneâyou go help nigel or something.  :)
<jtv> for danilo in danilos: danilo.thank()
<nigelb> heh
<nigelb> if I create a bug with factory, I can call its +index right?
 * nigelb checks zcml
<nigelb> oh, I need to call the +index of the bugtask?
<wgrant> nigelb: Bug has no +index, right.
<wgrant> It redirects to default_bugtask's +index.
<nigelb> so, create_initialized_view(self.bug.default_bugtask, name='+index', )()
<nigelb> I got a scary traceback for that one.
<nigelb> http://dpaste.com/605821/
<nigelb> I wonder if this has something do with layer = LaunchpadFunctionalLayer
<nigelb> I think I've mostly used DatabaseFunctionalLayer before.
<jtv> danilos: you'll notice I left one review on the queue.  It looks like nobody's worked on it but it's got other reviewers' names on it.
<rvba> jtv: thanks for the review mon cher.
<jtv> rvba: ton /quoi/?
<rvba> TrÃ¨s cher jtv ;)
<bigjools> le horreur
<jtv> I thought I was doing this work at quite a reasonable rate.
<bigjools> jtv is actually cheap
<jtv> Oh now that's rich.
<jtv> And is it le horreur or l'horreur?
<rvba> hehe
<rvba> "l'horreur" indeed.
<rvba> FTR this "cher" was "dear".
<jtv> Funnily enough, "dear" can have both meanings in English.
<bigjools> I was enunciating all the syllables
<jtv> (although "expensive" is somewhat archaic)
<jtv> bigjools: I see.  Maybe you should enunciate them twice in succession: le horreur, le horreur
<jtv> Wasn't until that bit in the book that I realized that the film was based on it.
<jtv> âle horreur, sans blagueâ?
<rvba> nigelb: I /think/ the problem you have is due to the fact that you don't pass any 'principal' to create_initialized_view.
<nigelb> rvba: principal being a person?
<rvba> nigelb: yes.
<wgrant> rvba, nigelb: It defaults to anonymous.
<wgrant> This is possibly another view that doesn't like being instantiated directly :/
<wgrant> It's a fairly horrific view.
<nigelb> I tend to pick the fun ones, yeah.
<rvba> I'm pretty sure I had this exact same problem.
<nigelb> Someday I'll learn how to do this right :|
<nigelb> Ha, solution?
<rvba> nigelb: please try principal=factory.makePerson()
<nigelb> mmm
<nigelb> NoCanonicalUrl: No url for None because None broke the chain.
<rvba> nigelb: Looks like something is trying to redirect to None.
<nigelb> rvba: Oh.
<nigelb> That's wweird :|
<nigelb> I should have started with fixing the bug isntead of tests
<danilos> jtv, on line 84 of the diff on https://code.launchpad.net/~jtv/launchpad/bug-834388/+merge/73483, "removables" is not a ResultSet, right?
<nigelb> I've spend 8 hours and I've nothing to show for it, except a broken test :)
<jtv> danilos: no, it's a set that's first painstakingly constructed by hand.  :/
<jtv> Sorry, list!
<jtv> Not set.
<danilos> jtv, ok, fair enough (btw, I am surprised you didn't know about resultset.set() :P)
<jtv> Well, now I do!
<rvba> nigelb: It's a good habit to start with the tests. And its absolutely normal to have trouble to write tests at first.
<jtv> I have to leave very very soon btw
<danilos> jtv, sure, I am almost done with this second review
<jtv> Great, thanks
<danilos> jtv, just one more non-critical question, why the changes to logger statements?
<jtv> They annoyed me.
<jtv> Why have special provision for debug
<jtv> but none for the other log levels?
<jtv> Someone's going to do a logger.info or a self.warn and find it doesn't exist.
<jtv> Or do you mean the string interpolation?
<jtv> Apparently the "proper" way to do it is logger.info("format %s string", variable)
<jtv> Not logger.info("format %s string" % variable)
<nigelb> Hrm, I wonder what I do next.
<nigelb> I can't get the view to get a test. My test is supposed to just check the existence of a URL.
<G> btw guys, what is the best way to find out what handles/generates a particular page?
<nigelb> configure.zcml
<nigelb> it will be inside browser folder
<danilos> jtv, I meant both, the fact that logger.info considers the first string a format string doesn't make it proper in my book, but I don't care either way :)
<G> nigelb: ah ha, thanks! :)
<nigelb> G: :)
<danilos> jtv, it'd make sense if it was something like structured() or sqlvalues() where stuff needs to be interpolated some how
<danilos> s/ //
<jtv> danilos: I never bothered to ask really, it just allows for slightly better formatting.  :)
<danilos> jtv, heh, sure, formatting is a good argument
<danilos> jtv, anyway, r=me already, but I assume you know what will this do for performance :)
<jtv> thanks!
<jtv> It'll improve the normal case, make the weird case worse.
<jtv> And it shows future engineers how to fix the weird case.
<nigelb> wgrant / rvba: Anything more I could try?
<rvba> nigelb: please paste your diff.
<nigelb> http://dpaste.com/605832/
<G> hmmm took a look at bug 61428 and I think I've identified a 3 line fix
<_mup_> Bug #61428: Want a "subscribed to teams" portlet... <feature> <lp-bugs> <trivial> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/61428 >
<stub> The 'proper' way of interpolation with the logger API saves interpolation time, but has a habit of hiding interpolation bugs as we end up with untested code paths.
<G> ahhh actually, not that simple
<G> danilos: if you've got a moment, question about the above mentioned bug, the portlet while nearly identical for a subscribed bugs page, is nearly the same as the current assigned bugs (person-portlet-team-assignedbugs.pt) is the attitude to create a seperate portlet, or try and introduce logic to the current portlet
<stub> gmb: Bug comment lazy loading - what will search engines see if we turned this on for everybody?
<gmb> stub: Uhm... Well, at the moment it's just JS on top of what's currently there. But fair point; if we did it automatically then it would be not good for our Googlejuice.
<danilos> G: I am looking at it
<nigelb> gmb: Or using text-only browsers :)
<gmb> nigelb: also true.
<stub> gmb: Yer. We might need to be more intelligent. Default batched comments with a sane batchsize, and js magic to improve on that if you have js.
<gmb> Thankfully, we're talking about something I haven't done yet :).
<nigelb> heh
<gmb> stub: Agreed. It wouldn't be too hard to do that, actually.
<G> danilos: a second portlet, would basically be :%s/assigned/subscribed/g w/ a little bit of extra CSS for the new div class id's
<stub> For insane people who want to scroll through the entirety of Bug #1
<_mup_> Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Confirmed for compscibuntu-bugs> <dylan.NET.Reflection:Invalid> <dylan.NET:Invalid> <EasyPeasy Overview:Invalid by ramvi> <GenOS:In Progress by gen-os> <GNOME Screensaver:Won't Fix> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <LibreOffice:In Progress by bjoern-michaelsen> <Linux Mint:In Progress> <The Linux OS Project:In Pr
<G> (at least my intpretation)
<nigelb> argh!
<nigelb> But there are probably Ubuntu bugs worse than that.
<nigelb> rvba: Were able to spare a minute to look into that diff?
<danilos> G: right, I think you are better off reusing it, and just introducing a few variables like CSS class and target view to decide what to do in the template
<danilos> G: though, that entire portlet looks like a very bad UI, but if you find it useful, it's better than no UI :)
<rvba> nigelb: Yes, nothing seems bad with your diff but this "NoCanonicalUrl: No url for None because None broke the chain." is a tricky error.
<rvba> nigelb: hang on.
<nigelb> okay, so I'm not going completely crazy :)
<G> danilos: so just cp it, and change where it's needed
<danilos> G: nope, with "reusing" I meant use the same template (perhaps rename it), but modify it so you can pass in the bits like the div ID and "+assignedbugs" part of URL from the view
<G> ahh yep, I get you
<G> so basically the second option (introduce logic/pass variables)
<danilos> G: btw, would it not be better if there was a page listing all the actual bugs any of your teams is assigned to/subscribed to?
<danilos> G: yes
<G> danilos: you mean something like +subscribedbugs as is, and then something like +subscribedbugs/+all ?
<danilos> G: yeah, something like that
<G> actually, +indirectsubscribedbugs
<G> danilos: couldn't that be quite 'expensive' if someone did that against say Mark Shuttleworth who I'm guessing is a member of a lot of teams?
<danilos> G: it could be nontrivial, sure, but if that's the question you really want answered with this page, it should be doable, imho
<G> (expensive as in time to generate & amount of data that could be retrieved from the DB for some people)
<danilos> G: of course, I am not suggesting you should block on this, but just wondering if that's what you really want?
<danilos> G: also, as far as data being transferred from the DB, we do batch most of the big requests
<rvba> nigelb: Why are you rendering a view on bugTask in test_bugattachment_edit_view?
<nigelb> rvba: I should move it to another file, but I wanted to write it some place where attachment bits existed.
<nigelb> Is there a better place already existing?
<G> danilos: actually seems +bugs does that already
<rvba> nigelb: I don't know that code (bugs) very well but I /think/ the bug is not properly setup for you to render bugTask:+index
<nigelb> hmm.
<nigelb> I should perhaps tell you what I'm testing so you can suggest a better, working alternative.
<rvba> nigelb: My advice would be to find in the code a properly setup bug (one that ressembles what you want to test).
<nigelb> When you attach a patch on LP, it shows up in the comments, but the edit link is a portlet on the left. I'm adding that edit link to the link in the patch as well.
<nigelb> Ah, perhaps I should try that.
<danilos> G: yeah, though I'd never claim to know what exactly "related" means there
<rvba> nigelb: I see.
<danilos> G: at least not before reading the code carefully :)
<G> danilos: I think 'related' is something all could easily be lost in translation
<danilos> G: I also dislike the concept of me being related to a bug ;)
<wgrant> G, danilos: "Related" is a combination of all the filters.
<wgrant> s/combination/union/
<G> danilos: but the way I see it is: all bugs (x) has had something to do with
<wgrant> Stuff you're subscribed to, or assigned to, or reported, or commented on.
<nigelb> hrm, I could do a browser test I guess.
<danilos> wgrant, that simple? no team subscriptions considered?
<wgrant> danilos: Correct.
<danilos> wgrant, right, thanks
<G> ahhh so it doesn't consider via teams
<rvba> nigelb: that's already what you tried to do.
<StevenK> G: Good to see you tackling more issues. :-)
<G> StevenK: seems a bit of a waste to sign a document to contribute and stop after one issue :)
<nigelb> rvba: No, the from lp.testing.BrowserTestCase
<nigelb> StevenK: Isn't it fun that there are two new people contributing and both of us have the same first name ;)
<danilos> G: so, my gut feel is that we should be able to generate a single pages with all related bugs for any of the filters which includes your team subscriptions: basically, the "only" difference would be a join through TeamParticipation table :)
<StevenK> nigelb: That isn't confusing *at all*
<nigelb> heh
<danilos> G: s/single pages/self-contained pages/
<G> StevenK: that and, I really like Launchpad, and the code is much better than another unnamed set of tools are :)
<nigelb> wgrant: Can I hope for mildly better success with BrowserTestCase? :)
<G> danilos: actually, what I was wondering is if the best way is to create an AJAXy portlet that uses the team list, to basically: "Also show bugs assigned to: <set of checkboxes>"
<G> or even a "Showing directly (assigned|subscribed) bugs, [link: include indirectly (assigned|subscribed) bugs]
<danilos> G: yeah, though choosing through a bunch of checkboxes is a nightmare imho :)
<G> danilos: yeah, I agree
<G> but with an on/off type thing, it'd also be possible to still list the user's teams that implements the existing behaviour as well
<danilos> G: yeah, though user's teams are listed elsewhere as well (I'd even argue that it's misplaced on +assignedbugs now)
<G> danilos: well for someone with a lot of team memberships, it allows drilling down to individual teams easily, but I agree to that view as well
<danilos> G: true, true
<danilos> G: well, I think providing the list won't be bad or wrong, so I suggest you just go for it :)
<G> (I just realised as well, that it'd likely have to be a non-AJAXy because the searching is also non-AJAXy
<rvba> nigelb: I think the only place where this (adding a comment with a attachement) is tested is bugtarget-filebug-views.txt
<nigelb> doctest :(
<rvba> Indeed.
<G> Maybe just a portlet that basically has: http://dev.nigelj.com:81/bugportlet.txt
<rvba> nigelb: my advice would be to see what your fix does to that test. If you only need to add a couple of lines in there, it's fine.
<G> or is that getting a little too complex?
<bigjools> anyone else seeing update-sourcecode breaking when run from rf-get?
<nigelb> bigjools: the cache file?
<bigjools> yes
<nigelb> yeah, I saw that yesterday
<nigelb> I was told to just use the merged version in devel
<nigelb> there's a conflict
<wgrant> Someone probably updated it on an i386 machine.
<wgrant> So the order is different.
<rvba> nigelb: I'm sorry but I'll have to leave you to it now. Maybe danilos can give you a hand if you need more help.
<nigelb> rvba: Sure, no problem, I'll dig deeper.
<bigjools> bzr resolve --take-other FTW
<nigelb> yeah, that's what I did
<nigelb> *too
<G> danilos: does that look semi-sane, or a bit too long?
<nigelb> This may come off as exceedingly stupid. How do I run a doctest?
<danilos> G: I think it'd be fine
<bigjools> nigelb: bin/test -t <name of doctest>
<danilos> G: and definitely an improvement over what we have now
<nigelb> bigjools: ah, thanks!
<G> danilos: and basically the idea is for it be no-AJAX, and just add a includeindirect=1 to the request to +subscribedbugs
<bigjools> nigelb: remember that doctests are loaded by unit tests
<rvba> nigelb: ./bin/test -cvv -t xx-doctest.txt
<G> (or assignedbugs for that matter)
<danilos> G: I'd also do it in two steps (branches) to make it easier: generalize what we have in one branch and make it work for the subscribed bugs case, and then another one to provide the indirect bug searches
<danilos> G: yeah, that sounds good, though that may conflict with the search parameters
<G> danilos: so basically 1 branch to include it for subscribedbugs, and then a second branch to implement show indirect
<jelmer> hmm, is something up with loggerhead on staging?
<nigelb> bigjools: oh. that's why everyone tells me doctests having a slightly weird loading.
<danilos> G: yep, that also ensures you solve at least one problem
<G> danilos: ahhh a good point
<danilos> G: (if the "include indirect" turns out to really be too expensive)
<wgrant> jelmer: Probably. asuka has been in swapdeath twice in the last 24 hours.
<wgrant> jelmer: Stuff may need restarting.
<wgrant> jelmer: although codebrowse is tellurium, so I am stupid.
<nigelb> Gah, I fixed it and I don't see a text breakage.
<nigelb> *test
<nigelb> The one time I'm unhappy a test didn't break.
<bigjools> nigelb: this is an example of one: lib/lp/archivepublisher/tests/test_publisher_documentation.py
<jelmer> wgrant: ah, ok
<G> danilos: alright I'll type it up in launchpad
<jelmer> wgrant: either way, I guess should ask a losa
<jelmer> wgrant: what was up with asuka?
<danilos> G: cool, I'll be switching locations soon, but if you want me to comment on the bug as well, just let me know :)
<wgrant> jelmer: I'm not sure that was ever determined.
<nigelb> bigjools: Ooooh. thanks!
<G> danilos: thanks!
<jelmer> wgrant: hmm, that makes me feel a bit uneasy about the stuff that is currently on staging..
<wgrant> jelmer: You could obtain process listings.
<wgrant> jelmer: I believe they are taken every minute or 5.
<G> danilos: does person-portlet-team-bugs.pt should better than ...-team-assignedbugs.pt for a generic version?
<danilos> G: it does, yeah
<bigjools> it's quite amazing when DF manges to load a page that [qa]staging is timing out on
<wgrant> bigjools: You used TRUNCATE, didn't you?
<bigjools> hmm?
<wgrant> The only explanation for that is that you TRUNCATEd the table.
<bigjools> nope
<wgrant> Lies.
<nigelb> heh
<nigelb> Is asking for a review of a branch I couldn't successfuly write a test cheating?
<nigelb> :)
<nigelb> wgrant: Is this fix released? bug 833743
<_mup_> Bug #833743: test_versioninfo fails when system has bzr 2.4 <test-system> <Launchpad itself:In Progress by nigelbabu> < https://launchpad.net/bugs/833743 >
<wgrant> nigelb: Indeed.
<nigelb> woah, too fast.
<bigjools> what are the runes to make bin/test pick up the failing tests in a testr repo?
<nigelb> testr init ; zcat ~/Downloads/4595-upgrade-bug-linking-r13582.subunit.gz | testr load; testr run --failing
<nigelb> where the ~/Downloads path is the subunit file.
<nigelb> bigjools: is ^ that what you meant?
<bigjools> nigelb: perfect, thanks
<nigelb> :)
<bigjools> how does it know to use bin/test?
<jelmer> bigjools: the .testr.conf file
<nigelb> I just assumed StevenK's magic would work.
<bigjools> which begs another question ... :)
<bigjools> how did I have that config already
<jelmer> bigjools: it's part of the launchpad tree
<jelmer> :)
<bigjools> ah!
<nigelb> heh
<bigjools> that would make some sense, yes
<nigelb> danilos: Could I add https://code.launchpad.net/~nigelbabu/launchpad/patch-edit-684548 to your list? :)
<nigelb> I've been unsucessful in getting tests for it sadly. Any help appreciated :)
<stub> Is there a helper to make non-blocking reads on a Python file object, or am I stuck with select() boilerplate?
<bigjools> Twisted? :)
<bigjools> are you in a loop that keeps checking?
<nigelb> excellent, TDD dream works
<G> danilos: hey quick question, I can seem to find any other pages that does the whole pass variables to a portlet thing, do you know of any so I can have a look at one
<nigelb> GAH, qastating. Y U NO let me file bug.
<nigelb> bigjools: is bug 284707 still trivial?
<_mup_> Bug #284707: When Launchpad Janitor closes a bug due to a package upload, the message should link to that upload page <lp-soyuz> <trivial> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/284707 >
<wgrant> nigelb: Yes. Should just need the template changed to insert a link -- which can be easily generated using canonical_url.
<bigjools> nigelb: mostly, you just need to get the DSPR and use canonical_url() on it
<nigelb> But how do I test...
<bigjools> find the existing test
<nigelb> I mean, not programmatically
<bigjools> do an upload locally
<bigjools> and run process-upload.py
<nigelb> hm, this is going to be fun.
<wgrant> nigelb: Nah, it was fun 2 years ago.
<wgrant> Now it's well-documented and partially scripted :)
<wgrant> https://dev.launchpad.net/Soyuz/HowToUseSoyuzLocally
<nigelb> wgrant: I'm still stuck on the test for that bug. The "fix" turned out to be 2 lines. Once again haunted by tests.
<StevenK> nigelb: Is your TDD experiment working?
<nigelb> StevenK: I failed spectacularly.
<bigjools> do it for this one, really
<nigelb> The second bug I picked up turns out to have been fixed by wgrant a few months back.
<nigelb> Also, what kind of box does qastaging run on? netbook?
<nigelb> My local instance seems fast than that.
<StevenK> Hahaha
<bigjools> staging is timing out today
<StevenK> nigelb: qastaging *and* staging both run on asuka. The poor thing is running *two* Launchpads
<G> hmmmm I don't get how I can pass data to a template portlet
<bigjools> G: "view" and "context" contain what you think.  So you can do view/property or context/property
<nigelb> mrevell: "The Great Dismal Swamp" isn't some part of Launchpad? ;)
 * nigelb runs
<bigjools> I read "The Great Dismal Swamp" but said "Soyuz"
<nigelb> heh
<nigelb> I was thinking of blueprints though
<wgrant> nigelb: What did I fix?
<nigelb> bug 667413
<_mup_> Bug #667413: "HowToTriage" URL in "thankyou for reporting this ubuntu bug" box should be a link <lp-bugs> <trivial> <Launchpad itself:Fix Released> < https://launchpad.net/bugs/667413 >
<G> bigjools: basically I'm trying to generic-ize the person-portlet-team-assignedbugs.pt portlet, (/+assignedbugs) but I can't see any way to pass "+assignedbugs" etc to the portlet
<G> unless I'm lost somewhere
<wgrant> nigelb: qastaging and staging share an appserver (asuka) and a DB server (sourcherry). I don't know asuka's specs, but a fair bit of the bottleneck is the DB. sourcherry has 32GiB of RAM and has two LP DBs. The production DB servers have 132GiB of RAM (I believe), a single DB, and there are three of them.
<wgrant> nigelb: So I did. Thanks for finding that.
<nigelb> wgrant: 32 GiB of RAM?
<G> 132GiB of RAM? nice :)
<nigelb> Oh, right. It has the eentire prodcution data
<StevenK> nigelb: The production database is like 240GiB
<wgrant> nigelb: Yes. The LP DB is hundreds of gigabytes, so...
<bigjools> G: that's a traversal, you need to define that in zcml.
<bigjools> and there be dragons
<bigjools> oh look, it's lunchtime!
<nigelb> tea time!
<StevenK> Wow. I reviewed that when I was still being mentored.
<nigelb> StevenK: wgrant's branch?
<StevenK> nigelb: Yup
<nigelb> Neat :)
<G> bigjools: so is traversals for this a good thing, or a bad thing (i.e. if it's a bad thing, something that should not be generic-ized)
<nigelb> I didn't know you were a full reviewer only recently.
<StevenK> nigelb: Commit message for that revision you linked in the bug starts with [r=stevenk, thumper]
<StevenK> nigelb: Graduated 15/03
<bigjools> G: I don't know what you're trying to do exactly, but normally if you want a generic portlet you need to attach the template to a view in zcml, and then you can use /@@ in the other template that needs the portlet
<nigelb> StevenK: 2003?
<nigelb> Or March of 2011
<bigjools> but I am vastly simplifying
<StevenK> nigelb: No, March.
<nigelb> Fairly recent :)
<nigelb> I remember wgrant being a full reviewer.
<nigelb> err, getting full review
<nigelb> GRAR
<nigelb> *reviewer
<StevenK> Almost six months
<nigelb> I now know I've learned quite a bit in the last 9 MPs.
<nigelb> Especially when I see G go through what I as going through branches 2 to 5 ;)
<G> haha
<StevenK> And what I went through for branches 2 until 30
<henninge> danilos: Hi! Can you please review my branch?
<henninge> https://code.launchpad.net/~henninge/launchpad/bug-824435-failure-reporting-4/+merge/73507
<G> ohhhhh I think I get what I'm trying to do :)
<StevenK> (I was working on Soyuz, give me a break. :-)
<nigelb> heh
<G> (as crazy as that sounds)
<nigelb> G: I think most of us know exactly what you mean :D
<G> nigelb: yeah, I'm a bit of a zope newbie
<nigelb> StevenK: I still don't know lots. I'm frequently getting stuck at tests as evidenced today.
<nigelb> G: Join the club! :)
<G> nigelb: but I think I just worked out the solution to my question, even though I'm still not 100% sure I've been asking the right question (or at least the question the right way)
<nigelb> That sounds /very/ familiar.
<mrevell> nigelb, Heh :)
 * nigelb heads for tea.
<deryck> Morning, all.
<henninge> Hi deryck! ;)
<danilos> henninge, sure
<henninge> I am getting this error from the LibrarianLayer. It seems that something is wrong with my local python2.6 installation, specifically setuptools.
<henninge> http://paste.ubuntu.com/678866/
<henninge> danilos: cool, thanks
<henninge> What can to to fix that?
<wgrant> henninge: https://bugs.launchpad.net/ubuntu/+source/python-setuptools/+bug/834698
<_mup_> Bug #834698: setuptools.egg-info can end up as a directory when it is meant to be a symlink <python-setuptools (Ubuntu):New> < https://launchpad.net/bugs/834698 >
<wgrant> henninge: rm -rf /usr/lib/python2.6/dist-packages/setuptools.egg-info, and apt-get install --reinstall python-setuptools if it is installed.
<henninge> wgrant: ah thanks. I alread tried re-installing but did not remove first.
<henninge> yes, now it is a symlink
<henninge> and the LibrianLayer works, too .. :-D
<wgrant> This broke buildbot a bit, and is causing delights in the staging mailbox.
<wgrant> A dozen emails a minute or so.
<wgrant> And launchpad-error-reports is getting similar, although not quite so frequent.
<G> hmmm so I think StevenK was right, I can't do this the way I was thinking
<deryck> adeuring, don't worry about marking bugs fix released. looks like wgrant got them all.
<adeuring> ah, thanks wgrant and deryck!
<rvba> danilos: Hi, can you please have a look at https://code.launchpad.net/~rvb/launchpad/dsd-api-bug-833080/+merge/73532 ?
<G> although it does seem to me a bit odd that I can't access variables from view/foo from a portlet that is included via tal:content
<lifeless> \o/
<bigjools> lifeless: I can only assume that means one thing
<lifeless> :)
<bigjools> congrats :)
<lifeless> sleep now. details later
<bigjools> details?
<bigjools> gah!
<G> lifeless: congrats
<nigelb> lifeless: Congrats!
<mrevell> hey, congrats Daddy Lifeless :)
<rvba> lifeless: fÃ©licitations !
<danilos> henninge, r=me, sorry for taking this long
<james_w> congratulations lifeless
<danilos> rvba, looking at yours now
<rvba> danilos: Great!
<nigelb> This calls for a party at next sprint/summit lifeless attends? ;)
* danilos changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 244 - 0:[#######=]:256
<danilos> rvba, is this how we generally export SPNs?
<nigelb> danilos: ouch, I had a small branch I needed a review :)
<rvba> danilos: yes, IÂ took the code from somewhere.
<danilos> nigelb, if it's really small, I might be convinced :)
<danilos> rvba, cool, just wondering
<nigelb> https://code.launchpad.net/~nigelbabu/launchpad/patch-edit-684548
<nigelb> I really tried my best to write tests for that, but failed.
<danilos> rvba, I also assume all of these are public, and no concern should be for privileges?
<rvba> danilos: that's right.
<nigelb> Gah, I mean - https://code.launchpad.net/~nigelbabu/launchpad/patch-edit-684548/+merge/73514
<danilos> rvba, cool, everything else is nice and simple, r=me
<rvba> danilos: Thanks a lot!
<danilos> nigelb, have you looked at lib/lp/bugs/stories/bugattachments/xx-attachments-to-bug-report.txt for testing?
<danilos> nigelb, (I wouldn't be surprised if that starts failing with your change)
<nigelb> danilos: It didn't fail :(
<nigelb> wait.
<nigelb> that's not the one I tried.
<nigelb> *checks*
<danilos> nigelb, you might still be right, but it shouldn't be hard to extend it to test for this as well
<nigelb> danilos: I'll do that. I never thought I'd be writing doctests :)
<nigelb> Yup, I broke that.
<danilos> nigelb, heh, well, if you really want, you can start a new view test if you want :)
<nigelb> danilos: Tried for about 4 to 5 hours :)
<nigelb> failed.
<danilos> nigelb, anyway, please extend the test to assert that the link is right as well
<danilos> nigelb, oh, I've noticed you've been discussing that with a few people, fair enough
<nigelb> danilos: Just fix what I broken right? [I've been at this for 12 hours now :P]
<danilos> nigelb, you should try to at least extend the pagetest to make sure it works
<nigelb> pagetest?
<danilos> nigelb, well, not really, I think that is only using extract_text which strips all the tags
<henninge> danilos: thank you very much! ;)
<danilos> nigelb, yeah, all the doctests in stories/* subdirectories are what we used to call "page tests"
<nigelb> so, I added an (edit) in the doctest now, it passes.
<nigelb> do I need to add another new test?
<danilos> nigelb, well, you can at least check the actual link tag in there as well
<nigelb> ah, right.
<nigelb> let me look at other tests and figure out how
<danilos> nigelb, in that same spot there is a getLink call, you should be able to emulate that
<danilos> nigelb, something like browser.getLink('(edit)')
<danilos> nigelb, or well, just "edit" since that's what the link is
<nigelb> yeah, got that bit.
<henninge> gary_poster: Good morning! ;)
<nigelb> Now I'm looking around to figure out the pattern to the dit link
<nigelb> *edit link
<gary_poster> hey henninge, good afternoon. :-)
<henninge> ;)
<danilos> nigelb, "sprite edit" on the containing <span> might be enough
<henninge> gary_poster: I am not sure if you are aware but that ZConfig change I made a few weeks ago actually helped.
<henninge> gary_poster: Now I think it is time to submit it upstream.
<danilos> nigelb, or <a> itself (i.e. <a class="sprite edit"></a>)
<henninge> gary_poster: can you help me with that?
<gary_poster> henninge, I am most definitely aware!  It is great!  I made sure you were recognized on a team lead call :-)
<nigelb> danilos: Oh, I didn't add that class.
<henninge> gary_poster: thanks ;-)
<danilos> nigelb, that adds the edit icon on the link
<danilos> nigelb, also, considering you've got parentheses following the edit link now, the layout is not very pretty :)
<nigelb> danilos: Ah. I just copied the format of the edit in the right portlet as well
<nigelb> I'll add the class so I can grab it.
<gary_poster> henninge, help with upstream: /me cringes, but yes, can do.  I would suggest that one of us email Fred Drake.  He knows me, so I might be a better choice, but I'd be very happy to let you do it.  He is the ZConfig author, and the author of that particular machinery.  I'd ask him for a review and a merge if he's willing; or if he doesn't have the time, he can toss it back to me, and I'll do it.
<gary_poster> henninge, you want me to write that?
 * gary_poster starts email ;-)
<danilos> nigelb, do you think you can finish these changes quickly? (I need to leave soon and have a few other things I need to finish, so if you can't do it quickly, we can perhaps come back to it tomorrow morning)
<nigelb> danilos: Please don't block on me, if I can't get finish it, I can start again tomorrow, no issues :)
<nigelb> *get it
<nigelb> *finished
<danilos> nigelb, cool, thanks
<danilos> nigelb, feel free to ping me tomorrow morning then :)
<henninge> gary_poster: sorry, phone call
 * henninge reads
<nigelb> danilos: sure :)
<G> danilos: thanks for your help btw
<gary_poster> henninge, I'm three paras in by now  ;-)
<henninge> gary_poster: no, was reading your explanation. that's only one para
<henninge> gary_poster: what are you reading? ;)
<gary_poster> henninge, I mean I'm three paras in to the email that I'm writing to Fred Drake
<henninge> ha!
<henninge> :-D
<henninge> gary_poster: can you please write the email ... ;)
<gary_poster> :-)
<henninge> gary_poster: thanks ;)
<gary_poster> welcome, henninge.  thanks for the great fix
<bac> bigjools: how do i add an ArchivePermission for a user to a private PPA?  can it be done via the UI?
<bigjools> bac: only on the API
<bac> bigjools: ok then, i'll stop looking
<bigjools> bac: good idea :)
* abentley changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: abentley | Critical bugs: 244 - 0:[#######=]:256
<nigelb> I did wonder why a test wasn't failing.
<nigelb> I wasn running it in the WRONG branch!
<G> nigelb: whoops
<G> I think my understanding of the zope templating methods must be completely off
<deryck> bac, ping
<bac> hey deryck
<cjwatson> lifeless: congratulations!
<cjwatson> have we heard anything new regarding bug 809123?
<_mup_> Bug #809123: we cannot deploy DB schema changes live <fastdowntime> <qa-ok> <Launchpad itself:Fix Committed by stub> < https://launchpad.net/bugs/809123 >
<benji> G: the ZPT reference is pretty good: http://docs.zope.org/zope2/zope2book/AppendixC.html or I might be bribed into answering a question or two
<G> benji: I think part of it is that I'm used to how other templating methods work, and just getting a tad confused about the boundries in ZPT
<G> benji: essentially in bug 61428 it'd be nice to make person-portlet-team-assignedbugs.pt reusable for both assigned and subscribed methods, but that portlet isn't the one that is the first template loaded (buglisting-embedded-advanced-search.pt)
<_mup_> Bug #61428: Want a "subscribed to teams" portlet... <feature> <lp-bugs> <trivial> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/61428 >
<benji> G: I'm looking at the templates in question to see if I can get up to speed with you.
<nigelb> Is this lint error okay? " narrative uses a moin header."
<G> the idea is, in person.py, I was thinking it should be possible to tell the template that the backlink should be "+subscribedbugs" or "+assignedbugs" or "+somethinginthefuture", and I can certainly tell the buglisting-embedded-advanced-search.pt about it (by setting a variable in the python class)
<stub> cjwatson: Still blocked on RT #47040
<_mup_> Bug #47040: while booting initial kernel, progress bar looks like Win98 <gfxboot-theme-ubuntu (Ubuntu):Confirmed> < https://launchpad.net/bugs/47040 >
<stub> cjwatson: (Which had its one month birthday and nobody noticed! Happy Birthday!)
<G> but the portlet just seems to have no access to it
<benji> nigelb: it's not bad, or hard to fix (instead of "== Title ==" replace it with "Title\n====="
<nigelb> benji: cool, thanks
<G> (I even tried: tal:define="link_back view/link_back" tal:content="existing loading of portlet-team-assignedbugs", without any luck
<G> (which reading the ZPT documentation, sounded like it should work
<nigelb> benji: BTW, Your quickstart guide helped me understand a lot of how things work :)
<benji> G: define "without any luck" and "work" :)
<benji> nigelb: I'm glad
<G> without any luck = "Expression: <PathExpr standard:u'link_back'> with a list of variables excluding link_back
<G> benji: work as in <div tal:content="link_back"> displays what I had set in the view
<benji> G: the first error makes no sense to me, so I will ignore it ;) the second should have worked with something like "<div tal:content="view/link_back"></div>"
<benji> (assuming that the view really has an attribute or method named "link_back" that holds/returns a string)
<G> that just makes it "Expression: <PathExpr standard:u'view/link_back'>
<G> benji: hold on, just pastebin'ing now
<G> benji: http://paste.ubuntu.com/678981/
 * benji looks
<benji> G: the first change in person-portlet-team-assignedbugs.pt looks like a typo; why the space and then "view/name"?
<G> (btw, I've had it with the same issue, with and without the nocall: (thought it might fix it) and with/without the seperate <tal:... define></ta/:...>
<G> benji: oh you can ignore that
<G> benji: you are right in pointing that out, but I thought the way I was joining the two variables together was the issue, it isn't actually trying to render view/name though
<G> flacoste: btw, thanks for sorting out the Contributor Agreement
<flacoste> G: thanks for you contribution!
<G> flacoste: no problem, I was originally confused about the agreement and the Code of Conduct thing, so it didn't click until late that I hadn't done it
<benji> G: if I apply that patch, what URL could I visit to trigger the error?
<G> on my system: https://launchpad.dev/~name16/+assignedbugs
<G> so really launchpaddomain/~user/+assignedbugs where ~user belongs to one or more teams
<deryck> bigjools, ping.
 * bigjools is in demand today
<nigelb> abentley: Can I bring https://code.launchpad.net/~nigelbabu/launchpad/patch-edit-684548/+merge/73514 to your attention?
<abentley> nigelb: sure.  Be with you in a moment.
<nigelb> thanks!
<benji> G: if I apply your patch but leave out the broken changes to person-portlet-team-assignedbugs.pt, I don't get any errors when rendering that page
<G> benji: yep, and if you add in the <div tal:content="link_back" /> the oops will occur
<G> (ignore the rest)
<G> and variants such as view/link_back also didn't work
<benji> G: each template has its own namespace, so that name isn't defined there, you can put "view/" on the front of "link_back" and it should work (assuming both templates have the same view object)
<benji> ah, so it may have a different view
<G> benji: thats the 'class="lp...."' bit in the .zcml file?
<benji> G: indeed, if you look to see where that template is wired up (lib/lp/bugs/browser/configure.zcml) it has a view class of lp.registry.browser.person.PersonView
<G> hmmmm I just changed it to the same as the handler for +assignedbugs
<G> and no oops
<G> and it looks correct
<benji> cool
<G> benji: thank you very much!
<benji> glad to help
<G> can't believe I was so close, but so far away at the same time
<G> benji: oh except it won't be generic still, because +assignedbugs and +subscribedbugs use different classes
<G> benji: I think you've got me in the right direction though
<benji> G: good luck, if you get in another bind feel free to ask for help
<abentley> nigelb: it looks like danilos is your reviewer on that one.
<nigelb> Yeah, I thought I'd finish it off.
<nigelb> You'd rather defer to him, I can put it off for tomorrow :)
<abentley> nigelb: I'd prefer that.
<nigelb> Cool, np
<nigelb> Onto next bug :)
<adeuring> abentley, deryck[lunch]: could one of you review a small fix for a larger part of the HWDB hassles? https://code.launchpad.net/~adeuring/launchpad/fix-broken-comment-nodes-in-hwdb-reports/+merge/73552
<abentley> adeuring: sure.
<adeuring> abentley: thanks!
<abentley> adeuring: ISTM that your re would match "<comment></comment><somethingelse></somethingelse><comment></comment>"
<adeuring> abentley: no, the '?' after '.*' prevents this
<abentley> adeuring, doesn't ? make the '.*' optional?
<adeuring> abentley: no, that means "non-greedy searching"
<adeuring> like "stop at the first match"
<abentley> adeuring: okay.  I didn't realize that *? was a single entity.
<abentley> adeuring: could you add a test for that, please?  Showing that "<comment></comment><somethingelse></somethingelse><comment></comment>" => "<comment/><somethingelse></somethingelse><comment/>" ?
<adeuring> abentley: not today.... I am really tired. I see the need but...
<abentley> adeuring: _fix_frequent_errors doesn't match our naming guidelines.
<adeuring> abentley: ouch, yes fixFrequentErrors, sure
<abentley> adeuring: with a test and the name change, this is okay to land.
<adeuring> abentley: thanks
<bigjools> mrevell: I requested your mince pies on a text change MP if you would be so kind
<mrevell> Haha, I'd be delighted to.
<nigelb> mrevell: Excellent email.
<bigjools> mrevell: splendid old bean
<nigelb> I'd be glad to be part of it as a contributor :)
<mrevell> Ah, thanks nigelb; you red fast!
<nigelb> heh
<mrevell> Very glad to have you on board :)
<nigelb> Wouldn't the blueprint bugs merge conflict with stakeholders?
<nigelb> Especially since Ubuntu uses it extensively.
<mrevell> nigelb, We won't throw away functionality that's important to stakeholders ... nor the distinction between a "blueprint" face and a "bugs" face, probably. It's more an under the hood thing that'll give us benefits such as bug dependencies.
<nigelb> Ah.
<mrevell> And it'd bring some much needed love to blueprints.
<nigelb> heh
<nigelb> YEah
<mrevell> jml and mpt started that work at our Dallas sprint.
<nigelb> Yeah, jml told me about the merge at UDS
<bigjools> good night everyone
<LPCIBot> Project devel build #1,017: STILL FAILING in 4 hr 45 min: https://lpci.wedontsleep.org/job/devel/1017/
<flacoste> deryck: your squad is on maintenance rotation this week?
<deryck> flacoste, yes.  on interrupt duties, you mean?
<flacoste> yes
<deryck> flacoste, yeah, indeed we are.  what's up?
<flacoste> did any of you guys picked up the email to Registry asking for https://launchpad.net/libreoffice/ and https://launchpad.net/df-libreoffice/ to be merged?
<flacoste> (in the latter)
<nigelb> Hi, what's the ideal way to test a validation change?
<nigelb> (I'm trying to fix bug 59301)
<_mup_> Bug #59301: Don't give me a vague "URL is already registered by another blueprint" error <lp-blueprints> <trivial> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/59301 >
<deryck> flacoste, I don't think so.  I can pick it up, though.  I'm the only one left today.
<flacoste> deryck: ideally, this should come through a question
<flacoste> nigelb: where are you implementing the validation change?
<deryck> flacoste, should I ping back and ask for them to open a question?  is that the normal way?
<flacoste> nigelb: iow, in the view, or the field validator
<nigelb> flacoste: In the interface
<flacoste> nigelb: field validator? or as an interface invariant?
<nigelb> er, I probably don't know what's the difference.
<nigelb> There's already validation
<flacoste> deryck: i'd suggest opening a question for it and pointing the user to the question for follow-up
<nigelb> I'm only changing the validation error
<deryck> flacoste, ack.  thanks.
<flacoste> deryck: one aspect that needs follow-up is that we don't support project merging per se
<flacoste> deryck: we can make one inactive
<deryck> right.
<flacoste> deryck: but it would be their responsibility that all contents is moved across
<flacoste> so bugs / translations / code / whatever
<deryck> gotcha
<flacoste> nigelb: what file are you changing?
<nigelb> blueprints/interfaces/specification.py
<nigelb> specifically L136
<nigelb> I'll be adding the specification title as well.
<nigelb> and if I can manage it, the URL.
<flacoste> nigelb: ok, looking for a decent suggestion
<nigelb> flacoste: :)
<nigelb> Is interface the Right Placeâ¢ for validation?
<nigelb> (Also, drat bigjools isn't here. Really wanted to poke him to look at cricket :P)
<flacoste> nigelb: yes, it's the right place
<nigelb> Ok, so theoretically, I should be able to look at other validation tests, if any.
<flacoste> nigelb: there is no interfaces/tests directory
<flacoste> nigelb: well, there is probably more bad examples of those than good ones
<nigelb> oh :(
<flacoste> especially in blueprints
<nigelb> heh
<nigelb> blueprints is especially unloved :)
<flacoste> so my suggestion is to add a unit test in blueprints/model/test_specification.py
<flacoste> ideally, i would go in interfaces/tests
<flacoste> but since that doesn't exists, might be more complicated to set-up
<flacoste> more cookie points to you if you go that extra miles
<flacoste> but the test should simply be something like
<flacoste> test_url_validation(self):
<flacoste>      spec = self.factory.makeSpecification()
<flacoste>        field = ISpecification['url']
<flacoste>          with ExpectedException(LaunchpadValidationError, regex_to_match):
<flacoste>             field.validate(the_value)
<flacoste> and that's it
<nigelb> s/url/specurl/g I think?
<flacoste> right
<flacoste> the above is a sketch :-)
<flacoste> you probably want one test per error condition you need
<nigelb> heh, ok. let me play with this
<flacoste> and you might want to validate a good one for completeness
<flacoste> that's how validation tests should be written
<flacoste> we probably have one example of those in the tree
<flacoste> but the more common cases was to test it through all the stack
<flacoste> which is bad practice
<flacoste> oh
<flacoste> you might want to use:
<flacoste>    field = ISpecification['specurl'].bind(spec)
<nigelb> Also, another thing.
<nigelb> that above test will probably pass.
<flacoste> since that allow the field to look at the object itself
<flacoste> nigelb: regex_to_match is matched against the exception message
<flacoste> so probably not if that's what needs to change
<nigelb> because the test failure is when you try to create a new spec with apiurl which exists.
<nigelb> so I'll just do an empty makeSpecification before that.
<flacoste> indeed
<flacoste> good point
<flacoste> existing_spec = self.factory.makeSpecification(specurl='...')
<flacoste> ISpecification['specurl'].validate('...')
<flacoste> should trigger your error
<nigelb> yeah :)
<flacoste> validate() works for both new object and existing ones
<flacoste> so you probably want a test that shows that validate('a_url') when the current value is 'a_url' pass
<flacoste> you'll need to call .bind() with an existing instance for that test
<flacoste> (as outlined in my first example)
<nigelb> right!
<flacoste> or correction to it :-)
<flacoste> that should get you started
<nigelb> so I get to do a success and failure.
<flacoste> let us know if you hit any snags
<nigelb> sure! :)
* abentley changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 244 - 0:[#######=]:256
<nigelb> Excellent. Finally got TDD working.
<nigelb> My tests fail successfully.
<nigelb> flacoste: Almost a success. All I'm missing is that validation is automatically escaped.
<flacoste> nigelb: you could simply check that the exception message is a structured() instance
<nigelb> flacoste: No, I mean, not just in the test. also in the output
<nigelb> so, I get <a href=... instead of that getting linked
<flacoste> nigelb: i don't understand?
<nigelb> one sec
<nigelb> flacoste: http://people.ubuntu.com/~nigelbabu/validation.png
<flacoste> nigelb: i see
<flacoste> that's more tricky
<nigelb> Ouch.
<flacoste> as we don't want to become another XSS vector
<nigelb> Can I use one of the formatters instead?
<flacoste> i think we now have a way to have HTML in error message
<flacoste> but i don't know how off the top of my head
<nigelb> Could I use the FormattersAPI?
<flacoste> nigelb: you need to pass use structured() to wrap the message creation
<flacoste> so structured('%s is already registered by <a href="%s">%s</a>', value, canonical_url(spec), spec.title)
<flacoste> that will do the right thing
<nigelb> okay!
<flacoste> bonus points if you can figure out how to use the the link formatter from there
<flacoste> note though that in some way this is a violation of our layering
<flacoste> we are introducing display logic at a very low level
<nigelb> Oh.
<nigelb> Wait, link formatter? Here?
<flacoste> putting a link in the error message is
<flacoste> more so if we going to reuse the implementing behind fmt:link
<nigelb> But is there a better way to do this?
<flacoste> but this seems to be the pattern we follow
<flacoste> the better way would just be more work
<flacoste> and indirection
<flacoste> for no real benefit
<nigelb> heh :)
<flacoste> you would make a subclass of LaunchpadValidation specific to this error
<flacoste> and then in the view look for that error
<flacoste> and replace the better error message there
<flacoste> that would preserve the layering of concerns
<flacoste> one drawback here is that there "HTML-formatted" error message will appear on the API as well
<nigelb> ah
<flacoste> but not sure if that's arguably a bug :-)
<nigelb> heh
<flacoste> anyway, using structured() wiht LaunchpadValidationError has precents
<flacoste> as its ErrorView supports it
<flacoste> and grepping should find prior art
<flacoste> so that should work for you :-)
<flacoste> ntrtam
<flacoste> -> need to run to a meeting :-)
<nigelb> ciao :)
<nigelb> Thanks!
<bac> benji: could i bother you for a review?
<benji> bac: my pleasure
<bac> benji: thx: https://code.launchpad.net/~bac/launchpad/bug-828914-test/+merge/73585
<benji> bac: looks good!
<bac> tahnks
<benji> vim protip: if you have ever lost some important edits down an undo/redo blind alley, look up the :earlier command
<benji> bac: do you have time for a very small review?
<bac> yep
<benji> here you go https://code.launchpad.net/~benji/launchpad/bug-810113/+merge/73591
 * benji probably should have mentioned bac's name.
<bac> i saw it.
<bac> benji: done
<benji> ah!, thanks
<bac> flacoste: that's a good idea.  i'll look at a sweep tomorrow
<soren> benji: Awesome review music.
<benji> soren:  :)
<flacoste> bac: thanks for fixing this!
<benji> bac: "Your review would've been much faster if it weren't for the insane music."  I LOLled
<soren> benji: Just bought the whole album. :)
<benji> soren: cool, he definately has some skill at what he does
<mwhudson> er
<mwhudson> would anyone like to talk about the lp javascript with me?
<mwhudson> ah
<LPCIBot> Project devel build #1,018: STILL FAILING in 3 hr 57 min: https://lpci.wedontsleep.org/job/devel/1018/
<mwhudson> "These installation instructions apply to the new PostgreSQL database infrastructure as found in Ubuntu Breezy and later, and Debian Etch and later. "
 * mwhudson thinks this line of prose has probably outlived it's usefulness
<lifeless> nuke from orbit
<wallyworld> mwhudson: did you find someone to talk about lp javascript?
<mwhudson> wallyworld: no
<wallyworld> did you have a question? not sure if i can help or not
<mwhudson> wallyworld: well, i dunno
<mwhudson> wallyworld: i sort of answered my own question
<wallyworld> ok
<mwhudson> wallyworld: do you know much about the ChoiceSource & related widgets?
<wallyworld> i know what i need to to use them
<mwhudson> there is code knocking around in there to replace icons with loading icons and so on
<mwhudson> but _most_ usages in lp now use sprites
<mwhudson> & so have to do different things
<wallyworld> yeah
<mwhudson> i was sort of vaguely wondering whether it makes sense to change the widget to assume sprites
<wallyworld> when you say loading icons, you mean display a spinner while loading takes place?
<mwhudson> wallyworld: yes
<mwhudson> wallyworld: the context for all this is that i've write a greasemonkey script for editing work items
<mwhudson> http://people.linaro.org/~mwh/work_item_editor.png
<wallyworld> the other pattern i've seen is to add "spinner" to the class of an item
 * wallyworld looks
<mwhudson> i had to copy paste hack the source of ChoiceSource to get this to work
<wallyworld> and you want to show a spinner when you clock the yellow edit icon?
<mwhudson> and i'd like to get my changes into lp
<mwhudson> no
<mwhudson> i want to understand the code a bit to know if my changes will do damage :)
<wallyworld> ah right. ChoiceSource is used in only a few places from memory eg blueprints
<wallyworld> i can look at your branch and see if anything jumps out if you like
<mwhudson> bugtask this affects me uses it too
<mwhudson> but yeah, it's fairly isolated
<mwhudson> wallyworld: https://code.launchpad.net/~mwhudson/launchpad/ChoiceSource-flexibility
 * wallyworld looks
<mwhudson> i should check if my greasemonkey script works with my changes too, or this is a bit pointless :)
<wallyworld> :-)
<mwhudson> hm
<mwhudson> wallyworld: why would LPS.WidgetPositionAlign be undefined on a blueprints.launchpad.dev page but there on production or staging?
<mwhudson> i guess the js is built differently?
<wallyworld> mwhudson: good question. i've recently noticed that too. the same source files are used in both dev and prod, but prod is minified, that's the only difference afaik
<wallyworld> mwhudson: we recently made a change to the build prcess so that yui is incuded as a tarball in downloadcahce and unpacked as part of the build
<wallyworld> so we can easily use a different version by modifying versions.cfg
<wallyworld> but i can't see offhand how that would account for the difference
<mwhudson> i can't really see how that would matter
<mwhudson> i think i can make it work by wrapping all my script in a LPS.use( '...', function (Y) { ... });
<wallyworld> yeah, so something else is happening
<mwhudson> which is probably a better idea anyway
<mwhudson> (it may be an undeployed change)
<mwhudson> ah uh, could it be a race condition?
<wallyworld> your branch looks ok to me. you could run all the yui tests but i don;t think we have coverage for what you've changed
<mwhudson> i don't know when greasemonkey scripts run
<wallyworld> could be
<wallyworld> i can't recall the exact error i've seen in the firebug console wrt an undefined symbol
<mwhudson> no, that doesn't make sense
<wallyworld> mwhudson: just checked lp.dev and lp.net. both give this error: yui: NOT loaded: widget-position-ext. so i misremembered what i saw. not related to your issue perhaps
<mwhudson> ah yeah i see that too
<wallyworld> but there's no apparent problems
<mwhudson> i don't see that on devel though...
<wallyworld> i do
#launchpad-dev 2011-09-01
<wallyworld> maybe you have an old yui lying around?
<wallyworld> perhaps make clean will make it show up
<mwhudson> well, i just set up the devel environment just now...
<wallyworld> hmmm.
<wallyworld> when i get a spare moment later i'll try and see what we're missing from the jsbuild target to cause the issue
<mwhudson> ah
<mwhudson> i need to load widget-position-align
<wallyworld> makes sense
<mwhudson> it has to be said, yui3 makes for very nicely greasemonkeyable pages
<wallyworld> yeah, from what i've seen yui3 rocks compared to earlier incarnations
<wallyworld> too bad yui isn't more popular. seems like jquery has all the mindshare?
<mwhudson> yay i can use the lazr widget with my lp changes
<wallyworld> mwhudson: when you say lazr, you mean from the lazr-js tree? for lp, we've stopped using lazr-js separately and pulled all the source in the lp tree and are mainatining it there
<mwhudson> wallyworld: no, i mean, not my hack that's in my greasemonkey script
<wallyworld> ah ok
<mwhudson> wallyworld: ok, i'd like to merge this branch i guess
<mwhudson> wallyworld: if i propose it will you review it?
<mwhudson> i'll run the yui tests if you tell me how :)
<wallyworld> sure. but i'm only a provisional reviewer. i'm being mentored atm
<wallyworld> mwhudson:  bin/test --layer=YUITest
<wallyworld> mwhudson: curtis has done an awesome job packaging the yui tests to be able to be run from the command line without  requiring a browser
<wallyworld> you can also run individual tests by loading the html page corresponding to the test js file in a browser
<mwhudson> yay
<mwhudson>   Ran 44 tests with 0 failures and 0 errors in 1 minutes 8.693 seconds.
<wallyworld> whoot
<mwhudson> hm, seems i don't need the editicon change any more
<mwhudson> think it's still dead code though
 * mwhudson repurposes wallyworld's existing mp
<wallyworld> mwhudson: i did that mp to quickly see the diff :-) i was just about to mention that the change now seems redundant.
<G> (potentially silly question alert) when running 'testr run', 'make run' should be killed right?
<wgrant> G: Shouldn't need to be. But it's possible some dodgy tests might require it.
<G> wgrant: okay thanks
<LPCIBot> Project devel build #1,019: STILL FAILING in 3 hr 59 min: https://lpci.wedontsleep.org/job/devel/1019/
<thumper> hi hackers
<thumper> how is the long-poll stuff going?
<thumper> I've been eagerly awaiting the auto-diff updating
<thumper> but it isn't happening :(
<StevenK> Waiting for red squad to finish derivation, and then they switch to long-poll
<spm> StevenK: isn't the correct repsonse "patches accepted"?
<StevenK> Damn it. I go on holidays for a week and I forget everything.
 * StevenK books in "re-training" for Monday
<wallyworld> thumper: how's dx?
<lifeless> digitally extreme :P
<wallyworld> lifeless: you had a kid yet?
<thumper> wallyworld: interesting
<lifeless> yes :)
 * wgrant gives lifeless a few looks of disapproval.
<thumper> lifeless: really?
<wallyworld> lifeless: congrats. details please
<wgrant> It was on G+ and FB hours ago!
<thumper> did I miss the announcement?
<wgrant> Social media, people :P
<lifeless> g+ 2 minutes ago
<lifeless> FB hours ago.
<wallyworld> wgrant: i hate social media
<lifeless> EBUSY.
<thumper> lifeless: congrats
<wallyworld> lifeless: so why ffs are you on irc?
<lifeless> wallyworld: easier than ringing the 100 off folk that Want To Know
<lifeless> s/off/odd/
<wallyworld> understood
<wgrant> Heh
<wallyworld> lifeless:  so, for those of us who hate fb, g+ - boy, girl?
<wallyworld> thumper: got my new coffee machine :-D Breville BES900
<thumper> wallyworld: awesome
<thumper> wallyworld: so now you never leave home?
<wallyworld> thumper: nope :-)
<wallyworld> thumper: so feature freeze on oneric. everything looking good?
<wallyworld> is the minimise behaviour or global menu stuff fixed?
<wallyworld> please say it is
<wgrant> wallyworld: Heh, good is not how Ubuntu release cycles ever look until the last two weeks.
<wallyworld> wgrant: understood. i was being optimistic :-)
<thumper> wallyworld: no
<wallyworld> thumper: you suck
<wallyworld> :-P
<thumper> wallyworld: no, you suck
<wallyworld> sometimes
<lifeless> wallyworld: answered privately :)
<wallyworld> lifeless: ah, so it is. thanks
<wallyworld> lifeless: love the slogan :-)
<lifeless> indeed ;)
<G> $ testr run - 3hrs 20min and counting, is this normal?
<StevenK> Yes
<StevenK> Sadly
<G> on the plus side, I did find a failing test in my changes :)
<StevenK> A full testsuite run will take approximately 4 hours and change
 * spm quaintly recalls complaining about the 90minute test runs
<G> (Don't take it for complaining, was just shocked/surprised)
<jtv> StevenK, wgrant: I tried an experimental domination run (source only and without judging phase) on dogfood's Debian yesterday.  I don't think it did anything because it ran suspiciously fastâ¦ I'll have to check.
<StevenK> G: I complain about it all the time.
<G> the only real problem I see is the inability to have 'make run' and 'testr run' going at the same time
<StevenK> That's why we use ec2 for test runs.
<StevenK> But that can get expensive.
<wgrant> G: What happens when you try? It should mostly work.
<G> wgrant: address already in use
<G> I'm guessing because testr has zope running on the same port that make run starts zope on
<wgrant> That's meant to not happen any more :(
<G> wgrant: seems to be happening on my VM that I set up
<wgrant> Which layer?
<wgrant> AppServerLayer just started up fine for me.
<wgrant> and I have a dev appserver running.
<G> http://pastebin.ubuntu.com/679377/
<G> runlaunchpad.start_launchpad()
<G> StevenK: by the way, thanks for the help last night
<StevenK> G: I didn't think I helped much, but you're welcome. :-)
<G> I ended up by solving it by turning the second template into a Macro in the end, but you got me thinking in the right direction
<nigelb> Good Morning!
<G> nigelb: good $localtime
<nigelb> heh
<G> testr run just finished
<G> lots of bzr tests failed hmmm
<wgrant> G: Which Ubuntu release are you using?
<G> natty
<G> I'm guessing that is why
<wgrant> Possibly. The test suite is only regularly run on Lucid, but it used to work on Maverick and mostly works on Natty.
<wgrant> And partly on Oneiric.
<nigelb> Ursinha: Happy Birthday!
<nigelb> StevenK: I DID IT.
<nigelb> One branch with TDD :)
<nigelb> (except now my test doesn't pass)
<G> nigelb: tut tut tut ;)
<nigelb> heh
<LPCIBot> Project devel build #1,020: STILL FAILING in 3 hr 59 min: https://lpci.wedontsleep.org/job/devel/1020/
<nigelb> what time zone is danilos?
<G> nigelb: LP says UTC+02
<nigelb> ah, right.
<nigelb> How do I compare strings in regular expression
<nigelb> I'm surpised that I have to ask this at all.
<nigelb> Someone has thoughts on this? http://pastebin.ubuntu.com/679424/
<wgrant> nigelb: It doesn't really look like you want a regular expression match.
<nigelb> wgrant: I don't,no.
<nigelb> what do I use instead?
<wgrant> Does assertIn not do what you wish?
<nigelb> Oh.
<nigelb> Hrm, not sure how I would do that.
<G> oh no....
<G> my make lint was clean, until I modified the pagetests
<wgrant> That's your first mistake right there.
<nigelb> heh
<G> now it includes various 'source has bad indentation' / 'source exceeds 78 characters'
<nigelb> G: lint checks modified files
<G> nigelb: yep, get that, just disappointed
<nigelb> heh
<wgrant> G: If it's not too huge, you should probably clean the doctest up, so it's not left for the next person to find the same way.
<wgrant> But if it's a few hundred warnings like they sometimes are, meh.
<G> should I try and fix it as part of the branch or propose a second merge to fix
<G> looks /ike 68 errors
 * G takes a look
<wgrant> Most of them probably just need you to s/^   /    / or so.
<nigelb> who's reviewer today? Technically lifeless I believe?
 * nigelb checks
<wgrant> Don't give him any ideas.
<wgrant> I can have a look if nobody else is around.
<nigelb> aha
<nigelb> well, not ready yet.
<G> wgrant: looks like the lint errors are a bit 'yeah right' to me
<wgrant> G: Oh?
<nigelb> OMG.
<G> for instance, I can't see any bad indentation on ./lib/canonical/launchpad/pagetests/basics/notfound-traversals.txt
<G> (line 7 that is)
<nigelb> I totally didn't recognize this launchpad feature till now.
<wgrant> G: The code blocks are meant to be indented by 4 spaces.
<wgrant> To distinguish them from narrative.
<nigelb> I was about to say that.
<wgrant>     >>> if foo:
<wgrant>     ...     bar
<G> wgrant: oh....
<wgrant> nigelb: Which feature?
<nigelb> So, if I have a bug number in my branch name
<nigelb> like foo-1234
<wgrant> nigelb: That feature is about a month old.
<nigelb> HA.
<nigelb> Its awesome.
<G> in that case I'd have to re-indent etc the whole file
<wgrant> G: Yes. It's rather unpleasant. I just ignore it in that case.
<wgrant> I might do a mass fixup over Christmas or something, when it's less likely to conflict with people.
<nigelb> ..
<G> nigelb: oh does it automatically associate the branch w/ the bug?
<G> I just do 'bzr commit --fixes lp:<bug>'
<nigelb> G: It doesn't. When I click the link to associate the bug, it already has the bug number from the branch
<wgrant> So it suggests it.
<nigelb> (well, I often forget to do that :D)
<wgrant> But doesn't do it automatically.
<G> oh yeah, I noticed that today too
<nigelb> which is nice!
<nigelb> But yah, I agree with wgrant, it needs to be a picker.
<nigelb> wgrant: how would I use assertThat in this scenario
<nigelb> https://code.launchpad.net/~nigelbabu/launchpad/specification-validation-59301/+merge/73631
<nigelb> L77
<wgrant> nigelb: You should probably use assertRaises.
<wgrant> nigelb: It's normally just used to check the class of the exception, but it also returns the exception object.
<wgrant> So you can then assertEquals on its content afterwards.
<G> ha, just got my first buildbot e-mail :) a couple of hours ago
<nigelb> Wait, why aren't you on Contributors page yet.
<nigelb> G: FYI - https://dev.launchpad.net/Contributions
<wgrant> Possibly the wiki upgrade... let me see.
<nigelb> we're all competing with wgrant.
<wgrant> I am invincible, nyahaha.
<G> I can wait :)
<nigelb> I'm landing 3 branches a week or at least trying to.
<nigelb> That means about a year until I beat wgrant :D
<G> hopefully I'll have 2 this week at least
<nigelb> hrm, I don't understand how to use assertRaises.
 * nigelb greps more
<G> pondering snatching up bug 410331 tbh
<_mup_> Bug #410331: PPA: should default to sensible/good name (or give example) <easy> <lp-soyuz> <ppa> <trivial> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/410331 >
<G> nigelb: are you having trouble w/ 59301 ?
<nigelb> G: I fixed the bug.
<nigelb> I'm fixing the test.
<nigelb> I always seem to have some or the other trouble with tests :D
<G> nigelb: whats the command to run the broken test?
 * G wants to have a look too
<nigelb> well, you just run the test and you'll see its not passing.
<nigelb> wgrant: I don't see a good enough example of assertRaises to understand how it works
<wgrant> nigelb: e = self.assertRaises(SomeException, some_function, some, arguments)
<wgrant> It will call some_function(some, arguments)
<wgrant> And assert that it raises a SomeException.
<nigelb> mmm
<mrevell> Hello
<nigelb> Morning mrevell!
<mrevell> Hey nigelb :)
<wgrant> nigelb: Contributions page unbroken.
<nigelb> wgrant: And I can use the 'e' to get the error message for an assertEqual?
<nigelb> wgrant: Ha, thanks!
<wgrant> nigelb: Yeah. Use str(e), perhaps.
<nigelb> Excellent, thanks.
<wgrant> Woah.
<nigelb> Heh
<nigelb> THIS NEEDS CELEBRATION.
<nigelb> At least we need to give him a certificate - "I purposefully quit IRC for X days"
<nigelb> spm: ^
<jtv> wgrant: well, I found out why domination on dogfood's debian was so fast.  All those active SPPHs for Debian are PENDING, not PUBLISHED.  Shouldn't gina set them to PUBLISHED?
<spm> nigelb: haha
<spm> nigelb: tbh, I call lies.
<nigelb> hehe
<rvba> Morning!
<nigelb> Morning rvba!
<rvba> G'day nigelb.
<nigelb> wgrant: Could you review https://code.launchpad.net/~nigelbabu/launchpad/specification-validation-59301/+merge/73631
<nigelb> I'm sure you'll find at least 3 things to be picky about ^-^
<wgrant> jtv: Probably, yes. Not sure why it doesn't.
<wgrant> jtv: Probably a holdover from when it imported Ubuntu.
<jtv> It did?  Owww
<wgrant> jtv: They needed to be Pending so the publisher would publish them.
<wgrant> That's how Ubuntu ended up in LP, yeah :)
<rvba> Unless you're playing with it ATM, could one of you guys update DF?
<wgrant> Feb 2006.
<stub> jtv: I lost your review stamp when I resubitted a MP (added a new pipe so needed to change the dependency branch). Can you click some buttons, and optionally review the nonblocking_readline()? https://code.launchpad.net/~stub/launchpad/branch-rewrite/+merge/73563
<jtv> rvba: I already Q/A'ed your ISD fix.
<stub> jtv: nm. just hit refresh :)
<rvba> jtv: I need it for another QA ;)
<wgrant> nigelb: Could you convince someone else to review? I'm trying to disappear now.
<rvba> jtv: but thanks for Q/Aing my fix for ISD.
<nigelb> wgrant: Ok
<jtv> stub: but I already approved that twice!
<nigelb> jtv: Can I grab your attention for a quick review? :)
<stub> yup. Needed to reload the page.
<jtv> stub: ah, you found it.  Sorry, fell a little bit behind with 4 simultaneous conversations.  :)
<jtv> nigelb: welcome, conversation #5!  :-)
<nigelb> heh
<jtv> Let me just catch up with the other 4.
<nigelb> I'll grab lunch meanwhile.  Here's the MP - https://code.launchpad.net/~nigelbabu/launchpad/specification-validation-59301/+merge/73631
<jtv> rvba: I tested your ISD fix because I thought we might be able to roll out some critical fixes.  But unfortunately it looks like abentley is the remaining blocker.
<jtv> rvba: how fresh does dogfood need to be?  Because I updated a few hours back.  I can easily do it again if you need it though.
<rvba> jtv: I tested it and I'm sure my changes are not on DF now.
<jtv> wgrant: sorry, got a bit distracted with various other conversations (including phone call from a friend who it turns out is turning back from the brink of death :-)
<rvba> It says my revisions where included in db-devel 2 hours ago.
<jtv> rvba: ok I'll do that again
<rvba> jtv: thank you.
<jtv> rvba: it'll take a bit longer than usual (though not as long as when that db patch wouldn't apply!) because I'm currently testing a script.  Hence the question.  But right after that I'll update & notify.
<jtv> wgrant: I'm currently running domination on debian, with the PENDING statuses updated to PUBLISHED.  Obviously it's going to be fairly intense.
<rvba> jtv: There is no rush for me to QA that /right now/ please take your time. Please just ping me when it's updated.
<jtv> rvba: btw is it appserver?  script?  both?
<rvba> jtv: it's an api change so appserver.
<jtv> OK
<wgrant> rvba: Can't you do that on qastaging?
<wgrant> jtv: Probably.
<rvba> wgrant: Oh ... I'm so used to qa things on DF that I did not even think about that ... ;)
<jtv> \o/
<wgrant> jtv: How does gina currently handle the multiple versions case? I forget if it imports both or just the latest.
<wgrant> jtv: If it imports all present versions, it's going to be amusing.
<jtv> wgrant: urrrâ¦ I think it just imports everything it finds.
<wgrant> jtv: Because each run will create all the old versions, then supersede them all.
<jtv> I was shocked the other day to find out I may accidentally have made myself the go-to gina coder.
<rvba> nigelb: If you're able to wait for 1 hour an official reviewer will be on duty.
<wgrant> Then the next run will create them all and supersede them again.
<jtv> wgrant: no I do think it skips everything that's already in the db; but if it finds multiple _previously unknown_ versions it imports them all.
<wgrant> jtv: I know bits of it from 2 years ago, but try to suppress those memories. It's not very relevant to normal LP development, so it gets paged out quickly.
<nigelb> rvba: Ha, just when I get back from lunch. Excellent.
<wgrant> jtv: Ew, that would be even worse.
<jtv> wgrant: it's like a clever woman I know who, when switching from microbiology to IT and clearly short of relevant experience, still left APL off her CV.  âNot getting the job scares me less than being made to work in APL.â
<jtv> I told her to chance it.
<wgrant> Heh.
<wgrant> But the behaviour you suggest would be very inconsistent, confusing and wrong.
<jtv> In what way?  I have a feeling we may be talking at cross purposes.
<rvba> (QA ok on qastaging \o/)
<jtv> OK
<wgrant> jtv: The dominator and gina will fight, creating and superseding over and over again.
<jtv> Why?
<jtv> Ah, I see the misunderstanding.
<jtv> wgrant: cancel
<wgrant> If gina imports all versions referenced in Sources, it will import stuff that the dominator has marked as superseded.
<wgrant> Heh.
<jtv> wgrant: When I said âthem all,â I was referring to the last bunch that I mentionedâwhen gina finds multiple previous unknown versions, AIUI it imports _all the unknown versions_.
<wgrant> jtv: What's "unknown"?
<jtv> Because it skips everything that's already in the db.
<wgrant> I'm not clear on the definition of "in the DB."
<jtv> I think the criterion was ânot having an SPPH.â
<wgrant> An active SPPH?
<jtv> No, I don't think so.
<jtv> But guessing.
<wgrant>         # Create the Publishing entry, with status PENDING so that we
<wgrant>         # can republish this later into a Soyuz archive.
<wgrant> Heh.
<wgrant> As I suspected.
<wgrant> 5 years ago, maybe...
<jtv> Soâ¦ proper behaviour now is to make that PUBLISHED?
<jtv> And maybe assert that the distro is debian for good measure, just so we find this spot back if we should ever change that?
<wgrant> I guess.
<jtv> And we'll also need a transitional measure of course.
<wgrant> It looks like it will create new SPPHs for anything that doesn't already have a matching active SPPH.
<wgrant> Probably including multiple versions of the same source.
<jtv> That would be bad.
<wgrant> But that needs testing.
<wgrant> It would be probably correct, but makes use of the dominator difficult.
<jtv> Well, it would be bad for my current purpoes.
<jtv> The only scenario that springs to mind where it helps to check only for active records is when an old version of a package is made the most recent again.  But I'm not sure that should ever happen.
<wgrant> Well, to check also for inactive records violates everything.
<jtv> "Everything" is a little broadâ¦ could you narrow it down a little?  Let's take it as read that Albanian traffic law is not, as a practical matter, violated whatever gina does here.
<wgrant> If it's published in the source archive, to say it can't be published in the target archive just because it has been published at some point in the past is a fairly hideous position to take. It causes gina to lose generality, and prevents us from partitioning SPPH.
<wgrant> And causes gina to give incorrect results for Debian imports.
<jtv> So you are saying that we must support Debian re-publishing older package versions.
<wgrant> I'm saying that to not support that would be limiting gina's usefulness, placing new requirements on our data model... and it's not a large amount of effort to support that.
<jtv> So you are saying that we must support Debian re-publishing older package versions.
<wgrant> As a result of the other constraints.
<G> wgrant: btw, thanks for my new-found fame on the launchpad wiki :)
<wgrant> G: You should have been there a while ago, but the upgrade to moin 1.9 broke stuff :(
<G> wgrant: it only passed QA this afternoon, which I don't class as a while ago :)
<wgrant> It landed like 12 hours ago.
<G> guess our definitions of a while ago differ ;)
<jtv> wgrant: So apart from the known, pre-existing problem that only one release of a package is considered published per series/archive/component, it sounds like there would be just 1 problem: âif Debian withdraws publication of the most recent version of a package, the previous version is not automatically reactivated.â  How serious would that bug be?
<wgrant> jtv: It also prevents us from reasonably archiving SPPH, which is problematic because it's a huge, slow table that we want to be able to archive.
<wgrant> I continue to maintain that using the dominator here is an unnecessary complication and an incorrect solution.
<wgrant> But I must concoct dinner.
<wgrant> So I shall return later.
<jtv> How does it prevent us from reasonably archiving SPPH?
<adeuring> good morning
<G> adeuring: good $localtime
<wgrant> jtv: We would have to adjust the query to look for both, which is slow and silly.
* allenap changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: allenap | Critical bugs: 244 - 0:[#######=]:256
<nigelb> allenap: Hi, can I add https://code.launchpad.net/~nigelbabu/launchpad/specification-validation-59301/+merge/73631 to your queue? :)
<nigelb> wgrant: Oh no. He's back.
<wgrant> nigelb: I know :(
<LPCIBot> Project devel build #1,021: STILL FAILING in 3 hr 57 min: https://lpci.wedontsleep.org/job/devel/1021/
<lifeless> StevenK: your hudson cert has expired
<nigelb> ...
<nigelb> lifeless: We've been plotting ways to keep you away.
<lifeless> why?
<nigelb> Because you're "supposed" to be away :)
<nigelb> Also, how's the baby?
<lifeless> still in ICU
<lifeless> I'm about to crash and sleep and then head back in in the morning
<lifeless> so - gnight.
<rvba> nn lifeless.
<nigelb> g'nite lifeless
<allenap> nigelb: Sure, I'll look at that.
<nigelb> allenap: Thanks :)
<nigelb> danilos: thanks for the review, updating.
<StevenK> wgrant: No.
<wgrant> StevenK: Well, they're certs, and they're free.
<wgrant> Not sure if IE likes them, but meh.
<nigelb> danilos: Hi, could you help me quickly with find_tags_by_class?
<nigelb> I'm just wondering how I would grab the right one :)
<G> danilos: can I put https://code.launchpad.net/~dev-nigelj/launchpad/bug61428/+merge/73632 in your queue (from our discussion last night)
<nigelb> *faceplam*, the nigel in that gets me pinged.
<nigelb> :D
<G> nigelb: don't worry whenever I see your name in chat, I think it might be poorly addressed to me :)
<G> nigelb: I think I'll just /nick Nigel ;)
<nigelb> G: haha, how confusing :D
<nigelb> if you /nick nigel, everyone who has to talk to both us are going to be surprised :D
<nigelb> hmmm, you'll get pinged for my test failures.
<G> oh... Happy Mailman Day :)
<nigelb> heh, I tweeted that when I woke up.
<danilos> nigelb, I can try, but generally, it should be simple to iterate over it and list all the returned items (the fact that you'll get only one should not hold you back from iterating over it)
<nigelb> danilos: nevermind, trial and error saved the day.
<danilos> G: in general, it is good to get more eyes to look at something, so I suggest you go over it with the current OCR, allenap :)
<G> danilos: okay
<nigelb> I printed the entire list and grabbed the index of the one I want. I hope that isn't evil.
<G> allenap: could you please, when you have time, take a look at https://code.launchpad.net/~dev-nigelj/launchpad/bug61428/+merge/73632 for me?
<danilos> nigelb, cool
<G> nigelb: sorry for the ping ;)
<nigelb> heh
<nigelb> heh
<nigelb> I wish bzr didn't throw a huge traceback when I Ctrl+C'd it.
<G> nigelb: if you use irssi, you could maybe modify https://github.com/nigeljonez/misc-scripts/tree/master/irssi to prevent the highlights on 'nigelj' just a thought
<nigelb> G: I could, but I don't want to :)
<G> okay, so I was looking at https://bugs.launchpad.net/launchpad/+bug/810551 how does at the bottom: "Announced: <date>" sound?
<_mup_> Bug #810551: +announcement/nnn lacks date information <anouncements> <trivial> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/810551 >
<G> (like the Updated on: <date> that shows on some announcements)
<G> that or: "Written for <project/distro/etc> by <person/team> on <date>. everywhere, and none of this: '<date>: Announcement title' thing on the +announcements list page
<G> So like: http://dev.nigelj.com/announcementchange.png instead of say https://launchpad.net/mailman/+announcements
<nigelb> Gah @ doctests.
<jelmer> Gah indeed.
<nigelb> woah, it passsed. *tries again to confirm*
<nigelb> \o/
<stub> allenap: https://code.launchpad.net/~stub/launchpad/pgbouncer-fixture-noca/+merge/73525 if you are free
<allenap> stub: Sure.
<rvba> allenap: wgrant I'm working on 837975 and perhaps one of you guys can give me a hand with something:
<wgrant> Bug #837975
<_mup_> Bug #837975: Distro +add and +edit page don't have settings for virtuality and restricted architectures <derivation> <Launchpad itself:Triaged> < https://launchpad.net/bugs/837975 >
<wgrant> :(
<allenap> rvba: I'm about to stop for lunch, but I'd be happy to help after that.
<rvba> If I change the field "Enabled restricted families" (I unselect the only available option "ARM Processors" I get "CannotRestrictArchitectures: Main archives can not be restricted to certain architectures"
<wgrant> rvba: Right, PRIMARY archives don't have architecture restrictions.
<wgrant> rvba: I have argued that this is a mistake.
<wgrant> But Julian has argued otherwise.
<wgrant> It seems he may have relented.
<rvba> wgrant: He told me you guys did not agree about something related to this bug ...
<wgrant> rvba: That was a different thing.
<wgrant> I argued there that we can just link to main_archive:+admin
<rvba> wgrant: ok, so what do you think about my problem? I don't see the point of adding this field if you cannot do anything with it.
<rvba> Am I missing something?
<wgrant> rvba: I suspect that Julian has forgotten it doesn't do anything.
<wgrant> It probably should do something, but that's more work.
<rvba> wgrant: I guess I'll put that on hold until he returns then.
<rvba> Unless you have a better idea?
<wgrant> rvba: Not sure. Fixing this is probably mostly a matter of deleting code and adding a few rows to the production DB.
<wgrant> It's unfortunate that he's off for a while :(
<rvba> Ok, I'll work on others bugs for now ...Â thanks for your help wgrant.
<nigelb> danilos: do you want to finish the review of https://code.launchpad.net/~nigelbabu/launchpad/patch-edit-684548/+merge/73514 or do you want me to ask OCR?
<danilos> nigelb, looks good, thanks for all the improvements
<danilos> nigelb, can you please set the commit message and I'll get it landed for you
<nigelb> danilos: done! Thanks :)
<StevenK> allenap: O hai -- how do you feel about reviewing a 1,300 line branch?
<allenap> StevenK: Of deletions, fine. Of Soyuz or Translations, not so fine.
<nigelb> haha
<allenap> G: Was the "okay, so I was looking at https://bugs.launchpad.net/launchpad/+bug/810551 ..." bit meant for me?
<_mup_> Bug #810551: +announcement/nnn lacks date information <anouncements> <trivial> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/810551 >
<G> allenap: meant in general
<StevenK> allenap: Based on the URL, you can guess. :-P https://code.launchpad.net/~stevenk/launchpad/no-more-staticdiff/+merge/72830
<G> allenap: basically to anyone that had any thoughts on it
<allenap> StevenK: I am happy to review that :)
<allenap> StevenK: I assume a follow-on branch will DROP TABLE staticdiff?
<StevenK> allenap: wgrant and I were not sure if slony would die horribly over that, but yes, that or something like it is the plan.
<wgrant> StevenK: We can't drop tables at the moment. We drop FKs, move the table to the todrop schema, and then stub probably does something to it.
<StevenK> So "something like it" it is. Fine.
<stub> Nah, upgrade.py does stuff to it
<wgrant> stub: Oh, I'd never seen that bit,.
<stub> Its all automated. Just change their schema rather than DROP TABLE.
<wgrant> But so it does.
<allenap> StevenK: The chunk at line 730, does that make sense any more? Ah, I assume requestUpgrade() creates a job?
<StevenK> So we land a patch to switch the tables namespace to 'todrop'?
<StevenK> allenap: Yes, requestUpgrade creates a job.
<StevenK> allenap: I've gone through and I'm reasonably sure that all of the tests pass and pyflakes/lint is happy
<nigelb> allenap: ouch, ran into some disturbing trouble.
<nigelb> allenap: its not actually escaping <script> blocks
<allenap> nigelb: It will if you use structured() in the way I suggest, because I tried it here :)
<StevenK> allenap: The diff at line 89{4,5} makes me sigh, once you hit it.
<wgrant> nigelb, allenap: What are we up to?
<nigelb> wgrant: https://code.launchpad.net/~nigelbabu/launchpad/specification-validation-59301/+merge/73631
<nigelb> allenap: oh? I did make that change first. Let me re-check
<allenap> StevenK: Is there no worth in modifying TestRevisionMailJob instead of ditching it? Erm, I guess I'm asking if removing it reducing our test coverage?
<StevenK> RMJ still does stuff, sadly
<wgrant> nigelb: Oh, you're not actually using a script tag, just using it to test injection. I am no longer terrified.
<wgrant> allenap: Thanks for picking that up.
<allenap> Hehe :)
<nigelb> wgrant: haha.
* jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: allenap, jcsackett | Critical bugs: 244 - 0:[#######=]:256
<nigelb> GAH.
<nigelb> Forgot to save the file.
<G> allenap: btw, if you do have views on my question, I'd be happy to hear them
<allenap> G: Yeah, I'm finishing something off then I'm going to take a look :)
<G> allenap: okay, thanks :)
<StevenK> He's finishing of a review of mine that is making his eyes bleed
<G> StevenK: does the code make his machine poke sticks of RAM in his eyes?
<nigelb> I the "making eyes bleed" is understood with StevenK's branches.
<nigelb> *I think
<allenap> G: That looks good. I am not a Launchpad UI god, but a change like that is undoubtedly an improvement regardless of whether or not it completely satisfies the UI gods.
<G> allenap: so basically that screenshot is okay?
<allenap> G: Yeah, I like it.
<G> (I was thinking of removing the trailing . though, because it looks odd)
<allenap> G: Ah yes. Bug pages don't have the trailing full-stop so that's a good call.
<StevenK> allenap: Thank you for the review.
<G> now to work out what tests I have to run
<G> don't exactly feel like waiting until 5am or later for tests to finish :)
<StevenK> G: Well, what did you change?
<allenap> StevenK: Welcome :)
<G> StevenK: just lib/lp/registry/templates/announcement-macros.pt
<nigelb> allenap: I fixed everything! Could you re-review https://code.launchpad.net/~nigelbabu/launchpad/specification-validation-59301/+merge/73631 ?
<allenap> nigelb: Sure :)
<StevenK> G: Right, so that seems a little far reaching. bin/test -vvt registry might be enough of a subset. Or you could try bin/test -vvt announcement and see what you hit
<G> StevenK: thanks
<StevenK> G: Or you can grep around and see which views use those macros and then see which tests use those views
<G> StevenK: is -t <foo> basically  tests matching views with <foo> in their path?
<StevenK> G: Right
<G> well lib/lp/registry/browser/tests/announcement-views.txt in particular passed fine
<benji> G: once you start running the same set of tests over and over again (doing TDD for example), you can use -m to tell the test runner to only look at a particular module; that will take 5-10 seconds off of the testrunner startup time
<G> benji: ahhh okay
<benji> (and -t and -m work well together too)
<G> so I could do something like ./bin/test -m registry -t announcement to only do announcement tests related to registry?
<StevenK> G: You can, yes
<nigelb> Of course you'll break some doctest you have clu about sometimes.
<G> thanks for that, learn something everyday :)
<deryck> Hello, all.
<deryck> adeuring, abentley, henninge -- sorry about missing the standup.
<deryck> I have a patched tire now. :)
<abentley> deryck: sorry to hear about the tire.
<deryck> abentley, thanks.  not too big a deal.  I could pull of to a station easily.  Just slow waiting.
<allenap> Oops, nigelb, I got distracted. r=me. Want me to land it for you?
<nigelb> allenap: Yes, please :)
<nigelb> ... \o/
<nigelb> I just moved up to 4th spot :)
<benji> G: to be exact, you could do "bin/test -m lp.registry -t announcement" (-m takes a module path)
<nigelb> Enough Launchpad for 2 full working days :)
<nigelb> Onto other projects
<G> benji: ahhh right
<Ursinha> nigelb: thanks :)
<nigelb> Ursinha: \o/ :)
<Ursinha> nigelb: :D
<G> woohoo, all registry & announcement tests passed
<G> allenap: for this template branch, would you be able to review it?
<allenap> G: Yeah, sure.
<G> allenap: any objection if I don't put a lot of content in the cover letter thing seems a bit hard to fit stuff in each section without repeating too much
<allenap> G: As long as I understand it, it's fine. Don't treat it like a tax form :)
<nigelb> That was...WOW. The best I've heard.
<G> allenap: oh gosh, tax forms... I had to fill out an Australian one once, and nearly blew my top
<mrevell>  /me wonders if we should rename the channel #nigel-dev
<nigelb> haha
<G> mrevell: haha
<mrevell> :)
<nigelb> I think G and I have led talking in this channel for the past few days
<G> that reminds me, I should invite nigelb into the super secret social club ;)
<G> (joking of course)
<nigelb> And the fun bit? If G /nicks to nigel, this will be a confusing channel all around.
<rvba> wgrant: About #827608, would you by any change have an idea where I could store announce_from_person=announce_from_person (passed to do_copy) to be able to use it in person._latestSeriesQuery (to populate ~xx/+related-software).
<_mup_> Bug #827608: Sync requester isn't credited with upload <derivation> <Launchpad itself:Triaged> < https://launchpad.net/bugs/827608 >
<G> allenap: MP: https://code.launchpad.net/~dev-nigelj/launchpad/bug-810551/+merge/73673
<rvba> wgrant: I know it's late for you, feel free to ignore my question ;)
<G> allenap: there is also https://code.launchpad.net/~dev-nigelj/launchpad/bug61428/+merge/73632 if you've got time
<allenap> G: Yeah, I've got time for both.
<G> except, if youcan wait a second, I'd like to make one more change for the 61428 one
<nigelb> mrevell: HAHAHA. Excellent tweet.
<G> nigelb: got a link?
<nigelb> https://twitter.com/#!/launchpad_net/status/109270580846534656
<G> mrevell: haha thanks
 * G follows @launchpad_net now :)
<mrevell> I kinda feel we're going for a buy two, get one free deal.
<mrevell> :)
<nigelb> haha
<G> nigelb: we could start a TV show... "The Two Nigels" :)
<nigelb> We could start a podcast :P
<jelmer> we used to have a Launchpad podcast, back in the day...
<nigelb> jelmer: revive it!
<jelmer> mrevell: this was one of the items I missed in your "my plans for Launchpad" email.
<wgrant> rvba: There is nowhere.
<rvba> wgrant: ok then maybe I need another plan.
<wgrant> rvba: There's no way to track copies.
<wgrant> rvba: I raised this a year ago, but it's been conveniently ignored for 12 months :)
<mrevell> jelmer, I'm recruiting a new "communications" person. If they have time for a such thing, it might be fun to do it again. I'd want to think about what sort of benefit it was bringing Launchpad, though. Ad-hoc episodes now and then seems fine to me, though.
<rvba> wgrant: and now it's on me ;)
<jelmer> mrevell: I was mostly kidding, though I did enjoy the podcast.
<wgrant> rvba: Fun. I think we need to discuss with bigjools next week. It requires a significant amount of thought.
<wgrant> Launchpod was good.
<mrevell> heh, I'm glad you liked it. Maybe we should do it again.
<nigelb> One every meetup would be nice
<nigelb> So, one from UDS, one from Thunderdome, etc
<G> allenap: I just pushed the change to the bug61428 branch, just adds an extra test
<rvba> wgrant: k
 * rvba marks another card 'blocked'. Sigh.
<mrevell> nigelb, That's definitely do-able.
<nigelb> mrevell: \o/
<jelmer> mrevell: that reminds, I wonder if it would be possible to do a blog post about the improvements for the code imports I've done recently
<jelmer> mrevell: if that's ok, should I just send you an email with a suggested text?
<mrevell> jelmer, Yes please!
<nigelb> I need to blog to the planet on some of the stuff I've done.
<nigelb> Some are nice!
<nigelb> Let me wait for the last two to land.
<mrevell> nigelb, If you have something for the Launchpad blog, let me know.
<nigelb> hrm I could just give you stuff that should go on the Launchpad blog instead, yeah.
<mrevell> Whichever you prefer.
<G> haha just red the person picker post... people would have fun finding me by IRC nick there ;)
<G> *read
<G> I'd imagine everyone with a 'g' in their name, e-mail address etc would appear
<allenap> G: I think three characters must be provided before it'll even attempt a search.
<G> allenap: ahh yeah, I forgot about that rule
<henninge> what's the reason for this failure again?
<henninge> ComponentLookupError: (<InterfaceClass canonical.launchpad.webapp.interfaces.IPlacelessAuthUtility>, '')
<henninge> in a test
<allenap> henninge: Try a different layer perhaps? The utility is not registered.
<henninge> allenap: yes, I am trying that currently. thanks
<G> allenap: how did you see leading dates in the titles on +announcements?
<G> allenap: the diff removed them
<G> (lines 8/9/10)
<stub> gary_poster: Do you happen to know where our current Storm is from? Trunk or a branch?
<henninge> much better
<stub> gary_poster: nm. annotate tells me it was StevenK. I'll sort it tomorrow.
<gary_poster> stub, lp:~launchpad-committers/storm/with-without-datetime
<gary_poster> ok stub
<timrc> I'm creating PPA's on staging but they do not seem to be getting whacked every 24 hours, is there a way to do this? deleting ppas is only a partial deletion (it doesn't seem possible to delete a PPA and then create one with the same name)
<allenap> G: Yes, of course, oops. Ignore me.
<G> allenap: I like your tests, I should've thought of it, I'll add them, plus an extra test :)
<allenap> G: Cool.
<G> allenap: actually, the idea for the third test was a bit flawed
<gary_poster> G, I am envious of your nick ;-)
<G> gary_poster: haha, had it since late `05
<cr3> if I have a question about login.launchpad.net, should I ask here or to the isd folks?
<gary_poster> :-) cool
<gary_poster> cr3, isd
<cr3> gary_poster: cheers!
<gary_poster> welcome :-)
<G> oh whoops didn't mean to do that
<G> I thought for some reason, selecting resubmit in the dropdown, would somehow mark the review as a resubmit request
<G> allenap: I updated the merge proposal, but may have stuffed it up at the same time
<allenap> G: Okay :) I'll take a look.
<allenap> G: Btw, the Resubmit status is meant for use by the reviewer. It's confusing, and others have had made this mistake too (including me). I assume this is what you meant by stuffing it up?
<henninge> adeuring: The script does not work.
<adeuring> henninge: what's the problem?
<henninge> "find" is not a method of ResultSet, it is a method of store.
<henninge> Store, even
<allenap> G: Want me to land it for you?
<G> allenap: yeah, thats what I meant
<G> allenap: yes please
<cjwatson> allenap: is it meant to be "Needs resubmission" or something?
<adeuring> henninge: ResultSet has such a method too. But it accepts only WHERE clauses
<G> cjwatson: I think that'd be a better status
<cjwatson> ah yes, https://help.launchpad.net/Code/Review
<allenap> cjwatson: Yeah, that would be a better name.
<henninge> adeuring: So I'll either need to extend getByStatus or put the Storm query in the script code.
<cjwatson> "Needs reworking" based on that help page
<henninge> adeuring: ForbiddenAttribute: ('find', <storm.store.ResultSet object at 0x104d09ec>)
<henninge> adeuring: I'll have a look at the api doc to see what you might be thinking of.
<adeuring> henninge: right... remove_security_proxy might work. but that's odd place to use it...
<henninge> adeuring: no, "ForbiddenAttribute" is not about security.
<henninge> adeuring: you'd see "AuthorizationError" in that case
<G> I will take it as a sign that I need sleep, so have a good day all :)
<henninge> ForbiddenAttribute is misleading ...
<adeuring> henninge: rihgt. but let me check the class resultset
<henninge> adeuring: I am doing that now
<henninge> adeuring: hm, you are right, there is one.
<henninge> in the api doc
<henninge> adeuring: so, removeSecurityProxy helps
<henninge> odd
<adeuring> indeed
<henninge> maybe I am wrong about ForbidenAttribute
<henninge> adeuring: I won't be able to add the support for distroseries search.
<adeuring> henninge: sure, no problem. Just try to finish what you have :)
<henninge> adeuring: that is something that should be included in HWSubmissionSet
<adeuring> right
<henninge> adeuring: So, it's almost done now but using removeSecurityProxy.
<adeuring> henninge: so, let's wait for a review :)
<adeuring> anybody around with some ELementTree-fu? this method http://paste.ubuntu.com/679902/ is supposed to move two node from one parent to another. Or to add missing nodes to the new parent. (The exact content of the old parent does not matter). Running a RelaxNG validation later fails, but if I move the data around in the XML text, things work...
<adeuring> flacoste: ^^^?
* allenap changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jcsackett | Critical bugs: 244 - 0:[#######=]:256
<henninge> adeuring: also, the find does not even work. :(
<adeuring> henninge: ouch.. which expression do you use?
<henninge> adeuring: what do you mean?
<adeuring> which parameter do you pass to find()? or... remember to use new_resultset = resultset.find(whatever)
<henninge> adeuring: I did not change that code
<adeuring> henninge: cyn you paste it?
<henninge> adeuring: let me try something first
<henninge> adeuring: there was no assignment
<henninge> see, now it passes ;-)
<adeuring> henninge: ahhh, sorry ofr this
<henninge> yeah, I missed that, too
<Ursinha> gary_poster: hi :) do you know if there's anything going on with launchpad? I can't view bugs, they're timing out
<gary_poster> Ursinha, hi!  happy birthday, yeah?
<Ursinha> thanks :)
<mrevell> Night all
<gary_poster> Ursinha, I'm about to go to a dr. appt.  deryck, benji, bac, are any of you all available to investigate the graphs and such?
<Ursinha> thanks gary_poster :)
<gary_poster> welcome :-)
<benji> gary_poster: sure (once I figure out what you're talking about) ;)
<henninge> adeuring, deryck: I have to run now, I am sorry. Got an appointment in an hour.
<adeuring> henninge: np, have a nice evening!
<henninge> I prepared the MP and made the branch owned to the squad.
<Ursinha> benji: I'm facing timeouts since this morning
<henninge> https://code.launchpad.net/~launchpad-orange-squad/launchpad/abel-broken-hwdb-reports/+merge/73697
<Ursinha> now I just can
<Ursinha> can't view bugs anymore
<Ursinha> they're timing out
<henninge> adeuring: I prepared the MP and made the branch owned to the squad.
<adeuring> henninge: ok, thanks
<Ursinha> benji: I'm trying to view this bug: https://bugs.launchpad.net/ubuntu/+source/gwibber/+bug/619654
<_mup_> Bug #619654: gwibber-service crashed with GError in __getitem__() <amd64> <apport-crash> <maverick> <natty> <ubuntu-une> <Gwibber:New> <gwibber (Ubuntu):Confirmed> < https://launchpad.net/bugs/619654 >
<gary_poster> benji, heh.  take a look at https://wiki.canonical.com/IncidentReports/2011-08-19-LP-confused-query-plan-degraded-performance .  It has links to a lot of tools I used to look at performance for a recent incident.
<benji> k
<gary_poster> benji in particular https://lpstats.canonical.com/graphs/OopsLpnetHourly/ ...
<benji> Ursinha: if you log out do you see the same behavior?
<Ursinha> haven't tried, a momen t
<gary_poster> benji, https://lp-oops.canonical.com/reports/production/2011/08/19/ (adjust for today)
<benji> gary_poster: thanks
<Ursinha> benji: cool, logged out that works fast
<gary_poster> benji, https://lpstats.canonical.com/graphs/OopsLpnetHourly/ looks fine
<gary_poster> benji, so does https://lpstats.canonical.com/graphs/DBCpuLoadAppServers/
<Ursinha> and logging in back times out again
<benji> Ursinha: I wondered if that would be the case, I suspect there is /something/ different about you that it is spending too much time on.
<deryck> henninge, ok, adeuring and I will make sure we get it landed.
<gary_poster> Ursinha, could you give us an OOPS please?  In a perfect world, it would be old enough to actually be synced to devpad, like at least 20 min or 30 min
<henninge> deryck: cool, thanks
<henninge> deryck, adeuring: Good luck and see you on Monday.
<Ursinha> sure
<adeuring> henninge: thanks and a nice weekend
<Ursinha> benji: I'm subscribed to tons of teams that are subteams of other teams and yada yada
 * henninge hugs Ursinha
<Ursinha> gary_poster: benji: OOPS-2070AW69
<Ursinha> henninge!
<henninge> Ursinha!
 * Ursinha hugs henninge back
<gary_poster> ack Ursinha, thanks, looking
<henninge> Happy birthday! (if FB is not lying)
<henninge> Ursinha: ^
<henninge> :-D
<benji> Ursinha: yeah, I guess that a recent deployment introduced a pessimization that hit you hard; we'll see if we can find it
<Ursinha> it's not :) thanks!
<Ursinha> thanks benji
<Ursinha> should I file a bug about it?
 * henninge has to run
<benji> yeah, a bug sounds like a good idea
<Ursinha> cool, I'll file it now
<benji> Ursinha: reference that oops too (although I can't see it yet, hopefully it'll show up shortly)
<gary_poster> benji, as I imagine you have discovered, OOPS is not there yet.  https://wiki.canonical.com/Launchpad/FreshLogs tells you how to request logs on devpad, and then you can look at them directly
<benji> gary_poster: thanks
<gary_poster> I'm requesting the oops summary page now...there it is
<Ursinha> benji: do you want the traceback?
<Ursinha> I can see it in the oops page
<benji> Ursinha: that'd be great
<gary_poster> Possible OOPS to investigate
<gary_poster> https://lp-oops.canonical.com/oops?oopsid=OOPS-2070DT30
<gary_poster> https://lp-oops.canonical.com/oops?oopsid=OOPS-2070DQ6
<Ursinha> benji: http://paste.ubuntu.com/679935/
<Ursinha> is the problem happening when trying to load the subscription portlet or I'm seeing it wrong?
<gary_poster> benji, Ursinha, this is https://bugs.launchpad.net/launchpad/+bug/811447
<_mup_> Bug #811447: BugTask:+index timeout - death by sql in PersonSubscriptions(user, bug) <timeout> <Launchpad itself:Triaged by gary> < https://launchpad.net/bugs/811447 >
<gary_poster> I have a branch to fix it that I was working on just then :-P
<Ursinha> gary_poster: cool! :)
<benji> Ursinha: yeah, it's building the subscription portlet when it runs out of time; of course something else might have used up all the time before that, but it's a smoking gun
<benji> gary_poster: very nice
<benji> gary_poster: cool, I'll put it in my notes
<Ursinha> thanks gary_poster and benji
<gary_poster> Welcome Ursinha.  It won't be rolled out today as a birthday present, but hopefully early next week :-)  fwiw, the trigger should generally be bugs with many duplicates, to which one or more of your teams are subscribed.
<Ursinha> I see.
<Ursinha> thanks anyway! :)
<gary_poster> jcsackett, I have three branches ready for review if you are willing to do it while I am absent for doctor's visit.  If not, no prob.
<gary_poster> https://code.launchpad.net/~gary/launchpad/bug838869/+merge/73695
<gary_poster> https://code.launchpad.net/~gary/launchpad/bug838878/+merge/73702
<gary_poster> https://code.launchpad.net/~gary/launchpad/bug811447/+merge/73704
<gary_poster> (The second two depend on the first.)
<gary_poster> # 2 is 913 lines, but #3 is 181.
<jcsackett> gary_poster, i saw. i'm finishing lunch now, then i'll start in on them. :-)
<gary_poster> thanks jcsackett :-)
<flacoste> adeuring: did you solve the issue?
<adeuring> flacoste: no :( It's really odd. Let me paste some related stuff...
<flacoste> adeuring: it's like if Relax-NG wasn't taken the new nodes into account?
<adeuring> flacoste: exactly
<adeuring> the odd thing is: If I write the modified tree to a file and read the file back into another tree, things are fine
<adeuring> flacoste: http://paste.ubuntu.com/679965/
<adeuring> ...like so
<adeuring> flacoste: if I remove line 23, things break...
<adeuring> flacoste: I am currently simply using regexes for the same job. ugly, but more promising :)
<flacoste> adeuring: i don't understand line 23 is identical to line 9
<flacoste> and it's not like if submission changed between the two?
<adeuring> flacoste: that should have been etrre.parse(f,...)
<adeuring> flacoste: let me check again...
<flacoste> adeuring: the error is when validating which works with a string anyway, no?
<adeuring> flacoste: arghhh sure!
 * adeuring is an idiot
<adeuring> so, regexes FTW
<adeuring> thanks!
<adeuring> so that's why writing to a file and reading (properly...) back worked
<adeuring> flacoste, deryck: but: seems that the idea to move two nodes around works. Have tested so far just one HWDB report, so I am not _that_ sure, but anyway
<deryck> cool
 * deryck has fingers crossed
<flacoste> adeuring: btw, getchildren is deprecated in 2.7, using list(node) would work in both 2.6/2.7
<adeuring> flacoste: ah, thanks.
<flacoste> adeuring: and you can node.findall('name') instead of the list comprehension you are currently doing
<flacoste> there is also a findtext() not sure if that would be useful to you currently
<adeuring> flacoste: ok, but I opted for a regex variant :)
<adeuring> since we need the string representation anyway
<flacoste> findall() takes a node name or a xpath subset: http://effbot.org/zone/element-xpath.htm
<adeuring> flacoste, deryck: with the data modification intended by the broken method, 848 from 858 reports that we unusable before can now be processed :)
<flacoste> adeuring: awesome!
<deryck> adeuring, very nice!
<adeuring> flacoste: what do you think: should I keep a regex based fix. or shouldd we goo back to tweaking the element trre, write it back to a string, and validate this string?
<flacoste> adeuring: what does the regexp-based fix look like?
<flacoste> it's doing substitution on the original document string?
<adeuring> flacoste: http://paste.ubuntu.com/679992/ (7 is unrelated to the current discussion -- that's about ESC characters in lots of other reports)
<adeuring> s7/line 7/
<adeuring> flacoste: http://paste.ubuntu.com/679995/ (with regexes inlcuded)
<flacoste> adeuring: it's not as gross as i thought it would be :-) probably faster also
<adeuring> right, I suspect too taht it's faster
<flacoste> since this is big kludge anyway... i'd be tempted to let it go
<flacoste> i'd suggest making the _udev_node_exists and _dmi_node_exists non-greedy
<flacoste> .*? instead of .*
<adeuring> flacoste: of course, sure
<adeuring> flacoste: they also need a DOTALL flag
<flacoste> it will fail faster
<flacoste> right, because of the newlines
<adeuring> exactly
<adeuring> I didn't test this branch against "working" reports yet...
<adeuring> wanted to see first if the approach is usabel at all
<adeuring> jcsackett: could you please review this mp: https://code.launchpad.net/~adeuring/launchpad/hwdb-test-natty-mess/+merge/73722 ?
<jcsackett> adeuring: i'll do it as soon as i finish the one i'm looking at now. :-)
<adeuring> jcsackett: ok, thanks!
<flacoste> jcsackett, adeuring: already done :-)
<jcsackett> flacoste: cool. :-)
<adeuring> flacoste: thanks!
<adeuring> flacoste: right, the test is really not very comprehensive -- but the later running parts of the processing script are quite paranoid, so I think we can be sure that we will not store nonsensical stuff
<flacoste> adeuring: right, you might add a comment to the test saying that the call would raise an exception without the work-around
<flacoste> or something like that
<flacoste> so that it's obvious that the assert is really not the meaningful test
<adeuring> flacoste: right
<flacoste> it might even be superfluous
<adeuring> night everybody
<thumper> morning
* flacoste changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jcsackett | Critical bugs: 237 - 0:[#######=]:256
* jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 237 - 0:[#######=]:256
<jelmer> hey thumper, mwhudson
<jelmer> hmm, the scanner doesn't seem to like ~vcs-imports/libreoffice/core
<mwhudson_> jelmer: it oopses with 'job ran too long'
<jelmer> oh, oops..
<mwhudson_> jelmer: in getAncestryData i think, before even it starts inserting stuff into the db
<jelmer> maybe 2.4 will help with that
<jelmer> jams recent fixes for get_parent_map will definitely  help
* wallyworld changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: wallyworld* (jtv) | Critical bugs: 237 - 0:[#######=]:256
<wallyworld> thumper: you must be missing launchpad - you've been poking around in here a bit of late :-P
<thumper> wallyworld: I've found some things, particularly in email, that I want changed
<wallyworld> i can imagine :-)
<wallyworld> have you raised bugs?
<wallyworld> thumper: what do you want fixed?
<thumper> bug email headers
<thumper> imapfilter only checks the first occurrance of a particular header it finds, like X-Launchpad-Bug
<wgrant> Really? That sounds like a pretty glaring flaw.
<wallyworld> thumper: and we stuff multiple X-Launchpad-Bug header items into the message?
<thumper> yep, one per task
<thumper> wgrant: it is defined in the imap rfc
<thumper> for remotely selecting messages
<wgrant> Still sounds like a pretty glaring flaw.
<wallyworld> thumper: so with the squad reorg, unless a bug is critical or part of feature work, it's unlikely to get done sinze we currently have so many criticals to burn down
<wallyworld> unlikely to get done soon
<wgrant> And even if it is escalated to critical, it's unlikely to be done within 6 months.
<wallyworld> thumper: so in other words, patches welcome :-)
<wallyworld> but changing mail headers would have a lot of downstream impact on people's existing mail filters etc, so I guess existing headers would need to be retained?
<wallyworld> and new ones added?
<wgrant> So we have to have three filtering methods.
<wgrant> Yay.
<wgrant> One for pathetic Google Mail, one for pathetic built-in IMAP filtering, and one for everything else.
<wgrant> Maybe we should have an "I use a retarded mail server" option...
<wallyworld> wgrant: why is gmail retarded?
<wgrant> It can't filter on custom headers.
<wgrant> At all.
<wallyworld> wtf? serious?
<wgrant> Yes.
<wallyworld> why????
<wgrant> That's why there's all this crap in the body of all emails.
<wgrant> To appease Gmail users.
<wallyworld> why would not being able to filter on custom headers appease gmail users?
<wallyworld> surely it would do the opposite?
<wgrant> Hm? We have to put all the filtering metadata in the body too, or Gmail users complain that they can't filter.
<wallyworld> ah i think you misunderstood my question - why doesn't gmail support filtering on custom headers i wonder?
<wgrant> NFI
<wallyworld> seems like a pretty basic omission
#launchpad-dev 2011-09-02
<wallyworld> wgrant: is one supposed to review a mp with a conflict or wait/ask for the conflict to be resolved?
<wgrant> wallyworld: That's one of the great unanswered questions of the modern age.
<wallyworld> hah
<wgrant> Are there more conflicts than just sourcecode.cache
<wgrant> ?
<wallyworld> the other issue is the mp was resubmitted (after the susperceded one was approved) so perhaps the original approver should be the one to look at the new one
<wgrant> That seems to be it.
<wallyworld> no, no other conflicts
<wallyworld> i could easily remove them when running locally
<wallyworld> just wondering as a matter of policy
<wgrant> It was last reviewed two months ago; I'm not sure that bac would have significantly more context than you at this point.
<wallyworld> sure. i'll jump in then
<mwhudson> wallyworld: what happens now with https://code.launchpad.net/~mwhudson/launchpad/ChoiceSource-flexibility/+merge/73611, does it need to be mentored?
<wallyworld> mwhudson: i'm thinking for such a small change it's ok
<mwhudson> wallyworld: can you land it then please?  I don't think i have all the relevant bits set up
<mwhudson> i guess it needs a commit message
<wallyworld> mwhudson: sure, i was just about to ask you if you wanted me tyo do that
<mwhudson> and uh, a bug report?
<wallyworld> mwhudson: yes, a bug report if you want to be able to qa it before it gets released
<mwhudson> i think that would be a good idea :)
<wallyworld> just to be sure nothing breaks
<mwhudson> there are semi related problems to do with the zIndex of the blocking div that prettyoverlay creates
<mwhudson> but they can wait for another day :)
<poolie> i'm getting persistent oopses on https://bugs.launchpad.net/ubuntu/+source/libreoffice/+bug/835153
<_mup_> Bug #835153: oosplash.bin crashed with SIGSEGV in XSetForeground() <amd64> <apport-crash> <oneiric> <running-unity> <libreoffice (Ubuntu):Triaged by bjoern-michaelsen> < https://launchpad.net/bugs/835153 >
<mwhudson> wallyworld: ok, should be good to land now
<wallyworld> mwhudson: cool. onto it
<mwhudson> thanks
<G> allenap: thanks for those two suggestions for the 61428 merge
<wallyworld> wgrant: looks like a buildbot test has hung. i assume we just force a new build in this case? i think this happens occasionally?
<wgrant> wallyworld: It's a regression in the last couple of weeks, but yes.
<G> allenap: I just made the change that you suggested on my MP (https://code.launchpad.net/~dev-nigelj/launchpad/bug61428/+merge/73632) so it should be good to land now
<LPCIBot> Project devel build #1,022: STILL FAILING in 4 hr 55 min: https://lpci.wedontsleep.org/job/devel/1022/
<jtv> hi wallyworld!
<nigelb> Morning!
<nigelb> oh oh.
<nigelb> 503 error from loggerhead.
<jtv> hi nigelbâ¦ sorry, never got around to looking at your MP yesterday.
<nigelb> jtv: hehe, np, I asked the OCR. I'm fixing test failings in it now :)
<jtv> OK great
<nigelb> What is the solution if the result of the doctest is > 78 characters?
<nigelb> wgrant: about?
<wgrant> nigelb: Unless you're using -NORMALIZE_WHITESPACE, you can just add newlines wherever you want.
<nigelb> Does this look fine?
<nigelb>     >>> browser.getControl('Title').value = 'Extension Manager '
<nigelb>     ...     'System Upgrades'
<nigelb> because lint complains for that.
<wallyworld> jtv: hi. sorry missed your ping. my stupid irc client doesn't pop up notifications for very long and then I miss them if i am afk eating or something :-(
<wgrant> nigelb: Oh, that's not the result... I'd do this:
<wgrant>     >>> browser.getControl('Title').value = (
<wgrant>     ...     'Extension Manager System Upgrades')
<nigelb> Ah.
<jtv> wallyworld: I was just saying good morning.  As you know I don't think you need any further mentoring, and since I can't find any further procedural notes, I hereby formalize your graduation.
<jtv> Note to mailing list to follow.
<nigelb> congrats wallyworld!
<wallyworld> jtv: cool :-) i did a largish review this morning but it needs fixing so i didn't ping you yet. but i guess now i don't need to
<wallyworld> nigelb: thanks
<jtv> That's right.
 * wallyworld goes back to fixing some failing tests !@%@!%@$!
<nigelb> haha
<nigelb> If my code passes tests, does it automatically get merged?
<nigelb> passes tests from ec2land.
<wallyworld> nigelb: yes, unless pqm is in testfix mode
<wallyworld> due to a buildbot breakage
<nigelb> hrm, I don't think my branch last night merged, despite tests passing.
<wgrant> buildbot was broken for a while.
<wallyworld> nigelb: check your email - there should be one saying it merged or else something like 0 tests blah blah
<mwhudson> nigelb: whoever ran ec2 land should have an email in that case
<mwhudson> heh, simultaneous contradiction!
<wgrant> The signer gets the PQM email.
<wallyworld> oops. i forgot that someone else would have landed it
<wgrant> Both parties get the EC2 email.
<nigelb> ok, I'll poke danilos later when he gets on.
<nigelb> mwhudson: Did you find configobj helpful? :)
<mwhudson> nigelb: i haven't gotten around to looking at it yet
<mwhudson> nigelb: i'd blanked on the fact that bzr used an external library for this sort of thing though
<mwhudson> well sort of external
<nigelb> :)
<nigelb> There are lots of things in ConfigParser which really irritate me
<nigelb> argh, doctests.
<nigelb> How do I reduce this one to less than 78
<nigelb>     'http://blueprints.launchpad.dev/firefox/+spec/extension-manager-upgrades/+edit'
<StevenK> http://blueprints...dev/firefox/+spec/exten.../+edit
<poolie> nigelb: do you have to silence the lint complaint?
<poolie> i don't know if doing so would be a good use of time
<poolie> hi, btw
<nigelb> poolie: Well, I can always ignore it :D
<StevenK> I didn't think we cared about 78 char limit in doctests ...
<nigelb> poolie: Morning! :)
<nigelb> StevenK: \o/
<poolie> hi there
<nigelb> Can someone kick this back into ec2? https://code.launchpad.net/~nigelbabu/launchpad/specification-validation-59301/+merge/73631
 * nigelb also notes the /topic needs updation
<nigelb> jtv: "to a vicious shark who will go as deep as it takes to find out what's wrong with your branch" -- you've trained him well :P
* wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: supreme ruler wallyworld | Critical bugs: 251!! - 0:[#######=]:256
<wgrant> Does anybody know of a Unicode explosion character?'
<wgrant> Or something similar?
<jtv> wgrant: an invalid UTF-8 sequence will usually do the trick :-P
<jtv> nigelb: thanks  âº
<wgrant> jtv: The graph in the topic will need it soon :(
<jtv> â¢ ?
<nigelb> Ha, that's why.
<wgrant> jtv: That works.
<wgrant> There are 257 tasks :(
<jtv> I take it the counter's been running the wrong way then?  Otherwise you'd be looking for â®
<wgrant> But 6 of them are private.
<wgrant> So webnumbr says we are safe and the world has not ended.
<jtv> â 
<jtv> Who is webnumbr?
<wgrant> http://webnumbr.com/launchpad-critical-bugs
<nigelb> wgrant: http://www.fileformat.info/info/unicode/char/1f4a3/index.htm
<wgrant> That is worryingly far outside the BMP. I wonder how widely supported it is.
<wgrant> But that looks helpful, thanks.
<wgrant> Hah.
<wgrant> Introduced in Unicode 6.
<wgrant> ie. 10 months ago.
<nigelb> Only 2 fonts support it.
* wallyworld changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: wallyworld | Critical bugs: 251!! - 0:[#######=]:256
<nigelb> wallyworld: Hi, I had a few test failures yesterday from this MP. Would you mind re-reviewing and kicking it back into ec2? https://code.launchpad.net/~nigelbabu/launchpad/specification-validation-59301/+merge/73631
<wallyworld> nigelb: sure
 * wallyworld looks
<nigelb> I should have commited test fixes and then fixed lint.
<nigelb> gah.
<StevenK> wallyworld: Nice work!
<wallyworld> StevenK: thanks. how's the battle scenario modelling going?
<StevenK> More painting :-P
<wallyworld> remember, i want to see a photo when it's done :-)
<StevenK> I've been linking you photos in privmsgs, you didn't comment :-P
<wallyworld> StevenK: ah bollocks. didn't notice :-(  Under Unity, quassell notifications aren't very obvious to me anymore for some reason so i miss them
<wallyworld> StevenK: so they start out blue? so you still have some work to do on those ones?
<StevenK> wallyworld: They start out grey and on a sprue
<wallyworld> nigelb: ec2 is off and running the tests
<StevenK> So I need to cut them out, assemble them and then paint them fully
<nigelb> wallyworld: thanks
<nigelb> StevenK: battle scenario modelling?
 * wallyworld types into google "define: sprue"
<wallyworld> nigelb: that was my made up term
<wallyworld> not sure what the official name for it is
<nigelb> What is he making? :)
<wallyworld> nigelb: painting little figurines of warriors and the like for display in a battle field scenario
<wallyworld> StevenK:  not sure if that's the correct description? ^^^^
<nigelb> Oooooh.
<StevenK> Close enough
<nigelb> StevenK: You made those? O_O
<nigelb> like, from scratch?
<StevenK> wallyworld, nigelb: http://wedontsleep.org/~steven/IMG_2807.JPG is what they start out like.
<nigelb> ah, not entirely from scratch.
<StevenK> That is for a different army, but you get the idea.
<nigelb> But still, excellent work.
<wallyworld> StevenK: oh, i didn;t realise you had to glue them together before painting
<nigelb> You manage to do stuff that does not involve computers during paid vacation.
<StevenK> wallyworld: That's the colour they start out as well.
 * wallyworld nods
<StevenK> That blue colour on them is 4 different blues
<nigelb> how do you attach them?
<nigelb> melt?
<nigelb> or glue?
<StevenK> nigelb: Plastic glue
 * wallyworld wonders how StevenK is typing with all his fingers stuck together
<StevenK> Plastic glue doesn't affect skin, it washes off easily
<nigelb> If his fingers do get stuck, I know which key it will be on.
<nigelb> ('Del')
 * G fears StevenK's army
<StevenK> Pft, I use vim, so it would be d
<nigelb> heh
<G> ohhh I may have spied on a another bugfix :)
<nigelb> G: You're +1200?
<G> nigelb: yep
<nigelb> Two people with first names Nigel and we're both not British? Talk about breaking stereotypes  :P
<G> in way of ancestry I'm mainly British
<nigelb> Ah.
<G> but I'm also partly Swede etc as well
<poolie> hooray for wallyworld!
<poolie> congratulations on your asterectomy
<nigelb> asterectomy sounds painful :P
<nigelb> "define asterectomy" got spell corrected to "define hysterectomy" making me LOL :)
<wallyworld> poolie: thanks
<poolie> the removal of his "i'm not a real reviewer" star
<nigelb> Ah.
<nigelb> aww wallyworld changed it, wgrant put a better /topic ;)
* Topic unset by poolie on #launchpad-dev
* poolie changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: wallyworld | Critical bugs: 251!! - 0:[#######=]:256
<LPCIBot> Project devel build #1,023: STILL FAILING in 4 hr 7 min: https://lpci.wedontsleep.org/job/devel/1023/
<jtv> wallyworld: gratifying that your graduation did not go unnoticed, no?  âº
<wallyworld> jtv: i was hoping to sneak under the radar and keep it quiet. but you put an end to that ambition :-P
<jtv> Why would you want to do that?
<wallyworld> jtv: i don't like attention :-)
<wallyworld> i'm shy
<jtv> Tip #1 for a reviewer: be feared.  Otherwise people start bothering you for reviews even when they have other options.
<wallyworld> hah. good advice :-)
<jtv> The attention comes in the one form or the other.  Trust me, you got the better deal.
<wallyworld> lol
<wallyworld> jtv: i notice you have a repuation as a feared reviewer
<wallyworld> now i know why
<jtv> "fascist" is the more common term, but alright
<wallyworld> jtv:  well, if hard questions are asked and you get good answers, then that's good
<jtv> I rejected a sentence added to an HTML page the other day.
<G> all this discussion about fearing reviewers when I want to run something by... :)
<jtv> wallyworld: see?  ^^
<wallyworld> G: i don't bite, but jtv does
<wallyworld> i think i need more mentoring therefore :-)
<jtv> I'm doing my best for you, wallyworld.  Don't throw it all away.
<G> wallyworld: well it's lucky I also don't fear people generally :)
<jtv> Repeat afer me: I bite.  I'm a shark.  I'll go as deep as it takes to find what is wrong with your branch.
 * wallyworld repeats I bite.  I'm a shark.  I'll go as deep as it takes to find what is wrong with your branch.
<wallyworld> jtv: why the rejected sentence?
<jtv> Sorry, is this _still_ not clear to you?
<jtv> Fear.
<wallyworld> ah, right.
 * wallyworld needs to grow a pair
<jtv> You don't have to be online who you are in real life though.  Ever seen BjornT and bigjools face off in the two arenas?  The outcomes are very different, I tell you.
<G> wallyworld: putting your self proclaimed fearsome-ness to the test... I'm looking at https://bugs.launchpad.net/launchpad/+bug/793670 and found the issue ln 1819 of lib/lp/registry/browser/person.py runs setPreferredEmail(None), but a reactivation doesn't know which e-mail to set preferred again
<_mup_> Bug #793670: User account missing preferred email after suspension/reactivation <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/793670 >
<jtv> wallyworld: What we have online is you with a huge automatic weapon.
<jtv> No go get 'em.  :)
<G> wallyworld: would the right way be to create a new EmailAddressStatus such as "OLDPREFERRED" to save that detail?
<wallyworld> G: i'll look, give me a second
<wallyworld> jtv: will do
<jtv> :)
<G> wallyworld: so maybe something like setSuspendedPreferredEmail() which changes the status of the current preferred e-mail to OLDPREFERRED and then does _setPreferredEmail(None)
<nigelb> G: jtv doesn't bite. Be makes you bite yourself after seeing his reviews.
<nigelb> *But he
<jtv> Well _if possible_ obviously I like to avoid even that bit of the work.  :)
<G> wallyworld: rationale for a new Status, is that otherwise the question is "which VALIDATED e-mail should become the new PREFERRED e-mail"
<jtv> StevenK, wgrant: we don't import binaries from debian, do we?  I suddenly wonder if there's still any need for gina to import binaries at all.
<wgrant> jtv: We don't use gina's binary importer at the moment, no.
<wgrant> jtv: But removing it right before we start migrating derivative distros onto LP would be somewhat foolish, I think.
<wallyworld> G: another option is to modify the preferredemail property on person to check the account status before returning the email object
<wallyworld> G: that seems cleaner to me
<G> wallyworld: ahhh now that is a good idea
<wallyworld> G: it's a smaller, more localised change with a smaller footprint
<G> wallyworld: but I think the logic to removing the preferred e-mail address is to avoid cases where it may not go through the preferredemail property?
<wgrant> G, wallyworld: This is a pretty difficult and all-encompassing issue.
<wgrant> we need to make person status explicit, and fix a lot of code that checks for preferredemail instead.
<jtv> wgrant: so when are we going to start using gina's binary importer?  What tells us that it will still work properly when we do?  What work have we done to verify that there are no harmful interactions between the binary importer and derived distributions?
<wgrant> DB queries and stuff that doesn't use the property.
<wgrant> jtv: None, but it will probably need to be done this year and deleting it is not a good idea.
<wgrant> G: So, in summary, "run away".
<wgrant> G: This is a deep rabbit hole.
<wallyworld> wgrant: you would hope there's a single piece of logic to do "get the email address to send stuff to for this person"
<wgrant> wallyworld: Hee hee hee.
<jtv> wgrant: so we have something that we don't use, and won't use, and may do some kind of unknown damage.  Therefore we can't delete it shortly before the point where it may start doing that damage.  Does that sum it up?
<wgrant> wallyworld: So, no, there isn't. And the preferred address is used not just for finding the preferred address, but also for checking whether the account is active.
<G> wgrant: so essentially noone understands the logic so run away?
<wallyworld> wgrant: that's dumb. account status should be via an "status" property of some sort
<G> wgrant: thats what I'm saying... introduce a new ENUM option "OLDPREFERRED" and set the PREFERRED to that instead of VALIDATED on suspension
<wgrant> jtv: gina is known to not be in a good state. I hope nobody is crazy enough to run it in anything except source-only Debian mode without first extensively checking the code.
<wallyworld> not piggybacked onto another property
<wgrant> G: But the account status is unrelated to email addresses.
<wgrant> G: It should be a property of the person.
<jtv> wgrant: so what else would we use it for?
<wgrant> jtv: Migrating Linaro and OEM and $OTHER_DERIVATIVES to LP.
<wgrant> wallyworld: You. Would. Think.
<wgrant> wallyworld: There is Account.status, but very little uses it, and Account needs to burn anyway.
<wallyworld> yeah, clearly i'm still too idealistic wrt lp code cleanliness
<G> wgrant: but isn't: make it so LOSA's don't have as much work to do, and "re-implement everything sanely" two different issues?
 * jtv gives up on wgrant
<wgrant> jtv: Hm? We've just enabled LP to handle derivative distributions.
<wgrant> jtv: We have several derivative distributions that need to be moved into LP.
<wgrant> The way to migrate a distribution to LP is to use gina on it, importing binaries too.
<wallyworld> G: you should talk to someone like cutis (who is on leave till next week). he know a lot about our email infrastructure i think
<wgrant> gina needs significant repairs to be able to do that properly nowadays, but deleting the code is hardly going to make it better.
<wgrant> G: Not really. Adding a new email address status is just perpetuating the insanity.
<wgrant> G: We should fix this properly.
<spm> G: fwiw. "so LOSA's don't have as much work to do" is a ... funky way of phrasing it. better would be "so losas can spend THAT time on other tasks equally valuable"
<spm> it's not like we're sitting around on our hands waiting to be pinged :-)
<G> spm: actually that is a better way of putting it :)
<spm> :-)
<G> spm: I should have said "So they don't have to worry about that issue anymore" :)
<G> or something like that :)
<G> wgrant: but yes, I see your point
<G> wgrant: and obviously the solution would be to find everything that generates and e-mail and make sure that it someway or another goes through a proper method to get the e-mail address, that also checks if the account is active
<wgrant> G: Roughly.
<wgrant> jtv: So, am I still given up on? I'm not quite sure what's so controversial about not deleting code that we will probably have to revive to bring this feature into use.
<G> and I'd imagine doing that, and all the subsequent tests to make sure it's working properly would be quite a lot :)
<jtv> wgrant: sorry, I get frustrated sometimes.  I now realize that you meant to give the right answer, but because you never answered the direct question, I completely misinterpreted the rest of what you said.  So for confirmation I posed that "so we're not going to use this code," but you didn't deny it.  After all that, it was a very disorienting realization that you meant the exact opposite of what I was getting all along.
<jtv> I asked a composite question, which in retrospect maybe I shouldn't have done, and so when you talked about "None" and "it," that left a lot of room for interpretation.
<wgrant> jtv: We are going to use the code, but not in its current state.
<wgrant> It was last used in mid-2007. People know not to use it now, so it is not harmful.
<jtv> You really know how to scare me!
<wgrant> But when we want to move distros onto LP, we will have to revive it and verify its correctness.
<jtv> That's the part I was curious about: I got the impression that this was basically no more than a leftover from the original Ubuntu import.
<wgrant> It *is* basically no more than a leftover from the original Ubuntu import.
<wgrant> But it was used 18 months later to import partner too, because it still mostly worked, and didn't break too much.
<jtv> Well one thing more, apparently: the start of derived-distros import.
<wgrant> And I imagine it will be used against soon.
<wgrant> again
<wgrant> Right.
<wgrant> But it's still just a leftoer.
<wgrant> Just one that will be reincarnated.
<wgrant> gina itself was just a leftover from the Ubuntu import.
<wgrant> It got reincarnated and fixed up to import Debian.
<wgrant> Some years later.
<jtv> I don't know how many changes it went through; usually I advocate ditching poorly-understood custom code over reviving it after a long period of falling out of touch with the working application.  It'd do this code a world of good, but I can see the point for not rewriting this kind of stuff on a whim.
<jtv> But oh, the stuff I keep prying out.
<wgrant> gina's not huge, and it's pretty awful, but given the way LP goes it's tempting not to throw away roughly working code.
<wgrant> Because it would naturally take at least 18 months to get a replacement working :/
<jtv> I finally figured out that some very hard-to-read code with practically untyped data was really iterating over a dict's elements in alphabetical key order.
<wgrant> Yeah, there is a bit of that sort of stuff.
<jtv> A bit, he says.
<wgrant> But it's normally possible to untangle.
<wgrant> Eventually.
<wgrant> ... occasionally by rewriting it and seeing what differs when you import an archive.
<jtv> That's what I did with the dominator: a huge sql-laden mega-method turned out to be a few completely independent, easily-described methods.
<jtv> Which reminds me: ran a full test domination of Debian on dogfood.  Seems to have gone pretty well, apart from the known problem with pre-existing publication records.
<jtv> I wonder why we have all this support for deleting a source package without its binary packages though.
<jtv> (Well, requesting deletion of a publication of a source package without doing the same for the publications of its binary packages)
<jtv> Things could be a whole lot simpler and faster without that ability.
<wgrant> SPPH<->BPPH links are special.
<wgrant> ie. implied and varying.
<jtv> Oh.
<jtv> Oh God no.
<jtv> Please no.
<jtv> Remember that complicated code that iterated over a dict's keys?
<jtv> That was for source packages.
<wgrant> And?
<jtv> It has a near-exact counterpart for binary packages.
<wgrant> Ah yes.
<wgrant> That also happens a bit.
<jtv>         for binary in sorted(packages_map.bin_map[archtag].values(),
<jtv>                              key=lambda x: x.get("Package")):
<jtv> âwhere packages_map.bin_map[archtag] is, of course, keyed by x["Package"].
<wgrant> Heh.
<wgrant> I'm glad to see you cleaned up the dominator more.
<wgrant> I did a bit a yearish ago.
<wgrant> But it was still pretty awful.
<wgrant> I didn't get all the way up to judgeSuperseded and judgeAndDominate... just the lower-level stuff.
<wgrant> You seem to have fixed the rest up nicely, however.
<jtv> Thanks.  Didn't even change much in terms of complexity, just moved large blocks of code.
<wgrant> And simplified the setdefault/sort/reverse mess.
<jtv> Oh, that :)
<wgrant> With the creative variable name 'L'.
<jtv>             package_name = binary.get("Package", "unknown")
<wgrant> Erm.
<jtv>  ^^ after all that trouble reconstructing the dict key, which is binary['Package']
<wgrant> s/.*/raise DebianIsInsaneError/
<jtv> I'll catch your DebianInsaneError and raise you a personal sanity failure.
<nigelb> danilos: Hi, around?
<wgrant> Oops.
<wgrant> I made the mistake of using qannotate to check the dominator's history.
<wgrant> Forgetting the qt apps now hang X.
<wgrant> Thanks unity :)
<nigelb> Excellent.
<nigelb> Aren't you going to go the extra mile and blaim thumper for it?
<wgrant> Nah, I've already done that once this week.
<nigelb> *blame
<wgrant> jtv: Launchpad is really good at consistency, isn't it?
<jtv> We're getting better.  I think squads are a step forward.
<wgrant> Even within a squad.
<wgrant> Consider the ?PPH state transitions from PUBLISHED.
<wgrant> supersede(), setDeleted() and requestObsolescence().
<jtv> Well, setDeleted is purely an internal support method.
<wgrant> Three different tenses, three different prefixes.
<wgrant> True.
<jtv> That part makes sense.  The real comparison I think is {requestDeletion, requestObsolescence} and those look consistent to me.
<wgrant> Indeed, I'd forgotten that setDeleted had not entirely replaced requestDeletion.
<jtv> It was never meant to.
<jtv> wgrant: here's a thought about debian domination.  I can make retireDeadSourcePublications find the latest source publication for each package; request deletion for that one and supersede the rest.
<wgrant> jtv: That would work.
<jtv> Oh, gotta help out with something else now.
 * wallyworld is off to see Australia vs Thailand in a soccer World Cup qualifier :-)
<adeuring> good morning
<stub> StevenK: So tell me about our Storm branch. Is there stuff in there that hasn't or cannot land upstream?
<wgrant> stub: Has with_ landed upstream?
<wgrant> The main issue apart from that is that storm now converts Date to a Python date, rather than a datetime.
<stub> I dunno. I think this happened while I was on leave.
<wgrant> which breaks tests and some code.
<stub> Do we have bugs open on Launchpad or Storm?
<wgrant> I don't recall. lifeless did the Date -> date revert initially.
<wgrant> StevenK and I just updated the branch later with fixes we'd landed on trunk.
<stub> ok. I'll trawl some bugs.
<jelmer> wallyworld, hey, congrats on becoming a reviewer and thanks for using your newly found powers to review my code-imports-ui branch !
<lifeless> is someone fixing the merge conflict?
<stub> I can get there in a minute
<lifeless> \o/
<nigelb> danilos: Aha, thanks for merging! I was about to poke you. :)
<mrevell> Hello
<danilos> nigelb, you are welcome (it failed to merge yesterday because we hit "testfix" mode: a mode which stops further landings from happening when someone merges a branch that fails the test suite)
<nigelb> hey mrevell :)
<nigelb> danilos: ah! that makes sense :)
<danilos> hum, how does one update sourcedeps.cache without doing it manually?
<jelmer> danilos, ./utilities/update-sourcecode ?
<danilos> jelmer: oh, thanks :)
<G> hmmm I was looking at the list of trivial Launchpad bugs, and found: https://bugs.launchpad.net/launchpad/+bug/137373 the funny thing is, now it seems the blacklist deosn't exist, and you can 'register' a project named 'codeofconduct' without error, but be generated to the code of conduct page
<mrevell> G: that sounds odd. Let me try that.
<G> mrevell: I just did a grep around, and can only see the error message in relation to person/team names
<mrevell> G: on staging,I get the message "The name 'codeofconduct' has been blocked by the Launchpad administrators."
<G> it's as if the concept of blacklisting project names went into a void/black hole
<G> mrevell: on my rocketfuel setup LP, it just proceeds like nothing happened
<mrevell> G: Weird. On product it's blocked. I did notice that initially it tried to give me a project name of "codeof", though.
<G> haha and now it says "codeofconduct is already used by another project" so the registration obviously worked
<nigelb> G: did you check if the blacklisting is in the db?
<nigelb> I'm just guessing. It is possible that its a table of some sort.
<mrevell> I'm not sure how the blacklisting works. Any ideas wgrant?
<G> for people/teams it's done in python in: ./lib/lp/registry/interfaces/person.py (class PersonNameField(BlacklistableContentNameField)
<StevenK> That's a person, not a project
<G> StevenK: I said that :)
<G> my point is, greping around in my checkout, I can't find anything similar for projects
<G> ohhh except I know what I mayhave done wrong
<nigelb> case?
* adeuring changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: wallyworld, adeuring | Critical bugs: 251!! - 0:[#######=]:256
<danilos> mrevell, I am pretty sure blacklisting is a table in the DB
<StevenK> G: ^
<G> hmmm okay, theory didn't work out
<G> I created a user that didn't belong to any teams in the DB and it still worked w/o error
<danilos> mrevell, "psql launchpad_dev" and then "\d nameblacklist" :)
<G> danilos: ahhh ha
<G> and select * from nameblacklist; doesn't include a regex that matches codeofconduct
<nigelb> Which is why it worked for you.
<StevenK> It's in the production database
<G> yep, okay that makes sense now
<StevenK> SELECT regexp from nameblacklist where regexp like '%code%'; => ^codeofconduct$, ^code$, ^decode$
<nigelb> \o/
<nigelb> Test results: specification-validation-59301 => devel: SUCCESS
<nigelb> Oh. I just landed 2 successive branches.
<nigelb> Achivement Unlocked.
<G> so what would be a better way to put it? "The name (foo) has been blacklisted in Launchpad, please select another project name" ?
<G> nigelb: I'm catching you up ;)
<nigelb> G: Which is still winning. We're making Launchpad better.
<nigelb> :)
<G> exactly, not a race either
<nigelb> Well, it is a race, except we're trying to beat wgrant, not each other :D
 * wgrant forces a /Contributions update.
<wgrant> Not bad, not bad.
<nigelb> Climbed up to two digits in a leap.
<wgrant> It was 10 before.
<wgrant> Hmm.
<nigelb> well, 10 was today as well.
<wgrant> Ah, right.
<wgrant> It appears that the race may be on, and I may have competition :(
<G> wgrant: but aren't you a Canonical employee now?
<nigelb> wgrant: I also love that you can't compete
<wgrant> G: Yes :(
<nigelb> G: He is, his numbers are static.
<nigelb> (Those are pre-Canonical numbers)
<G> oh I get it, that is just tracking the pre-employment commits?
<wgrant> So I have a 130 branch head start, but no velocity.
<wgrant> Right.
<wgrant> Everything post-employment is with my Canonical address, so is conveniently excluded.
<G> okay, I think I've come up with another good tiny bug to kill, one that actually confused me :) https://bugs.launchpad.net/launchpad/+bug/306569
<_mup_> Bug #306569: Link to https://help.launchpad.net/Code from project branch listing page <lp-code> <trivial> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/306569 >
<nigelb> wow
<nigelb> OOPS-2071AT29
<nigelb> when I propose a merge
 * nigelb kicks _mup_ 
<LPCIBot> Project devel build #1,024: STILL FAILING in 4 hr 8 min: https://lpci.wedontsleep.org/job/devel/1024/
<wgrant> nigelb: I find it encouraging that timeouts surprise you now.
<nigelb> wgrant: Well, it does. LP has been fairly untimeouting except for staging and qastaging.
<nigelb> wallyworld: can I add https://code.launchpad.net/~nigelbabu/launchpad/merge-review-mail-824487/+merge/73765 to your list as well?
<wgrant> I hope wallyworld is not still here.
<nigelb> In which case, adeuring would you review this branch ^^
<adeuring> nigelb: sure
<nigelb> thanks!
<adeuring> nigelb: nice work, r=me
<adeuring> should I land it?
<nigelb> adeuring: Yes, please :)
<adeuring> ok
<danilos> anyone feels like reviewing a simple loggerhead branch?
<jelmer> danilos, might as well
<danilos> jelmer, cool, https://code.launchpad.net/~danilo/loggerhead/bug-839395/+merge/73766
<danilos> jelmer, actually, got a review from jam in the meantime
<jelmer> actually, it looks like jam beat me to it :)
<danilos> jelmer, yeah, though his vote was not registered by LP
<jelmer> danilos, he's hanging out in #bzr
<gary_poster> A bit of insomnia has me noticing that buildbot is unhappy.  The failing tests are all bzr bits and bobs, which no-one seemed to have touched.  Has anyone investigated?  I might try running some of the failing tests locally, and if they pass, I might force a build.
 * jelmer has touched some bzr bits recently
<jelmer> I'll have a look
<wgrant> jelmer: We've had several lots of spurious bzr (particularly puller and import) failures since your recent series of branches.
<wgrant> They've always been fragile, but they were mostly sorted for the last 6 months.
<jelmer> well, I did reenable some that were supposedly going to be fixed with the new version of twisted
<gary_poster> jelmer, thanks.  FWIW, they do pass locally, so it sounds like it is fragility
<jelmer> actually, it looks like these are not the tests I was expecting to fail
<jelmer> these are all failing with "no such module bzrdir", which is pretty weird
<gary_poster> all sent by the server too
<gary_poster> rather than in the main process
<jelmer> yeah
<gary_poster> would be nice if we could dupe locally :-/
<jelmer> I wonder if somehow the system bzrlib gets loaded
<gary_poster> Lemme see if I can find out how that subprocess is started...
 * jelmer has to run off for lunch, back in ~30
<danilos> adeuring, hi, a very simple branch up for review on https://code.launchpad.net/~danilo/launchpad/bug-839395/+merge/73779
<adeuring> danilos: ok, I'll look
<danilos> adeuring, thanks
<gary_poster> FWIW, all tests using ForkingServerForTests failed
<adeuring> danilos: r=me
<gary_poster> Going to ask for some diagnostics from the bb machine
<danilos> adeuring, thanks, landing already :)
* bac changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: wallyworld, adeuring, bac | Critical bugs: 251!! - 0:[#######=]:256
* bac changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer:adeuring, bac | Critical bugs: 251!! - 0:[#######=]:256
<bac> hello adeuring
<adeuring> hi bac
<jelmer> gary_poster: ah, interesting
<jelmer> gary_poster: I wonder if we're using bzrlib.bzrdir somewhere but don't do "import bzrlib.bzrdir" in the forking service
<stub> gary_poster: Is this an intermittent problem? I just resolved a merge conflict between stable and db-devel dealing with bzrlib imports
<gary_poster> stub, yes intemittent, AFAICT
<stub> so this might go and infect db-devel now
<gary_poster> jelmer, sorry, I have to do morning things with children now.  Will be back in < hour
<stub> gary_poster: Interestingly, the imports where the same but the order was different.
<nigelb> Is Julian away today?
<jelmer> gary_poster: np, talk to you later
<wgrant> nigelb: Yes.
<wgrant> nigelb: He will return next week.
<nigelb> Ah, ok. I found a few bugs he'd opened which doesn't seem reproducible.
<wgrant> nigelb: Link?
<nigelb> Let me hunt them down.
<nigelb> bug 814696, bug 814697
<_mup_> Bug #814696: Link to show inline diffs in merge proposals should be green <code-review> <trivial> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/814696 >
<_mup_> Bug #814697: Inline diffs in merge proposals don't have a spinner when it's waiting for the diff <code-review> <trivial> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/814697 >
<nigelb> Inline diffs in an MP is green. And there's no spinner since it just points to the diff further down in the page.
<wgrant> He may mean inline diffs *for* MPs.
<wgrant> eg. on branch, bug pages.
<wgrant> But they are also green and have a spinner.
<nigelb> Right.
<wgrant> It was broken due to a partial revert for a few hours a month or so ago... not sure if the timelines coincide.
<nigelb> It was filed a little over a month ago.
<nigelb> So, entirely possible.
<nigelb> Hrm, is bug 787798 trivial?
<_mup_> Bug #787798: Launchpad should use Google web fonts for the Ubuntu font <css> <trivial> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/787798 >
<nigelb> I tend to think not.
<wgrant> nigelb: Why not?
<wgrant> It's basically one line of HTML.
<wgrant> Although it needs a bit of thought as to which subset of the font we want.
<wgrant> Probably all of it.
<nigelb> One line of HTML and another line of CSS.
<wgrant> Oh?
<nigelb> It might end up looking ugly for monospace.
<wgrant> We already use the Ubuntu font in our CSS.
<nigelb> AH!
<nigelb> one line of HTML then
 * nigelb grabs
<nigelb> hrm, interestingly. I have no idea which is the page template thing for LP.
<wgrant> nigelb: base-layout.pt
<nigelb> Interesting.
<nigelb> \o/ 4 branches in a day? I'm aiming for my personal best today.
<jelmer> nigelb: whoa
 * jelmer is glad if he manages one lp branch in a day
<nigelb> jelmer: 2 of them got merged only today. I worked on them for 2 days
<nigelb> the other 2 are small fixes :)
<nigelb> wgrant: Does this need tests?
<nigelb> (is there a way to test this?)
<bac> go nigelb, go!
<nigelb> bac: heh thanks! And for that, can you review https://code.launchpad.net/~nigelbabu/launchpad/ubuntu-font-787798/+merge/73793
<bac> nigelb: would be glad to, in just a bit.  i'll claim it now.
<nigelb> cool!
<gary_poster> jelmer, I'm back, but assuming that you are on the test failure.
<gary_poster> jelmer, heh
<jelmer> hah, brain wave sync. or something. :)
<gary_poster> :-)
<jelmer> I haven't found much yet, other than that previous hypothesis is almost certainly wrong.
<gary_poster> jelmer, the bzrdir not imported idea you had is reasonable, though I would have expected...heh, ok
<nigelb> bac: That article about checklists was "enlightening"!
<bac> i should re-read it.  saw it when it first appeared in the NYKR
<gary_poster> jelmer, I'm not sure where to go with this one.  site.py is where we set up paths, and it is obviously working fine for the main process, and the environ passed to the forked server subprocess on my system was correct
<gary_poster> I looked at site.py on buildbot and it was fine...
<gary_poster> Everything was also OK here before 2.4.x so that's the most obvious smoking gun
<gary_poster> but I certainly would not like to roll that back, and I'm sure you feel even more strongly that way
<jelmer_> yeah
<jelmer_> gary_poster, db-devel has had 2.4 for a while
<gary_poster> If I could dupe that would sure be nice.  I tell you what jelmer, I'm going to ask for a screen session on the buildbot and try running the tests there and see if I learn anything.  Are you around for awhile?
<jelmer_> gary_poster: I'll be out for about an hour soon, but should be able to help after that
<gary_poster> ack thanks jelmer_
<nigelb> Is buildbot temporarily broken?
<StevenK> Testfix
<gary_poster> nigelb, we are in testfix
<gary_poster> nigelb, I'm working on it
 * gary_poster has to step away for 20
<nigelb> Ah!
 * gary_poster back
<bac> hi nigelb, thanks for the font fix.  i'm not sure making the change in the base page template is the right place to put it.  there are still other fonts being used, on buttons for example.
<bac> nigelb: i'd like to talk to sinzui to get his thoughts
<nigelb> bac: I'm only making sure google web fonts get included.
<nigelb> The font gets changed to Ubuntu wherever the font is set as 'Ubuntu'
<nigelb> Also, hrm, I was looking for sinzui the other day as well. Has he not been around?
<bac> nigelb: i don't know his schedule
<bac> there is a US holiday on monday though, so if he isn't around today it'll be tuesday before he comes back
<StevenK> sinzui has been on holidays this week, like I have.
<nigelb> Ah!
<nigelb> I'll wait for next week :)
<bac> nigelb: ok, sorry to put a speed bump in your mojo today!  :)
<nigelb> heh
<nigelb> no worries!
<nigelb> I'll applaud sinzui for *really* being away when he's suspposed to be away.
 * nigelb glances at StevenK and wgrant.
<nigelb> mrevell: Might be interesting to you - http://nigelb.me/ubuntu/launchpad/2011/09/02/some-new-improvements-to-lp.html
<wgrant> nigelb: Heh.
<danilos> adeuring, bac: anyone eager to pick up a branch for review: ://code.launchpad.net/~danilo/launchpad/bug-302449/+merge/73813
<bac> danilos: sure
<LPCIBot> Project devel build #1,025: STILL FAILING in 4 hr 9 min: https://lpci.wedontsleep.org/job/devel/1025/
<danilos> bac, thanks :)
<mrevell> thanks nigelb
<bac> danilos: why does 'make lint' not report problems for you?  i see it complaining about the use of a==b in storm clauses with no spaces.
<danilos> bac, I thought I fixed that
<danilos> bac, :/
<bac> oh, maybe you didn't push
<danilos> bac, actually, I didn't even commit :)
<danilos> bac, pushing now
<bac> you need the 'bzr-telepathic' plugin
<danilos> bac, heh, I thought I had that
<bac> danilos: done.  thanks.
<stub> https://code.launchpad.net/~stub/launchpad/dboptions/+merge/73824 needs a review. I can deal with it in 15 mins, or tomorrow so schedule accordingly :-)
<bac> stub: done
<stub> bac: ta
<nigelb> I just got an interesting comment :)
<nigelb> "Do you know how to make lp sexy? MAKE A DESCRIPTION HOW TO CLONE REPO! "
<G> actually, I was thinking about that myself
<nigelb> There is instructions here - https://code.launchpad.net/~launchpad-pqm/launchpad/devel
<nigelb> Get this branch: bzr branch lp:launchpad
<G> I was looking at https://bugs.launchpad.net/launchpad/+bug/306569 and thinking of fixing it
<_mup_> Bug #306569: Link to https://help.launchpad.net/Code from project branch listing page <lp-code> <trivial> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/306569 >
<G> because I must admit, it is sometimes a bit vague
<gary_poster> jelmer_ or abentley, I'm about to submit a diagnostic hack for when the forked server might fail again.  The output when we have an error will be something like this. http://pastebin.ubuntu.com/680623/  Unfortunately, you'll notice that the message, delivered straight from bzr, does not give me a  traceback for where this bad flafla import was (it was in the lpserve plugin).  Is there a way to make it give me one?
<nigelb> G: Oooh, good one
<gary_poster> -v does not
<abentley> gary_poster: yes.  From the commandline, you'd use -Derror.  I don't know what the equivalent in bzrlib is.
<G> nigelb: I just assigned it to $me :)
<gary_poster> abentley, would it hurt anything to start lpserve up in that way for the tests?
<gary_poster> I can try :-)
 * gary_poster does
<abentley> gary_poster: I think that would be fine.  There's a small chance that a test is expecting the non-Derror output.
<jelmer_> gary_poster: from bzrlib import debug; debug.debug_flags.add("error") is the python equivalent of using -D
<gary_poster> jelmer_, cool, thanks.  The forking server uses subprocess to run lpserve on the commandline, so -Derror works fine.  So far tests  are passing with the change and the error message is much more informative, as you'd expect.
* adeuring changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: bac | Critical bugs: 251!! - 0:[#######=]:256
<gary_poster> abentley, thanks that worked well.  Could you take a glance at https://code.launchpad.net/~gary/launchpad/testfix/+merge/73840 before I submit it to see if you have any better ideas that I can implement now or later?
<abentley> gary_poster: what does "we are not on a testcase" mean?
<gary_poster> abentley, it is supposed to mean that self is not a testcase
<gary_poster> which is where the "addDetail" lives
<gary_poster> abentley, better?
<gary_poster> We cannot use the "addDetail" method because this class is not a TestCase and does not have access to one.  It runs as part of a layer.
<abentley> gary_poster: I'm seeing that now.
<abentley> gary_poster: Since it's part of the layer, we can't even pass a TestCase in.
<jml> fixtures have addDetail, or something similar
<gary_poster> right, abentley.  jml, this is a tool used by a Zope layer.  Our layers are not fixtures, right?
<jml> gary_poster: some of them are almost fixtures. The LibrarianLayer, for example.
 * gary_poster goes base class hopping...
<abentley> gary_poster: I guess I wonder whether the return value isn't a sufficient test by itself.
<gary_poster> abentley, that's what I had initially.  It wasn't sufficient practically: the return value was None.  The process hadn't completely died yet.  I could do a time.sleep before the poll, but I thought the error string check was the lesser of those two evils.
<abentley> gary_poster: I'm mildly surprised that Popen.wait doesn't accept a timeout.
<gary_poster> ack abentley, me too.  It looks like the underlying C module, exposed as _subprocess, takes either a 0 or INFINITE timeout.  Maybe it has other options, but that's all that subprocess.py uses.
<gary_poster> ah, no, that's windows...
<abentley> gary_poster: So it seems okay to me.  Perhaps you could check for success, rather than failure, which might look a bit less kludgy.
<gary_poster> abentley, ok thanks.  I thought about checking for success, but it seemed more fragile.  The success output is this:
<gary_poster> Preloading 0 modules
<gary_poster> Listening on socket: /var/tmp/launchpad_forking_service.sock
<gary_poster> well, not 0 modules
<gary_poster> but anyway
<gary_poster> the first line seems to happen no matter what
<gary_poster> but I could be wrong
<gary_poster> abentley, if you feel stongly about it, I'll do it.  I don't.  Otherwise I'll land as is.
<gary_poster> ("I don't" == I don't feel strongly about it.)
<abentley> gary_poster: I don't feel strongly.  I was just trying to help you find a solution that made you happier.  My theory is that success is predictable, but failure can take many forms.  So if you don't get something you recognize, it's a failure.
<gary_poster> Hm, yeah, there's something to that abentley.  OK, I'll maybe look for "Listening on socket" and fall over if not found.  Thank you very much.
<abentley> gary_poster: np.
<LPCIBot> Project devel build #1,026: STILL FAILING in 4 hr 55 min: https://lpci.wedontsleep.org/job/devel/1026/
* bac changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 251!! - 0:[#######=]:256
#launchpad-dev 2011-09-03
<LPCIBot> Project devel build #1,027: STILL FAILING in 4 hr 6 min: https://lpci.wedontsleep.org/job/devel/1027/
<StevenK> wgrant: lpci has a new cert
<wgrant> StevenK: Indeed, thanks!
<nigelb> Morning!
<nigelb> *yawn*
<nigelb> are we still in testfix?
<nigelb> hmm. QA.
<LPCIBot> Project devel build #1,028: STILL FAILING in 4 hr 8 min: https://lpci.wedontsleep.org/job/devel/1028/
<G> nigelb: kia ora
<StevenK> G: O hai. You haz qa
<StevenK> G: I can give you run-down if you need it.
<G> StevenK: huh?
<StevenK> G: Launchpad has a deployment report for which revisions are deployable to production. Before they can deployed they need to be QA'd and marked as such in the bug.
<G> StevenK: oh so this is something that I have to / can do?
<StevenK> Sadly, the deployment report is private, but I've started the process to open it up.
<StevenK> G: We prefer people do their own QA if they can.
<G> sure I can do it myself
<StevenK> G: r13839 / bug 837290 and r13856 / bug 61428
<_mup_> Bug #837290: When there is no "Other bug subscribers" incorrect descriptive text is displayed <qa-needstesting> <Launchpad itself:Fix Committed by dev-nigelj> < https://launchpad.net/bugs/837290 >
<_mup_> Bug #61428: Want a "subscribed to teams" portlet... <feature> <lp-bugs> <qa-needstesting> <trivial> <ui> <Launchpad itself:Fix Committed by dev-nigelj> < https://launchpad.net/bugs/61428 >
<G> StevenK: yep I was just loading up the bugs myself :)
<StevenK> G: You can use qastaging.launchpad.net to see if your code fixes the problem
<G> okay
<StevenK> G: If it looks good, then remove the tag of 'qa-needstesting' and add one of 'qa-ok'
<G> is qastaging clean data or what?
<G> oh nevermind, I clicked login and openid logged me in
<StevenK> qastaging is using the production database, which is probably a few months out of date
<StevenK> qastaging and staging use their own databases, so it's fine to play with
<G> StevenK: okay, 837290 is qa-ok now
<StevenK> G: Excellent!
<StevenK> We have 29 revisions in the queue, so I doubt we're fully deployable until AU Tuesday morning, with probably some deployable UK Monday morning
<G> ditto for 61428
<G> StevenK: done
 * G hits the big read "That was Easy!" button
<G> *red
<G> StevenK: btw, if there are any other bugs that need a look over for verification just let me know
<StevenK> G: Your 3 revisions are all qa-ok :-)
<G> StevenK: yep :)
<LPCIBot> Project devel build #1,029: STILL FAILING in 4 hr 8 min: https://lpci.wedontsleep.org/job/devel/1029/
<nigelb> StevenK: are we still in testfix?
<nigelb> One of my branches is unmerged I think.
<StevenK> Sigh. Yes.
<StevenK> I can force, but there is little point, since it will fail that test.
<nigelb> Ah
<nigelb> this is the bzr-related test which was talked about yesterday?
<StevenK> Yeah, one of the lpserve tests
<nigelb> *whee*
<nigelb> I think I was on the blamelist of buildbot when it failed.
 * nigelb remembers getting a failure email.
<G> hmmm interesting
<G> I was looking at the code behind https://bugs.launchpad.net/launchpad/+bug/697157 (which I'm going to fix) and noticed something interesting that I hadn't really realized
<_mup_> Bug #697157: Streamline "You have assigned this bug to yourself" message  <bugs> <trivial> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/697157 >
<G> sometimes an action can generate two e-mails (i.e. comment on bug, and assign to $self) first been the generate comment bug, and then another e-mail implying that the bug is NEW sent 2 seconds later
#launchpad-dev 2011-09-04
<nigelb> hrm, I didn't know sinzui was a Gnome/gtk person as well.
<nigelb> Ah,gedit developer plugins.
 * StevenK grumbles at Jenkins four persistant test failures
<StevenK> ... which don't fail on db-devel. Bleh.
<nigelb> StevenK: I will gladly grumble more about buildbot
<nigelb> :)
<StevenK> No one else seems to
<nigelb> My merge is pending because of buildbot :(
<StevenK> No, your merge was rejected due to buildbot-poller and testfix mode
<nigelb> oh.
<StevenK> buildbot just handles builds of devel and db-devel. buildbot-poller runs via cron, and checks to see how the last builds finished and changes the config of PQM.
<nigelb> Ahhh.
<StevenK> nigelb: A build has been forced, so we are likely out of testfix if you want your branch landed
<nigelb> StevenK: \o/
 * nigelb heads for lunch and offline-ish things
<StevenK> Which requires me to know which branch ... :-)
 * G can't get my dev instance of launchpad working with e-mail :S
<wgrant> G: What's not sending mail?
<wgrant> G: Appserver mail will go to root@localhost. Script mail is not sent at all.
<G> wgrant: yeah, I think I just realised my problem... bug mail is scripted
<wgrant> send-bug-notifications.py -vv or so
<G> I've got postfix setup as a localhost mail server, with @canonical.com @ubuntu.com going to my user's mbox
<wgrant> Dev appservers redirect mail to root@localhost, and scripts (well, all those that use zopeless, which is just about all) send to nowhere. Most scripts will output the mail to stderr when given a -v or two.
<wgrant> I run LP is a lucid LXC container... I have script mail turned on there, because its postfix relays through my host, which delivers everything to my user.
<G> wgrant: how did you enable the script-mail?
<wgrant> I may tell you once you ensure everything goes to a local user, not just @canonical.com and @ubuntu.com :)
<G> wgrant: oh I have :)
<wgrant> Tested various non-Canonicalish domains, and none leave your machine?
<wgrant> Anyway, for bug notifications just give send-bug-notifications.py a couple of -vs, and it will show you what it "sends"/.
<G> http://pastebin.ubuntu.com/681695/
<wgrant> The second one resulted in an NDR...
<G> wgrant: yep, but so does my real @gmail.com
<G> I've got inet_interfaces set to loopback_only & default_transport to error
<wgrant> Ah. I use default_transport = local, luser_relay = wgrant
<G> and I have virtual_alias_maps w/ @canonical.com & @ubuntu.com njones
<G> don't worry, it was the first thing I checked, last thing I wanted to do, was spam Mark Shuttleworth :)
<StevenK> It would be amusing.
<StevenK> For those of us that aren't you, that is.
<G> hahah yeah ):
<G> only reason why I want to play w/ bug e-mails is because I think I spotted a bug where e-mails are sent out in the wrong order
<wgrant> I only know of one case.
<wgrant> Subscribed/assigned emails can be sent from the appserver.
<wgrant> During the request.
<wgrant> So they're not queued like the rest.
<wgrant> It can be a bit strange.
<wgrant> I forget exact details.
<wgrant> Anyway, in configs/development/launchpad-lazr.conf, right down the bottom, there should be a sinzui comment about zopeless mail. Flip that setting.
<G> well take this for example: http://dev.nigelj.com/buge-mail.png logically speaking, the [NEW] should arrive before the non [NEW]
<G> but I've also seen it recently when doing: assign to me, + comment at the same time
<wgrant> G: That's probably more to do with the Message-IDs and References headers. Have you checked the timestamps?
<G> wgrant: yeah, 2 seconds in it
<G> wgrant: which makes me think it's the order it's put in the queue
<G> hmmm looks like watching TV + LP 'make run' and the python send bug mail script is too much for my laptop
<wgrant> G: [NEW] is probably sent direct from the appserver.
<wgrant> Yeah, LP likes its RAM.
<wgrant> Particularly on amd64.
<wgrant> Yay Python.
<G> wgrant: I had a poke round, and it seems they are both sent the same way
<G> I think it's a case of wrong order, which is why I wanted the e-mails via postfix just to test the theory and have everything in one place
<G> LP likes it's RAM, and TV likes it GPU/CPU for the h264
<wgrant> G: If you have an example from production, the headers will tell you where they are sent. loganberry/ackee run send-bug-notifications and other scripts, while chaenomeles/gac/soybean/wampee run appservers.
<G> wgrant: ahhh thanks
<G> in that cause you are right
<G> [NEW] comes from the appservers, the others are from send-bug...
 * G takes it servers are named after fruits or something there?
<wgrant> There have been a few naming schemes.
<wgrant> First was penguins. Which is why there is gentoo.ubuntu.com.
<wgrant> Then there were Antarctic bases.
<G> oh I thought that was a big joke
<wgrant> Then elements (you'll see lots of these on https://launchpad.net/builders)
<G> I'd seen it mentioned somewhere
<wgrant> Then fruits.
<wgrant> And now stars, although there aren't many of them yet.
<G> heh, Native New Zealand Birds for my network + VPS'
<G> unfortunately some aren't overly suitable, because although they are Maori, they could be read the wrong way :)
<wgrant> There have been a number of names vetoed, sadly :(
<G> I bet :)
<G> come to think of it, amazing that only 4 machines are needed to run the main part of launchpad.net
<wgrant> There are lots more behind the scenes, but yes, those four serve all the production appserver requests.
<G> oh yeah, I saw some comment the other day about the DB servers and that
<G> and I'd hate to think how much storage is backing the PPAs and that but still
<wgrant> A bit :)
<G> (For what it's worth I used to be a Fedora guy, but I kept running into brick walls, and PPAs was actually something that converted me
<G> hated having to roll my own RPMs and keeping track of them somehow
<wgrant> Arch has AUR, openSUSE has OBS... does Fedora have anything?
<G> not unless they added something to Koji in the last year
<G> (Koji been the buildsystem)
<wgrant> I've never understood that. Everyone else has a TLA... even Debian. But noooo, Fedora just has to go and use a word instead :P
<G> haha
<G> but yeah, one of the things I like about Launchpad, is that it's all interconnected like it should be
<G> (at least imo)
<wgrant> Well, a bit :)
<G> Fedora has about 5 or 6 core 'applications' and then they still have to use RH's Bugzilla ;)
<wgrant> There is great benefit in having them all linked together. But also a significant cost with the current way things are done.
<wgrant> ie. a 650KLOC monolithic Zope webapp.
<wgrant> But things are moving in the right direction now.
<G> tbh, I thought Zope had died out
<wgrant> I'm not sure that Zope 3 was ever really alive.
<G> hmmm can't seem to confirm the bug I was looking for but I think I found a completely different one
<G> It wouldn't be an issue for launchpad.net, but for instance, my machine is +1200, script mail, comes through with the correct date/time, but e-mail from the app layer, comes through at NZST, but saying it's -0000
<wgrant> Yeah, there's a bug for that.
<wgrant> Until a year or so ago the appservers ran in BST, so some mail was hour off for a few months a year.
<wgrant> I'm pretty sure I filed it years ago...
<wgrant> Bug #262040
<_mup_> Bug #262040: Bogus timestamps on some unbatched bugmail <email> <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/262040 >
<LPCIBot> Project devel build #1,030: STILL FAILING in 4 hr 38 min: https://lpci.wedontsleep.org/job/devel/1030/
<G> wgrant: btw, thanks for the help before, managed to get at least one of the issues I wanted sorted
<nigelb> staging is down?
<nigelb> I love who StevenK's branches are always deleting or killing or burning :D
<nigelb> *how
<nigelb> GAH
<mwhudson> qa-ing merge proposals on qastating seems to be fun
<wgrant> mwhudson: All the files are restricted, so yeah, you basically have to create a new one.
<mwhudson> yeah, it's not so bad i guess, just a bit tedious
<wallyworld_> wgrant: i have a packaging question for you. i upgraded to oneiric beta over the w/e and it hosed my system so i reinstalled from scratch
<wallyworld_> wgrant:  and now lp devel dependencies won't install
<wallyworld_> because postgressql-8.4-debversion can't be installed
<wgrant> What's the error?
<wgrant> Hmm.
<wgrant> Ah.
<wgrant> It's only here for 9.1
<wgrant> Grab it from Natty for now.
<wallyworld_> because libapt-pkg.4.11 is instgalled with oneiric and it needs libapt-pkg4.10
<wgrant> Ah.
<wallyworld_> i forced libapt-pkg-4.10 to install
<wallyworld_> but synaptic keeps removing it
<wallyworld_> because it says it's broken
<wgrant> It is.
<wgrant> GIve me a few minutes and I'll have an oneiric postgresql-8.4-debversion in the PPA...
<wallyworld_> cool! thanks
<wallyworld_> i don't know much about this stuff
<wgrant> But you should use a lucid LXC :)
<wallyworld_> wgrant: yeah, haven't reinstalled lxc yet
<wallyworld_> but lp does run ok with my current broken set up
<wgrant> Hmm. We really should upgrade to 9.1 soon.
<wallyworld_> yes, an upgrade to 9.1 would be good
<wallyworld_> scary though given the pain with 8.3->8.4
<wgrant> Hmm, not sure if the new version will actually build with 8.4. May have to rename the old one and build that as well.
<wallyworld_> wgrant: i think this happened because there was a recent apt upgrade in oneiric. so is your system broken yet? or are you yet to see this issue locally?
<wgrant> wallyworld_: I use LXC :)
<wallyworld_> ah of course
<wallyworld_> i'll reinstall lxc but the memory saving didn't seem worth the hassle. but this on the other hand...
<wgrant> wallyworld_: Could you test the postgresql-8.4-debversion from ppa:wgrant/experimental?
<wgrant> It finally built.
<wallyworld_> wgrant: sure will. thanks very much
 * wallyworld_ opens synaptic
<wallyworld_> wgrant: it works \o/
<wgrant> Yay.
 * wgrant copies.
<wallyworld_> now all i need to do is disable zeitgeist because it keeps easting all my cpu and oneiric is looking good
#launchpad-dev 2012-08-27
<wgrant> StevenK: More importantly you should check what the subscribers are
<StevenK> wgrant: So I guess I should set a bug supervisor too, then
<wgrant> Right
 * StevenK picks wgrant.
<StevenK> wgrant: https://bugs.qastaging.launchpad.net/auditorclient/+bug/939552 looks good to me
<_mup_> Bug #939552: Juju should support MAAS as a provider <juju:Fix Released by allenap> < https://launchpad.net/bugs/939552 >
<StevenK> wallyworld_, wgrant: https://code.launchpad.net/~stevenk/launchpad/contains-to-match/+merge/121354
<wallyworld_> +1 from me
<StevenK> wallyworld_: Thanks
<wallyworld_> np
<wgrant> webops: Could you ppa-reset marid, please?
<wgrant> It's on furud
<spm> ta
<spm> wgrant: hmm. oki, how does one reset a single server. ppa-reset seems to be an all or nothing affair?
<wgrant> spm: I think 'ppa-reset marid' should work
<spm> I don't have rights for that
<wgrant> Ah
<wgrant> You'll have to do it from alphecca, I guess
<wgrant> sec
<spm> and looking at scripts, I'm going to need gsa intervention
<wgrant> Nope
<wgrant> ssh -i /home/lp_buildd/.ssh/ppa-reset-builder ppa@furud.ppa ppa-reset marid
<spm> ah yes, the builddmaster would have stab ability.
<spm> Mon, 27 Aug 2012 01:20:18 +0000: Clearing all marid Copy-On-Write devices.
<spm> device-mapper: create ioctl failed: Device or resource busy
<spm> Command failed
<wgrant> That's rather unpleasant of it
<spm> that doesn't look happy.
<wgrant> Sounds GSAy
<StevenK> wgrant: Shall I put together a deploy?
<wgrant> StevenK: Worth a try
<wallyworld_> wgrant: according to bug 1040999, branches should always be able to be marked as security fixes, with userdata only available if branch is linked to a userdata bug. so i'm going to make this change
<_mup_> Bug #1040999: Cannot use branch information type portlet to set type <disclosure> <information-type> <javascript> <Launchpad itself:In Progress by wallyworld> < https://launchpad.net/bugs/1040999 >
<wgrant> wallyworld_: Sounds reasonable
<wallyworld_> but proprietary cannot be done just yet i don't think?
<wgrant> It can be
<wgrant> It should always be shown if it's allowed
<wgrant> Like Public
<wgrant> Hm
<wgrant> Actually
<wgrant> We can't really hide userdata until nobody's using BVPs
<wallyworld_> it's always there now, but the code comments say it should only be shown if branch linked to proprietary bug
<wgrant> That's meant to apply to Public Security, Private Security and Private
<wgrant> Not Proprietary
 * wgrant checks the comments
<wgrant> Oh
<wgrant>             # Once Proprietary is fully deployed, it should be added here.
<wgrant> "it" == Private there
<wgrant> I must have removed a line describing why Private wasn't included
<wallyworld_> ok, i'll fix the comment
<wgrant> Thanks.
<wgrant> The idea is that we need to show Private now since it's what BVPs use for privacy
<wallyworld_> so just to confirm, userdata is to be updated once bvps go away
<wgrant> But once everyone's using Proprietary, Private is no longer going to be common at all for branches
<spm> wgrant: it's being stubborn, but looked at.
<wgrant> spm: Thanks
<StevenK> wgrant: The amount of cowboys is terrible :-(
<wgrant> Yeah...
<wgrant> All of them have landed, though
<wgrant> Only one code changes
<wgrant> -s
<wgrant> Rest are security.cfg
<wgrant> So we can ndt without a problem
<wgrant> Um
<wgrant> Though
<wgrant> StevenK: Have you checked for new DB perms?
<StevenK> I have not.
 * wgrant does so
<StevenK> garbo is one I can think of, I think
<wgrant> Oh
<wgrant> Can't pull
<wgrant> Blah
<StevenK> Hahaha
 * wgrant does it manually
<StevenK> wgrant: Is this going to involve a second call to Optarse?
<wgrant> Already done
<wgrant> Two sets of DB perms
<StevenK> garbo and what else?
<spm> wgrant: we believe that's back
<wgrant> StevenK, webops: There's an SQL request at the usual place to manually apply this nodowntime's DB security changes
<wgrant> Which will take us up to 5 live cowboys :)
<spm> ./ignore wgrant
<wgrant> spm: marid looks healthy again, thanks
<StevenK> wgrant: Do you even have an ETA from them?
<wgrant> No
<StevenK> Because routing to Europe is hard or something.
<wgrant> Maybe I can convince a GSA to check what the melons think of the BGP state
<wgrant> L3's looking glass looks fine
<wgrant> Maybe Datahop is breaking things
<bigjools> urls for multi-task bugs are weird
<bigjools> I entered a bug on maas, url has maas in it.  Add a task for cloud-init and the url changes to one for cloud-init
<wgrant> bigjools: Right, when you add a new task it sends you to the bug in that context
<StevenK> wgrant: No test failure for me. :-(
<lifeless> StevenK: auditor today?
<StevenK> Haha
<StevenK> wgrant: http://pastebin.ubuntu.com/1169187/ == no failure after make schema
<wgrant> StevenK: The problem is creating the link from a --fixes
<StevenK> wgrant: I thought that just linked the bug?
<wgrant> StevenK: Yes
<wgrant> But it's the bug linking that crashes
<wgrant> Not scanning a branch with a linked bug
<wgrant> You can probably reproduce by switching to the DB user before calling linkBug
<StevenK> wgrant: Calling db_branch.linkBug() in the with dbuser block == no crash
<wgrant> StevenK: Hm, possibly it's not calling the notify methods?
<wgrant> I forget where in the traceback it died
<wgrant> But I linked the OOPS in the bug
<StevenK> wgrant: Ah, yes, that would likely be it.
<StevenK> Actually, I think it's because the project that the bug is created has no structural subscribers.
<StevenK> And maybe no notification
<wgrant> That could do it too
<StevenK> wgrant: Right, I'm hooking it into the bug linking bit, I needed a revprop with the right format.
<StevenK> But still no failure, which is annoying.
<StevenK> Oh, hah. My contains-to-match trips over test_getAllPermissions_constant_query_count
<wgrant> Heh
<StevenK> wgrant: Still no failure. :-(
<spm> wgrant: psql:tmp/wg.sql:8: ERROR:  cannot execute GRANT in a read-only transaction
<wgrant> spm: tThat's no druk
<spm> oh wait. nm. my bad. wrong server.
<spm> yah
<spm> I blame mondays.
<wgrant> I blame Optus/Datahop/NTT :)
<spm> ad that tomorrow is tuesday. and yesterday being sunday.
<wgrant> For everything
<spm> wgrant: applied
<wgrant> spm: Thank
<wgrant> s
<wgrant> Hopefully you can now ndt without the world burning down
<wgrant> more than it already is
<StevenK> spm: http://images.ucomics.com/comics/ga/1991/ga910304.gif
<wgrant> woah
<wgrant> ppa queue almost caught up
<StevenK> Wah, still no failure.
<StevenK> Ah, the job running code isn't sending a notification.
<StevenK> wgrant: I think this test is being defeated by caching
<wgrant> StevenK: :(
<wgrant> StevenK: Then clear the cache :)
<StevenK> Which doesn't help either
<StevenK> Sigh
<StevenK> wgrant: Store.of(obj).invalidate() only invalidates only that object? What if I want to invalidate everything?
<wgrant> StevenK: That invalidates the whole store
<StevenK> Then I'm not sure why notify isn't calling back into getBugNotificationRecipients :-(
<StevenK> wgrant: The notify(ObjectCreatedEvent(bug_branch))
<StevenK> line in linkBranch() is directly implicated in the OOPS, but my test causes it to notify noone
<wgrant> You've verified that there are adequate subs to the bug?
<StevenK> I've added a direct subscriber with an APG
<StevenK> I'm trying to work out why that notify call is deciding to do nothing
<StevenK> And failing, I might add
<StevenK> wgrant: The bugchange ignores private branches. The notification code has to run before the branch is made private.
<wgrant> StevenK: Why is the branch private?
<StevenK> Because I was forcing it to be.
<wgrant> Ah
<wgrant> (and yes, it does ignore private branches -- I reported that leak a few years ago :))
<StevenK> wgrant: BTF isn't in branchscanner's security.cfg, too
<wgrant> StevenK: It probably inherits that
<wgrant> eg. from write or something
<wgrant> Yeah, from write
<StevenK> Ah. I think APG is fine, and we have a test that will blow up if that check changes.
<StevenK> Oh, sigh.
<StevenK> I bet the scanner has cursed this branch
 * StevenK stabs the branch scanner over and over.
<wgrant> I have mail
<wgrant> So I take it you uncursed it
<StevenK> No, rename and push again dance.
<wgrant> You do love to crush my hopes and dreams.
<StevenK> And putting a bloody knife in a Express Post and addressing it to celeryd@ackee
<StevenK> wgrant: Well, I do need a hobby ...
<wgrant> StevenK: Care to check that the bug actually gets linked?
<StevenK> That's probably a good idea.
<adeuring> good morning
* adeuring changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 4.0*10^2
<StevenK> wgrant: Done. Look again?
<wgrant> StevenK: r=me
<wgrant> thanks
<stub> wgrant: Do you think http://paste.ubuntu.com/1169485/ will work?
<stub> wgrant: I'm thinking this behaviour makes the improved FDT process much simpler. I just need to stop the slaves replaying WAL, disable master connections, apply patches to the master, enable master connections, disable slave connections, reenable replication, wait for sync, back to normal.
<stub> Which I think I can do without swapping pgbouncer config files around, which seems fragile.
<wgrant> stub: That was exactly the process I had in mind, but let me read the code...
<wgrant> stub: I think that would work
<wgrant> But we'd want to go a bit further eventually :)
<wgrant> A bit of refactoring to allow generic support for fallbacks would probably make it all a bit nicer
<stub> In what way? We can also cause master requests to get a slave, which is reinventing lp's read-only mode.
<wgrant> stub: Well, webapp and API requests always use the master
<wgrant> Erm
<wgrant> Webapp requests in recently-POSTed sessions
<wgrant> Because they want up to date data
<stub> But then we have a little risk with scripts, as we are deliberately returning a broken result.
<wgrant> But if the master's not available then they should fall back
<wgrant> xmlrpc-private also usually uses the master policy
<wgrant> Because it wants to be as up to date as possible
<stub> Yeah, so we can do that for all master requests, which would be a lie. Or make a master + fallback policy for them, and switch them to using that policy.
<wgrant> Right, I think we want a MasterIfYouCan policy which all those use
<wgrant> So the slave policy should always allow fallback to master
<wgrant> the masterifyoucan policy can always fall back to a slave
<wgrant> and the master policy just fails
<stub> Yes, slave falling back to master is documented as allowed.
<wgrant> Oh right, I think it even already does that
<wgrant> It must
<stub> I don't think we do that dynamically anywhere
<wgrant> So I think default_flavor = MAIN_FLAVOR becomes eg. flavours = [MAIN_FLAVOUR, SLAVE_FLAVOUR]
<wgrant> stub: I thought the slave policy respected lag
<wgrant> But I can't remember exactly.
<stub> oh yes
<stub> Only in the LaunchpadPolicy
<wgrant> Indeed
<wgrant> SlaveDatabasePolicy doesn't respect lag
<wgrant> That's probably a bad idea
<stub> We only choose the default based on lag.
<wgrant> Right, and only in LaunchpadDatabasePolicy
<stub> master requests still get a master if explicit, and slave requests still get a slave if explicit, no matter lag.
<wgrant> Oh hm
<wgrant> True
<wgrant> That sucks
<wgrant> So
<wgrant> I think most of dbpolicy.py wants a bit of a rethink
<wgrant> and
<wgrant> most importantly
<wgrant> a de-Americanisation
<wgrant> :)
<wgrant> Because there's not much reason to ever not respect lag
<wgrant> Lag should probably be treated as failure
<wgrant> Although not failed enough that it won't use it as a last resort
<stub> I think there is plenty of stuff that is happy using a slave even if it is an hour behind, and we raise alerts when things are 5 minutes behind
<wgrant> True
<wgrant> So yeah
<wgrant> xmlrpc
<wgrant> xmlrpc-private
<wgrant> recently-POSTed webapp
<wgrant> and API
<wgrant> probably all want to fall back to slaves
<wgrant> Webapp writes shouldn't
<wgrant> So we need a new policy
<stub> Try slave first, fallback to master ;)
<wgrant> Right, that's correct for webapp
<wgrant> But xmlrpc probably wants the opposite
<wgrant> Or at least a very low lag limit
<stub> I'm joking there.
<stub> I'd like to try to use the LaunchpadDatabasePolicy logic if there is a session cookie
<wgrant> Right, that logic is still good
<wgrant> Except that that should only influence the default
<stub> And I'm still annoyed nobody would put in a session token to the webapi, killing its scalability. But I suspect a lot of clients give us a cookie anyway.
<stub> I think the way forward is to try this with just slave fallback, which is the original ppa use case. Get the bugs ironed out on the production side before complicating things further.
<wgrant> I'm just worried that we're complicating things unnecessarily by adding a hacky single-case fallback
<stub> We only have 2 types of connections, 3 if you count 'DEFAULT'. We don't really need a generic framework.
<stub> Or do you mean this shouldn't be in the BaseDatabasePolicy?
<wgrant> Right, I don't think this belongs in BaseDatabasePolicy
<wgrant> I'm not quite sure where is better
<stub> I can put it in SlaveDatabasePolicy and SlaveOnlyDatabasePolicy
<wgrant> (also, am I missing something or do you try to reretrieve the same store there? you don't change the flavour)
<wgrant> Which means it'll just try to regrab the slave
<stub> typo
<stub> Not tried actually using this yet, just thinking through the idea :)
<wgrant> Heh
 * wgrant foods
<czajkowski> wgrant: stub you guys doing anything to LP right now? getting timeouts doing the licience review
<czajkowski> (Error ID: OOPS-e3753aed5cfe86fe227192e43be904c1)
<stub> nope
<czajkowski> hmm
<stub> Bah. Need to rethink this, again. DBPolicy will happily hand out stores from the ZStorm cache even if they won't work, and I don't want to test if connections work every time the policy is invoked.
<wgrant> stub: Hm
<wgrant> stub: Well
<wgrant> stub: It should be done the same way as the lag check, right?
<wgrant> I forget at what stage that is done, but whatever it is it's right at the start of the request
<wgrant> And for non-request-based stuff we probably just want to switch when a connection fails, maybe?
<wgrant> Although that makes it harder to fail back
<stub> We might not have a request
<stub> I think I can ask the store if it is in a disconnected state or not before handing it out. If it is disconnected, attempt reconnection
<wgrant> Yeah
<stub> So a script running during fdt will get disconnected and need to handle that. And when it handles it, it will get the master store if it asks for the slave.
<stub> Just need to wade through to see how to detect disconnected state, and to force a reconnection attempt.
<wgrant> stub: Yeah, we may just have to wait for a disconnectionerror to be raised, I suspect
<wgrant> And teach stuff to deal
* benji changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: benji | Firefighting: - | Critical bugs: 4.0*10^2
<deryck> abentley, adeuring, rick_h_ -- let's be ready for a longer stand-up today, to let rick_h_ lead us through the mockups he has.
<abentley> deryck: okay.
<rick_h_> party
<rick_h_> drink refill before the meeting, got it
<abentley> rick_h_: hey, don't party too hard :-)
<jam> jelmer: I added a card to track the translations stuff, but I don't have any specific insight into it.
<jam> My first guess is that there is an issue with a cron job that is supposed to be running.
<jam> jtv is someone you can poke for translations background, but he doesn't necessarily know more intimate details if it is an operational issue
<jam> wgrant is generally the person with the most ops ideas
<rick_h_> gary_poster: ping, hazmat ping'd me about looking over their YUI work on the juju js app/ui and deryck mentioned that since you guys were coming into that work should someone from your squad take part in the discussion
<rick_h_> bah
<gary_poster> rick_h_, hey thank you.  why the bah?  It would be great to be a part of it, I think, though I'll check with hazmat
<czajkowski> jam: jelmer jtv will be on later if that helps, in the mean time I'm going to put ana annoucement out on Twitter and places as we're getting more bugs/questions logged about it
<czajkowski> it's added to the topic as well in case people ask, that way you're not under as much pressure to find an answer
<deryck> abentley, ready for call?
<deryck> stand-up hangout is fine
<abentley> deryck: sure.
<czajkowski> hiya someone help me for a momen, bug https://bugs.launchpad.net/launchpad/+bug/1041864  I wanted to change the URL for the import, but when I do i get invalid as it's being used elsewhere and I've never seen that issue before :/
<_mup_> Bug #1041864: Badly named weston import <Launchpad itself:New> < https://launchpad.net/bugs/1041864 >
<maxb> czajkowski: Hm.. I don't think the user wants the import URL change
<maxb> +d
<maxb> Can someone look up OOPS-eb261d3e309c39d6948f60de23422af9 for me?
<rick_h_> maxb: loaded, the user isn't a member of the team
<czajkowski> rick_h_: you're faster than I was
<czajkowski> maxb: http://pastebin.ubuntu.com/1170166/
<maxb> Oh, right, this is because ~vcs-imports members has slightly weird hybrid edit permissions on code imports, I remember now
<jtv> czajkowski, jelmer: will be at least 8 more hours before I'm here â provided I get well enough.  What's the crisis?
<czajkowski> jtv: translation imports seem to have stopped
<czajkowski> jtv: https://bugs.launchpad.net/launchpad/+bug/1041858
<_mup_> Bug #1041858: No daily translation export anymore <Launchpad itself:Triaged> < https://launchpad.net/bugs/1041858 >
<jtv> Import or export?
<czajkowski> export sorry
<jtv> And it's not the normal exports, but the exports to branches, I see.
<jtv> Now, there's always a few exports that are skipped because the branches are locked, or there are concurrent translation updates that the exporter doesn't want to overwrite, etc.
<jtv> So there's a big difference between âseveral people haven't seen it workâ and âit's stopped.â  Do we know which it is?
<czajkowski> https://answers.launchpad.net/launchpad/+question/206912  https://answers.launchpad.net/launchpad/+question/206948
<czajkowski> jtv: I asked jelmer to look into it today
<czajkowski> not sure he made progress or what update he got with it
<jtv> If there's no breakthrough, chances are it's hard to diagnose â which most likely means a crash in a C-level library.  Might be worth finding out if the outage coincides with an upgrade.
<jtv> Hmm the log on crowberry simply stops on the 17th.  We haven't disabled the whole cron job by any chance?
<czajkowski> jtv: no clue :s
<czajkowski> jtv: but as soon as jelmer and jam come on tormorrow will get them to loook
<czajkowski> or ask wgrant in a bit when he arrives
<czajkowski> august 17th was when we did a lot of the DC move
<jtv> There's more log on taotie.
<jtv> I'll see if anything jumps out, but will leave it to the Dutch Cavalry otherwise then.
<czajkowski> thanks jtv
<jtv> czajkowski: I can give you my quick & vague impressionâ¦  The exporter writes to branches, which triggers branch-scan jobs (to make Launchpad notice the branch changes).  These seem to go into Celery now, via a RabbitMQ queue.  It looks as if the connection to RabbitMQ started breaking on the 18th (possibly from a change on the 17th, since this job runs in early morning UTC) and eventually on the 21st somebody may just have killed and disabled the
<jtv> Well, not disabled exactly.  Maybe the lock file from the aborted run is still there; I seem to remember that logging is a bit asymmetrical when it comes to those lock files.
<jtv> It may mean that that instance from the 21st is still hanging around trying to request a branch scan for Stellarium, and the subsequent runs are quietly giving up as they notice that.
<lifeless_> flacoste: o/ - just settling Cynthia a little, back soon
<flacoste> lifeless_: o/
<lifeless> flacoste: ok, ready when you are.
 * deryck heads home, back online shortly
* benji changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 4.0*10^2
<wallyworld> wgrant: StevenK: mumble?
<lifeless> http crackheads of the world, does bug 1040689 strike you as add?
<lifeless> s/add/odd/
<_mup_> Bug #1040689: add api to refresh an existing token <escalated> <Canonical SSO provider:In Progress by ricardokirkner> < https://launchpad.net/bugs/1040689 >
<StevenK> lifeless: Hello replay attack?
#launchpad-dev 2012-08-28
<StevenK> wallyworld: Oh good god, http://pastebin.ubuntu.com/1170963/ solves the test issues I brought up on the call.
<wallyworld> yay
<StevenK> And I was the one who added filter_visible ... :-(
<wallyworld> lol
<wallyworld> that sort of thing happens to me too
<StevenK> wallyworld, wgrant: https://code.launchpad.net/~stevenk/launchpad/drop-disclosure-feature-flags/+merge/121523
<wgrant> +271? :(
<wallyworld> what did you expect? it's a net loss of code
<wgrant> Well, normally I say "+foo? :(" then StevenK goes and reduces it to about 30% of that
<wgrant> I'm hoping it works here
<StevenK> Haha
<StevenK> Not this time
<StevenK> Far too much reflowing
<wgrant> :(
<wgrant> StevenK: I don't quite understand what's going on in bugsubscription.txt
<wgrant> Is there some broken unsubscription logic?
<wallyworld> mr ocr - https://code.launchpad.net/~wallyworld/launchpad/branch-infotype-portlet-1040999/+merge/121404 https://code.launchpad.net/~wallyworld/launchpad/branch-infotype-portlet2-1040999/+merge/121527
<wallyworld> add to the queue :-)
<wgrant> Sure
<wallyworld> thanks
<wallyworld> food time, me hungry
<wgrant> I think I'll do the same after StevenK's branch
<wallyworld> yeah, my branches will be in a huge bb queue anyway
<wgrant> Oh hm
<wgrant> No, that doesn't explain it
<StevenK> wgrant: I tried to explain it in the MP description.
<StevenK> wgrant: It seems the test subscribes mark, and then sets the bug to private, which boots mark off, and then the rest of test doesn't include him.
<StevenK> s/off, and/off, sets the bug back to public, and/
<wgrant> But why has that changed?
<StevenK> Because I removed the section in transitionToInformationType that ripped out the subscriptions if the job feature flag was disabled
<StevenK> Since RASJ will do it
<wgrant> StevenK: Right, but shouldn't he have a grant from the sub?
<wgrant> The subs were only ripped out if there was no grant
<StevenK> wgrant: He was subscribed when the bug was public -> no grant
<wgrant> Oh, right
<wgrant> Makes sense, then
<wgrant> But please do dispose of enhanced_picker fully
<StevenK> Eh? I missed something?
<wgrant> 69 @property
<wgrant> 70 def enhanced_picker(self):
<StevenK> Oh, that should get pulled in properly. Right.
<rick_h_> wallyworld: tossed a side note out to your MP
 * wallyworld looks
<wallyworld> rick_h_: agreed 100%, but this branch is more concerned with extending existing code to deliver what's required rather than fixing the inherit design issues
<wallyworld> now that the core banner stuff is out, it can be decoupled later
<rick_h_> wallyworld: ok, yea just gave it a quick glance and seemed to introduce some new stuff in the information type stuff for code.
<rick_h_> wallyworld: cool, ok
<wallyworld> yeah, that was to avoid code duplication with what was there already
<wallyworld> thanks for looking though
<rick_h_> with this can you use the BranchInformationTypeWidget without a portlet/banner?
<rick_h_> e.g. from the actual edit UI for a branch or bug? I guess you'd have to since I've seen the bug UI have it as a normal UI field in edit
<wallyworld> rick_h_: the banner etc is currently expected to be there since it makes direct calls to the various methods
<wallyworld> the portlet is expected to be there and that's what the widget is manipulating
<rick_h_> wallyworld: ok, I guess I'll be peeking at it soon enough. I was thinking of the case of during bug submission/etc
<wallyworld> it's not relevant during bug submission
<wallyworld> it's only for editng the info type via the bug/branch overview pages
<rick_h_> ok, so the widget for selecting information type during submission is a different widget? /me hasn't gotten down to all the code yet
<wallyworld> yes - the one during submission is just a popup field, the one on the overview pages is a portlet with extra info rendered
<wallyworld> extra info = description
<rick_h_> ah ok. I'll peek at it again in the morning then. Thanks.
<wallyworld> rick_h_: so with yui 3.5, i notice Y.Lang.substitute is replaced in some places by Y.Lang.sub
<rick_h_> yea, substitute is on the way to deprecation I think. Those will probably have to be updated.
<wallyworld> ah ok. thanks. i was really happy to see we are on 3.5 \o/
<rick_h_> there's discussion of a micro template feature added into 3.7 that would replace sub all together and be something less than handlebars, but more than sub
<wallyworld> and soon 3.6 maybe :-)
<wallyworld> cool, sounds good
<wgrant> Speaking of 3.5
<rick_h_> yea, I already had tests for 3.5.1 and 3.6 was new enough I didn't want to jump that far
<wgrant> I haven't seen any issues
<wgrant> Should we deploy it to everyone? :)
<rick_h_> cool, I'll open it up to beta tests later this week
<rick_h_> beta usres
<rick_h_> beta users...gah late here
<wallyworld> very late
<rick_h_> we have to wait for the RT on the squid proxy before we can enable the combo loader for all users and then get everyone on 3.5...hopefully in a month
<wgrant> Oh, we can't do non-comboloaded 3.5?
<rick_h_> no, we can't
<wgrant> :(
<rick_h_> well, not without a lot of extra work
<rick_h_> and we're one RT from combo loader for all so holding onto that and then it's a non-issue any more
<rick_h_> should make the upgrade to 3.6 a very easy process
<wgrant> Yep
<rick_h_> the only big missing part after that is some way of running our JS tests on each version we have running
<wallyworld> what's the rt for? caching?
<rick_h_> yea, combo loader wsgi traffic caused a 1-2% cpu spike on the app servers it's running on.
<rick_h_> since the urls never change (they're unique per revid) they can be 100% cached in squid
<wallyworld> right, sounds logical
<rick_h_> and should cause no real load for us at that point
<rick_h_> got an email that someone was oging to work on it after the DC move, but we know how much fnu that's been :)
<wgrant> wallyworld: Oh right, we can use set literals now, can't we...
<wallyworld> yeah
<wallyworld> i just did it as a drive by
<wgrant> wallyworld: Why do we use a custom form for this widget, rather than the API?
<wallyworld> so i can re use all the existing back end logic
<wallyworld> to keep html and ajax back end code the same
<wallyworld> and for bugs, the form returns the subscribers list. we will ultimately need to do that for branches too
<wgrant> The only thing that should be interesting to share is getInformationTypesToShow, right?
<wgrant> Still, there has to be a better solution to that
<wallyworld> maybe. code reuse is good :-)
<wgrant> Yes
<wgrant> But doing it through a custom form isn't :)
<wallyworld> why?
<wallyworld> it is a form submission type operation
<wgrant> I think this is the only case where we don't use the API
<wgrant> Everything else does
<wgrant> Even milestone creation
<wallyworld> and we get to use all the existing validation hooks, error handling etc
<wgrant> Milestone creation loads a form from the server, but then submits via the API
<wgrant> wallyworld: Sure, but we need the validation hooks and error handling through the API too
<wgrant> What does the form need that the API doesn't?
<wallyworld> bug privacy also works this way
<wgrant> Sure
<wgrant> That's the one exception
<wgrant> In all of Launchpad
<wgrant> To the rule
<wgrant> I understand that you're cloning bug privacy
<wgrant> But I think bug privacy is very wrong
<wallyworld> i disagree :-)
<wgrant> Everything else manages to use the API
<wgrant> The only possibly valid argument is the custom return value
<wgrant> Validation/errorhandling are not valid arguments, since the API needs those too
<wallyworld> the form infrastructure is what's being used here - why does the transport matter?
<wgrant> If the API isn't sufficient, we probably have a problem with our API
<wgrant> And it'd be nice to understand what that problem is :)
<wallyworld> does the api have declarative validation hooks and all the rest of the stuff that the form submisison stuff relies on?
<wgrant> We expose this exact transition as an API method
<wgrant> If that API method doesn't do the necessary validation, we have a serious bug
<wallyworld> bugs checks that a bug is being made invisible, and there's a workflow there to manage that with the user
<wallyworld> you can't really do that via the api in the same way
<wgrant> Why not?
<wallyworld> so you need some front end to handle that
<wgrant> You have the current bug state in the JS
<wgrant> You know the new state because the user clicked on it
<wallyworld> yes, but the api has to communicate the potential issue back to the caller and then process their response
<wgrant> Well, the API can either do that by accepting a keyword arg for confirmation
<wgrant> Or, and this would be my preference, the web UI does the check before it gets to the API
<wallyworld> 2 xhr calls = yuck
<wgrant> Huh?
<wgrant> How's that different from now?
<wallyworld> well one to check,one to do the work
<wgrant> You don't necessarily need one for the check
<wallyworld> now, it optimistically assumes it will succeed
<wgrant> Since you have the old and new information_types
<wgrant> On the client
<wallyworld> the client can't make the decision
<wgrant> Why not?
<wallyworld> it doesn't have all the information
<wallyworld> a back end sharing service api call is required
<wgrant> Oh, right, we only warn if there's no APG?
<wgrant> Forgot that detail
<wgrant> A way to do that with one API call is to have an even_if_there_is_no_apg=False kwarg
<wgrant> So the normal case would be a single call
<wgrant> On failure, retry setting that to True
<wallyworld> at the end of the day, both ajax widget and html form are the same thing - submit of data to be processed, so should be handled consistently by the back end
<wgrant> But the API needs that too
<wallyworld> using common code
<wgrant> So the validation code has to be in the model
<wgrant> Not the view
<wallyworld> here it's no so much validation as in a check that's only relevant to the ui at the monent
<wallyworld> and the check is done using model code
<wgrant> Right, this is an interesting special case
<wgrant> But I don't think it's special enough to warrant not using the API, if this wasn't just a clone of the Bugs code
<wallyworld> the view layer is a logical thing to use to solve any impedance mismatch between web and model api
<wgrant> Probably, yes
<wgrant> But not as the application is architected at present
<wallyworld> especially with html forms for the exact same editing operation is use
<wallyworld> we want to keep the code common
<wgrant> Right, we need a way to do that
<wgrant> But currently we don't do it in many places
<wallyworld> if no html form was there, then i'd have used the api since all the view code would be in javascript
<wgrant> And when we need to, it involves adding self.request.is_ajax special-cases everywhere
<wallyworld> the is_ajax is just used to determine what to return from the actions
<wgrant> Sure
<wgrant> And the fact we have to repeatedly hack this in is a sign that we're Doing it Wrongâ¢
<wallyworld> ideally there would be no html forms, just ajax :-)
<wgrant> Indeed
<wallyworld> and then the issue would be moot
<wallyworld> wgrant: bug 1040989 "For commercial projects we can choose Public or Proprietary as the default...." - this seems to imply just to check for OTHER_PROPRIETARY when registering a project, but surely a commercial subscription is also required to allow proprietary sharing policies?
<_mup_> Bug #1040989: new project have legacy sharing policies <disclosure> <sharing> <Launchpad itself:In Progress by wallyworld> < https://launchpad.net/bugs/1040989 >
<wgrant> wallyworld: Projects with Other/Proprietary get a complimentary 30 day commercial subscription.
<wgrant> I'm not sure at which stage this is applied
<wallyworld> ah ok,thanks
<wallyworld> i'll look into it
<wgrant> Personally I think we might be best to just always default to Public/Public for now
<wgrant> But if it looks easy, I guess the Proprietary defaults couldn't hurt
<wallyworld> i'm doing it because the card was put into the next lane as an item to do to finalise stuff for the beta
<wallyworld> so i assume it's required
<wgrant> The really important bit is that it not default to legacy any more
<wgrant> Since we want to be migrating existing projects away from legacy without creating new ones
<wallyworld> yes, true
<wgrant> We can't make the columns NOT NULL if new projects have them null
<wgrant> That's the main thing
<wgrant> StevenK: Hah
<wgrant> StevenK: I just tripped over the makeBug(product=) thing myself
<StevenK> Hahaha
<wallyworld> wgrant: another one to add to your queue, sorry  https://code.launchpad.net/~wallyworld/launchpad/new-project-sharing-policies-1040989/+merge/121532
<wgrant> Oh, that was easier than expected
<wallyworld> hope i haven't missed anything
<wgrant> Nope, r=me
<wgrant> Thanks
<wallyworld> coo :-)
<wallyworld> cool
<wallyworld> luckily createProduct calles setLicence which does the subscription thing
<wgrant> Yup
<wgrant> Not quite sure how we're going to do the not-null thing
<wgrant> But I guess whoever does that can work it out :)
<StevenK> wgrant: Your comment on the MP pre-dates our discussion about bugsubscription.txt?
<wgrant> StevenK: It was between me asking and you responding, so yes, that's no longer a concern
<StevenK> wallyworld: http://pastebin.ubuntu.com/1171109/
<StevenK> wallyworld: Opinions, have I missed something/looks okay?
 * wallyworld looks
<StevenK> wallyworld, wgrant: Now the deploy is done, any reason I can't toss my branch that ALTER TABLE product/distribution DROP COLUMN security_contact at ec2 to land?
<wgrant> StevenK: db-devel has enough stuff in it already
<wgrant> There's already two things, AFAIK
<wgrant> Let's not add more
<wallyworld> StevenK:  99  -  show_create_team=self.enhanced_picker)
<wallyworld> should be            show_create_team=self.show_create_team)
<StevenK> Hmmm, I think show_create_team_link in that class can die too
<StevenK> Mmm, maybe not
<wallyworld> you need to keep the show_create_team boolean
<wallyworld> StevenK: also, in test_vocabulary, you can delete tests
<wallyworld> the tests without enhanced_picker=True which have matching =True versions can go
<wallyworld> and maybe rename the tests to remove _enhanced_ fromthe name
<wallyworld> looks ok otherwise, but do run it up locally and test project creation and editing of owwner/driver
<StevenK> wallyworld: Not sure which tests can die, but http://pastebin.ubuntu.com/1171125/
<wallyworld> StevenK: i'm not sure either, ignore that dumb comment of mine. i think it looks ok
<wallyworld> wgrant: distributions don't have bug/branch sharing policies yet, right?
<StevenK> wallyworld: I just tried New Team. It looks *awesome*
<wallyworld> StevenK: oh, cool, ta :-)
<StevenK> wallyworld: It needed to be 'show_create_team=self.show_create_team_link' anyway
<wallyworld> ah, right, sorry. i misrembered
<StevenK> The tests picked it up
<wallyworld> so the create team link should only be there for registering new projects and editing owners/drivers
<wallyworld> yay for tests \o/
<StevenK> wallyworld: Yeah, I tried assigning a bug and there was no create team link
<wallyworld> good!
<StevenK> Rargh
<wallyworld> oh no, what?
<StevenK> Damn it, auditor, stop failing on db-devel :-(
<StevenK> Now wallyworld will yell at me again :-(
<wallyworld> i didn't yell :-)
<wallyworld> just asked politely
<StevenK> I know, I'm teasing
<wallyworld> irc is so devoid of emotional clues
<StevenK> Hold on, I'll get in the car and drive 12 hours. Will that help?
<wallyworld> ok, and bring cake
<StevenK> You don't deserve any, due to being a large cake tease with spm.
<wallyworld> i actually made one and sent him a picture :-D
<wallyworld> it was sooo yummy too
<StevenK> Yes, but he didn't get to eat any!
<wallyworld> i know! hence the :-D
<wallyworld> i was trolling him
<StevenK> And then devel fails with test_getAllPermissions_constant_query_count
<StevenK> Seriously
<StevenK> This is why we can't have nice things.
<wallyworld> i think the query count test was added relatively recently
<StevenK> wgrant was talking about it yesterday
<wallyworld> yeah, it's all soyuz fault
<StevenK> wgrant: Terribly hard review up for you. https://code.launchpad.net/~stevenk/launchpad/clean-up-ibranch/+merge/121536
<StevenK> Oooh, Postgres 9.2 RC1
<stub> yeah, yeah.
<StevenK> stub: Not a dig at you, just reading the blog post
<stub> haha, yeah. Just don't get too excited :)
<stub> There is some cool stuff in there though, such as statement profiling that might actually work for us. Nice to have pg spit out our slowest common queries.
<StevenK> 'Fix bugs in pg_trgm' looks relevant too
<StevenK> wallyworld: Since wgrant appears to have disappeared without a trace, can you review https://code.launchpad.net/~stevenk/launchpad/clean-up-ibranch/+merge/121536 ? A few tears would be okay, too, since it's the last mention of transitively_private in model code.
<wallyworld> StevenK: sorry, was out buying dinner, looking now
<wallyworld> StevenK: there's still a update_transitively_private Postgres function? Which updates Branch table transitively_private column.  And there's also a security.cfg entry for said function.
<wallyworld> i guess these will be killed next
<wallyworld> r=me
<StevenK> I thought that was dead
<StevenK> ALTER TABLE Branch DROP COLUMN transitively_private;
<StevenK> DROP TRIGGER maintain_branch_transitive_privacy_t ON Branch;
<StevenK> DROP FUNCTION maintain_transitively_private();
<StevenK> In 16-6
<StevenK> steven@undermined:~/launchpad/lp-branches/clean-up-ibranch% grep -c maintain_transitively_private database/schema/security.cfg
<StevenK> 0
<wallyworld> StevenK: in security.cfg, look for public.update_transitively_private(integer, integer, boolean) =
<StevenK> With no perms
<StevenK> But yeah
<adeuring> good morning
* frankban changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: frankban | Firefighting: - | Critical bugs: 4.0*10^2
<jam> mgz: morning, /wave jelmer
<jelmer> hello mgz, jam
<jam> I'm technically off today, but I'd like to chat with you guys on mumble if you're available.
<mgz> I'll hop on
* rick_h_ changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: frankban, rick_h | Firefighting: - | Critical bugs: 4.0*10^2
<mgz> jelmer: so, I take it this is not the intended plan with the lucid/py27 ppa: <http://pastebin.ubuntu.com/1171529/>
<jelmer> mgz: hehe, no
<mgz> it also needs the normal launchpad ppas, and to upgrade everything?
<mgz> or should just leave those packages alone?
<jelmer> mgz: it should leave those packages alone
<jelmer> mgz: are you looking at the bzr 2.5.1 upgrade, or should I ?
<mgz> I can do that.
<jelmer> mgz: either works for me; I'm a bit stuck with the python upgrade, the bzr 2.5.1 upgrade seems more straightforward
<maxb> Which PPA is this? https://launchpad.net/~canonical-bazaar/+archive/py27/+packages ? If so, I'd be quite surprised if you were able to get away with versioning python-defaults as 2.7anything, rather than 2.6something
 * maxb has battle scars from backporting Python 2.7 to Debian etch.
<maxb> Yes, etc.
<maxb> h
<jelmer> maxb: that sounds painful
<maxb> It was... interesting :-)
<jelmer> mgz: why is that?
 * mgz reads maxb's mind
<stub> So why aren't we just switching to Precise? The machines are all due for that upgrade anyway.
<mgz> stub: this is what I keep asking.
<mgz> it's meant to be 'easier' and 'safer' to do all this crazy backport stuff with packages no one else has used or needs
<mgz> but seems like it's just introducing yet another environment for things to potentially fail in.
<stub> I don't see how using untested bespoke packages is considered safer than the versions all the devs have been testing for months.
<mgz> indeed.
<stub> Especially if there do turn out to be problems, the solution would be to upgrade to precise rather than stick with 2.6 forever.
<maxb> On a similar topic, why not precise/py26 rather than lucid/py27?
<lifeless> we know that we're broken on 2.7 atm AFAIK.
<lifeless> calling it tested for months is a massive exaggeration.
<lifeless> maxb: because its a lot easier to rollback lucid/py27.
<mgz> broken in non-testsuite ways? I'm reasonably sure I got all the test failures.
<lifeless> yes
<maxb> lifeless: Fair enough - on the other hand, it's probably a lot easier to create a viable precise/py26 PPA than a lucid/py27 one
<lifeless> maxb: we depend on very few system packages
<maxb> hmm, ok, I was just looking at canonical-bazaar/py27 and noticing an awful lot of red Xes
<stub> If we are broken under 2.7, we are broken under 2.7. At least with Precise we know it is broken with the real environment, not a bespoke environment.
 * lifeless shrugs
<lifeless> we had a disasterous time last time we did a lock step upgrade.
<stub> But we are switching to 2.7 plus all our packages, which is pretty much a lock step upgrade. Except we are planning on doing it twice.
<lifeless> stub: how do you figure that ?
<stub> Once to a version of 2.7 nobody except staging and buildbot will be using, plus lucid debs. Then again to Precise 2.7.
<lifeless> many devs develop on lucid in lxc; so nobody is an exaggeration. But sure.
<lifeless> There are two separate transitions.
<lifeless> There were three.
<lifeless> postgresql
<lifeless> python
<lifeless> base os
<lifeless> we did the same strategy for postgresql.
<lifeless> and the associated libraries
<lifeless> and we had small issues and friction
<stub> postgresql is unrelated, as that is server migration.
<lifeless> its entirely related; we had to deal with it in buildbot, we had to upgrade libpq because of (I forget the exact issue)
<stub> We *want* things to fail under buildbot and staging. What we are worried about is production failures, and there is unrelated.
<lifeless> if it was unrelated, we wouldn't have had to upgrade packages on all the production machines :)
<stub> It is unrelated because the DB servers are due for upgrade to Precise as soon as webops have bandwidth. It isn't blocked on the clients.
<lifeless> they are *now*
<lifeless> I'm not talking about precise on those servers
<lifeless> I'm talking about 8.4->9.1
<lifeless> which was part of the overall upgrade-to-precise plan
<adeuring> stub: thanks for the review
<wgrant> stub: FYI in 9.1 we can now add constraints without too much downtime
<wgrant> Using NOT VALID
<wgrant> Then ALTER TABLE ... VALIDATE CONSTRAINT
<wgrant> That may influence your opinion on adeuring's MP
<stub> Oh, hadn't seen that. Ta.
<lifeless> shiny
<stub> Its only 100k rows to scan
<wgrant> True
<wgrant> But yeah
<wgrant> that was a relatively late addition to 9.1
<wgrant> It was the last common really expensive operation
<wgrant> that is now doable without major downtime
<wgrant> makes me very happy
<lifeless> 100k rows is probably too many
<wgrant> Yeah
<wgrant> Well
<wgrant> It's only validation
<wgrant> And it's pretty simple
<wgrant> So it might be OK
<wgrant> But I think you're still probably looking at >5s for that
<wgrant> But I haven't thought that through properly
<lifeless> I'm fairly sure we would be
<lifeless> but data will tell
<wgrant> I'd sort of expect 5-10s for a single node there
<wgrant> but that's mostly a gut feeling
<lifeless> I see progress from stub on rolling schema patches.
<wgrant> oh?
<lifeless> which will be awesome
<wgrant> oh
<wgrant> you mean nodowntimefastdowntime?
<lifeless> nearlynodowntimefastdowntime
<wgrant> right
<wgrant> noreaddowntime
<wgrant> yeah
<wgrant> we discussed that last night
<lifeless> great.
<wgrant> i saw a branch
<wgrant> looks ok
<lifeless> I should spend more time not looking in the channel, you guys get up to good stuff :)
<wgrant> it will be marvellous :)
<lifeless> stub: sorry I didn't catch up w/ you last week, was -hectic-.
<lifeless> stub: this week may be too, for different reasons.
<StevenK> And in Sydney
<lifeless> stub: if you want to catch up, wed is probably the day, or we can raincheck it - whatever you prefer
<lifeless> *yawn*, ciao.
<stub> lifeless: whatever :)
<stub> see how we go tomorrow
<wgrant> Night lifeless
<stub> and from above, just about to push the tests.
<wgrant> stub: yeah, i'm glad we have pgbouncer fixtures working so we can actually test that excellently
<wgrant> this is not the sort of stuff we want untested
<czajkowski> mgz: you about?
<mgz> yup
<mgz> will need some lunch shortly though.
<czajkowski> mgz: can you look at https://bugs.launchpad.net/launchpad/+bug/1042351  when you're free please
<_mup_> Bug #1042351: Bzr import fails with NotImplementedError in broken_remote_exceptions <Launchpad itself:New> < https://launchpad.net/bugs/1042351 >
<czajkowski> else I'll poke jelmer when he gets back
<mgz> it's an old format, which may upset things, otherwise need to look at logs/code for more info
<rick_h_> abentley: let me know if there's anything I can do to help ease the JS pain in that stuff.
<abentley> rick_h_: Thanks.
<jcsackett> anyone know how up to date the data on staging is?
<abentley> rick_h_: Could you please review this? https://code.launchpad.net/~abentley/launchpad/blueprint-info-type-code/+merge/121683
<rick_h_> abentley: sure thing
<rick_h_> abentley: is the diff off or are the other changed files just part of a merge conflict resolve business?
<abentley> rick_h_: The changed files listed in the lint?
<rick_h_> yea, there's many more files as 'changed' but only the one small diff in specification.py
<rick_h_> the patch, garbo, etc
<abentley> rick_h_: The lint is stale.
<abentley> rick_h_: It's from before abel duplicated my work.
<rick_h_> ok cool, just wanted to make sure I wasn't missing things
<rick_h_> abentley: ok r=me
* rick_h_ changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 4.0*10^2
<lifeless> o/
<jelmer_> g'morning lifeless
<abentley> rick_h_: I think I've got an improvement on the JS_YUI rule: http://pastebin.ubuntu.com/1172484/
<abentley> lifeless: Hi.  Can you help me with the DB patch for indexing Specification.information_type ?
<lifeless> abentley: sure thing, how can I help?
<abentley> lifeless: First thing, I didn't test it on qastaging.  Is that essential?
<lifeless> the risk of not testing it is that it triggers poor queries for the table
<lifeless> we wouldn't expect a hot patch to fail to apply (as long as its not adding uniqueness that is)
<abentley> lifeless: Since this column is not used anywhere, I don't think there's a risk of that.
<lifeless> abentley: its a single column index?
<abentley> lifeless: Yes.
<lifeless> I agree.
<abentley> lifeless: It is r15850.  I can put up a deploy request.
<lifeless> hot patches are still done by stub
<lifeless> so the best thing is to email him and ask for it to be applied
<abentley> lifeless: it says TA does them here: https://wiki.canonical.com/Launchpad/PolicyandProcess/ProductionChange
<lifeless> I can if it's urgent, but the production environment right now is in a great deal of flux
<lifeless> we're running on temporary servers, have just migrated off of slony etc.
<lifeless> I'd rather avoid fragmenting the state about where we are at by applying it myself, until we're back to normal.
<abentley> lifeless: Okay, just wanted you to know.  If the info's incorrect, we should change it, but it sounds like it's just temporary.
<lifeless> Getting stuart to do it will add ~8 hours delay - he's asleep at the moment.
<lifeless> It is,.
<lifeless> abentley: so, you'll email stub? Or would you like me to?
<abentley> lifeless: I've emailed him.  Thanks.
<lifeless> Cool, de nada.
<lifeless> http://lipka.github.com/piecon
<wgrant> lifeless: Can we convince ops to do hot patches now that slony's gone, or are we still blocking on them not have to transform them to CONCURRENTLY manually?
<lifeless> wgrant: its a good question, I was considering this earlier.
<lifeless> There were two complexities - per-node + concurrently transform.
<lifeless> We've halved that.
<wgrant> Right
<lifeless> I would like to switch us to lazr-postgresql's patch applier now.
<lifeless> Which understands this more or less.
<lifeless> So it wouldn't need special handling.
<lifeless> That would make the ops discussion real easy.
<wgrant> ops certainly seem to be a lot more comfortable with manual stuff like grants now that it only has to be done on a single node
<lifeless> just needs it glued into the FDT scripts.
#launchpad-dev 2012-08-29
<wgrant> StevenK: wallyworld mentioned on the call that you probably forgot to remove the feature flags from sampledata
<StevenK> Drag
<StevenK> *Drat
<wallyworld_> wgrant: with the change to give all new projects default sharing policies, setPrivateBugs() become irrelevant and broken. so i think i should remove it
<wallyworld_> or wil that break apport?
<wgrant> wallyworld_: Things that test private_bugs behaviour (which is still important, not for apport but for all unmigrated projects) will have to setBugSharingPolicy(None) first
<wallyworld_> there's a garbo job which has migrated all projects except a few commercial ones
<wgrant> private_bugs/setPrivateBugs will die with BVP
<wgrant> Sure
<wgrant> And the commercial ones are the ones that use private_bugs :)
<wallyworld_> hmm. ok. will set to None
<wgrant> If it was killable already, it would be dead already :)
<wgrant> But sadly not quite yet
<wallyworld_> yeah, me got a lot of test failures related to all this to fix :-(
<wgrant> wallyworld_: Not tooo many things should use setPrivateBugs
<wallyworld_> it's more than just that
<wgrant> wallyworld_: Most will probably be fixed by fixing makeProduct, I suspect
<wallyworld_> bvp stuff fails too etc
<wgrant> Ah, true
<wgrant> StevenK: Hm, why does lib/deb822.py still exist?
<wgrant> It's a broken symlink now
<wgrant> Did you miss it in your cleanup?
<StevenK> Hmm
<StevenK> Seems I did, indeed
<StevenK> Sigh, I bet https://code.launchpad.net/~stevenk/launchpad/feature-flags-sampledata is cursed.
<wgrant> [2012-08-29 02:03:46,105: INFO/PoolWorker-2] Job resulted in OOPS: OOPS-f59d1eb05281222ebbe8ce921e02b2d8
<wgrant> d
<wgrant> cursed indeex
<wgrant> StevenK: Well
<wgrant> StevenK: I was going to say that was a trivial self-review
<wgrant> But you made a mistake :P
<StevenK> :-(
<StevenK> I was like "Green? There's not supposed to be any green in this branch ... Oh."
<wgrant> Ah
<wgrant> I see the next rev
<wgrant> heh
<StevenK> That's better, +0, -3
<StevenK> wgrant: So now you've made fun of me for it, are you going to review it, or shall I self-review?
<wgrant> That's a self-review if I ever saw one
<lifeless> jam: what tz are you now?
<bigjools> lifeless: he's +4
<lifeless> noice
<lifeless> halfway to a good place  :)
<jam> lifeless: +4
<jam> morning, btw
<wallyworld_> StevenK: auditor broke db_devel again :-( i need to land a branch to devel but have to wait till db devel is restarted with a testfix or completes fully?
<wgrant> wallyworld_: Force, wait 10 minutes, land
<wgrant> No need for a testfix or a green run
<wallyworld_> ok, will do, ta
<StevenK> wallyworld_: The new auditor wasn't in that db-devel run
<wallyworld_> ah ok, explains it.
<wallyworld_> were you going to land a testfix for it?
<StevenK> devel finally passed, so it will be in the next build after buildbot-poller actually merges in stable.
<wallyworld_> which will be when do we know?
<StevenK> PQM probably rejected it due to db-devel being broken
<StevenK> Force a build of db-devel
<wallyworld_> ok
<StevenK> Bleh, new auditorfixture
<StevenK> r15871 has hit stable at least.
<StevenK> Now to wait for the 8 revisions to be merged into db-devel
<StevenK> Hmmmmm
<jam> bigjools: /wave for being online
<bigjools> hi jam
<bigjools> gimme 5 mins and I'll grab you
<StevenK> bigjools: Hahaha. Grab him where?
<lifeless> jam: o/ :)
<StevenK> Come *ON* buildbot-poller
<lifeless> jam: its probably faster to do remote hands (tuolumne)
<bigjools> jam: ok, hangout?
<jam> bigjools: sure on the hangout, let me grab the camera
<StevenK> RARGH
<StevenK> buildbot-poller only merges if both builds worked
<wgrant> Yes
<jam> bigjools: https://plus.google.com/hangouts/_/70125daf3bc2bab87f46d925ed9e5fc229cdf07e?authuser=0&hl=en#
<wallyworld_> StevenK: so pqm is ignoring my lp-land and not even sending back hate mail. is that expected?
<StevenK> Haven't seen anything in the pqm logs
<StevenK> But archvsync is still chewing on my request
<wallyworld_> but lp-land should be expected to work right now i think?
<wgrant> yes
<wgrant> Which branch?
<wgrant> new-project-sharing-policies-1040989
<wgrant> ?
<wallyworld_> no, branch-infotype-portlet2-1040999
<wgrant> Hm
<wgrant> No conflicts
<wgrant> How many times have you tried to land it?
<wallyworld_> 3 maybe
<wgrant> :(
<wgrant> One more time for good luck?
<wallyworld_> once when it complained about a testfix, 2 since after forcing bb
<wallyworld_> ok, here goes....
<StevenK> How long after forcing buildbot?
<wgrant> It should have been fine after the first */10 after the forcing
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/shift-get_distroseries_pofiles/+merge/121751
<wallyworld_> wgrant: yeah, i waited maybe 20 minutes
<bigjools> jam: you're frozen
<wgrant> wallyworld_: :(
<wallyworld_> wgrant: and lestest attempt, nothing
<wallyworld_> i gotta go get the kid, back soon
<wgrant> wgrant@carob:~$ rsync prasÃ©::
<wgrant> rsync: getaddrinfo: pras\#303\#251 873: Name or service not known
<wgrant> fail
<wgrant> wallyworld_: Hm, it's either hung or a wallyworld-denier
<wgrant> I see nothing in its logs from you in more than 50 minutes
<wgrant> wallyworld_: Ah
<wgrant> wallyworld_: Landed just now
<StevenK> Ah ha! It was waiting for wallyworld_ to be AFK.
<StevenK> wgrant: No review for me? :-(
<wgrant> StevenK: Where's the cleanup?
<wgrant> Adding more methods to DistroSeries is probably not removing tech debt.
<StevenK> wgrant: Destroying half of a translations module doesn't do it for you?
<StevenK> wgrant: I'd argue that they probably belonged their to begin with
<wallyworld_> well, next time something fails, i'm going away again and it wil work
<wgrant> StevenK: Adding app-specific stuff to registry modules when it can be avoided makes me a very sad panda
<StevenK> Obviously, -60 doesn't make you happy either
<wgrant> model/distroseries.py is already 1800 lines long
<wgrant> -60 is nice
<wgrant> But bloating (I)DistroSeries is less so
<StevenK> wgrant: Then where?
<wgrant> IVPOExportSet? :P
<StevenK> Bleh
 * StevenK deletes the MP
<wgrant> Code reduction is nice, but Distribution/DistroSeries/Archive/Person are already huge
<wgrant> We need a better solution than just adding every method in the application to them
<lifeless> stop using subclassing ?
<wgrant> lazr.restful
<jam1> bigjools: it seems to be fairly specific to g+, do you want to try the hangout without video, or maybe mumble?
<wallyworld_> StevenK: wgrant: did you guys find any more email addresses on dogfood?
<bigjools> jam1: looks like your whole internets given the disconnection from irc!
<jam> bigjools: well, hangouts triggers the disconnect.
<bigjools> I don't have mumble set up on here.  I think we're pretty much done for now, perhaps you can try out the code and that'll generate more questions
<bigjools> yeah I guess they don't like all the bandwidth it seems to eat
<wgrant> wallyworld_: Wasn't there only one missing?
<wallyworld_> wgrant: not sure. i thought there were a few but could be wrong
<adeuring> good morning
<jelmer> mgz: done now
<mgz> ta.
<bigjools> Given a base url like http://host/path and an additional path/to/thing, what would you use in Python to join them? Yes, this is a trick question.
<wgrant> bigjools: urlparse.urljoin, as long as the first URL has a trailing /
<bigjools> bzzzzzzzzzt
<bigjools> wrong
<wgrant> Oh?
<bigjools> >>> urljoin("http://10.0.0.9/MAAS/", "/api/thing")
<bigjools> 'http://10.0.0.9/api/thing'
<wgrant> Well, yes, if the path is absolute :)
<bigjools> this is the trick part :)
<wgrant> But yours wasn't
<nigelb> wait, what is the trick part?
<bigjools> aha stripping the leading / from the path works
<wgrant> >>> os.path.join('where/did/this/go', '/hello/there')
<wgrant> '/hello/there'
<bigjools> nigelb: the trick is that if you have a leading slash on the path part it doesn't work
<wgrant> It trips everyone up, but it is consistent with other path handling and sort of makes sense.
<nigelb> oh wow. I totally didn't notice that :)
<bigjools> it's a nasty trap for URLs
<bigjools> makes total sense for is.path though
<bigjools> os.path, even
<adeuring> what does this buildbot failure http://lpbuildbot.canonical.com/builders/lucid_db_lp/builds/2252 mean?
<wgrant> adeuring: There's a new version of auditorfixture that will hopefully fix that spurious failure
<adeuring> wgrant: thanks
<wgrant> It should be in the next db-devel build
<mgz> how do I unbreak my copy of launchpad today? <http://pastebin.ubuntu.com/1173583/>
<wgrant> bin/buildout
<wgrant> mgz: ^^
<wgrant> Or just make
<mgz> ta.
<mgz> so, basically I need to run make after bzr switch before running bin/test? I will get used to this in the end...
<wgrant> If you're changing between trees of significantly different versions, yes.
<wgrant> You do need to rebuild :)
<mgz> too used to either python, where you don't, or C, where the way you run tests builds as a side-effect anyway
<wgrant> You do in Python unless you have modules installed system-wide.
<wgrant> Which is becoming a rarity
<wgrant> buildout and pip have this same problem
<wgrant> and virtualenv
<jam> mgz: any luck with the ppa updates?
<mgz> jam: no, we need some major packaging changes
<mgz> see the log from yesterday morning, maxb suggested a few things
<jam> mgz: "we need major changes" means we should bring up the idea of just going to P w/ flacoste
<mgz> jam: I think so, yes, but lifeless wasn't keen
<jam> mgz: I'm not specifically keen, but if it is 'spend another 2 weeks sorting out py2.7 on lucid' vs, 'slowly migrate one appserver at a time to P' where the latter is where we want to be anyway...
<jam> mgz: I really like the idea of every step being something we can easily roll back, but remember my "installing py2.7 seems tractable"
<jam> if that part is invalidated
<mgz> okay, lib/lp_sitecustomize.py is what made this so confusing
<mgz> ...why is all that work being done pre-normal imports?
<jam> mgz: bin/py does not load 'site.py' the regular python site package, instead it uses a custom site.py so that it *only* loads eggs from the buildout location.
<jam> to promote isolation from the system.
<mgz> sure, but that's site.py
<mgz> this sitecustomize.py stuff is all just working around zope being weird and (failing to) silence various loggers
<mgz> and importing a barrel of cruft in the process
<mgz> deliberately importing bzrlib plugins before script execution seems to serve no purpose, and breaks our api version check stuff
<Beret> wgrant, just an FYI, the landscape project wasn't changed to proprietary by SQL
<Beret> wgrant, I did it yesterday, so I think we hit a bug
<Beret> (I just read the backlog of your conversation with fcorrea)
<wgrant> Beret: Around what time did you do it, do you recall?
<wgrant> 'cause when I set it away and back to the same setting, it worked fine.
<wgrant> Hmmm
<Beret> I would guess around 22:00 GMT
<wgrant> Thanks
<Beret> I'm not certain I just knowit was later in the day
<Beret> after the email went out
<wgrant> Beret: Ah, found the bug
<wgrant> A cleanup job is being a tad aggressive
<Beret> sweet
<wgrant> Only affects the initial setup, before there are any new bugs or branches
<deryck> abentley, ping for standup
<abentley> deryck: Sorry, having trouble with it on this machine.  Can you please invite me and I'll answer on my phone?
<deryck> abentley, sure
<rick_h_> abentley: that makefile update looks good from here. Thanks for that. Did you want to submit that?
* abentley changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: abentley | Firefighting: - | Critical bugs: 4.0*10^2
<wgrant> abentley: Hi, do you have a few minutes to review https://code.launchpad.net/~wgrant/launchpad/bug-1043319/+merge/121874 shortly? It's a pretty simple regression fix for the sharing migration that I'd like to get landed tonight.
<abentley> wgrant: certainly.
<wgrant> Bah
<wgrant> Curtis beat you to it
<wgrant> Even though he's not here
<wgrant> this week
<wgrant> Bad sinzui
<wgrant> Thanks anyway :)
<abentley> wgrant: np
<abentley> rick_h_: Sure, I'll get that in.
<abentley> rick_h_: Whenever you're ready for a review: https://code.launchpad.net/~abentley/launchpad/makefile-tweak/+merge/121885
<rick_h_> abentley: r=me thanks for the fix
* abentley changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 4.0*10^2
<wgrant> StevenK: Morning
<StevenK> wgrant: O hai
<wgrant> StevenK: Can you QA your rev?
<wgrant> And I will QA my fix
<wgrant> And then we can deploy the bugfix
<StevenK> I am in the middle of doing so
<wgrant> thanks
<wgrant> StevenK: Did you see the garbo bug?
<wgrant> Quite amusing
<StevenK> wgrant: Not I recall
<wgrant> launchpad.net/bugs/1043319
<wgrant> dammit firefox
<_mup_> Bug #1043319: UnusedSharingPolicyPruner doesn't preserve permitted access policies <disclosure> <qa-needstesting> <regression> <sharing> <Launchpad itself:Fix Committed by wgrant> < https://launchpad.net/bugs/1043319 >
<wgrant> stop pruning schemes
<StevenK> wallyworld_: Did you cast your eyes over qas sharing?
<wallyworld_> StevenK: yes, looks ok
<StevenK> wgrant: Just waiting for your QA and we can deploy
<wgrant> Great
<StevenK> And I have a row ready to paste into LPS.
#launchpad-dev 2012-08-30
<StevenK> wgrant: Your card for respecting BugSharingPolicy is in review. Is it really in review?
<wgrant> I submitted an MP last night...
<wgrant> but it doesn't seem to be there
<wgrant> maybe I had too many MPs open and it hung and I didn't notice
<wgrant> oops
 * wgrant will resubmit shortly
<StevenK> wallyworld, wgrant: https://code.launchpad.net/~stevenk/launchpad/destroy-target-uses-malone/+merge/121958
<wallyworld> looks ok to me
<wgrant> official_malone is still a bit evil, but less so than target_uses_malone
<wgrant> https://bugs.launchpad.net/launchpad/+bug/1043591 makes me happy
<wgrant> Private security bug with notifications but no security contact
<StevenK> Linux Mint 12 and he's filing it in LP against LP? :-(
<StevenK> wgrant: We can't destroy the team until the DB patch is on prod?
<wgrant> StevenK: And it's all that gives us access to a few private bugs in other projects
<StevenK> But sharing! We can fix it
<StevenK> wallyworld: Is your branch that creates sharing policies on creation up?
<wgrant> It's in buildbot
<wallyworld> StevenK: in bb
<StevenK> It's landed, so fine, I'm not blocked by it to fix bug 1043366
<_mup_> Bug #1043366: Create Private and Private Security policies if sharing policy allows them <disclosure> <sharing> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1043366 >
 * StevenK smacks wallyworld for duplicated code.
<wallyworld> where?
<StevenK> setB{ug,ranch}SharingPolicy
<StevenK> I've refactored it out
<wallyworld> i wasn't the original author
<wallyworld> blame wgrant
<wgrant> I wrote them originally and they were like two lines each
<wgrant> Then sinzui and I added to them
<wgrant> And now they're probably worth refactoring
<wallyworld> yes. best to to try and generalise too early
 * wallyworld smakes StevenK for jumping to conclusions :-P
<wgrant> Particularly now that they're becoming more sensible (creating PRIVATE and PRIVATESECURITY themselves), it probably makes sense to refactor now
<StevenK> Bah, still +2
<StevenK> However, setBugSharingPolicy is two lines, and Branch is 3
<wgrant> :)
<StevenK> No, +3, missed an import
<wgrant> :(
<StevenK> wgrant: http://pastebin.ubuntu.com/1175144/
<wgrant> mmmmmmmm
<wgrant> maybe
<wgrant> I'd prefer you moved the setattr out, really
<wgrant> Only saves a single line, and makes things far less greppable
<StevenK> +4, but the extra one is your fault. :-P
<wgrant> Heh
<StevenK> I thought test -vvt 'TestProduct$' would work :-(
<wgrant> That should
<StevenK> steven@undermined:~/launchpad/lp-branches/shift-ap-creation% bin/test -vvt 'TestProduct$'
<StevenK> Running tests at level 1
<StevenK> Total: 0 tests, 0 failures, 0 errors in 0.000 seconds.
<wgrant> Oh
<wgrant> Well
<wgrant> There's no tests named that
<wgrant> Do you mean TestProduct\..*?
<StevenK> Probably
<lifeless> that setattr s bong anyhow
<lifeless> obj name value
<wgrant> Well, yes.
<wgrant> But it's also pointless so it's dead :)
<StevenK> lifeless: Yeah, I was going to introspect the variable's name out
<lifeless> side effects bad in that sort helper
<StevenK> And then get murdered by wgrant
<wgrant> I'm not that murderous today
<StevenK> Now to figure what is creating these APs
<wgrant> StevenK: Your new code probably is...
<wgrant> StevenK: makeLegacyProduct just unsets foo_sharing_policy afterwards
<wgrant> They start of as PUBLIC
<wgrant> off
<wgrant> (you'll probably need to fix createProduct to not set the to PUBLIC if it's about to override them to PROPRIETARY, too)
<StevenK> wgrant: The default argument is None, and makeProduct only sets them if they're not None?
<wgrant> StevenK: See createProduct
<wgrant> It defaults them to PUBLIC
<wgrant> Then if the license is Other/Proprietary, sets them to PROPRIETARY
<StevenK> Hmmm
<StevenK> Indeed, that is a pickle
<StevenK> wgrant: It doesn't actual set until after that block
<StevenK> bug_sharing_policy_to_use = BugSharingPolicy.PUBLIC
<wgrant> Oh, right
<wgrant> That makes more sense
<wgrant> But sti.ll
<wgrant> makeLegacyProduct will create the APs due to your new code
<wgrant> Because it calls createProduct
<StevenK> wgrant: So, did you put that card's branch up for review?
<wgrant> I have the final set of 20 test failures, so will repropose once those are fixed
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/shift-ap-creation/+merge/121985
 * StevenK prepares for the wgrant sadface
<wgrant> D:
<wgrant> StevenK: Is it perhaps better to nuke the APs in the test rather than adding the skip_sharing_policy arg?
<wgrant> Regardless, makeLegacyProduct needs those two APs
<wgrant> So you cannot skip_sharing_policy unconditionally in there
<StevenK> wgrant: If we're just going to nuke them, the test is pointless
<wgrant> StevenK: Mm, true
<wgrant> StevenK: I guess what you want is a couple of tests for setFooSharingPolicy
<StevenK> Which already exist
<StevenK> I added the checks for APs to the bottom
<wgrant> Create a free project, check that it has [PRIVATE, PRIVATESECURITY], set bug_sharing_policy, confirm it creates PROPRIETARY
<wgrant> Create a proprietary project, check that it has PROPRIETARY, set bug_sharing_policy, confirm it creates [PRIVATE, PRIVATESECURITY]
<StevenK> wgrant: http://pastebin.ubuntu.com/1175188/
<wgrant> StevenK: LGTM
<wallyworld_> wgrant: StevenK: i think there's a bug in createProduct - it always creats AP for UD and PS even for sharing_policy = PROPRITARY which only allows PROPRIETARY
<wgrant> wallyworld_: That's what Stevenk just fixed, isn't it?
<StevenK> ...
<StevenK> wallyworld_: Read the diff on https://code.launchpad.net/~stevenk/launchpad/shift-ap-creation/+merge/121985
<wallyworld_> oh, right. ok. i wasn't paying too much attention
<wgrant> OK, good
<wallyworld_> the branch i'm working on has been affected by this bug
<wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/bug-restrict-type/+merge/121989
<StevenK> wgrant: Fantastic, looking
<wgrant> There's a few hundred lines of testfixes that I haven't pushed yet, to avoid killing you
<wgrant> too bad
<wgrant> ly
<wgrant> Our last few final cleanup branches are all a bit intertwined :(
<StevenK> wgrant: +107 :-(
<wallyworld_> sigh, new code not always bad
<StevenK> No, just seeing if what wgrant does to me works with him.
<wallyworld_> let's see....
 * wallyworld_ gets popcorn ready
<wgrant> Heh
<wgrant> I bitch about new code on pure cleanup branches :)
<wallyworld_> sometimes cleanup requires new code
<wgrant> Sure
<StevenK> wgrant: r=me, with one comment.
<wgrant> sadface
<wgrant> Ah yes
<wgrant> I meant to do that, actually
<wgrant> Thought I caught all of them
<wgrant> But apparently not
<wgrant> wallyworld_: Ah, you're working on the SharingService bug I reported last night?
<wallyworld_> getAllowedInformationTypes, yes
<wgrant> Great
<wgrant> I think that's the last important bit
<wallyworld_> i need to introduce some new methods on IProduct, IDistro
<wallyworld_> getAllowedBranchInformationTypes()
<wallyworld_> to match getAllowedBugInformationTypes()
<wgrant> Yeah
<wgrant> I guess just push that down from BranchNamespace
<wgrant> ish
<wallyworld_> i just used POLICY_ALLOWED_TYPES or whatever it;s called
<wallyworld_> imported it
<wgrant> Right
<wgrant> We probably want to rename that now
<wgrant> to say BRANCH or something
<wgrant> I didn't initially expect it to be exported
<StevenK> Right
<wgrant> But it's now used in a few places
<StevenK> There's a bug one which is also used in a few places
<wallyworld_> wgrant: i aliased it on the import
<wallyworld_> same with P_A_T for bugs
<wgrant> Right, but everyone's doing that now
<StevenK> I'll fix it tomorrow
<wgrant> So we should possibly just rename it at the source
<wallyworld_> not everyone needs to branch version, just me
<wallyworld_> but yes we can rename
<StevenK> wallyworld_: Just alias it, I'll do a branch to just rename tomorrow
<wallyworld_> i will also need a followup branch when setting the sharing policy using the ajax popups
<wallyworld_> since i need to reset the json cache
<wgrant> Ah, yes
<wallyworld_> to allow for the different types
<wallyworld_> hence i need a view layer :-P
<wgrant> We also need to refresh parts of the sharing table, if possible
<wgrant> eg. if I set from Public to Proprietary, it'll add a grant for the owner on Proprietary
<wallyworld_> yes
<wgrant> But not show up until I refresh
<wallyworld_> will do all that next
<wallyworld_> but it all works now, i merged in StevenK's branch. just need to add tests
<wgrant> Great
<stub> ImportError: No module named sysconfig
<stub> oh... probably because it has a hardcoded python2.6, and we still need that for production :-/
<wgrant> stub: What's that from?
<wgrant> Not seen it before
<stub> Running preflight locally
<adeuring> good morning
<wgrant> Ah, right
<lifeless> o/
<wgrant> wallyworld_, StevenK: Product._ensurePolicies (as called by setFooSharingPolicy) currently creates any missing policies, and grants the owner access to all of the requested types. I wonder if we want to restrict the grants to just the new ones. Otherwise eg. changing PUBLIC to PROPRIETARY_OR_PUBLIC will grant the owner access to everything, even though only Proprietary is new
<wgrant> Opinions?
<jam> jelmer: any luck with bzr-2.5 on staging?
<wallyworld_> hmmm. sounds reasonable at first thought
<jam> (anything I can help with?)
<StevenK> wgrant: As wallyworld_ says, sounds reasonable. Bring it up on the call and the four of us can discuss it?
<wgrant> Or I could land the fix in 5 minutes...
<wgrant> Seems pretty unobjectionable
<StevenK> Or that
<wallyworld_> i just wonder if we are relying on ensurePolicies to correct mistakes
<StevenK> My fix won't land, ec2 will send hate mail
<wgrant> wallyworld_: I think it's more likely to cause mistakes.
<wgrant> But it's not huge either way
<jam> lifeless: did you see any of the conversation about lpstats? I'm thinking we just want to nuke all of the extra rabbit queue data, but I'd like sign off from someone else before we delete it.
<lifeless> +1
<wallyworld_> wgrant: i guess fixing it makes things more failsafe
<wgrant> Oh, actually, I might have misread
<wgrant> It may do that already
<wgrant> But the variables are a bit ambiguously named
 * wgrant rereads
<lifeless> jam: Do what needs to be done :)
<wgrant> wallyworld_, StevenK: Yeah, sorry, required_types had me confused. It's actually the required types that must be created
<wgrant> So it already has the behaviour I said
<wallyworld_> cool
<wgrant> Otherwise everything seems pretty nice
<wallyworld_> well, after my next 2 branches land to fix +sharing
<wallyworld_> for proprietary projects
<wgrant> Right
<wallyworld_> one is in review
<wgrant> But it's perfectly usable atm, just a few too many UI elements sometimes
<wgrant> that don't do anything
<wallyworld_> yeah
<wgrant> Tomorrow I need to analyse the damage from the buggy garbo job yesterday
<wallyworld_> there was also confusion between Private vs Proprietary
<lifeless> jam: if you think we'll confuse tuolumne, I'm happy to help with analysis or whatever, but I presume you looked at the structure in enough detail to avoid such side effects.
<wallyworld_> i wonder if we need to fix thsat
<wgrant> wallyworld_: Yeah, I've always been a little concerned about that
<wallyworld_> wgrant: buggy job? what happened?
<wgrant> wallyworld_: It will be less of an issue once your current fix is deployed
<wgrant> wallyworld_: As commercial projects will never see private
<wallyworld_> let's hope so
<wgrant> Currently even once they set everything to Proprietary, there are still Private and Private Security grants showing up on +sharing
<wgrant> https://bugs.launchpad.net/launchpad/+bug/1043319
<_mup_> Bug #1043319: UnusedSharingPolicyPruner doesn't preserve permitted access policies <disclosure> <qa-ok> <regression> <sharing> <Launchpad itself:Fix Released by wgrant> < https://launchpad.net/bugs/1043319 >
<wallyworld_> yes
<wallyworld_> ah right, that one
<wallyworld_> affected landscape
<wgrant> Right
<wgrant> And will have affected lots of non-commercial projects
<wgrant> Mostly by deleting their Private policies
<wgrant> I suspect
<wallyworld_> do we know how many commercial projects still have sharing policies needing to be set?
<wgrant> 540 or so
<wallyworld_> because all public projects should have been updated via garbo job
<wgrant> 583, sorry
<wgrant> yeah
<wgrant> https://pastebin.canonical.com/73308/
<wgrant> All public projects should have been, yes
<wgrant> Is it in hourly? Can't remember
<wgrant> But it won't matter after tomorrow, as they'll be created pre-configured
<wallyworld_> houry
<wallyworld_> wgrant: so it would be considered failsafe just to update those commercial projects to policy = proprietary
<wallyworld_> id the owners don't do anythiong
<wgrant> Right
<wallyworld_> can't wait for that so we can nuke bvp :-)
<wgrant> It's likely to cause a few people to be unable to push branches
<wgrant> Yes :)
<wallyworld_> well, it will be a good way to shake things out :-)
<wallyworld_> because the pushers will complain to get access and the owners will have to get things correctly configured :-)
<wgrant> yep
 * wallyworld_ hears a cork pop. mmmm. tasty drink time
<wgrant> heh
<jelmer> jam1: mgz and I have made some progress, I'm about to chuck it into ec2 again
<lifeless> jam1: did you see my replies ?
<jam1> lifeless: yeah
<lifeless> jam1: kk
<wallyworld_> wgrant: so, fdt soon
<wgrant> wallyworld_: yup, double patch, and hopefully about 7 seconds
<wgrant> But this is the first super-short one
<wgrant> So will be interesting
<jam> lifeless: thanks.
<wallyworld_> excellent. i want to land the followup devel branch :-)
<dpm> hey all, could somebody help me with an issue I believe to be with the LP API? I'm trying to create language packs for Q, and the process is failing to download static translations from LP through the API. I hit a similar issue a few weeks ago when trying to build the 12.04.1 language packs, which wgrant and others helped to solve. I'm wondering if it's the same issue again. The tool that makes the API call is langpack-o-matic, running on a Canonical s
<dpm> erver, could someone give me a hand debugging this and getting the langpack to build?
<mgz> do you have irclog from that a few weeks ago and output for the failure now?
<dpm> mgz, I can find the log, but the failure output is simply what langpack-o-matic outputs (basically interpreting "gzip: stdin: not in gzip format" in http://pastebin.ubuntu.com/1175572/ as "could not download file")
<dpm> mgz, here's the IRC log from last time: http://pastebin.ubuntu.com/1175613/
<mgz> dpm: so, we're presuming we got an error page rather than a .tar.gz
<mgz> that might be a timeout, but if it got bumped last time, it might not be. log doesn't have timestamps, but it didn't run any time around 10UTC did it?
<dpm> mgz, it might have, let me check...
<dpm> mgz, yeah, the script was started at 10:02UTC actually
<mgz> hm, probably past the window
<dpm> I'll try to re-run langpack-o-matic
<mgz> but logging the actual response code from urlretrieve and actually checking what is fetched is a tarball would help regardless
<mgz> how do you deploy code for this? can give you a patch for that quickly
<dpm> mgz, MP on https://code.launchpad.net/~ubuntu-langpack/langpack-o-matic/, and I'll ping pitti as the maintainer about it
<dpm> mgz, on http://bazaar.launchpad.net/~ubuntu-langpack/langpack-o-matic/main/view/head:/import#L217
<mgz> heh, urllib seems to actually chuck the status code away...
<mgz> dpm: did rerunning the script work?
<dpm> mgz, it seems it did, yes
<mgz> okay, I'll file bug and fix for the error handling. seems like you just don't want to run it near launchpad downtime.
<dpm> yeah, I'll try to remember that :) thanks mgz
<cjohnston> Hey guys... on bug #1035661 Its fix released.. It's still effecting me (screenshot attached). should I open a new bug or reopen that bug?
<_mup_> Bug #1035661: group membership expire date "self renewal" can not be set: "Wrong Type" <disclosure> <regression> <sharing> <Launchpad itself:Fix Released by sinzui> < https://launchpad.net/bugs/1035661 >
* jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: jcsackett | Firefighting: - | Critical bugs: 4.0*10^2
<abentley> rick_h_: Do you know how to debug "extend failed, verify dependencies"?
<rick_h_> abentley: it's a tough one, usually I comment out all objects/extend() in the file and open them up one by one
<rick_h_> to find the missing dep
<abentley> rick_h_: The file has no objects.  It's all functions.  So I'm assuming one of the dependencies, itself, is broken.
<rick_h_> abentley: try to run the YUI test layer and see if one of them blow up to hint?
<abentley> rick_h_: I guess I can try.  I don't think any other files have been changed.
<rick_h_> abentley:  xvfb-run ./bin/test -x -cvv --layer=YUITestLayer  should run pretty quick
<abentley> rick_h_: Only failure was in my file.
<rick_h_> abentley: can you pastebin and I can look for anything that jumps out ?
<abentley> rick_h_: The source file itself?
<rick_h_> abentley: yea, the file with the error
<abentley> rick_h_: http://pastebin.ubuntu.com/1176379/
<rick_h_> abentley: lazr.choiceedit does that still existing? I thought it was killed
<rick_h_> I wonder if including that requirement is causing it to fail, check the build/js/meta.js
<abentley> rick_h_: It still existing.  at build/js/lp/app/choiceedit/choiceedit.js
<rick_h_> ah ok. There's some work on killing lazr and I don't see that module used in your code
<abentley> rick_h_: It provides Y.ChoiceSource.
<rick_h_> yea, see it. Is the branch pushed up to check out?
<rick_h_> nothing jumps out in there, but wonder if it's the used location
<rick_h_> is this in a test file? is the test file YUI() vs YUI?
<abentley> rick_h_: branch is at ~abentley/launchpad/blueprint-info-type-ui and up-to-date.
<abentley> rick_h_: The test runs fail with "extend failed, verify dependencies", but only if information_types.js includes lp.app.choice.
<abentley> s/includes/requires
<rick_h_> abentley: in test_information_type.html it's including information_type.js from the build directory, but then also including it via ../information_type.js
<abentley> rick_h_: Ah, that's got it.
<abentley> rick_h_: Human error while merging.
<rick_h_> cool
<abentley> rick_h_: Thanks very much.
<abentley> jcsackett: Could you take a look at https://code.launchpad.net/~abentley/launchpad/blueprint-info-type-ui/+merge/122140?  Sorry it's a bit long.
<jcsackett> abentley: sure.
<jcsackett> abentley: r=me, but some comments about QA on the MP.
<jcsackett> information_type_choice has been a hotbed of accidental breakages, so it's worth your time to double check things that seem somewhat redundant. :-P
* jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 4.0*10^2
<wgrant> wallyworld_: I think your AP changes have caused a sampledata conflict with db-devel? Could you have a look at that soonish before PQM spams us to death, please?
<wallyworld_> sure
<wallyworld_> but i didn't change sample data? i guess there's a constraint violation
<wgrant> Hm
<wgrant> Maybe it's not your fault, then
<wgrant> I thought you might have prepopulated sampledata
<wallyworld_> noooo
<wgrant> Might just be fallout from my merge, then
 * wgrant fixes
<wgrant> Probably related to the reverted reverted reversion
<wallyworld_> bb is ok, did you see it locally?
<wgrant> PQM's failing to merge
<wgrant> Anyway, fixing
<wgrant> It's indeed a criss-cross from the mismerge
<StevenK> Ah, I was about to look at that.
<wgrant> StevenK: How'd your AP creation branch go in ec2 last night?
<StevenK> wgrant: 4 failures, I'm starting to peer at them now.
<wgrant> If you fix them in a few minutes then we can massacre buildbot and deploy them today
<StevenK> But, oh, no pressure. :-)
<StevenK> Hah
<StevenK> wgrant: Hah. All four failures are caused by factory.makeLegacyProduct(licenses=[License.OTHER_PROPRIETARY]) and then having the bugs forced to USERDATA
<wgrant> StevenK: Ah, right. Maybe have makeLegacyProduct set them to PUBLIC and then unset them
<wgrant> to ensure the APs exist
<StevenK> Why? The whole reason they don't get set is because setB{ranch,ug}SharingPolicy says oh, License.OTHER_PROPRIERAY, here, have PROPRIETARY only
<wgrant> StevenK: Not for a legacy product
<wgrant> License.OTHER_PROPRIETARY is handled by createProduct
<wgrant> to set the sharing policies to PROPRIETARY
<wgrant> makeLegacyProduct then unsets them, to get the old behaviour
<wgrant> But it relies on the fact that the USERDATA/PRIVATESECURITY APs will exist already, because they did until your branch
<wgrant> Now you'll need to create them by either calling _ensurePolicies directly, or setBugSharingPolicy(PUBLIC) before setting it to None
<StevenK> Yeah, I get it now.
<wgrant> (IIRC the tests that use makeLegacyProduct with License.OTHER_PROPRIETARY are testing that owners can set private_bugs if there's a commercial sub?)
<Laney> is there a way to delete a packageset?
<Laney> via API
<cjwatson> Not even sure there's one internally
<wgrant> There isn't one
<wgrant> Easy enough to add and expose if someone wants to
<Laney> hmm
<Laney> This causes me to reconsider how much we care about doing this
<Laney> vs, say, deleting all the uploaders and leaving them
<cjwatson> It'd be about one hour to develop, test, and write up the MP, I suspect
<wgrant> Agreed, just have to check that all the permissions are gone, basically
<StevenK> wgrant: Test failures fixed, pushing now.
<StevenK> Branch is now +0
<wgrant> :)
<Laney> possibly could be something to do on the train on Saturday morning
<StevenK> wgrant: r15890
#launchpad-dev 2012-08-31
<StevenK> wallyworld_, wgrant: Your thoughts on http://pastebin.ubuntu.com/1176978/ ?
<wallyworld_> BRANCH_POLICY_ALLOWED_TYPES perhaps?
<StevenK> Which is just awkwardly long
<wallyworld_> but more clear
<wallyworld_> i don't agree that it's awkward
<wallyworld_> but that's just mho
<wallyworld_> wgrant can have the casting vote
<StevenK> But he isn't old enough to vote?
<wallyworld_> lol
<wgrant> Hey, I can even drink the US as of a few months ago!
<wgrant> I agree with wallyworld_
<wgrant> Both on that BRANCH_POLICY_ALLOWED_TYPES is good, and that I should have the casting vote because I'm awesome.
 * wallyworld_ gets a bucket
<wgrant> :)
 * StevenK pushes up the branch, since you both suck
<StevenK> wallyworld_, wgrant: https://code.launchpad.net/~stevenk/launchpad/exported-policy/+merge/122184
<wgrant> StevenK: 8	- 'POLICY_ALLOWED_TYPES',
<wgrant> 9	+ 'BUG_POLICY_ALLOWED_TYPES',
<wgrant> 10	'POLICY_DEFAULT_TYPES',
<wgrant> Perhaps rename POLICY_DEFAULT_TYPES as well?
<wallyworld_> +1
<StevenK> wallyworld_: And POLICY_REQUIRED_GRANTS in branchnamespace?
<StevenK> Rargh, tab fail
<wallyworld_> why not, might as well
<StevenK> wgrant: Diff updated
<wgrant> StevenK: r=me, thanks
<wgrant> StevenK: Your security_contact droppage is good to go out tonight?
<StevenK> wgrant: Yeah, applied in 0.2 on sourcherry
<StevenK> Sigh. We use apt-ftparchive generate for contents generation
<adeuring> good morning
<mgz> so, I had a message from buildbot about my (ec2 tested) change failing when merged along with one from wallyworld
<wgrant> mgz: It's on production now
<mgz> given there was a NDT since that d... right
<mgz> so, is my "just ignore it" attitude right, or should I be following stuff like that up?
<wgrant> That was probably the run that we selfishly had killed so a later rev could be deployed
<mgz> that sounds likely.
<mgz> aborted after running a few hundred tests, from the report
<wgrant> The rule of buildbot failures: if the current/last build in either lane of http://lpbuildbot.canonical.com/waterfall is red, you need to do something
<wgrant> otherwise ignore it, because someone else has fixed it
<mgz> okay, thanks!
* bac changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: bac | Firefighting: - | Critical bugs: 4.0*10^2
<abentley> jcsackett: I had a look at the filebug implementation of information_type, and it looks totally separate from the bug one.
<bac> i'm attempting to login to staging.launchpad.net but am getting invalid password errors from the two-factor auth query.  could this be something unique to staging or likely my 2fa is broken?
<nigelb> lifeless: How'd your keynote go?
<nigelb> Or is it later today?
<lifeless> nigelb: in 1 hour
<wgrant> bac: ISD manage even staging SSO, but I believe they know of issues with 2FA there (mostly that they don't properly disable it when they copy the DB across)
<bac> wgrant: thanks i got it worked out.
<bac> staging and production cannot use the same devices or they get out of sync
<bac> and if you set 'always use 2fa' then you are screwed b/c staging will try to use it...but you don't have a registered device.
<wgrant> bac: Right, I believe they intended to disable that setting for everyone on staging, because you obviously can't use the same device on both
<wgrant> So everyone would be locked out
#launchpad-dev 2012-09-02
<cr3> hi folks, is there a wiki page that describes the policy used by Launchpad for triaging bugs? for example, I heard you don't use the medium priority, is that documented somewhere?
<cr3> nevermind, I just found my answer in the aptly named page: https://dev.launchpad.net/BugTriage
<lifeless> cr3: there is also
<lifeless> cr3: https://dev.launchpad.net/BugTriage/Background
<cr3> lifeless: do you happen to know of tools to generate more elaborate reports of bugs for a project or a person? I know of jkakar's kanban that's pretty cool, but I'd like to have a look at other options
<lifeless> theres the ubuntu qa tools
<cr3> lifeless: thanks, I'll pull the master branch and have a look
<wgrant> wallyworld_: I wonder if you could just return the types of the accesspolicies that exist?
<wgrant> I think that would work
<wallyworld_> perhaps. i'd rather filter by ones in use
<wgrant> The access policies that exist are the access policies that are in use, Â± a day
<wallyworld_> sure, we now have a garbo job of course. but a day is a long time
<wgrant> I was thinking last night that we should probably prune on sharing_policy change, too
<wgrant> So if you change it and they're unused then they evaporate
<wgrant> immediately
<wallyworld_> that sounds best to me. i don't like the job method of cleaning these things up. fine for a safety net, but i think we can do better
<wgrant> Well, the job is necessary unless we want to be firing off more jobs every time an information type changes
<wgrant> But we should probably do it inline on sharing_policy change
<wallyworld_> that's what a meant - do it inline, not more jobs
<wallyworld_> if it can be done efficiently
<wgrant> Even when an artifact's information_type changes?
<wgrant> We *could* do that, but it's likely to cause some contention and isn't obviously a significant benefit
<wallyworld_> it's a benefit since it allows the in use policies to be maintained. but how much so, not sure
<wallyworld_> if the info type changes, so long as we ensure there's a policy if there were none previously
<wgrant> You can't change the info type to something not allowed by the sharing policy, and changing the sharing policy ensures that there's an access policy, so that direction is not an issue
<wgrant> It's just cleanup that needs consideration
<wallyworld_> it may ensure there's an access policy but the policy may be cleaned up before it can be used
<wgrant> It won't be, since we don't clean up anything that's allowed by either sharing_policy.
<wallyworld_> that's good
<wgrant> (modulo that bug last week)
<wallyworld_> ah right, yes
<wgrant> So, I think it's probably sensible to just return the APs that exist. Separately we should factor most of the pruner into a model method, and call it from both the garbo job and setFooSharingPolicy
<wgrant> Since your new method has to return all the stuff permitted by sharing policies, and all of the types involved in access policies, but the model ensures that everything permitted by a sharing policy has an access policy
<wallyworld_> the method already returns the permitted stuff, just is missing the current in use stuff
#launchpad-dev 2013-08-26
<stub> Anyone up for Swift reviews?
 * stub wonders if it should be a single MP
#launchpad-dev 2013-08-27
<cjwatson> wgrant: Did you have a chance to look at my series-alias branch while I was on holiday?
<cjwatson> wgrant: I'm trying to sort out the Deferred handling in https://code.launchpad.net/~cjwatson/launchpad/buildmaster-cancel-properly/+merge/177580.  Why would requestAbort fail if the slave is already ABORTING?  My reading of the slave code is that it should just return immediately.
<cjwatson> wgrant: My inclination is that if requestAbort fails then I should immediately enter the resume-slave-host-or-fail-builder path.  But I think I can only take that approach if such a failure isn't going to happen any time somebody restarts buildd-manager in the middle of a cancellation.
<cjwatson> I believe that http://paste.ubuntu.com/6034419/ implements this if it's a tenable approach.
<wgrant> cjwatson: Ah, you're right, abort() shouldn't normally fail.
<wgrant> resetting is indeed probably sensible, but the CancellationError due to a manager restart is indeed an issue.
#launchpad-dev 2013-08-28
<wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/builderinteractor-c-b-b/+merge/182290
<StevenK> I have fear.
<StevenK> Oh, only 700 lines
<wgrant> The next one's 1800 lines, but it's just moving code so I'll self-review.
<StevenK> wgrant: r=me with one niggle
<wgrant> StevenK: Those tests need to be moved and rewritten later in this series
<wgrant> I just did enough to make them work here.
<wgrant> Thanks.
<StevenK> wgrant: Oh, right, they will get burnt down and replaced later. Fine, niggle dropped.
<StevenK> wgrant: Hmm, have you added a race?
<wgrant> StevenK: Where?
<StevenK> wgrant: db-devel buildbot failure
<wgrant> No, that's a normal failure.
<stub> Anyone up for Swift branch review?
<cjwatson> wgrant: I still don't quite understand.  What CancellationError do you expect on manager restart with my patch?  AFAICS it will either send abort to a slave that's already aborting, in which case the slave will say "ok then" and carry on, and the master will just have a timeout that extends from the restart rather than the original cancel; or the slave will have died in some other way such that the abort fails, in which ...
<cjwatson> ... case my patch will cause the master to immediately resume-or-fail, which seems right.
<wgrant> cjwatson: Nevermind, I misunderstood.
<wgrant> cjwatson: The slave abort() method should never fail, so you can probably do resetOrFail if it does.
<cjwatson> wgrant: I think https://code.launchpad.net/~cjwatson/launchpad/buildmaster-cancel-properly/+merge/177580 is ready now.  Dunno if you care about reviewing e.g. the inlineCallbacks conversion I did after getting lost in a maze of callbacks one too many times.
<cjwatson> (Just in checkCancellation, though, not all over)
<cjwatson> Need to do some QA on dogfood builders first.
<wgrant> cjwatson: I plan to port an awful lot of stuff to inlineCallbacks over the next week or so. Your port looks sane.
<cjwatson> Good.  I think we're ready to go on this once we finish fixing recipe builds in launchpad-buildd 115 (see -ops) and deploying everywhere.
<cjwatson> Only four hours spent back-and-forthing on that this evening ...
#launchpad-dev 2013-08-29
<wgrant> Yeah, I knew it would be bad, and we've been down some IS staff here for the last couple of weeks, otherwise I would have done it myself.
<wgrant> Thanks :)
<wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/builderinteractor-updateBuild/+merge/182600
<StevenK> wgrant: So there's no code changes, just moving it?
<wgrant> StevenK: Apart from porting TTBB to override handleStatus instead, yeah.
<wgrant> Otherwise it's basically just s/self._interactor/self/
<StevenK> wgrant: r=me with a small niggle
<wgrant> Commenting circular imports is pointless.
<wgrant> I've never understood why we do it
<wgrant> Need to know whether an import is circular? If it's inside a function, it's circular.
<wgrant> Need to find all circular imports? bzr grep '^\s+import'
<StevenK> True. But currently, we tend to comment on them.
<wgrant> It might become more reasonable if we have a sane code structure such that they don't appear in almost every file.
<StevenK> That would be nice.
<wgrant> But now you can barely read a notable function without seeing at least one, so commenting them is silly.
<StevenK> Right.
<cjwatson> wgrant: How about http://paste.ubuntu.com/6038589/ as a refactoring of the Sources/Packages path handling in series-alias (untested, but general idea)?  It's more LOC than before, but perhaps that's worth it.  I'm on the fence.
<cjwatson> I thought about making it a method on the publisher config.  Not sure about that.
<wgrant> cjwatson: I think that's clearer. I think having them as functions is fine.
<wgrant> I like keeping the path joining and string formatting rules in one place
 * cjwatson sets the archivepublisher tests going and really goes to bed, then
<cjwatson> Thanks
<wgrant> Thanks, and night
<jtv> wgrant: excellent downtime announcement email BTW.  I saw one from Microsoft the other day and the difference is palpable.
<wgrant> I think our announcements win simply by not quoting the times in some random timezone
<wgrant> Particularly when someone quotes an outage time in PST when they actually mean PDT...
<jtv> Seriously?
<jtv> It's a bit like that ad a friend spotted in late 1999...
<wgrant> Yeah, Americans don't do timezones, even their own.
<jtv> "Even in the year 2000 we will be there for you 365 days."
<wgrant> Heh
<jtv> He was itching to call them and ask: given that 2000 is a leap year, and surely you have dealt with Y2K bugs such as mistaking it for a non-leap year, which day will you be unavailable?"
 * jtv was sad to discover at a quiz that only one other contestant knew how to calculate leap years
<jtv> Doubly sad: one, a room full of teachers was largely unaware; and two, there was the bare minimum of awareness to rob our team of that 1-point advantage.
<wgrant> Well, everyone'll hopefully learn by 2100 :)
<jtv> They'd better!
<jtv> Loved that Sun joke at the time...  "Y5K bug (or whatever the number was) discovered in Java.  A team at Sun is reportedly rushing to fix the problem."
<wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/builderinteractor-cookies/+merge/182813 https://code.launchpad.net/~wgrant/launchpad/axe-the-ibb/+merge/182814
<StevenK> wgrant: r=me for both
<wgrant> StevenK: Thanks.
<wgrant> Nearly disentangled.
<StevenK> wgrant: One more branch
<StevenK> ?
<wgrant> I've got one more branch of cleanups, and then I think I can start pulling the DB dependencies up.
<wgrant> That mostly involves heavily caching a few builder attributes and the slave build cookie.
<wgrant> and then a SlaveScanner should be able to handle most operations other than the start and end of a build without touching the DB.
<StevenK> So its another 3 or 4 branches, right
<wgrant> And I will have succeeded in stopping the commits.
<wgrant> Something like that.
<StevenK> wgrant: So why was there an IBB, that's what I can't understand
<wgrant> StevenK: BuildFarmJobBehavior used to have updateBuild etc. on it
<wgrant> Including updateBuild_IDLE
<wgrant> Every scan would go through to a BFJB.
<wgrant> And the BFJB could do with the result whatever it wanted.
<wgrant> But nothing ever overrode anything except updateBuild_WAITING, and I don't want to have to create a BFJB from the DB for every scan.
<wgrant> So I moved those general methods up into the new BuilderInteractor.
<wgrant> In related news, https://code.launchpad.net/~wgrant/launchpad/direct-action/+merge/182818
<wgrant> It sorta made sense back in 2009 when it was designed, as not everything was going to have a BuildFarmJob.
<wgrant> So the original TTBJ implementation overrode a couple of the others.
<wgrant> But that's long fixed.
<wgrant> StevenK: Do you vaguely see what I'm working towards here?
<wgrant> BuilderInteractor will cache enough information to do normal scans without hitting the DB. A single new worker in buildd-manager will scan the DB every n seconds and update all the cached information on each SlaveScanner's BuilderInteractor, and another new worker will grab the logtail from each SlaveScanner and flush them all to the DB at once.
<StevenK> wgrant: r=me
<wgrant> Thanks.
<cjwatson> wgrant: I don't suppose you fancy canonicalising the spelling of "behaviour" in your buildmaster rearrangements? :-)
 * cjwatson was just bitten by that again
<wgrant> cjwatson: I think I've been consistent, but I will happily switch to the real spelling now that we've exterminated the Americans :)
<wgrant> Though knowing my luck that will cause lots of extra lines to wrap, giving me a good excuse to rename current_build_behavior to something less silly too.
<wgrant> cjwatson: That branch will fail with devel merged
<wgrant> The middle argument to handleStatus is gone.
<wgrant> (it was an ILibrarianClient that hadn't been used in 3 years)
<cjwatson> I'd better merge devel into buildmaster-cancel-properly then
<wgrant> It won't conflict with that
<wgrant> As the new handleStatus was added in the branch you landed a couple of weeks ago.
<cjwatson> Yeah.  Fixed now.
#launchpad-dev 2013-08-30
<StevenK> wgrant: Bad you for using Equal/NotEqual for None
<wgrant> StevenK: Where?
<wgrant> In my IBB rework?
<StevenK> No, your log fix for r16750
<wgrant> That was cjwatson, I just reviewed it and missed that :)
<StevenK> % bzr grep 'assertEqual.*None' | wc -l
<StevenK> 512
<StevenK> So I can't complain too much. :-)
<stub> Who's up for a Swift branch review?
<cjwatson> StevenK: I had to use assert[Not]Equal there, because that mixin method is used in test case classes that derive from TrialTestCase rather than from the LP TestCase variants, and that don't have assertIs[Not]
<cjwatson> StevenK: (I tried using the more correct methods first, of course)
<cjwatson> No doubt we should fix that, but I didn't feel like pulling in a complete test rework
<stub> I just use self.assert_(foo is None), even if the test fail message is suckier.
<stub> Or maybe I abuse assertEqual too... sucks this is all so fragmented.
<cjwatson> I was following usage in nearby tests, anyway.
<cjwatson> W: Conflicting distribution: http://gb.archive.ubuntu.com devel Release (expected devel but got saucy)
<cjwatson> Oh dear
<cjwatson> I did not predict that :-(
<wgrant> Huh, really.
<mpt> Wow, dogfood.launchpad.net still exists, separately from staging.launchpad.net
<lifeless> mpt: yes, dogfood was an entirely separate cluster IIRC
<wgrant> "cluster"
<StevenK> cjwatson: Ah ha, Trial impacts again.
<StevenK> lifeless: "is"
<lifeless> wgrant: "pedant"
<StevenK> lifeless: Duh? That's wgrant means.
<lifeless> StevenK: EPARSE :)
<StevenK> lifeless: As in duh, wgrant is just another word for pedant
<lifeless> StevenK: Ah :).
<cjwatson> wgrant: Hey, it has builders attached.  That makes it a cluster, right? :)
<wgrant> Heh, I guess it does now.
<cjwatson> wgrant: I'm seeing a lot of tracebacks like this in buildd-manager:
<cjwatson> https://pastebin.canonical.com/96660/
<cjwatson> is that new?
<lifeless> cjwatson: private bin :(; probably a -ops channel question?
<cjwatson> sorry, got confused about where I was
<cjwatson> http://paste.ubuntu.com/6045865/
<lifeless> cjwatson: I don't remember seeing that before
<cjwatson> wgrant did quite a lot of rearrangement of buildd-manager that went into the last deployment, and I suspect a regression from that
<cjwatson> unless it's one of my buildd-manager changes, but I don't think it is
<lifeless> seems likely
<wgrant> Hi
<wgrant> Hmm
<wgrant> Probably a regression from my stuff, but I tested TTBJs.
<wgrant> I think it only happens when they fail.
<wgrant> And anyway, TTBs aren't critical, and this doesn't seem to have wide impact, so can probably wait until Monday.
<wgrant> cjwatson: Thanks for poking.
<cjwatson> ok
#launchpad-dev 2013-08-31
<wgrant> Reproduced the exception on DF
<wgrant> There's no way it could have worked yesterday
<wgrant> Yet I quoted the log showing that it did
<wgrant> I wonder if the code was out of date.
<cjwatson> wgrant: https://code.launchpad.net/~cjwatson/launchpad/failure-count-reset/+merge/183320
 * cjwatson goes off for a while
<wgrant> Looking
<wgrant> Thanks
<cjwatson> wgrant: One of these days I will get the hang of Twisted.
<wgrant> cjwatson: Heh
<wgrant> The key thing here is that a Deferred isn't executed until the reactor gets to it, and the reactor won't get it unless you return it all the way up.
<cjwatson> wgrant: Yeah.  It's not that I don't know it, but I still find it excessively easy to forget.
<wgrant> Indeed.
<cjwatson> wgrant: I must admit I'm fairly confused about how/where the reactor is started in test code, though.
<cjwatson> Oh, "run_tests_with = AsynchronousDeferredRunTest.make_factory(timeout=20)" maybe ...
<Guest46080> cjwatson: Right, AsynchronousDeferredRunTest lets tests return a deferred
#launchpad-dev 2014-08-25
<lifeless> wgrant: hey when I change target milestone in bugs it now shows a broken image after changing the target
<wgrant> lifeless: There's a bug for that.
<wgrant> I believe it's done it since like 2009.
<lifeless> wgrant: hmmm
<lifeless> wgrant: ok, I just don't remember seeing it until this weekend
<lifeless> wgrant: thansk
<lifeless> wgrant: oh btw - https://plus.google.com/u/0/+RobertCollins/posts/TsubZFt7Ce2 may interest
#launchpad-dev 2014-08-29
<jtv> wgrant: can I bother you with another IPv6 question?  I've been looking for a way to create IPv6 host maps through OMAPI, but it doesn't seem possible
<jtv> Maybe dhcpctl is the only way to do it.
<wgrant> jtv: I haven't tried. Have you checked the code?
<jtv> Yes...  not very hopeful.
<lifeless> jtv: 6to4 stuff?
<jtv> No, plain IPv6.
<jtv> Doesn't look as if OMAPI supports it.
<lifeless> ah well
<lifeless> not too surprising since ISC want to kill it
<jtv> On a sidenote, so do I.  :)
<jtv> Yeah I figured they'd let this fall by the wayside, but had to be sure.
<jtv> It's an awful, awful thing.
<jtv> Tried compiling a basic dhcpctl program, but... missing header.
<jtv> lifeless: dhcpctl is meant to be the replacement for omapi, right?
<lifeless> jtv: not sure TBH
<wgrant> jtv: https://translations.launchpad.net/+imports/+index?field.filter_target=all&field.filter_status=NEEDS_REVIEW&field.filter_extension=all is looking pretty terrifying nowadays, mostly because it's full of POs for inactive languages (the gardener never guesses a POFile for an inactive language). What should be done about them?
<jtv> wgrant: we have a dict somewhere providing an expiry time per status.  Add one for NEEDS_REVIEW?
<jtv> Or does it have one already?
<wgrant> It seems a bit inefficient to keep them around forever if they're never going to be approved.
<jtv> Quite.
<jtv> Making the gardener cull old uploads was scary, but also massively useful.
<wgrant> But this also seems like a pretty special case.
<wgrant> It seems like we could handle the inactive language case better.
<jtv> Don't assume too much about the permanence of the current situation though.  An inactive language may at some point be activated.
<jtv> I think there's a general problem of old uploads never being approved.
<wgrant> Right, but an old unapproved upload usually indicates something that wants manual intervention.
<wgrant> These do not.
<jtv> Then maybe the gardener should just approve them as normal, but the importer should ignore them.
<jtv> And then if they stay Approved for too long, they can expire.
<wgrant> Then we have the importer having to test and skip a hundred thousand jobs each run instead.
<wgrant> Doesn't seem like a necessarily good idea.
<jtv> The idea is to make that a temporary problem.
<jtv> Just so long as the gardener kicks them into a state where eventually they expire.
<wgrant> Also, approving them would create a POFile for the inactive language.
<jtv> Ah, fair point.
<jtv> Then... Blocked?  Needs Information with a message saying "this language is deactivated"?
<wgrant> I've never quite been sure what Blocked was used for.
<jtv> Manual blocking, normally.  Mostly interesting for templates so far.  Automatically blocks all POFile uploads for the same template.
<jtv> But maybe a disabled language should do the same.
<wgrant> Ahh
<wgrant> That sounds sensible.
<jtv> I think eventually they'd still expire.
<jtv> Keeping that queue to a reasonable size is key to smooth operation, I think.  (Well, along with suggestions not timing out.)
<wgrant> Nothing automatically processes Blocked, apart from the gardener pruning old records, right?
<wgrant> So it wouldn't be a problem to shove a heap of stuff in there.
<jtv> As far as I know the gardener is the only thing that's going to care about Blocked entries.
<jtv> I probably wouldn't block deactivated languages immediately, in case they get reactivated soon after.
<jtv> It seems to me that over 99.9% of the queue is entries that have been waiting for review for more than 2 years.
<jtv> That suggests to me that we currently have those items expire in 3 years...
<jtv> Or some period like that.
<jtv> 99.9%?  No, wait, I've got my ordering the wrong way around.
<jtv> Surely I do.
<wgrant>     RosettaImportStatus.NEEDS_REVIEW: timedelta(days=DAYS_IN_HALF_YEAR),
<wgrant> hmmmmmm
<jtv> Maybe that's only for Ubuntu uploads?
<jtv> I seem to remember it didn't just blindly apply to all uploads.
<wgrant> It removes >1y Ubuntu blocked POs.
<wgrant> But that's the only Ubuntu-specificity I can see.
<wgrant> Weird
<wgrant> I think date_status_changed is being touched on those.
<wgrant> Oh
<wgrant> I guess that could just be because the entry was updated by a new upload?
<jtv> That's possible, yes.  It resets the clock.
<wgrant> So we'd end up with 500,000 entries in Blocked.
<wgrant> Which might not be a problem.
<jtv> If the Blocked ones get cleaned up eventually, you can shave a few orders of magnitude off the queue...
<wgrant> Except that these keep being uploaded.
<jtv> But you'll know a lot more about how many those are, I suspect.
<jtv> Can't imagine it being almost all of them...
<wgrant> If it wasn't almost all of them, wouldn't they have been approved?
<wgrant> Approved or expired.
<wgrant> Hmm
<wgrant> Actually, almost all of them are Ubuntu uploads.
<wgrant> wat
<wgrant> jtv: https://launchpad.net/ubuntu/utopic/+source/libreoffice-l10n/+imports
<jtv> wgrant: oh dear, another project that doesn't use the language code for the PO file name...
<wgrant> 424000 of the unapproved items are in libreoffice, libreoffice-l10n or python-django.
<wgrant> debian/python-django-common/usr/share/python-django-common/django/contrib/admin/locale/udm/LC_MESSAGES/django.po
<wgrant> That's quite a path.
<wgrant> And the same sort of problem.
<jtv> We've always had some special-case code in the approver for this sort of thing.  And it's horrible.
<wgrant> jtv: I'd be tempted to Block all the queue entries for those packages for now, to get the gardener down from 14 hours. Thoughts?
<jtv> wgrant: first thought is... â14 hours!?â
<jtv> Second: looks like we'll need to tweak the special-case code again.  :(
<jtv> But yeah, an oversized queue is poison.
<wgrant> Is this the multidirectory special case?
<wgrant> I saw that in the guessing code earlier.
<wgrant>                 pofile = self._guess_multiple_directories_with_pofile()
<wgrant> Yeah, looks like it
<wgrant> There are a whole lot of similar cases hardcodedish there.
<wgrant> Hm
<wgrant> Except python-django in utopic has no templates, and there are no POTs of any status in the queue.
<jtv> wgrant: there seem to be a few variants of multi-directory special cases, and it can be hard to be sure why none of them "takes" for a particular set of uploads.
<wgrant> In this case it might just be because there's no template at all.
<jtv> Oh, it has POs but no POTs?
<wgrant> https://launchpad.net/ubuntu/utopic/+source/python-django/+imports?field.filter_status=all&field.filter_extension=pot
<wgrant> 'https://translations.launchpad.net/ubuntu/utopic/+source/python-django
<jtv> I guess that means "yes."
<wgrant> As far as I can tell.
<wgrant> But you might know of some dark corners where they could be hiding.
<jtv> It could be that one of the PO files is effectively also the template.
<jtv> There might even be an "English translation" (in which case we hope and pray that the translations are either blank or identical to the translatable strings).
<jtv> I think you can even approve a .po file as a template, and have it imported.
<jtv> And then maybe re-approve it in its original form, for its translations...
<wgrant> jtv: Or I could just set them to Needs Information and pretend they don't exist.
<jtv> wgrant: then do set a message.  After a while IIRC they'll be rejected, and then deleted.
<jtv> (Or maybe that was the same thing.  I *think* there was a stage inbetween.)
#launchpad-dev 2015-08-24
<wgrant> Right, TryOutBuildSlave was written by Translations people, IIRC.
<wgrant> We should modernise HowToUseSoyuzLocally.
<wgrant> HTUSL is a bit shorter than it was originally, so TOBS can probably be easily folded in.
<blr> wgrant: yep, poking at it today
<wgrant> blr: Do let me know if you run into roadblocks; I know it all very well.
<blr> wgrant: are all the buildd's on 12.04?
<wgrant> blr: Most are trusty nowadays.
<wgrant> But some are still precise.
<blr> wgrant: was there anything other than the rpc section worth preserving? If not I'll delete it.
<blr> by _it_, I mean the TryOutBuildSlave page.
<wgrant> pbuilder section is silly, build the package section is out of date, chroot uploading is still relevant but I think the other page includes it.
<wgrant> Ah yeah, the other page uses manage-chroot now, so the manual curl | xargs is no longer required.
<wgrant> blr: So the rest of the page can burn. Only the manual RPC stuff is still valuable.
<blr> cool, thanks
<wgrant> cjwatson: Oops, pasted and failed to edit that line. Thanks.
<wgrant> cjwatson: So you have a viable plan for porting upload notifications to BaseMailer?
<cjwatson> wgrant: Yes, I'm just beating tests into submission but it mostly works.
<wgrant> Yay
<cjwatson> All is easier with BaseMailer being a little more flexible so that a single mailer can emit everything for one action.
<cjwatson> It'll fix things like https://bugs.launchpad.net/launchpad/+bug/117155 in the progress
<mup> Bug #117155: soyuz emails are vague about why they are being sent to a given user <email> <lp-soyuz> <notifications> <soyuz-build> <soyuz-upload> <Launchpad itself:Triaged> <https://launchpad.net/bugs/117155>
<cjwatson> *process
<blr> morning
#launchpad-dev 2015-08-25
<blr> wgrant: sensible subdomain name for the endpoint configuration? builderproxy?
<blr> builderproxyservice?
<blr> perhaps builderproxyapi
<blr> naming things is hard.
<blr> cjwatson: I can see instances of the git hosting client mocked for testing, but no integration tests - I suppose that's rather difficult (thining in terms of the client for the builder proxy)
<cjwatson> That's right, I never got round to working out how to integrate that into the build system
<cjwatson> I can't remember whether it was a problem with installing pygit2, or a problem with getting it to actually run in individual tests
<cjwatson> Possibly both
<cjwatson> The interface isn't so complicated that that scares me *too* badly - it'd be nice to fix, but I don't think it's a showstopper for your new thing if you have similar trouble
<cjwatson> It is possible that this was one of the many problems I decided was blocked on moving off buildout ...
<cjwatson> How's the local buildd setup coming along?
<blr> cjwatson: bob appears to be happy, just fleshing out a client now
<cjwatson> Cool
<cjwatson> I'm kind of hoping that I'll emerge from mailer code this week sometime :-)
<lifeless> cjwatson: thank for the testtools patch update, shall look this weekish
<cjwatson> lifeless: cool, thanks
#launchpad-dev 2015-08-26
<blr> wgrant: cjwatson: is there much to be gained by using request's collection pooling/sessions do you think?
<wgrant> blr: Not for this. Several HTTP requests are alreadymade per dispatch.
<wgrant> If this were an otherwise fast path then sure.
<blr> wgrant: thanks, I didn't think so (we use it in the git hosting client)
<wgrant> blr: Oh, you probably can't use requests here anyway
<blr> wgrant: err why's that?
<blr> from the build master?
<wgrant> blr: buildd-manager is all asynchronous Twisted, so blocking requests are a bad idea. You could deferToThread, but the requests are simple enough that using native twisted.web is probably easier
<blr> wgrant: ah, I appear to have wasted most of the day :)
<blr> hmm
<blr> well, the interface is probably fine
<wgrant> Likely, will just need to be deferred-ified
<blr> wgrant: will have a look at twisted.web.. sorry, I should have talked to you about this in more depth perhaps
<wgrant> I should have remembered you hadn't properly encountered twisted in depth before
<wgrant> twisted.web isn't too bad, compared to the classic python librariea
<wgrant> s
<blr> wgrant: so I gather I want an Agent?
<blr> ah found, some client documentation that is more informative than the api docs.
<wgrant> blr: twisted.web.client.downloadPage is used in a few places, and may be sufficient for your needs.
<wgrant> I forget the details, but let me know if your digging is unsuccessful.
<blr> wgrant: thanks
<cjwatson> wgrant: https://code.launchpad.net/~cjwatson/launchpad/answers-mail-permissions/+merge/269241 if you're still up, otherwise I'll self-review in a bit to fix the current qa-bad
<cjwatson> OK, SRing
<blr> wgrant: colin mentioned there was a method which took a build type argument, do you recall which that was - I only see snappy used explicitly in the builder browser code
<cjwatson> blr: The build behaviour tells the slave which of its build managers to use: in this case, SnapBuildBehaviour.composeBuildRequest
<cjwatson> blr: The build manager in question is called "snap", not "snappy", which is perhaps why your search failed
<blr> ah thanks cjwatson
<cjwatson> And you can follow through the build argument handling from there
<cjwatson> There's a build behaviour for each build type
#launchpad-dev 2015-08-27
<blr> wgrant: are you aware of an alternative to setting juju config via juju deployer (i.e. the 'deploy' yaml in mojo) in a spec?
<blr> a manual juju set in postdeploy?
<wgrant> blr: I've just had sysadmins run juju set manually in the past.
<wgrant> What're you trying to do?
<blr> wgrant: updating rutabaga's mojo spec to reflect the changes we made to auth_params in the charm a while back, and I think there is possibly a bug in juju in that it appears to strip quotes when passing the list of dicts to juju deployer, which blows up
<blr> Invalid config charm squid-forwardproxy auth_params=[{'scheme': 'basic', 'program': '/srv/rutabaga/code/rutabaga/scripts/rutabaga_auth_helper.py'}]
<blr> juju set squid-forwardproxy auth_params="[{'scheme': 'basic', 'program': '/srv/rutabaga/code/rutabaga/scripts/rutabaga_auth_helper.py'}]" is fine however
<blr> escaping quotes results in yaml errors
<blr> hmm double quoting the string passes the correct string, yet juju deployer still complains. Invalid config charm squid-forwardproxy auth_params='[{'scheme': 'basic', 'program': '/srv/rutabaga/code/rutabaga/scripts/rutabaga_auth_helper.py'}]'
<wgrant> blr: Are you sure your version of the charm has that parameter?
<blr> wgrant: it does, the correct rev of the charm is in the build directory
<wgrant> blr: What's your syntax in the YAML?
<blr> http://paste.ubuntu.com/12203792/
<wgrant> blr: I used syntax like this to avoid escaping issues:
<wgrant>   auth_params: |
<wgrant>     [some, literal, yaml]
#launchpad-dev 2015-08-28
<cjwatson> wgrant: Does my amendment to https://code.launchpad.net/~cjwatson/launchpad/delay-ppa-publication/+merge/269090 look OK?
<cjwatson> wgrant: Also, want me to do your pending QA?
<wgrant> cjwatson: Ah, if you could.
<wgrant> Sorry, been distracted the last couple of days...
<wgrant> That MP looks better now, thanks.
<cjwatson> OK, thanks.
<cjwatson> wgrant: I seem to get "URL not allowed" surprisingly often
<cjwatson> e.g. http://lies.example.com/ resulted in that
<cjwatson> When I'd have expected "DNS lookup failed"
<wgrant> Hm.
<cjwatson> I mean, it's not actually breaking AFAICS, just surprising.
<wgrant> Yeah
<wgrant> I guess it falls afoul of the IP address restrictions because it has none.
<cjwatson> Ah, could be.
<wgrant> I'll need to refactor the rules to remove the positive dst assertion, I think.
<cjwatson> http://www.ubuntu.com/ also gave me "URL not allowed" but that's more understandable.
<wgrant> Which is possible if they get split up, I think.
<wgrant> Right, that is intended.
<wgrant> I expected the DNS error to happen before the ACL check, TBH.
<cjwatson> Trying to think of a way to provoke a different error without actually having to bother setting up my own endpoint.
<wgrant> You can get connection errors by using a host that doesn't listen on HTTPS.
<wgrant> Or HTTP.
<cjwatson> Oh that's true
<wgrant> Has to be port 80
<wgrant> And HTTPS is dodgy because of httplib, so 80 is better to test.
<cjwatson> wgrant: https://oops.canonical.com/?oopsid=OOPS-86856e3812ab2bada21cf3f23ab8f83b
<cjwatson> Though apparently not a regression.
<wgrant> Hrmph.
<wgrant> Did you provoke a timeout?
<wgrant> It indeed shouldn't OOPS.
<cjwatson> (That's from trying my ADSL, riva.dynamic.greenend.org.uk, which doesn't listen on HTTP but also apparently doesn't actually reject it)
<wgrant> Ah.
<wgrant> Right.
<cjwatson> Useful misconfigurations FTW
<cjwatson> Anyway, qa-ok since that's a pre-existing bug
<cjwatson> Shall I file a bug?
<wgrant> Please do.
<wgrant> cjwatson: https://code.launchpad.net/~wgrant/launchpad/bug-1489674/+merge/269480
<cjwatson> wgrant: https://bugs.launchpad.net/launchpad/+bug/1489792
<mup> Bug #1489792: WebhookClient.deliver crashes if the endpoint times out <fallout> <oops> <Launchpad itself:Triaged> <https://launchpad.net/bugs/1489792>
<wgrant> Thanks
<cjwatson> wgrant: Mystifying, why is to_person ever a team there, let alone a private one?
<wgrant> cjwatson: If the team has an email address.
<cjwatson> Oh, right.
<cjwatson> And we don't need this in BaseMailer too?
<wgrant> BaseMailer already seems to touch other attributes without rSP
<cjwatson> It does, but I wonder if it has hitherto only been called from scripts
<wgrant> So I thought it must have done it globally somewhere else.
<wgrant> That is possible indeed.
<cjwatson> One of these days the word order in ZopelessDatabaseLayer vs. DatabaseFunctionalLayer will stop confusing me.  Or I'll crack and rename one of them.
<wgrant> ZopelessDatabaseLayer needs to be renamed for two reason.
<cjwatson> wgrant: Anyway, r=me but I should check whether any of my BaseMailer conversions are potentially affected.
<cjwatson> I think team-mail for example can go wrong if you renew a team membership and one of the admins of the team is an inaccessible private team.
<wgrant> Yep
<cjwatson> get_recipients does rSP but BaseMailer doesn't.
<cjwatson> Can you think of any reason not to plaster rSP over all recipient attribute accesses in mailers?
<cjwatson> With BaseMailer, it's always a separate mail to each recipient so there should be no leakage.
<wgrant> Indeed.
<wgrant> Ideally you could do it higher.
<cjwatson> Maybe in get_recipients?
<wgrant> That might not help with eg. the rationale calculation in TeamMembershipMailer.
<cjwatson> Which bit of it?
<wgrant> It unusually grabbed the displayname directly for something.
<wgrant> One of the templates had %%(blahblah)s because it was formatting the format stirng.
<cjwatson> forExpiringMembership does admin.unique_displayname, if that's what you're thinking of
<cjwatson> The stuff in TeamRecipientReason is fine because it always refers directly to the member or the team being operated on, rather than other members/admins of the same team; and when performing one of these operations you always have access to both the member and the team at hand.
<cjwatson> team.teamowner is used, but that's constrained to be a public person.
<wgrant> Eeh.
<wgrant> Indeed, probably safe.
<cjwatson> It's true that forExpiringMembership would need its own handling as well.
<cjwatson> But it actually needs some special handling no matter what, because it tells you the names of the team's admins.  If you're in the bizarre situation of being a member of a team with some admins you aren't allowed to know about, the expiring membership mail shouldn't tell you
<cjwatson> (Is this really a possible setup?)
#launchpad-dev 2015-08-29
<rpadovani> cjwatson, o/ how are you? Sorry for the long absence after the first MR (btw, thanks for the review) but I was on holiday. Any hint on which easy bug related to python I can try to fix? Or I just look the list and do something?
#launchpad-dev 2015-08-30
<cjwatson> rpadovani: I think the best thing is to look for something that grabs your attention, and ask for advice on whether it's reasonably tractable
<rpadovani> cjwatson, sorry to bother you - I updated the 'devel' branch and now when I type `make run` it says Error: Couldn't find a distribution for 'celery==3.1.18'. I have it on a lxc container with 12.04, what's wrong? I tried to pip install -U Celery but doesn't change anything...
<blr> morning
<cjwatson> rpadovani: We don't use pip (yet).  Either you need to use rocketfuel-get to update, or you need to update your lp-sourcedeps/download-cache checkout
<cjwatson> (Doing the former will do the latter)
<rpadovani> thanks, worked as charm
<blr> cjwatson: what is turku-agent in the git mojo spec?
<blr> something related to backups?
<elmo> blr: yes
<elmo> blr: https://insights.ubuntu.com/2015/08/04/introducing-turku-cloud-friendly-backups-for-your-infrastructure/
<blr> elmo: thanks
#launchpad-dev 2016-09-01
<oSoMoN> arenât launchpad builders affected by https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=827315 ?
<oSoMoN> apparently not
<cjwatson> not as yet I think, see #ubuntu-devel
<cjwatson> possibly only because our chroots haven't been refreshed lately
<xnox> i believe that bug was transient, and with reverted stuff in sbuild, all is good again.
<xnox> so only affected combos was 0.70 sbuild and gpg2 in chroot. Newer sbuild is fine. Older as well, if one purges the apt-keys on the host.
<xnox> ..
<xnox> i just noticed new launchpad certificate, and it's digicert now. Old certs used to be thawte, if i recall correctly.
#launchpad-dev 2017-09-01
<xnox> lxd snap builds appear to be painfully slow
<xnox> https://code.launchpad.net/~xnox/+snap/xnox-subiquity/+build/75817
<xnox> it's been 9 minutes doing apt update =(
<xnox> and previous builds at https://code.launchpad.net/~xnox/+snap/xnox-subiquity would take just 1 minute to get a lot further (and fail, but whatever)
<cjwatson> It may have hung
<xnox> it is progressing, but slowly
<cjwatson> Bluff
<wgrant> That Ign sounds a bit suspicious
<xnox> i made coffee refreshed and it had an extra 'Ign:5 http://ppa.launchpad.net/snappy-dev/tools/ubuntu xenial Release'
<xnox> before refresh it was at Ign:3
<wgrant> There's no way the chroot should have the snappy-dev PPA up to date already.
<cjwatson> Yeah, it's almost certainly timed out
<cjwatson> We'll hopefully get more information once it finishes timing out
<xnox> https://launchpadlibrarian.net/335415335/buildlog_snap_ubuntu_xenial_amd64_xnox-subiquity_BUILDING.txt.gz -> took 1 minute to do everything, and it has a lot of Get rather than Ign.
<cjwatson> Yeah yeah I know
<cjwatson> It's not "slow", it's broken :)
<xnox> ok, thanks.
<cjwatson> But I won't have much idea of why until it gets further
<cjwatson> (worked fine on dogfood)
<cjwatson> Huh, there it managed to get something from ppa.launchpad.net
<cjwatson> let's see what dogfood lcy01 makes of it
<cjwatson> trying https://code.dogfood.paddev.net/~cjwatson/+snap/subiquity/+build/65491
<cjwatson> Looks like it's doing the same thing
<cjwatson> wgrant: Do you have your "get shell on dogfood builder" package handy somewhere?
<wgrant> cjwatson: labbu:~wgrant/buildd-rooter.tar.xz
<cjwatson> thanks
<wgrant> You need to adjust the key in wgrant-rooter, build it, publish it, then install wgrant-rootit
<wgrant> cjwatson: Should we roll prod back for now?
<wgrant> s/install wgrant-rootit/build wgrant-rootit and wait for it to hang/
<cjwatson> wgrant: Let's.  Could you organise that?  If you could leave buildd-staging alone that'd be good.
<wgrant> cjwatson: kk
<cjwatson> I wonder if breaking out of the buildd chroot will work when it's using schroot, though.
<wgrant> I don't see why not.
<wgrant> Unless schroot has started using mount namespaces.
<wgrant> Pretty sure I put in way too many ..s for that reason.
<cjwatson> Seems to leave me in /home
<cjwatson> At least when I try locally in schroot -c unstable-amd64 -u root
<cjwatson> I guess overlayfs might matter there
<wgrant> Yeah, I see the same here, but that is aufs so is going to be pretty weird.
<wgrant> Slightly surprised it wouldn't work, but I don't see how schroot could possibly break it with the dir backend and no mount namespaces.
<cjwatson> Well, let's try I guess.
<wgrant> cjwatson: Wait no I typoed, it works fine
<wgrant> >>> import os
<wgrant> >>> os.getcwd()
<wgrant> '/root'
<wgrant> >>> os.chdir('..')
<wgrant> >>> os.chroot('/root')
<wgrant> >>> os.getcwd()
<wgrant> '(unreachable)/run/schroot/mount/artful-amd64-a0c7ae2d-d598-4f05-8b8f-2e60df09df2a'
<wgrant> >>> os.chdir('../../../../../../../../')
<wgrant> >>> os.getcwd()
<wgrant> '(unreachable)/'
<wgrant> cjwatson: All active vbuilders are 147, though lgw01 is being sluggish at coming back, as always
<cjwatson> wgrant: https://code.launchpad.net/~cjwatson/launchpad-buildd/lxd-clamp-mss/+merge/330078
<cjwatson> I don't feel too bad about not spotting that in advance ...
<wgrant> Indeed
<wgrant> cjwatson: lgtm
<cjwatson> thanks
<cjwatson> will get it onto dogfood today but I'm not going to attempt another prod rollout until next week
<wgrant> cjwatson: The non-vbuilders are probably 1500 anyway, so I suppose we need not downgrade them.
<cjwatson> z13-* are 1500 at least
<cjwatson> wgrant: Would you be OK with reversioning our currently-forked dependencies using PEP 440-compliant versions (e.g. Twisted 13.0.0-p2 -> 13.0.0.post3 or similar)?  I had another poke at my pip branch today and discovered that this would be helpful for that project; notably, unlike old pkg_resources, modern pkg_resources (and hence pip) considers zope.pagetemplate 3.5.0-p1 < 3.5.0, so much ...
<cjwatson> ... confusion arises.
<cjwatson> wgrant: I think we just need to make sure that whatever versions we use are also handled correctly by the old pkg_resources we're currently using, but that's possible.
<cjwatson> (I made some progress; I have a bin/py now, though it's currently a pkg_resources entry point which means that running it causes pkg_resources to try to resolve stuff and hence I run into the above.
<cjwatson> )
#launchpad-dev 2017-09-02
<wgrant> cjwatson: I mean if it works
<cjwatson> it actually seems to
<cjwatson> almost.  haven't got lp_sitecustomize working yet
<cjwatson> putting sitecustomize.py in the right place by hand gets bin/test working though
<cjwatson> and it doesn't totally explode as soon as it hits storm or webservice code or anything
<wgrant> Not bad.
<cjwatson> not everything's happy - bson is unpacked with totally wrong module contents because pymongo is evil beyond belief, and that makes oops_datedir_repo blow up
<cjwatson> and I need to clean up a huge pile of hackery, come up with a less painful way to manage requirements (especially the ztk and zopeapp bundles), and probably a few other things.  but I think it's now in debug-into-existence territory rather than being a big pile of fail
<cjwatson> solution to the pymongo/bson nonsense is probably to upgrade the oops stack, so running the test suite on that overnight
