#launchpad-dev 2009-11-23
 * mwhudson lunches
 * thumper lunches
 * thumper frowns
<thumper> Kontact has gone awol
<mwhudson> haha
<mwhudson> ec2 test launched an instance with the same ip as another i'd previously launchpad
<mwhudson> so ssh complained about the key changing
<thumper> heh
<thumper> chalk up another failing git import with submodules
 * mwhudson finds why ec2 instances aren't shutting down, oops
 * mwhudson kills the wrong instance by mistake :(
<thumper> bugger
<thumper> :(((
<thumper> I have a bad feeling about this...
<thumper> mwhudson: got a minute for a quick call?
<mwhudson> thumper: sure
<mwhudson> how did it get to be 17:20 ?
 * mwhudson eods
 * thumper EODs
<adeuring> goodd morning
<mrevell> Morning
<jml> mrevell, hi
<mrevell> hey there jml
<jml> mrevell, the boiler in my house is on the blink
<jml> yay
<jml> I mean, umm, ahem.
<jml> mrevell, how's things?
<mrevell> jml, Ah, that's no good and explains the "Hot water doesn't work" comment :) Thing ah okay, even if my head is slightly in Dallas still.
<mrevell> How was your first trans-Atlantic flight jml?
<mrevell> s/ah/are
<jml> safe :)
<mrevell> man alive
<mrevell> jml, heh, that's a fairly minimal commendation of a flight :)
<jml> brb
<jtv> hi al-maisan, care to help me out a little bit with orientation w.r.t. the buildd work?
<al-maisan> jtv: of course! How cam I help?
<al-maisan> *can
<jtv> al-maisan: great.  Well first of all, my understanding is that the build slave writes its output files into a directory, and then process-upload.py finds the files there & does the processing on the "real LP" side.  Is that right?
<al-maisan> jtv: yes, after the build slave finishes a job successfully, the buildd master will upload the files to whereever they are needed next
<jtv> that "upload" happens in the LP db (librarian etc.)?
<al-maisan> jtv: I was just looking at the code but, yes, I believe the librarian is used as a buffer
<al-maisan> i.e. the files go to the librarian first
<jtv> al-maisan: I won't bite if you get the details wrong :-)
<noodles775> IBuilder.transferSlaveFileToLibrarian() ?
 * al-maisan is relieved ;P
<jtv> noodles775: That sounds probable.  :-)
<wgrant> They don't normally go straight to the librarian.
<jtv> who calls process-upload.py though?
<al-maisan> jtv: cron?
<jtv> I don't see any calls to it, and it's not in cronscripts
<wgrant> buildd-manager grabs them from the slave, dumps them in a directory.
 * jml back
<wgrant> It calls process-upload.py (it's in the production configs, so grep won't find it)
<al-maisan> ah, there you go
<wgrant> process-upload.py looks in the directory, uploads stuff to the librarian, and fiddles with the DB.
<jtv> very succinct :-)
<jtv> answers pretty much everything.
<al-maisan> wgrant: thank you for explaining the workflow.
<jtv> and throws in a free use of "to fiddle with" as well
<wgrant> I guess transferSlaveFileToLibrarian might be used for the build log.
<wgrant> jtv: Sorry.
<jtv> wgrant: no I wasn't being sarcastic!
<wgrant> Oh.
<jtv> wgrant: it just sums up everything I wanted to ask (for now :) and leaves out the stuff that wouldn't mean anything to me yet.
<jtv> There are build ids encoded in the directory names, right?
<wgrant> Yes, but that's not important for anything.
<wgrant> (And they're easily faked by malicious users on the buildds)
<wgrant> The build ID is passed to process-upload.py directly.
<jtv> ah!  I thought it was the other way around: the build id being kept in the db, and the slave not trusted about things like which ubuntu release it's working for
 * jtv should have waited before talking :)
<wgrant> There's just a bug or two which means some untrusted values show up in logs and directory names.
<jtv> can't check everything all the time I suppose
<jtv> so something in a cron job figures out which builds are finished, and invokes process-upload.py on each in turn?
<wgrant> No. buildd-manager is a daemon, and it polls every five seconds.
<wgrant> It starts new builds as appropriate, and uploads complete ones.
<jtv> oic
<jtv> so how does a slave signal completion?  write a marker file somewhere?
<noodles775> wgrant: when you say uploads complete ones - doesn't it just dump the files, and process_upload takes them from there (as you mentioned above)?
<wgrant> jtv: The daemon running on the slave reports a status of WAITING.
<jtv> oh, that's done in xmlrpc?
<wgrant> jtv: Remember that buildd-manager polls the XML-RPC status() method every five seconds.
<wgrant> Right.
<wgrant> noodles775: Well, yes.
<jtv> So for generalizing the process, either the daemon could call a different script for other job types, or the process-upload script could diversify to handle them
<wgrant> The way buildd-manager handles uploads is going to change a lot soon.
<wgrant> Because at the moment it is synchronous.
<wgrant> Which really sucks.
<noodles775> yep, there's a spec for that too (which I'm sure you know ;) ).
<wgrant> Although I'm not sure it's changing in the right way, since an in-process solution will be necessary for the poppy replacement anyway.
<jtv> I didn't want to get too far ahead of things...  still exploring the existing system :)
 * noodles775 too :)
<jtv> wgrant: I may have known at some point what poppy was (in this context ;-) but if so, I forgot... are you saying it's to be generalized into the same build-job system but needs its processing to happen in the daemon rather than a child process?
<wgrant> jtv: No, no. poppy is the external FTP source upload processor. The only relation is that it needs to have done to it exactly the opposite of what is proposed be done to buildd-manager's upload processing.
<jtv> ah, was it the one that used to have (by far) our longest-running test?
<wgrant> I do not know.
<wgrant> But it breaks a lot.
<wgrant> And confuses users.
<jtv> wgrant: never mind the details, you sold me at "FTP" :-)
<wgrant> Heh.
<deryck> Morning.
<jtv> hi deryck!
<jtv> wgrant: does a slave gets a source deb or a tarball in the filesystem or librarian as its "starting capital"?
<jtv> I'm asking because I saw mention of branches as well.  The Translations jobs will want to read branches (though afaik no private ones)
<jtv> The mention I mention was in the past days' mailing-list discussions
<wgrant> jtv: It gets a number of SHA1s and filenames that it has to download from the librarian itself.
<jtv> so I guess it gets specific access to the librarian and little else
<wgrant> I imagine you'll pass in a branch URL of some kind, and the slave will check it out of bazaar.launchpad.net itself.
<jtv> networking-wise, I mean
<wgrant> Right.
<wgrant> librarian, plus the PPA and primary archives.
<wgrant> I imagine that will have to be adjusted, though.
<jtv> probably less fragile than checking out the branch, rolling a tarball, and feeding it into the librarian :)
<wgrant> Very probably.
<jtv> and if we have no private branches to worry about, the auth can be the simplest one of all: anonymous-only
<jtv> so a malicious package could subvert its build slave and get... publicly available source code
<wgrant> Right.
<wgrant> Which isn't good, but it can be done with the librarian already.
<jtv> I think the list of packages installed on the slave also came up... any idea whether that list is explicit somewhere?
<wgrant> A plain buildd-flavour debootstrap, with some added lamont magic.
<wgrant> But pretty much just the basic buildd installation.
 * jtv looks intrigued in a mountaineer's "do we really have to climb this thing?" way
<wgrant> How are dependencies going to be specified in your case?
<jtv> that's something I don't know about yet: there'll be some fixed ones I'm sure, and the package's build dependencies may come into it as well
<jtv> oh
<jtv> I do remember reading that the slave starts by doing an apt-get upgrade
<wgrant> Right.
<jtv> so presumably it can do an apt-get install just as easily
<wgrant> It can do anything.
<jtv> and that's done before any untrusted code runs, so probably not too hard to splice in
<wgrant> Right.
<jtv> actually there's something else _very basic_ that I haven't asked about (and thanks for your patience!) and that is, what about the vms & chroots?  Do we kick off vms inside a chroot each?
<wgrant> you don't know about the VMs. They just appear as normal machines that happen to be reset before a new build is started.
<wgrant> The slave then starts up inside the VM.
<wgrant> So everything happens inside the VM.
<jtv> oh, and then the package build happens inside a chroot inside the vm?
<wgrant> Exactly.
<jtv> and the vms run on different archs for the various builds, right?  (something we won't have any interest in)
<wgrant> Right.
<jtv> oh, but there must be a provision for this already since there are any/all packages
<wgrant> arch-indep ('all') packages are built on the i386 builders at the moment.
<jtv> that's fine for us, and not having to care is fine as wll
<jtv> *well
<wgrant> The problem there is that it overloads the i386 builders.
<wgrant> But no point worrying about that now.
<jtv> right
<jtv> now actually we won't really be building a package, but a branch, so all this stuff won't be presentâbut maybe it'll make sense to pretend
<wgrant> "all this stuff"?
<jtv> (sorry, distractions elsewhere)
<jtv> by "all this stuff" I meant the build dependencies and the "architecture"
<wgrant> You probably care about build dependencies, since I preusme you're executing untrusted code.
<jtv> right; we don't have the deb metadata in the branch though afaik
 * jtv hasn't even looked at deb packaging in a while
<jtv> so for those, we don't have them; we care about them; and it may make sense to pretend to some extent that we're working from an uploaded package
<jtv> (and dig up the info from elsewhere)
<wgrant> Right, in most cases you won't have a debian/control to slurp build-dependencies from.
<wgrant> But you are going to need to get them from somewhere.
<jtv> will we need them for building the pot files in the most common intltool setup?
<wgrant> I know very little about translations.
<jtv> the answer may even be something complicated like "technically yes but there's no absolute need to support that off the bat"
 * wgrant sleeps.
<jtv> wgrant: g'night and thanks!
<wgrant> np
<jtv> henninge: you had a chance to look into the intltool stuff yet?
<henninge> jtv: nope
<jtv> henninge: ah ok... I just got a lot of things explained to me... most of it not relevant to the parts you're set to work on, but one thing I don't know yet is whether I'll need to install a package's source dependencies in order to build the templates
<jtv> (in the initial case at least; I guess it becomes more and more likely as we support more cases that we'll need them)
 * jtv rests his wrists
<henninge> jtv: I wouldn't think that source dependcies are needed
 * jtv hopes not
<jtv> but not ruling it out :)
<jml> deryck, rockstar: either of you want to land https://code.edge.launchpad.net/~brian-murray/launchpad/bug-485229/+merge/15040 ?
<deryck> jml, i will.
<jml> deryck, thanks.
<deryck> jml, ec2 had test failures so need to chase those and then will land.
<jml> deryck, ahh ok.
<jml> deryck, Brian asked me in person what the deal was with landing so I thought I'd chase it up.
<deryck> jml, yeah, no worries.
* salgado changed the topic of #launchpad-dev to: This is Launchpad Development Channel | Week 3 of 3.1.11 | PQM is open  | I am Zero OOPS and So Can You! http://is.gd/4fkLl |   https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in #launchpad-reviews | Use http://paste.ubuntu.com/ for pastes | This channel is logged: http://irclogs.ubuntu.com/
 * jml -> errands
<sinzui> salgado: ping
<salgado> hi sinzui
<sinzui> salgado: do you own a mirror in production?
<salgado> sinzui, yep
<sinzui> staging is not updating so I cannot test that I fixed the change owner permissions
<salgado> I'll test it on edge
<sinzui> salgado: thanks
<salgado> sinzui, it works!
<sinzui> \o/
<flacoste> morning folks
<sinzui> EdwinGrubbs: bug 297833
<mup> Bug #297833: Attempting to invite a private team fails with "Constraint not satisfied" <oem-services> <oops> <Launchpad Registry:Triaged> <https://launchpad.net/bugs/297833>
<jml> beuno, hello
<jml> flacoste, hello
<flacoste> hello jml!
<flacoste> call in 5?
<beuno> jml, hey
<jml> flacoste, indeed
<jml> beuno, guess what I just bought
<flacoste> silent headphones!
<beuno> jml, PALM PRE!
<jml> beuno, \o/
<mars> those work in Argentina?
<beuno> jml, woooooooooooooo
<beuno> jml, and?  I can't see your happy face through IRC very well
<jml> beuno, well, I haven't actually got it to do anything yet
<flacoste> yeah, staging was updated!
<flacoste> everyone can clear their QA queue!
<jtv> beuno, have you seen this sprite problem before? https://translations.edge.launchpad.net/ubuntu/karmic/+source/apturl/+pots/apturl/av/+translate
<beuno> jtv, I haven;t
<beuno> jtv, the sprite is applieed on the wrong element
<beuno> (on the div, on the p)
<beuno> or something like that
<jtv> beuno: I don't see the sprite class on anything in the dom so far... do you see it?
<sinzui> beuno: jtv: I still see developers doing that
<sinzui> jtv: the sprite class just sets the padding, but you are using the sprite TYPE class
<beuno> sinzui knows all about it
<beuno> btw, sinzui, are you still doing UI reviews?
<jtv> ah, this one says "error message" as classes for the div
<beuno> I'd like to get us back to the graduation process  :)
<sinzui> jtv: If a CSS class shows an image, it is a sprite that is designed to be on an inline element with the default line-height
<sinzui> beuno: I have. The only one of interest though I did with noodles775 for henninge.
<beuno> sinzui, please continue adding me to the MPs and pinging me
<jtv> sinzui: in this case it's the "error" class that produces the spriteâbut I think it's there to produce the big red "stop" sign
<sinzui> jtv: error is the big sign that is used in notifications
<jtv> sinzui: that's the one I'm talking about, yes
<jtv> I don't really see what this div does differently from other ones, apart from having more text
<jtv> e.g. product-purchase-subscription.pt (arbitrary example) seems to do much the same thing
<jtv> ahh, p vs div maybe
<sinzui> jtv: p and div and both block elements
<sinzui> jtv: we need to copy the notifications markup
<jtv> sinzui: copy from where?
<sinzui> notifcations are global lp/app/template/base-layout-macros.pt
<sinzui> jtv: Consider that form errors and and information notifications display correctly, so that is the markup we want to copy
<jtv> sinzui: thanks, I'll see what I can figure out from there
<jtv> filing bug first
<mars> deryck, around?
<deryck> mars, hi
<mars> hi deryck, would you have some time to review my one-line CSS change to fix bug #484848?
<mup> Bug #484848: Subscriber icons in the subscribers portlet appear on a separate line from the subscribers' names <bug-page> <ui> <Launchpad Bugs:In Progress by mars> <https://launchpad.net/bugs/484848>
<deryck> mars, sure
<mars> done
<sinzui> beuno: jtv: firebug shows each paragraph in that block is inheriting the div's .error .message classes. That is wrong
<jtv> right, it's the surrounding div that currently has those classes
<jtv> should I use view/notifications instead?
<deryck> mars, Looks good.  r=me.
<mars> deryck, ok.  beuno, ping, care to review my CSS hack?
<beuno> mars, sure, just waiting for the diff to come up
<mars> deryck, I think there is a better fix, by applying the correct styles to the surrounding <a> tag
<mars> however, I ran face-first into a limitation of our framework - we can't call something like person/fmt:css-styles-that-should-be-applied
<beuno> mars, just one question, why do we need to special-case subscribrers?
<deryck> mars, I did add a question to the review for the need for XXX, but will defer to your final judgment on that.
<mars> beuno, the markup in the subscribers portlet is such that the sprite gets assigned a 90% width value
<mars> beuno, an it also gets assigned the inline-block style.  Those push the sprite and the text onto different lines.
<mars> This pulls the text back inline.  The correct fix is to change the markup, but like I said, our framework doesn't allow for that.
<beuno> mars, gotcha
<beuno> mars, I'm +1, said so on the MP
<sinzui> Chex: Do you know why staging has not updated in 2 days?
<mars> to be honest, what should be a trivial fix was very difficult.  There were a dozen ways to do it, and even when I chose a way to do so, I didn't know if it was right or not.
<mars> and it took a lot of investigation - many hands have touched that code for specific reasons.  You can't touch it without worrying about regressing someone else's work.
<mars> sinzui, didn't flacoste say earlier that staging had finally updated?
<sinzui> It was update friday. and was not updated 1.5 hours ago
<mars> beuno, deryck, thanks for the reviews.
<sinzui> mars: staging is definitely stale.
<deryck> mars, np
<mars> abentley, around?
<abentley> mars: Hi.
<mars> hi abentley.  We are doing the YUI 3.0.0 upgrade for Launchpad, and it is live on staging.  We need help testing the pages for errors.  Would you or rockstar be able to give us a hand?
<abentley> mars: I guess so.  What do you want me to test?
<mars> abentley, well, anywhere that you came up with custom javascript, that windmill may not cover so well.
<rockstar> mars, I'm unavailable this week.
<rockstar> mars, why can't you just run the windmill tests?
<mars> rockstar, ok, thanks for the heads up.
<rockstar> mars, I think if you just pass the staging url to the CodeWindmillLayer tests you should be to use our windmill tests against staging.
<mars> rockstar, if you feel that is sufficient, then OK.  I can't be sure what your windmill-to-feature coverage is like.
<rockstar> mars, we've been pretty strict about making sure that at least the interaction is tested.
<mars> rockstar, abentley, ok.
<abentley> mars: Replying to a code review comment doesn't work because "Error: Y.one("label [for=field.vote]") is null"
<mars> abentley, alright, first question then: did the windmill test catch it?
<abentley> mars: No.
<intellectronica> i don't think you can use dots in identifiers in selectors
<mars> bin/test --layer=CodeWindmillLayer --test=???
<mars> right
<mars> that little issue :(
<mars> abentley, ok, please file a bug, tag it yui-3-upgrade
<BjornT> intellectronica, mars: you could use a dot in selectors in the previous yui version we used.
<mars> BjornT, ah, but can you do so now?
<BjornT> mars: iirc bug commenting selects by an id containg a dot
<BjornT> mars: yes, Y.get('[id="field.comment"]'); works
<mars> abentley, ^
<mars> maybe it is the quoting?
<abentley> mars: Isn't this a Y.one vs Y.get issue?
<mars> abentley, they should be the same function.  Y.get() is deprectated.
<jml> beuno-lunch, my palm hard locked during an update
<BjornT> anyone seen this psycopg error before? http://paste.ubuntu.com/326281/
<mars> abentley, BjornT, those selectors worked for me, with and without quotes.  abentley, I suggest checking that the element is actually in the HTML.
<abentley> mars: I did.
<abentley> mars: And it works on edge.
<BjornT> abentley-lunch: where are you seeing this failure? latest devel works for me
<abentley-lunch> BjornT: On staging
<BjornT> abentley-lunch: at what url? staging worked for me as well
<BjornT> abentley-lunch: i tested it here: https://code.staging.launchpad.net/~bjornt/launchpad/form-overlay-render-by-default/+merge/10249
<sinzui> EdwinGrubbs: I volunteered your time to help QA https://dev.launchpad.net/LaunchpadTestPlan/3.1.11 lazr-js items in exchange for a a speedy review
<EdwinGrubbs> sinzui: yeah, I heard that on the call. Is the only page that I need to look for tests on? I didn't know we were still tracking bugs on "Launchpad itself".
<sinzui> That page is the failover. testplans are bogus because they claim to test applications, but it is actually team members
<sinzui> EdwinGrubbs: All my answers and blueprints fixes were wrongly assigned to the the registry app for example. The Launchpad page is for fixes landed by a non-launchpad team member
<salgado> intellectronica, I just found an issue with the save/cancel buttons when editing a bug's description. the buttons move to the middle of the screen after clicked: http://people.canonical.com/~salgado/edit-bug-description.png
<salgado> is that a known problem or should I report it?
<jml> beuno, but now it's working again, thanks to the very cool people in #webos
<mwhudson> jml: you have a pre?
<beuno> jml, wooooo. What happened?
<salgado> deryck, maybe you know (^)?
<mwhudson> salgado: i think it's yui3 fallout, you can see on staging but not on edge, not sure if there's a bug though
<jml> beuno, I tried updating to webOS 1.3.1, it failed.
<beuno> salgado, I suspect it's the newer version of that widget not playing nicely with the way it's implemented in LP
<jml> beuno, I had to use their firmware fixer thingy, which of course doesn't work with 64 bit linux
<salgado> mwhudson, good catch.  I'll check with mars to see if he knows about it
<salgado> mars, ^
<mars> reading
<mars> salgado, could be YUI3, or lazr-js changes from the sprint.  Difficult to isolate :(
<salgado> mars, yeah, as I imagined.  I'll see if it affects any other places where we use a multi-line widget and report a bug
<beuno> salgado, mars, it is
<mars> salgado, first, try the lazr-js example
<beuno> deryck made the widget fixed width or scalable
<mars> it is sort of the same
<beuno> my guess is that Launchpad's implementation assume the old implementation of it
 * deryck is on the phone right now, will look shortly
<salgado> mars, both the fixed and fluid examples in lazr-js work fine
<mars> so it is an integration issue.  good, now we know where the effort must be focused.
<mars> salgado, the widget still works, right?
<mars> on launchpad?
<salgado> mars, yes
<mars> also good!  so it isn't fallout from another subtle bug.  It is a CSS-only fix.
 * deryck looks at what salgado and beuno were saying about description widgets
<mwhudson> morning
<salgado> deryck, bug 487263
<mup> Bug #487263: Save/Cancel buttons move to the middle of the line when editing the description of a bug <Launchpad Bugs:New> <https://launchpad.net/bugs/487263>
<deryck> salgado, (and beuno, if interested), I would guess it's the one bit of local CSS on the bug page.  The fonts made lining up hard initially, so had to hard code something for that top line.
<deryck> I would guess this could be dropped now.
<salgado> if it can be dropped, it'd be perfect
<beuno> deryck, that would be my guess as well
<mwhudson> gar, landing the bzr-svn code import worker failed again :(
 * deryck pulls db-devel
<mwhudson> ffs it passes locally :(
<thumper> morning
<thumper> mars: morning
<thumper> mars: have you looked at editing a commit message on staging?
<thumper> mars: the styles are all fubared
<thumper> mars: haven't checked editing a bug description
<thumper> mars: but I suspect it may be the same
<mars> thumper, salgado is looking into it
<thumper> mars: awesome, ta
<salgado> thumper, bug 487263
<mup> Bug #487263: Save/Cancel buttons move to the middle of the line when editing the description of a bug <bug-page> <js> <ui> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/487263>
<thumper> salgado: ta
<bigjools-afk> morning thumper
<deryck> salgado, I don't think it's the local rule after all. not sure what, though.  and I was just about to step out for the day, until later tonight.
<salgado> deryck, since it affects editing commit messages as well, I suspect it's something else. I won't be around for long either, so maybe thumper can investigate it
<thumper> salgado: do you have anything for me to go on?
<beuno> salgado, deryck, I think it's the padding-right on .yui-ieditor
<thumper> mars: what is the process for merging yui3 back into devel when we need to fix bits?
<thumper> beuno: morning
<thumper> beuno: how long are you around for?  I'd like a chat
<thumper> beuno: but I'm tied up for about the next hour and a bit
<beuno> thumper, hey, I'm around for an hour or two
<mars> thumper, good question.  flacoste, were we planning to land yui-3 fixes on db-devel, or should I keep an integration branch?
<deryck> ok, got to be out for a bit.  catch you all later.
<salgado> thumper, nothing, really.  I just stumbled upon it but didn't dig any further
<thumper> salgado: ok
<thumper> who knows zope the most?
<thumper> if we have IExtended inheriting from IBase
<thumper> and register a utility using IExtended
<thumper> and ask for the utility with IBase
<thumper> will it get the IExtended object?
<mwhudson> i don't think so
<mwhudson> but try it, i guess
<mwhudson> aargh
<mwhudson> so the reason bzr-svn-imports tests pass locally but fail remotely is that i have python-xdg installed
<mwhudson> preeettty ooobscure
<thumper> mwhudson: what is python-xdg?
<mwhudson> thumper: not really sure
<mwhudson> but bzr-svn has a try: except: around it's import
<mwhudson> and the reason ec2 land didn't work on my ec2 test branch was because i hadn't pushed a revision but just ran "ec2 land" and didn't look a the terminal again :(
<thumper> mwhudson: I'd file a bug on bzr-svn :)
<mwhudson> thumper: well, it's caused by an underlying bzr bug really
<mwhudson> (which i'm fixing)
<thumper> mwhudson: :(
<dobey> thumper: python-xdg is a python module which makes using the xdg specs easier for a lot of things (XDG Base Directory spec for example)
<wgrant> Everyone should use it.
<dobey> except for people on !linux
<dobey> it doesn't really make a whole lot of sense on windows or osx
<wgrant> On !freedesktops, you mean.
<dobey> no
<dobey> i mean /usr/share/ doesn't exist on windows :)
<wgrant> But it does on other UNIXy systems.
<dobey> xdg doesn't take into consideration filesystem layouts which aren't FHS really
<dobey> well there are a couple of distros where i don't think it would work well either, that try to simulate the osx layout on top of linux or bsd, for example
<dobey> but they also have like 2 users, so not really a worry :)
<thumper> beuno: still around?
<beuno> thumper, yes
<thumper> beuno: skype?
<beuno> thumper, sure
<beuno> thumper, ready
<mwhudson> wow, launchpad currently prevents non-ascii branch names by having about 7000 bugs in the area
<mwhudson> (and an actual policy somewhere i'm sure)
<mars> lol
<dobey> mwhudson: yeah, i tried to name a branch â¥ once, but it wasn't happy
<mwhudson> argh, one of these problems would be fixed in python 2.6 :)
<dobey> heh
<dobey> it will be nice in 10 years when we're all finally using python 3 :)
<mwhudson> now bzr crashes on the client side \o/
<mwhudson> oh maybe not
<mwhudson> falling over in the smart server now i think
<mwhudson> lawl
<mwhudson> bzr: ERROR: Permission denied: "~mark/gnome-terminal/ddÃ©/": : Invalid%20branch%20name%20%27dd%C3%A9%27.%20Branch%20names%20must%20start%20with%20a%20number%20or%20letter.%20%20The%20characters%20%2B%2C%20-%2C%20_%2C%20.%20and%20%40%20are%20also%20allowed%20after%20the%20first%20character.
<wgrant> Nice.
 * mwhudson is tempted to just give up
<mwhudson> mwh@grond:unicod-branch-names-bug-449528$ bzr push  lp://dev/~mark/gnome-terminal/dÃ© -d ~/tmp/german/
<mwhudson> bzr: ERROR: Received bad protocol version marker: 'u"Invalid branch name \'d\\xe9'
<mwhudson> oh
<mwhudson> my stupid fault
<mwhudson> i wonder if it would be practical to use a different xmlrpc lib for python
<bigjools-afk> hey wgrant, still not got to your branch, sorry.  I will do it tomorrow, promise (I am still in Texas BTW)
<wgrant> bigjools-afk: Ahh, I was wondering why you didn't appear last night.
 * bigjools-afk wonders why I am -afk
<wgrant> bigjools: So, yeah, I think jdong would hunt me down and shoot me if I asked for a dpkg backport.
<bigjools> hmm :/
<bigjools> I wonder if we can get someone in the distro team to do it
<wgrant> Indeed.
<bigjools> after all, it would unblock them, so you'd think they'd have a vested interest
<wgrant> Yes.
<bigjools> I wll poke someone tomorrow
<wgrant> And it's a simple merge, so it wouldn't take much of their time.
<wgrant> Thanks.
<bigjools> nay wurries
<wgrant> 3.0 penetration will exceed 1% later today. Debian is actually moving quickly for once.
<bigjools> eek
<bigjools> ok
<lifeless> dpkg changed its default behaviour
<wgrant> Er, when?
<lifeless> at least, thats what I thought ;)
<wgrant> I see no mail to d-d-a about that, and it wasn't the case a couple of days ago.
<lifeless> wgrant: I must have gotten the wrong idea from an irc conversation
<wgrant> They were going to do it.
<wgrant> But then an #ubuntu-devel conversation suggested that it wasn't such a good idea.
<wgrant> bigjools: I also fixed gina a couple of days ago, so I think all the Soyuz end of things is done.
<wgrant> And exams are done tooooo.
<bigjools> woo
<bigjools> excellent, with any luck we'll get everything landed this week so we're ready for release next week
<wgrant> Yep.
<wgrant> As long as lamont is on the buildd side of things.
<spm> wgrant: I believe he is; had a chat with him yesterday arvo; he seemed cool and froody with it all
<wgrant> spm: Great.
<wgrant> Although powerpc will be Fun.
<bigjools> spm: froody?
<spm> bigjools: you don't know your hitchhikers. for shame. :-) http://www.urbandictionary.com/define.php?term=froody
<bigjools> indeed I never read/watched that
<wgrant> O_O
<bigjools> it's not my bag, baby
 * wgrant prefers the original radio series.
<spm> wgrant: I'll have a chat with the powers that be; see if we can get bigjools suspended until he's forcibly watched, read and listened to the lot
<wgrant> spm: Excellent idea.
 * spm is fond the radio series as well - but prefer the books. the pictures are better.
 * bigjools goes to monster.com
<spm> heh
 * wgrant quickly adds "Hitchhiker's Guide expertise a must" to all the descriptions.
<spm> my superb wife actually bought me the hardcover of the 6th book in the trilogy for my birthday. Not written by DA aiui, but a fan with assist.
<wgrant> Well, yes, he's not awfully good at writing any more.
<spm> no. :-(
<thumper> anyone know where some inpage help that works is in LP?
<wgrant> thumper: Check a PPA page.
<wgrant> thumper: 'Help with installing'
<wgrant> Although "works" isn't quite true.
<wgrant> It has an AJAX spinner stuck in the middle for some reason that I cannot fathom.
<mwhudson> bah, postgres doesn't have "median"
<thumper> hew
<bigjools> wgrant: yui3 borked it
<bigjools> noodles is on the case
<thumper> bigjools: yui3 isn't on devel
<wgrant> bigjools: It has been like that for a few weeks now.
<bigjools> oh this is edge?
<thumper> bigjools: so that isn't it
<bigjools> hmmm
<wgrant> And prod, IIRC.
<bigjools> hmm x2
<wgrant> Yeah, prod as well.
<bigjools> isn't that something to do with the missing styles as well?
<wgrant> Possibly. Something isn't unsetting the background image.
<wgrant> Which could conceivably be another style, I guess.
<bigjools> it sounds like a corrupted icing dir location
 * bigjools EODs
<wgrant> Night.
<bigjools> cheers, night all
<mwhudson> spm: what's your R magic qqplot oneliner?
<spm> mwhudson: something like: qqplot(c(seq(0,1, by=0.001)), y2, main="Page Responsiveness", xlab="Percentiles", ylab="Time in Microseconds", type="p",tck=-0.01,col="blue", lab=c(20,20,20), sub="2006 Week 27")
<spm> where y2 is the data being analysed
<spm> fiddle around with by=0.001 to get finer/coarser views
<spm> the rest is mostly fluff
<mwhudson> ta
<spm> mwhudson: I find using quantile() helps show where the points of interest are, and thus can fiddle to that; eg quantile(y2, seq(0, 1, by=0.05))
<wgrant> lifeless: Looks like we're safe from the default format changing for now; a GR to prevent it has even been threatened. Yay Debian.
<mwhudson> spm: tanks
<mwhudson> spm: 90% of successful code import runs complete in 90s
<mwhudson> 50% in about 30
<spm> sweet!
<wgrant> And linux in two months.
<mwhudson> well, most import runs are just updates
<mwhudson> i guess i could filter in/out the initial runs
<mwhudson> probably a fairly expensive bit of sql though
#launchpad-dev 2009-11-24
<wgrant> Weren't the "technical difficulties" resolved yesterday?
<spm> if you mean a somewhat critical server that haemorraged? yeah. whats up?
<wgrant> It's still in the #launchpad topic.
<spm> haha. thanks! :-D
<ajmitch> 'technical difficulties' is such a useful term
<wgrant> Yep.
<ajmitch> it covers everything from things going a bit slow to the data centre burning down
<wgrant> Or just SKS being crap, in this case..
<ajmitch> which managed to hold up the appserver?
<wgrant> Yes, all of them, I believe.
<ajmitch> ouch
<mwhudson> it wasn't sks' fault
<mwhudson> (this time)
<maxb> How long is it supposed to take for Launchpad to notice a new source upload to Debian?
<wgrant> It is meant to look twice a day, or maybe four times if the sysadmins have actioned that RT.
<wgrant> Note that some appeared to be failing a couple of weeks ago, and slightly over 1% of the archive is currently unimportable because of the format transition.
<maxb> It's times like these when it would be handy for the production crontabs to be opensource
<thumper> mwhudson: probably not best to reply to Bug 236973 with "oh crap!"
<mup> Bug #236973: the interval between launching a new code import job should be more flexible <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/236973>
<mwhudson> thumper: no, probably not
 * spm subscribes losas to that one
<spm> mwhudson: fwiw, that "cheat" in the crontab is evil; so just getting rid of that would be wonderful.
<mwhudson> spm: so I guess you wouldn't be super happy for me to suggest fudging it harder?
<spm> meh
<spm> :-)
<mwhudson> thumper: the fix proposed in the description would be very simple and certainly help a bunch
<spm> mwhudson: srsly tho - fudge in what way? we could do some things that wouldn't be quite so ... yuk, but still be a fudge.
<mwhudson> spm: well, s/sleep 30/sleep 20/ and similar
<mwhudson> spm: probably better to do it in the script and avoiding paying the startup cost
<spm> 'k. a proper wrapper script vs evil.
<spm> right
<spm> mwhudson: have we discovered/can-find the actual startup time of the python code to make stuff happen? that'd assist in giving a lower bound.
<mwhudson> spm: it's probably 1.something seconds in this case
<spm> oki
<mwhudson> spm: you could look for the times of the requests to the CodeImportScheduler endpoint on arsenic
<mwhudson> that's the first actual useful thing the script does
<spm> as an offset from 0 or 30? yeah! good idea!
<mwhudson> so if requests come in at xx:xx:05, it's a 5 second startup
<spm> nods
<spm> mwhudson: eyeball suggests ~ 3-5 secs off :00 and ~ 36-38 off 30 - but I'd suggest the latter is simply due to the effect of the 1st startup etc
<mwhudson> spm: right, that makes sense
 * spm ponders if breaking out R and Doing It Right is worth it.... ;-)
<spm> mwhudson: thinking aloud here - so shoot down if needed - but I wonder if having a random offset sleep wuold be better. eg a random time between 5-15 seconds - would go some small way to avoid the thundering herd of 3 machines all doing the same stuff at the same moment.
<mwhudson> spm: yes, that's certainly possible
<spm> and being random, would rely on us losas having to sync eg 15 machines down the track with subtle offsets
<spm> *wound't*
<spm> wouldn't even bleh
 * wgrant wonders how they could place so much load on anything in order to make that necessary.
<spm> mainly as evetually you will; better to design in the workaround up front
<spm> peak loads always bite you hard when you least want them too :-/
<spm> esp as these sort of things tend to not impact linearly - more like exponential; (not) literally the straw breaks the camels back
<spm> I think it was cnet? one of the big news orgs. in the earlier days of RSS, they used to get *hammered* at specific times. all due to gazillions of teeny trivial requests for rss feeds from clients using the same s/w that was hard wirded to timee X; vs time X + random
<mars> so, bug tags don't apply across projects?  I can't click the "yui-3-upgrade" tag to see all problems across all of LP??
<mars> ah, to see all of the bugs with a particular tag across projects, I need to search for that tag on the super-project.
<mars> jtv, ping
<jtv> mars: yes?
<mars> hi jtv, just wondering, I need someone to help test the translations JavaScript upgrade on staging.  Would you be willing to do so?
<mars> or perhaps volunteer a teammate? :)
<jtv> I can help, but hang on
<jtv> got a bit of a situation involving contact lenses :)
<spm> jtv: you pinged last night? I gather all is solved? or ... ?
<mars> :)
<jtv> spm: yes, thanks
<spm> sweet
<jtv> mars: I know one of ours broke with the upgrade (something I wrote in fact), but last night db-devel already had that code changedâby henning I guess
<jtv> ahhh! staging's had a code update
<jtv> mars: so... what is it you want to test?
<spm> jtv: gawd I hope so - it's been runing for long enough.....
<jtv> spm: a code update?  I thought those were very lightweight?
<spm> was a full one. DB ++
<jtv> ah well that takes a while yes :)
<mars> jtv, everything in translations that might not be covered by windmill.
<mars> jtv, if you do find anything, could you please tag it as "yui-3-upgrade"?  That way we can see what blocker we have for the next rollout.
<jtv> mars: ok... we don't have much, so it shouldn't take long
<mars> jtv, cool, thank you for the help!
<jtv> np
<jtv> mars: oh, and there's something unrelated I've been wondering about
<jtv> the spinner
<jtv> I can see why it's not a sprite,
<jtv> but that does mean it can take a while to load
<jtv> so what I did on the import queue is put a spinner at the bottom of the page while the JS is setting things up.
<jtv> It really helps, but... have I been re-inventing the wheel?
<jtv> I should be clearer:
<mars> jtv, soyuz had the same problem.  Each page has to decide when to load that image so that it is available.
<jtv> I made the JS load the spinner early on (by showing it at the bottom of the page) so that when it's actually needed, it's in the browser cache
<mars> At some point, we hit the lower limit of our ability to serve files over-the-wire.
<lifeless> mars: well, that limit can be lowered :)
<jtv> the lower limit is zero... not sure I follow
<jtv> lifeless: GMTA
<mars> jtv, once someone caches the spinner, it should be available right away, no?
<mars> are you only worried about initial page loads?
<jtv> I find the spinner needs to be re-fetched fairly often
<jtv> spm: there is something _else_ you could help me with...
<spm> jtv: sure
<jtv> spm: I need Q/A on a bit of UI that's currently only available to admins
<jtv> spm: e.g. here: https://translations.staging.launchpad.net/ubuntu/+source/apport/+custom-language-codes
<spm> jtv: so you want rubber ducky to give it a whirl?
<jtv> do you see a list there that you can add items to and remove items from?
<jtv> I am not familiar with this rubber ducky you refer to
<spm> jtv: the rubber duck is the icon of LP admins
<spm> https://edge.launchpad.net/~admins
 * thumper EODs
<jtv> thumper: g'night
<spm> jtv: so that came up with "No custom language codes have been defined." and "Add a custom language code " with a green plus
<spm> night thumper
<spm> selecting the + then gave a list of langs to choose from
<jtv> spm: good so far
<jtv> spm: maybe I should just join that team... it's got open membership anyway
<spm> jtv: whatever you think is best? it's only for a short period anyway :-)
<jtv> spm: I was lying for comic effect... it's restricted.  :)
 * spm is scored upon. again. :-)
<jtv> hardly!
<jtv> you didn't throw a tantrum
<jtv> (unfortunately, but that's another matter)
<jtv> anyway, can you now click into the custom language code you just created to view it?
<spm> jtv: https://translations.staging.launchpad.net/ubuntu/+source/apport/+customcode/spm  spm-is-ace (hahahahah)
<jtv> mars: I was wondering if maybe we should have some easy-to-use boilerplate for "go fetch my spinner in case I need it later."  To be done at the very end of loading everything else.
<spm> jtv: "For apport in ubuntu, uploads with the language code âspmâ are associated with the language Acehnese"
<jtv> spm: yup, works for me too.
<spm> sweet
<mars> jtv, I just tried loading the bugs page on edge, and the spinner was cached by the browser.  Of course, that doesn't mean something something else could not be blocking it from rendering instead
<jtv> spm: and as is appropriate, I don't see the "remove" link but you probably do
<spm> jtv: sure do. give it a twirl?
<jtv> spm: oh yes... the red icon shown in the listing, please!
<mars> jtv, we thought about that for soyuz, but we did not have a second case for a while.  Bugs also has a spinner, but I don't know if they optimized it as well.
<jtv> mars: some pages you end up reloading a lot
<spm> jtv: "Removed custom language code 'spm'." == gone
<mars> jtv, reloading the page contents, or the spinner graphic?
<jtv> mars: the page contents, but my impression was that that made the spinner appear slowly as well
<jtv> spm: cool!
<jtv> spm: now, the same thing should appear also in a different kind of context:
<jtv> https://translations.staging.launchpad.net/expono/+custom-language-codes
<jtv> spm: that's a project, whereas the previous one was a package
<spm> kk, go thru the same process?
<mars> jtv, it could.  Depends on what is in the page.  For instance, putting <script> nodes at the bottom could speed things up.
<jtv> spm: please!
<spm> jtv: https://translations.staging.launchpad.net/expono/+customcode/spm
<spm> will remove on your nod
 * jtv checks one little thing first
 * jtv looks satisfied & nods at spm
<spm> is gonÃ©
<jtv> spm: and for our last trick, could you verify also that the links to these overview pages appear on the following pages?
<jtv> https://translations.staging.launchpad.net/ubuntu/+source/apport
<jtv> https://translations.staging.launchpad.net/expono/
<jtv> In both cases, the link should be in a little paragraph at the bottom of the right-hand column
<spm> jtv: no on the first[1]; yes on the 2nd "If necessary, you may  define custom language codes  for this project."  [1] or I can't find it....
<jtv> spm: I always have trouble finding it on source packages myself tbh...
<jtv> spm: ah, I think that's because I put it here:
<jtv> https://translations.staging.launchpad.net/ubuntu/karmic/+source/apport
<spm> right. there it is. same general place as the 2nd
<jtv> spm: wonderful!  Another qa-needstesting tag bites the dust
<spm> sweet
<spm> after that gruelling (***gruelling!!!*** i tells ya!) session of clicking on a web page; /me goes to make a cuppa tea
<jtv> spm: thanks!  you've earned it
<jtv> just one cookie, mind you
<jtv> hey stub
<stub> yo
<spm> yo stub
<spm> jtv: just one? awwww.
<jtv> spm: no worries, we'll do more of this some other time and you can have another
<spm> oh thankyouthankyouthankyouthankyouthankyouthankyouthankyouthankyouthankyouthankyouthankyouthankyou
<jtv> go now, and enjoy your tea
<spm> it's seeping as we speak. figured aftr that hard work I needed the stronger one, so went for the assam.
<jtv> mars: I just noticed that the notification box for login failure also has that ugly sprites problem
 * mwhudson is really severely tempted to switch to gmail for all his mail
<mars> jtv, probably present on edge as well
<mars> mwhudson, until you have one of those days where it is slow
<jtv> mars: I guess... I happened to see it on staging
<mwhudson> mars: well what i really want is a desktop client with the good ideas stolen from gmail
<mwhudson> i think
<jtv> "keep it in beta so people can't complain"?
<mars> mwhudson, there was a thread on hackernews about that being the next big thing for the GNU guys, now that they have an OS
<mars> not Gmail specifically, but good webapps for personal use
<mwhudson> mars: erm, that sounds a little bit like the opposite of what i'm asking for
<mwhudson> unless the idea is you run the server on localhost
<mars> heh
<mars> well, yes.  Or on your personal server
<mars> same as people run bip now
<mwhudson> of course there's this notmuch thing
<mars> notmuch thing?
<mwhudson> http://keithp.com/blogs/notmuch/ http://notmuchmail.org/
<mars> mwhudson, interesting.  I wonder if the same principle behind sup is how they constructed raindrop?
<mwhudson> my turn
<mwhudson> mars: raindrop?
<mwhudson> oh that
<mars> I don't buy into the Raindrop UI changing the world, but the backend has potential as a platform.
<jtv> mars: found 1 thing broken with YUI3 upgradeâbug 487428
<mup> Bug #487428: Mouse-overs in template links broken by YUI upgrade <ui> <yui-3-upgrade> <Launchpad Translations:New> <https://launchpad.net/bugs/487428>
<mars> A simple mechanism for plugging all of your remote accounts into a local datastore, then viewing that datastore with whatever local apps you want.
<mars> jtv, ah, that might have been me!  I changed the code to use the new Y.delegate() mechanism.
<mars> Saves much processing.
<mars> Lacks much testing.
<jtv> *cough*
<jtv> :)
<jtv> spm: are emails not going out?  I tried to reset a password in LP and got no email
 * mwhudson wants sqlflakes
<jtv> mars: I don't see how anything delegates-related comes in here, so I wouldn't suspect you of this one.
<spm> jtv: how long ago? and was it to a gmail and/or hotmail like address?
<jtv> spm: twice in the past hour or so, and no.
<spm> jtv: worked for me. practically instant.
<jtv> gah
<jtv> spm: ah, since this was Q/A I was probably stupid enough to do it on staging!
<spm> um.... :-)
<spm> jtv: s/ignorant/distracted/ ;-)
<spm> ignorant/stupid whatever :-)
<jtv> very charitable :)
<spm> jtv: I can hardly complain. I spent ~ 5-10 mins yesterday wondering why a "tar cf --option tarfile files" wasn't working. face? meet palm.
<jtv> spm: not working?  Didn't you get a nice tarfile called --option ?
<spm> yes.... it was the "where are the (*&^(*&ing backups?!!?!?"
<spm> the error message about "tarfile" being unable to be fstat'ed too was the other weirdness. :-D
<mars> spm, how much trouble is it to patch and/or upgrade our AWStats instance?  I would like to track the numbers for Chrome.
<spm> mars: no idea. I'd suggest raising an RT
<mars> spm, k
<mars> ok, I'm done.  Good night!
<spm> g'night mars!
<jtv> mars: good night!
 * mwhudson eods
<spm> night mwhudson
<jtv> g'night mwhudson
<spm> hey al-maisan!
<al-maisan> hello spm :)
<jtv> hi al-maisan!
<al-maisan> how are you?
<al-maisan> hello jtv!
<stub> Exception: apr-config not found. Please set APR_CONFIG environment variable
<stub> Thats a new one today!
<stub> Anyone fixing the stable -> db-devel conflict in translations.js?
<jtv> stub: I'll take a look, thanks
<stub> jtv: I've already got a patch landing - hopefully I guessed correctly ;)
<jtv> stub: oh, I just wrote one up as well
<stub> looks like it is playing now
<henninge> Is anybody onto the "APR_CONFIG" build failure?
 * henninge re-runs rocketfuel-setup.
<henninge> No joy.
<henninge> al-maisan: Did you solve the APR_CONFIG failure?
<al-maisan> henninge: unfortunately, no.
<henninge> al-maisan: Guten Morge, Ã¼brigens... ;)
<henninge> n
<al-maisan> henninge: Hallo :)
<henninge> al-maisan: too bad, I am stuck on that now
<al-maisan> henninge: stuck while doing what? "ec2 test"?
<henninge> al-maisan: make build
<al-maisan> ah
<jml> Turn around bright eyes!
<jml> uhh, I mean, Good Morning Launchpadders
<al-maisan> :)
<henninge> Hi jml, great to see you are in a good mood ... ;)
<henninge> al-maisan: I just ran update-manager and is *has* an update for libapr !
<al-maisan> henninge: oh, what a coincidence ;)
 * al-maisan checks on his side
<henninge> it's still downloading updates
<henninge> oh, and here comes a new kernel
 * henninge will have to reboot in a minute
<al-maisan> henninge: a new kernel for karmic?
<henninge> 2.6.31-15
<henninge> yup, "Reboot required"
<henninge> al-maisan: back in a sec ;)
<al-maisan> henninge: sure, enjoy rebooting :)
 * henninge will have to let the filesystem check run through sometime ....
<henninge> but there never is time ...
<henninge> al-maisan: ok, let's do "make build"
<al-maisan> yup
<henninge> looking good
<henninge> al-maisan: yes, all is well again :-D
<al-maisan> henninge: great!
<mrevell> morning
<al-maisan> Good morning mrevell
<jml> mrevell, good morning
<mrevell> how's the Pre working out jml?
<jml> pretty good so far
<jml> haven't yet figured out how to get my mammoth lists out of emacs and onto it
<mrevell> I'm sure it won't be long until there's an emacs port for it, heh
<jml> also, I had this horrible incident where it borked on upgrade. The guys on #webos were very helpful in getting it back up.
<mrevell> Do Pres upgrade over the air?
<jml> yes.
<jml> except not this one just then :)
<jml> brb
<mwhudson> al-maisan: ec2 test should select image 110 now, can you try it?
<al-maisan> mwhudson: will do.
<al-maisan> mwhudson: yes, it's using version 110 now. Thanks!
<mwhudson> al-maisan: yay
<al-maisan> :)
<mwhudson> and i'm glad i checked my mail after getting in tonight...
<al-maisan> mwhudson: thank you very much for fixing ec2 test so promptly!
<mwhudson> al-maisan: it was just me failing to make it public, i'd already fixed it for me :-)
<al-maisan> very nice, we're all benefiting from it now :)
<deryck> Morning, all.
 * jml lunch
<noodles775> Hi leonardr ! When you get a chance, could you read through the comments on bug 487522 and let me know if there's something obvious we missed?
<mup> Bug #487522: getPublishedSources() does not support batch operations <Soyuz:New> <https://launchpad.net/bugs/487522>
<bigjools> oO
<bigjools> james_w has been using that for many months
<leonardr> "getPublishedSources()[:10] might not be applying the slice to the server-side query, but rather to the result that never gets back to you"
<leonardr> that's correct
<leonardr> a python slice is applied to its lhs, and the lhs is not being calculated
<leonardr> you need to forgo the syntactic sugar
<leonardr> let me find the right syntax
<noodles775> leonardr: but https://help.launchpad.net/API/launchpadlib#Collections seems to imply that it works for other CollectionFields?
<noodles775> OK, thanks!
<noodles775> bigjools: without any args? (ie. ubuntu.getPublishedSources() ;) ).
<leonardr> noodles: a lhs like "launchpad.bugs" is resolved without going to the server
<bigjools> noodles775: he uses published_since_date IIRC
<leonardr> launchpadlib doesn't go to the server until it sees the slice
<leonardr> but if you call a named operation, it goes to the server immediately
<noodles775> I see. Would it be possible for it to behave similarly for named operations?
<noodles775> (at least, if it's followed directly by a slice?)
<leonardr> in theory, yes. the function call would return some kind of 'defered' object
<leonardr> noodles775: try passing "ws.start" and "ws.size" parameters into the named operation
<leonardr> i know that if you send those parameters lazr.restful will respect them, but lazr.restfulclient might reject them because they're not found in the wadl
<noodles775> leonardr: thanks! Will do.
<jtv> danilos: my near-term fix of choice for that ongoing approver failure is to combine it with bug 487447 by creating a db permissions group for anything that needs to approve translations uploads (instead of duplicating permissions between the branch approver and the regular one).  The real fix is still to factor out that canEditTranslations check from newPOFile though.  OK if I proceed with the first?
<mup> Bug #487447: TranslationBranchApprover: permission error on TranslationMessage <oops> <Launchpad Translations:In Progress by jtv> <https://launchpad.net/bugs/487447>
<jtv> (Also solves some oopses, which is why I'm working on that one anyway)
<danilos> jtv, considering how it keeps coming back, you should know what I am going to say :)
<jtv> danilos: "yes, and do the other one after that"?
<danilos> jtv, heh, well, something like this, but a bit simpler in terms of effort involved :)
<jtv> "delete the whole damn thing"?
<danilos> jtv, anyway, we have to talk about the buildfarmjob stuff; have you had a chance to catch up with bigjools?
<jtv> danilos: I haven't spoken to him, but did catch up with wgrant and al-maisan
<danilos> jtv, yeah, "delete" is much more of what I'd say
<jtv> The oopses are not caused by the same root problem, so I still think it makes sense to reorganize the permissions.
<jtv> Now, the _testing_ would get simpler if I also remove the pointless check at the same time.
<danilos> jtv, it does make sense to re-organize them, but if removing the useless check helps... :)
 * jtv grins with blood-lust
<jtv> cut!
<jtv> hack!
<jtv> slash!
<jtv> mostly hack.
<danilos> jtv, there you go :) traditional hacking is done with an axe, so let's start with that
 * jtv grinds
<jml> leonardr, fwiw, I've got a branch that splits deferreds out of twisted. that said, it'd probably be nice to have a launchpadlib that did nonblocking IO.
<salgado-lunch> sinzui, I think bug 33145 may not have been fixed by the ajax picker, after all. check my comment there
<mup> Bug #33145: Cannot submit form if Person requires clarifying without doing a non-obvious no-op <fix-it-friday> <Launchpad Foundations:Fix Released by kiko> <https://launchpad.net/bugs/33145>
<sinzui> oops
<salgado-lunch> flacoste, re: yui combo loader, is the plan to have it use the minified files or the raw ones?
<sinzui> salgado-lunch: Yes, I experienced that problem last week too
<flacoste> salgado-lunch: minified files for prod, raw ones for devel
<barry> salgado-lunch: ping when you're back, re: bug 482176
<mup> Bug #482176: Should be possible to add a member without leaving a team's main page <Launchpad Registry:In Progress by salgado> <https://launchpad.net/bugs/482176>
<EdwinGrubbs> bac: ping
<abentley> sinzui: I'm in the middle of moving webapp.errorlog into lp.services, and I just came across this comment from you: "XXX sinzui 2008-08-15 bug=258423: We should be importing from lazr.errorlog"
<abentley> sinzui: Do you think it makes sense to move errorlog into lp.services?
<sinzui> abentley: I am not certain any more. oopses are becoming a lazr lib
<bac> EdwinGrubbs: hi
<abentley> sinzui: Is someone working on moving oopses into lazr now?
 * sinzui thinks
<EdwinGrubbs> bac: nevermind, I found the email which explains how to clean the twisted directory.
<sinzui> abentley: I recall that u1 and landscape have reworked our oops code. That may only be the backend reporting.
<bac> EdwinGrubbs: it is easier now that there is a 'clean' target in sourcecode
<sinzui> abentley: I think flacoste/gary know the answers now. I do know that everything in webapp should be errorlog
<abentley> sinzui: I don't understand.  Do you mean that everything in webapp.errorlog should be in lazr.errorlog?
<sinzui> webapp == lazr
<sinzui> abentley: there was a sprint earlier this year that tried to move everything in webapp to lazr
<EdwinGrubbs> bac: really? I ran rocketfuel-get yesterday, and I still don't have the clean target.
<abentley> sinzui: I don't understand.  Webapp is part of the launchpad codebase, lazr is not, and lazr shouldn't be specific to web AIUI.
<bac> EdwinGrubbs: /me looks
<bac> EdwinGrubbs: it is in mine
<EdwinGrubbs> bac: wait, I was going directly to lp-sourcedeps/sourcecode instead of lp-branches/trunk/sourcecode
<bac> ah
<sinzui> abentley: we do not want webapp in the launchpad code base. We did not want to write it. We will be happy when that code is out of launchpad.
<gary_poster> abentley: (maybe I misunderstand you but) lazr does not contain only web-specific code, but can contain web-specific modules.  I do want our oops generation code, and our oops tools, to be open sourced.  The first will certainly be a lazr.* thing, I think.  The oops tools I'm not as clear on right now.
<gary_poster> s/modules/packages/
<abentley> gary_poster: I thought sinzui meant that lazr should only be web-specific stuff because he said "webapp == lazr".  I think our expectations of lazr are similar.  I want to hack on our oops generation code, so I'd like to know if anyone else is hacking on it.
<gary_poster> abentley: cool.  I don't think so.  matsubara-lunch is the person to coordinate with.  I'm reasonably confident that he's not working on the generation code right now though.
<salgado> barry, hi there
<barry> salgado: hi.  otp w/sinzui
<sinzui> barry: https://edge.launchpad.net/mailman
<sinzui> https://edge.launchpad.net/ubuntu/lucid/+source/mailman
<sinzui> https://edge.launchpad.net/ubuntu/breezy/+source/mailman
<barry> sinzui: skype fail, calling back
<sinzui> barry: visit https://edge.launchpad.net/mailman/+packages, then choose the Source package link for each that is wrong
<gmb> Does anyone know how to express the following expression using storm: http://pastebin.ubuntu.com/326956/
<gmb> I've tried using In(BugSubscription.bug, duplicates_ids) (where duplicate_ids is a list of the IDs of duplicatebugs) but I just get a CompileError.
<EdwinGrubbs> gmb: you should be able to use: BugSubscription.bug.is_in(duplicates_ids)
<gmb> EdwinGrubbs: Let me try that...
<EdwinGrubbs> gmb: you may also add in the subquery as: BugSubscription.bug.is_in(SQL('SELECT id FROM Bug WHERE duplicateof = 15'))
<gmb> Eurgh. Let's hope it doesn't come to that.
<EdwinGrubbs> gmb: actually, is there a reason you don't use a join instead of a subquery?
<gmb> EdwinGrubbs: No reason other than habit
<gmb> EdwinGrubbs: Incidentally, I just got "AttributeError: 'Reference' object has no attribute 'is_in'" when using is_in().
<EdwinGrubbs> gmb: oh, you have to do: BugSubscription.bugID.is_in(foo)
<gmb> EdwinGrubbs: Aha. That's got it. Nice and non-obvious answer. thanks!
<sinzui> barry: I was hearing you
<sinzui> I hear you
<barry> sinzui: i'm back when you are
 * jml gone
<mrevell> night all
<barry> salgado: ping
<salgado> hi barry
<barry> salgado: hi.  i'm wondering if i should finish up bug 482176 for you
<mup> Bug #482176: Should be possible to add a member without leaving a team's main page <Launchpad Registry:In Progress by salgado> <https://launchpad.net/bugs/482176>
<salgado> barry, if you'd like, sure!  as I said in my email, I don't think I'll have time for it any time soon
<barry> salgado: how close to landing is it?  what still needs to be done?
<salgado> barry, it's missing proper handling of two corner cases (currently handled with alert()s) and some windmill tests
<barry> salgado: okay.  between that branch and my own, i'm trying to land our sprint branches before i disappear for the long thanksgiving weekend
<salgado> barry, cool.  if you don't get to it I'll try to do it on the first week of 3.1.12
<barry> salgado: +1
<mwhudson> good morning
<thumper> morning
<sinzui> Chex: ping
<thumper> jml: please QA your landings :)
<barry> salgado-afk: ping
<Chex> sinzui: hello
<sinzui> Chex: I need help testing the first item on on this page: https://dev.launchpad.net/RegistryTeam/RegistryTestPlans/3.1.11
<sinzui> Chex: Can you create a series on staging and run the initialise-from-parent.py script?
<dobey> doh. just missed leonard
 * sinzui has no idea how long it takes to run initialise-from-parent.py with real data
<wgrant> sinzui: The release process says around eight minutes on production.
<bigjools> wgrant: is this the review you're waiting on me for? https://code.edge.launchpad.net/~wgrant/launchpad/distroseries-source-format-selection/+merge/14977
<sinzui> wgrant: thanks
<wgrant> bigjools: That's the one.
<bigjools> ok doing it now
<Chex> sinzui: ok, looking.. what kind of a series are you looking for?
<sinzui> Chex: create Ubuntu/frumpy and set the parent series to jaunty
<thumper> flacoste: call now?
<flacoste> thumper: in one hour?
<thumper> flacoste: I thought we had moved it forward
<thumper> flacoste: however one hour is ok
<thumper> barry: ping
<barry> thumper: pong
<thumper> barry: will mailman strip image attachments throught the LP lists?
<barry> thumper: no, but it might hold them for approval if the size of the message is "too big"
<thumper> ok, should be pretty small
<thumper> barry: total of 25k :)
<barry> thumper: that should make it
 * thumper turns attention to email
<Chex> sinzui: here is the full error I am seeing with this script for you: https://pastebin.canonical.com/25018/
<Chex> sinzui: I think its a DB access issue on sourcherry, looking at it now
<sinzui> Chex: we may need to pass the config in the environment so that the correct credentials are setup
<sinzui> Chex:try prefixing the script with
<sinzui>     LPCONFIG=staging;
<Chex> sinzui: same thing.  credentials are where I am looking now.
 * sinzui thinks
<sinzui> Chex: you can use
<sinzui>     utilities/lsconf.py -s archivepublisher configs/staging/launchpad-lazr.conf
<sinzui> to verify that staging has the information setup
<sinzui> well I suppose it does since the user is known. Maybe this cannot be tested on staging
<sinzui> Chex: I suspect that dogfood is where archive publishing is tested. If that is the case, I have wasted your time. I will ask someone from Soyuz tomorrow.
<bigjools> sinzui: it is, how can I help?
<Chex> sinzui: ah ok..
<Chex> sinzui: no worries, I learned a little something in the process...
<sinzui> bigjools: Chex and I have tried to run initialise-from-parent on staging
<bigjools> what happened?
<sinzui> bigjools: https://pastebin.canonical.com/25018/
<bigjools> db perms
<sinzui> looks like that script and lucille have never been tested on staging
<bigjools> yeah we usually do it on dogfood
<sinzui> Is dogfood up
<bigjools> it runs as "lucille" which is obvioiusly not permissioned for the local user
<bigjools> it's not :/
<bigjools> it's restoring the latest prod database
<bigjools> but hs been doing that for 9.5 days now
<sinzui> luckilly we have a few months to test my change to initialise-from parent
<bigjools> I asked stub to check if I should let it continue or if I should apply his speedup and re-try
<bigjools> heh :)
<cody-somerville> bigjools, holy crap. its still restoring?
<sinzui> and I very confident that it works since the test worked and I ran the script in production
<bigjools> cody-somerville: yarp
<wgrant> Is dogfood still entirely running on poor old mawson?
<bigjools> yarp
<bigjools> but there ws a bug in the db restore script that made it slow
<bigjools> it normally takes about 1.5 days
<wgrant> Ah.
<bigjools> ideal to heat the DC over a weekend
<ajmitch> if it normally takes 1.5 days, what is 'slow' by those standards?
<wgrant> bigjools: Which version is cjwatson's dpkg backport?
<bigjools> still going after 9!
<ajmitch> it'll finish in another week?
<bigjools> no idea
<bigjools> wgrant: not sure, you need to ask him
<wgrant> bigjools: Will do.
<lifeless> spm: lp is rejecting bugmail for me
<thumper> lifeless: it doesn't like you any more
<lifeless> thumper: oops
<spm> lifeless: that's a cool trick. have you any examples you can show?
<lifeless> spm: I deleted them, but its logged mail oopses
<spm> oki
<bigjools> good night everyone
#launchpad-dev 2009-11-25
 * mwhudson lunches
<mwhudson> oh right
<mwhudson> libapr-dev and libsvn-dev need to be installed *everywhere* or make build dies
<mwhudson> er
<wgrant> mwhudson: do you have access to import CAT packages into the ppa and ec2 images?
<mwhudson> wgrant: um, maybe
<mwhudson> wgrant: which images do you mean?
<mwhudson> ec2 images that is
<wgrant> both
<maxb> CAT packages?
<mwhudson> wgrant: i don't have access to the buildbot images
<wgrant> not sure what it stands for, but it's the sysadmins' internal archive
<wgrant> in other news, this onscreen keyboard sucks.
<spm> it stands for notdog
<mwhudson> wgrant: i think i can get the source package from inside the db and shove them into the ppa
<mwhudson> but it might be better to ask someone who knows what they're doing
<wgrant> well, i need the karmic->hardy dpkg backport everywhere in the next day or two
<mwhudson> once it's in the ppa it's fairly easy to get it into the ec2 test images
<wgrant> i have the diff, so could produce an identical package if required
<wgrant> great
<mwhudson> the buildbot images require a bunch of work
<mwhudson> for the losas, not me
<wgrant> ungreat
<mwhudson> it's not hard, but it is time consuming
<mwhudson> getting it onto the DC machines is very much not my problem
<wgrant> right
<mwhudson> but if it's in CAT already that shouldn't be too hard
<spm> mwhudson: fwiw, despite my protestations otherwise - and they won't stop regardless of what I say next ;-) - updating the AMI's isn't that hard. It's fiddly and mindnumbing; but not hard. :-D
<mwhudson> spm: that's what i was trying to say
<spm> ha! great minds etc.
<thumper> sometimes firefox's caching drives me up the freaking wall
 * thumper wonders who reviewed the new comment js work
 * thumper is frustrated it has no tests
<thumper> the fix for the review replies on staging is deleting exactly one space
<thumper> however
<thumper> adding a tests sucks arse
<wgrant> Does capitalisation count when sorting imports?
<mwhudson> wgrant: case insensitive sort
<wgrant> mwhudson: Thanks.
<mwhudson> wgrant: do you know how published binaries get linked to the build that created them?
<wgrant> mwhudson: buildd-manager gives process-upload.py the build ID.
<mwhudson> wgrant: ah ok
<mwhudson> i guess that'll be easy enough to mimic for building source packages from recipes
<wgrant> Something could be easily enough arranged, I'm sure.
<thumper> wgrant: I'll give you a chocolate fish if you rename SourcePackage to DistroSeriesSourcePackage :)
<wgrant> Mmmm. Chocolate fish.
 * wgrant WTFs at the rather preemptive SourcePackageFileTypes.
<adeuring> good morning
<mrevell> Morning compadres.
<noodles775> Hi mrevell !
<wgrant> al-maisan: In r9913 you added two new imports (getFileType and SourcePackageFileType) to lp.soyuz.model.publishing. They seem to be unused, and conflict with one of my branches. Can I safely remove them?
<al-maisan> wgrant: if they are unused, yes, feel free to remove them.
<al-maisan> wgrant: sorry for creating problems
<wgrant> al-maisan: Thanks
<jml> good morning Launchpadders.
<al-maisan> Good morning jml
<ajmitch> hi jml
<jml> ajmitch, al-maisan: hello
<jml> mwhudson, are you around?
<lifeless> jml: 11:45pm there
<lifeless> jml: signs point to no.
<jml> xchat is being incorrect.
<intellectronica> jml: dog bite man
<jml> intellectronica, heh
<al-maisan> hello wgrant, just going through https://dev.launchpad.net/Soyuz/HowToUseSoyuzLocally
<al-maisan> the "gpg --keyserver keyserver.launchpad.dev --send-keys <keyid>" part fails
 * wgrant fixes a bug in it.
<wgrant> Did you create /var/tmp/zeca?
<wgrant> If you didn't, you'll get a 500.
<al-maisan> ah, I missed that!
<al-maisan> wgrant: thanks !
<al-maisan> wgrant: that was the issue, after creating /var/tmp/zeca everything works
<wgrant> al-maisan: Great.
<jml> so much stuff.
 * wgrant should get some of those changes merged at some point.
<wgrant> al-maisan: No other issues so far?
<al-maisan> wgrant: I got entangled in postfix issues but that's unrelated to your instructions page
<bac> hi jml.  just looking at your cookbook.  nice idea
<jml> bac, it's Marjo's idea.
<jml> I'm just the postman.
<bac> jml:  your example can be simplified to login_with('hello-world', 'production')
<bac> jml: or did you really want to show the other params in use
<jml> bac, oh. cool.
<jml> bac, umm, I don't so. Could you please update the example?
<bac> jml:  sure
<jml> bac, thanks.
<bac> deleting code is easy
<jml> and fun!
<al-maisan> wgrant: I have a new LP user with a PPA, moving on to the "Configure a buildd" section..
<jtv> henninge: want me to write up how to test the storm fix?
<sinzui> mthaddon: can is be an admin on staging to test
<mthaddon> sinzui: can is?
<sinzui> mthaddon: sorry, I have not had coffee yet
<mthaddon> sinzui: what's your LP username?
<sinzui> mthaddon: May I be an admin on staging so that I can test
<sinzui> sinzui
<maxb> Twisted egg! Woot!  However, sourcecode/Makefile and utilities/sourcedeps.conf haven't been updated
<mthaddon> sinzui: done
<flacoste> morning launchpadders
 * bigjools waves at flacoste
 * flacoste waves back
<adiroiban> hi. I'm having some problems with pagetests/stories http://paste.ubuntu.com/327670/
 * jml off to get some lunch
<adiroiban> i tried to follow in information from https://dev.launchpad.net/Debugging but with no luck
<intellectronica> adiroiban: this style of pagetest is particularly annoying to debug, since one failure trips the entire test and you get a lot of output which you can't really work with
<adiroiban> still I need to write the test and pass it
<adiroiban> in the devel branch everthing is ok
<adiroiban> and i have just added a new column
<adiroiban> but don't know how to modify the pagetest
<intellectronica> adiroiban: right, so you've added a new column to the table, but not in the test?
<adiroiban> I have added in the test
<salgado> adiroiban, the existing page test expects to find a  'Last Update' in the text but it finds a 'Last update' instead
<salgado> as intellectronica pointed out, these failures are really painful to debug
<adiroiban> uh... stupid me
<adiroiban> the diff was not very useful
<adiroiban> I thought the fortmating in wrong
<adiroiban> as each cell was on each line
<adiroiban> instead of being alligned
<salgado> adiroiban, that doesn't happen because our tests run with +NORMALIZE_WHITESPACE
<salgado> I mean, the formatting in the expected output doesn't have to match that of the given one
<adiroiban> this is my first attempt to fix a bug in LP
<adiroiban> this is  why I'm just lost :)
<adiroiban> is there a way to improve the way that diff was displayed ?
<adiroiban> https://dev.launchpad.net/Debugging#Debugging stories/pagetests as it was not very useful for me
<intellectronica> adiroiban: you could try to parse the table with beautiful soup and reformat it for printing line-by-line, so that at least if there's an error you get to see on which line it is
<salgado> adiroiban, there's a --ndiff option you can pass to bin/test.  that might help
<flacoste> gary_poster: did you see the good news, mwhudson updated to twisted 9 and as an egg!
<intellectronica> salgado: what does --ndiff do?
<salgado> intellectronica, not sure; never tried it
<salgado> I just remember somebody mentioned it as useful in these cases
<gary_poster> flacoste: wow, awesome!  no, I didn't.  I think that will allow a number of cleanups.
<adiroiban> salgado: same result with --ndiff
<adiroiban> will try with udiff and cdiff
<bac> barry: are we having a reviewers meeting?
<barry> bac: we should, but i'm otp
<intellectronica> adiroiban: i don't think there's any diff that can help. the problem is that the output really is like what you see when the test fails. the fancy formatting in the document itself relies on doctests treating all whitespace equally
<henninge> jtv: yes, please
<barry> reviewers, developers, lurkers: apologies for the delay, i had some last minute phone calls.  i'd like to do the reviewers meeting in 5m.  i think it will be short -> #launchpad-meeting
<barry> reviewers -> #launchpad-meeting in 2m
<barry> reviewers -> #launchpad-meeting
<allenap> sinzui: flacoste has approved https://code.edge.launchpad.net/~sinzui/launchpad/parent-packaging-links-bug-484828/+merge/15092 but there's an outstanding review request for ~launchpad. Is that correct? If so, I'll do it now.
<sinzui> No it does not need review
<flacoste> allenap: i think that's a bug in the code review system
<allenap> sinzui, flacoste: Cool, thanks.
<sinzui> allenap: I will delete it. The script was run once and it will not be merged
<allenap> leonardr: There are a couple of quite old lazr branches of yours still showing as needing review: https://code.edge.launchpad.net/~leonardr/lazr.restfulclient/option-introspection/+merge/5800 and https://code.edge.launchpad.net/~leonardr/lazr.restful/service-request-to-web-request/+merge/7027
<allenap> leonardr: Actually, one of them has been approved.
<allenap> leonardr: Do they still need reviewing, or can they be marked as approved, or merged?
<jml> beuno, can we have a call in a little while?
<beuno> jml, sure, although I'm about to step away for 30' of lunch
<jml> beuno, after lunch is fine
<gmb> jml, allenap: Either of you got any idea why ec2 land would say "running tests," give me the submit message bit and then shut down all on its own?
<jml> gmb, is that really the full output?
<mrevell> gmb, I can't answer you but I have seen that before.
<gmb> jml: No. I should have said "get as far as saying "running tests"". https://pastebin.canonical.com/25039/ is the full output.
<gmb> Aha!
<gmb> So it's not necessarily me being rubbish.
<mrevell> gmb, That looks familiar. I have no idea why it happened. I think I tried again and all was well. I hope that this greatly detailed reply will be of help.
<gmb> Heh.
<jml> gmb, no, I don't know.
<gmb> This is the fifth or sixth attempt at landing this bloody thing... I'll keep trying.
<jml> gmb, my hunch is that something is failing on the server side, and another bug is preventing it from being reported properly
<gmb> jml: Okay, no worries. I'll see if I can work it out, or at least just get it to actually play.
<jml> gmb, try using ec2 land to get the ec2 test command, then invoking ec2 test directly
<gmb> jml: Right, that was going to be the next thing I did :)
<jtv> henninge: hi, I wasn't really EOD, just not paying attention
<henninge> jtv: is this the right call: http://paste.ubuntu.com/327794/
<henninge> ?
<gmb> It's amazing how quickly you forget how to format a PQM message.
<jml> yes :)
<jtv> henninge: that looks right, yesâalthough creepy things happen with the gtk+2.0 package name in URLs.
<henninge> gmb: [r=..][ui=..][bug=..]
<henninge> jtv:  but that is not a url
<jtv> right
<jtv> henninge: what I mean is, there's a potential pitfall in the obvious way to get the package name.
<jtv> it's close to midnight, don't be picky :)
<jml> staging cronjobs are slow
<henninge> jtv: np
<henninge> jtv: how long did it usually run, what am I to expect?
<henninge> ok, it never finished ....
<jtv> henninge: hours.
<henninge> jtv: so should I ask for staging updates to be stopped while it runs?
<jtv> it'll still run for a long time (assuming all is well), but it'll no longer grow to 2.x GB of memory within the hour
<henninge> jtv: oh, so if it does not do that, it can be stopped after an hour or so ?
<jtv> right
<jtv> just make sure it gets to at least stage 2; stage 3 if you can
<henninge> I assume it says so in the output
<jtv> yes, but there can be a lot of the stuff.
<jtv> hang on, I'll look it up
<henninge> jtv: I guess it can be tee'd.
<jtv> I think you get log level "info" by default, right?  That prints "Removing duplicate messages" (start of stage 1), "Merging POTMsgSets" (start of stage 2), and "Merging TranslationMessages" (start of stage 3).
<jtv> I would, yes, or nohup.
<henninge> oh, it does detach?
<jtv> not by itself, no
<henninge> man nohup
<jtv> nohup does two things:
<jtv> 1. make the command not die on HUP
<jtv> 2. write stdout & stderr to a log file, nohup.out
<jtv> If you combine it with &, you get a background process that doesn't die when your ssh connection does
<henninge> I had thought it was for keeping jobs from detaching.
<jtv> no, it's usually used in combination with detaching
<jtv> $ nohup run_my_process.py &
<jtv> [12345]
<jtv> $ tail -f nohup.out
<henninge> ah, that was my ascociation then
<henninge> jtv: thanks, I'll let you EOD now ... ;-)
<jtv> In stage 2, you'll see messages like "Message 10/254: 5 subordinate(s)."
<henninge> jtv: what about them?
<jtv> That's one way of seeing where you are despite tons of output.
<henninge> ah!
<jtv> I'd try a smaller package first though, and work your way up.
<henninge> jtv: which is a smaller package, for example?
<henninge> size is measured by number of strings, I guess
<jtv> pretty much... of course how long it's been in main also matters
<jtv> argh my wrists hurt
<jtv> I'm looking for something small
<henninge> jtv: I think I got it all. Rest your wrist (and your eyes and your head)
<henninge> jtv: no, I can do that myself
<jtv> thanks
<henninge> jtv: I have to go afk for a little now. Have a good night!
<jtv> thanks!
<jtv> you too, good luck
<sinzui> EdwinGrubbs: I was able to restore the milestone table to as it was before you changed it. I added a test to verify it. The page is was still broken because the tbody's children is not null instead of an empty list. We have a test for this but it does not run
<sinzui> EdwinGrubbs: javascript/test/test_milestone_table will not play in my browser because YUI is undefined
<EdwinGrubbs> sinzui: I don't remember changing that. You should check that the path for the <script> tag in javascript/registry/tests/milestone_table.html actually points at the current location of yui.
<sinzui> EdwinGrubbs: you made the table conditional in your bug-435260-duplicate-links branch. That is not important since the fix took a few minutes. I am more concerned that the JS did need a change and that we had a test for the function
<sinzui> EdwinGrubbs: icing/yui/current no longer exists
<sinzui> I have run make build of course
<sinzui> is there something else I need to run?
<EdwinGrubbs> sinzui: are you branched off of db-devel or devel?
<sinzui> EdwinGrubbs: devel
<EdwinGrubbs> sinzui: I'm cleaning and rebuilding to see if I get the same problem.
<sinzui> I build db-devel just now. icing/yui/current does not exist
<sinzui> oh!
<sinzui> EdwinGrubbs: I see this in the log in db-devel:
<sinzui> utilities/shhh.py bin/jsbuild  -b lazr-js/build -x testing/
<sinzui> The found YUI root isn't valid: /home/curtis/Work/launchpad/db-devel/lib/canonical/launchpad/icing/yui/current/build
<sinzui> EdwinGrubbs: I can manually make the 'current' symlink, The test shows the _prepend_node fails function  and that my fix does pass.
<leonardr> allenap, one is merged, one is dead
<allenap> leonardr: Cool. I marked the merged one as merged already (I eventually realised I should look at the branch page to check), and added a comment to the other.
<leonardr> allenap: i marked the other one as rejected
<allenap> leonardr: Great.
<sinzui> BjornT: EdwinGrubbs: Do you know if anyone has tried to write a wrapper for YUI.Test so that those tests (which rock) are automatically tested?
<EdwinGrubbs> sinzui: I'm surprised by the error you got with jsbuild on db-devel. It created the yui/current/build directory structure correctly for me. BTW, it appears that for yui 3.0.0 that YUI().use('yuitest') needs to be changed to YUI().use('test')
<sinzui> EdwinGrubbs: I see 3.0.0pr2 in db-devel not 3.0
<sinzui> I think something else is wrong
<EdwinGrubbs> sinzui: does your symlink in db-devel point to this version of lazr_js?  lazr-js/build/yui -> /home/egrubbs/canonical/lp-sourcedeps/eggs/lazr_js-0.9dev_r153-py2.5.egg/lazrjs/yui/
<sinzui> no
<sinzui> It points to lazr-js/build/yui
<sinzui> EdwinGrubbs: do I make clean or just blow the link away
<sinzui> I am very frustrated with this. I have a great test. It showed there was an error, but it does not run anymore
<EdwinGrubbs> sinzui: no, the second symlink. lib/canonical/launchpad/icing/yui points to lazr-js/build/yui which itself is a symlink to the lazr-js egg.
<sinzui> WTF, the link ti to a py2,4
<sinzui> I removed py2.4 on my test machine. maybe I should remove it from this one
<mrevell> night all
<mwhudson> jml: hello
<jml> mwhudson, hi
<jml> mwhudson, would you be up for a skype call?
<mwhudson> jml: my coffee is still brewing; after it's made sure
<jml> mwhudson, cool.
<mwhudson> jml: in the mean time, you could ready my first mail to the "Immediate plan for Build Farm generic jobs" thread
<mwhudson> jml: that might explain what SourcePackageRecipeBranch is for
<jml> ok. :)
<mwhudson> jml: ok ready now
<wgrant_> Is bigjools still in Texas?
<adiroiban1> hi, is there a way for creating pagetests based on regex?
<adiroiban1> when testing a table
<lifeless> adiroiban1: no
<lifeless> adiroiban1: pagetests can do pattern matching with ...
<lifeless> or you can manually use a regex and check that it matches, but thats ugly and hard to debug (which is a major limit of doctests)
<lifeless> sinzui: ping
<gary_poster> fwiw, cases like tables is what manuel addresses.  FIT tables are one of his examples.  I hope to bring that in to Launchpad sometime.
<sinzui> Hi lifeless
<gary_poster> s/is/are/
<lifeless> sinzui: I'm concerned we're speaking past each other on the bug about contact addresses for teams.
<lifeless> sinzui: I'm wondering if you have time for a brief chat (here is fine) about it.
<sinzui> okay
<lifeless> As a project owner, I need to create a number of 'noise' teams.
<lifeless> teams to control commit and bug triage for instance.
<lifeless> these are not distinct communities
<lifeless> secondly, the default 'contact this team' feature in launchpad is really really really annoying, because it direct mails all the members, and noone can tell if someone replies.
<lifeless> I should add another really there.
<lifeless> So, what I want is for 'contact this team', on the 'noise' teams, to send mail to the main dev list.
<thumper> beuno: ping
<thumper> flacoste: confirmation for LCA miniconf has come through
<flacoste> thumper: awesome!
<lifeless> Now, I can setup a mail alias on my home mail server that will forward back to launchpad at the main dev list if launchpad won't permit what I want; but I don't think what I want is that unusual.
<lifeless> sinzui: does this make sense?
<beuno> thumper, pong
<sinzui> lifeless: it does. I know teams are doing this now
<lifeless> sinzui: alternatively/better would be for 'contact this team' to not exist for the housekeeping teams.
<thumper> beuno: I think I have been convinced to show the review fields all the time
<thumper> beuno: but instead of showing " - Select - " as the default
<thumper> beuno: have something like "Just comment"
<sinzui> lifeless: I think we should fix launchpad so that there are no housekeeping teams
<wgrant> I think the problem here is actually that LP notifications are just too inflexible and really need to be completely rethough.
<thumper> beuno: as we really want to suggest to people that they should give their opinion :)
<beuno> thumper, that's crazy enough it could work
<thumper> but don't have to
<sinzui> lifeless: I think a team is for communication, and it should be clear who is involved in the communication
<thumper> beuno: I'll mock it up
<lifeless> sinzui: that would be nice too, though 'group of people who can commit' is obviously still a useful concept ;)
<mwhudson> thumper: woo
<sinzui> that is an ACL issue
<sinzui> lifeless: My test for this issue is if had a patch that fixed all the insanity with teams and email addresses, and permitted an email address to be shared by many teams, I do not think I would accept it
<lifeless> sinzui: Why not though?
<lifeless> what does it do _for launchpad_ to require that uniqueness
<sinzui> lifeless: because when launchpad tries to communicate with just that team, the message goes to the wrong people.
<lifeless> sinzui: I disagree.
<bac> hi nhandler
<lifeless> if, as the owner of that email address (e.g. the list moderator) I assert that two different teams should be comunicated with at that address
<nhandler> Hi bac
<lifeless> then what doe it have to do with lp?
<bac> nhandler: what happened with your branch?  did it fail in ec2?
<nhandler> bac: No clue. I haven't gotten an email about it like I have in the past
<bac> nhandler: hmm, i wonder if i messed it up?  i got no email either.  i'll try again now
<sinzui> lifeless, that is essentially the same argument for why we accept a launchpad mailing list as a contact address even though we know no one is subscribed to it. I think that tis wrong.
<lifeless> sinzui: can you describe how it is wrong?
<sinzui> lifeless, The point that sways me to your side is that this is what really happens. I think we need a better solution to ensure the correct users are getting messages.
<lifeless> sinzui: The thing I'm struggling with is that the limit seems arbitrary, and made by the sender, not the recipient. But if you consider business cards: I hand out *my address*, I don't ask the person I'm giving the card to for the address.
<lifeless> sinzui: it sounds like one constraint/goal you have is 'when someone clicks Contact XXX in the web UI, you want to be sure that a mail was sent'
<lifeless> sinzui: but you seem to be conflating it with 'when someone clicks Contact XXX in the web UI, you want to be sure that the message was received'
<lifeless> sinzui: and these are very very diferent statements.
<lifeless> For instance, contact me, on my home page - lp cannot tell that I received the email.
<sinzui> lifeless: I am concerned about the ambiguity of responsabulity
<lifeless> for a list with no subscribers, someone may be reading it on the web. Or via rss.
<lifeless> sinzui: Please consider making that the problem of the folk who own the team & email address.
<sinzui> lifeless: If I am a member of your list and can confirm an email address for my team, then I can co-opt it. I can even disable it for you when I disable it
<lifeless> sinzui: the latter point sounds like a bug, not a must-have.
<lifeless> sinzui: coopting is not possible without me knowing because I'll see the confirmation mail come through and be able to click on the 'cancel' link, or contact the sysadmins.
<lifeless> sinzui: and if its a launchpad hosted list, you can require a list-team-admin to ack it directly in the UI.
<sinzui> lifeless: I am not trying to be mean on this issue, even if I accepted it, I do not think it will be fixed in my lifetime given the complete and utter madness or accounts + email addresses + teams + ACL + mailing lists.
<lifeless> sinzui: I appreciate that scheduling it may take forever :)
<lifeless> but people work around this already, it seems more useful to me to have it acked as wishlist, and then (for instance) wgrant might fix it.
<sinzui> lifeless: why are there multiple teams for a single address?
<lifeless> sinzui: because discussion forums are not equivalent to teams.s
<lifeless> sinzui: (I don't think you're being mean)
<bac> nhandler: something odd is happening.  i resubmitted it and it seems to have terminated without running the tests.
<wgrant> bac: gmb reported the same thing on launchpad-dev.
 * bac looks
<sinzui> lifeless: I am going to accept the bug purely on your point that users are working around this now. I think though that this may be a case there are other solutions to this problem. Launchpad does a poor job of dispatching emails to teams and their members. We have decided this issue cannot be solved, but I think it improving how we dispatch and who we dispatch to can be improved to make this issue moot.
<lifeless> thanks:)
<mwhudson> bac: it's probably my fault but i can't see how
 * mwhudson decamps to a cafe for a change of seen back online in a few mins
<bac> mwhudson: i just did two runs
<bac> the first ended with: instance shutting-down
<bac> the second with: instance running
<bac> but both terminated immediately after detaching
<bac> nhandler: i'm resubmitting using the work-around of not using --headless.  hopefully that'll work
<nhandler> Thanks bac
<sinzui> bac: I still have the test and submit command when I landed nhandler's last blueprint branch
<bac> nhandler: please ping me if you don't get an email in 4-5 hours
<bac> sinzui: i've hacked 'ec2 land' to not detach and will see if that works
<bac> sinzui: it's not a matter of invoking it incorrectly.  the tool seems a bit broken.
<sinzui> bac: this is the command I used: http://pastebin.ubuntu.com/328057/
<wgrant> bigjools: When you have a moment, can you please have a look at my MP again?
<bigjools> wgrant: doing it right now, nearly finished
<wgrant> bigjools: Great. Thanks.
<bigjools> wgrant: craploads better in some parts BTW
<bac> sinzui: *when* it works, you can just do:  ec2 land <MP_URL>
<sinzui> bac: emphasis on *when*
<bac> sinzui: yes.  that's why i emphasized it.  :)
<wgrant> bigjools: I thought so.
<wgrant> I wish MPs had a ToC.
<wgrant> The comments tend to be pretty massive.
<bac> i wish MPs didn't hide the bug link so far down the page
<bigjools> yeah, they need to be collapsable
<bigjools> thumper, fix that will ya :)
<bigjools> wgrant: ok done
<bigjools> just a few more changes
<bigjools> also, you didn't change the existing tests at all, do they still pass?
<wgrant> bigjools: They still pass.
<bigjools> ok cool
<wgrant> bigjools: OK, fixing all those now. Thanks.
<mwhudson> soyuz is eating my brain, again
<bdrung> mwhudson: regarding bug 487897, when can we expect your suggested solution in the productive system (= when will i have a working bzr import to work with)? what's the release policy for the vcs importer?
<mup> Bug #487897: VCS import from audacity CVS failed with invalid value for commit message <Launchpad Bazaar Integration:New> <https://launchpad.net/bugs/487897>
<bigjools> mwhudson: may I assist with that^W^Wyou there?
<wgrant> bigjools: Any hints for where to put the mixin?
<bigjools> wgrant: I thought about that as soon as I hit send.  Eummmm....
<bigjools> wgrant: lib/lp/soyuz/model/files.py I think
<wgrant> bigjools: Ah, yes, that makes sense.
<wgrant> Thanks.
<bigjools> although archiveuploader shouldn't really import from there :/
<bigjools> but that's the least of the problems with soyuz
<wgrant> I don't see how (archive(uploader|publisher)|buildmaster) can possibly avoid using soyuz stuff.
<bigjools> it actually does for the most part
<bigjools> does avoid, I mean
<wgrant> bigjools: I cannot think of a good name for this mixin either. SourceFileMixin, or something more specific?
<bigjools> wgrant: that would be fine
<wgrant> bigjools: I guess I should add explicit tests for the new property as well?
<bigjools> wgrant: ideally yes but it's a mixin so don't worry too much
<wgrant> bigjools: Ah, OK.
<thumper> sinzui: ping
<sinzui> Hi thumper
<thumper> sinzui: can you explain to me why some pages have a <base> tag and some don't ?
<sinzui> really?
<thumper> yeah
<thumper> I'm wanting to have a # anchor
<thumper> but the base tag screws it up
<thumper> and some pages don't have it, and some do
<thumper> and I can't figure out why
<sinzui> I do not know why pages want a base tag. It might help with some script urls, but I think we fixed the cross-domain issue a long time ago
<thumper> sinzui: for example, a branch listing has a base tag, but the active code reviews doesn't
<thumper> sinzui: and I can't work out how the active code reviews one didn't get one
<sinzui> I think annotate might help explain the issue
<thumper> sinzui: I'd explicitly like to have a page not have one
<thumper> sinzui: they use the same base template...
<sinzui> base-layout?
<mwhudson> impressive, i moved from the cafe to the library and my connection to freenode stayed up
<mwhudson> bigjools: possibly
<mwhudson> bigjools: i'm looking at Build and wondering how many fields apply to building recipes into source packages
<mwhudson> bigjools: and thinking "quite a lot of it, possibly"
<thumper> sinzui: metal:use-macro="view/macro:page/main_only"
<bigjools> mwhudson: I think I know where this is going as well.  You're scaring me.
<thumper> mwhudson: how far is the cafe to the library?
<mwhudson> bigjools: i don't, but it's scary too
<mwhudson> thumper: not that far i guess, but my machine was asleep for at least a couple of minutes
<thumper> mwhudson, bigjools: table inheritance fixes everything \o/
<mwhudson> thumper: i'm not sure i believe you
<bigjools> the bigger the inheritance the better, I say
<thumper> mwhudson: heh
<sinzui> thumper: that is base-layout, all pages use it...I am certain because I remove the old main-template Monday
<bigjools> but old tables need to die first
<thumper> sinzui: so why does one page have a <base> tag and the other not?
 * sinzui looks
<bigjools> mwhudson: so are you designing a RecipeBuild or similar?
<mwhudson> bigjools: well, i've got as far as thinking we need one
<bigjools> I coulda told you that :)
<mwhudson> bigjools: well you did, but i also needed to see it for myself :-)
<wgrant> bigjools: Note that this cannot land until the new dpkg-dev is in the various images.
<sinzui> thumper: base-layout does not create a base tag. does one our the templates have a head_epilogue?
<bigjools> wgrant: which images?
<wgrant> bigjools: ec2test and buildbot AMIs.
<thumper> nope
<bigjools> mwhudson: yeah it links recipes to source packages, so it's kinda necessary
<thumper> sinzui: ^^
<mwhudson> bigjools: do you think it's better to have a RecipeBuild that has many similar fields to Build or do something different, like have RecipeBuild that points to a Build row?
<mwhudson> bigjools: well, we could use the RecipeJob for that
<bigjools> mwhudson: the former
<mwhudson> (but i don't think we should)
<mwhudson> bigjools: okay
<bigjools> Build is for Source<->Binary package linkage
<bigjools> best not confuse that
<sinzui> thumper: there are no <base.* tags in the tree
<thumper> :(
<bigjools> mwhudson: yes you could also use RecipeJob, I think I mentioned that in an email, it depends on how much fluff you want in there etc.
<mwhudson> bigjools: how is source binary pakage linkage done?
<thumper> where the f*ck are they coming from then?
 * sinzui looks at branch listing
<thumper> sinzui: firefox maybe?
<thumper> can't be
<mwhudson> bigjools: i guess those scary *packagepublishinghistory tables are involved?
<thumper> it doesn't know the underlying view name
<bigjools> mwhudson: build has a sourcepackagerelease column, binarypackagerelease has a build column
<thumper> it has to be created by us somewhere
<bigjools> mwhudson: publication is orthogonal
<mwhudson> bigjools: oh good
<sinzui> thumper: YUI more likely. scripts can inject nodes
<mwhudson> bigjools: what's a sourcepackagerelease again?
<thumper> sinzui: no way
<thumper> sinzui: it was there before yui
<bigjools> mwhudson: build is the *how*, publishing is the *where*
<mwhudson> bigjools: is it created during upload processing?
<bigjools> mwhudson: yes, when a source package is uploaded
<bigjools> uploading currently the only creation point of all source-related objects
<sinzui> thumper, branch-index.pt has in invalid js element. It must have a close tag, it cannot be <empty />
<mwhudson> bigjools: so this is jumping ahead a bit but if you wanted to link a binary to a recipe it would be
<bigjools> mwhudson: I wouldn't do that
<sinzui> thumper: that is a separate issue
<thumper> sinzui: it isn't branch index
<mwhudson> binarypackagerelease -> build -> sourcepackagerelease -> ??? -> recipe?
<bigjools> oh indirectly
<mwhudson> bigjools: i mean in the ui, sorry
<thumper> sinzui: but lib/lp/code/templates/product-branches.pt
<bigjools> yeah something like that
<mwhudson> ok
<mwhudson> i guess upload processing or something near that will have to change a bit to make the ??? part of that work
<bigjools> mwhudson: well we'll be uploading the result of the recipe build - which is a source package - so we can do what we do for build uploads and tell process-upload which recipe generated it
<bigjools> pretty trivial
<mwhudson> bigjools: right
<mwhudson> a column on sourcepackagerelease to sourcepackagerecipebuild would seem to fit the existing pattern
<bigjools> if you look at process-upload it has a -b flag, this is only used by buildd-manager when it uploads binary packages
<bigjools> mwhudson: I think the other way around
<sinzui> thumper: I just tried epiphany for a sanity check. this is strange. <base> is being injected after the head element. It is not in out template rules. Maybe this is the rendering engine.
<bigjools> sourcepackagerelease column on sourcepackagerecipebuild
<thumper> sinzui: perhaps, maybe gary or some other zope person knows
<thumper> sinzui: it is very frustrating
<mwhudson> bigjools: oh right yes
<sinzui> thumper: I am looking in web app. We have some rules there.
<mwhudson> sinzui: i don't think it can be client-side; as thumper says how would it know the view name?
<bigjools> mwhudson: if you can get to grips with the existing model then this becomes a little easier to visualise I think
<sinzui> mwhudson: I agree
<mwhudson> bigjools: no kidding
<mwhudson> bigjools: i know it a lot better than i did two weeks ago! :)
<bigjools> mwhudson: do you remember my database diagram that everyone laughed at at the epic? :)
<mwhudson> bigjools: vaguely
<bigjools> https://dev.launchpad.net/Soyuz/TechnicalDetails?action=AttachFile&rename=SoyuzBuilders.dia
<bigjools> ah bollocks
<bigjools> where's it gone
<wgrant> Not copied from the old wiki?
<bigjools> I uploaded my copy again
<bigjools> now, how do I embed the png
<sinzui> thumper: I think the template class (which has a __call__) is doing it
<thumper> hmm
<thumper> sinzui: so why on some and not on others?
<sinzui> I do not know
<mwhudson> bigjools: is it overly optimistic for me to ignore everything in the "publish" box?
<wgrant> It's mostly wrong, so I would actively avoid reading it anyway.
<bigjools> Oo
<sinzui> thumper: I think this is zope
<wgrant> Oh, no, that bit's actually right.
<bigjools> it's all right
<wgrant> Just years out of date.
<bigjools> mwhudson: yes you can avoid that box
<sinzui> thumper: and I think this has something to do with zcml registration
<mwhudson> bigjools: yay
<bigjools> right, I am getting attacked by insects.  And in Texas, everything is bigger.
 * bigjools heads indoors
<mwhudson> bigjools: sorry if i'm just being dense, but
<mwhudson> <bigjools> mwhudson: I think the other way around
<mwhudson> <bigjools> sourcepackagerelease column on sourcepackagerecipebuild
<mwhudson> this doesn't seem like what happens currently
<mwhudson> build has a link to what it's building (the spr)
<bigjools> mwhudson: build has a sourcepackagerelease column
<mwhudson> what's built (the bpr) links to build
<mwhudson> so the analogy would be that the spr that's built would like to the recipebuild
<mwhudson> s/like/link/
<bigjools> mwhudson: true, but it would be a nullable FK in this case
<mwhudson> bigjools: i guess in this case sourcepackagerelease <-> recipebuild is 1-1
<mwhudson> bigjools: certainly!
<mwhudson> i don't really have an opinion which way round is better, i guess
<bigjools> hmmm
<bigjools> yeah I guess I don't on reflection
<bigjools> hmmm can't avoid the nullable FK either way
<mwhudson> i think what's built linking to why it's built is a little more natural
<mwhudson> bigjools: indeed
<bigjools> agreede
<mwhudson> ok cool
<mwhudson> bigjools: did you see my email with a proposed db patch?
<mwhudson> (i know you've had problems with that)
<bigjools> mwhudson: still no email :/
<mwhudson> bigjools: ok
<bigjools> mwhudson: send it to bigjools@gmail.com and I can see it there
<mwhudson> bigjools: i talked to jml and changed my mind on lots of things anyway :-)
<bigjools> fook knows what's happened to my home server, I ordered a new PSU and UPS just in case!
<mwhudson> bigjools: how much longer are you going to be around for today?
<bigjools> mwhudson: not much
<mwhudson> bigjools: ok
<mwhudson> bigjools: i'll send an email with an updated patch later today, i guess you won't see it for some time
<bigjools> I can hang around on here though
 * mwhudson tries to summarize
<bigjools> I won't be going out for an hour or so but I will need to scrub up a bit
#launchpad-dev 2009-11-26
<mwhudson> bigjools: the idea is to have the actual recipe part (the payload?) or both manifests and recipes stored the same way, in some SourcePackageRecipeData table
<mwhudson> then SourcePackageRecipe (the editable thing) and SourcePackageRecipeBuild will both refer to this table
<mwhudson> SourcePackageRecipe will have the stuff needed to locate the recipe
<mwhudson> there will be a linking table linking SourcePackageRecipeData and Branch
<bigjools> ok
<mwhudson> SourcePackageRecipeBuild will mostly be like Build i guess
<bigjools> I can pore over this on my flight home on Friday
<mwhudson> and we'll add a (nullable) link from SourcePackageRelease to SourcePackageRecipeBuild
<bigjools> I am not working tomorrow (Thursday here)
<mwhudson> ok
<mwhudson> i'll send my followup mail to your gmail address too
<bigjools> I guess the FK makes more sense in a way if you treat it like "this package came from a recipe, here it is"
<bigjools> ok I am going AFK for a bit but I will check back before I go out
<mwhudson> ok
<mwhudson> thanks for the hints
<mwhudson> bigjools/wgrant: what creates Builds out of curiosity?
<wgrant> mwhudson: An upload.
<wgrant> Or queue-builder, but not normally.
<mwhudson> wgrant: so they're created at the same time as sourcepackagerelease?
<wgrant> mwhudson: Right.
<mwhudson> ok
<wgrant> sinzui: Is there a good reason that lp.registry.interfaces.sourcepackage.SourcePackageFileType doesn't live somewhere like lp.soyuz.interfaces.files?
<mwhudson> i guess this build from recipe plan doesn't have an analogue of sourcepackageupload really
<mwhudson> i wonder if that's a problem
<wgrant> You mean a PackageUploadSource?
<wgrant> Or something else?
<mwhudson> sorry
<mwhudson> sourcepackagerelease
<wgrant> Ah.
<wgrant> That would be a manifest, I guess, but there isn't one of those yet...
<mwhudson> yes i guess so
<thumper> damn
<thumper> up arrow in the terminal to get 'make run' takes longer if the last was 'make clean run'
<wgrant> Heh.
<mwhudson> :-)
<mwhudson> thumper: make schema run is worse i think
<thumper> not any more
<thumper> make schema runs faster for me than make clean biuld
<thumper> hah
<thumper> bent javascript to my will yet again!
<thumper> ...
<thumper> grr
<thumper> not working
 * thumper needs food badly
<wgrant> mwhudson: Are jtv and yourself the only non-Soyuz BFB people?
<mwhudson> thumper: javascript is such a tease like that
<mwhudson> wgrant: i guess
<mwhudson> wgrant: jml is pretty involved too, and some work from the bzr end of course
<nhandler> Just out of curiosity, is there an easy way to manage launchpad branches? In the past, I've followed the guide on the wiki to get the source, made my changes, commited, and pushed. Then, once it was accepted, I reverted my changes and updated the branch. Is there a better workflow?
<mwhudson> nhandler: you shouldn't need to revert the branch
<wgrant> nhandler: You should probably have a separate branch for each piece of work.
<mwhudson> nhandler: i have a slightly extreme set up, i have repo with no trees and a directory that contains lightweight checkouts from the repo
<wgrant> Ah.
<mwhudson> and make a new branch + checkout for each new bit of work
<nhandler> wgrant or mwhudson: How would I go about doing that? Would I just pass an additional argument to bzr commit once I make my changes?
<mwhudson> nhandler: no
<mwhudson> i do bzr cb trunk feature-X; cd feature-X; hack hack hack; bzr ci; bzr push
<mwhudson> cb is an alias for "cbranch --lightweight --hardlink"
 * wgrant is boring and uses 'bzr branch devel some-feature'
<nhandler> Ok thanks a lot
<nhandler> And I'm not sure if it is just backed up, but I haven't gotten an email yet bac
<wgrant> nhandler: it has only been 2.5 hours, hasn't it?
<wgrant> The test suite takes more than 3.
<nhandler> wgrant: I wasn't aware of that, thanks
<wgrant> There's a looooot of sloooow tests.
<mwhudson> bigjools: http://pastebin.ubuntu.com/328119/ <- updated patch, if a wad of sql makes sense to you :-)
<lifeless> mwhudson: wgrant: bzr switch -b FTW
<lifeless> or branch --switch too
 * mwhudson definitely wants to write some code that uses this before landing the patch tho
<mwhudson> lifeless: i like my working trees to be named after the branch
<wgrant> lifeless: Handy.
<wgrant> lifeless: Although I'd prefer it if it shelved my uncommitted changes, like switch-pipe.
<lifeless> wgrant: patches reviewed.
<mwhudson> i guess i could abuse branch --switch to something like that but the tree build time for launchpad is ok now tbh
<thumper> I use lightweight checkouts too
<thumper> although --hardlink isn't supported in 2a yet
<thumper> lifeless: what does branch --switch do?
<thumper> lifeless: can it rename the directory?
<lifeless> thumper: no
<sinzui> wgrant:  no reason. We have not spent much time breaking the larger classes into domains yet. Your welcome to reorganise them.
<nhandler> I'm thinking about fixing LP Bug #400393. It looks pretty straightforward. I personally am in favor of mpt's suggestion, but I think either would work.
<mup> Bug #400393: "Updating branch..." message reads oddly <trivial> <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/400393>
 * wgrant wonders why people want digests.
<mwhudson> wgrant: because they want to make their email harder to read i guess
<lifeless> because they don't want that mail at all
<mwhudson> i guess
 * mwhudson hates at the fact that the launchpad database only accepts file://localhost/ urls but bzrlib only accepts file:/// ones for the first time in a while
<lifeless> mwhudson: you can fix lp if you like ;)
<mwhudson> yeah, probably the right thing to do
<mwhudson> where's that dba
<mwhudson> lifeless: can i fix bzr too?
<lifeless> mwhudson: yes.
<lifeless> file:///localhost/ should be accepted already, I thought.
<lifeless> bah, s////////
<mwhudson> :)
<mwhudson> anyway, nope, url_to_local_path still complains in trunk
<mwhudson> man i suck at typing my gpg passphrase today
<ajmitch> just don't type your passphrase in irc
<mwhudson> well, judging by today's performance i wouldn't be able to
<mwhudson> i think this traceback means i should stop work for the day: http://pastebin.ubuntu.com/328196/
<wgrant> mwhudson: I would say so.
<mwhudson> hm, probably python version fun actually
<wgrant> I considered that, but it looks too obscure.
<mwhudson> yes
<mwhudson> bzrlib was built with 2.6
<mwhudson> pretty obscure yes :)
<mars> why is edge still running 9928?  Should it not have updated last night?
<wgrant> I heard whispers of a couple of security fixes that I haven't seen landing on public branches, so perhaps updates are turned off for those.
<mars> I don't see anything in the branch lists.  Could have been a while ago, in a sub-component, or I'm looking at the wrong list.
<spm> mars: was a landing made by a lp-bzr dev in northern NZ; it would have broken at least one edge server, so we disabled
<jml> mwhudson, I've replied to your "first cut" thread.
<stub> Anyone know of any simple Python libraries to help write /etc/init.d type control scripts? I'm getting sick of hacking up makefiles or shell scripts.
<wgrant> stub: Upstart!
<stub> Hmm... user services is a planned feature
<stub> This is all in tree dev stuff
<wgrant> Ah. Sad.
<wgrant> I don't know of anything Pythony to help generate the boilerplate.
<jml> stub, you could make a quickly template, perhaps
<stub> A whatly template?
<jml> stub, 'quickly' is a python app made by didrocks and rickspencer3 that is all about making projects quickly.
<jml> basically by doing the boilerplate generation for you
<jml> maybe that's not what you want.
<BjornT_> danilos: didn't your patch to make test_template_listing_admin_links pass land?
<jtv> Does anyone know of a clever way to "mock up an import"?
<jtv> I'm trying to write a test for test the class we use to run tests on EC2 instances, but afaict it imports python modules that are available in the EC2 image but not in LP itself.
<jml> jtv, you can insert something into sys.modules, I think.
<jml> jtv, it's just a dict.
<jml> jtv, but you can probably do something nicer by splitting the code you are testing away from the code that's difficult to test.
<jtv> jml: yes, I guess I'll do that
<jtv> the latter
<jml> jtv, good call :)
<jtv> I wonder if it'd be terribly bad of me to isolate the invocation of the function I can't import, _plus its import_, in a method that I can override for the test.
<jml> it depends.
<jml> without context, it sounds a little dodgy
<jtv> jml: wait, I'll paste
<jtv> jml: https://pastebin.canonical.com/25062/
<jtv> dodgy, yes, but perhaps not so compared to not testing this stuff at all and suffering regular breakage
<jml> jtv, you want to test that method?
<jtv> jml: no, I want to test the class I put it in without ImportErrors
<jtv> And incidentally of course I also want to mock up the call to that function it wraps.
<jtv> Without this my test can't even import the class it wants to test
<jml> jtv, I'd suggest changing it so that the method you've added only gets called if a launchpad login is not provided as a parameter.
<jml> jtv, also, I can tell you how to import that
<jtv> jml: I'd really hate to change the API though, just to test a fix for the current widespread breakage
<jml> jtv, from bzrlib.plugin import load_plugins; load_plugins()
<jtv> jml: how will that affect the other stuff we do in ec2?
<jml> jtv, I don't know.
<jtv> hmm
<jtv> and it's not tested (which is what I'm trying to make a small first step away from)
<adiroiban1> how come that this URL load a answer component, while it's in the translations namespace ? https://translations.launchpad.dev/rosetta/+addquestion
<jml> jtv, what do you mean it's not tested?
<jtv> jml: the class I'm trying to write a test for isn't tested
<jtv> I don't know how much more of this stuff is around the corner, so frankly I just want a quick fix for now
<jml> jtv, devscripts.ec2test should be running load_plugins() on import.
<jml> jtv, running load_plugins() is the quick fix.
 * jtv tries
<danilos> adiroiban1, it might be registered on more than just translations layer/facet
<danilos> adiroiban1, you'd need to look into lp/answers/browser/configure.zcml to see what's the case there :)
<adiroiban1> i tried to search the answer/browser/configure.zcml for "translations" .... but there was not such hint
<danilos> adiroiban1, btw, I've seen your branch, I'll take a look later, but I don't understand why you've got that big wdml change in there? how did you get it?
<jtv> jml: I guess I'd have to load_plugins() from the module that currently tries to import that function?
<adiroiban1> danilos: regarding the branch. I fixed it :) ... don't know how I got that
<jml> jtv, yes
<jml> jtv, is this module in the devscripts.ec2test package?
<jtv> yes
<danilos> adiroiban1, ok, cool
<adiroiban1> danilos: i think that file was modified by one of the debugging scripts.... but I have reverted and pushed a new revision... it should be nice and clean now
<jml> jtv, then load_plugins should already have been called by the time you import it.
<danilos> adiroiban1, well, it's not registered in the translations facet, it's just not limited to the answers one :)
<jml> jtv, see devscripts/ec2test/__init__.py
<jtv> jml: not much luck anyway:
<jtv>         from bzrlib.plugin import load_plugins
<jtv>         load_plugins()
<jtv>         login = get_lp_login()
<jtv> NameError: global name 'get_lp_login' is not defined
<jml> jtv, it doesn't work like that.
<jml> https://pastebin.canonical.com/25063/
<jml> https://pastebin.canonical.com/25064/ should also work
 * jml apologises for using the secret pastebin
<jtv> jml: first one doesn't...
<jtv>   File "/home/jtv/canonical/lp-branches/bug-488695/lib/lp/scripts/utilities/importfascist.py", line 163, in import_fascist
<jtv>     module = original_import(name, globals, locals, fromlist)
<jtv> ImportError: No module named launchpad.account
<danilos> adiroiban1, right, and about the answers, this is what would make the difference: http://paste.ubuntu.com/328447/ (though, I am sure some links would break if you just did this, which is why nobody ever bothered to fix it up for all the pages)
<jml> jtv, I'll try.
<adiroiban1> danilos: still lost in url traversals ... and a quick search on the net was not enough for finding a good tutorial
<jtv> jml: btw I hope the "thank you for helping with this" is implicit, but just in case: thanks for helping with this.  :-)
<danilos> BjornT_, btw, the test passes on db-devel for me locally, I'll try to figure out what's going wrong
<danilos> adiroiban1, I don't think there's one, we are doing a lot of custom things in LP as well (like one layer for each application/virtual host)
<danilos> adiroiban1, if I were you, I wouldn't worry about it too much, because Zope is very flexible in how you do it, and we use all of those possible ways :)
<adiroiban1> danilos: i was just looking at bug 193750 , and this is the only reason I bumped into url traversal .... otherwise I try to avoid it :)
<mup> Bug #193750: remove references to +addticket <trivial> <ui> <Launchpad Translations:Triaged> <https://launchpad.net/bugs/193750>
<jtv> jml: I think I'll just circumvent this entire factory that's giving me trouble.
<jml> jtv, ok
<jtv> jml: I'll still have to silence the import error of course
<jtv> how about I move the import into the factory, so circumventing the factory also circumvents the import error?
<adiroiban1> danilos: but I learned something today about the usage of layer from configure.zcml
<jml> jtv, fwiw, get_lp_login is called on the client side
<jml> jtv, so if the 'ec2' command is working at all, it should be able to be imported
<jtv> jml: any idea why I can't import it from the test?  Sorry for leaning so much on you here; I'm terrified of getting stuck on this silly detail!
<jml> jtv, which means that something in the test environment is substantially different to something in the normal usage environment
<jml> jtv, I don't know what that difference is
<jml> jtv, and I would figure out that difference using normal debugging techniques.
<jml> but I'm not going to do that, because now I am going to have lunch and start doing the things that I ought to have started four hours ago.
<jtv> jml: I understand... I'm almost out of time myself.
<jml> jtv, np
<jml> jtv, well, good luck. sorry I couldn't help more.
<jtv> hope to have good news tomorrow...
<danilos> adiroiban1, :)
<danilos> BjornT_, entire TranslationsWindmillLayer passes for me
<adiroiban1> danilos: should I then look at bug 192775 ?
<mup> Bug #192775: Notification after upload refers to "1 files" <trivial> <ui> <Launchpad Translations:Triaged> <https://launchpad.net/bugs/192775>
<danilos> adiroiban1, sure, though 193750 is easy as well (it's about replacing addticket with addquestion, not really a big deal :)
<adiroiban1> danilos: i can replace that text, I was thinking it also requires adding a "sane" link
<danilos> adiroiban1, btw, for Zope Page Templates, you might want to bookmark http://www.zope.org/Documentation/Books/ZopeBook/2_6Edition/ZPT.stx and http://www.zope.org/Documentation/Books/ZopeBook/2_6Edition/AdvZPT.stx (from ZopeBook; it's for Zope 2.x, but these chapters are very much still relevant)
<adiroiban1> to answers.lp.net ... and not to translations.lp.net/rosetta/+addquestion
<danilos> adiroiban1, you can construct such a link in the view using canonical_url with rootsite='answers', but I wouldn't worry about it for now
<bac> jtv:  are you working on the ec2 shutdown problem?
<jtv> bac: yes
<bac> jtv:  how's it going?  i was looking at it a bit too
<jtv> bac: the fix is easy, adding a test is hard
<bac> jtv: :(
<adiroiban1> danilos:  ok. I have already changed that text in my local branch (replace all +addticket with +addquestion)... I will add that branch as a merge proposal
<danilos> adiroiban1, cool, do make sure to run the full test suite, or at least all the translations tests (bin/test -vvt lp.translations)
<adiroiban1> danilos: ok
<danilos> adiroiban1, also, it's useful to note in the MP how one can easily test/see what you are changing, i.e. "go to translations.launchpad.dev/evolution/+blahblah to see how the links now point to a different location"
<adiroiban1> danilos: yes... I read the MP cover letter wiki pages and I will try to use that format.
<danilos> adiroiban1, cool, thanks :)
<adiroiban1> danilos: in lp parlance, "in the view" mean in the .pt files ?
<danilos> adiroiban1, no, it means in the browser/*.py classes :)
<danilos> adiroiban1, all those classes have names like POTemplateView or POTemplateAdminView or...
<adiroiban1> danilos: ok... then templates are just the look :)
<danilos> adiroiban1, yeah
<danilos> adiroiban1, in traditional sense, zope view is a bit of a controller and a bit of a view, and zope page template is the view
<adiroiban1> adiroiban1: ok. I can understand it... also "class" from configure.zcml can gives tips about view class
<danilos> adiroiban1, that's right, it's all tied together in ZCML
 * jml going off IRC for a bit. Back later.
<danilos> BjornT_, I've seen many (16 out of 33) tests fail with something like http://paste.ubuntu.com/328473/ in the browser with full make jscheck, so I don't know what's wrong; all translations tests pass locally
<BjornT_> danilos: that's odd. i've seen one jscheck run where a lot of tests failed, probably due to that same failure. i also can't get the translations tests to fail locally, i'm trying to run the full suite now, to see whether that makes any difference
<flacoste> morning launchpadders
<mars> morning flacoste
<BjornT_> danilos: the tests pass when running the full suite as well. i'm not sure what is going on here...
<mars> BjornT_, we may want to enhance the build output so that the LP webserver's log is also available.  Tracebacks will be printed in there.
<BjornT_> mars: yeah. it would also be nice, if you could easily boot up a buildbot slave image, and run the tests interactively
<matsubara> Chex, rockstar, danilos, sinzui, allenap: LP production meeting in 13 min @ #launchpad-meeting
<matsubara> noodles775, hi, is bigjools on holiday today?
<noodles775> matsubara: Well, I was aware that he wouldn't be here for the second half of the day, but it was a surprise to me that he's not here now too.
<noodles775> He was heading home today.
<matsubara> noodles775, right. we'll have the LP production meeting in 10 min. would you be able to cover for him?
<noodles775> matsubara: sure.
<matsubara> noodles775, thank you
<adiroiban1> danilos: should I add you as the review of MP for bug 193750 or should I found someone else ;)
<mup> Bug #193750: remove references to +addticket <trivial> <ui> <Launchpad Translations:Triaged> <https://launchpad.net/bugs/193750>
<adiroiban1> ?
<danilos> adiroiban1, you can probably get a faster review if you go to #launchpad-reviews
<danilos> adiroiban1, look at the topic to see who's online
<danilos> adiroiban1, and leave the default reviewer (it's entire LP team), so anyone can claim the review
<adiroiban1> ok. i'll do
<danilos> adiroiban1, I'd be happy to do it, but just exposing you to more of the processes :)
<adiroiban1> danilos: yep. As I'm still learning, I'm happy to go by the book
<danilos> adiroiban1, thanks :)
<adiroiban1> danilos: should I also ask for someone else to review MP for bug 487970
<adiroiban1> ?
<mup> Bug #487970: distroseries/+templates page should display last update date for templates <trivial> <Launchpad Translations:Confirmed for adiroiban> <https://launchpad.net/bugs/487970>
<danilos> adiroiban1, no, that's fine, I'd be happy to take a look at that
<adiroiban1> danilos: ok
<nhandler> I'm thinking about fixing LP Bug #400393. It looks pretty straightforward. I personally am in favor of mpt's suggestion, but I think either would work. I'm not sure how much discussion is really involved for it
<mup> Bug #400393: "Updating branch..." message reads oddly <trivial> <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/400393>
<mars> nhandler, does grep-find turn up the string "Launchpad is processing new changes" in the templates or the tests?
<mars> if so, then it should be an easy fix :)
<nhandler> mars: Yeah, a grep comes up with: code/templates/branch-index.pt:          Launchpad is processing new changes to this branch and will be
<mars> nhandler, excellent, and quite easy to fix then.  You just need to pick the new text.
<nhandler> mars: Unless there are any objections, I think I'll go with mpt's suggestion.
<nhandler> And I'm not sure if this is a known issue or an issue on my part, but I tried to do a 'rocketfuel-branch' yesterday, and got a few messages similar to "This Python has API version 1013, module gpgme._gpgme has version 1012."
<mars> well, I personally find the voice that mpt uses in his text to be a bit too advanced.  I'm partial to Tieflegger's suggestion - closer to the "Don't make me think" mantra.
<mars> nhandler, you need to run 'make clean'.  That is Python2.4 compiled code getting in the way of the Python2.5 runtime.
<mpt> My words didn't have enough syllables in them? :-)
<nhandler> mars: In that case, I might use a combination of the two. I like the more personal feel of "We'll have the changes here for you shortly"
 * jml likes mpt's suggestion
<nhandler> mars: Ok, I thought doing a rocketfuel-setup would have cleaned that stuff
<mars> mpt, well, it reminded me a bit of reading Dickens - you have to hold more than seven words in your head to understand the sentence in context.  High writing, but not right for the lows of the internet
<jml> mars, are we reading the same thing?
<nhandler> How about "Launchpad is processing new changes to this branch. We'll have them here for you shortly." ?
<mars> that's the one I liked
<mpt> mars, that one isn't in the bug report
<mars> jml, mpt used a comma, which makes the voice more sophisticated, and in my opinion, a bit more difficult to read.
<mpt> mars, no I didn't.
<jml> mars, No he didn't. This is the suggestion: "This branch has been changed in the past few minutes. Weâll have the changes here for you shortly."
<mars> mpt, wasn't that jml's suggestion?  It's in a comment.
<nhandler> Here are the 2 suggestions: https://bugs.edge.launchpad.net/launchpad-code/+bug/400393/comments/1
<mup> Bug #400393: "Updating branch..." message reads oddly <trivial> <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/400393>
<mars> jml, I was reading "Launchpad is processing new changes to this branch, which will be available in a few minutes."
<mpt> mars, that's Colin Watson
<jml> mars, that would be cjwatson's suggestion
<mars> argh, sorry mpt
<mars> I heard "mtp's bug", and missed the actual reporter :(
<mars> so, I /do/ like mpt's suggestion.  nhandler, I think you should just go with whatever is right.  And I think mpt is right :)
<nhandler> mars: Alright, that is fine by me ;)
 * mars goes to file a bug titled "A bug's reporter should stand out more on the bug report"
<mpt> mars, that's already reported :-)
<mpt> Well, actually, there's an open bug report on highlighting comments from the reporter, which wouldn't have helped here
<mpt> There's also a wontfix (iirc) report that the reporter should be more closely tied to the description, which was wontfixed on the grounds that the description can be edited by other people
<mpt> (though it wasn't in this case)
<mars> I wonder, is it in any significant number of cases?
<mars> I lack the SQL skill to whip up a query for that
<cody-somerville> yes
<cody-somerville> descriptions are edited frequently
<cody-somerville> Its part of the SRU process in fact
<cody-somerville> (for Ubuntu that is)
<nhandler> It is also a common part of the Ubuntu bug triage process
<cody-somerville> indeed
<mars> I guess knowing the reporter is only a problem if the bug is opinionated.  Ubuntu bugs are not, they are subjective shared tickets?
<mars> by opinionated, I mean it somehow makes a suggestion on a subjective topic
<mars> Ubuntu bugs are objective
<cody-somerville> I only care about who the reporter is when I have strong urges to hurt them.
<mars> which is also a subjective opinion :)
<cody-somerville> (or when I want to get in touch with the person more directly because they aren't responding to comments to the bug)
<mars> I think the original reporter does matter.  It matters more as you introduce indicators as to a person's standing in the community.
<mars> So for ubuntu, it does not matter right now
<cody-somerville> noodles775, I think it does matter
<mars> but it may matter later if we add badges
<cody-somerville> err
<cody-somerville> mars, I think your point has merit
<cody-somerville> I'm sure I would have filed a bug sooner or later
<cody-somerville> as I've been annoyed several times at the few ms delay to find the reporter name
<mars> I'm thinking of what stackoverflow does, actually
<mars> they have badges to show when a developer poses a question.  It helps you to understand the context, and allows you to make some assumptions.
<jelmer> what sort of hardware do the code imports run on, 32-bit or 64-bit?
<elmo> jelmer: a mixture
<abentley> mars: do you have a sec?
<mars> abentley, I have a few minutes
<abentley> mars: How do I make a test pause until the contents of an element have been updated?
<adiroiban1> when doing pagetest is there a way to get a link by id ? It looks I can only get on by text or url
<abentley> mars: A windmill test.
<abentley> mars: The element is a textbox.
<mars> abentley, you could try waits.forElement on something in the box that you want to see
<mars> oh
<mars> abentley, for a textbox you may have to put in a hard waits.sleep() call
<abentley> mars: I have: client.waits.forElement(xpath='//[contains(., "> content")]')
<mars> but not too long, as it should update quickly
<mars> abentley, interesting, and that doesn't work?  Try it with the sleep() and see if the XPATH works after a long enough wait
<abentley> client.waits.sleep(5) gives TypeError: __call__() takes exactly 1 argument (2 given)
<mars> ?
<mars> that is odd
<abentley> mars: Does it require named parameters?
<mars> abentley, check the docs, maybe the parameters are less than obvious
<abentley> the docs say waits.sleep takes a milliseconds parameter.
<mars> do any other tests call that function?
<abentley> mars: I tried passing it in as a named argument, and that seemed to work.
<mars> odd
<mars> abentley, salgado, I'm off for lunch, back in a while
<jml> thumper, you around?
<nhandler> To test out a bzr-related change, do I need to add my ssh key to Sample Person in order to push a branch ?
<mwhudson> jml: evening
<jml> mwhudson, hello
<jml> nhandler, I tend to use ./utilities/make-lp-user $USER
<jml> nhandler, which adds an ssh key automatically if it finds one
<nhandler> jml: Will that let me push to any user's branch in the dev environment?
<jml> mwhudson, I'm glad I made you laugh :)
<jml> nhandler, no, just the user that it creates.
<jml> nhandler, but, umm,
<jml> nhandler, './utilities/make-lp-user $USER bzr-experts' will probably let you push to any branch in the dev env
<nhandler> jml: I just looked at my ~/.ssh/config, it looks like rocketfuel setup a ~/.ssh/launchpad_id_dsa key and configured it for bazaar.launchpad.dev. It looks like it uses the user, 'mark'
<jml> nhandler, get rid of that. use ./utilities/make-lp-user
<nhandler> lol, ok
<jml> mwhudson, do you know if thumper is around today?
<mwhudson> jml: should be
<mwhudson> jml: it's only 8:40 though, he's probably herding kids
<jml> ahh right
<mwhudson> jtv: thanks for spotting the problem with ec2 test
<mwhudson> has anyone fixed it yet?
<mwhudson> my survey says no
<nhandler> jml: I've run the make-lp-user script, modified the entry rocketfuel had made in ~/.ssh/config to use the ssh key linked to the new launchpad.dev account, but I get a 'Received disconnect from 127.0.0.88: 2: Too many authentication failures for nhandler'. Is there an equivalent of bzr launchpad-login that I need to do?
<jml> nhandler, uhh, maybe
<jml> nhandler, what does 'ssh bazaar.launchpad.dev' tell you?
<nhandler> jml: Same message as above
<jml> nhandler, did you use your unix user name as the argument to make-lp-user
<nhandler> jml: I used $USER. I confirmed on the web interface that an nhandler user was created (nhandler@example.com:test) and it was linked with one of my ssh keys (which I have listed for this host in ~/.ssh/config)
 * nhandler loves how he spends more time testing his fixes than creating the patches ;)
<mwhudson> someone review this: https://code.edge.launchpad.net/~mwhudson/launchpad/ec2-test-oops/+merge/15294
<jml> nhandler, you don't need to do a bzr launchpad-login thing if ssh bazaar.launchpad.dev is failing
<jml> nhandler, umm
<jml> nhandler, try that command with -v or -v -v and pastebin it maybe?
<nhandler> jml: http://paste.ubuntu.com/328759/
<thumper> morning
<jml> mwhudson, done
<jml> nhandler, looking
<jml> thumper, hi
<mwhudson> jml: thanks
<thumper> jml: morning
<jml> nhandler, do you still have ~/.ssh/launchpad_id_rsa?
<jml> http://paste.ubuntu.com/328765/
<jml> (or dsa or whatever it is)
<jml> nhandler, add that clause to your config file and tell me what happens
<jml> thumper, would you have a moment to talk?
<thumper> jml: I have a quick moment before the stand up
<jml> thumper, when is the standup?
<thumper> in 7 min
<jml> hm
<jml> thumper, after the standup then?
<thumper> jml: suer
<thumper> sure
<jml> thumper, ok. thanks.
<mwhudson> morning
<thumper> abentley, mwhudson: skype?
<mwhudson> thumper: 1 sec
<abentley> thumper: 1 sec, skype's being slow.
<nhandler> jml: Didn't change anything. Rocketfuel setup that key for the user 'mark', should I use that user?
<jml> nhandler, delete the user line
<nhandler> jml: Still gives the same message
<jml> no it doesn't
 * jml is a little tired.
<jml> nhandler, I bet if you replace the clause with the one I pasted, make sure that you have ~/.ssh/launchpad_id_dsa then it will work
<jml> nhandler, in the SSH output you pasted, it's trying multiple SSH keys, which it wouldn't be doing if it were using a key specified in the config file
<nhandler> jml: I missed your paste. Now it is saying: ssh: connect to host launchpad.dev port 5022: Connection refused
<jml> nhandler, oh, that's much better.
<jml> nhandler, bazaar.launchpad.dev, not launchpad.dev
<nhandler> jml: I entered 'ssh bazaar.launchpad.dev' to get that error
<jml> nhandler, maybe delete the Hostname entry?
<mwhudson> thumper: ok ready
<nhandler> jml: That gives: ssh: connect to host bazaar.launchpad.dev port 5022: Connection refused
<jml> nhandler, is the server running?
<nhandler> jml: Yes
<jml> nhandler, you'll need 'make run_all' not just 'make run'
<mwhudson> nhandler: did you run 'make run' or 'make run_all' ?
<nhandler> jml and mwhudson: I had previously done 'make run'. I just did 'make run_all', and I get: ssh: connect to host bazaar.launchpad.dev port 5022: Connection refused
<mwhudson> thumper: try again?
<mwhudson> abentley: or you?
<jml> nhandler, can you please pastebin the output of 'make run_all' as ran and also 'sudo netstat -ltpn'
<nhandler> jml: Here is make run_all: http://paste.ubuntu.com/328777/
<jml> nhandler, thanks.
<nhandler> jml: Here is netstat: http://paste.ubuntu.com/328779/
<jml> nhandler, and the IP address of bazaar.launchpad.dev is...?
<jml> nhandler, what does 'ssh -p 5022 127.0.0.88' say?
<nhandler> jml: That ssh command gives me a "No shells on this server" :)
<jml> nhandler, ok. so the server is actually working.
<jml> good.
<lifeless> does anyone know all the products that launchpad uses?
<jml> lifeless, I think kfogel listed them in a recent email. you can also find them on https://launchpad.net/launchpad-project -- unless you have a narrower definition of 'uses' than that.
<jml> nhandler, so the problem is in your local config, somehow.
<lifeless> thats precisely what I wanted.
<lifeless> though ajaxbrowswer doesn't seem that lp related :P
<nhandler> jml: Strange, this is a fresh setup, so I would have thought rocketfuel would have taken care of everything.
<lifeless> jml: actually, why is https://edge.launchpad.net/ajaxbrowser in lp-project?
<jml> nhandler, funny that :)
<jml> lifeless, no clue
 * nhandler loves how the patch (which involves changing 2 lines) took about 2 minutes, and the testing for it is taking hours ;)
<lifeless> flacoste: do you know (why is https://edge.launchpad.net/ajaxbrowser in lp-project?) ?
<jml> thumper, skype audio bugs, one sec.
<lifeless> nhandler: you should reply to jml's mail ;)
<nhandler> lifeless: I've already started drafting a reply ;)
<ajmitch> nhandler: something along the lines of "the testing burns!"
<flacoste> lifeless: i have no idea :-)
<ajmitch> maybe someone randomly decided to add their project to lp-project, if that were possible
<flacoste> lifeless: i think this is because anyone can add any project to any project group
 * ajmitch wouldn't think there'd be much PHP code in launchpad-project
<ajmitch> jml: are you working on bug 146389 which you started at UDS? :)
<mup> Bug #146389: api for blueprint tracker <api> <feature> <Launchpad Blueprints:Triaged by jml> <https://launchpad.net/bugs/146389>
<lifeless> flacoste: I'd assume so too; I'll get spm to remove it later.
<flacoste> thx
<lifeless> flacoste: how many devs does the lp community have - ~ 30 ?
<flacoste> lifeless: you mean apart from the Launchpad team ? https://dev.launchpad.net/Contributions list that we have 12 contributors and two of that list are other Canonical employees
<lifeless> flacoste: no, I mean 'folk that work on it'
<lifeless> flacoste: total set of active people, not including canonical folk like me that very rarely cut code or do bug triage etc on lp itself.
<lifeless> flacoste: just want a ballpark.
<flacoste> lifeless: yeah, about 30
<jml> ajmitch, no, I'm not
<ajmitch> jml: ok, I just saw that people are screen-scraping blueprints for work items, and your demo branch from UDS looked easy enough to start from
<mars> flacoste, quick question: was one of the future plans for zc.buildout to pull the Makefile targets into their own recipes?
<flacoste> mars: it can
<flacoste> depends if you prefer make run over bin/run
<mars> ok.  I was wondering if the JavaScript and CSS build steps should be pulled into their own Makefile, or possibly a python script or zc.buildout recipe.  I guess there is no system-wide plan for that?
<mars> no plan for other components to be separated in a similar way
<lifeless> Make is pretty good :)
<mars> the question is build-system agnostic
<mars> And I see that the database does have its own Makefile
<flacoste> mars: looking at the lazr-js Makefile...
<flacoste> i think the build: target could become a recipe
<mars> flacoste, I was looking at our own Makefile, actually
<flacoste> the eggs and download-cache target should be integrated into our own bootstrap.py
<mars> that too
<flacoste> and then there is nothing in the Makefile that pulls its weight
<flacoste> now, looking at the Launchpad one
<flacoste> ...
<mars> simple for now, but it is getting a bit more cryptic with the combo loader
<flacoste> i think making a lazr-js build recipe would be a good thing
<flacoste> that can be parameterized for other projects
<mars> flacoste, ok, looks like no one has created a custom recipe so far?  I don't see anything in doc/buildout.txt.  I'll keep it in mind for the future then.
<flacoste> mars: it was part of Bjorn's plans, but he moved on
<lifeless> jml: bombs away
<lifeless> flacoste: you might like my mail (just sent) to lp-dev :)
<wgrant> lifeless: I don't see that happening without tag subscriptions.
<wgrant> Although those are pretty easy.
<mwhudson_> right
<mars> I want those now, so I can track the 'javascript' tag
<mwhudson_> yes
<jml> wgrant, mars, maybe someone will do that during the experiment
<lifeless> wgrant: I don't see why that should block the experiment.
 * wgrant sees a screaming horde of developers jump onto lifeless.
<wgrant> But maybe not.
<mars> fans!
<lifeless> wgrant: bzr has 20 times the bug density lp devs deal with, and its quite ok.
<thumper> beuno: ping
<mars> lifeless, you know what, when you put it that there are only 19 bugs per developer, then eliminating the entire backlog all of a sudden seems possible
<lifeless> mars: its interesting to me that bzr has 18 per
<lifeless> I start to wonder if its some sort of common ratio
<mars> interesting
<jml> which numbers are we using?
<lifeless> I don't think the low number means 'we can close them all', because its an instantaneous view, not the rate of change.
<lifeless> jml: 'open bugs' on the right hand side in 'launchpad-project' and 'bzr'
<jml> ~5800 / 30 = ~190
<lifeless> jml: oh,my math broke did it ? :) yes , factor of 10 on both figures.
<mars> my dreams are shattered
<jml> mars, start dreaming about being able to fix bugs 10 times faster maybe?
<ajmitch> though 19 bugs for each developer would be rather nice
<mars> jml, that is going into my email to you :)
<jml> heh heh
<lifeless> I shifted right to divide and forgot to shift left again :)
<ajmitch> bzr diff really does suffer with a cold cache :)
<lifeless> it should be legal to shoot people who operate whippersnippers
<lifeless> ajmitch: ext3 suffers :P
<ajmitch> doesn't help that I'm on a laptop with a bit of a slow drive
<wgrant> And when I throw eCryptFS and encrypted swap into the mix, it is even slower.
<lifeless> ajmitch: you could strace it next time.
<ajmitch> it was down to 1 second when I re-ran bzr diff
<lifeless> ajmitch: yes
<lifeless> 15+ seconds will be starting bzr itself from cold cache
<beuno> thumper, pong
<thumper> beuno: can you take a look at the commit message branch?
<thumper> beuno: I also wanted to talk to you about the comment layout change as I'm not happy with it
<thumper> beuno: but ...
<beuno> thumper, did you tweak it after my comment?
<thumper> beuno: I need coffee and my ride to the coffee shop is heading off
<thumper> beuno: which comment?
<thumper> damn
<thumper> too many unread emails
<beuno> :)
<beuno> we'll talk later
<thumper> beuno: do you mean move it above the review block?
<thumper> beuno: after the heading buts, but before the review table?
<thumper> s/buts/bits/
<beuno> thumper, yes
<thumper> beuno: +1 from me
<thumper> beuno: lets talk later
<beuno> thumper, super, you already have my approval, but a screenshot before landing would be nice  :)
<beuno> sure
<beuno> I'mm be around on and off
<mars> odd.  +icing is broken in all of my branches, and now in trunk, too.  I can't load files from it any more.
 * ajmitch wonders how long the test suite will take on this laptop
<mars> "The connection was interrupted" when trying to visit https://launchpad.dev on a new system setup.
<mars> it doesn't even make it to the Zope server (which is running)
<wgrant> mars: Need to restart Apache.
<wgrant> mars: One of the modules doesn't like being loaded during a reload.
<wgrant> So immediately after rocketfuel-setup, some requests will fail like that.
<ajmitch> wgrant: which one is broken like that?
<wgrant> ajmitch: Haven't got that far
<lifeless> ajmitch: test suite ... laptop. hahahahahhaa
<wgrant> lifeless: Hey, it's not too bad on mine.
<ajmitch> lifeless: some of us don't have access to ec2 instances :)
<wgrant> Once it un-swaps.
<lifeless> ajmitch: sure you do
<mars> wgrant, here's the error log.  Something is segfaulting: http://paste.ubuntu.com/328835/
<wgrant> mars: Right, there is an Apache bug somewhere.
<wgrant> I must debug that one day.
<ajmitch> lifeless: meaning that I don't want to pay too much just to run some tests :)
<ajmitch> at least my laptop has 4GB of RAM, or I could shove stuff onto my desktop with 8GB
<mwhudson> i must say, i've come to respect apache a noticeable amount less over the last year
<maxb> hmm?
<lifeless> familiarity
<mwhudson> oh just a few little things
<mars> restart fixed it, but that is a bit of an obscure bug.  New lp developers will *love* that one.
<mwhudson> mostly mod_rewrite, come to think of it
<mars> should we restart apache as part of rocketfuel-setup ?
<ajmitch> change rocketfuel-setup to do an apache restart then
<mars> ajmitch, care to review the patch? http://pastebin.ubuntu.com/328843/
<ajmitch> I'm not a reviewer, sorry, though it looks sensible
<wgrant> mars: Would it be better to alter Makefile where it's already reloaded?
<ajmitch> it should probably be done later on, once all the apache config has been done
<mars> it is, isn't it?
<mars> wgrant, and I don't see it being reloaded anywhere else?
<ajmitch> actually the reloading of apache is done in the main lp Makefile
<wgrant> Right, in the reload-apache target.
<ajmitch> 'sudo make install' calls the reload-apache target
<mars> we have a 'make install' target?
<ajmitch> yep :)
<mars> and people use it?
<ajmitch> install: reload-apache
<ajmitch> rocketfuel-setup uses it
<wgrant> I've never used it, except in rf-setup
<mars> so if rocketfuel-setup already does the right thing, then I guess there is nothing to do, without knowing what triggered the bug.
<wgrant> It currently reloads. That is the right thing, but it doesn't work. If we change it to a restart, it will work.
<mars> ah
<ajmitch> if the problem only happens on the initial rf-setup, then it may be best fixed there (after the make install), otherwise change Makefile
<wgrant> Probably.
<ajmitch> running tests on the laptop is nice until the tests sit around & do nothing, specifically testPPAUploadResultingInNoBuilds waiting forever
<mars> the only callsite for the install target that I can find is in rocketfuel-setup.  Changing the Makefile looks easiest to me.
<mwhudson> i run make install occasionally when testing changes to branch-rewrite.py
<mwhudson> if it did restart, that'd be fine
<ajmitch> most people wouldn't be relying on having apache hold connections open at that point
<lifeless> bug 488762 is a good example of the confusion I think we're having at the moment
<mup> Bug #488762: SnapShot OOPS editing team with 13000 members <oops> <Launchpad Foundations:Triaged> <Launchpad Registry:Triaged> <https://launchpad.net/bugs/488762>
<lifeless> within the same overall team (lp), its simultaneously low and high priority.
<lifeless> [note that this appears to be deliberate, it was triaged on both 'projects' simultaneously by the same dev
<wgrant> The current LP use of importance is fucked.
<mars> mwhudson, you were a build engineer; do you have a moment to review this patch? http://pastebin.ubuntu.com/328861/
<lifeless> wgrant: care to expand?
<wgrant> lifeless: Importance is treated as binary by some of the LP projects.
<lifeless> mars: why the change?
<lifeless> wgrant: important/not important?
<wgrant> lifeless: Important enough to schedule, and everything else.
<lifeless> wgrant: thats a good thing IMO
<mars> lifeless, see the scrollback for the discussion about apache segfaulting randomly after rocketfuel-setup, and how that will be real fun for new developers.
<lifeless> wgrant: as long as everything else gets broadly sorted into wishlist/defects
<lifeless> mars: \o/ then JFDI
<lifeless> +1
<lifeless> rs=lifeless
<mars> :D
<lifeless> mars: but be sure to say /why/ in your commit message to trunk, not just what.
<mars> lifeless, will do
<mars> wgrant, are you saying that the fact that some projects us a binary approach while others do not is the problem?
<wgrant> mars: I'm suggesting that the binary approach does not make sense.
<mars> mwhudson, unping, thanks
<wgrant> There are things that should not be scheduled now, but are more important than other things.
<lifeless> wgrant: what would you propose? (And are you familiar with LEAN) ?
<mars> lifeless, I don't know if lean has anything to do with it.  I think the YUI project follows the same principle.
<mars> 3.1, 3.NEXT, Wishlist
<lifeless> mars: a bunch of arguments Lean proponents make tie to together to support doing it
<wgrant> lifeless: Vaguely.
<lifeless> mars: so it can provide a good shorthand for discussing it.
<mars> ah
<lifeless> wgrant: ok, so, from the bottom then :). Some bugs may never get worked on right?
<wgrant> lifeless: Right.
<lifeless> wgrant: And others will go stale because their root cause goes away, or other things improve to stop the bug mattering
<lifeless> wgrant: One part of the argument then, is that /all/ effort - triage, analysis, discussion, filtering - done to such bugs is of no benefit to the project or the users
<lifeless> wgrant: But we don't know when a bug is in that set of bugs without putting some effort into it.
<wgrant> Hmm.
<lifeless> wgrant: now, open source projects allow people to scratch their own itches.
<lifeless> wgrant: and in a commercial project an unimportant bug may become important if a lot of users want it.
<lifeless> wgrant: so just saying 'we will close the bug if it appears low priority' isn't all that useful.
<lifeless> wgrant: but setting the relative important of two bugs which are both below the fold, isn't very useful.
<lifeless> wgrant: specifically, its not useful at all, as neither may get worked on /at all/ until either a) someone scratches their own itch (so the importance to the project does not matter at all).
<lifeless> or b) something changes to make one of them more important (so the relative importance to that other bug is invalidated anyway)
<wgrant> Hmmmhmm.
<lifeless> testtools is running an experiment at the moment
 * mwhudson looks at https://code.edge.launchpad.net/~vcs-imports/b2evolution/trunk
<mwhudson> is this an import branch or not?
<wgrant> Odd.
<mwhudson> it appears not to have an associated CodeImport
<lifeless> mwhudson: yes
<lifeless> http://bazaar.launchpad.net/~vcs-imports/b2evolution/trunk/revision/5950?start_revid=5950
<mwhudson> lifeless: indeed
<lifeless> wgrant: now, another angle is the GTD angle, where you decide once and don't change your mind. Which I think has to be transcribed for dynamic systems
<lifeless> to 'only touch it when it changes'
<lifeless> or better 'only touch it when touching it will change what you /do/'
<lifeless> ok, so dug up the testtools experiment mail. We have three importances in that project: wishlist, medium, critical.
<lifeless> Items are either 'must do before we can release'. 'Something isn't working as advertisd'. 'Something that would be nice to do'.
<lifeless> wgrant: ^
<wgrant> OK.
<mwhudson> hnngh
<mwhudson> is it bzr-builddeb that has the pristine tar stuff in it?
<lifeless> It doesn't have 6K bugs :) but its an interesting experiment.
<lifeless> mwhudson: yes.
<mwhudson> lifeless: thanks
<mwhudson> that naming thing is really going to be a pain
<wgrant> Naming thing?
<mwhudson> wgrant: bzr-builder vs bzr-builddeb
<wgrant> Ah.
<lifeless> perhaps bzr-recipe instead
<mars> wgrant, are you familiar with how the registry team uses bug importance?
<wgrant> mars: I am.
<thumper> damn, missed gary again
<mars> thumper, he wasn't here today
<thumper> oh, yeah, thanksgiving
<thumper> d'uh
<thumper> mars: so why are you?
<mars> wgrant, ok, do you find their strategy sensible?
<mars> thumper, because I'm not an American?
<mars> ;)
<thumper> mars: are you north of the border then
<thumper> ?
<thumper> mars: I'm not sure why I thought you were...
<wgrant> mars: No, but I need to think through lifeless' explanations/
<lifeless> wgrant: there's more :) I'd /love/ to see the infinite granularity stuff abentley proposed years back.
<lifeless> wgrant: but I wouldn't want to care about the relative importance of stuff below the fold.
<mars> lifeless, infinite importance granularity... you mean, an ordered list?
<lifeless> mars: yes. Imagine if you will that there was a partial sort on bugs, and the UI would let you influence that sort as much as you want.
<lifeless> so you could create a complete sort if you wanted, or just group bugs into a few bags the way we do today
<mars> lifeless, yep, thought of that, it's right out of SCRUM.  Foundations uses a Google spreadsheet as a proxy.
<mars> well, we did use it.  We dropped the experiment because it did not fit our team's style.
<lifeless> could you expand on that?
<mars> lifeless, we had a team-wide list of bugs, ordered by priority, top of the list first.  You worked down the list through the cycle.
<mars> the idea is that anyone on the team could pop the top bug off the list, but for Foundations that doesn't work too well: too many domain experts
<lifeless> mars: do you mean 'people wanted to work on thing that had not been assessed as the most important' ?
<lifeless> mars: and did you do a complete sort, or a partial sort ?
<mars> lifeless, more like it was terribly inefficient for some team members to work on the highest priority item.  Like me working on openid, or stub working on javascript.
<mars> lifeless, we did a complete sort on the list for the number of items that we could reasonably finish in one cycle.  Items in the following cycles were roughly sorted, usually at the team leads sprint.
<lifeless> mars: so, I think there are three necessary things for a sorted approach to work.
<lifeless> Firstly, I think you need plenty of slack. Must do items should be no more than 50% of estimated capacity, preferrably less.
<mars> true.  Lean says 80% capacity.
<mars> so does server load planning :)
<lifeless> Secondly, the team must support people doing things outside of the must-to items and ignoring the sort when they do this. Itch scratching/personal assessment.
<mars> queue theory, apparently
<lifeless> mars: very much so.(My dad's Master's was on queue theory ;))
<mars> lifeless, yep.  We added those items to the top of the list, coloured them red.  During the cycle retrospective, you take that into account.
<lifeless> Thirdly, everyone must be willing to work outside their comfort zone when the must-do list is in trouble.
<mars> extra items only cause the cycle's lowest priority work to get pushed a bit.  You just have to keep an eye to make sure that everything doesn't turn into an unscheduled item :)
<mars> lifeless, also true
<mars> lifeless, we did that, too
<lifeless> mars: lastly, but I don't think its necessary, just preferred, is to make it a partial sort.
<lifeless> all of the must-do items should be the same priority
<mars> ah, that is a bit different.  I think the idea from SCRUM is that some items logically come first, so there is some order in there to take advantage of.
<lifeless> mars: and yet from what you said, thats the thing that didn't work for you.
<mars> lifeless, in SCRUM, everyone is a generalist, and you do pair programming.  Our team only did out-of-domain work and paired up on occasion.
<lifeless> mars: sure
<mars> so breaking up the work into an ordered list didn't work.  In reality, everyone had their own personal list.
<mars> So we abandoned the team list experiment
<mars> And we rely on each developer to know what they are doing.  The team lead helps with knowledge sharing and workloads by moving people between domains.
<mars> when it makes sense
<lifeless> so, it seems to me that that increases coupling and latency.
<mars> coupling and latency between what, exactly?
<lifeless> the overall system
<lifeless> e.g. if there are mounting numbers of critical js bugs, you need to start to feel overload before you say to the teamlead, hey I'm overloaded
<lifeless> then they have to ask everyone 'how loaded are you'
<lifeless> and manually rebalance.
<mars> lifeless, yep!
<mars> lifeless, the counter-argument is, who will save me?  I'm the only domain expert.
<lifeless> mars: if folk overcome the fear/efficiency concerns and help out because there are critical bugs in your area, they may end up asking for mentoring - but there will be more folk hopping in and helping out.
<lifeless> the impression I got from the poppendiecks about this was that its a difficult change to make, but actually works very well in practice.
<mars> lifeless, yes, if the bug pool is open, and if they have slack.
<lifeless> mars: see necessary conditions above ;)
<mars> lifeless, one of the biggest concerns I have with our project is that very few people feel that they have slack time.  That is why I want to kill the entire bug backlog.  All of it.
<lifeless> mars: I would support mass moving them all to wishlist.
<lifeless> mars: but I don't support deletion.
<mars> hehe
<mars> lifeless, have you read Implementing Lean: Concept to Cash?
<lifeless> mars: people scratch itches; bugs are a valuable resource as well, as data for defects.
<lifeless> mars: I believe poolie is about to lend it to me; Are you talking about the honest queues aspect?
<poolie> no that's something else
<poolie> the only one i have is Lean Software Development, i think
<lifeless> ok
<lifeless> mars: no.
<poolie> pretty sure
<poolie> you know adam smith said most of this ~180 years ago :)
<lifeless> \o/
<mars> lifeless, ok.  They have an example in there about how a team reworked their 5-year bug backlog to zero, then kept zero bugs from there on out.  I think that would be amazing.
<mars> well, kept zero bugs per cycle
<lifeless> mars: do you mean 'no new defects' ?
<poolie> how?
<mars> they found new bugs, but dealt with them right away
<lifeless> mars: so, see the testtools experiment I mentioned before - 'things that might be nice', 'defects' and 'things we must do before the next release'.
<mars> I'll have to look at it again, but they just re-prioritized.  Some of them were real issues, a lot were bugs that the customers thought "Yeah, it would be nice to fix, but you don't have to"
<poolie> mars: did they mostly fix the backlogged bugs (and lift their speed greatly?) or reject them or ...?
<lifeless> in those terms, if I understand you right, the defect set would be empty, and new reports would either be 'things that might be nice' or 'things we must do before the next release'.
<mars> The "nice to fix" bugs were cleared from their radar (shelved, or something similar)
<mars> poolie, they rejected them
<poolie> (our comments overlapped)
<lifeless> mars: So, I don't like doing that, if its a good idea, I think its worth keeping it - but outside the queue.
<lifeless> mars: people can accrue data on them. I don't think it invalidates the principle to do that.
<mars> ah, found it!  Page 171
<mars> lifeless, well, they didn't say how they kept them out of the queue.  But the queue was reduced to zero.
<mars> lifeless, your "All wishlist" idea is the same thing
<mars> that would work
<mars> ah, yes, so they did a pareto analysis on the bugs, found most of them were stale
<mars> the truely high priority bugs were worked on within 90 days
<mars> everything else just went on the queue
<mars> so, dump everything that is older than 90 days :)
<mars> from lean: if you calculate the project's bug lead-time, you will see this
<mars> the high priority stuff gets worked on
<mwhudson> not all bugs (in launchpad) are defects though
<mars> it happens naturally
<mwhudson> but that's a bit sideways
<mwhudson> in many many ways one of the things that makes launchpad terrible (and also is it's USP) is that it does so goddamn much
<poolie> so
<poolie> i think it depends on what benefit you hope to obtain whether it's better to have zero bugs, or zero non-wishlist bugs
<poolie> is it a matter of clutter, or being honest about when they will get fixed
<poolie> and is it ever useful to have wishlist bugs?
<poolie> you could alternatively just sometimes make small improvements
<poolie> perhaps people will do them independently of whether there is a bug for them or not
<mars> I don't think there is an easy answer for that
<poolie> so speaking to spiv this morning
<mars> but hey, that's why we are at the cutting edge of software development processes here :)
<poolie> he pointed out that we have >100 High bugs
<poolie> it's hard to know where to start in them
<poolie> some of them could be downgraded
<jml> man I hope not.
<poolie> hello jml
<mwhudson> it's not what i actually think, but it's tempting to say "avoiding fruitless discussions" on irc could be a goal
<poolie> :)
<jml> poolie, hi
<poolie> in favor of fruity discussions
<mwhudson> for a while for launchpad-code specifically it felt like we had policies that meant various bugs were automatically categorized as high, but not that they were fixedx
<mwhudson> (we're a bit better at this now, but still have more than a page of high bugs)
<mars> mwhudson, killing the backlog is a fruitless discussion?  I think you underestimate the power that team leads have over their goals :)
<poolie> i think priorities only make sense if they determine what tasks people should do next
<poolie> the now/next/sometime idea
<poolie> "oopses are high" is meaningless unless you also say something like "do high bugs before new features or lower priority bugs"
<mwhudson> mars: any discussion is fruitless if it doesn't lead to anything
<mwhudson> mars: but as i said, it's not what i actually think
<mars> or you give the team enough slack so someone can kill the OOPS as soon as it pops up
<mars> mwhudson, well, I like the discussion.  I now know poolie and lifeless are thinking in the same space, are also experimenting, and my ideas weren't called ludicrous.
<mwhudson> mars: i was just being silly, ok?
<awilkins> This would be a stupid time to ask if there's any integration of wikis going on then?
<mars> mwhudson, ah, no problem :)
 * awilkins runs for the hills
<jml> awilkins, it's like Captain Planet says
<jml> awilkins, the power is yours
<awilkins> But what if I just have the power of... Wind  (mmmm. burritos)
<poolie> i think the bug tracker should be set up so that on any day you can choose a good thing to work on next at the minimum amortized cost
<poolie> if you have a zillion open bugs but they never get in the way of seeing the important ones, it doesn't matter
<poolie> and vice versa
<lifeless> awilkins: then you will get a windy wiki
<lifeless> poolie: did you see mars note about their team trying-and-not-liking-this (his team is a bunch of domain experts e.g. dba, js specialist, etc)
<poolie> this==single bug queue?
#launchpad-dev 2009-11-27
<lifeless> poolie: more detailed sorting than just importance
<lifeless> ah no.
<awilkins> Query on tags?
<awilkins> Hmmph, already there
<lifeless> poolie: msged you that part of the discussion
<lifeless> mars: http://en.wikipedia.org/wiki/Pareto_analysis has some interesting notes :)
<mars> lifeless, and jml posted that nice table of bug data to the mailing list a little while ago...
<lifeless> :>
<jml> mars, I have a branch on lp:~jml/launchpad/bug-stats that kind of crappily adds such charts to each project on Launchpad
<jml> mars, I'd be very grateful if you made it less crappy and landed it.
<lifeless> we could use tags to describe caues
<lifeless> like missing-workflow
<mars> jml, early next year, when I switch to lean tooling as my main priority for a quarter.
<lifeless> mars: lean tooling?
<spm> leans his tools against the nearest wall
 * jml drags his mind out of the gutter
<jml> mars, I might have made it less crappy myself by then :)
<spm> jml: hi-5 the gutter team ;-)
<jml> spm, :)
<mars> jml, LP Foundations: permanently snowed under
<lifeless> mars: on a larger scale, foundations might get more help from e.g. code  if folk saw that daily ;)
<mars> lifeless, saw what daily?
 * lifeless would actually like to disolve all the subteams.
<lifeless> mars: bugs in foundations that are critical/high
<mars> lifeless, others have thought the same
<mars> ah
<lifeless> http://en.wikipedia.org/wiki/Conway%27s_Law
<jml> good night all.
<mars> good night jml
<lifeless> ... "A real life example: NASA's Mars Climate Orbiter crashed because one team used United States customary units (e.g., inches, feet and pounds) while the other used metric units for a key spacecraft operation." :)
<mwhudson> mars: s/LP Foundations/working at Canonical/
<mars> lol
<mars> see why it is so difficult to get that critical 20% slack? :)
<mars> I'm serious :)
<mars> anyway
<lifeless> same
<mars> lifeless, so my wishlist: zero held-over bugs, all problems like OOPSes, testmode, and PQM queue visible on the same screen, 80% utilization for the project.
<lifeless> mars: I would express it something like 'zero queued defects, all team tasks on one screen, 50% load from stakeholder requests'
<lifeless> mars: (I think we still estimate too low, and need a buffer while we rearrange things)
<mars> yes, that sounds reasonable
<lifeless> mars: I use the word defect to separate out from wishlist bugs, which I think many things should just be punted to.
<mars> right
<lifeless> I think it scares folk though - folk /want/ a cozy little area-of-concern.
<lifeless> which is the big thing to break though I think. I'll need to be really clear about this in replying to the experiment thread.
<poolie> thumper: fwiw https://code.edge.launchpad.net/~vila/bzr/releasing-clarified/+merge/10854 is oopsing consistently (oops-1427ec27)
<thumper> ??!?
 * thumper wonders why
<thumper> poolie: does it oops on prod?
<poolie> yes
<poolie> i was playing with vote requests to understand them
<poolie> and i think i have the db in a state where it can't render that page
<poolie>     Module lp.code.model.branchmergeproposal, line 600, in getUsersVoteReference
<poolie>     query).one()    Module storm.store, line 1117, in one
<poolie>     raise NotOneError("one() used with more than one result available")  NotOneError: one() used with more than one result available<br />
<lifeless> \o/
<thumper> heh
<thumper> poolie: were you reassigning a pending review?
<poolie> is that even possible?
<thumper> yes
<thumper> poolie: so what were you doing?
<poolie> i was playing with bug 456643 and bug 487327
<mup> Bug #456643: Merge proposal claims a review even though I specified a different type <code-review> <Launchpad Bazaar Integration:Incomplete> <https://launchpad.net/bugs/456643>
<mup> Bug #487327: Floating "claim review" button <Launchpad Bazaar Integration:New> <https://launchpad.net/bugs/487327>
<poolie> so requesting reviews of various types from people and teams
<poolie> i meant to do this on staging but i think url completion trapped me
<thumper> poolie: you found a bug
<thumper> poolie: surprise
<thumper> poolie: please find bugs by breaking staging next time
<lifeless> thumper: it may not be possible on staging
<thumper> yeah, I know
<thumper> but in this case it is
<poolie> staging breaks itself without much help
<poolie> seriously, i did mean to be screwing around on staging
<thumper> poolie: can you file a bug with what you most remember doing?
<thumper> poolie: tag it with oops and mark it high
<thumper> poolie: I'll try to get this fixed early next week for a release-critical
<poolie> sure
<poolie> how do you reassign a pending review?
<poolie> bug 489019
<mup> Bug #489019: requesting code reviews puts the mp into a state where it can't be displayed <code-review> <oops> <Launchpad Bazaar Integration:Confirmed> <https://launchpad.net/bugs/489019>
<poolie> oh i ese
<poolie> see
<poolie> that yellow button
<poolie> so
<poolie> no, i wasn't doing that
<thumper> poolie: you used claim review on the team?
<poolie> i don't recall
<thumper> damn
<poolie> i could well have done
<poolie> i didn't reassign the reviews because until now i didn't know it was possible
<poolie> there's no log?
<thumper> no
<thumper> well, there may be an apache log
<thumper> there is an outstanding bug about history/log
<thumper> which is kinda in progress, but not
<poolie> thumper: ok i have it
<thumper> poolie: yes...
<poolie> https://bugs.launchpad.net/launchpad-code/+bug/489019/comments/2
<mup> Bug #489019: requesting code reviews puts the mp into a state where it can't be displayed <code-review> <oops> <Launchpad Bazaar Integration:Triaged by thumper> <https://launchpad.net/bugs/489019>
<poolie> basically, claim two reviews simultaneously
<thumper> beuno: if you get back, and want to talk pics, email me
<thumper> s/email/poke/
 * mwhudson :(s at the failure on buildbot
 * mwhudson also :(s at the way the code import worker tests are parameterized
<mwhudson> i really think we should switch to a single url in the code import code
<mwhudson> not have this svn_branch_url/git_repo_url/... distinction
<thumper> mwhudson: that has been requested by stub for the bzr-hg addition
<mwhudson> thumper: yeah i know
<thumper> oh, ok
<poolie> thumper: when you say in bug 382825 there's no more 'other' section
<mup> Bug #382825: code review confusing 'other' section <code-review> <confusing-ui> <Launchpad Bazaar Integration:Invalid> <https://launchpad.net/bugs/382825>
<poolie> is that because of a new change you've just done? because there is one at the moment
<thumper> poolie: yeah, it has another title now :)
<poolie> which is?
<thumper> poolie: and there is a different bug for that one
<poolie> oh ok
<thumper> ...
<thumper> how'd we end up with so many new bugs...
<thumper> "New" not new
<thumper> just in case there was confusion ;-)
<thumper> I'm sure there was something else I was going to try to fix today
<lifeless> thumper: which do you mean ? :)
<thumper> status: new
 * lifeless waits for apt
<poolie> i decided to file bugs rather than grumbling or wondering :)
<poolie> but it's ok, you had your revenge by filling my mailbox while i had lunch :)
<thumper> poolie: it wasn't just yours I was referring to
<mwhudson> thumper: you can try sticking boto in download-cache as an egg, might fix buildbot
<thumper> mwhudson: is that what is missing?
 * lifeless hates on boto
<mwhudson> thumper: yes, the tests now try to import it
<thumper> mwhudson: which tests?
<thumper> does this have something to do with twisted 9 ?
<mwhudson> thumper: no, this is the fix for the ec2 test --headless problem jtv landed
<thumper> mwhudson: didn't you have a fix too?
<mwhudson> thumper: mine didn't have a test
<thumper> heh
 * thumper suggests jtv fix it :)
<mwhudson> i guess he'll be up fairly soon
 * thumper is about to hurl in the towel
<mwhudson> for today, or because of something specific?
<thumper> for today
<thumper> feels like it has been a draining afternoon
<mwhudson> fair enough
<mwhudson> have a good weekend!
<thumper> I'll pop back later to try to land those branches
<poolie> cheerio thumper
<poolie> mwhudson: i can't remember if you actually answered before:
<poolie> is me getting rid of python logging going to actually break anything for you?
<poolie> re https://code.edge.launchpad.net/~mbp/bzr/remove-logging/+merge/15306
<mwhudson> poolie: i'm not sure you actually asked, i did wonder why you requested a review from me
 * mwhudson looks
<mwhudson> poolie: i can't see anything obvious
<poolie> k
<poolie> more just thought i should let you know
<poolie> in case you were hooking into bzrlib this way
<mwhudson> we were sort of planning too
<mwhudson> but luckily we never got around to it :-)
<poolie> <spm> [...] would necessitate a short outage. I'd estimate ~ 10-20 seconds.
<poolie> hope springs eternal ;)
<spm> :-)
<spm> it should be just shutdown apache; start it up with shiny new configs and things just work.
<spm> But it's a major - potential - disruption to a critical service, so not one I'm willing to JFDI
<poolie> agree
<spm> ideally - we'd schedule it for when I'm in a car and driving up to sydney for the xmas party and thus unable to assist in any failures. but that'd be lazy/optimisim at work ;-)
<adeuring> goodmonring
<MTecknology> golly... you guys should come up with some way to help users see if there's an existing bug that matches the subject of the one they want to report to help eliminate duplicates...
<MTecknology> something like.. idk.. you type the short description; click a button before continuing that gives you a list of possible dupes..
<MTecknology> -_- | sorry, just getting irritated when there's 32 dupes of a single bug that very accurately describes the issue. Even if they don't see that one, they should at least see the 32 others
<jml> flacoste, does having testtools on PyPI make it easier to use as a Launchpad dependency?
<BjornT_> jml: it shouldn't matter much where it's located. the most important thing is that it has a proper setup.py, so that eggs can be generated. having it on PyPI only makes it slightly easier to download the sdist.
<jml> BjornT_, thanks.
<jml> BjornT_, you might be interested in the assertThat stuff that lifeless recently added to testtools.
<BjornT_> jml: what does it do? or where can i easily see it?
<jml> BjornT_, http://bazaar.launchpad.net/~testtools-dev/testtools/trunk/annotate/head:/MANUAL is probably the best starting point
<jml> BjornT_, it's a way of avoiding adding lots of domain-specific assertFoo methods to a base class by moving the assertion logic out to separate classes that implement a well-defined interface
<jml> what needs to be done to close off bug 139855?
<mup> Bug #139855: Display stats about PPA usage <feature> <Soyuz:Triaged> <https://launchpad.net/bugs/139855>
<noodles775> I think it just needs scheduling - afaik, wgrant has done a lot of the required backend work for it.
<jml> noodles775, that's good. :)
<noodles775> Well, when I say 'just', I'm sure it depends on exactly *what* stats are required etc. :).
<jml> noodles775, but I actually meant, what if someone off the street wanted to fix it.
<wgrant> noodles775, jml: The log parser has been refactored, so the backend work is just a couple of dozen lines of code. But then there's UI.
<noodles775> The UI shouldn't be complicated - we've left space in the portlet on the PPA index page for it.
<jml> wgrant, what is it that makes the UI more complicated than just a number?
<wgrant> jml: A number where?
<wgrant> Per binary?
<wgrant> Per binary name?
<wgrant> Per source?
<wgrant> Per release?
<wgrant> Per version?
<wgrant> Argh.
<noodles775> lol
<jml> I see.
<wgrant> Yay Soyuz/
<jml> what do you think people care about?
<noodles775> So it's aggregating the appropriate stat that's the issue?
<wgrant> noodles775: Right.
<wgrant> jml: I do not know.
<jml> wgrant, were it me, I think I'd like to have the raw data available for all of those, and I'd probably like some kind of aggregated statistic.
<wgrant> jml: Right.
<jml> so how do we go about picking one?
<wgrant> The stats will more than likely be keyed on (day, country, archive, binarypackagerelease), and from there you can do a lot.
<jml> is this the kind of change that would show up on edge before a rollout?
<wgrant> Not unless the backend makes it onto production first.
<jml> and that's work that's not yet done, right?
<wgrant> Correct.
<wgrant> Most of it is done (the refactoring was the big bit), but not the actual feature addition.
<jml> ok. I was just wondering, maybe we could land an obviously wrong UI shortly after the release, and then learn from our experiences using it on edge.
<jml> by "obviously wrong" I mean "our best guess made with little deliberation", which is roughly the same thing.
<wgrant> Right.
<jml> we could even blog about it and post to the mailing list to get feedback from users who aren't intimately involved with Launchpad development.
<jml> I know the Bazaar guys really want this, and I'm confident they'll have informed & interesting opinions :)
<wgrant> Definitely good ideas.
<jml> I'm glad you think so. :)
<jml> so, about that backend work...
<jml> wgrant, are you planning on doing it before the rollout?
<wgrant> jml: I'm not. I'm busy with real work and other stuff.
<wgrant> jml: Basically, it needs PPA versions of cronscripts/parse-librarian-apache-logs.py and lib/canonical/launchpad/scripts/librarian_apache_log_parser.py
<wgrant> Plus the DB tables.
 * jml looks.
<wgrant> There's not much remaining in either of them.
<jml> indeed not.
<jml> wgrant, which DB tables, exactly?
<wgrant> jml: ArchivePackageDownloadCount, or something along those lines.
<wgrant> LibraryFileDownloadCount is the existing one.
<jml> wgrant, thanks. I've added a comment to the bug summarizing this discussion, in the hope that I or someone else will act on it.
<wgrant> jml: Looks good.
 * wgrant sleeps.
<jml> wgrant, g'night
<noodles775> Hi BjornT_ , would you be able to look over the last comment on bug 487009 (and the linked diff) with your SA goggles on?
<mup> Bug #487009: Generalise IBuilder <Soyuz:In Progress by michael.nelson> <https://launchpad.net/bugs/487009>
<flacoste> jml: testtools in PiPy: it makes thing a little bit easier, in that you can rely on the default buildout behaviour to get updates, but once the release is in the download-cache, it doesn't make a big difference
<BjornT_> noodles775: it seems a bit odd to adapt a job type. adapting an actual job seems more natural
<noodles775> BjornT_: yes - that would be much better (I'd headed down that path as I initially had a factory there).
<noodles775> BjornT_: although, that would then require adding a factory that knows about the different types, wouldn't it?
<noodles775> That was what I was trying to avoid too (having multiple places that need to know about the different types)
 * bigjools-afk wonders if he has some electromagnetic field that causes power cuts where ever he goes
<BjornT_> noodles775: no. you can mark the job as providing the right interface, depending on which job type it is
<noodles775> BjornT_: ah, great!
<BjornT_> noodles775: you can see an example in BugTask._init()
 * noodles775 looks
<bigjools-afk> noodles775: same sort of thing happens in Archive.__init__
<noodles775> Right.
<zul> hi guys, im doing the canonical-application-support sec (https://blueprints.edge.launchpad.net/ubuntu/+spec/server-lucid-canonical-application-support) I was wondering why launchpad doesnt use python-subversion rather than python-svn since python-svn is not in main?
<jml> zul, If I had to guess, it's because either launchpad-cscvs or bzr-svn depends on python-svn in particular
<zul> jml: thats reasonable then
<jml> zul, it's probably worth sending an email to mwhudson (and the launchpad-dev list) about this.
<zul> jml: will do
<jml> zul, if the python-svn dep is because of launchpad-cscvs, it's possible we'll phase this out.
<beuno> rockstar, how's your UI review crusade going?
<jml> lifeless, I can't remember if it was you or poolie who suggested I open N tabs of bugs in (firefox, chrome) and time how long it took for all the tabs to load
<mars> does anyone have a moment to verify a JavaScript bug on db-devel?  I'm getting it consistently on local, but staging is clean.
<jml> lifeless, but for ten tabs, ~24s for firefox, ~13s for chrome. same bugs.
<mars> ah, wait a minute, it might be the devmode switch
 * mars tries it
<mars> jml, is that with a clean firefox profile?
<jml> mars, no, it's with my dirty chrome & firefox profiles.
<mars> ah.  Just wondering.  Things like Firebug can really mess with the results of such a test.
<jml> mars, kind of.
<jml> mars, depends on what I'm measuring, really. I'm not going to uninstall firebug just because it makes Launchpad faster :)
<mars> :)
<mars> ok, the error is still present on my local setup
<mars> so, can anyone spare a moment to verify that the bug is present on db-devel when run locally?
<jml> I can have a go
<mars> jml, cool, thanks
<jml> I might give up if it's too hard for a friday evening
<mars> jml, shouldn't be
<jml> ~17s on "safe mode" firefox, no extensions or themes.
<mars> jml, please pull the latest db-devel, edit configs/development/launchpad.conf and shut off devmode, make schema && make run, and visit any bug.
<jml> mars, this will take a while. I haven't run make clean in my db-devel branch since the python switch
<mars> no problem
<mars> thanks for the heads up though.  I'm doing the same here.
<mars> and the bug is /still/ there
<jml> mars, is it in db-stable?
<mars> hmm, haven't tried that yet
<mars> jml, watched your "Teach me Packaging" talk.  That was interesting, and a nice presentation idea.  It worked really well with the skills of the audience.
<jml> mars, thanks
<jml> mars, I stole the presentation idea from Steve Holden, who did a "Teach me Twisted" talk at the preceding PyCon.
<jml> mars, he had two hours, and a bottle of whisky.
<mars> heh
<mars> jml, so could you successfully create your own python packages after that talk?
<mars> I know other people appear to have an easy enough time creating them
<jml> mars, almost. It's not actually a great way for me personally to learn -- I should have been more prompt in writing up the notes and blogging them.
<mars> and the bug is present on db-stable as well
<mars> with devmode on and off
<jml> but not on staging
<jml> thus, it might be caused by something in r8722..8729
<mars> yes
<mars> when does the next staging update take place?
<jml> that, I don't know.
<mars> something to add to the future developer control-panel
<mars> in the name of making blockers visible
<mars> losas, do you know when the next staging update will be?
<mars> jml, any luck reproducing the error on your local system?
<jml> mars, I'm looking at a bug page. What's the error?
<mars> jml, A JavaScript error, 'display_name' is undefined
<jml> mars, yes, I get that in firefox
<mars> jml, ok, thanks.  So it does happen locally.  And I don't know why I am not seeing it on staging.
<mars> it is present on r8721 as well, so before the last update.
<jml> mars, http://pastebin.ubuntu.com/329617/ does rather look like the bug subscriber code has changed
<jml> mars, which seems to be where the js error is coming from
<mars> jml, yes, so perhaps the CSS classes are generated differently now
<mars> yep, quite possibly.  Line 769 of the diff switches the CSS to use a different execution path
<mars> unlike line 756, which uses the old code-path
<mars> I am still suspicious though, because r8721 locally shows the error.  This diff is for r8722..
<jml> mars, I don't get it in r8721
<mars> jml, bzr branch -r8721 ?
<mars> I did have that correct?
<jml> mars, 'bzr revert -r8721' in db-stable is what I did
<jml> then Ctrl-Shift-R to be sure I wasn't getting stale js
<mars> yes, I did control-F5
<jml> oh wait
<mars> I'm seeing it in trunk as well
<mars> so it must not be a YUI3 problem
<jml> no, I do get it in r8721
<mars> ok
<jml> mars, if you are seeing it in trunk, then I recommend using the stable branch and winding back (starting from 9944, which is 8722 in db-stable) until you don't see the error any more
<Chex> mars: staging update is finishing up now, should be done in 2-3 hours or so
<mars> Chex, great, thank you
<mars> jml, ok, bug filed: #489342.  Thanks for all the help so far.
<mup> Bug #489342: "'display_name' is undefined" JavaScript error on development <javascript> <Launchpad Bugs:New> <https://launchpad.net/bugs/489342>
<jml> np
 * jml off downstairs to stay awake for another couple of hours
<mwhudson> zul: launchpad (through cscvs) depends on *both* python-svn and python-subversion
<mwhudson> zul: if from this you deduce that cscvs isn't the greatest thing ever, well...
<lifeless> jml: me
<lifeless> jml: poolie may or may not have suggested it; I khnow I have :)
#launchpad-dev 2009-11-28
<henninge> Is PQM really still open?
<henninge> allenap: still around?
<allenap> henninge: I'm here now.
<allenap> henninge: And I guess that PQM is not open. I get the feeling that it was shut deliberately overnight.
<henninge> allenap: yes, usually it shuts down Friday night.
* henninge changed the topic of #launchpad-dev to: This is Launchpad Development Channel | Week 3 of 3.1.11 | PQM is appearently closed.  | I am Zero OOPS and So Can You! http://is.gd/4fkLl |   https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in #launchpad-reviews | Use http://paste.ubuntu.com/ for pastes | This channel is logged: http://irclogs.ubuntu.com/
<allenap> henninge: Right, thanks, shame, have a good weekend :)
<henninge> allenap: you too ;)
#launchpad-dev 2009-11-29
<mwhudson> morning
<thumper> mwhudson: hi
<thumper> mwhudson: call?
<mwhudson> thumper: couple minutes, but yes
<thumper> ok
<mwhudson> thumper: choose your technology
<thumper> mwhudson: skype
<mwhudson> thumper: calling
<lifeless> http://pypi.python.org/pypi/gerald/0.3.1
#launchpad-dev 2010-11-29
<poolie> is it normal that i have to run 'bzr update' in download-cache as well as update-sourcecode?
<jml> poolie: yes.
<wgrant> rocketfuel-get does both.
<poolie> but it's deprecated, according to some people?
<wgrant> NAFAIK
<poolie> spm: when lpnet is deployed, do you run 'make' in an existing tree, or a clean tree?
<jml> poolie: I've never used it, since I felt it was a bit crappy to be working on bzr support for Launchpad and not actually using bzr to get my code
<spm> poolie: completely clean, the process is something like:
<spm> copy new tree as launchpad-revno-<revno>; make build in that; shutdown service; ln -s switchero (launchpad -> newshiny revno); startup
<lifeless> mwhudson: hi
<lifeless> mwhudson: re apache mangling; is there a bug for codebrowse?
<mwhudson> lifeless: hi
<spm> poolie: I grossly simplfy, but that's the gist.
<mwhudson> ah no, i don't think so
<mwhudson> let me search a bit
<lifeless> mwhudson: we'll need one marked as a blocker for becoming an open id consumer
<lifeless> mwhudson: probably needs a task on foundations
<lifeless> not because they should do it but because they are doing the openid stuff and need to see it
<poolie> spm that's what i thought
<poolie> i was just trying to work out bug 680339, but
<_mup_> Bug #680339: 'Entry' object has no attribute 'markAsDuplicate' <Launchpad Bugs:New> <https://launchpad.net/bugs/680339>
<poolie> i suspect it's an interaction thing and not a real bug
<wgrant> poolie: markAsDuplicate is a mutator.
<wgrant> It's not exposed as a method; you just set the attribute.
<StevenK> I thought that changed, depending on the API version used?
<wgrant> Probably.
 * jml gone again, back slightly later than UK starting time tomorrow morning
<wgrant> But I really hope not.
<wgrant> But probably.
<poolie> wgrant: it was exposed as a method
<poolie> it is now exposed as .duplicate = ...
<StevenK> wgrant: Would you believe I still can't nail down where the spph is created for an nascent upload
<poolie> when i first wrote my client code, the mutator didn't exist
<wgrant> StevenK: PackageUploadSource.blahblah
<poolie> now the method seems to have disappeared from the 1.0 api
 * StevenK declares tea will magically point it out to him
<wgrant> poolie: Well, did you know that Launchpad likes to make API stability guarantees without actually thinking about whether it can support them?
<poolie> i was kind of suspecting that
<poolie> the "api stability" thread prompted me to think about it again
<poolie> but that turned out to be a somewhat different question
<wgrant> AIUI the discussion basically went like this: "We need to have a stable API for Ubuntu releases to use." "OK, you can now annotate methods as being for a particular version." "OK, you must now support beta and 1.0 for 5 years, with no warning."
<wgrant> Now we cannot change our datamodel.
<lifeless> so
<lifeless> thats an exaggeration, and you know it :)
<poolie> i'm curious where those annotations go
<lifeless> in the export() call
<lifeless> see salgados blueprint branch, for instance.
<lifeless> that exports only on devel (at my request)
<wgrant> API stability guarantees are good. Simply freezing an entire API version is not.
<lifeless> agreed
<lifeless> EBeforeMyTime
<wgrant> The bits of the 1.0 API that need to be stable for 5 years are probably limited to a couple of methods.
<lifeless> perhaps
<wgrant> StevenK: Found it yet?
<StevenK> wgrant: Nope
<wgrant> StevenK: It should be in queue.py somewhere.
 * wgrant looks.
<wgrant> PackageUploadSource.publish
<wgrant> newSourcePublication
<StevenK> Ah ha. There we get the SPR, so digging back for the history should be easy
<wgrant> HA HA HA
<thumper> gah
<thumper> I have a test that regularly fails in ec2 but passes locally
<thumper> s/regularly/always
<thumper> has to do with recipe breadcrumbs of all things
<poolie> lifeless: you're saying the export decorator controls which versions it appears in
<poolie> that seems plausible but markAsDuplicate calls it with no parameters
<wgrant> lib/canonical/launchpad/rest/configuration.py:    last_version_with_mutator_named_operations = "beta"
<wgrant> So in beta, mutators appear as named operations.
<wgrant> In 1.0 and devel they do not.
<thumper> wallyworld_: ping
<wallyworld_> thumper: hello
<thumper> wallyworld_: we should have a chat
<thumper> although I might go make a coffee first
<wallyworld_> thumper: ok
<thumper> sound good?
<wallyworld_> thumper: ack
<thumper> wallyworld_:  bzr+ssh://bazaar.launchpad.net/~thumper/launchpad/recipe-binary-builds r11904
<thumper> TestSourcePackageRecipeView
<thumper> ICanHasLinkedBranch(product).branch
<thumper> ICanHasLinkedBranch(sourcepackage).branch
<thumper> ICanHasLinkedBranch(productseries).branch
<lifeless> === Top 10 Time Out Counts by Page ID ===
<lifeless>     Hard / Soft  Page ID
<lifeless>      260 / 6568  Archive:+index
<lifeless>      142 /  301  BugTask:+index
<lifeless>       26 /  290  Distribution:+bugs
<lifeless>       26 /  126  ProjectGroupSet:CollectionResource:#project_groups
<lifeless>       10 /    4  Cve:+index
<lifeless>        9 /   29  Milestone:+index
<lifeless>        9 /   15  DistroSeries:+queue
<lifeless>        8 /  107  Archive:+packages
<lifeless>        8 /   12  DistroSeriesLanguage:+index
<lifeless>        6 /    3  Person:+bugs
<thumper> Difference: !=:
<thumper> reference = 'Request builds for cake_recipe\nMaster Chef\nRecipes\ncake_recipe\nRequest builds for cake_recipe\nArchive:\nSecret PPA (chef/ppa)\nDistribution series:\nSecret Squirrel\nHoary\nWarty\nor\nCancel'
<thumper> actual = u'Request builds for cake_recipe\nMaster Chef\nRecipes\ncake_recipe\nArchive:\nSecret PPA (chef/ppa)\nDistribution series:\nSecret Squirrel\nHoary\nWarty\nor\nCancel'
<wallyworld_> lifeless: aren't you meant to be on leave?
<lifeless> wallyworld_: I am
<wgrant> wallyworld_: He's not very good at it.
<thumper> lifeless: if you want something to do, fix my bug
<thumper> lifeless: it passes for me and wallyworld_ but fails in ec2
<lifeless> thumper: I'm not doing work
<wallyworld_> lifeless: well fcuk off then and enjoy your break :-)
<thumper> lifeless: no, this would be fun :)
<lifeless> wallyworld_: I am, I've scrastched a bunch of itches in lp:testrepository
<lifeless> and watched a movie and a half, plus judge john deed
<lifeless> cleaned up a bit in my office
<lifeless> and got wow reinstalled.
<wallyworld_> lifeless: i still think you need some "downtime" :-)
<StevenK> lifeless: O.O at the last one
<ajmitch_> StevenK: you managed to break free from that?
<StevenK> ajmitch_: Er, not quite
<lifeless> StevenK: google for wine + could not fix permissions, or thereabouts - launcher.exe fail.
<ajmitch_> let me guess, waiting impatiently for the 7th?
<StevenK> ajmitch_: Yes :-)
<lifeless> wallyworld_: thats why I have a long block of leave. It takes me weeks to spin down.
 * wgrant never fell into the WoW trap.
<lifeless> we can help
<StevenK> lifeless: Is that what you're currently tripping on?
<StevenK> Haha
<lifeless> StevenK: was
<ajmitch_> wgrant: be glad :)
<lifeless> StevenK: thus the reinstall.
<StevenK> lifeless: Has it downloaded the 10GiB of patches it needs?
<lifeless> the incremental downloader is damn awesome
<lifeless> yeah
<lifeless> its all good, I logged in and checked its working. Still haven't decided if 'm going to get the upgrade or not.
<wgrant> StevenK: Ah, so crazy Blizzard patches aren't just limited to SC2? When I reinstalled last week it must have reached 100% eight or nine times...
<StevenK> wgrant: Oh no, WoW is where they perfected that
<lifeless> wgrant: on wine or windows?
<wgrant> lifeless: Both.
<StevenK> ajmitch_: Does that mean you pulled yourself out of it?
<ajmitch_> StevenK: well I stopped raiding at least, that has to count for something?
<StevenK> Haha
<StevenK> Neat. parallel-test #16 has been building for 2 days 18 hours
 * StevenK kills it with fire
<LPCIBot> Project parallel-test build (16): ABORTED in 2 days 18 hr: https://hudson.wedontsleep.org/job/parallel-test/16/
<lifeless> ajmitch_: I found getting a new job worked wonders
<ajmitch_> lifeless: worked wonders for getting you away from wow for a bit?
<StevenK> A bit?
<StevenK> Like 10 months, more like
<lifeless> moving country helped too
<lifeless> StevenK: 6
<lifeless> StevenK: april
<StevenK> lifeless: That's 7, nearly 8 :-P
<ajmitch_> I would complain about the state of internet connectivity here, but it wasn't much different in .au I guess
<lifeless> I had a loose heatsink then
<lifeless> wondered why I was getting thermal dmesgs and low performance
<LPCIBot> Project parallel-test build (17): ABORTED in 12 min: https://hudson.wedontsleep.org/job/parallel-test/17/
<LPCIBot> * William Grant: PgTestSetup needs BaseLayer now, so set up BaseLayer and use the PID when checking that it respects LP_TEST_INSTANCE.
<LPCIBot> * William Grant: test_mlists now invokes scripts with the correct appserver config.
<LPCIBot> * William Grant: Fix restful-cache.txt to not depend on 'testrunner'.
<LPCIBot> * William Grant: Fix missed rename.
<LPCIBot> * Robert Collins: Merge wgrants fixes.
<LPCIBot> * Robert Collins: Refactor db layer db setup/teardown to make the db available to subordinate layers.
<LPCIBot> * William Grant: Scripts' -d arguments now properly override the DB name again.
<LPCIBot> * William Grant: Fix two missed canonical.lp.dbname references.
 * StevenK blinks.
<StevenK> I didn't do that
<lifeless> I did
<lifeless> the branch it was testing has landed in db-stable
<lifeless> my librarian branch is the Next Big Thing
 * spm gibbers
<lifeless> spm: does that mean you need gibbing?
<wgrant> The librarian branch seems to work. There's just a couple of dozen tests that need to be dehardcoded, and another one or two which have more complex failures.
<StevenK> lifeless: Should I change it to devel, or you've done it?
<lifeless> StevenK: i changed it to my librarian branch
<wgrant> The main issue after that is the SMTP server.
<lifeless> yes
<lifeless> bugs filed for that :)
<wgrant> Ah, great.
<lifeless> which interested people need to do
<spm> lifeless: needing gibbing? that sounds a touch personal?
<wgrant> It looks like we might have to write out ZCML... or otherwise hack around with Zope internals.
<lifeless> spm: hey, you're the gibberer
<lifeless> bah
<lifeless> OperationalError: FATAL:  database "launchpad_ftest_template" does not exist
<wgrant> Hm? Where?
<wgrant> In a parallel run?
<lifeless> https://hudson.wedontsleep.org/job/parallel-test/18/console
<lifeless> theres a test somewhere that removes it
<StevenK> lifeless: That might be fallout from using the same build slave as #17
<lifeless> StevenK: no
<lifeless> I filed a bug on it back a month
<lifeless> I was hoping it had been caught
<wgrant> Possibly test_sampledata
<wgrant> It does a pg_restore
<LPCIBot> Project parallel-test build (18): STILL FAILING in 33 min: https://hudson.wedontsleep.org/job/parallel-test/18/
<StevenK> That's a bonus, it actually fails
<adeuring> good morning
<poolie> hi abel
<mrevell> Hi
<wgrant> bigjools: Morning.
<bigjools> morning
<wgrant> bigjools: What's the current status of the logparser?
<bigjools> wgrant: I landed the gzip fix but we're stuck on that error I showed you
<wgrant> Oh, right, forgot about that.
<bigjools> steve thinks it might be the 32 bit field I am reading in overflowing but that raises the question about how did gzip itself write the field
<wgrant> Hm? gzip rights the lowest 32 bits of the size.
<wgrant> Anything higher is stripped.
<bigjools> indeed
<wgrant> s/rights/writes/
<wgrant> It's a completely irrelevant traceback, but that probably is it.
<bigjools> it's a Storm traceback
<bigjools> so it's caused by writing something bad into the db
<wgrant> I wish we could turn off write caching.
<wgrant> So we could actually see which line it was.
<bigjools> hmmm interesting idea
<wgrant> ... oh.
<bigjools> there might be a way
<wgrant> It's a postgres integer.
<wgrant> Which is 32-bit.
<wgrant> That would be the problem.
 * bigjools gets coffee to think about it
<wgrant> Can you alter it to a bigint and see if it works?
<wgrant> Although that means we're above the 32-bit limit, so the gzip size calculation will be screwed.
<wgrant> Yya.
<deryck> Morning, all.
<bigjools> wgrant: still there?
<bigjools> howdy deryck
<wgrant> bigjools: I am.
<deryck> gmb, hey, man.  I need to settle in a bit this morning and then I'll turn to that troublesome lazr-js tarball.
<bigjools> https://launchpad.net/~openswan/+archive/openswan-testing/+packages
<bigjools> observe and wonder
<wgrant> Uhoh.
<wgrant> What's broken?
<wgrant> Why's it not published?
<bigjools> two i386 builds ...
<wgrant> Oh.
<bigjools> one from a different PPA
<wgrant> WTF.
<wgrant> Yeah.
<bigjools> WTF indeed.
<gmb> deryck: Sure, no worries. I'm working on other stuff at the moment anyway.
<deryck> gmb, ok, cool.
<gmb> (I looked at the lazr-js stuff, decided that I didn't have enough brainjuice to comprehend the problem and thought it best to leave it til later)
<gmb> deryck: If I can help in any way, though, let me know.
<deryck> gmb, sure, thanks!  will do.
<wgrant> bigjools: I recall being scared away by SPR.getBuildByArch a few months ago.
<wgrant> There was something strange with how it determined the build given published binaries.
<wgrant> I don't quite remember what, but it sounds relevant.
<wgrant> Let's see...
<wgrant> Oh, that's right.
<bigjools> wgrant: somehow, the copy he did from his personal PPA to the openswan one  overlapped with a build in the openswan PPA
<wgrant> It made me cry because it issues so many damn queries.
<wgrant> bigjools: The openswan build is newer.
<bigjools> yes
<wgrant> Yessssss, I remember this code now.
<wgrant> Potentially hundreds of queries per build!
<wgrant> bigjools: Hmm. The creation date on the build appears to be ~10min after publication.
<wgrant> Oh.
<wgrant> No.
<wgrant> 1:50 *before*.
<wgrant> So it was copied, had the build created, deleted, then copied back.
<bigjools> the first had rebuild, the second didn't
<wgrant> Presumably.
<wgrant> Can you confirm that?
<bigjools> no :/
<wgrant> Hm, the API should be helpful here.
 * wgrant tries.
<bigjools> new b-m running as of 30 mins ago BTW
<wgrant> What's changed?
<bigjools> the extra pre-build ping
<wgrant> Oh, right.
<wgrant> Does it work?
<bigjools> shipova correctly failed
<bigjools> needs more load to be certain
<bigjools> we'll find out when the faily builds are dispatched
<bigjools> wgrant: so I suspect he deleted that package while it had a build in progress.  Will have to play on that and see what DF does.
<wgrant> Did you deliberately schedule the daily builds so you're asleep? :P
<bigjools> I didn't schedule anything
<henninge> jtv: name an arbitrary upstream project, please. ;-)
<jtv> openobject-addons
<jtv> If this is for verifying the migration script I'd look for at least: one with multiple series, one that's in ubuntu main, and one that's just very bigâideally of course one that has all.
<jtv> Are you going to compare exports from staging (post-migration) to ones from production?
<jtv> XPI won't work yet; KDE may possibly also still be broken.
<henninge> jtv: as a start I wanted to see how the merge affects upstream translatoins.
<henninge> openobject-addons is not linked to an ubuntu-packge, though
<jtv> No.
<jtv> But it's nice and large.
<jtv> Just to make sure I'd also see if there's any effect on the Ubuntu package.
<jtv> (For a product that _is_ linked to main packages, of course)
 * bigjools -> fud
 * henninge lunches
<bigjools> wgrant: argh
<gary_poster> abentley, hi.  Do you happen to know what would happen to a bzr checkout of a branch that is upgraded to 2a after the checkout?  Would bzr up fix it, or would devs need to blow the branch away and check it out again?
<abentley> gary_poster: bzr up would not revert it to the previous format.  You'd need to upgrade the branch or downgrade the checkout.  You can downgrade the checkout by blowing it away and re-checking out, or using "bzr upgrade --FORMAT".
<lifeless> bzr upgrade should work - we designed it to handle most cases.
<gary_poster> abentley: sorry, I meant the original branch was upgraded, but ok, thanks lifeless, sounds like we can make it work one way or another.
<abentley> lifeless: well, he was asking about "bzr up", which is "bzr update" IIRC.
<gary_poster> that's correct abentley
<gary_poster> the story is this:
<gary_poster> I check out the download-cache, which is not 2a
<gary_poster> I update the launchpad branch of download-cache to 2a
<gary_poster> what do I do now to make my local checkout happy?
<abentley> gary_poster: "bzr upgrade" in your checkout should do the trick.
<gary_poster> ok awesome, thanks abentley
<gary_poster> trying :-P
<gary_poster> gmb, deryck, did you get the lazr-js version mismatch resolved?
<deryck> gary_poster, heh.  I literally *just* worked this out. :-)
<gary_poster> heh, ok :-)
<deryck> gary_poster, I believe http://pastebin.ubuntu.com/537940/ shows the issue.
 * bigjools dances around the room, I finally sorted out buildd-manager timeouts
<Ursinha> flacoste, hi
<gary_poster> deryck: ah-hah, yes, I agree
<deryck> gary_poster, so I'm fixing lazr-js now and will roll a new tarball.
<gary_poster> cool deryck.  /me is sympathetic to your many woes with this stuff :-/
<deryck> thanks. :-)
<deryck> it's been so amazingly painful, I've quit complaining or worrying about it really
<gary_poster> probably a good idea. :-
<deryck> rockstar`, you around?
<bigjools> anyone know if I can turn off caching in storm so the traceback I get on a store flush is attached to the code that caused it?
<bigjools> ah nm
<gary_poster> Can I have a volunteer to try upgrading your download cache to see if these instructions work for you?  Assuming you have a checkout (which rocketfuel uses), you would do a bzr upgrade followed by a bzr up.  It works for me, but I own the LP branch.  Could someone verify that it works for them too before I send an email out?
<abentley> bigjools: mazel tov!
<bigjools> abentley: I'm not jewish but thanks :)
<bigjools> abentley: feel free to push ahead with your recipe changes
<bigjools> I'm landing another branch later that makes all the built file downloads asynchronous
<abentley> bigjools: Cool.  Does this mean we can drop this rlimit on recipe memory use?
<bigjools> abentley: I don't know, we could try and see what happens.  The worst is that the builder blows up and the manager detects that and fails it
<abentley> bigjools: that sounds appropriate.
<bigjools> abentley: we need to address the root cause though, is it bzr that eats memory when checking out those big branches?
<abentley> bigjools: Yes.  Though another cause is we're using a version of bzr that is 2x worse in that regard.
<bigjools> ugh
<deryck> rockstar, hey, dude.  If you can spare a moment, can you review  https://code.edge.launchpad.net/~deryck/lazr-js/fix-build-assuming-branch/+merge/42130 ?
<rockstar> deryck, did you see my email about this?
<deryck> rockstar, yeah, just replied.  That was my plan, too.
<rockstar> deryck, okay.
<bigjools> gary_poster: I am upgrading mine now, I'll let you know
<gary_poster> thanks bigjools
<bigjools> gary_poster: seems fine here
<gary_poster> great, thank you bigjools
<bigjools> my pleasure
<bigjools> gary_poster: can I tap you for some help now please? :)
<gary_poster> bigjools: sure :-)
<bigjools> gary_poster: http://librarian.dogfood.launchpad.net/57530918/1z7LhiYuLtwxWSJXAxHNvbyksXZ.txt
<bigjools> that's an error from the apache log parser
<bigjools> however i've got no idea what's causing it as the trace is not particularly useful :/
<gary_poster> ah, errors during the commit phase are always fun
<bigjools> I've tried setting the storm cache size to zero but no difference
<bigjools> so if you can offer any help here that'd be great :)
<gary_poster> so, bigjools, do I assume correctly that this is some error received under more-or-less unknown circustances, and we don't know how to dupe?
<bigjools> gary_poster: I can dupe easily
<bigjools> it's processing the production logs that I copied to dogfood
<gary_poster> ah ok
<gary_poster> bigjools: I'd do the fairly obvious as a first step and try to see what the args to execute are.  If I had to guess, you are trying to store a number that is too big for the DB columb
<gary_poster> column
<gary_poster> I'd see what the args are by hacking the egg
<gary_poster> Because I'm evil that way
<bigjools> yeah, that much is obvious I think.  I suspect it's the download counts themselves
 * bigjools breaks an egg
<bigjools> I presume that DataError is from psycopg
 * gary_poster wonders if the re-raise-an-exception-with-the-original-exception in Py 2.7 (IIRC) would make it easier to have reasonable behavior in situations like this, so that you can actually have a traceback that tells you what is going on
<gary_poster> yeah, think so.  there is a DataError in that package
<bigjools> useful tracebacks?  sounds...useful :)
<james_w> does LP_DEBUG_SQL_EXTRA or whatever not work here?
<bigjools> james_w: it depends on whether the error is from psycopg or the pg server
<bigjools> I'm not sure where it logs, but I might try it if I am desperate, the log will be swamped.
<bigjools> gary_poster: so, this is what is failing: args = ('UPDATE ParsedApacheLog SET date_last_parsed=%s, bytes_read=%s WHERE ParsedApacheLog.id = %s', ('2010-11-29 17:13:54.534869+00:00', 2623267714L, 802))
<gary_poster> bigjools: looks big. ;-) have you already looked up the ParchedApacheLog table to see what the constraints are?
<bigjools> looks like a signed integer overflow
<gary_poster> heh, ParsedApacheLog
<gary_poster> ParchedApacheLog needs water
<bigjools> :)
<bigjools> that could be a bigint (?)
<bigjools> it's a big freaking log file
 * gary_poster will try to find table, but has little Postgres fu atm
<bigjools> gary_poster: http://www.postgresql.org/docs/7.4/interactive/datatype.html#DATATYPE-INT
<gary_poster> bigjools: right, and this appears to be an integer.  bigint seems to be the way to go if this is not a "rethink the problem" sign
<gary_poster> bigjools: is there a reason to not make it a bigint?
<bigjools> gary_poster: I can't think of one
<gary_poster> then sounds like a win :-)
<bigjools> the file it's falling over on is a gzip which is 189M itself
<gary_poster> heh
<gary_poster> so is it worth doing step-back sorts of questions?
<gary_poster> otherwise, +1 on going with a bigint change
<bigjools> well it's mainly a question of how big the logs get
<bigjools> and if this is a one-off, we can split the log up
<gary_poster> well, presumably we'd want to do that in an automated way
<gary_poster> why do we care about the precise number of bytes?
<bigjools> no idea!
<gary_poster> heh
<gary_poster> ah, it's to determine if we've read the whole thing or not, it looks like
<bigjools> yeah, just saw that too
<bigjools> hmmm, I think bigint is safer anyway
<bigjools> I think it will Just Work, no Python changes?
<gary_poster> bigjools: looks that way to me, yes
<bigjools> okidoki, I'll get it sorted
<gary_poster> cool
<bigjools> thanks gary_poster
<gary_poster> np, thank you :-)
<bigjools> dogfood FTW
<bigjools> oh man, pushing db-devel branches is ball-achingly slow now
<lifeless> bigjools: setup the script
<bigjools> lifeless: que?
<lifeless> I sent a python script to fix it to the dev list
<lifeless> days ago :)
<bigjools> seems like I missed that
<lifeless> in the thread where thumper announce the lp:launchpad change
<lifeless> needs someone to do a rt and shepard it through
<bigjools> lifeless: I didn't understand who or where it has to run
<bigjools> and if it needs something like this running, why don't we just go back to how it was
<lifeless> because the way it was causes lots of grief we can't fix.
<lifeless> the performance, we can fix.
<bigjools> I didn't see much evidence of grief, just a couple of pebkac moments
<bigjools> anyway ...
<lifeless> alternatively
<lifeless> we could stack db-stable on stable
<lifeless> thumper: ^
 * lifeless shrugs
 * lifeless does the on leave dance
<bigjools> lifeless: step away from the laptop
 * lifeless watches the mentalist and hacks on testtools
<lifeless> I love the roomba, its really quite effective
<jelmer> 'morning lifeless
<lifeless> hi
<bigjools> lifeless: I wanted one of those and someone warned me off it!
<lifeless> bigjools: they were jealous :)
<bigjools> lifeless: no, they already had one!
<jelmer> lifeless: which generation do you have?
<jelmer> mine has a fondness for cables, I can't leave it alone
<lifeless> jelmer: 4 weeks old
<bigjools> heh
<bigjools> lifeless: timeouts in b-m are no more \o/
<lifeless> it will wind them up, but loose cables are fugly anyway, so just routing them along the wall and putting U restraints in is sufficient
<lifeless> bigjools: congrats
<bigjools> that was a *nasty* bug
<bigjools> and I am going to celebrate with a beer now, good ngiht
<lifeless> cody reckons theres more to it
<lifeless> I reckon fixed is grand
<bigjools> well if he wants to contribute something concrete then I'll listen ;)
<elmo> bigjools: what was the problem?
<bigjools> elmo: the problem is still there, I just worked around it
<elmo> oh, I see
<bigjools> it sends a dummy connection before the real work starts
<bigjools> then waits for it to either work or time out
<cody-somerville> I believe the hypothesis was that Xen guests drop the first packet or something?
<bigjools> I still see the timeouts in the log but we don't chuck the toys out the pram now
<bigjools> cody-somerville: that's the behaviour I was seeing, but only a) randomly, b) when we connect *immediately* after the guest is reset
<bigjools> there's some interesting stuff on google about xen and dropped packets
<bigjools> anyway, I have to run
<bigjools> g'night all
<lifeless> jml: hi
<cr3> is there a preferred xml parser in launchpad? minidom, expat, cElementTree, etc.?
<thumper> abentley: bzr+ssh://bazaar.launchpad.net/~thumper/launchpad/recipe-binary-builds
<thumper> TestSourcePackageRecipeView
<lifeless> cr3: anything-but-xml ?
<cr3> lifeless: heh, I'd normally agree with you, but I'm helping with another project which uses junit xml files and I usually try to inspire myself from the tools used by launchpad :)
<lifeless> cr3: outputting or reading?
<cr3> lifeless: just reading
<cr3> lifeless: come to think of it, you might already have something in subunit
 * cr3 looks
<lifeless> mmm, I don't htink there is a junit library for python yet - all the code I've seen is like junitxml - output only.
<lifeless> however
<lifeless> if you are writing parsing support, let me encourage you to submit patches to python-junitxml, I'd be happy to have a parser in there.
<cr3> lifeless: ugh, I really don't like the print of xml strings in junitxml :(
<lifeless> if the project is python based they may be using subunit anyway and only doing xml casting for hudson
<lifeless> cr3: anyhow to answer the qyestion
<lifeless> using anything baked into python is going to be tolerable
<lifeless> though I'm fairly sure that modelling junit in launchpad would be a mistake ;)
<lifeless> jml: https://code.launchpad.net/~lifeless/testtools/listtests/+merge/42166 :) - coming after that, load-list.
<cr3> lifeless: different project with the server team
<cr3> lifeless: but, for the sake of future considerations, I agree with you. if there's anything junit related, it would be client side
<lifeless> cr3: Seriously though, if you do write python modelling code for junit, please supply patches to python-junitxml, I'd be very happy to have a reader exist there.
<cr3> lifeless: sure thing
<thumper> WARNING:root:Memcache set failed for pt:testrunner:lp/registry/templates.....  WTF??
<thumper> seeing lots of these
<lifeless> memcache isn't running ?
<thumper> I'm running tests
<thumper> why isn't it running?
<lifeless> dunno, its just a thought
<thumper> lifeless: I had that thought too, but not sure why it isn't running
 * thumper sighs
<thumper> lifeless: the output of --list-tests can't be fed back into --load-list
<lifeless> thumper: doctests?
<thumper> lifeless: yes, and others
<lifeless> https://bugs.launchpad.net/launchpad-foundations/+bug/682772
<_mup_> Bug #682772: doctest construction generates duplicate test ids <Launchpad Foundations:New> <https://launchpad.net/bugs/682772>
<thumper>   test_unknown_baseurl (lp.bugs.tests.test_bugwatch.SFExtractBugTrackerAndBugTest)
<thumper> is another
<lifeless> thumper: what else?
<thumper> lifeless: abentley helped me debug my test issue a little, and it is a test isolation problem
<lifeless> that should work, if thats the test id; its an unusual id but no reason it wouldn't work.
<thumper> so I was trying to bisect
<thumper> lifeless: it isn't working
<lifeless> thumper: why doesn't it work ? [also, file a bug :)]
<thumper> lifeless: I don't know, but it just isn't running them
<thumper> lifeless: file a bug on what?
<lifeless> launchpad-foundations
<lifeless> whomever analyses further (not me, I *am* on leave), can move it to some other project if needed - but its probably something in LP.
 * thumper sighs agani
<Guest66935> lifeless: in bug 638924, you mention that bugs and milestones have existing off-by-default eager loading. I don't know if I'm missing something, but the only eager loading I see for bugs is the indexed_messages attribute, and I don't see any eager loading for milestones.
<_mup_> Bug #638924: Milestone:+index timeouts with many bugs <pg83> <timeout> <Launchpad Registry:In Progress by edwin-grubbs> <https://launchpad.net/bugs/638924>
<EdwinGrubbs> hmm, I wonder why I was Guest66935.
<jelmer> EdwinGrubbs: freenode will rename you when you fail to identify in time
<jelmer> bac: Hi!
<bac> hi jelmer
<jelmer> bac: Thanks for handling CHR today. I noticed you assigned the recipe build issues to Soyuz, they should actually be in launchpad-code.
<gary_poster> bac, you back in the states?
<bac> jelmer: oh, sorry
<bac> gary_poster: yep
<gary_poster> cool, bac.  hope it was a good trip
<bac> gary_poster: yeah, it was.  glad to be home.
<jelmer> bac: No problem, just thought I'd mention it if there are any more.
<gary_poster> Great.
<bac> jelmer: no more.  thanks for the heads up.
<mars> thumper, ping
<thumper> mars: hi
<mars> Hi thumper
<mars> thumper, I was just looking at r11995 of devel, which just got merged into db-devel, and started dying there
<thumper> mars: yes...
<thumper> mars: my question would be "what else changed around that time?"
<mars> thumper, do you see any way that your change may be related?  The suite has failed in a similar place twice in a row
<thumper> because that revision touches exactly zero to do with the librarian
<thumper> mars: similar, but not the same
<mars> yep
<mars> thumper, thank you for the info
<thumper> np
<mars> 'what else changed' is a good question.  But according to buildbot - nothing else changed? :(
<lifeless> EdwinGrubbs: hi
<lifeless> EdwinGrubbs: searchTasks has eager loading support
<lifeless> EdwinGrubbs: and theres an eager load flag I added when improving blueprints performance a couple of months back
<gary_poster> deryck: is there a generic "our windmill tests suck" bug?  If there is, I figure you'd know :-) (I would add https://bugs.launchpad.net/launchpad-foundations/+bug/681817 to the list if so?)
<_mup_> Bug #681817: lp.registry.windmill.tests.test_team_index.TestTeamIndex.test_addmember fails intermittently <Launchpad Foundations:New> <https://launchpad.net/bugs/681817>
<EdwinGrubbs> lifeless: is it prejoins parameter for searchTasks()? That would enable to pull in the BugTask.assignee, but it would be less straightforward to pull out the info from the ValidPersonCache.
<lifeless> vpc isn't needed
<lifeless> what you need is Person._validity_clauses
<lifeless> yes, prejoins is it, I think.
<lifeless> for bugtasks
<lifeless> see Person._validity_queries
<lifeless> for a reused code fragment that will handle validity checks
<lifeless> that is in fact used from milestones in milestone prejoining
<lifeless>  - grep for it
<lifeless> read a bit, theres a lot of unrefactored code, you'll need to page it into your head to see how the bits hang together (sadly)
<lifeless> its generally taken me a couple of hours to find a good place to push on, when making these changes.
<deryck> gary_poster, no, there isn't a single bug for the issue.  Just individual bugs on trouble tests.
<deryck> gary_poster, we should probably use a tag
<lifeless> but it gets better each time
<gary_poster> ack thanks deryck .  ok, will start with "windmill" ?
 * lifeless trolls: windfail
<deryck> heh
<lifeless> or perhaps failmill
<gary_poster> :-P :-)
<deryck> windflail
<deryck> gary_poster, yeah, just windmill for now works
<gary_poster> ok
<deryck> ok, out for my evening.  Until tomorrow all.
<wgrant> Grarrrrr.
<mwhudson> morning william
<lifeless> henninge: hai
<lifeless> bah
<lifeless> wgrant: hai
<henninge> ;-)
<henninge> hi wgrant ;)
<wgrant> Bug #682738
<_mup_> Bug #682738: Split the PPA publisher into public/private processes <ppa> <soyuz-publisher> <Soyuz:Triaged> <https://launchpad.net/bugs/682738>
<wgrant> Grarrring at the response to that.
<lifeless> mine or bigjools?
<wgrant> bigjools.
<lifeless> well, say so :)
<wgrant> Already did.
<lifeless> in the bug ?
<wgrant> Yeah.
<LPCIBot> Project devel build (259): FAILURE in 3 hr 46 min: https://hudson.wedontsleep.org/job/devel/259/
<LPCIBot> * Launchpad Patch Queue Manager: [r=henninge][ui=none][no-qa] Convert some webservice tests for
<LPCIBot> blueprints into model tests.
<LPCIBot> * Launchpad Patch Queue Manager: [r=mwhudson][ui=none][bug=146389] Start exposing ISpecification and
<LPCIBot> IHasSpecifications attributes on the webservice API.
<wgrant> Hah, ENOSPC
* flacoste changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 2 of 10.12 | Performance Tuesday | PQM is open | firefighting: db-devel broken | https:/â/âdev.launchpad.net/ | Get the code: https:/â/âdev.launchpad.net/âGetting
<flacoste> wallyworld, thumper: are you guys doing anything special for performance Tue?
<thumper> flacoste: no, I'm attempting to find out why I have test isolation failures
<flacoste> yeah, that seems sensible
<wallyworld> flacoste: not today, i've got a branch to finish and another to start after that
<wallyworld> wgrant: seems a build upload is stuck, any ideas? https://launchpad.net/~nova-core/+archive/trunk/+build/2069633
<wallyworld> wgrant: it says it finished building over an hour ago
<wgrant> wallyworld: Ugggh.
<wgrant> Binary builds aren't meant to get stuck :(
<wgrant> Can you see if others have been uploading?
<wallyworld> not sure how to do that :-)
<wallyworld> wgrant: can you tell me where to look?
<wgrant> wallyworld: Can you look for process-upload.py logs from cesium?
<wallyworld> wgrant: will do
<wgrant> Argh, the daily builds have just hit.
<wgrant> but they seem to be running OK.
<wgrant> THere was a binary build uploaded <30 minutes ago.
<wgrant> So it's not a general problem.
<wgrant> Any errors in the p-u log?
<wallyworld> wgrant: looking....
<spm> wgrant (who has not yet started with us...): ref https://bugs.launchpad.net/soyuz/+bug/663562 this appears to have bit again on linux 2.6.24-28.81 in hardy. anything I can poke that may give more useful info?
<_mup_> Bug #663562: duplicate orig for "linux" package in hardy <soyuz-core> <soyuz-security> <Soyuz:Incomplete> <https://launchpad.net/bugs/663562>
<wgrant> spm: Grar. We'll have to recowboy.
<wgrant> I was hoping a real Soyuz person would fix the data before we had to do this again.
<wgrant> spm: Do you need a new patch?
<spm> there are no real soyuz persons. they're unreal.
<spm> wgrant: I suspect so
<wgrant> spm: http://pastebin.ubuntu.com/516480/ is what I gave mbarnett last time, and it still works fine.
<wgrant> Cowboy it on cocoplum.
<wgrant> And get jdstrand to use unembargo-package.py
<spm> hookay, ta
<wgrant> Myes.
<spm> is that something we apply, get the unembargo'd, then revert? or is safe to leave? or?
<wgrant> And then uncowboy it. Preferably in quick succession.
<spm> great minds.
<wgrant> Not safe to leave, no.
<spm> it didn't look it, just looking at the removals
<wallyworld> wgrant: logs appear to indicate that the ulpoad of the nova ppa went ok. there's some process messages and then "Finished Checking Upload"
<wgrant> wallyworld: Yay... might need someone with both log access and uploader knowledge to look at it tonight.
<wgrant> spm: Huh?
<wgrant> Oh, I see.
<spm> yeah - rm'ing those checks was setting off alarm bells for me
<wallyworld> wgrant: can i check anything else? this is in response to a question on #launchpad so i'd like to be able to tell them something if possible
<spm> and that's *before* coffee too
<wgrant> As long as no AAs are doing any copying at the moment (it's rare), it should be fine.
<wgrant> wallyworld: Not entirely sure. If the uploader log looks fine, there's something pretty broken.
<wgrant> Because there are no binaries.
<wallyworld> wgrant: want me to pastebin the log info just so you can eyeball it?
<spm> wgrant-aaaaaa.patch <== it seems like a suitable name to give this patch
<wgrant> wallyworld: Probably a good idea.
<wgrant> spm: Yep.
<spm> dry-run applies cleanly. I'll do the juju when jdstrand is around again. thanks muchly wgrant!
<mbarnett> hah, i was totally PMing spm this in a far less intelligible way.   Perhaps i should pay attention to what's going on around me...
<spm> NEVER
<mbarnett> okay, good.
 * mbarnett goes back to staring blankly at the floor.
<spm> I'd suggest you're a crazy texan, but you might pull a gun on me
<wallyworld> wgrant: https://pastebin.canonical.com/40234/
<wgrant> wallyworld: That's private.
<wallyworld> wgrant: what's the public url?
<wallyworld> wgrant: nevermind, found fit
<wgrant> paste.ubuntu.com
<wallyworld> wgrant: http://paste.ubuntu.com/538151/
<wgrant> wallyworld: Those are different builds (same package, different series). Anything mentioning 2069633?
 * wallyworld looks
#launchpad-dev 2010-11-30
<wallyworld> wgrant: there's some stuff from 23rd Nov and also 29th: http://paste.ubuntu.com/538152/
<wgrant> wallyworld: Any references in the p-u log to the directory mentioned in that b-m line?
<wgrant> I wonder if this is the race where p-u will have moved it to failed.
<wgrant> Because it ran before the status was set.
<lifeless> does anyone else get a 404 on this - http://ppa.launchpad.net/testing-cabal/ppa/ubuntu/dists/maverick/main/source/Sources.gz ?
<wallyworld> wgrant: you mean this directory? 20101129-213106-2078946-PACKAGEBUILD-2081126
<wgrant> wallyworld: That one.
<wgrant> lifeless: Yes. Should it be there?
<lifeless> I assume so
<lifeless> there are packages...
<lifeless> http://ppa.launchpad.net/testing-cabal/ppa/ubuntu/dists/maverick/main/binary-amd64/Packages.gz
<lifeless> is 404ing for me too
<lifeless> https://code.launchpad.net/~testing-cabal/+archive/archive
<lifeless> reckons theres a bunch of stuf in the ppa
<wgrant> Ugh, indeed.
<lifeless> I'd like to use this stuff
<wgrant> lifeless: er, the PPA is named 'archive', not 'ppa'.
<wgrant> Your URL is wrong.
<lifeless> oh right
<lifeless> apt-add-repository fail
<wgrant> Hm, it let you do that?
<lifeless> yes
<wallyworld> wgrant: don't appear to be any references to that exact directory. there are references to 213106 from earlier dates eg 20101115
<wgrant> wallyworld: That's just a timestamp.
<wallyworld> wgrant: yeah but i was thinking earlier uploads may have worked
<wallyworld> wgrant: so you think it may be a race condition? can we kick off a manual upload?
<wgrant> spm: Do you have a moment?
<spm> wgrant: sure
<wgrant> spm: Can you see if there's a 20101129-213106-2078946-PACKAGEBUILD-2081126 directory in cesium's buildd-manager upload root somewhere?
<wgrant> /srv/launchpad.net/builddmaster
<spm>  ./failed/20101129-213106-2078946-PACKAGEBUILD-2081126
 * jelmer is here but was about to get some sleep
<wgrant> jelmer: You've been working on this, right?
<wgrant> I wonder if we should just throw it back into incoming.
<jelmer> wgrant: I haven't read the context - what's going on exactly?
<wgrant> jelmer: Binary upload stuck UPLOADING, with its directory in 'failed'.
<wgrant> jelmer: No logs.
<jelmer> wgrant: Yeah, moving it back to failed should be ok in that case. I did a fix recently to not move things to failed as quickly but keep them in incoming until we've committed the change to the build status.
<jelmer> wgrant: That hasn't been deployed yet though.
<wgrant> spm: Could you move that dir from failed back to incoming?
<spm> sure, one sec (was encoffeenating)
<spm> done
<jelmer> spm: Thanks
<wgrant> spm: Thanks.
<wgrant> And now we have an upload log.
<spm> np, and heyo jelmer
<wgrant> Hah, it's been superseded already.
<wallyworld> wgrant: spm: thanks for helping with this. i'll let the user know the upload is retrying
<lifeless> wgrant: at what point should a ppa package become installable on client machines
<wgrant> lifeless: After the binaries are built and published.
<lifeless> grrr
<lifeless> why isn't apt seeing it
<lifeless> ahh
<lifeless> http://ppa.launchpad.net/testing-cabal/archive/ubuntu/pool/main/p/python-testtools/
<lifeless> i386 for all.
<lifeless> https://code.launchpad.net/~testing-cabal/+archive/archive/+packages
<lifeless> wgrant: expand ython-testtools - 0.9.7+bzr146~ppa116~maverick1
<lifeless> Publishing details
<lifeless>     * Published 15 minutes ago
<wgrant> lifeless: That be the source.
<wgrant> Note how it has a green flashing gear.
<lifeless> no
<wgrant> And it says the binaries aren't pblished.
<lifeless> is that what it is?
<wgrant> lifeless: Which series?
<lifeless> wgrant: colour means approximately nothing to me in UIs
<wgrant> Natty is still building, and Maverick and Lucid should have a warning about unpublished binaries when you expand their row.
<lifeless> wgrant: i see a dark octagon, which might be greed.
<wallyworld> thumper: ping
<lifeless> *might be green*
<thumper> wallyworld: aye
<lifeless> someone needs to do user testing on this
<wgrant> lifeless: Right, there's that, but you'll get more details if you expand the row.
<lifeless> wgrant: i did
<wgrant> lifeless: Launchpad does not do UI.
<lifeless> wgrant: Launchpad does.
<wallyworld> thumper: unknown dailydeb implies out of date bzrlib?
<lifeless> some things have more room to improve than others.
<lifeless> wgrant: is there a 'this needs work' bug ?
<thumper> wallyworld: eh?
<wgrant> lifeless: I don't think so.
<wallyworld> thumper: https://pastebin.canonical.com/40235/ see the error at the end. i seem to recall dailydeb is a recent addition to bzrlib?
<thumper> Unable to load plugin 'builder' from '/usr/lib/python2.6/dist-packages/bzrlib/plugins'
<thumper> line 434
 * wallyworld slaps forehead
<thumper> which hints at a bigger problem
<wallyworld> thumper: the log came from https://launchpadlibrarian.net/59824676/buildlog.txt.gz
<wallyworld> thumper: does that mean we have something to look at on our end?
<thumper> certainly looks like it
<thumper> I would have thought that this would have come up in QA testing
<wallyworld> thumper: yeah. so is it a matter of updating some servers to get the necessary plugin installed?
<thumper> wallyworld: I'm not entirely sure
<thumper> wallyworld: check with abentley or lamont
<wallyworld> thumper: thanks, will do
<thumper> GRRR!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!1111
 * thumper runs bzr blame
<wallyworld> thumper: ?
<thumper> wallyworld: I think I've found "my" bug
<wallyworld> thumper: you mean the test isolation one
<wallyworld> ?
<thumper> yep
<thumper> a monkeypatch
<thumper> Hierarchy.makeBreadcrumbForRequestedPage = lambda self: None
<wallyworld> what was it put there for?
<thumper>     # Monkey patch Hierarchy.makeBreadcrumbForRequestedPage so that we don't
<thumper>     # have to create fake views and other stuff to test breadcrumbs here. The
<thumper>     # functionality provided by that method is tested in
<thumper>     # webapp/tests/test_breadcrumbs.py.
<thumper> but the method is never put back
<wallyworld> doh!
<thumper> aye
<thumper> lifeless: do you know if we have a way to nicely run something at the end of a doctest?
<thumper> can we use addCleanup?
<lifeless> no
<thumper> :(
<lifeless> you could add an empty fixture to globs
<lifeless> e.g. 'test_fixture' : Fixture()
<lifeless> then in the setUp set it up, and in the teardown do cleanUp
<lifeless> and in the test do test_fixture.addCleanup(...)
<wallyworld> rockstar: ping!
<thumper> that sounds like a good idea...
<thumper> I may well do it
<thumper> I'm running all the tests in the layer again to see if my monkey patch reversal now makes it work
<thumper> I think my test failed due to adding more tests caused some re-ordering
<thumper> which caused the failure to be shown
<thumper> the problem was there before
<wallyworld> thumper: you know the rules: last person to touch it broke it :-P
<thumper> :P
<spm> ugh. does anyone happen to know which table project groups live in? need to rename a new one.
<lifeless> project
<lifeless> projects are products
<lifeless> project groups are projects
<spm> so projects are in product; and project groups are in project? logical :-)
<spm> snap!
<lifeless> echangingourmindisn'teasy
<spm> :-)
<lifeless> spm: so how about that proxypass rule
<spm> slowly atm
<thumper> why do I not have memcached working?
<thumper> anyone got any ideas?
<rockstar> wallyworld, hi
<wallyworld> rockstar: just wondering if you wanted to review the product merge queue branch :-)
<thumper> rockstar: also, you have a qa task on our board
<thumper> rockstar: can you move it / do it?
<rockstar> thumper, sure.
<thumper> ta
<abentley> wallyworld: It looks like bzr-builder is there, but apt_pkg is not.
<wallyworld> abentley: so this is on one of our servers I think? wonder how that could have happened?
<abentley> wallyworld: This would be on one of our builders.
<abentley> wallyworld: Please post a link to the actual build-- the build log doesn't provide navigation to it.
<wallyworld> abentley: i'll have to get it. i got the log from someone on #launchpad. i'll see if i can ping him
<abentley> wallyworld: I believe it is this one: https://code.launchpad.net/~audacity-team/+recipe/daily.karmic/+build/9502
<wallyworld> abentley: right you are
<wallyworld> abentley: so, who do i talk to to get the issue looked at?
<abentley> wallyworld: james_w could tell you what's apt_pkg, because it doesn't look like it's bzr-builder directly.
<thumper> \o/
<abentley> wallyworld: lamont can install updated packages on the builders.
<thumper> finally landing the binary build change
<thumper> I emailed the list with my problem resolution
<wallyworld> abentley: thanks. so it seems we just need to install and/or update apt_pkg on that server?
<wallyworld> thumper: most importantly, did you figure out who to blame? :-)
<abentley> wallyworld: No, there's detective work needed.  What is apt_pkg used by?  Why didn't apt install it when it installed whatever requires it?
<thumper> yes
<thumper> but I won't publicly shame
<abentley> wallyworld: Possibly we need to fix the dependencies on some packages.
<wallyworld> thumper: didn't expect you to, just having the satisfaction of knowing it wasn't you is enough :-)
<thumper> :)
<thumper> yes, it was not me
<wallyworld> abentley: thanks. will pass on the details. does it seem like it may only be an issue with one server? should I tell the user try again?
<abentley> wallyworld: it seems likely this is a single-server issue.  We also had another bug with the wrong version of something being installed: https://bugs.launchpad.net/bugs/682941
<_mup_> Bug #682941: no such option --append-version in recipe build <Launchpad Bazaar Integration:New> <https://launchpad.net/bugs/682941>
<wallyworld> abentley: cool. many thanks for your help.
<abentley> wallyworld: It appears that a few servers were recently added to the build farm.  That may be at the root of this: https://lpstats.canonical.com/graphs/BuildersActiveVirtual/
<thumper> abentley: I'm smiling at the daily build spikes :)
<wallyworld> abentley: could be
<abentley> thumper: They're nice, but we may want to look into staggering them to make the build farm's responsiveness more consistent.
<thumper> abentley: yeah.  I've heard bigjools request as much
 * thumper goes to collect kids
<spm> wgrant: fyi. patched, zapped, unpatched. looks good. thanks!
<abentley> wallyworld: in the meantime, see if any new reports were built on sandpaperfig or lansones, because maybe only those two are problematic.
<wallyworld> abentley: ok
<lifeless> === Top 10 Time Out Counts by Page ID ===
<lifeless>     Hard / Soft  Page ID
<lifeless>      287 / 6525  Archive:+index
<lifeless>      170 /  390  BugTask:+index
<lifeless>       43 /  260  Distribution:+bugs
<lifeless>       27 /  140  ProjectGroupSet:CollectionResource:#project_groups
<lifeless>       18 /    1  Archive:EntryResource:syncSource
<lifeless>       14 /   35  DistroSeries:+queue
<lifeless>       13 /  259  Distribution:+bugtarget-portlet-bugfilters-stats
<lifeless>       12 /   38  Milestone:+index
<lifeless>       12 /   28  MailingListApplication:MailingListAPIView
<lifeless>       12 /    4  Person:+bugs
<wgrant> spm: Yay, thanks.
<lifeless> wgrant: I can has Archive fix.
<lifeless> wgrant: iz getting worse.
<thumper> \o/ db-devel builder passed third time lucky
<poolie> wow: http://odata.stackexchange.com/ubuntu/s/665/too-many-comments
<poolie> > Stack Exchange Data Explorer allows you to run arbitrary SQL queries on the Stack Exchange family of sites, share those queries with your friends and explore interesting queries.
<wgrant> I imagine it's not really SQL.
<wgrant> Well, at least not direct.
<poolie> i think it is
<poolie> but it's against a warehouse db, not the real one
<wgrant> Ah.
<lifeless> may still be filtered
<poolie> i think they strip private stuff on the way to the warehouse
<poolie> they may have much less private stuff, and a simpler delineation of it, than lp would have even now
<wgrant> Yeah, true.
<beuno> thumper, these new incremental diffs in merge proposals are the best thing since slide bread
<thumper> beuno: I'm glad you like them, abentley is fixing a bug in them :)
<beuno> I lovez em!
<wgrant> They are really great.
<wgrant> Now all we need is comments on lines, somehow.
<beuno> yes we do
<abentley> beuno, thumper: they're disabled until we can get that bug sorted, which unfortunately is a bzr bug, but I've submitted a fix.
<beuno> abentley, that's fine, I already got a taste of them, will keep me warm and fuzzy for a good week
<spiv> abentley, thumper: yes they're nice, thanks for adding them!
<abentley> spiv: you're welcome.
<wgrant> Are they restricted to betatesters?
<spiv> abentley, thumper: I was wondering if it might be nice to be able to have a javascript thing to collapse them all for times when you want to just skim the conversation
<wallyworld> thumper: why does self.assertEqual(x, y) fail unless i removeSecurityProxy(x) and removeSecurityProxy(y)? i'm logged in as the owner of x and y
<wgrant> +1
<spiv> e.g. if you are a patch pilot and want to quickly get a sense of where the discussion is at
<lifeless> wallyworld: probably a buggy __eq__
<abentley> spiv: Could be.
<lifeless> wallyworld: sp(x).__eq__(sp(y)) -> x.__eq__(sp(y))
<abentley> lifeless: Or __eq__ falling back to is.
<lifeless> abentley: sure, which in the absense of sp's is reasonable. In our context its arguably buggy though
<wallyworld> in this case i have two sets of branches, and i have to iterate over the contents of each set and remove the security proxies
<wallyworld> for now, i would like to do the workaround and file a bug, does that sound ok?
<wgrant> StevenK:
<wgrant> StevenK: Morning.
<spm> publicrestrictedlibrarian	default	0	on <== re-enabled
<lifeless> wgrant: give it a spin ?
<wgrant> lifeless: It's unbroken! Well done.
<lifeless> spm: looks good to me too
<spm> \o/
<wgrant> lifeless: Is it intentional that bzr won't stack if you terminate the initial push?
<lifeless> wgrant: there is a bug
<wgrant> Ugh. Why do project bug pages now advertise Ubuntu?
<lifeless> the downstream link?
<wgrant> Yes.
<lifeless> they should show any distro packages, not just ubuntu
<wgrant> The last thing Launchpad needs is more Ubuntu-specific stuff.
<lifeless> I suspect the links are only in place for Ubuntu.
<wgrant> Big upstreams *do not want that sort of thing*.
<lifeless> mmmm
<lifeless> wgrant: I think the placement needs tuning
<lifeless> wgrant: but I don't buy your assertion - I think its overstated.
<lifeless> wgrant: many upstreams *want* easier querying of bugs in their stuff in Ubuntu.
<wgrant> lifeless: Hm? I thought it was an undeniable fact that Ubuntu-specificity was hurting Launchpad adoption.
<lifeless> wgrant: including bug ones.
<lifeless> s/bug/big/
<wgrant> Sure.
<lifeless> poolie: http://rbtcollins.wordpress.com/2010/11/30/testrepository-iteration-for-python-projects/ fyi
<wgrant> But it needs to not just scream about Ubuntu on every page.
<lifeless> wgrant: file bugs on making it more subtle - point out that it needs to scale to baltix, linaro etc.
<wgrant> Ah yes, I keep forgetting that Linaro might force people to be less insane.
<lifeless> wgrant: Ubuntu specificity is a mixed blessing
<lifeless> wgrant: its totally unclear whether its a net win or negative
<wgrant> lifeless: It will get you lots of nice little projects.
<wgrant> But scare away any big ones.
<lifeless> wgrant: !cite
<lifeless> wgrant: seriously, hyperbole doesn't help analyse this.
<lifeless> wgrant: Ubuntu specific stuff can be harmful in at least two distinct ways
<lifeless> there may be defaults or hardcoded behaviour that make less sense for upstreams
<lifeless> which is a special case of 'less than the best of breed $tool than LP could be'
<lifeless> wgrant: secondly theres stuff that presents Ubuntu where it isn't welcome.
<lifeless> wgrant: its my opinion that the former has been a frequent problem in the past
<lifeless> wgrant: but I don't think this falls into that category.
<wgrant> lifeless: There have been both progressions and regressions around the former over the last few months, but it's probably not too bad now apart from the portlet on Product:+index.
<wgrant> And maybe this bug listing thing, but that may just be an artifact of the portlet.
<lifeless> on the positive side
<lifeless> Ubuntu is a very demanding and large customer
<wgrant> Indeed.
<lifeless> many things that Ubuntu experiences problems with other large users experience problems with.
<wallyworld> lifeless: what do i do about bug 615788? it's been fixed but the user has reopened it against launchpad because they don't agree with the resolution. should i change to Invalid and ask them to file a new bug against ubuntu-website?
<_mup_> Bug #615788: gpg should use port 443 by default in order to work from behind firewalls <Launchpad itself:Opinion> <synaptic:New> <Ubuntu Website:Fix Released> <https://launchpad.net/bugs/615788>
<wgrant> But the Ubuntu-specificity reduces Launchpad adoption, which reduces bzr adoption, which reduces Launchpad adoption, which...
<lifeless> wallyworld: file an rt ticket asking for a gpg server on port 80
<lifeless> wgrant: Ubuntu may sometimes ask for things to solve their issues that don't work well for other users, but its *our job* to find a way to solve their problem creatively, in a way other folk also benefit from.
<wallyworld> lifeless: ok. but we don't want the bug to stay assigned to launchpad do we?
<lifeless> wallyworld: why not?
<rockstar> lifeless, because it's not launchpad's issue?
<rockstar> "(I don't have permissions to reopen the original bug for ubuntu-website, so I reopened for launchpad instead - sorry for the hassle)"
<LPCIBot> Project devel build (260): STILL FAILING in 4 hr 5 min: https://hudson.wedontsleep.org/job/devel/260/
<wallyworld> lifeless: the bug when first filed was tiraged to ubuntu-website so i assumed that's where we wanted it to go
 * lifeless shrugs
<lifeless> I'm on leave.
<lifeless> I'm telling you what I'd do.
<lifeless> but I really don't want to go deep on the discussion.
<wallyworld> lifeless: ok. i wasn't going to bother you but you keep popping up :-)
<rockstar> wallyworld, I'd close it as "Won't Fix" for launchpad.
<wgrant> wallyworld: It doesn't need an RT.
<wgrant> It's Invalid in Launchpad.
<lifeless> and reading the bug, its already fixed.
<lifeless> he wants a bug open on gpg upstream.
<lifeless> that will need some clear communication.
<wgrant> It needs a fix in python-software-properties, and possibly Soyuz.
<rockstar> Er, yeah, Invalid.  What wgrant said.
<wgrant> And maybe gnupg.
<lifeless> gnupg could certainly try 80 if the default fails epically.
<lifeless> wgrant: back to reputation
<lifeless> wgrant: Ubuntu may ask for crack, and we may do it occasionally, in response to its [large] scaling problems.
<wgrant> lifeless: Sure.
<wallyworld> rockstar: so should the user file his own bug upstream?
<lifeless> wgrant: its up to us to help them solve their issues in such a way that our other users are helped too.
<lifeless> wgrant: the pressure to solve these issues is a positive pressure
<rockstar> wallyworld, well, he certainly shouldn't just re-open the bug on another project just because he doesn't like the fix.
<wgrant> rockstar: Well, what he did was more correct than what he initially tried to do.
<lifeless> wgrant: and secondly many upstreams care passionately about the quality of their *delivered to users* software.
<wgrant> It needs an LP config change and possibly an Ubuntu gnupg or python-software-properties change.
<wgrant> The sysadmin side is done.
<rockstar> wgrant, I guess I'm confused on how this issue relates to Launchpad at all.
<lifeless> wgrant: tighter reporting and easier analysis of their bugs in Ubuntu is desired and positive
<rockstar> It's a keyserver/client issue.
<lifeless> rockstar: it doesn't, he's confused.
<wgrant> rockstar: Launchpad produces links to the keyserver. It uses port 11371
<lifeless> wgrant: we should change those links.
<wgrant> lifeless: Hence the config change.
<lifeless> I'd file a new bug for that though.
<lifeless> rockstar: so it does apply, though he is confused about how it all hangs together.
<rockstar> wallyworld, ^^ Close the bug as invaled, reference a new bug about ports for 11371
 * rockstar goes to bed
<wallyworld> rockstar: thanks
<wgrant> wallyworld: Also Invalidate the synaptic task, while you're there.
<wallyworld> wgrant: what should i ask for the 11371 port number to be changed to in the new bug?
<lifeless> 80
<wallyworld> lifeless: thanks. just wanted to make sure so i didn't enter something stupid in the bug report
<wgrant> gpghandler.port is the relevant config key.
<wallyworld> wgrant: thanks to you too
<lifeless> wgrant: anyhow, think on those points
<lifeless> wgrant: I think its very much undecided whether the net influence is win or lose
<wgrant> Hmmm.
<lifeless> and we have plenty of room to make great decisions going forward to make the most of having a huge customer
<LPCIBot> Project db-devel build (175): FAILURE in 4 hr 16 min: https://hudson.wedontsleep.org/job/db-devel/175/
<wgrant> losa ping
<mthaddon> wgrant: hi
<wgrant> mthaddon: germanium's publisher has issues.
<wgrant> Hasn't published anything for a couple of hours, AFAICT.
<mthaddon> ok, what stage are we at in terms of troubleshooting it?
<wgrant> mthaddon: 0
<wgrant> Well.
<wgrant> I know that process-accepted.py isn't finishing.
<wgrant> I'm not sure if it's starting.
<lifeless> am?
<lifeless> 19:35 < lifeless> wgrant: its up to us to help them solve their issues in such a way that our other users are helped too.
<lifeless> 19:36 < lifeless> wgrant: the pressure to solve these issues is a positive pressure
<lifeless> bah, copy paste faillll
<mthaddon> wgrant: would that be cron.publish-ppa or cron.daily-ppa (I'm assuming the former)?
<wgrant> mthaddon: The former.
<mthaddon> wgrant: it seems to be doing stuff - the logfile is being written to fine with non-error messages (things like: 2010-11-30 08:42:53 INFO    Processing http://ppa.launchpad.net/some/url)
<wgrant> mthaddon: What time does cron.daily-ppa run?
<mthaddon> wgrant: 5:59 UTC
<wgrant> Hah.
<mthaddon> er, 05:59 UTC, that is
<wgrant> Then I bet it takes a little over two hours to run, blocking publication.
<wgrant> Since everything woke up about 10 minutes ago.
<wgrant> As it did last time I poked you about this.
<mthaddon> interesting - they write to the same log file
<wgrant> They share a lock too.
<mthaddon> yeah
<mthaddon> but it does look like it's cron.publish-ppa running now
<wgrant> Yeah, it's all good now.
<mthaddon> k
<wgrant> I just have awful timing.
<wgrant> Again.
<bigjools> hello
<StevenK> s/Again/Still/
<wgrant> Morning bigjools.
<bigjools> I love the smell of burning Soyuz in the morning
<wgrant> bigjools: It's been a pretty boring day, apart from the usual hardy linux cowboy.
<wgrant> And a b-m race.
<StevenK> Oooh, I missed the race
<bigjools> ?
<wgrant> Uploads stuck in UPLOADING, fixed by moving them back from failed to incoming.
<wgrant> But I believe that bug is fixed?
<bigjools> the race is not fixed
<wgrant> :(
<bigjools> *cough* jelmer
<bigjools> wgrant: I can't fathom why cron.daily-ppa takes so long
<wgrant> bigjools: It shouldn't.
<wgrant> The tree isn't that huge.
<bigjools> 2.5 hours ....
<wgrant> I guess we should make DiskPool take care of it.
<bigjools> and why does it need to share a lock
<wgrant> Because otherwise it'll crash the publisher.
<bigjools> how
<wgrant> It could remove a dir just after the publisher creates it.
<wgrant> Boom.
<bigjools> grar
<wgrant> Jar.
<bigjools> this Is Bad
<wgrant> Barelt.
<wgrant> DiskPool can do it easily.
<wgrant> Hmm.
<wgrant> But not easily all the way up.
<wgrant> I guess we could.
<bigjools> I want to know why this is so slow first
<bigjools> and, I am sick to death of PFOE
<wgrant> bigjools: Did you get anywhere with that logparser error? Did my theory about postgres 32-bit integer limits being exceeded hold any water?
<bigjools> wgrant: yes, that is it
<wgrant> Hah.
<wgrant> Does bigint work?
<bigjools> yes
<wgrant> Yay.
<bigjools> did you see my bug about dupe builds?
<wgrant> Vaguely.
<bigjools> there's 2 PPAs like that now
<bigjools> same looking problem
<mrevell> Hello
<bigjools> hey
<allenap> mrevell: Bug imports are quite manual; someone files a question to request it, provides us with some XML, we massage that and check it imports (and go back to the requester if there are problems, rinse, repeat), import it to staging so they can see it, then import into production.
<allenap> mrevell: The whole process can take a few days or a few weeks depending on what issue crop up.
<mrevell> allenap, So, same as before pretty much. How long do they take ... on average? I'm wondering how strongly we should offer this as a service.
<mrevell> allenap, Ah right ... how much dev time?
<mrevell> roughly
<allenap> mrevell: Right now they do suck up the team's time quite a lot. Best talk to deryck before promoting things too heavily,
<allenap> mrevell: On the other hand, if we don't promote it, we won't spend time improving the story.
<mrevell> allenap, We barely mention them at all, right now, precisely because we don't want to overload ourselves. I think you're right, though, probably best to speak Dezza as this is somewhere that we do a really good job and make people happy.
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build (261): FIXED in 3 hr 32 min: https://hudson.wedontsleep.org/job/devel/261/
<LPCIBot> Launchpad Patch Queue Manager: [r=abentley][ui=rockstar][bug=602508] Show related binary builds on
<LPCIBot> the recipe index page.
<bigjools> wgrant: can you check the ppa stats on DF?  the log parser process has been busy this morning
<wgrant> bigjools: When are the logs from?
<bigjools> wgrant: production logs, latest is from Oct 27
<wgrant> bigjools: Ah, I see.
<wgrant> bigjools: re. bug #663562, I recall we discussed this last time and worked it out.
<_mup_> Bug #663562: duplicate orig for "linux" package in hardy <soyuz-core> <soyuz-security> <Soyuz:Incomplete> <https://launchpad.net/bugs/663562>
<wgrant> The upload file checker only looks for published files.
<bigjools> wgrant: shame that wasn't logged on the bug :(  I can't remember
<wgrant> linux was uploaded with one orig.tar.gz, deleted, then uploaded a while later with a different one.
<bigjools> ah
<bigjools> that sucks
<bigjools> that reall sucks
<bigjools> that explains how people are doing it in PPAs
<bigjools> sigh
<wgrant> bigjools: DF seems to be from Oct 7 or so. Do the logs go back that far?
<bigjools> wgrant: yes, it's catching up
<bigjools> it has all the prod logs evar
<wgrant> .. oh.
<wgrant> bigjools: I've checked a few and they are all 0, but I've no idea what date range to look for.
<bigjools> ok
<bigjools> erm
<bigjools> it's catching up, try later
<bigjools> it just did the file for Oct 27 actually
<wgrant> That won't have done anything, since DF doesn't have data that late.
<wgrant> Will have to wait.
<wgrant> bigjools: !!
<wgrant> https://dogfood.launchpad.net/api/devel/~launchpad/+archive/ppa/+binarypub/10649139?ws.op=getDownloadCounts
<wgrant> One download of python-psycopg2 from the launchpad PPA.
<wgrant> Hopefully more will show up eventually..
<bigjools> GTK file open dialog is a total utter shambolic PoS
<wgrant> You're using a GTK application!?
<bigjools> Firefox
<bigjools> about the only one
<wgrant> Ah.
<wgrant> Chromium FTW :)
<bigjools> you'd think that when it wants an application to run for a file download, it would match stuff in the PATH, but no, you need to type the full path out.
<wgrant> Oh yes, that bit is just stupid.
<bigjools> I use Chromium on my laptop.  I am thinking of using it as default everwhere now
<bigjools> but I hate its windown decorations
<bigjools> window, even
<wgrant> I tell it to use the system ones.
<bigjools> then you get two lots .... :/
<wgrant> ... not for me.
<bigjools> I mean - you get none at all, sorry
<bigjools> it doesn't like kwin
<wgrant> https://chrome.google.com/extensions/detail/elnmibmpefhmfgphdphdncoogpbfmlbp
<wgrant> That's what it looks like in sane WMs :P
<bigjools> it's using GTK, no?
<wgrant> Maybe.
<bigjools> the world is heading to QT anyway
<wgrant> Both of them suck.
<bigjools> qt is great
<wgrant> It has its own C++ preprocessor.
<wgrant> That disqualifies it from greatness.
<bigjools> heh
<wgrant> It is otherwise less abhorrent than GTK, though, I agree.
<henninge> jtv: the old updateStatistics method relied a lot on the filtering methods "getPOTMsgset*". Your last branch seems to break that relationship. I assume you are aware of that?
<jtv> henninge: yes
<henninge> jtv: I see the advantage in the old approach because it keeps filters and statistics in sync.
<jtv> yes
<jtv> it was unfortunately also intractably slow
<henninge> but I understand you did that for performance reasons, right?
<henninge> no need to react like that .... ;)
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build (176): FIXED in 3 hr 33 min: https://hudson.wedontsleep.org/job/db-devel/176/
<LPCIBot> * Launchpad Patch Queue Manager: [rs=buildbot-poller] automatic merge from stable. Revisions: 11999
<LPCIBot> included.
<LPCIBot> * Launchpad Patch Queue Manager: [r=stub][ui=none][bug=368600,
<LPCIBot> 678289][rollback=9997] Revert database patch application timing
<LPCIBot> report,
<LPCIBot> which isn't working as expected when patches are all applied in a
<LPCIBot> single transaction.
<LPCIBot> Project devel build (262): FAILURE in 8 min 19 sec: https://hudson.wedontsleep.org/job/devel/262/
<bigjools> StevenK: ^ it's run out of disk space :)
<StevenK> Yes, I saw
<wgrant> bigjools: What are we doing for builds that were failure-counted to death?
<wgrant> They appear at the top of recipe build listings.
<bigjools> wgrant: nothing
<wgrant> wallyworld_: ^^
<wgrant> They will appear there forever.
<bigjools> url?
<wgrant> https://code.edge.launchpad.net/~videolan/+recipe/master-daily
<bigjools> what's the problem?
<wallyworld_> wgrant: i was being asked a question about that just now in #launchpad
<wgrant> wallyworld_: That's why I pinged you.
<wallyworld_> wgrant: just realised :-)
<wgrant> bigjools: That build is from two weeks ago.
<wgrant> bigjools: It was failure-counted out, so has no finish time.
<wgrant> It sorts above all others.
<wgrant> (this affects binary build listings too)
<wgrant> (failing a build without setting a finish time is crack)
<bigjools> wgrant: then that's a UI bug
<wgrant> I don't think so.
<bigjools> not really, it didn't finish
<bigjools> this makes me sad that people don't understand packaging: https://edge.launchpad.net/~stilia-johny/+archive/xwinwrap-gui/+packages?field.name_filter=&field.status_filter=&field.series_filter=
<bigjools> I need to fix that file re-upload bug
<bigjools> did you investigate where in the code the bug is, before I waste time?
<wgrant> bigjools: _getFileByName in lp.archiveuploader.dscfile, which uses Distribution.getFileByName.
<wgrant> It should probably use Archive.getFileByName instead.
<bigjools> ta
<wallyworld_> bigjools: do you know if all members of a team should be notified of recipe and ppa build failure/success
<wallyworld_> or wgrant: ^^^^
<bigjools> BFNs go to the team, yes
<bigjools> dunno about recipes though
<wgrant> Members should.
<wgrant> But owners probably won't, despite having privs.
<wallyworld_> a guy on #launchpad says he gets notified for a team he owns, but not a team he is admin of
<jelmer> wallyworld_: same thing for recipes
<wallyworld_> so i should just tell him to make himself a member?
<wgrant> wallyworld_: The issue is that ~videolan has a contact address set.
<wgrant> So failure emails will go to that mailing list, not the members.
<wgrant> Also, it's 11pm.
<wgrant> EOD was a while ago?
<wallyworld_> wgrant: ok. i would have expected members get notified too but i can see that if the contact address is a mailing list for eg that you wouldn't want that
<wgrant> wallyworld_: It's pretty horrible, but that's how LP works.
<wallyworld_> wgrant: it's only 10pm here - we don't have daylight saving :-(
<wgrant> If there is no team address, it goes to the members. If there is, it doesn't.
<wgrant> wallyworld_: Oh, right. Lucky people.
<wallyworld_> wgrant: since i'm on chr, i figured i should hang around as long as i could :-)
<wallyworld_> wgrant: i wish we had dls. you don't like it?
<wgrant> We do need more CHR coverage, but I'm not really sure that's the solution...
<wgrant> wallyworld_: It's confusing and pointless!
<wallyworld_> wgrant: well, each is entitled to their opinion :-)
<jelmer> wallyworld_: What are dls?
<wallyworld_> jelmer: daylight saving
<jelmer> ah :-)
<wallyworld_> jelmer: it's too late for me to type properly and my typing is bad enough as it is :-)
<jelmer> heh
<bigjools> we need permanent DLS here but the Scots get all huffy about it :)
<wallyworld_> bigjools: thanks for not mentioning the cricket :-)
 * jelmer hadn't realized there was a difference in daylight saving across Australia
 * jelmer wonders what this "cricket" thing is people keep talking about...
<wgrant> jelmer: Western Australia a couple of years back decided to trial DST... with three weeks of notice.
<wallyworld_> jelmer: where you from?
<bigjools> the funny thing about Aus is the lack of DST in the north, coupled with the 1/2 hour TZs makes for an interesting time if you're travelling
<wallyworld_> wgrant: so, there is *no* way a user can get notifications if a contact address has been set for a team?
<wgrant> Yeah, SA just had to be different...
<wgrant> wallyworld_: They'll go the ML.
<bigjools> and NT
<wgrant> wallyworld_: The ML might reject them.
<wgrant> bigjools: Everyone ignores NT.
<bigjools> heh
<jelmer> wallyworld_: the Netherlands
<wallyworld_> wgrant: :-( we need a checkbox to say "send to team members" *and* contact address
<wgrant> wallyworld_: No, we need to redesign Launchpad notifications :/
<wallyworld_> jelmer: the netherlands had a cricket team in the last world cup :-)
<wallyworld_> wgrant: well maybe, but I was conceptualising :-)
<wallyworld_> jelmer: cricket for dummies :-P http://www.angelfire.com/dragon2/kamrantariq/
<jelmer> wallyworld_: Apparently we do, but nobody seems to care about cricket.. I didn't know we had a team until somebody congratulated me with our victory over (I think) the UK.
<jelmer> wallyworld_: heh, thanks :-)
<wallyworld_> jelmer: i guess football is your thing. what else do you play over there?
<jelmer> yeah, the popular things to watch on tv here are football, (indoor) speed skating and cycling
<bigjools> UK?!  The UK does not have a cricket team.
<jelmer> Wait.. there's separate teams for England, Wales, Scottland and Northern Ireland?
<jelmer> *Scotland
<bigjools> yep
<bigjools> "United" Kingdom :)
<bigjools> jtv: reckon we should change the make at the bottom of utilities/rocketfuel-get to use -j2?
<jtv> bigjools: yesâ¦ I'll bet there will be more failures, but we'll want to solve those not hide them.
<bigjools> indeed
<bigjools> I tried it on dogfood and get failures after a make clean
<jtv> Something to be announced on the mailing list.  We also want to maintain healthy interest in the topic of faster builds.  :-)
<bigjools> hell yes
<jtv> How about an environment variable, and asking everyone to experiment and look for failures?
<jtv> Once we feel we've ironed out the immediate failures, we can jack that variable up to -j2 by default.
<bigjools> yup
<jtv> And then we go to work on zcml time.  :)
<bigjools> fun
<jtv> Pickle their data and maintain them with "make."
<wgrant> bigjools: Why'd the buildd-manager async thing land on db-devel?
<bigjools> I can't remember
<bigjools> probably a dodgy MP
<wgrant> It seems like the sort of thing that we want to get out ASAP so we can revert it, since it's going to break....
<bigjools> ?
<wgrant> All major buildd-manager changes break at least twice.
<bigjools> sigh
<wgrant> It's far easier to roll back if we don't have a DB rollout in the middle.
<wgrant> Like last time.
<wgrant> Although this one does look safe enough.
<bigjools> hmmm, how is Distrubution.getFileByName() failing for deleted orig files
<wgrant> bigjools: From the views:
<wgrant>   WHERE securesourcepackagepublishinghistory.dateremoved IS NULL;
<bigjools> nope
<wgrant> Oh?
<bigjools> it's using SPFP.selectBy(distro, lfa, archive)
<bigjools> then ordering on ID
<bigjools> which sucks that it has to order at all
<wgrant> bigjools: SPFP is a view.
<wgrant> bigjools: That view has the WHERE clause that I just pasted.
<bigjools> I..... hate views
<wgrant> :D
 * wgrant hugs FTPArchiveHandler.
<bigjools> I hate  regex
<bigjools> trying to work out if re_issource will match orig fiels
<bigjools> ah yes, it will
 * bigjools back later, fud time
<LPCIBot> Project devel build (263): STILL FAILING in 2 hr 38 min: https://hudson.wedontsleep.org/job/devel/263/
<deryck> gmb or allenap -- what was the bug # you guys used for "use notification level for structural subscriptions" ?
 * allenap looks
<allenap> deryck: Ah, what am I thinking. Structural subscriptions have always worked with levels, but levels were not exposed in the UI. I think gmb knows about that part.
<deryck> allenap, gmb, yeah, that's what I mean, the ui bit.  the recent work we've done.  There's an older bug, but I'd like to dupe it against the recent one we worked on.
<deryck> I think bug 213261 could be duped against the bug representing the work we just did.
<_mup_> Bug #213261: Allow specifying the notification level for structural subscriptions <email> <story-better-bug-notification> <Launchpad Bugs:In Progress by gmb> <Launchpad Bugs 1.2:Triaged> <https://launchpad.net/bugs/213261>
<gmb> deryck: bug 672507 is the one I used.
<_mup_> Bug #672507: Add bug_notification_level to the structural +subscribe view <qa-ok> <story-better-bug-notification> <Launchpad Bugs:Fix Released by gmb> <https://launchpad.net/bugs/672507>
<deryck> ah ha.  Thanks, gmb!
 * gmb notices that he has a branch called "lp:~gmb/launchpad/oh-my-god-my-eyes". No idea what that's about.
<LPCIBot> Project db-devel build (177): FAILURE in 3 hr 47 min: https://hudson.wedontsleep.org/job/db-devel/177/
<LPCIBot> Launchpad Patch Queue Manager: [r=gary,
<LPCIBot> stub][ui=none][bug=682728][no-qa] Make ParsedApacheLog.bytes_read a
<LPCIBot> bigint so that it can cope with larger files.
<jelmer> mrevell: hi
<mrevell> Hello jelmer
<jelmer> mrevell: I vaguely recall that Launchpad used to have a small banner on the top or bottom of all pages saying "Launchpad will go down for 1 hour 23 hours from now.". What happened to that?
<mrevell> jelmer, AFAIK we have the facility to do that 15 minutes from the impending down-time and mthaddon and friends switch that on.
<mrevell> jelmer, There has been talking, on several occasions, of creating a general purpose notification system.
<jelmer> mrevell: Something like that seems useful for this sort of thing. Before I was a lp developer I know the only thing I'd really notice for sure would be the notification on Launchpad itself.
<mrevell> jelmer, I'd love to see something like that. I guess it's a lower priority than implementing features that stakeholders need.
<LPCIBot> Project db-devel build (178): STILL FAILING in 3 hr 50 min: https://hudson.wedontsleep.org/job/db-devel/178/
<LPCIBot> * Launchpad Patch Queue Manager: [r=gmb][ui=none][bug=662631,
<LPCIBot> 668060] Make file uploads from builders asynchronous so that it
<LPCIBot> doesn't block the rest of the build manager. This is the last
<LPCIBot> part of the build manager to be made asynchronous.
<LPCIBot> * Launchpad Patch Queue Manager: [rs=buildbot-poller] automatic merge from stable. Revisions: 12000
<LPCIBot> included.
<gary_poster> sinzui: could you take a look at https://bugs.launchpad.net/canonical-identity-provider/+bug/556680 for me and then talk with me?  I'm hoping you will have some brilliant insight, or at least that you will agree with Anthony
<_mup_> Bug #556680: attempting to create a new account with an existing email address at login.ubuntu.com oopses after you follow the email link back to create the name/password <isd-logging-sprint> <Canonical SSO provider:Incomplete by stuartmetcalfe> <Canonical ISD QA:New> <https://launchpad.net/bugs/556680>
<gary_poster> 's analysis and be able to help me come up with an actionable plan for his description of "let Launchpad refuse an account when it tries to sign in, it it notices it's related to a team Person."
<lifeless> http://pypi.python.org/pypi/injector/0.3.1 looks kindof interesting
<lifeless> poolie: ^
<poolie> hi lifeless
<poolie> lifeless: it is a bit interesting
<poolie> it seems like it brings quite high-proof abstraction
<poolie> or rather, indirection
<poolie> such that you wouldn't want it unless you needed it
<poolie> what do you think?
<lifeless> I'm reminded of greenspuns law, or whatever it is
<lifeless> about half baked lisp implementations
<poolie> indeed
<poolie> :(
<lifeless> I think that 'injector' would be overkill quite often
<lifeless> OTOH perhaps its solid, thought out, reliable
<poolie> it seems like having the option of dynamic scope would mostly obsolete this
<poolie> the decorators to mark what's injected are kind of nice
<lifeless> perhaps... might bend folks brains a bit much
<lifeless> anyhow
<lifeless> I'm not proposing to use it, just thought it was interesting
<poolie> it is
<benji> is anyone else getting a "Error: Couldn't find a distribution for 'testtools==0.9.8-r137'.
<benji> ... error when running make?
<benji> I've updated the download-cache with no effect.
<mars> benji, check the mailing list, Nov 10th, message from Gary 'Fix your testtools', looks relevant?
<benji> looking
<mars> benji, ah, maybe not.  There was something about this recently though
<benji> mars: nope; it's simply that the r137 version is missing
<benji> I've changed the versions.cfg in my branch to let me build, but it looks like someone checked in a new requirement without adding the distribution for that version
<james_w> anyone have a clue how specification-bug links should be exported?
<rockstar> james_w, specs can be linked to bugs?
<james_w> the only other user of IBugLinkTarget is answers, which is not exported either
<james_w> rockstar, they can indeed
<james_w> rockstar, through different code to branches apparently
<wgrant> benji: You download-cache update succeeded?
<mars> benji, bzr annotate versions.cfg, then check the revision history of download-cache for something in the same day
<wgrant> benji: The format changed recently, so you'll need upgrade before you can update.
<benji> wgrant: that's the problem; it attempted an on-the-fly upgrade, but I didn't notice that that failed
<mars> ah
<mars> gary_poster, ping, ^ do we need to tell everyone to manually upgrade their download-cache?
<gary_poster> mars, I did
<mars> gary_poster, ah, sorry, I just saw it got threaded (darn Thunderbird :)
<gary_poster> :-)
<benji> mars: at least you have an excuse; I saw the thread but didn't realize its significance
<maxb>  $ bin/jslint -a
<maxb> jslint: No files to lint.
<maxb> Help? I want to test an updated spidermonkey (/usr/bin/js) package I just built
<wgrant> maxb: Have you built the JS?
<maxb> I need something other than "make" ?
<wgrant> No.
<wgrant> jslint is currently consuming all of one core, so I presume it's working for me.
<maxb> hah - it did that to me, then eventually just stated "jslint: No files to lint."
<wgrant> Oh.
<wgrant> Well, I guess we'll see.
<wgrant> Eventually.
<maxb> Oh. That would be it doing "for file_id in self.tree" on the LP bzr tree, I guess
<maxb> And skipping lib/* and build/* ???
<maxb> Seeing as all the versioned .js files in the tree are under lib/, this is rather stupid
<wgrant> Where's this? In lazr.js?
<wgrant> Huh, that is a bit special.
<wgrant> I'm wondering whether I want to remove that check or not. I fear the number of errors...
<maxb> $ wc -l jslint-all.out
<maxb> 1449 jslint-all.out
<maxb> lots of lint
<maxb> lib/canonical/launchpad/icing/yui_2.7.0b/ ?!
<maxb> someone explain to the novice why we still have YUI 2 in tree? :-)
<maxb> jslint: Lint found in '/home/maxb/launchpad/lp-branches/devel/lib/lp/app/javascript/tests/test_lp_collapsibles.js':
<maxb> Line 20 character 9: Expected '}' and instead saw '}'.
<maxb> uhm thanks :-)
<wgrant> maxb: IIRC YUI2 is used for the calendar widget.
<maxb> yuck
<maxb> that is all I have to say
<wgrant> Of course we should never have had any YUI version at all in the tree.
<maxb> Can someone delete these for me? https://launchpad.net/~launchpad/+archive/ppa/+delete-packages?field.name_filter=xulrunner
<maxb> The binaries packages built from that source are now overridden by the spidermonkey source package
<wgrant> maxb: Is that going to end up in the primary archive at some point?
<maxb> I've given up following that saga
<maxb> It's easier to just PPAize the thing
<wgrant> Heh.
<wgrant> Also, we can probably delete psycopg2 now.
<wgrant> Since I'm running it fine with 2.2
<wgrant> Which must mean that launchpad-dependencies has been fixed.
<maxb> oh, point
<maxb>   * Remove dependency on python-debian, now in sourcedeps.
<maxb> too
<wgrant> I was just wondering about that.
<wgrant> So, can someone in ~launchpad please remove xulrunner, psycopg2 and python-debian from all series in the ~launchpad PPA?
 * maxb copies lp-deps 0.85 to lucid too, and sighs a bit that people don't follow the instructions
<wgrant> When did you do the spidermonkey copy?
<maxb> About a minute before I asked for deletion here
<wgrant> Hm.
<wgrant> Publisher is being slow again :(
<wgrant> But soon I will have logs, so I can maybe optimise it without harrassing too many people.
<lifeless> wgrant: A r c h i v e : + i n d e x
<wgrant> I tried to find my branch for that yesterday, but failed.
 * wgrant looks harder.
<poolie> mkanat: so it seems to me that if we're going to have a stable branch
<poolie> if there's going to be any point in it
<poolie> first, launchpad needs to deploy it
<poolie> second, we ought to do non-intrusive fixes onto that
<mkanat> poolie: Oh, launchpad is already running identical code to the stable branch (except for a fix in start-loggerhead, which they don't use).
<mkanat> poolie: Yeah, I agree about non-intrusive fixes.
<poolie> can you expand 'identical code'?
<poolie> if you land new fixes on 1.18, will they get them?
<mkanat> poolie: I branched the stable branch at the same point that launchpad is currently running from, I believe. I can double-check that, though.
<mkanat> poolie: They'll get them if I merge the stable branch into their branch when there are fixes.
#launchpad-dev 2010-12-01
<mkanat> poolie: The launchpad loggerhead branch is still named devel, though, it looks like.
<poolie> ok
<poolie> so istm we will stick with the 1.18 branch on launchpad for the medium term
<poolie> so i'd like you to concentrate your efforts on things that can go into that
<mkanat> poolie: Yeah, there are no fixes on the 1.18 branch that would affect loggerhead on launchpad, right now. I can still merge it, but it's mostly stuff like test changes, NEWS updates, release stuff, etc.
<poolie> thumper: ^^ agree?
<mkanat> poolie: Well, I think you might need to be more specific than that.
<poolie> perhaps i should nominate particular bugs?
<mkanat> poolie: Well, perhaps, but are we shifting away from our original plan?
<mkanat> poolie: See, I think that the trunk work that I'm doing is going to eliminate all of the problems in 1.18 with actually far less work than hunting down memory leaks, etc. will be.
<mkanat> poolie: Some of these things I can backport, if in a hackish fashion, though.
<mkanat> poolie: For example, the "don't annotate by default" thing I could backport by adding a URL parameter like ?annotate=1 to the "annotate" controller, instead of refactoring the controllers like I did for trunk.
<mkanat> Of course, that means that trunk and 1.18 will use different URLs for the same thing, and that the default behavior of 1.18 (which is to annotate when asked for /annotate/) will change.
<poolie> i guess this depends a bit on when we expect to do a 1.19, and how traumatic it will be
<james_w> anybody know what's going on with http://paste.ubuntu.com/538491/ when I do a "make"?
<mkanat> poolie: Well, in an ideal world I was hoping to be ready for 2.0 instead of 1.19.
<mkanat> poolie: I'd like say of 2.0 "this will absolutely scale to launchpad size".
<poolie> james_w: i haven't seen that but typically it means an  update in download-cache, and then make clean make
<poolie> it rings a bell
<mkanat> poolie: I'm totally with you on wanting to see these improvements on launchpad as quickly as possible, though, particularly the performance-related ones.
<james_w> oh, if you read up there are errors further back, sorry
<poolie> james_w: there was a thread on launchpad-dev about this last friday
<poolie> "Fwd: failure"
<poolie> mkanat: looking back at my mail "next steps on codebrowse" from november
<mkanat> poolie: I think that one of the problems I'm running into is that the template code and the CSS are great, in loggerhead, but the python pieces are pretty messy, so every time I go in to make a change, it's generally much more complex to do than it ought to be.
<poolie> :/
<mkanat> poolie: I'm improving that as I go a bit, though.
<mwhudson> james_w: you might need to upgrade your download-cache
<mwhudson> (the one on lp got upgraded to 2a lately)
<james_w> been through that one :-)
<james_w> seems I'm missing libpq-dev
<mwhudson> ah
<poolie> "obviously" :-|
<mkanat> poolie: So, re: the "next steps for codebrowse" mail, those are all still totally things I'd like to focus on.
<mkanat> poolie: There's just a lot of technical debt, too--to the point where there often aren't really great ways to do those changes in a non-invasive way. Sometimes there are hacky ways we could do branch fixes, though.
<lifeless> FWIW
<lifeless> and I really don't want to get into detail
<lifeless> I am happy with LP running off trunk.
<lifeless> as long as the existing process and QA are followed.
<mkanat> lifeless: Does the LP team have a QA process for loggerhead?
<lifeless> same as all other changes
<poolie> so trunk currently contains history-db etc
<lifeless> which I and francis described in a thread with you a few weeks back
<poolie> rolling that out will have some operational implications
<poolie> ie it'll create different cache dbs
<mkanat> lifeless: I really would be opposed to running trunk on LP, FWIW.
<lifeless> if its ready for use
<lifeless> then its inventory we're not benefiting from.
<lifeless> mkanat: lp itself is vastly larger and we run trunk.
<mkanat> lifeless: LP also has a reasonable test suite and FAR more development effort available.
<lifeless> sure
<mkanat> lifeless: I don't think it's as much about size as it is about how many people you have on hand who can immediately spend the next week solving a critical issue.
<elmo> wgrant: is poppy known to spew exceptions pretty much non-stop?
<mkanat> I'd say that it's actually easier to run trunk of a large project with excellent tests and a large development team than it is to run trunk of a small project with worse test coverage and a small team.
<lifeless> lp teams can put time into loggerhead if needed - its something we've previously said we'll maintain. We've no feature work planned for loggerhead though, FWIW.
<elmo> alternatively - how soon should I expect packages I upload to a PPA to appear in the webUI?
<poolie> elmo: 5-10m
<poolie> (unauthoritative answer)
<elmo> k
<mkanat> lifeless: Ultimately, I think that I am the person with the most general loggerhead knowledge (although jam is probably pretty good at it by now too), though, so one way or another, if we put trunk into production, I will be on the hook.
<poolie> i don't think we should have non-staff on the hook
<poolie> i talked to francis about doing a squad rotation onto loggerhead
<lifeless> mkanat: I'm not asking for us to run trunk.
<lifeless> mkanat: I'm being clear that I consider it an ok option.
<mkanat> lifeless: Okay.
<lifeless> When I get back from leave I'd be happy to discuss further.
<lifeless> thats jan.
<mkanat> lifeless: That's fine, I think that we have a plan that will work one way or another.
<mkanat> lifeless: By January I'd like to see what's currently on trunk be stable, in any case.
<mkanat> poolie: I like the idea of just having a "no annotate and other codebrowse stuff" release. It's possible that I could create a branch and cherry-pick stuff from trunk for that.
<mkanat> poolie: And that could be 1.19.
<poolie> in some ways that would be nice to do because it would give us an easier dry run for deploying your work to lp
<poolie> well, semi-wet
<poolie> and then you don't need to backport them in a hacky way
<mkanat> poolie: Yeah. It would still be additional work, probably, because I'll have two different branches to test, there will be code differences, etc.
<poolie> right, i was just considering the idea that in some ways doing any branches is a waste of time and effort
<poolie> at least as far as the lp deployment
<mkanat> poolie: Sure, except for when we are ready to actually say, "this is table".
<mkanat> *stable
<mkanat> poolie: For example, so far, the 1.18 branch is very little work, and I *have* landed a fix on it.
<mkanat> poolie: (It just happened to be a fix that didn't affect LP.)
<poolie> can we keep trunk just adequately stable all the time?
<mkanat> poolie: Ah, I'm not sure. That's some fair bit of effort for a web app. I mean, it's possible, but I'd have to look at the test suite quite a bit, probably.
<mkanat> There are decent-coverage tests, but I know that some of them don't even pass on 1.18 right now (although the failures don't represent bugs).
<mkanat> poolie: Also, we'd have to be sure that all reviewers required unit tests on checkins.
<mkanat> poolie: There would be more process involved.
<mkanat> poolie: I don't know if adding that sort of process to a smaller project is a good idea.
<poolie> it may not be
<poolie> otoh having trunk always stable does avoid some kinds of churn and waste
<mkanat> poolie: Yeah, it totally does. It's absolutely more efficient in many or most ways, it's just not necessarily *easier*. :-)
<mkanat> poolie: I'm thinking about the ideal solution.
<poolie> the other thing from that november thread was adding etag and http cache support
<mkanat> poolie: Yes, although that might be less of a general win than the other improvements, since I suspect that a lot of page loads in loggerhead are things that will not be cached.
<mkanat> poolie: I think there could be some reasonable ways to do it without too much effort, for some pages, though.
<mkanat> poolie: What I most *want* to do is to do everything from the "next steps" email, improve performance in a few more places, get great coverage on the test suite, and *then* branch for a new release.
<poolie> i would like that too
<poolie> two questions:
<poolie> 1- does that make sense to do in one block, or should we do the annotation changes separately?
<poolie> 2- roughly how long elapsed/effort would it take?
<mkanat> poolie: Okay. Well, re: (1), I think that with the current development model of loggerhead, and since history-db which came before was such a big change, I think we should do all the changes, then stabilize, then after that release, look into doing smaller, more rapid releases.
<mkanat> poolie: For (2), I'd guess that we're looking at 50-75 hours, although it could be less (and of course, could be more).
<mkanat> poolie: In terms of time, I'd like to get it done in two weeks or less--I'm focusing heavily on loggerhead work right now (and also very much enjoying doing the work).
<poolie> good
<poolie> i'd like to keep you in the rhythm
<poolie> i think, even if it's a bit less efficient, i would like to do a 1.19 that's got the annotation changes and that's deployable
<poolie> and, in fact, deployed
<rockstar> BLUE SQUAD!
<lifeless> mkanat: btw
<poolie> then go on with other things later
<lifeless> mkanat: if the loggerhead caches are k-v, you might consider moving to cassandra
<mkanat> lifeless: Can the value be a tuple or a dict?
<mkanat> poolie: Okay, I can totally do that.
<lifeless> mkanat: sure; I mean it serialises.
<mkanat> poolie: Or I could just merge the annotation changes into the LP branch.
<lifeless> mkanat: see my posts to the lp-dev list for a summary/description
<poolie> yeah
<mkanat> lifeless: Is Cassandra the DB system you posted about on the list...yeah.
<poolie> i think conceptually "here's the loggerhead branch" and "here's what lp deploys" are a bit different but it's not a big deal
<mkanat> lifeless: I don't think we'll need that level of performance from the cache alone, at this point.
<mkanat> lifeless: Right now we're using sqlite and it's good enough.
<mkanat> lifeless: Although loggerhead could be a reasonable PoC for Cassandra usage, if somebody felt like doing that work.
<lifeless> mkanat: we have no stats to show if its good enough or not
<lifeless> mkanat: if we instrument lock latency on sqlite and various related things, then we could say that
<mkanat> lifeless: Well, I wouldn't worry about it at this point.
<elmo> mm
<elmo> the publisher is deadlocking
<poolie> mkanat: in some ways i like the idea of having a separate branch for 1.19 vs the lp deployment branch
<poolie> if only for the reviews: "i'm fixing this bug" vs "i'm deploying this block of stuff"
<mkanat> poolie: It's a possibility. It will be more work, but I suppose I'd be happy to do it if you'd like me to.
<poolie> perhaps we should just be lazy there
<mkanat> poolie: That's sort of my inclination.
<poolie> fair enough
<poolie> you can ping me, spiv, jam, etc, for reviews
<mkanat> Okay, sweet.
<poolie> the default reviewer may not be who you want
<mkanat> poolie: Hmm, okay. Any recommendations for who to specify when making a new MP?
<poolie> ~bzr and ~launchpad?
<mkanat> poolie: Okay. I'll do ~bzr--I suspect that will get me the most likely reviewers.
<mkanat> I'll do ~launchpad for the deployment merges, I suppose, yeah?
<lifeless> mkanat: the default is appropriate there
<mkanat> lifeless: Okay, will use the default then, for that.
<mkanat> Is something up with LP right now?
<mwhudson> mkanat: data centre issues it seems
<mkanat> okay
<wgrant> The frontends are in the other DC?
<mwhudson> seems so
<lifeless> apache is in one dc
<lifeless> some appservers are in each
<lifeless> === Top 10 Time Out Counts by Page ID ===
<lifeless>     Hard / Soft  Page ID
<lifeless>      324 / 7217  Archive:+index
<lifeless>      148 /  451  BugTask:+index
<lifeless>       36 /  150  ProjectGroupSet:CollectionResource:#project_groups
<lifeless>       19 /  296  Distribution:+bugs
<lifeless>       18 /   13  Cve:+index
<lifeless>       13 /  278  Distribution:+bugtarget-portlet-bugfilters-stats
<lifeless>       11 /   38  MailingListApplication:MailingListAPIView
<lifeless>       11 /   19  DistroSeries:+queue
<lifeless>        9 /   16  Person:+specworkload
<lifeless>        8 /   51  Milestone:+index
<lifeless> wgrant: every binary build makes a:+i a little worse ;)
<poolie> could someone please sponsor merging of https://code.launchpad.net/~mbp/launchpad/dkim/+merge/41819 for me?
<StevenK> poolie: Sure, let me toss at ec2 for you
<poolie> thanks
<lifeless> poolie: I'll be in syd tomorrow through sunday early avo
<poolie> oh, really?
<poolie> can come over if you want
<poolie> will just check with s
<poolie> got to go now
<lifeless> ciao
<jtv> Anyone else getting test failures on db-devel?  I just saw a whole lot of tests for the ec2 script itself fail.
<bigjools> morning all
<adeuring> good morning
<mrevell> Morning
<lifeless> wgrant: http://panospace.wordpress.com/2010/11/24/thank-you-sourceforge-hello-launchpad/
<wgrant> lifeless: I was with it until "The web interface is light"
<lifeless> wgrant: forest for the trees ;)
<lifeless> wgrant: have you used the SF tracker recently? or jira?
<bigjools> anyone seen stub?
<wgrant> lifeless: The project I worked on at uni used to use SF's tracker... we moved it to LP.
<wgrant> lifeless: So I haven't used it in a year or so. But I recall it was pretty damn awful.
<mkanat> I think I'd say that SF's tracker is the worst I've used.
<mkanat> It's too bad, really. SF, I think, was probably the only site that had the potential to be THE site, where every open source project could or would have hosted their stuff.
<mkanat> It just wasn't very good.
<henninge> jelmer: Hi!
<jelmer> henninge: hi
<henninge> jelmer: last Friday night I merged our recife feature branch into db-devel
<henninge> now we have to roll back that merge unfortunately
<henninge> and two others that landed on db-devel after it.
<jelmer> henninge: Ah
<jelmer> henninge: Thanks for the heads up, you need me to reland my branch?
<henninge> jelmer: not sure, I'd like to ask your help on the procedure.
<henninge> This is more a bzr question, actually.
<jelmer> henninge: ah, ok
<henninge> jelmer: actually, I don't think anybody else's branches need to be re-landed.
<henninge> So we'd roll back the changes by reverse-merging those revisions in db-devel.
<henninge> but we also want to use our feature branch again.
<henninge> the feature branch is based on db-stable and has had it merged into it regularly.
<henninge> We'd plan to continue that while we still work on it.
<henninge> The question is: At one point we will merge db-stable into recife and it will contain that roll-back revision.
<henninge> I'd expect that to clean out all of our feature from the feature branch.
<henninge> How do we get around it?
<henninge> Would it be okay to simply not merge that revision from db-stable?
<henninge> jelmer: can you follow? ;)
<jelmer> henninge: yeah :)
<henninge> (feature branch == recife branch)
<jelmer> You can't really not merge that revision from db-stable
<jelmer> but what you could do is mark that revision as merged but not take any of its changes ("bzr merge"; "bzr revert .")
<henninge> ah!
<henninge> I have done that before but thought it would remove any trace of the merge from the branch.
<henninge> So, it leaves a "merged revision X" mark on the target branch?
<jelmer> The "." is important because it means the pending merges are still remembered. "bzr status" should still show them.
<henninge> so "bzr revert ." is different from "bzr revert" at that point?
<jelmer> henninge: Yeah (arguably a bit of a UI bug in Bazaar)
<henninge> I see. Yeah, I would have missed that.
<henninge> jelmer: thanks, that is exactly what we need, then.
<wgrant> bigjools: The getFileByName bug that let the dupe linux orig.tar.gz into hardy.
<bigjools> wgrant: but only if it was deleted from disk, right
<bigjools> wgrant: the dude has copied two versions at the same time
<bigjools> ah, I see, his source PPA had conflicting orig files....
<wgrant> Exactly.
<wgrant> It can't cause a PFOE itself.
<bigjools> yeah
<wgrant> A new pub of the old one has to be created.
<bigjools> did you see my fix BTW?
<wgrant> Was there one?
<bigjools> https://code.launchpad.net/~julian-edwards/launchpad/upload-file-conflict-bug-663562/+merge/42264
<wgrant> bigjools: Is the archive ever None?
<bigjools> yes, in a  very tiny corner case
<bigjools> partner....
<wgrant> Where partner doesn't exist?
<bigjools> the lovely partner
<wgrant> Kill it.
<wgrant> That fix looks fine.
<wgrant> Can you remove Distribution.getFileByName?
<bigjools> not looked tbh
<wgrant> I recall there were only a couple of callsites.
 * wgrant looks.
<wgrant> (I investigated on my view extermination campaign)
<wgrant> bigjools: That same bug exists in PackageUploadSource.verifyBeforePublish.
<wgrant> bigjools: And the only other callsite is in SyncSource, and it can be adjusted the same way.
<bigjools> yeah
<wgrant> I have to wonder about the sanity of triplicating that logic.
<bigjools> wgrant: ignorance, no doubt.  And will it get any better with the team change ....
<wgrant> :D
<bigjools> wow, all builder queues are empty
<wgrant> Even powerpc?
<wgrant> Hm, indeed.
<bigjools> doko wants to do a rebuild
<wgrant> doit
<bigjools> including universe
<wgrant> We need to test.
<wgrant> Although I'd like to get a daily builds session through the async download code first, I guess.
<jtv> henninge: my rollback branch is still pushing at lp:~jtv/launchpad/recife-rollback-10-12.  I gather you've been able to extract the secrets from jelmer.
<deryck> Morning, all.
<jml> bigjools: hi. just finishing something up. won't be long.
<jml> bigjools: re StevenK's database patch... we are building a linked list structure into SPPH?
<wgrant> jml: It's a tree of pain, not a linked list, but yes.
<jml> ... because many things can have the one parent, I see.
<wgrant> In the case of a copy, yeah.
<StevenK> jml: Has wgrant answered your questions?
<StevenK> wgrant: Tree of pain? :-(
<wgrant> There will be pain. But it won't be our problem any more :)
<deryck> *sigh* I continue to have failure at landing this lazr-js upgrade.
<deryck> gary_poster, would love a moment of your time when you're available.
<jml> StevenK: no he has not, see the MP.
<StevenK> jml: Commented.
<james_w> salgado, hi. Did you look at exposing blueprint-bug links by any chance?
<salgado> james_w, nope, do you want me to?
<maxb> Where are the buildds supposed to get their bzr-builder for recipe builds from these days?
<maxb> I am asking because of bug 682941
<_mup_> Bug #682941: no such option --append-version in recipe build <Launchpad Bazaar Integration:Confirmed> <https://launchpad.net/bugs/682941>
<bigjools> jml, sorry was at lunch.  Hope you got your questions all answered.
<james_w> salgado, it's the only thing missing to port the current collect script to the API. If you could give it a quick look with me that would be great.
<james_w> salgado, Specification implements(lp.bugs.interfaces.buglink.IBugLinkTarget)
<salgado> james_w, you don't need to be able to create new links, right?  Just reading existing ones should be enough?
<james_w> and that interface has a bugs attribute that looks like it is what we want to export, but as the interfaces aren't linked will doing the expose be enough?
<james_w> salgado, for this use case, yes
<salgado> james_w, what do you mean by interfaces are not linked?
<james_w> salgado, ISpecification doesn't inherit from IBugLinkTarget
<james_w> salgado, does the webservice code traverse the model to see what it implements too?
<salgado> oh, good question
<salgado> I think it does
<salgado> but we'd need to export IBugLinkTarget
<salgado> just like we export ISpecificationBranch
<salgado> yeah, that should be everything needed to export .bugs
<salgado> james_w, would you like me to do that?  should be quick
<james_w> salgado, I have a test written, but I forgot that I hadn't set up an LP dev environment on this machine yet, so it's taking me a while to get to actually running it :-)
<james_w> salgado, if you think it's just a few minutes I'm happy for you to take over, but I can keep plugging away at it
<salgado> james_w, whatever you prefer.  I'd expect a bit more than a few minutes to get something landable, but it should be faster as a quick hack for you to test
<james_w> salgado, ok, I'll keep going at this, as it will be useful for me to have a working LP dev environment again. I'll let you know if I hit any roadblocks.
<salgado> ok
<james_w> thanks for the help
<salgado> np
 * salgado breaks for lunch
<gary_poster> deryck, hey, pong, though other things going on
<deryck> gary_poster, hey, just finishing standup now myself
<gary_poster> cool
<deryck> gary_poster, off now.  So my lazr-js upgrade gets the same error again.  and it builds locally -- i.e. python setup.py build -- just fine....
<deryck> gary_poster, however python setup.py --version reports 1.5DEV and versions.cfg expects 1.5DEV-r190.  Could this be the issue?
<gary_poster> deryck, remind me the error please?  can't quite place it
<deryck> gary_poster, I'll forward you an email if that's ok.  basically just "failed to build" or some such.
<gary_poster> sure, deryck .  and yes, that sounds suspicious
<deryck> gary_poster, mail sent
<gary_poster> ack
<deryck> gary_poster, is there something I can do locally to make my build *just* like what pqm tries to reproduce this?  Feels a bit hit or miss to hack a new tarball, fire off to pqm, rinse/repeat
<gary_poster> deryck: if your download-cache is pristine and like the upstream, you should be good to go for testing locally as long as your eggs have not been messed up by hacking.  if you cd into your download-cache and run bzr status, what does it report?  Also, does bzr info look like http://pastebin.ubuntu.com/538665/
<gary_poster> deryck: if bzr status is clean and bzr info is similar, then I suggest wiping out the lazr-js egg(s) from your eggs directory and rerunning bin/buildout
<gary_poster> at that point, if you haven't duped, I'll be surprised
<deryck> gary_poster, bzr st reports nothing, and yes, my bzr info reports that same as your paste.  bzr log -l 1 shows my commit to add the current lazr-js tarball.
<deryck> gary_poster, ok, I'll try that.  thanks!
<gary_poster> np, lemme know
<jml> do we have an end-to-end overview of our dev process?
<gary_poster> I don't recall ever seeing one, jml.
<jml> gary_poster: ta, I might have to make one then.
<gary_poster> sounds good jml :-)
<jml> flacoste, mrevell: just making a cup of tea before the standup. will be a tiny bit late.
<mrevell> jml, np
<deryck> gary_poster, it worked fine.  weird.  https://pastebin.canonical.com/40332/
<deryck> bac, appologies but I won't be able to make reviewer meeting today.
<gary_poster> deryck: very weird.  only other possibility I can think of is that there's a built egg on pqm and lazr-js had two revisions with the same name or something...
<gary_poster> deryck: I'll try to build locally in a min
<deryck> gary_poster, ok, thanks!
<gary_poster> deryck: to be clear, the pqm hypothesis seems pretty unlikely to me.  I think we wipe out built eggs on pqm
<gary_poster> between builds
<deryck> ah
<deryck> hmm, yeah, that's weird.
<jml> mrevell: http://pastebin.ubuntu.com/538678/
<deryck> gary_poster, so I could do a hand-code version string in the setup.py and ensure that matches and try again.  if so, I can commit to lazr-js then.
<bac> meeting ping: abentley, adeuring, allenap , bac, benji, danilo, sinzui, deryck, EdwinGrubbs, flacoste, gary, gmb, henninge, jelmer, jcsackett, jtv, bigjools, leonard, mars
<deryck> where, if so == it pqm build works
<bac> deryck: ok
<flacoste> bac: apologies from me
<jtv> bac: anon!
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build (264): FIXED in 4 hr 2 min: https://hudson.wedontsleep.org/job/devel/264/
<gary_poster> deryck: I have been under the impression that the lazr-js egg does some funky things (possibly because it is trying to build js resources in a Python packaging world, dunno).  In any case, buildout/setuptools/distribute typically is just fine with the -r stuff that setuptools/distribute produces
<gary_poster> so it's worth a try, but still pretty mysterious
<LPCIBot> Project db-devel build (179): STILL FAILING in 4 hr 5 min: https://hudson.wedontsleep.org/job/db-devel/179/
<gary_poster> deryck, I think I have a clue
<mars> flacoste, ping, I have a question about the simplified merge machinery tracking
<gary_poster> deryck: (that comes after discovering that it also worked for me)
<deryck> gary_poster, ok, awesome.  on call now....
<gary_poster> deryck: call: ack.  Clue from your failure email: "Downloading http://pypi.python.org/packages/source/d/distribute/distribute-0.6.10.tar.gz".   Distribute is not included it our download-cache, and I expect that it is included on most/all dev machines but not on pqm.
<gary_poster> Core problem IMO: lazr-js explicitly depends on distribute.  I think this is a bad idea, because it unnecessarily excludes setuptools users.  In contrast, specifying setuptools *includes* distribute.  My suggested fix is to change lazr-js's setup.py to require setuptools, not distribute.
<gary_poster> Other possible fixes are to include distribute in the download cache or to make pqm have distribute.  Maybe those are also good ideas, but specifying distribute seems unnecessarily antagonistic to setuptools to me.
<bigjools> abentley: I just noticed that recipe builds are ending up with a buildqueue score of 2605 on dogfood.  Is that intentional?
<abentley> bigjools: I haven't changed anything related to that.  I expect them all to be 2505 ATM.
<bigjools> ok
<flacoste> mars: what's the question?
<james_w> leonardr, hi. We have a case where we want to export bugs on IBugLinkTarget for blueprints. ISpecification doesn't inherit from that, Specification just implements() it. Just exporting IBugLinkTarget doesn't seem to be enough. Do you have any advice on how to proceed?
<deryck> gary_poster, thank you.  A very clear analysis that helps me understand.  I'll look into why lazr-js requires distribute and fix if possible.
<deryck> rockstar, ping.  do you know why we require distribute explicitly for lazr-js, instead of just using setuptools?
<deryck> hmmm
<deryck> gary_poster, so I see a merge that led to this -- https://code.launchpad.net/~sidnei/lazr-js/newer-buildout/+merge/21417
<deryck> gary_poster, how would you feel about me temporarily adding distribute to download-cache, and opening a bug against lazr-js for this?
<deryck> since I'm not sure why we went this way in the first place.
<gary_poster> deryck: on call
<deryck> ack
 * deryck needs to eat
<rockstar> deryck[lunch], the distribute thing is a big reason why I want to kill the build system in lazr-js.
<rockstar> It always either Just Works for landscape or for Launchpad, but never both at the same time.
<bigjools> jtv: can you remind me where to look to see finished translations template jobs?
<jtv> bigjools: errr didn't they get cleaned up?
<bigjools> jtv: I am just QAing on dogfood, I want to make sure the job finished
<jtv> Anyway I don't have database access now so I'm falling out of touch with the schema.
<jtv> It's job type 6 IIRC; let me check.
<bigjools> jtv: there was a page I could go to to see the template?
<bigjools> 4 actually
<bigjools> jtv: I need a page, not a query :)
<jtv> Ah.  One of the job types for this class was 4 and the other 6, and I always forget which is which.
<jtv> Try the import queue page for the productseries.
<jtv> https://translations.dogfood.launchpad.net/test_product/test_series/+imports
<jtv> If the job completes successfully, you'll see a template upload appear there, owned by "VCS Imports."
<jtv> This last bit is significant; if the branch contains a template file, that one will also show up (probably earlier) under the identity of, if I remember correctly, the person who committed to the branch.
<bigjools> ok, thanks
<jtv> Since there's no importer running on dogfood, the queue entry will be in Needs Review state.
<bigjools> I see the one I did last time we talked but not the one I just did :/
<jtv> Don't blindly trust the date on the entry; if there was already a previous upload with the same path, productseries, and uploader, it will go into the existing entry without updating its creation date.
<jtv> So try opening the link to the file itself (<something>.pot) and reading the POT header inside the file.
<jtv> It'll have a timestamp.
<bigjools> jtv: aha, that's it
<bigjools> thanks
<jtv> no worries
<bigjools> I was panicking for a while about competing jobs on a single builder until I realised that the translations job creation script sends them to a nonvirtual builder by default!
<gary_poster> rockstar: do you have time for a five min call?  I'll even set a timer if you want :-)
<gary_poster> deryck[lunch]: the distribute thing is a pain; I had completely forgotten about sidnei's branch.  If putting distribute in the download-cache works, I guess that's ok, though it makes me unhappy.  Whatever--if you can move on with this landing, it's not that big of a cost.
<deryck> gary_poster, ok, thanks.  I'll add to download-cache to unblock me, and then follow up with a bug and discussions to see if we can fix lazr-js.
<deryck> well, scrollback suggests rockstar is going to fix it anyway.
<gary_poster> Cool deryck.  yeah, but rockstar has approx 1000000 things on his plate AFAIK. :-)
<rockstar> deryck, and by fix, you mean "destroy with a firey passion"
<rockstar> gary_poster, yes, yes I do, but lazr-js is on the official plate, rather than the unofficial plate.
<deryck> gary_poster, yes, but see "firey passion" above ;)
<gary_poster> :-)
<gary_poster> rockstar: you cleverly showed up at the time when I only have four min
<gary_poster> quick call now?
<gary_poster> like, three min :-)
<rockstar> gary_poster, pots?
<lifeless> jml: hi
<lifeless> abentley: hi
<abentley> hi.  in TL meeting
<lifeless> https://dev.launchpad.net/PolicyAndProcess/ReleaseManagerRotation?action=diff
<lifeless> abentley: a tweak to your edit, FYI/clarity.
 * lifeless goes to hop on a plane
<lifeless> I hope its clear.
<lifeless> ciao
<sinzui> flacoste, benji-lunch, gary_poster ping
<flacoste> hi sinzui
<gary_poster> sinzui: hi
<sinzui> IHugeVocabulary requires a displayname for the picker. I think we need to provide a step_title too since UI testing revealed users were not sure what would be matched
<gary_poster> sinzui: I was also going to try and ping you about the question I gave you yesterday.  Trying to get it again
<deryck> dear lazr-js, I like you, do you like me? check one.  [  ]yes  [x]no [  ]maybe
<gary_poster> lol
<gary_poster> sinzui: I'm not clear yet on what a step_title would be
<sinzui> This is certainly the case with the two vocabularies I have changed. I ponder changing all implementers to provide the generic "Search" we see now, but my branch would be smaller if I could just add the attr to the vocabularies I care about
<sinzui> I am not sure how to add a readable attr to just a few vocabs. They are secured utilities
<sinzui> gary_poster, When you see a picker, you might read "Add a member", and the smaller text says "Search" But it does not say that only teams, or only public teams will be matched
<sinzui> gary_poster, flacoste. I am trying to avoid duplicating the text in 3 many places in our code.
<gary_poster> sinzui: OK, I think I understand.  I don't quite see how the term "step_title" comes from that, but not going to worry about it.  So, no, you won't be able to add titles to just a few.  I would be tempted to add "Search" to the class
<gary_poster> expose it
<gary_poster> override it on instance
<gary_poster> and be done
<gary_poster> that seems simplest on the face of it.  is there a reason that is a bad idea?
<gary_poster> or, problematic, or whatever
<sinzui> gary_poster, step_title is indeed bad. The picker assume there are one or more steps to complete an action.
<gary_poster> I see
<sinzui> gary_poster, it inflates my diff by 150 lines, and only my vocab will use it. I could expand scope to change hoe the picker works `step_title = getattr(vocab, 'step_title', 'Search')`
<gary_poster> sinzui, but then you have to expose the attribute on security or rip off security wrappers and so on, right?
<gary_poster> if you already have the 150 line diff, I am in favor of the additional lines.  If you don't have it already, then I can see the value of more discussion
<sinzui> gary_poster, I /think/ that I need to add it to the interface so that the views can access the attr
<sinzui> gary_poster, given that the is used to vocab is build javascript, I do not want to consider removing the security wrapper
<sinzui> gary_poster, sorry about the bad typin
<sinzui> g
<sinzui> :(
<gary_poster> sinzui, add to interface: right, since that's usually what we use in the zcml to specify the proxies.  But if you have to do that, conceptually that means that you are saying the attribute will exist.  Maybe that's a niggle, but I must admit that it is how I feel about it.
 * gary_poster types poorly most of the time.  never learned how to do it properly, and now is reasonably fast the wrong way
<gary_poster> sinzui, btw, this is what I wrote you yesterday.  I have to go to do some things over lunch at 1 PM, and we should finish this conversation now, but maybe we can talk about itthis afternoon.
<gary_poster> could you take a look at https://bugs.launchpad.net/canonical-identity-provider/+bug/556680 for me and then talk with me?  I'm hoping you will have some brilliant insight, or at least that you will agree with Anthony's analysis and be able to help me come up with an actionable plan for his description of "let Launchpad refuse an account when it tries to sign in, it it notices it's related to a team Person."
<_mup_> Bug #556680: attempting to create a new account with an existing email address at login.ubuntu.com oopses after you follow the email link back to create the name/password <isd-logging-sprint> <Canonical SSO provider:Incomplete by stuartmetcalfe> <Canonical ISD QA:New> <https://launchpad.net/bugs/556680>
<sinzui> okay, I will continue on the path of exposing a new attr
<gary_poster> ok
<gary_poster> sinzui: the other option is an adapter, but that seems a bit heaviweight.  But it might have the characteristics you are looking for
<gary_poster> implementation:
<gary_poster> well, you can come up with an implementation
<gary_poster> Maybe I have too high a barrier in my mind for when an adapter makes sense these days
<gary_poster> it feels a little heavy
<sinzui> gary_poster, yes. I am looking for the minimal amount to work to close a security issue
<gary_poster> I see
<gary_poster> then
<sinzui> I happen to know there is a UI issue too
<gary_poster> You could
<gary_poster> - have the widget try to adapt the vocab to the other interface
<gary_poster> - if it can't be adapted, use "Search"
<gary_poster> - if it can be adapted, use the value
<gary_poster> - make your other vocab implement the other interface directly so it is a no-op and involves no other classes and stuff for you to write
<sinzui> gary_poster, Yes. I fear that if someone like flacoste were to review the branch, he would point out we already have an interface for that...IHugeVocabulary
<sinzui> gary_poster, as to the other bug. We do not allow users to reuse team bugs.
<gary_poster> sinzui, right, that's kind of why I thought an adpater was too heavy
<sinzui> I think that is the underlying surprise in the bug
<gary_poster> sinzui: "We do not allow users to reuse team bugs." -> "We do not allow users to reuse team emails." ?
<sinzui> yes, I mean emails, I think our email rules are broken
<gary_poster> I see
<sinzui> so I typed bug
<gary_poster> :-)
<gary_poster> So..."let Launchpad refuse an account when it tries to sign in, it it notices it's related to a team Person." -> we do this already?
<sinzui> gary_poster, I believe Lp
<sinzui> gary_poster, I believe Lp's email rules account for taken addresses. Registration did too. I think the rule was lost in translation to Django
<sinzui> gary_poster, I think the bug is making a leap issues. The bug starts by talking about ubuntu, then later I see Lp mentioned. Lp can never let someone authenticate as a team. It should never let someone authenticate if there is no preferred address
<gary_poster> sinzui: ack.  I *think* the connection is this:
<sinzui> gary_poster, but is Lp really an issue here? I know Ubuntu got the email address from Lp. IT might get emails from other systems...like forums.ubuntu.com. I think ubuntu needs to check that the email address is available, and inform the user that is it not
<gary_poster> - SSO used to check with LP for used emails, and team emails
<gary_poster> - now they don't because, as Anthony Lenton writes, "we're not receiving updates for the emailaddress table."
<gary_poster> - or maybe they do, but the data is old
<gary_poster> - so they want to remove those constraints entirely on their side AIUI
<gary_poster> - And then we need to make sure that we don't allow people to authenticate if the preferred email from SSO is a team email on our side
<gary_poster> Which seems like a really bad UI  on our side :-(
<gary_poster> which inherently comes from the two systems being separate
<gary_poster> I'm inclined to say that we treat the SSO email as a hint
<gary_poster> because when we switch to accepting any openid provider, which AIUI is in the cards, we won't care about their email unless we use it as a trusted, verified email source
<gary_poster> '
<gary_poster> sinzui, I have to go run do lunch things.  Any quick thoughts on the above, or should we try to continue later this afternoon?
<sinzui> gary_poster, This scenario is familiar. I think it is like debian package uploaders. We know their addresses and we might know they are teams. but some users what to *become* the team in Lp for certain operations. We reject the notion.
<gary_poster> I see
<gary_poster> So let me rephrase
<sinzui> gary_poster, have lunch. ping me when you are available. I think we do care about email and we can talk on mumble
<gary_poster> - if a user comes in with an openid we've seen before, we go to that user.  If there has been craziness and Launchpad now thinks that their preferred email is actually owned by a team, whatever, that's fine: they will never log in as that team, only as the user that we've already identified with that openid.
<gary_poster> - if a *new* user comes in from SSO with a preferred email that is a team's, we should reject the user creation.  That's the important thing here.
<gary_poster> OK thanks sinzui, will do
<gary_poster> I'll ping you later
 * gary_poster linches
<gary_poster> heh
<gary_poster> lunches
<benji> gary_poster: heh
<rockstar> matsubara, ping
<matsubara> hi rockstar
<rockstar> matsubara, could you join #tarmac ?
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build (180): FIXED in 3 hr 42 min: https://hudson.wedontsleep.org/job/db-devel/180/
<LPCIBot> Launchpad Patch Queue Manager: [rs=buildbot-poller] automatic merge from stable. Revisions: 12001,
<LPCIBot> 12002, 12003, 12004, 12005, 12006 included.
<gary_poster> deryck: do you object to me making a branch of lazr-js that does not depend on distribute?
<deryck> gary_poster, not at all.  I have no objections.
<gary_poster> deryck, then I would make a local release, and merge that.
<deryck> gary_poster, I was hesitant to do that not knowing the impact to landscape and not having sidnei around to ask.
<gary_poster> deryck, I would fork for now
<deryck> gary_poster, ah, right.
<deryck> gary_poster, sure, sounds like a good plan to me.
<gary_poster> ok
<deryck> gary_poster, will you submit to pqm, or just make the tarball and I will submit?
<gary_poster> deryck: so, I just want bzr branch lp:lazr-js, right?
<deryck> gary_poster, right
<gary_poster> ok.  I'll submit deryck.  I'm pretty optimistic that this is the problem.
<gary_poster> famous last words :-P
<deryck> heh
<deryck> gary_poster, I agree the log looks like this is it.
<gary_poster> cool
<deryck> gary_poster, and thanks so much for the help!
<gary_poster> deryck, I figure somebody else should help you with the pain :-/
 * gary_poster is confused, deryck.  I see "install_requires=['setuptools',]," in setup.py.  not clear on why distribute is included when you make an egg yet
<deryck> gary_poster, yeah, I don't understand either.
<mars> gary_poster, check lazr-js versions.cfg (if it has one)
<gary_poster> lazr-js's buildout and Makefile specify distribute, but that's nothing to do with when you actually build the egg
<deryck> gary_poster, well, there is the distribute_setup that is imported, if that's what you mean
<gary_poster> mars, that won't be run, but deryck, that must be it!
<gary_poster> mars, versions.cfg will affect buildout, but not when the egg is built
<mars> right
<deryck> setup.py imports distribute_setup
<gary_poster> right
<gary_poster> that might be it
<gary_poster> meh, distribute vs setuptools is yucky :-(
<deryck> and it just uses setuptools, so I don't see why we can't just use setuptools.
<deryck> right, I don't get it, gary_poster  ;)
<gary_poster> deryck: http://pastebin.ubuntu.com/538783/ ?  Do you have lazr-js review privs?  If so, want me to try to land that?
<gary_poster> deryck, argh, my first comment is backwards
<deryck> gary_poster, right.  we do prefer the local tools right?
<gary_poster> deryck, the term "local" is not clear.  I did this, but it is still confusing
<gary_poster> http://pastebin.ubuntu.com/538784/
<gary_poster> I think it is good enough though ;-)
<deryck> gary_poster, yup, r=me.  land away.
<gary_poster> ok thanks deryck
<gary_poster> deryck: meh, what's the pqm dance for this?  I can figure it out, but if you have a cheat-sheet I'll go faster :-{
<gary_poster> :-P
<deryck> gary_poster, I just do bzr pqm-submit from lazr-js and have my locations.conf set.
<gary_poster> deryck, ack
<deryck> gary_poster, I can send you my locations.conf section
<deryck> if that helps
<gary_poster> deryck: s'ok, I'll just use command line for now.  thank you
<deryck> gary_poster, np
<wallyworld> abentley: quick chat?
<gary_poster> deryck: still here?
<deryck> gary_poster, hey, yup.
<gary_poster> cool deryck.  So, lazr-js merged.  I have a 191 lazr-js sdist locally.  I have a LP branch that is merged with devel and has uses the new sdist.
<gary_poster> I don't have a review;
<gary_poster> I am not sure about whether pqm is open
<gary_poster> I guess it must be
<gary_poster> I just heard mutterings that confused me
<deryck> gary_poster, you can *not* merge with devel.  You (or I) need to merge with my branch...
<deryck> gary_poster, lazr-js is using yui 3.2 now and requires lp changes to work.
<gary_poster> deryck: oh...I'm confused.  I took devel, merged your branch, and committed.  That was when I was seeing if I could dupe the problem.
<gary_poster> deryck, then I just upped the version in versions.cfg
<deryck> ah ok
<deryck> gary_poster, I didn't realize you had merged my branch.  That should work then.
<gary_poster> deryck: you want to do the honors, since otherwise I have to get reviews and stuff, and I made things confused mby merging devel?
<gary_poster> uh-oh
<gary_poster> you have lazr-js 1.5.1DEV
<gary_poster> I'm adding lazr-js 1.5DEV-r191
<gary_poster> do you care?
<gary_poster> do we care?
<deryck> gary_poster, right, I can change to the sdist you added to download cache.  doesn't matter.
<gary_poster> :-) ok
<deryck> gary_poster, so to be clear... :-)
<deryck> gary_poster, I'll update my branch with this new version of lazr-js and land it.  right?
<deryck> update in versions.cfg, I mean
<gary_poster> deryck: right.  Here's hoping.  lazr-js = 1.5DEV-r191.
<deryck> alright.  thanks, gary_poster!
<gary_poster> deryck: I'll accept the thanks if it works, deryck ;-)
<deryck> heh, fair enough
<deryck> sent to pqm.  we'll know shortly.
<gary_poster> cool
<gary_poster> sinzui: ok, until informed otherwise, I think I'm open for a call about that email thing whenever you are (but no rush also)
<sinzui> okay
<jcsackett> leonardr: i'm trying to create an anonymous launchpadlib connection in a test, but all my attempts come back with a server based on a newInteraction while another interaction is active.
<jcsackett> any thoughts?
<leonardr> jcsackett: i think you should talk to salgado, who just did a branch for this
<jcsackett> alas, he is afk.
<jcsackett> salgado-afk: when you return, if you have time i could use some help with the above.
<wallyworld> abentley: http://people.canonical.com/~ianb/recipe-related-branches.png
<leonardr> jcsackett: he ended up adding a new helper method, if that helps
<jcsackett> leonardr: it gives me something to look for, at least. thanks!
<salgado-afk> jcsackett, yeah, I've seen that too.  had to endInteraction() before calling my helper method
<jcsackett> salgado-afk: what's your helper called?
<abentley> wallyworld: https://launchpad.net/thekraken
<salgado-afk> jcsackett, launchpadlib_for_anonymous
<jcsackett> salgado-afk: is that merged?
<salgado-afk> jcsackett, yes, as of a couple hours ago
<jcsackett> well, i'll just update then--i was writing the exact same method. :-p
<jcsackett> thanks!
<deryck> gary_poster, success!  hurrahs heard round the world!
<gary_poster> deryck: HURRAH!
<deryck> and with that, I shall away. ;)
<gary_poster> :-)
<deryck> gary_poster, thanks again for the help today!
<gary_poster> np deryck :-)
<rockstar> \o/ New lazr-js has landed!
<LPCIBot> Project devel build (266): FAILURE in 3 hr 39 min: https://hudson.wedontsleep.org/job/devel/266/
<LPCIBot> * Launchpad Patch Queue Manager: [r=jelmer][ui=none][bug=683106] Add a launchpad.View security adapter
<LPCIBot> for ISpecification so that collections of it can be exposed on
<LPCIBot> the API for anonymous users.
<LPCIBot> * Launchpad Patch Queue Manager: [r=gmb][ui=none][bug=60055] Removes unused and untested parameters
<LPCIBot> from ProjectGroup.search
<wgrant> Grah.
<wgrant> canonical-launchpad@l.c.c hates me :(
<spm> wgrant: nah. that's the filter I put on for you t'other month. was feeling bored.
#launchpad-dev 2010-12-02
<mwhudson> james_w: https://code.launchpad.net/~james-w/launchpad/export-specification-bug-links/+merge/42426 has conflicts
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build (267): FIXED in 3 hr 9 min: https://hudson.wedontsleep.org/job/devel/267/
<LPCIBot> Launchpad Patch Queue Manager: [r=deryck][ui=none][bug=514190, 672576] Update lazr-js to 1.5DEV-r191.
<james_w> mwhudson, so it does, thanks
<lifeless> === Top 10 Time Out Counts by Page ID ===
<lifeless>     Hard / Soft  Page ID
<lifeless>      387 / 6695  Archive:+index
<lifeless>      155 /  482  BugTask:+index
<lifeless>      126 /  226  Question:+index
<lifeless>       49 /  104  RootObject:+login
<lifeless>       39 /  247  Distribution:+bugs
<lifeless>       37 /   50  DistributionSourcePackage:+index
<lifeless>       33 /  283  POFile:+translate
<lifeless>       27 /   39  Distribution:+questions
<lifeless>       26 /  126  BranchSet:CollectionResource:#branches
<lifeless>       26 /    0  https://api.launchpad.net
<lifeless> wgrant: its spiking...
<poolie> lifeless: did you ever see
<poolie> /usr/lib/python2.6/atexit.py:24: DeprecationWarning: Attempt to tearDown inactive fixture.
<poolie>   func(*targs, **kargs)
<poolie> at the end of an lp testn
<StevenK> I keep seeing that with ec2 land and such like
<poolie> or LayerIsolationError: Test left new live threads: [<_Timer(Thread-1, stopped 140604130006800)>]
<lifeless> basically
<lifeless> zope.testing sucks
<lifeless> layers and reinvoking processes sucks
<lifeless> and rather than have unclean code in the core, I'm building out a stack of clean
<lifeless> the downside is some things will warn
<poolie> as long as it's not my fault :)
<lifeless> probably isn't
<lifeless> thumper: was baxters keynote good ?
<spiv> lifeless: heh, trying to decide whether you'll go to SyPy tonight? :)
<lifeless> spiv: exactly!
<lifeless> spiv: having yak shaved the day on a modem issue + chatting with poolie.
<lifeless> meh, its hot, long walk isn't what I need.
<lifeless> not with a 4:40am - nzdst - start time.
<spiv> lifeless: I feel sleepy just thinking about that!
<lifeless> yah
<thumper> lifeless: yes
<adeuring> good morning
<mrevell> Morgen
<adeuring> moin mrevell
<mrevell> :)
<bigjools> helleau
<salgado> can somebody running Maverick run a 'grep shared_buffers /etc/postgresql/8.4/*/postgresql.conf' for me?
<wgrant> salgado: shared_buffers = 28MB			# min 128kB
<salgado> wgrant, thanks.  care to also grep max_connections and tell me what you have in /proc/sys/kernel/shmmax ?
<wgrant> salgado: shmmax is 33554432
<wgrant> max_connections is 100
<salgado> if I have exactly that, postgres fails to start but tells me I have max_connections==103 rather than 100
<salgado> wtf!?
<wgrant> Hah.
<wgrant> Hmm.
<wgrant> I recall that it reserves connections for superusers.
<wgrant> salgado: Where are you getting the 103?
<salgado> wgrant, the error message:
<salgado> [2010-12-02 09:22:38 BRST] HINT:  This error usually means that PostgreSQL's request for a shared memory segment exceeded your kernel's SHMMAX parameter.  You can either reduce the request size or reconfigure the kernel with larger SHMMAX.  To reduce the request size (currently 33562624 bytes), reduce PostgreSQL's shared_buffers parameter (currently 3584) and/or its max_connections parameter (currently 103).
<salgado> and it's lying about shared_buffers; it's currently at 28M
<wgrant> salgado: #superuser_reserved_connections = 3	# (change requires restart)
<wgrant> Looks relevant.
<wgrant> But the docs say that should be included within max_connections.
<StevenK> 28MiB in bytes is 29360128, which is smaller than 33562624 ... Silly Postgres
<salgado> wgrant, changing that doesn't have any effect; postgres still fails and reports 103 max_connections
<salgado> however, changing max_connections causes it to start just fine
<salgado> reducing it from 100 to 99 is enough
<wgrant> Odd.
<salgado> indeed
<wgrant> Which arch?
<salgado> x86_64
<wgrant> I'm plain i386 here.
 * salgado creates a fresh new cluster to see what it's going to look like
<salgado> by default it comes with 24MB for shared_buffers, so it starts up just fine
<salgado> but then if I change that to 28MB it fails
<salgado> and when it fails it reports the same 103 max_connections
<wgrant> bigjools: Do we have a rebuild in our immediate future?
<deryck> Morning, all.
<bigjools> wgrant: as soon as jelmer lands his fix for not leaving uploads in limbo, yes
<wgrant> bigjools: Yay.
 * jelmer waves to wgrant
<wgrant> Morning jelmer.
<bigjools> wgrant: something is odd with domination
<wgrant> bigjools: Slow?
<wgrant> For those two tortoisehg PPAs?
<bigjools> wgrant: some PPA's publications don't seem to get scheduleddeletiondate set
<bigjools> so they appear in the getPendingPublicationPPAs every time
<wgrant> I haven't been near the condemnation stuff in a while.
<wgrant> Let's see..
<wgrant> bigjools: Do you have a specific example?
<bigjools> yeah
<bigjools> a few, hang on
<wgrant> The code looks reasonably sane.
<bigjools> mirabilos/ppa is one
<wgrant> Which pub?
<bigjools> working it out, I mention it because it appears in the list of PPAs being processed every time
<bigjools> and it's not seen any activity for ages
<wgrant> Hmm. We need to ban animated GIFs.
<wgrant> bigjools: Do you have any idea what's going on with those two tortoisehg PPAs which take >70s to publish?
<bigjools> wgrant: no not yet, I suspect it's related to this issue
<wgrant> They only have ~850 source pubs in total...
<wgrant> It could be, but it seems too small.
<wgrant> Let's see.
<wgrant> bigjools: Are you sure that mirabilos doesn't just have some orphaned binaries?
<bigjools> also the ubuntu langpack PPA is taking 20 minutes
<wgrant> Indices, or something else?
<bigjools> will look later
<wgrant> The tortoisehg sources seem to all be condemned
<bigjools> wgrant: https://dogfood.launchpad.net/~mirabilos/+archive/ppa/+sourcepub/761302/+listing-archive-extra
<bigjools> http://pastebin.ubuntu.com/538967/
<bigjools> created and deleted without getting published
<wgrant> Probably has orphaned binaries.
<wgrant> There is a bit of a race there.
<wgrant> We need something like NBS.
<wgrant> Yep. Binaries still published.
<bigjools> why are they still published?
<wgrant> It was deleted after the build uploaded, but before process-accepted ran.
<wgrant> So the upload was accepted, but the publications weren't there when the deletion happened.
<wgrant> So, in other words, custom uploads fuck the world over again :)
<bigjools> awesome
<bigjools> custom?
<wgrant> Custom uploads are the only reason that the upload doesn't create the publishing records.
<wgrant> Like it already does for sources.
<bigjools> ah right
<bigjools> I wonder if we can do it
<wgrant> We do need to fix custom copies at some point, and it is the same fix...
<bigjools> ok so for ubuntu-langpack, we've got 2 seconds doing nothing but publishing empty Sources/Packages
<bigjools> so that will be the same for all PPAs
<bigjools> what a waste
<wgrant> All pending PPAs.
<bigjools> domination is 2.5 minutes
<wgrant> on langpack?
<bigjools> yes
<wgrant> domination, or judgmenet?
<bigjools> both
<bigjools> superseding processing is the long bit
<wgrant> Hmmm.
<wgrant> A lot of that is probably the stupid cache invalidation.
<bigjools> http://pastebin.ubuntu.com/538970/
<bigjools> ummm
<bigjools> wtf
<bigjools> oh ignore that
<wgrant> It just tried to start again?
<bigjools> anyway Packages for non-existent arches take too long
<bigjools> yeah it tries to start every 5 mins
<bigjools> and given that we're taking 35-40m....
<wgrant> Non-existent arches?
<bigjools> yes, powerpc....????
<wgrant> I think we need to move to using ArchiveArch even for non-virt PPAs.
<bigjools> which it then generates for each component
<bigjools> which is probably why the whole thing is like treacle
<wgrant> Restricting that to main is trivial now.
<wgrant> + tests
<bigjools> that would be a *great* start
<wgrant> I made sure to factor that out sanely in my last publisher refactor.
<bigjools> jeez, 4 minutes on writing all those empty files!
<wgrant> Are they really empty?
<wgrant> langpacks are arch-all.
<bigjools> ah maybe
<bigjools> I assume too much
<bigjools> release files, 4 seconds
<wgrant> For a single PPA?
<bigjools> yes
<wgrant> Ouch.
<bigjools> components ....
<wgrant> True.
<bigjools> if we remove non-main we'lll get a huge win
<bigjools> I am going to fix this today
<wgrant> Archive.getComponentsForSeries
<wgrant> Not sure about tests.
<bigjools> wgrant: I'll send you a log snippet if you want to examine it more?
<bigjools> for a single run, with -v
<wgrant> bigjools: I'm about to head off to bed, but I'd be glad to look tomorrow.
<bigjools> wgrant: yeah no rush, just thought you'd be interested :)
<bigjools> wgrant: so I reckon we should disallow package deletions where there's a build in progress
<wgrant> bigjools: This situation can also arise in the normal NBS case, when a source no longer builds a binary.
<bigjools> wgrant: I suspect we have a bug for that
<bigjools> anyway, errands and food
<james_w> benji, hi, could you give me some advice on the webservice?
<james_w> benji, we want to export the "bugs" attribute that IBugLinkTarget contributes to Specification. Currently ISpecification isn't related to IBugLinkTarget, Specification just implements(IBugLinkTarget).
<james_w> benji, exporting IBugLinkTarget and its bugs attribute and adding it to the webservice interfaces isn't enough, but if I then have ISpecification subclass IBugLinkTarget everything works. Is this the right thing to do, or should I use a different approach to achieve this?
<james_w> benji, https://code.launchpad.net/~james-w/launchpad/export-specification-bug-links/+merge/42426 for more context
<benji> james_w: taking a look at the MP
<james_w> thanks
<benji> james_w: a digression: are you aware that the diff in that MP has conflict markers in it?
<james_w> benji, yup, just fixed them
<benji> james_w: I haven't forgotten about you.  Sill looking.
<james_w> thanks
<bigjools> jml, leonardr: only series are derived, not distros, in the model we're doing
<bigjools> this implies a distro of course
<leonardr> bigjools: does that mean that Natty is derived (from Ubuntu or from Maverick?), but Linaro is not? will that change anytime soon?
<bigjools> leonardr: Linaro distros would be dervied from a series in Ubuntu.  Ubuntu is dervied from Debian unstable.
<bigjools> leonardr: once a distro is derived it cannot be derived again, but you can keep syncing packages
<bigjools> we'll track in each Ubuntu series which Debian series we sync from (it can change during LTS releases)
<leonardr> bigjools: so, if you were to search for a bug in "ubuntu or its derivatives", you'd be searching across a lot of distribution *releases*, not distributions
<bigjools> leonardr: well bugs are different in that regard because they are not targeted to distroseries
<bigjools> I think for the case of bugs you can say Ubuntu or its derivatives but I've not really thought about it
<bigjools> (and mean the whole distro, I mean)
<leonardr> a bug can be targeted to a distro source package, and that's associated with a distribution
<leonardr> not a distro series
<benji> james_w: I've resorted to building your branch; Do I assume correctly that test_representation_contains_bug_links fails with an attribute error?
<james_w> benji, my branch works for me, I'm just not sure that it's the right approach. You get the attribute error?
<bigjools> leonardr: I think you'll be ok, I just wanted to ward off any incorrect assumptions in case it causes issues in the future after we finish the derived distro feature
<benji> james_w: nope, no error.  It sounds like all classes that implement ISpecification also implement IBugLinkTarget, if so, then your approach is fine.  If not, you should decompose IBugLinkTarget into two interfaces, one that ISpecification implementations can provide and one with the rest of the bits and IBugTarget would inherit from both of those but otherwise be empty
<james_w> benji, ok, thanks
<sinzui> allenap, ping
<cody-somerville> bigjools, Would it make sense to post something in topic/identica about publishing slow down? There seems to be a lot of questions about it.
<bigjools> cody-somerville: it's been slow for weeks
<cody-somerville> bigjools, No disagreement here about that! :P
<bigjools> cody-somerville: :)  I thought it was just the high load - it is, but it's other problems too.
 * cody-somerville nods.
<bigjools> anyway, I gotta dash
<cody-somerville> bigjools, Cool. FYI, expect a trivial branch from me soon.
<bigjools> cody-somerville: great!  I'll check later.
<sinzui> lifeless, I want to replace the launchpad.Append rules in lp.answers with launchpad.AnswerContact. The use of "Append" is ambiguous and when an object like a distros/project can have many ".Append" rules that conflict. I think .AnswerContact is clear, like Driver and BugSupervisor permissions
<deryck> leonardr or benji, hi.  Could one of you look at bug 680339 to see if you can offer insight in what might be happening?
<_mup_> Bug #680339: 'Entry' object has no attribute 'markAsDuplicate' <api> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/680339>
<benji> deryck: sure
<deryck> thanks benji!
<deryck> thumper, hey. you want to do a call today?
<leonardr> deryck: the thing you 'should' be doing is setting duplicate_of, but 1.0 should be allowing the old behavior for backwards compatibility
<leonardr> one possibility is that the wrong wadl is generated for 1.0
<leonardr> if the trunk wadl is presented as the wadl for 1.0, then launchpadlib will think there is no such thing as markAsDuplicate
<deryck> leonardr, what do you mean "old behavior?"  I'm not following what as changed, though I see what you mean about should be using duplicate_of based on the interface declarations now.
<lifeless> sinzui: did you have a tab complete fail there?
<leonardr> deryck: in later versions of the web service, methods like markAsDuplicate are no longer exposed as such
<sinzui> lifeless, maybe
<leonardr> instead, you set duplicate_of the way you would set any other value, and it calls markAsDuplicate behind the scenes
<allenap> sinzui: pong
<leonardr> the old behavior is that a mutator_for method also shows up as a named operation
<deryck> leonardr, got you.  and this was a web service change, nothing that we on the bugs team changed?
<leonardr> deryck: right, it is a totally general change affecting every mutator_for method
<sinzui> allenap, as I said in reply, our version of simplejson is too old. so I cannot HTML encode
<leonardr> i suspect you are getting the 'devel' version of the wadl when you should be getting the '1.0' version--that would tie it in to benji's recent hcnage
<deryck> leonardr, ok.  I recall all this for the transitionToStatus, etc., but didn't make the connection here for markAsDuplicate.
<allenap> sinzui: Ah, fair enough. It doesn't seem to be causing us bother so far so I'm not really worried.
<benji> leonardr: I don't quite follow, will you explain that some more?
<allenap> sinzui: Although I think I'll file a bug about it anyway.
<lifeless> sinzui: I don't have an opinion about lp.appent/lp.answercontact until jan 4th
<sinzui> lifeless, we have a regression. Ubuntu answer contacts cannot manage FAQs because the permission was changed from something removed to a permission that quietly conflicts with Ubuntu's rules for adding series
<lifeless> sinzui: ok
<thumper> deryck: hi, I'm on leave today :)
<thumper> deryck: in fact I'm not working a Friday until Jan :)
<deryck> well ok then.  enjoy the leave, thumper :-)  We'll chat in Jan then ;)
<thumper> deryck: so perhaps we could arrange a call earlier next week?/
<lifeless> sinzui: do you need anything from me
<sinzui> no. thanks lifeless
<deryck> thumper, sure, let's try to find a time when next week is here.
<thumper> deryck: ok
<deryck> probably easier now that our times overlap a bit more.
<thumper> deryck: yeah, it will be
<leonardr> deryck: can you look in your .launchpadlib/api.whatever.launchpad.net/cache/*1.0*wadl and see if the base of the wadl:resources tag really says "1.0"? a
<leonardr> (you can make it easier to find the file by removing /cache and running your script again
<deryck> leonardr, I can.  this was poolie who reported the bug though.
<lifeless> leonardr: the online api doesn't show markAsDuplicate in 1.0
<leonardr> lifeless: since the apidoc is generated from the wadl, that means it's not in the wadl
<leonardr> there are a couple places where that could go wrong but i would look at the change benji made recently
<leonardr> since the other place i can think of involves a change nobody would reasonably make
<maxb> *blink* wtf
<maxb> I just typoed a PPA Upload and got an email whose subject referred to a totally random other PPA
<lifeless> maxb: \o/
 * maxb searches bugs for "ppa" and starts reading the resultant list to check for similar
<maxb> bug 684450 filed. ftr
<_mup_> Bug #684450: Totally unrelated PPA referenced in subject of upload rejection email <Soyuz:New> <https://launchpad.net/bugs/684450>
<gary_poster_> sinzui: I added some notes from our conversation yesterday at the bottom of https://dev.launchpad.net/LEP/OpenIdRoadmap .  Your review, additions, and corrections would be appreciated.
<gary_poster_> See everybody Monday!
<sinzui> okay
<mwhudson> maxb: fun bug, it seems
<maxb> it's a bit mad, isn't it :-)
<maxb> (and scary)
<mwhudson> maxb: i just commented
<wgrant> maxb: If we have a PPAish upload path that doesn't map to a PPA, we deliberately pick the first PPA we can find. Probably to get PPA-specific error messages.
 * wgrant checks the code.
<wgrant>             # XXX cprov 20071212: using the first available PPA is not exactly
<wgrant>             # fine because it can confuse the code that sends rejection
<wgrant> Yay.
<wgrant>             # messages if it relies only on archive.purpose (which should be
<wgrant>             # enough).
#launchpad-dev 2010-12-03
<mwhudson> i predict this new will make jml unhappy
<wgrant> Which new?
<wgrant> The restructure?
<poolie> spm it turns out, just fyi, that qastaging has no mx record
<poolie> i will send a mail to the list
<spm> ah. k
<wgrant> flacoste: Thanks for forwarding that to the list.
<lifeless> wgrant: hey
<lifeless> wgrant: so, I don't know if you noticed, but the archive:+index timeouts are going up daily. I don't think this is random.
<wgrant> lifeless: That's odd.
<wgrant> (sorry, I've been distracted with other stuff the last couple of days)
<lifeless> doesn't deeply worry me
<lifeless> its a little concerning, and I want to be worry free on my holiday
<lifeless> 21st:      387 / 6695  Archive:+index
<wgrant> Hmmm.
<lifeless> bah
<lifeless> 1st:      387 / 6695  Archive:+index
<lifeless> 30th:      324 / 7217  Archive:+index
<lifeless> 29th:
<lifeless>      287 / 6525  Archive:+index
<wgrant> The other pageids aren't showing a similar trend?
<wgrant> That's a pretty massive escalation.
<lifeless> 29th:
<lifeless>      287 / 6525  Archive:+index
<lifeless>      170 /  390  BugTask:+index
<lifeless>       43 /  260  Distribution:+bugs
<wgrant> If it's the only pageid doing it, that's more 'deeply concerning' than 'a little concerning'.
<lifeless> 28th:
<lifeless>      260 / 6568  Archive:+index
<lifeless>      142 /  301  BugTask:+index
<lifeless>       26 /  290  Distribution:+bugs
<lifeless> 27th:
<lifeless>      115 / 5419  Archive:+index
<lifeless>       92 /  233  BugTask:+index
<lifeless>       23 /  244  Distribution:+bugs
<wgrant> Uh.
<wgrant> I think we have a general problem :)
<lifeless> possibly
<lifeless> 30th:
<lifeless>      324 / 7217  Archive:+index
<lifeless>      148 /  451  BugTask:+index
<lifeless>       36 /  150  ProjectGroupSet:CollectionResource:#project_groups
<lifeless>      19 /  296  Distribution:+bugs
<lifeless>   1st:
<lifeless>      387 / 6695  Archive:+index
<lifeless>      155 /  482  BugTask:+index
<lifeless>      126 /  226  Question:+index
<lifeless>       49 /  104  RootObject:+login
<lifeless>       39 /  247  Distribution:+bugs
<lifeless> These are all genuinely problematic pages
<lifeless> so I'm not inclined to shout fire yet
<lifeless> rather fix the pages.
<lifeless> and the worst.. well you know the worst.
<lifeless> it is, on its own, 25% of the timeouts on the site.
<lifeless> besides which
<lifeless> I know flacoste will be looking at these reports and biting his teeth in my absence ;)
<wgrant> Heh.
<wgrant> Ah, I remember why I didn't get to it last time. download-cache took forever to upgrade.
<wgrant> It's done now, so the appserver might finally start...
<lifeless> pillarname ||  963021.70 tuples/sec
<lifeless>                                   branch ||  443538.40 tuples/sec
<wgrant> WTF?
<lifeless> archive ||   41660.33 tuples/sec
<lifeless> 28th
<lifeless>                               pillarname ||  825961.90 tuples/sec
<lifeless>                                   branch ||  463744.27 tuples/sec
<lifeless>                               sl_confirm ||  133113.62 tuples/sec
<lifeless>           binarypackagepublishinghistory ||   49345.86 tuples/sec
<lifeless>                                  archive ||   33620.98 tuples/sec
<lifeless>                          shippingrequest ||   24934.80 tuples/sec
<lifeless> 1st:
<lifeless>                            pillarname ||  827065.04 tuples/sec
<lifeless>                                   branch ||  469628.31 tuples/sec
<lifeless>                               sl_confirm ||  130679.92 tuples/sec
<lifeless>           binarypackagepublishinghistory ||   53159.91 tuples/sec
<lifeless>                                  archive ||   40228.91 tuples/sec
<lifeless>                        sourcepackagename ||   24256.23 tuples/sec
<lifeless> not hugely different on a day to day basis
<lifeless> it may be that traffic spike that I called out on the list causing trouble
<wgrant> 'the list' being the internal one, or have I missed something?
<lifeless> wgrant: uhm
<lifeless> checking
<wgrant> lifeless: Aw, is that really my first? :(
<lifeless> wgrant: no :)
<mwhudson> something must be sequentially scanning branch to get numbers like that
<mwhudson> i'd love to know what
<mwhudson> well not 'must' i guess
<mwhudson> seems very likely though
<wgrant> And Julian doesn't think I'm crazy :D
<abentley> poolie: Any progress with QA for r12008?
<poolie> abentley: hi, is that the dkim and gpg bugfixes?
<abentley> poolie: yes.
<poolie> i presume yes, as that's the only one i landed
<poolie> it turns out qastaging doesn't receive mail
<poolie> it doesn't even have an mx
<poolie> that's where i'm up to
<poolie> my next thing is to send mail about perhaps turning it on
<poolie> it may take too long to do it
<poolie> i mean, it may take a while to turn it on and get it working, and that could cause rollouts to stall for too long
<lifeless> poolie: does your patch make anything worse, do you think?
<poolie> perhaps there's a better approach?
<abentley> poolie: normally, I'd suggest using staging instead, then.  But unfortunately, we are doing a full db restore in order to ensure we don't run overtime.
<lifeless> poolie: and if so, can it be disabled easily?
<poolie> i thought that qastaging would be like staging in this regard (ie reads mail, doesn't send it)
<lifeless> abentley: Will my subunit db patch be in the rollout, do you think?
<poolie> but it's not
<poolie> lifeless: it changes existing code
<abentley> lifeless: Yes, that's the plan.
<poolie> so, it's possible it will cause a regression
<lifeless> abentley: \o/
<poolie> and it's not isolated by a flag
<abentley> poolie: I think qastaging is still being configured.
<poolie> so if it does turn out to break anything that was previously working, i think we would have to roll back or reverse-cherrypick or i guess live with it until a new fix is landed :/
<wgrant> poolie: It's not on staging yet either?
<lifeless> poolie: db deploy - we can't roll those back.
<abentley> poolie: like it doesn't have import slaves or a build farm at the moment.
<abentley> wgrant: It would be, but staging is doing a full restore ATM.
<wgrant> abentley: Yay :(
<poolie> i'll send mail first at least
<poolie> should we/do we have a bug asking for real-time log feeds?
<abentley> poolie: do you mind testing it in ~24h?
<poolie> do you mean <24h or >24h?
<poolie> i mean, do you mean "will you please test it on saturday au/sydney?"
<abentley> poolie: that is what I mean.  If that's okay with you, it's probably the simplest solution.
<poolie> and you mean test it on staging?
<abentley> poolie: yes.
<poolie> that's ok with me
<abentley> poolie: Thanks.
<poolie> is there any more specific estimate when it'll be live there?
<poolie> just trying to work out if i can do it first thing saturday or something (like 20h from now)
<abentley> Chex: Do you have an estimate when the staging update will complete?
<poolie> abentley: i might also work out/document/improve testing it locally
<poolie> manually testing it locally, i mean
<poolie> but i think checking it in the dc would be really good too
<poolie> there are enough different moving parts
<abentley> poolie: Yes.  We don't normally consider it QAed until it's tested in the datacentre.  And I know our mail handling can be a bit odd, so I think testing it on staging is really worthwhile.
<poolie> me too
<poolie> thanks for following up on it
<abentley> poolie: np
<lifeless> wgrant: so, please look at that branch :)
<lifeless> ciao
<poolie> hey, can someone help me work out how to land max's loggerhead changes
<poolie> is it enough to just merge them to the relevant branch?
<poolie> spm? wallyworld?
<spm> poolie: I'm not sure at all tbh
<maxb> poolie: Don't you just PQM them onto lp:~launchpad-pqm/loggerhead/devel, and then update utilities/sourcedeps.conf?
<poolie> i think so; i'll try it later
<poolie> are one-letter project names banned?
<wgrant> Hmm, are the appservers overloaded?
<wgrant> I'm getting truncated responses again.
<wgrant> Less frequently than last time, though.
<spm> a couple are, yeah
<spm> https://lpstats.canonical.com/graphs/AppServerRequestLpnet/ wheeee
<spm> wgrant: with any luck, things should be better. looks like a buggy script was being rude.
<wgrant> spm: Thanks.
<spm> yeah the # requests graph is dropping off due to our heavy hand of stabbing application
<cody-somerville> wgrant, Do you know much about SQL indexes?
<wgrant> cody-somerville: Some. What's the issue?
<cody-somerville> wgrant, I'm wondering if an index could be used to make finding the latest of something that relates to something else faster - ie. lets say finding the latest build for a project in a CI.
<wgrant> cody-somerville: Of course.
<cody-somerville> Would just creating an index on say the finished_at field of the buildresult table be sufficient?
<cody-somerville> or would an index on the foreign key to the project and the finished_at field be better?
<wgrant> Probably the latter.
<spm> be aware that adding indexes can have major negative performance impacts. if you're looking to modify LP stuff, I'd strongly urge you to check with stub, and at least get some explain's happening to show if an index *may* help.
<wgrant> launchpad.dev seems to be just about unusable for me.
<wgrant> It has to grab several hundred YUI files on every page load...
<spm> O_O
<wgrant> Yeah, more than 400 YUI JS files are included in each page's <head>. This seems questionable.
<StevenK> wgrant: Whereas, usually, launchpad.dev is *fast*
<wgrant> StevenK: I think it would be almost acceptably fast if the JS was all in one file like it is on prod.
<StevenK> I thought the Makefile did combine-css?
<wgrant> The main issue is that only a single request can be processed at a time.
<StevenK> Oh, wait
<StevenK> I'm a bozo
<StevenK> Doesn't lazr-js do that for us?
<wgrant> The dev appserver doesn't use the combined launchpad.js, even if it has been built.
<wgrant> I wonder which config key it is.
<wgrant> Ah, devmode.
<wgrant> OMG so fast
<wgrant> There's also a disturbing number of 404 OOPSes that have just shown up in the last few minutes.
<wgrant> Like 3000 of them.
<wgrant> I think something might be wrong with the lazr-js upgrade :/
<wgrant> StevenK: Could you pastebin the query log from OOPS-1780G1265 for me?
<StevenK> No, you can bloody well wait 10 days
<wgrant> Heh.
<wgrant> But IS probably won't get around to creating my account for years :P
<StevenK> I doubt it
<StevenK> wgrant: http://pastebin.ubuntu.com/539296/
<StevenK> Oh sigh, that's the wrong bit, isn't it
<StevenK> wgrant: http://pastebin.ubuntu.com/539297/
<wgrant> StevenK: Thanks.
<lifeless> === Top 10 Time Out Counts by Page ID ===
<lifeless>     Hard / Soft  Page ID
<lifeless>       80 / 5210  Archive:+index
<lifeless>       62 /  271  BugTask:+index
<lifeless>       21 /  248  Distribution:+bugs
<lifeless>       18 /    0  BinaryPackageBuild:+retry
<wgrant> lifeless: Yes, I'm working on it right now :P
<lifeless>       15 /    3  ProjectGroup:+milestones
<lifeless>       12 /  276  Distribution:+bugtarget-portlet-bugfilters-stats
<wgrant> But the factory is being shitty.
<lifeless>       12 /   19  Archive:+copy-packages
<lifeless>       10 /  123  ProjectGroupSet:CollectionResource:#project_groups
<lifeless>        9 /  146  POFile:+translate
<lifeless>        8 /    2  Person:+bugs
<lifeless> *interesting*
<wgrant> The 300 or so missing ones?
<lifeless> cool
<lifeless> wgrant: factory fixin time?
<wgrant> lifeless: I think it's OK now.
<wgrant> I was confused for a while after I fixed it, since the baseline assertion had started failing without me noticing.
<wgrant> But I think it's sorted now.
<lifeless> kk
<adeuring> good morning
<mrevell> Morning Launchpad devvers
<gmb> So, StevenK... about that failure in db-devel. Is anyone looking at why it happened?
<gmb> jml: Given the nature of the current db-devel problem (i.e. ongoing) is there any reason not to force the build so that we don't block on it? I'll add a comment to the bug, obviously.
<wgrant> Hudson's happy. Let's use Hudson.
 * gmb can't stop thinking people are referring to mwhudson when they say "Hudson."
<wgrant> My statement may still stand, even in that case.
<gmb> Hudson, old boy, merge this branch would you? There's a good chap.
<gmb> True.
<gmb> jml: Ah, ignore me; it's obviously not all that intermittent; 3 out of the last 4 builds failed because of this, if I'm reading buildbot right.
<wgrant> How easy is it to find a timeout by its pageid?
<deryck> Morning, all.
<StevenK> gmb: I utterly forgot to look and got distracted by work
<StevenK> gmb: But if I'm reading your last statement rightly, it's not really not my fault, it's buildbot being crap?
<wgrant> When did it start?
<gmb> StevenK: Yeah, it looks like it's an annoying but ongoing problem. I haven't had time to dig further.
<wgrant> Around the time of the databasefixture landing?
<gmb> wgrant: The bug was filed on the 29th, FWIW: https://bugs.edge.launchpad.net/launchpad-foundations/+bug/682861
<_mup_> Bug #682861: db-devel librarian hung with '[Errno socket error] timed out' <build-infrastructure> <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/682861>
<wgrant> That's, er, slightly suspicious timing.
<wgrant> The databasefixture stuff landed on the 28th.
 * StevenK is going to bang the Hudson drum in the new year
<StevenK> I might actually *gasp* do some work on it while I'm off next week
<wgrant> Damn, then who am I going to harass during my last week without ~launchpad superpowers?
<StevenK> wgrant: Go play in the traffic
<wgrant> :(
<bigjools> wgrant: well done on passing
<wgrant> bigjools: Thanks!
<bigjools> did you get graded?
<wgrant> Overall for the degree? No.
<bigjools> when do you know?  or don't you?
<wgrant> I'm not sure if they do it... but let's see what it would be.
<wgrant> Oh good, H1.
 * bigjools has no idea what that means
<wgrant> First-Class Honours, ie >=80%.
<bigjools> I did get a first class honours in hindsight from the university of life
<wgrant> Haha.
<bigjools> well done, anyway!
<wgrant> Thanks.
<wgrant> It is great to be done.
<leonardr> jml, i sent email about this, but what do you mean when you refer to the set-based design we discussed 3 years ago?
<wgrant> deryck: Hi.
<deryck> hi wgrant
<wgrant> deryck: I'm seeing around 450 YUI JS files included in <head> when it devmode. It makes page loads sort of slow. Is this likely to be related to your recent lazr-js upgrade?
<deryck> wgrant, this is definitely related.  After I do qa today on the production changes, then I'll turn my attention to making that better.
<wgrant> deryck: Thanks.
<deryck> I should have sent mail about that already, sorry.
<wgrant> Turning of devmode makes it fast again, since it just uses launchpad.js.
<wgrant> I cannot type tonight, apparently.
<deryck> heh
<deryck> yeah, that is definitely a work around if you're not hacking on js itself.
<gmb> flacoste: with db-devel being broken because of this ongoing but not-all-the-time timeout problem, does it make sense to force the build so that people can land things in the interim (and who knows, maybe it'll work this time)
<gmb> ?
 * gmb JFDoesIt.
<mars> flacoste, ping
<mars> gmb, +1 on forcing the build, +1000 on finding a possible clue as to its cause
<gmb> mars: Yeah. I'll take a look later if I get the chance.
<wgrant> mars: It's probably the librarian fixture changes.
<wgrant> In db-devel r10013
<bigjools> wgrant: awake enough for a very quick chat about removing non-PPA pockets from their indexes?
<wgrant> bigjools: 'course.
<wgrant> bigjools: They shouldn't be there, though...
<bigjools> wgrant: I think it's an easy case of fixing the loops in C_writeIndexes and  D_writeReleaseFiles
<bigjools> I was going to abstract the pockets to a call on IArchive
<wgrant> bigjools: They're not actually ever written at the moment.
<wgrant> Because they're never dirty.
<bigjools> right
<bigjools> but it loops them nonetheless
<wgrant> We should skip them in A and B as well.
<bigjools> ah yes
<wgrant> bigjools: So, I'd just replace all the 'for pocket in PackagePublishingPocket.items' calls with 'for pocket in self.archive.getPocketsForSeries(distroseries)'
<bigjools> wgrant: why by series?
<wgrant> bigjools: Because you never know what Ubuntu will want to do next. But I guess for now we can skip that bit.
<wgrant> Since it'll only be used in a couple of places.
<wgrant> Anyway, it is getting late.
<flacoste> hi mars
<bigjools> aww jml you painted over my green bikeshed!
<bigjools> foundations guys: I think I need to add lazr.enum._enum.EnumItems in lp_sitecustomize.py so that it has no security proxy, any thoughts?
* flacoste changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 2 of 10.12 | PQM is open | firefighting: db-devel broken | https:/â/âdev.launchpad.net/ | Get the code: https:/â/âdev.launchpad.net/âGetting
<mars> flacoste, pong
<flacoste> mars: what can I do for you?
<mars> bigjools, I think Gary would be best positioned to answer that question, but unfortunately he is out today
<bigjools> mars: yeah, he's blowing out a lot of candles
<mars> :)
<bigjools> I can't talk, I've got the same number in 3 months...
<jcsackett> how often is the data on qastaging restored?
 * bigjools shakes fist at zope testrunner
<bigjools> jcsackett: one a week
<bigjools> once*
<jcsackett> thanks.
<bigjools> so, if I use an argument to bin/test like "-t soyuz -t !windmill" it ends up running every single test except windmill ones rather than all the soyuz  tests except windmill.
<jelmer> -t is unfortunately || rather than &&
 * jelmer often just uses a file with a list of tests to run and calls ./bin/test --load-list=FILE
<bigjools> jelmer: yeah, && would be more useful
<bac> jtv: are you disappearing soon?
<jtv> bac: from the channel, yes.
<bac> jtv: rats.  i'd like to work with you on a problem but i realize it is really late for you.
<jtv> bac: oh, you're back on the West Pole already?
<bac> yep, for a week
<jtv> A week.
<jtv> bac: but hey, see if you can grab my attention.  What is it you wanted to work on?
* abentley changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 2 of 10.12 | PQM in RC mode | firefighting: - | Get the code: https:/â/âdev.launchpad.net/âGetting
* abentley changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 3 of 10.12 | PQM in RC mode | firefighting: - | Get the code: https:/â/âdev.launchpad.net/âGetting
<cody-somerville> libmozjs2s from LP-PPA-Launchpad is trying to uninstall my xulrunner :(
<deryck> sinzui, ping
<mkanat> Is there an icon that would mean something like "give me the raw content of this" in Launchpad that I should re-use for loggerhead?
<maxb> Not that I have seen
<sinzui> hi deryck
<maxb> Also you'd have to jump through lots of licensing hoops if you wanted to take an icon from LP into Loggerhead
<deryck> hi sinzui.  Is the calendar widget used on registry pages in js or python?  I can't find tests or a js file for it.
<sinzui> deryck, the calendar is yui2. I think it was moved to lp.app
<sinzui> deryck, we want the widget to be used every time there is a date or datetime field, but we do not seem to know how to ensure the js loads because the field is present
<deryck> sinzui, ok, cool.  That helps then.  The lazr-js upgrade has borked the style of this widget, but I suspect it's an easy fix via class names.
<mkanat> maxb: Why, are the LP icons not GPL?
<maxb> mkanat: No, they are copyright Canonical, restricted rights for use in development only
<mkanat> maxb: Oh, okay.
<sinzui> deryck, I thought there was a windmill test for the calendar, but I do not see it
<deryck> sinzui, yeah, it doesn't appear to have any test coverage, as far as I can see.
<deryck> but no biggie.  I'll fix it all up next week.
<sinzui> I there a yui3 calendar widget yet?
<deryck> hmmm, not sure.
<deryck> I can look into that a do a replacement.
<benji> Is there a design document (or something similar) that describes the librarian?
<lifeless> yes
<lifeless> librarian.txt, docstrings.
<benji> k, thanks
<lifeless> jkakar:
<lifeless> jkakar: hey
<jkakar> Hi! :)
<lifeless> jkakar: wondered what you think of my object graph to model query proposal.
<lifeless> I'm sure its not new, but none of the python ORM's I've seen do it ...
<jkakar> lifeless: It's still marked as "read" in my mail... I haven't made the time to read through it properly. :(
<jkakar> lifeless: I want to, just been pressed with lots of other things.
<jkakar> lifeless: Will try to get to it this weekend.
<wgrant> flacoste: Do you really think that Soyuz can be simplified such that domain expertise is no longer required?
<flacoste> hi william!
<jcsackett> the whole pqm/release thing now largely has to do with stuff in db-devel, right? things done in devel will still roll out on a fairly regular interval, so RCs are less important, aren't they? or have i completely misunderstood something?
<wgrant> Heh, morning.
<flacoste> wgrant: i do believe that we can make it so that many changes can be done without requiring vetting by somebody who "understands the whole system" and without the risk of breaking everything
<flacoste> wgrant: but people will have to understand the basics of package building and archive management
<jelmer> jcsackett: No, that's right.
<flacoste> but i don't think it requires a year of dedicated Soyuz hacking to get that
<jcsackett> jelmer: thanks.
 * jcsackett relaxes a touch.
<flacoste> jcsackett: you should still make sure that all your revisions before 12019 are qa-okl
<flacoste> or get a rc to fix any qa-bad in that range
<flacoste> jcsackett: but yeah, we'll continue nodowntime deployment afterwards from devel
<jcsackett> flacoste: i may need to do that. it's not a qa-bad, per se, but a qa-ok that's an incomplete solution to an inprogress bug. but i think i'm about to have the full solution, which would be nice to release all together.
<flacoste> jcsackett: right, you can negotiate that with abentley
<jcsackett> flacoste: right. first though i have to know if have anything to negotiate. was just checking on time tables. :-P
<abentley> jcsackett: are any of your changes going to introduce regressions?
<jcsackett> in the thing i'm working on?
<jcsackett> abentley: in the thing i'm working on, no. in the branch that's qa-oked but we've discovered is incomplete, it wouldn't appear so. nothing is worse than it was, or we'd qa-bad it.
<lifeless> wgrant: required? no. God Like Knowledge Of Every Line Of Code - that can be fixed.
<abentley> jcsackett: Okay, so it won't need to block deployment.
<lifeless> wgrant: for instance, concurrent uploads to the uploader, what unique keys are :)
<jcsackett> abentley: yeah, the branch already landed isn't a blocker.
<abentley> jcsackett: We can give it an RC if you think it's necessary to deploy your changes at the first available opportunity.
<wgrant> lifeless: It's not quite that simple to fix, I'm afraid.
<abentley> jcsackett: That would be Wednesday next week at 10:00 UTC.
<jcsackett> abentley: dig.
<lifeless> wgrant: is it worth fixing?
<lifeless> wgrant: (rhetorical)
<abentley> jcsackett: Otherwise, land it normally, and it can be part of the first nodowntime deployment of the next cycle.
<jcsackett> abentley: okay. thanks.
<lifeless> wgrant: I think its worth fixing because the current set of folk that can work on soyuz is the intersect of 'knows zope well', 'knows twisted well', 'knows debian packaging well', 'knows debian archive mgmt well'
<lifeless> wgrant: ... very small set.
<wgrant> lifeless: Sure, that sucks.
<lifeless> every one of those that we can relax by better layering, clearer code, safer system, the larger the set of folk that can contribute sensible.
<wgrant> But the new model means that the last two sets will vanish.
<lifeless> uhm
<wgrant> And after that it's just a matter of time until someone mistakenly qa-ok's something and tens of millions of machines notices.
<lifeless> wgrant: I think you're mistaken.
<abentley> rockstar: chat?
<lifeless> wgrant: right now, the review policy does not require domain reviews of changes.
<lifeless> wgrant: the reorg asks folk to take on the challenge of a wider view of the system. It doesn't expect them to suddenly know things they didn't.
<lifeless> wgrant: learning will be involved.
<wgrant> lifeless: No, but few outside the team make Soyuz changes. And I know that at least I review everything Soyuzy.
<lifeless> wgrant: what will stop you doing that?
<wgrant> lifeless: Trusting everyone to make changes to such a fragile, critical piece of Launchpad. Probably starting with myself.
<lifeless> wgrant: do you mean 'there will be so many soyuz patches wgrant cannot look at them all' ?
<wgrant> lifeless: No. I mean what I said. When I can be reasonably confident that people aren't able to accidentally break things, I will have a reason to stop watching.
<wgrant> Soyuz is fragile. Some of this fragility is implied by the domain. And the consequences of a mistake can be immense.
<rockstar> abentley, sure.  Mumble?
<abentley> rockstar: sure.
<lifeless> wgrant: you're talking about breaking the primary archive in some fashion, right ?
<wgrant> lifeless: That's the worst case, but sure.
<lifeless> so, some of the fragility is due to apt disk layout, sure.
<flacoste> wgrant: i really don't get 'fragility implied by the domain'
<lifeless> but a lot of it is due to our code.
<flacoste> i think that's my main point of disagreement
<flacoste> what is so fragile in archive management
<wgrant> lifeless: A lot of the code is atrocious, sure.
<lifeless> wgrant: I'll buy 'expected cost of a qa failure is huge'
<wgrant> I'll also throw in "QAing completely is just about impossible"
<lifeless> wgrant: but I don't see a reason for a qa failure to be more or less likely simply because its archive mgmt
<flacoste> the fact that we relied on poor perl code for stuff isn't something implied by the domain
<lifeless> wgrant: builds are already more-than-soyuz-team today.
<wgrant> lifeless: Um, not really, but OK.
<lifeless> wgrant: buildd manager isn't, but it breaking isn't catastrophic (compared to the main web UI going down, APIs going down, or codehosting going down)
<lifeless> wgrant: its not less catastrophic either.
<wgrant> lifeless: buildd-manager problems are always handled by Soyuz.
<wgrant> Still.
<lifeless> yes, I know
<wgrant> Despite it in theory being shared for 11 months now.
<lifeless> well
<lifeless> buildd manager wasn't ever shared yet AFAIK
<lifeless> slaves are
<wgrant> Hm?
<lifeless> and they get worked on
 * jelmer waves
<jelmer> flacoste: I don't think the fragility is implied by the domain. Even if it is, it's certainly not why things are fragile at the moment.
<wgrant> Some of it is. A lot of it isn't.
<flacoste> jelmer: thanks! i was starting to think i was blind to something
<wgrant> Yes, Soyuz is far worse than it needs to be.
<jelmer> flacoste: I do agree with Julian's concerns about Soyuz' fragility in the shorter term though.
<jelmer> perhaps I should follow up on the ml
<cody-somerville> Sounds like there is an interesting discussion going on somewhere that I haven't seen yet.
<wgrant> On canonical-launchpad@l.c.c
<flacoste> cody-somerville: 'Announcing Launchpad Squads' on canonical-launchpad mailing list
<lifeless> flacoste: perhaps its time to take it to lp-dev ?
<flacoste> perhaps
<flacoste> or start a different thread there
<lifeless> the latter is what I meant
<elmo> "Grave mistakes caused by soyuz rookies setting off booby traps is a future problem that might not ever actually happen.", i.e. "this risk may not happen, therefore it does not exist"
<elmo> this thread is really beginnning to depress me
<cody-somerville> :-(
<wgrant> lifeless: On a lighter note, my Archive:+index and Archive:+packages fix is reviewed and tested.
<lifeless> wgrant: sweet.
<lifeless> wgrant: land it!
<wgrant> lifeless: Aren't we still r-c?
 * lifeless twitches
<wgrant> The topic suggests it, although db-stable was merged recently.
<lifeless> yes
<lifeless> we need to qa *past* the db merge
<lifeless> and then we can unlock
<wgrant> Ah.
<lifeless> thats the critical section
<lifeless> currently waiting for buildbot
<flacoste> elmo: i think i understand your concerns, iow, fuck ups in Soyuz can have a high-impact, we shouldn't thread this lightly
<flacoste> elmo: it does mean that we should be careful when reviewing changes and QA there, but not sure that keeping the people who can make changes there is a solution
<flacoste> elmo: it didn't really got us far now
<EdwinGrubbs> lifeless: if a page explicitely gets the master store, doesn't it mean that every *Set class will load objects with the wrong store since they usually use IStore to get it? It seems like passing a store object into methods on *Set classes is a an inefficient way to handle this.
<lifeless> EdwinGrubbs: there may be many cases where its a problem.
<lifeless> EdwinGrubbs: my primary point was you're adding a new instance of the problem :)
<lifeless> EdwinGrubbs: we had timeout/performance issues on some pages due to this with LibraryFileAlias, for instance.
<lifeless> EdwinGrubbs: I don't mind if you want to search for a cool elegant way to handle it, but supplying the store is a fairly obvious and will-work approach.
<lifeless> EdwinGrubbs: essentially, IStore should be used as a matter of last resort, because its a global lookup rather than related-state lookup.
<wgrant> The Sets traditionally use I(Master|Slave)?Store.
<wgrant> Since they don't have an object on which they can call Store.of.
<lifeless> right
<lifeless> its a bug
<lifeless> the whole set structure is a bit buggy
<lifeless> they are exposed as singletones
<lifeless> s/es/s/
<wgrant> Right.
<lifeless> but correct behaviour for them depends on non-global state
<wgrant> I'm hoping that your next piece of work will make this a bit more fixable.
<wgrant> cody-somerville: What do I have to do to convince you not to use PPAs for obsolete series?
<lifeless> wgrant: customers which we have contracts with need them.
<wgrant> Well that's stupid.
<wgrant> s/stupid/insecure/, if you must.
<lifeless> s/.*/reality/
<wgrant> Shh.
<lifeless> insecure?
<wgrant> Or does OEM maintain security updates for obsolete series?
<lifeless> a per device thing, I believe
<lifeless> (so yes)
<wgrant> :(
<jpds> Ew.
<lifeless> jpds: well, its essentially a LTS for a specific customer, AIUI. cody-somerville will know all the gory details.
<lifeless> point is, *some* PPA's today need to keep publishing non-regularsupported suites.
#launchpad-dev 2010-12-04
<cody-somerville> wgrant, Although it is unfortunate for several reasons that we have to support obsolete series, I'm curious why you care as it isn't you that has to do the work to support the obsolete release. :P
<wgrant> cody-somerville: Publisher performance.
<wgrant> cody-somerville: We can save a lot of time if we have to publish 6 releases, not 14.
<wgrant> We should probably be querying for dirty series anyway, but it's a whole lot easier to just skip obsolete ones.
<cody-somerville> wgrant, sounds like a scaling issue.
<wgrant> cody-somerville: It is.
<wgrant> But I'd also like to reject uploads to obsolete series.
<lifeless> we has a lot of them
<lifeless> I can haz perf problems
<wgrant> I now haz full publisher log, so I can actually see where the problems lie.
<lifeless> https://devpad.canonical.com/~lpqateam/qa_reports/deployment-stable.html is updated
<wgrant> Need simple eager loading :(
<lifeless> wgrant: well, I've specced it now.,.. you can write if you like :)
<wgrant> Hmm.
<wgrant> I'd really like to see how much faster the PPA publisher is if we turn off cache flushing.
<lifeless> elmo: have you seen http://blog.benstrong.com/2010/11/google-and-microsoft-cheat-on-slow.html ?
<elmo> flacoste: I wasn't arguing against the desiloization in general or arguing for soyuz retaining specific people - as I think I've said to you and certainly to lifeless before now, I'm all for desiloization, I think it's important.  what I was reacting to was the idea that soyuz is no different to e.g. bugs or jml's "be bold, be confident, do what the fuck you want" (I may be paraphrasing) and wanted to reinforce the idea that soyuz is not like the rest of L
<elmo> lifeless: I hadn't - that's interesting
<lifeless> elmo: your long statement cut off at 'the rest of L'
<elmo> lala
<elmo> P - it a)  doesn't have the safety net of the browser in terms of impact on systems and b) can impact orders of magnitudes more  systems in an automated way compared to other LP systems that also don't have the browser safety net (e.g. code)
<elmo> flacoste: ^--
<elmo> man, now i'm going to be reading this tcpm list for hours
<lifeless> elmo: sorry!
<lifeless> === Top 10 Time Out Counts by Page ID ===
<lifeless>     Hard / Soft  Page ID
<lifeless>      168 / 6026  Archive:+index
<lifeless>       94 /  256  POFile:+translate
<lifeless>       90 /  269  BugTask:+index
<lifeless>       19 /  268  Distribution:+bugs
<lifeless>       16 /  118  ProjectGroupSet:CollectionResource:#project_groups
<lifeless>       15 /  282  Distribution:+bugtarget-portlet-bugfilters-stats
<lifeless>       15 /    2  ProjectGroup:+milestones
<lifeless>       10 /   21  DistroSeries:+queue
<lifeless>       10 /   19  DistroSeriesLanguage:+index
<lifeless>        8 /    0  ProductSeries:+edit
<jcsackett> lifeless, what exactly is the difference between a hard and a soft timeout? is it where it ran up against our time limits vs when the server actually gives up and dies?
<jcsackett> that may be a stupid question, but i just realized i don't actually know.
<lifeless> soft timeouts are requests that complete ok, longer than X where X is configured in lp-production-configs
<lifeless> hard timeouts are requests that trip the timeout code (grep for RequestExpired) and get interrupted
<lifeless> the default hard timeout is currently 15 seconds
<lifeless> and some page ids have higher ones, while we recover from pg8.4
<wgrant> lifeless: Any chance you could throw one of the recent DistroSeries:+queue timeout query logs my way?
<lifeless> wgrant: sorry no
<lifeless> wgrant: the grouping in the report means that high frequency things crowd out the details for lower ones
<lifeless> wgrant: so I don't have a +queue OOPS that is a timeout.
<lifeless> wgrant:  https://bugs.launchpad.net/soyuz/+bug/276950 is the bug
<_mup_> Bug #276950: DistroSeries:+queue Timeout accepting many packages queue page <queue-page> <timeout> <Soyuz:Triaged> <https://launchpad.net/bugs/276950>
<lifeless> wgrant: and it has a one month old bug
<lifeless> one month old OOPS
<wgrant> lifeless: Ah, OK.
<wgrant> Thanks.
<lifeless> I'll pull out data from it in a sec
<lifeless> wgrant: updated
<wgrant> lifeless: Thanks.
<lifeless> wow
<wgrant> Hm?
<lifeless> https://bugs.launchpad.net/soyuz/+bug/276950/comments/12
<_mup_> Bug #276950: DistroSeries:+queue Timeout accepting many packages queue page <queue-page> <timeout> <Soyuz:Triaged> <https://launchpad.net/bugs/276950>
<wgrant> Uh.
<wgrant> Why didn't it use packageupload__distroseries__status__idx?
<wgrant> Or does it really need a full (archive, distroseries, status) index?
<wgrant> (that should be the index anyway, but I still think it should have been able to get some benefit from the existing one)
<lifeless> it may be outdated statistics
<lifeless> its the order by
<lifeless> you need an index with the order key, to use the index
<lifeless> SELECT PackageUpload.archive, PackageUpload.changesfile, PackageUpload.date_created, PackageUpload.distroseries, PackageUpload.id, PackageUpload.pocket, PackageUpload.signing_key, PackageUpload.status FROM PackageUpload WHERE packageupload.distroseries = 103 AND  packageupload.archive IN (1, 534) AND packageupload.status IN (0) LIMIT 31 OFFSET 0;
<wgrant> Interesting.
<lifeless> (1 row)
<wgrant> Despite there being only one row.
<lifeless> Time: 1.601 ms
<lifeless> standard sql rules
<wgrant> So I guess we want an (archive, distroseries, status, id) index?
<lifeless> -id
<lifeless> but yeah
<lifeless>  Limit  (cost=0.00..90.66 rows=31 width=36) (actual time=58.704..58.709 rows=1 loops=1)
<lifeless>    ->  Index Scan using packageupload__distroseries__status__idx on packageupload  (cost=0.00..8946.13 rows=3059 width=36) (actual time=58.701..58.704 rows=1 loops=1)
<lifeless>          Index Cond: ((distroseries = 103) AND (status = 0))
<lifeless>          Filter: (archive = ANY ('{1,534}'::integer[]))
<lifeless> without the order by
<wgrant> That's a little better.
<lifeless> 1.5ms hot :)
<lifeless> anyhow
<lifeless> theres another index that I created, you do archive, distroseries, status, id DESC
<lifeless> or something like that
<wgrant> That's not the normal timeout, but it'll do for now.
<lifeless> we may be able to do
<lifeless> shaving 1.5s of normal is WIN
<lifeless> packageupload__distroseries__status__idx" btree (distroseries, status)
<lifeless> packageupload__distroseries__status__idx" btree (distroseries, status, id DESC)
<lifeless> may be enough
<lifeless> experiment in your local machine
<lifeless> disable all scans
<wgrant> How do I disable scans?
<lifeless> http://www.postgresql.org/docs/8.4/interactive/runtime-config-query.html#RUNTIME-CONFIG-QUERY-ENABLE
<wgrant> Thanks.
<lifeless> so its set
<lifeless> set enable_seqscan 0
<lifeless> wgrant: did that help?
<lifeless> wgrant: also see http://www.postgresql.org/docs/8.4/static/sql-createindex.html
<lifeless> look for ORDER BY
<wgrant> lifeless: Debugging Soyuz DB corruption issues at the moment.
<lifeless> \o/
<lifeless> looking at this
<lifeless> I'd do
<lifeless> distroseries, status, archive, id
<lifeless> as the index, replace status__idx
<wgrant> lifeless: I've tried a few combinations, but it will always use an explicit sort step. If I disable explicit sort steps, it reverts to using distroreleasequeue_pkey
<wgrant> Regardless of sort direction.
<wgrant> Odd.
<wgrant> Oh.
<wgrant> There we go.
<wgrant> .. no.
<wgrant> It will not use the index.
<wgrant> Can I deprioritise distroreleasequeue_pkey somehow?
<lifeless> well
<lifeless> the stats will play into what it uses
<wgrant> I guess I really need to try on real data.
<wgrant> Right.
<lifeless> hey stub
<lifeless> reckon you create an index on qastaging to test a query improvement for wgrant and I ?
<lifeless> create index packageupload__distroseries__status2__idx on packageupload(distroseries, status, archive, id);
<lifeless> this is the query 'SELECT PackageUpload.archive, PackageUpload.changesfile, PackageUpload.date_created, PackageUpload.distroseries, PackageUpload.id, PackageUpload.pocket, PackageUpload.signing_key, PackageUpload.status FROM PackageUpload WHERE packageupload.distroseries = 103 AND  packageupload.archive IN (1, 534) AND packageupload.status IN (0) ORDER BY packageupload.id desc LIMIT 31 OFFSET 0;
<wgrant> sinzui: The bugjam doesn't start for another 9 days. You don't need to start drowning us already :P
<stub> lifeless: yo
<stub> Currently that is a crap query (28 sec)
<wgrant> It wasn't always :(
<stub> yer - surprised it isn't using the distroseries,status index
<wgrant> Exactly.
<wgrant> That should really be archive, distroseries, status nowadays, but still.
<stub> We are missing an index on archive
<stub> (different problem)
<wgrant> Oh, true, I misread the fkey as an index.
<stub> Just adding an archive, status index brings it down to 1.6 seconds
<stub> Oh... must be warm cache now. It isn't using that index ;)
<lifeless> stub: it needs an index on id
<lifeless> stub: because of the order byu
<stub> yeah - warming up to that
<lifeless> stub: thus the 4-element one I'm proposing
<stub> Its always wanting to use the primary key index
<wgrant> That's what I found locally.
<wgrant> I hoped real data would work better :(
<stub> Define 'better'. PG is sometimes smarter than we are when picking plans. It might be correct that this approach is faster with the real data
<stub> Ok... id needs to be first
<lifeless> stub: ?!
<wgrant> ... what?
<lifeless> stub: win
<stub> Limit  (cost=0.00..518.75 rows=31 width=36) (actual time=0.061..92.017 rows=1 loops=1)
<stub>    ->  Index Scan Backward using pu_test7 on packageupload  (cost=0.00..52209.53 rows=3120 width=36) (actual time=0.059..92.014 rows=1 loops=1)
<stub>          Index Cond: ((distroseries = 103) AND (status = 0))
<stub>          Filter: (archive = ANY ('{1,534}'::integer[]))
<stub>  Total runtime: 92.065 ms
<stub> (5 rows)
<wgrant> How does that make sense?
<lifeless> stub: whats the defn of that index?
<stub> It doesn't have to materialize the whole thing before truncating 'first 30 items'
<stub> It can pull suitably filtered rows straight off the index in order.
<wgrant> Ahh.
<lifeless> stub: thats still way worse than if you drop the ORDER BY :(
<stub> create index pu_test7 on packageupload(id, distroseries, status, archive);
<lifeless> stub: 50* worse
<stub> Looking at that, it doesn't bother using the index for archive
<lifeless> what about
<stub> (not surprising if those archives are big ones... it might get used for small archives so we can leave that column in the index)
<wgrant> Given the distribution that might not be unreasonable.
<lifeless> create index packageupload__distroseries__status2__idx on packageupload(distroseries, status, archive, id desc);
<stub> lifeless: tried that one
<lifeless> stub: with the DESC ?
<stub>     "pu_test1" btree (archive, status)
<stub>     "pu_test2" btree (archive, status, id)
<stub>     "pu_test3" btree (distroseries, archive, status, id)
<stub>     "pu_test4" btree (archive, status, id DESC)
<stub>     "pu_test5" btree (distroseries, status, archive, id)
<stub>     "pu_test6" btree (distroseries, status, archive, id DESC)
<stub>     "pu_test7" btree (id, distroseries, status, archive)
<lifeless> kk
<stub> DESC shouldn't make a difference in this case - it can use the index backwards just as easily. If you were ordering by 'archive, id DESC' that would be different.
<lifeless> yeah, theory n prac though
<stub> Yup. I always pick my theory after empirical results are in ;)
<stub> I sound smarter that way.
<stub> Do we need the index for this rollout?
<wgrant> It's not a recent regression.
<wgrant> So I'd imagine not.
<stub> I can apply the patch live if we want to skip getting a patch on the production branch
<stub> Tomorrow though - already got some indexes building on production for translations
<lifeless> no panic
<lifeless> but we should figure it out :)
<stub> If you put in a db patch, please add the index on archive too... I don't like having ON DELETE CASCADE foreign key reference without the index to support that quickly.
<stub> Shall I drop these indexes on qastaging now so we are qaing what we think we are qaing?
<lifeless> stub: please
<lifeless> stub: I don't know that we've got a good enough index yet.
<lifeless> stub: for the archive one, tell wgrant what it should be, he can add a patch for that for your happiness :)
<stub> CREATE INDEX packageupload__archive__idx ON PackageUpload(archive);
<stub> I can't see how to make that index better
<stub> and 91ms is pretty good
<lifeless> unordered its 1.6ms :)
<stub> Yes, but that is pulling rows in disk order rather than randomly - far fewer pages to pull up off disk.
<stub> One block vs 30 odd.
<stub> I can make an index that is used that runs slower ;-)
<lifeless> oh?
<stub> yer - you want the terms that filter the most rows on the left.
<lifeless> bbiafew hours
<stub> But with timings this low, we are dealing with shape of the data for those parameters
<stub> Ditto
<sinzui> wgrant, sorry. This is not about the bugjam. I want every registry bug tagged with something I can find before the bugs are moved to launchpad.
<jelmer> 'morning
<lifeless> sinzui: I think its great that you're tagging separately from 'closing'... I'm expecting the bugjam to close bugs it shouldn't really.
<lifeless> theres not much nuance in the message :)
<sinzui> lifeless, I filter all bug mail from me to junk :)
<lifeless> sinzui: I read all bugmail on all lp projects :)
<lifeless> + all bzr projects
<lifeless> + my own things
<sinzui> I think bug mail is easy to read with the right filers. Maybe we should start a project/wiki of bug mail filters
<lifeless> might be interesting
<lifeless> I'm reading it without filters atm
<sinzui> wow. There are toddlers who are faster at reading than myself. I cheat to keep up
<lifeless> I've got fairly good at short circuiting things as I go
<lifeless> sinzui: what do you think of alexanders comments on bug 525235
<_mup_> Bug #525235: milestone page: expected release said XX hours ago, when scheduled for today <releases> <Launchpad Registry:Triaged> <https://launchpad.net/bugs/525235>
<sinzui> I have put some thought to that
<sinzui> we use a date field, not a date time, but our formatter assumes it is a datetime
<sinzui> I think we want to assume the date + 23:59:59.9 seconds so all the UI cases look right
<sinzui> lifeless,  the case of "today" is like the case of "minutes ago" our formatters are ridiculously precise (2 seconds ago).
<lifeless> right
<lifeless> another bug is open about that :)
<sinzui> I think we are tracking that issue in launchpad-web tagged with tales
<lifeless> \o/ one bug trackker
<sinzui> I think an engineer could close all the tales bugs in 2 days
<lifeless> that would be awesome
<lifeless> then we could delete the project :)
<lifeless> sinzui: what about bug 212439 - application vs acceptance ?
<_mup_> Bug #212439: Incorrect date for "member since" on +members, round 2 <Launchpad Registry:Won't Fix> <https://launchpad.net/bugs/212439>
<lifeless> sinzui: it seems reasonable to me that until you're accepted, you're not a member.
<sinzui> I think there are really two issues here. One is an effort to fix the existing data with data that does not exist. We really do not know the answer to when you were accepted
<lifeless> sinzui: poll support is being yanked, right?
<lifeless> sinzui: so, I accept that we can't retrospectively fix, but when someone goes through the process now, do we show 'accepted' or 'applied' in the 'member since' field ?
<sinzui> As to recording that information, we could...but Launchpad is not hosting the team. The correct were is register a team. Lp could learns of the team, and the membership years too lae
<lifeless> sinzui: if we show applied, I think its a legit bug - we should update the application date when they are accepted.
<sinzui> Polls will be removed. Someone will get to close twenty bugs soon
<lifeless> sinzui: you've already started :)
<lifeless> sinzui: so, https://edge.launchpad.net/~pythonistas/+members *is* an lp team
<lifeless> sinzui: this leaves me confused
<sinzui> the member since/accepted issue assumes the team only exists in Lp. I can be a member of a team before Lp learns of it and before I decide it is now time to register with Lp and join the team
<lifeless> sinzui: right, but thats a bit meta
<lifeless> I mean, its addressable by saying 'member of (in launchpad)'
<lifeless> as the column heading
<sinzui> The admin will need a new field to track that data....
<lifeless> sinzui: why?
<sinzui> note that we do not have a date when a team was registered.
<lifeless> sinzui: I'm really confused.
<lifeless> sinzui: the bug doesn't talk about membership *outside* lp.
<sinzui> The issue is that there is an underlying assumption that Lp knows about these dates
<lifeless> sinzui: thats *an* issue, not the only one.
<lifeless> sinzui: I agree that addressing the issue with the underlying assumption might also address the issue the users are talking about.
<sinzui> We do not know when a team was registered. membership dates in a team are mutable, they change often as the user's role changes
<lifeless> sinzui: mmm, holding-a-role-dates change, part-of-team doesn't fluctuate like that, neither in reality or as lp models it
<lifeless> join-exit-join does cause terrible modelling issues.
<sinzui> lifeless, I think my tagging of "teams" now shows that that is the most common problem in the registry domain. Teams do not work as users expect.
<lifeless> sinzui: ok
<lifeless> sinzui: so, I guess I'm saying: bug 212439 sounds like a reasonable thing we could fix, even if its not the big picture. You seem to disagree (its marked wont fix); why?
<_mup_> Bug #212439: Incorrect date for "member since" on +members, round 2 <Launchpad Registry:Won't Fix> <https://launchpad.net/bugs/212439>
<sinzui> lifeless, I cannot fix that user's data. that is why.
<lifeless> (indeed, it sounds like a trivial thing to fix)
<sinzui> The data does not exist
<lifeless> sinzui: its not a request for data fixing
<sinzui> We could record the data, then tell admins that we have a field that can be set to correct the lost data
<lifeless> they are asking why lp didn't record the time that the admin clicked on 'accept' in the team.
<lifeless> sinzui: separate the concerns. a) fix existing data, b) do better in future.
<lifeless> I'm talking about b) only
<sinzui> The reason is because all membership changes over write the date
<lifeless> sinzui: they seem to be saying that they *don't*
<sinzui> see the other bugs the not that that underlying issue is wrong
<lifeless> or are you saying 'if they had joined recently the acceptance date is what would have been shown' ?
<sinzui> bugger
<sinzui> I misread the other bugs
<sinzui> The other bugs are about *expiration* date, not acceptance date
<lifeless> sinzui: I might reopen this one ?
<sinzui> There is still an issue that we do not  know how to fix the data
<lifeless> right
<lifeless> but I think might be able to do better going forward
<sinzui> (date_review, date_accepted) -> date_joined. They are used based on subscription policy, and that too is mutable
<lifeless> wgrant: care to put a patch up for CREATE INDEX packageupload__archive__idx ON PackageUpload(archive);
<wgrant> lifeless: Will do it later today.
<lifeless> wgrant: kk
<wgrant> Hmmmmmmmmmmmmmmmmmm.
#launchpad-dev 2010-12-05
<lifeless> === Top 10 Time Out Counts by Page ID ===
<lifeless>     Hard / Soft  Page ID
<lifeless>      131 / 5921  Archive:+index
<lifeless>       64 /  255  BugTask:+index
<lifeless>       21 /  263  Distribution:+bugs
<lifeless>        9 /  115  ProjectGroupSet:CollectionResource:#project_groups
<lifeless>        8 /   15  NullBugTask:+index
<lifeless>        6 /   37  Milestone:+index
<lifeless>        5 /  202  POFile:+translate
<lifeless>        5 /   87  Archive:+packages
<lifeless>        5 /    4  ProjectGroup:+milestones
<lifeless>        5 /    0  BinaryPackageBuild:+retry
<lifeless> wgrant: still interested in DistroSeries:+queue
<lifeless> /srv/launchpad.net/production/launchpad-rev-12004/lib/lp/soyuz/browser/../templates/distroseries-queue.pt
<lifeless>    - Line 281, Column 11
<lifeless>    - Expression: <PathExpr standard:u'custom/libraryfilealias/content/filesize/fmt:bytes'>
<lifeless> ...
<lifeless> LocationError: (None, 'filesize')
<wgrant> That's not meant to happen.
<lifeless> 0 LocationError: (None, 'filesize')
<lifeless> bah
<lifeless> 10 LocationError: (None, 'filesize')
<wgrant> I think Translations may be expiry overzealously.
<wgrant> s/expiry/expiring/
<lifeless> well
<lifeless> the page should handle gc'd content anyhow
<wgrant> No.
<lifeless> no?
<wgrant> It's not allowed to be GC'd...
<lifeless> obviously nothing is enforcing that
<lifeless> I'd accept not /meant/ to be GC'd
<wgrant> There's nothing we can sanely do if it is GC'd.
<wgrant> It is forbidden.
<wgrant> But someone broke it.
<lifeless> but permitted
<wgrant> By the DB? Sure.
<wgrant> It is not possible to sanely apply that constraint in the DB.
<lifeless> I can think of at least one way.
<wgrant> Mortifyingly expensive triggers?
<lifeless> separate tables
<lifeless> so the rules in each table are succinct
<lifeless> is bug 262767 related?
<_mup_> Bug #262767: Expire translations-tarball file contents once they get successfuly processed <soyuz-upload> <trivial> <Soyuz:Triaged> <https://launchpad.net/bugs/262767>
<lifeless> https://bugs.launchpad.net/soyuz/+bug/592417
<_mup_> Bug #592417: LocationError in DistributionSourcePackageRelease pages <oops> <qa-ok> <Soyuz:Fix Released by julian-edwards> <https://launchpad.net/bugs/592417>
<wgrant> The idea in 262767 is what I was going to blame. But it seems to not be implemented anywhere yet.
<wgrant> 592417 is about real things. Not custom uploads.
<wgrant> So it's not really related.
<lifeless> bug 685401
<_mup_> Bug #685401: LocationError in DistroSeries:+queue <oops> <Launchpad Translations:New> <https://launchpad.net/bugs/685401>
<wgrant> lifeless: So, it could just not show them. But the point of the page is to allow packages to be accepted.
<wgrant> Can't accept something that's not there.
<lifeless> somethings thats gc'd isn't there, right ?
<wgrant> Right.
<lifeless> so theres no need to accept it
<lifeless> add a garbo job/fix the expiry code to /really/ delete it.
<lifeless> and we're done, no?
<wgrant> Hmmm? Expiring an unaccepted queue item doesn't make sense.
<wgrant> It is illegal.
<wgrant> It *should* crash.
<lifeless> I don't see why,.
<lifeless> you may have some axiom you need to share.
<wgrant> Hmm, actually, do you have the URL for that OOPS?
<lifeless> yes
<lifeless> in da bug
<wgrant> No query string?
<wgrant> I think it must have been the Done queue.
<wgrant> The query string will tell us.
<wgrant> Maybe Translations did actually run the query that they suggested.
<lifeless> queue_state=3&queue_text=&start=126240
<wgrant> Aha, it is indeed Done.
<wgrant> So we can probably reasonable hide expired files in those cases.
<wgrant> But Translations is still doing something that is technically illegal.
<wgrant> And will break once we unfuck custom upload handling.
<lifeless> whats the illegality
<wgrant> lifeless: Before acceptance: you're deleting part of an upload that hasn't yet been processed.
<wgrant> After acceptance: you're deleting part of an upload that will be used during a copy once we remove 2004-era insanity.
<wgrant> Perhaps for concerningly more broad values of 'we' than would be traditional.
<lifeless> wgrant: if you don't want the upload processed, why is deleting it a problem?
<lifeless> wgrant: what copy?
<wgrant> lifeless: If you don't want the upload processed, you should reject it.
<wgrant> lifeless: Any copy.
<wgrant> Translations has removed all old translations tarballs.
<lifeless> wgrant: so, will you expand the bug appropriately? And suggest the right way forward to achieve their goals
<lifeless> ?
<wgrant> lifeless: I will.
<lifeless> keeping old data because it was used once, is nuts.
<wgrant> To an extent.
<wgrant> We need to actually develop a policy.
<wgrant> And make the code respect it.
<wgrant> Not develop over 5 years.
<lifeless> we have a policy - don't delete anything ever. Except where we need to (which is broadly everywhere), then we can, if we need to.
<wgrant> Well.
<wgrant> The Soyuz policy until 18 months ago was "never delete anything".
<wgrant> Then it was changed.
<wgrant> Without actually making the code work like that.
<wgrant> Or thinking much about the consequences.
<lifeless> waheee
<wgrant> The fact that the hashes are stored in the deleted LFC is a significant problem :(
<lifeless> don't worry, md5 has been broken right?
<wgrant> I really don't like deleting data until we're really, really sure.
<wgrant> We have lost far too much already.
<lifeless> -> nz
<wgrant> See you.
<wgrant> Damn.
<mtaylor> off topic - but anybody here happen to be a god with vim modelines in source files?
<lifeless> mtaylor: yo
<mtaylor> lifeless: so, I have vim set up in my .vimrc to use spaces and to to do 2-space tabs (since that's what we do in drizzle) ... but I'm editing this file that uses tabs at the moment... and I was hoping there was an easy way to put a modeline into the file so that vim would behave properly
<lifeless> noexandtab
<lifeless> blah
<lifeless> noexpandtab
<lifeless> sw=8
<lifeless> that sort of thing
<mtaylor> lifeless: you make me happy
<lifeless> its a gift
<lifeless> probably sts=8 too
<mtaylor> lifeless: works perfectly. thanks
<lifeless> de nada
<thumper> morning
<thumper> it's going to be a quiet day
<jelmer> how can you tell?
<lifeless> I'm on leave
<thumper> and wally is on leave
<thumper> and mwhudson is more quiet than normal since the linaro move
<thumper> so I'm mostly left to myself...
 * thumper goes to make a coffee
<thumper> lifeless: is the launchpad copy of testtools up to date?
<thumper> lifeless: it seems to be lacking some matchers
<jelmer> thumper: EndsWith ?
<thumper> jelmer: yeah
<thumper> also, I'd find IsNot(None) easier to read than Not(Is(None))
<jelmer> thumper: That was added fairly recently, and IIRC the revision after the revision that lp uses.
<jelmer> thumper: I was planning to update lptools to a newer revision anyway, jml added some nice code to help with debugging of "lost" deferreds. Perhaps I should just do that..
<thumper> jelmer: sounds good to me
<lifeless> tip is broken.
<lifeless> jelmer: ^
<lifeless> jelmer: if you wanted to review my fix for the critical bug, that would be good.
<jelmer> lifeless: Sure, I can do that.
<jelmer> lifeless: Ah, that one! I should've remembered.
<thumper> gym time
#launchpad-dev 2011-11-28
<wgrant> huwshimi: lp:~wgrant/launchpad/the-one-domain isn't perfect yet, but it's 90% done.
<huwshimi> wgrant: Dude, that's fantastic!
<wgrant> A few links still lead to the wrong place, but it pretty much works.
<huwshimi> wgrant: That's really awesome. Did you manage to solve breadcrumb stuff?
<wgrant> huwshimi: Yeah, that all works.
<huwshimi> wgrant: :D
<wgrant> Inline help needed a bit of a rework too, but I'll hopefully land that and the breadcrumb stuff in the next day or two, as it's an improvement even if we don't go for one-domain-to-rule-them-all soon.
<huwshimi> awesome
<huwshimi> wgrant: It's so nice browsing through the facets with the urls like that
<wgrant> I just used the existing view names. some like +specs probably want to be renamed.
<wgrant> And we probably want to get rid of +bugs-index
<wgrant> (that's the crippled version of +bugs which only shows the top 10 bugs by heat)
<StevenK> Isn't that used by a projects page?
<huwshimi> wgrant: Yeah, but a great start
<wgrant> StevenK: Yes, but it is pointless and there was discussion last week that it might be removed.
<poolie> lifeless, incidentally it looks like at least in dev, memcached is used for comment rendering still
<lifeless> yes, if the flag is off
<StevenK> Rargh, why is creating a view raising LostObjectError
<wgrant> StevenK: O_o have you been aborting transactions or something?
<StevenK> Certainly not.
 * lifeless stabitty stabitty openidloginview
<lifeless> wgrant: can I nab you for a minute
<wgrant> lifeless: Sure
<lifeless> skype?
<wgrant> sec
<lifeless> want to get this openid fail fixed
<lifeless> wgrant: e.g. https://launchpad.dev/%7Emario-sitz/+archive/ppa/+login?field%85ries_filter=lucid
<lifeless> wgrant: there?
<lifeless> wgrant: hahahahahahahahahahahahahahahaa
<wgrant> Oh god.
<wgrant> What now?
<lifeless> eggs/zope.publisher-3.12.0-py2.6.egg/zope/publisher/http.py line 522
<wgrant> lifeless: That doesn't sound particularly relevant here.
<wgrant> It's the query string we really care about.
<lifeless> wgrant: the old code before the 6* bug was fixed called getURL
<lifeless> wgrant: anyhow
<lifeless> wgrant: yes, its broken
<lifeless> AssertionError: 'key%3Dvalue%85' not in 'http://launchpad.dev/+openid-callback?starting_url=http%3A%2F%2F127.0.0.1%3Fkey%3Dvalue%26%23x2026%3B'
<lifeless> wgrant: thats using a not-unicode-at-all value in the current code
<wgrant> Hah
<lifeless> note the x2026
<lifeless> entity escaping
<wgrant> Yeah.
<mwhudson> this brings back horrible memories
<lifeless> mwhudson: indeed
<mwhudson> mod_rewrite isn't involved is it?
<lifeless> mwhudson: now
<wgrant> Not this time.
<lifeless> mwhudson: no, just some dubious code in the openid login view
<mwhudson> oh good
<lifeless> mwhudson: see the form_args property in login.py
<lifeless> mwhudson: and see if you can spot the two separate defects
<mwhudson> mind you doing, string ops on urls unless you are exceedingly careful about normalization is ... a bad idea, i guess
<lifeless> indeed
<lifeless> I keep meaning to write a stringlike url class
<lifeless> that will refuse unsafe things and not be horribly slow
<lifeless> then I realise I have useful things to do
<mwhudson> lifeless: well, i don't know what UnicodeDammit does
<mwhudson> and well yeah, no escaping?  i don't know enough about request.form.items() to be specific
<wgrant> It's good for screen-scraping.
<wgrant> It's not good for this.
<mwhudson> e.g. if value_list_item was 'a&b=c' this could obviously be hilarious
<lifeless> mwhudson: as is a&b=c\x85
<lifeless> mwhudson: thats bug one - unicodedammit does entity escaping
<lifeless> ok, time for zope esoterica
<lifeless> whats teh difference between request.items() and request.form.items() ?
<wgrant> lifeless: request.items() includes cookies, environ and form.
<wgrant> Aha
<wgrant> Just caught an ec2 run with the poppy error before it finished.
 * wgrant investigates.
<wgrant> There's already a poppy-sftp running. Something's not cleaning up.
<wgrant> It's lp.poppy.tests.test_poppy.TestPoppy.test_bad_gpg_on_changesfile(ftp)
<lifeless> well, thats something we have prod issues with
<lifeless> \o/
<wgrant> Baaaaaaaaaaah
<wgrant> Does ec2 shut down if it sees the testrunner hung?
<wgrant> I thought it just shut down after 8 hours or something.
<jtv> StevenK: the BPPH.bpn populator is done.
<jtv> It's not very easily noticed in the logs, since it basically blocks, blocks, blocks, and then gets killed.
<lifeless> wgrant: up for a review of what we discussed earlier ?
<wgrant> lifeless: Sure
<lifeless> diff soon I expect - https://code.launchpad.net/~lifeless/launchpad/oops-polish/+merge/83544
 * wgrant is looking for a reviewer for https://code.launchpad.net/~wgrant/launchpad/globalise-help-folder/+merge/83543
<wgrant> Got enough criticals linked to that?
<wgrant> Oh, it's the old branch, I see.
<lifeless> yeah
<lifeless> it crept
<wgrant> Woot
* wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugtasks: 299
<lifeless> I didn't realise I was working on the wrong thing initially
 * wgrant finds another OOPS
<wgrant> There must be one here somewhere...
<wgrant> Tempting to make bug #890976 critical, as it hinders determination of timeout causes.
<wgrant> And it would make 300 tasks.
<wgrant> Which is an important milestone.
<lifeless> bug 890976
<wgrant> enomup
<wgrant> Bug #890976: hard to determine causes of late evaluation
<lifeless> its fixed ;)
<wgrant> Bah.
<wgrant> That's not very useful, then.
<lifeless> dunno why it wasn't marked f-r when deployed
<wgrant> lifeless: Anyway, I'm looking at your branch and trying to break it.
<lifeless> thanks
<wgrant> O
<wgrant> h
<wgrant> A new critical
<wgrant> Where'd that come from...
* wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugtasks: 300
<wgrant> victory
<wgrant> Ah, it was you.
<lifeless> timeout errors apprpving your MP
<wgrant> Awesome.
<wgrant> BranchRevision?
<lifeless> no OOPS reported
<wgrant> Or Branch FKs/
<wgrant> Pardon?
<wgrant> 502s, then?
<lifeless> just a just popup saying 'timeout error'
<wgrant> oh
<wgrant> heh
<wgrant> OOPS ID should be in the top right AJAX log thingy.
<lifeless> no
<wgrant> Lies.
<lifeless> ID: 0, status: 503 (Service Unavailable)
<wgrant> Hmmmm.
<wgrant> Oddity.
<lifeless> ENOTLYING
<wgrant> Liar.
<wgrant> I wonder if its OOPS matching breaks with the new scheme.
<wgrant> But surely not.
<wgrant> IIRC it just checks X-Lazr-OopsId
<lifeless> case sensitive ?
<wgrant> No idea.
<wgrant> Anyway.
<wgrant> Keep trying, I guess. If it works, no problem. If it keeps timing out, it will appear on the OOPS reports where it will be summarily ignored.
<wgrant> :)
<wgrant> lifeless: You'll be pleased to know that the URL in https://bugs.launchpad.net/launchpad/+bug/61171 now causes a post-login OOPS.
<lifeless> good :)
<wgrant> s@/bugs/@/firefox/@ to make it mostly work.
<wgrant> Why?
<lifeless> closer to the right place
<wgrant> Heh
<wgrant> Looks relatively sane otherwise.
<lifeless> wgrant: doesn't oops for me
<lifeless> wgrant: it just does a 404 error
<wgrant> lifeless: /bugs/+filebug doesn't exist any more.
<lifeless> wgrant: (a 404 same-domain oops that is)
<wgrant> Use eg. /firefox/+filebug instead
<wgrant>     Module zope.app.form.browser.widget, line 522, in renderTag
<wgrant>     attr_list.append(u'%s=%s' % (key, quoteattr(unicode(value))))
<wgrant> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe9 in position 79: ordinal not in range(128)<br />
<lifeless> I bet that is able to be provoked directly
<wgrant> Probably, yes.
<wgrant> Now I actually read it.
<lifeless> which is why I said good
<lifeless> this is just more tech debt in our stack
 * StevenK grumbles at LostObjectError some more.
<wgrant> StevenK: What's the test?
<StevenK> wgrant: http://pastebin.ubuntu.com/752154/
<StevenK> view.errors is throwing LostObjectError.
<lifeless> wgrant: you, hitting enter on that just breaks directly in the form widget
<lifeless> wgrant: so any logged in browser getting that from apport will barf in-place
<wgrant> lifeless: It works on prod
<lifeless> how verra interesting
<wgrant> Rather
 * wgrant checks what the post-login URL is.
<lifeless> given I only touch openidloginview
<lifeless> https://bugs.launchpad.dev/firefox/+filebug?field.title=package%20uml-utilities%2020070815-1.1ubuntu2%20failed%20to%20install/upgrade%3A%20el%20subproc%E9s%20post-installation%20script%20retorn%E0%20el%20codi%20d%27eixida%20d%27error%201
<StevenK> wgrant: I get the same issue in a unit test
<wgrant> https://bugs.launchpad.net/ubuntu/+filebug?field.title=package%20uml-utilities%2020070815-1.1ubuntu2%20failed%20to%20install/upgrade:%20el%20subproc%C3%A9s%20post-installation%20script%20retorn%C3%A0%20el%20codi%20d'eixida%20d'error%201
<lifeless> https://bugs.launchpad.net/launchpad/+filebug?field.title=package%20uml-utilities%2020070815-1.1ubuntu2%20failed%20to%20install/upgrade%3A%20el%20subproc%E9s%20post-installation%20script%20retorn%E0%20el%20codi%20d%27eixida%20d%27error%201
<lifeless> is a prod url that barfs
<wgrant> Yep
<wgrant> Wait
<lifeless> the reason the +login transition url works is because of the entity escaping
<wgrant> Your prod URL is a post-login one that breaks, or one that works?
<lifeless> wgrant: breaks
<lifeless> wgrant:
<lifeless>     attr_list.append(u'%s=%s' % (key, quoteattr(unicode(value))))
<lifeless> UnicodeDecodeError: 'ascii' codec can't decode byte 0xe9 in position 79: ordinal not in range(128)<br />
<lifeless> on prod
<wgrant> So, looks like you need to fix that first.
<wgrant> Rather than rebreaking apport again.
<wgrant> However, looks like your login fix is good.
<lifeless> wgrant: you misunderstand
<wgrant> It's just +filebug that's crap now.
<lifeless> wgrant: apport starts with a normal url right ?
<lifeless> wgrant: *if* the browser isn't logged in, they get a login bounce around
<lifeless> wgrant: right now, if the browser *is* logged in, it will simply fail straight up
<wgrant> Huh, indeed it does.
<wgrant> Unless the redirect *to* +login is breaking stuff.
<lifeless> wgrant: If the browser isn't logged in, it will get a login bounce around, which will 'fix' it while making the search for dupes unable to find anything.
<lifeless> wgrant: so no, I don't think the +filebug issue needs fixing before landing the login fix.
<wgrant> lifeless: Do we know that the original URL doesn't work?
<lifeless> wgrant: we don't have the original url anywhere that I can see
<wgrant> The query string may already have been mangled, because we're looking at URLs that already contain +login.
<wgrant> Your explanation seems reasonable, however.
<lifeless> possibly in a referer if we have any of the oopses around
<wgrant> Did we get a new spinner last week?
<wgrant> It looks different now.
<lifeless> none of the oopses have a referer
<lifeless> twitch
<wgrant> That's upsetting.
<lifeless> it is
<lifeless> including ones on shipit
<StevenK> wgrant: Any ideas? :-(
<wgrant> StevenK: Sorry, distractions abound.
 * wgrant looks.
<StevenK> Clearly.
<wgrant> StevenK: What's the traceback on the LOE?
<wgrant> However, it'spossible that create_initialized_view is aborting the transaction on error.
<wgrant> If you commit before it, it might be happier.
<wgrant> Because the person and project will still exist after the view transaction is aborted.
<StevenK> wgrant: http://pastebin.ubuntu.com/752157/
<wgrant> Yeah, I suspect the transaction is being aborted. Try committing before create_initialized_view
<StevenK> I get it. The rest of that doctest uses sampledata. Which does not need to be nailed into the DB.
<StevenK> Bloody sampledata.
<jtv> StevenK: wanna review?  https://code.launchpad.net/~jtv/launchpad/bug-849683/+merge/83546
<jtv> Updates sample data, funnily enough.
<StevenK> I'd rather sort out this validate() method, sorry.
<jtv> OK, I'll pounce on someone else.
<jtv> But I thought you'd at least want to know!
* lifeless changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugtasks: 3*10^2
<StevenK> Oh, fan-bloody-tastic, we're at 300?
<wgrant> Hah
<wgrant> 3E2? :)
<jtv> In its own small way, dropping the 3rd significant digit is progress.
<StevenK> Critical bugtasks: 0x12c
<wgrant> StevenK: That doesn't quite so effectively prevent me from updating it a couple of times a day.
<lifeless> jtv: reviewed
<StevenK> But it *looks* smaller
<jtv> Thanks!
<lifeless> jtv: tl;dr - I'd keep the job for now.
<lifeless> jtv: as insurance
<jtv> Against what, though?
<lifeless> jtv: bugs that we don't know about causing NULL values in the column
<jtv> We're not going to start using these until we have the constraint in place, are we?
<lifeless> jtv: the constraint adding will need the migrator to run on any such faulty rows, should they appear.
<jtv> From this point on, I'd rather have breakage when adding the constraint than bugs silently hidden by the jobs.
<lifeless> jtv: breakage adding the constraint extends downtime
<wgrant> Well
<lifeless> jtv: I'm very much against extending site wide downtime vs having bugs turn up later
<jtv> Not if we disable the jobs first; then we can detect them with a simple pair of queries.
<wgrant> There are very probably still creators that need to be updated.
<wgrant> As jtv says, we can detect them easily this way.
<wgrant> although it might be better to just disable the migrator for a few days first.
<jtv> I'm even more against hiding bugs until they hit production.
<lifeless> you'll need to reinstate the migrator to correct the rows any which way
<lifeless> how about making sure the test suite passes with such a constraint added, as a first step ?
<jtv> That's what we have bzr for, isn't it?
<lifeless> jtv: bzr doesn't really cut down on the 8 hour latency to land something
<jtv> Yes, running the test suite with the constraint is a good one.
<StevenK> Don't remove it, move it to --experimental
<StevenK> Then it doesn't get run by default
<StevenK> See if the numbers are creeping up
<lifeless> wgrant: for your entertainment - bug 897053
<wgrant> Hm, must be only a few days left.
<wgrant> Until 900000
<wgrant> lifeless: Thanks.
<lifeless> over 900000
<wgrant> Unconventional, but indeed.
<jtv> StevenK: see if what numbers are creeping up exactly?  It doesn't look like the populators have found any new rows to populate recently.
<StevenK> jtv: So. Disable the populators by moving them to experimental. Get that rolled out to prod. Check if sourcepackagename IS NULL is non-zero after 24 hours, same for binarypackagename IS NULL.
<jtv> StevenK: but what would be the difference with checking the populator logs now?
<StevenK> jtv: That will tell you if there are other codepaths are not creating SPPHs or BPPHs without setting the name.
<StevenK> Which will blow up if you add the NOT NULL constraint.
<lifeless> I'd suggest moving to experimental; so that *if, when the constraints time comes*, we have a readily available plan B, rather than rollbacks and 8+ hours of delay
<jtv> StevenK: but what would be the difference with checking the populator logs now?
<lifeless> costs nothing, provides support.
<lifeless> jtv: there isn't any difference vs checking the logs now, you are correct on that
<jtv> Checking the logs now costs less.
<jtv> Which is what I've been doing.
<StevenK> jtv: Less risk
<lifeless> jtv: the main thing is the cost of recovery, if its needed.
<StevenK> Versus 8+ hours of delay due to rollbacks
<jtv> More steps, more latency, more complication, no extra coverage â that's *more* risk.
<lifeless> no additional latency, one more step.
<lifeless> wow,  https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-99869e94b028f4641c1feeb4806ed88e has a horrible query count
<jtv> How is an additional step not additional latency?  And we know for a fact that no unpopulated rows have been found since Wednesday.
<lifeless> because its not serialised
<lifeless> the critical path to using the columns is 'add constraint [if possible]; profit'
<lifeless> thats the same critical path if you keep the job or not
<jtv> If you omit âchange the branch, come back and clean up the jobs again later.â
<lifeless> those things don't impact the ability to use the columns
<lifeless> why should they be included in the critical path?
<jtv> Okay, so they're not in the critical path.  Then why should I do them?
<jtv> Note that apparently we can't proceed with the critical path if I don't.
<lifeless> because once we're able to use the column they become tech debt
<lifeless> I don't understand why you can't proceed. Can you expand on that?
<jtv> You seem to insist that I move the populators back in before landing this branch.
<jtv> Here's another consideration though:
<jtv> We haven't seen any broken rows appear for 5 days.
<jtv> If we still have code that creates broken rows, how big are the chances that it will create enough of them to necessitate a loop tuner between the time we retire the populators and the time we add the constraints?
<jtv> AFAICS we're just optimizing for failure rather than protecting from it.
<lifeless> moving the populator back in would be very simple here and avoid latency later when it might be needed. I proposing to optimise for less team-wide and user-wide impact should failure occur
<lifeless> the probability thing is a good question
<jtv> Anyway, I'm shelving this for now; I do have more urgent things to work on.  Maybe I'll check the logs again in a few days or something.
<StevenK> There were 1 imports of names not appearing in the __all__.
<StevenK> You should not import _details_to_str from testtools.testresult.real:
<StevenK>     canonical.launchpad.scripts.runlaunchpad
<wgrant> poolie: Is it meant to be .tgz rather than something sensible?
<wgrant> We aren't using DOS or 16-bit Windows any more :)
<poolie> isn't it tgz?
<wgrant> It's a gzipped tarball, yes.
<poolie> oh '.tar.gz'
<wgrant> But the conventional extension for a gzipped tarball is .tar.gz.
<poolie> http://achewood.com/index.php?date=11222006
<wgrant> Heh
<poolie> it was slightly easier to write this way
<poolie> i can change it
<wgrant> I haven't seen anybody use .tgz in yeeeears.
<wgrant> So if it's easy...
<wgrant> Anyone keen to review another oversized branch? :)
<wgrant> Though this one's only 2000 lines, not 3300.
<poolie> i'm going to revisit it to add more links
<wgrant> +124/-441
<poolie> as long as most of them are minuses :)
<StevenK> wgrant: What are you ripping out?
<wgrant> StevenK: CodeLayer and TranslationsLayer
<wgrant> Won't land this one until I've confirmed with mrevell, though.
<StevenK> And renaming TranslationsLayer:DistroSeries:+admin ?
<wgrant> Not yet.
<StevenK> Does that make it unreachable if it lands?
<lifeless> shouldn't the translations admin become an expandable onthe one +admin/
<lifeless> if you're thinking streamlined UI that is
<wgrant> I'm thinking about destroying subdomains.
<wgrant> Not thinking streamlined UI.
<wgrant> StevenK: No.
<wgrant> StevenK: It's still on translations.launchpad.dev, and this doesn't change any links.
<wgrant> It just makes the views accessible in more places.
<StevenK> Right. Link me?
<wgrant> The layers still exist, and are bound to the relevant publishers. But there's only one view left bound to them.
<wgrant> https://code.launchpad.net/~wgrant/launchpad/delayer-views/+merge/83547
<StevenK> lib/canonical/launchpad/pagetests/basics/notfound-traversals.txt is quite disgusting
<wgrant> Actually, I lie. There are three ancient pre-1.0 tour redirects left bound to each layer, but they can die later.
<wgrant> Because nothing links to them.
<wgrant> yes.
<wgrant> It doesn't take 4 minutes to run any more, though.
<wgrant> Only like 90 seconds.
<wgrant> Not sure why.
<StevenK> wgrant: Why did you remove TranslateRedirectView and not TranslationsRedirectView, the latter of which is actually implicated in the ZCML?
<wgrant> StevenK: Both were used in ZCML, to redirect to the corresponding pages on translations.launchpad.net. But TranslateRedirectView has one remaining use: SP:+translate -> SP:+translations. I believe that's in place due to old liblaunchpad-integrations, from before Gutsy, but I need to confirm that nobody's still using it.
<StevenK> wgrant: Why not remove both, then?
<wgrant> Because SP:+translate may still be pointed to by something.
<wgrant> It redirects to +translations, not just to a different vhost.
<wgrant> That would break a URL that works now, which is not something I want the branch to do.
<StevenK> wgrant: You say TranslateRedirectView might still be used, but you're still removing it?
<wgrant> StevenK: TranslationsRedirectView is removed
<wgrant> TranslateRedirectView remains.
<StevenK> The diff says otherwise.
<StevenK> 2008 -class TranslateRedirectView(PageRedirectView):
<wgrant> Ah, right, I misremembered.
<wgrant> Was thinking that it was named after where it redirects *from*, which of course makes no sense.
<wgrant> We still need to redirect to +translations, to TranslationsRedirectView remains.
<wgrant> s/to Trans/so Trans/
<wgrant> The tests pass like this, anyway :)
<StevenK> And .dev behaves like you expect it to, re SP:+translate ?
<wgrant> Yes.
<wgrant> TranslationsRedirectView will later be changed to redirect to mainsite instead of translations, but not yet.
<wgrant> And then everything except mainsite will hopefully mysteriously evaporate next week.
<jtv> Weren't we still using that one for the licensing-agreement redirect?
<StevenK> wgrant: Approved.
<wgrant> jtv: The only redirects I touched were unconditional ones to +translate or +translations.
<wgrant> +licensing is now available on mainsite too instead of just translations, but that's all I've changed relating to that.
<jtv> Very possible that I'm wrong; just asking.
<wgrant> Sure, always best to be safe in stuff like this :)
<jtv> The redirect was covered in a pagetest, at least at some indefinite point in the past.
<wgrant> Hmm.
<wgrant> Which redirect?
 * StevenK grumbles.
<jtv> I'm just having a quick check.
<wgrant> StevenK: SourcePackage:+translations is still hit... 10000 times last month. I wonder where from.
<StevenK> +translations or +translate?
<wgrant> Bah, +translate
<StevenK> This validate method gets no owner in two cases. :-(
<wgrant> If it's hit that often, I should have a referrer in some logs.
<StevenK> No owner when the specified team is open, or if the field is empty.
<StevenK> Maybe I should just reach into view.errors and pull out the ConstraintNotValid
<StevenK> That will let me tell the difference.
<wgrant> Hmmm
<wgrant> Firefox 4
<wgrant> Old.
<wgrant> Although it's natty.
<wgrant> huuuuh
<wgrant> /ubuntu/gutsy/+sources/kdebase/+translate
<wgrant> Since when does +sources exist?
<wgrant> Isn't it +source?
<StevenK> I thought so.
<wgrant> Is anyone still running natty?
<wgrant> Ah, I have a VM.
<wgrant> Natty's Firefox still uses SP:+translate :(
<wgrant> liblaunchpad-integration was fixed years ago.
<wgrant> So we can't drop the redirect yet.
<wgrant> And Firefox to this day uses https://launchpad.net/distros/ubuntu/oneiric/+sources/firefox/+gethelp
<wgrant> Should probably fix that before Precise locks us into supporting that triply deprecated URL for another 5 years.
 * wgrant files.
<wgrant> I guess we can SRU it away in a few weeks, too.
<wgrant> jtv: What's the best view to link to for source package translations? SP:+translations?
<wgrant> +translate is clearly deprecated, but I don't know if there's something better.
<StevenK> wgrant: At least it doesn't use edge :-P
<jtv> wgrant: was +translations that complete rewrite of the view that we didn't quite complete about 2 years ago?
<wgrant> jtv: +translate redirects to +translations now
<jtv> wgrant: I can only speculate at this point.  This may have been the new view that we kept unlinked in parallel to the old one, under a separate name, until it was completed.  That _could_ be the full explanation for the redirect and the new name.
<jtv> To verify that, you could dig up the history to see if the views ever existed side by side.
<jtv> With very similar class names.
 * jtv digs up timing information
<jtv> wgrant: they'd probably have existed side by side in early 2010.
<wgrant> jtv: Aha, thanks.
<wgrant> +translations looks to be the way to go.
<jtv> Speaking purely aesthetically, I'm not entirely happy with that.  It's pretty much all âtranslations,â isn't it?
<wgrant> Hmm?
<wgrant> The subdomains are probably not long for this world.
<jtv> Arguably one could call just about any level of navigation to that page "translations."
<wgrant> Well, I gues.
<jtv> The things I like about +translate are its brevity and its specificity.  OTOH it doesn't quite cover translation review.
<wgrant> s
<wgrant> But we also have +answers, +bugs, +questions, +branches
<wgrant> Er, not +answers, that's +questions.
<wgrant> But you get the idea.
<jtv> True.  But with translations, we have extra layers of granularity: source package or product series; template; po file; individual translated message.
<jtv> The latter would be +translation if anything, but the rest are all âthe translations forâ¦â at some aggregation level.
<wgrant> Sure.
<wgrant> +translations is a list of translations.
<wgrant> Seems to make sense :)
<StevenK> If your membership does expire, we'll send you one more message to let
<StevenK> you know it's happened.
<StevenK> LIES
<wgrant> StevenK: It's true.
<wgrant> It just doesn't tell you about the intervening 6 messages before the expiry.
<StevenK> It should
<StevenK> "By the way, we'll spam you every day until you fix it, you dork."
<StevenK> wgrant: Can you review https://code.launchpad.net/~stevenk/launchpad/better-error-open-owner/+merge/83550 ?
<jtv> wgrant: anyway, de gustibus and all that.  Maybe the only important question is whether a new URL with an extra "/translations/" in the path (to replace the hostname) will look ridiculous.
<StevenK> Why would do that if we don't need to?
<jtv> I thought we were getting that?
<wgrant> There's one ambiguous view name in the entire application, DistroSeries:+admin.
<wgrant> At this stage I'm just flattening everything into mainsite, without changes names.
<wgrant> The facet menu will link to +translations, +bugs, etc.
<wgrant> Instead of the default view on the relevant vhost.
<jtv> No clashes?
<wgrant> Only DistroSeries:+admin.
<wgrant> I was quite impressed.
<wgrant> We've been on separate layers for more than 5 years.
<wgrant> But have only made effective use of the separation *once*.
<wgrant> StevenK: Is there really no better way?
<StevenK> wgrant: For the isinstance garbage?
<wgrant> For raising a sensible error.
<wgrant> Rather than deleting bits of self.errors.
<StevenK> The delete is not needed, that was to make the test cleaner.
<wgrant> Even so.
<wgrant> There has to be a better way.
<wgrant> Also, that error message is silly.
<wgrant> It shouldn't include the project name, and it should say what makes a person or team valid.
<StevenK> wgrant: "You must choose a person or a Moderated/Restricted team to be the owner."
<StevenK> wgrant: I've been trying to come up with a better way and have been failing.
<wgrant> I believe we're moving to the term "exclusive team"
<wgrant> Talk to Curtis tomorrow.
<wgrant> He may also have a suggestion for making the error replacement less nauseating.
<wgrant> Bah.
<wgrant> stub's DB patch is ahead of mine :(
<jtv> Questions: we sync SPPHs from Debian, which then triggers builds in Ubuntu, and process-upload then processes the resulting BPBs/BPRs?  Is that correct?  If so, how does the sync trigger the build?
<wgrant> packagecopier
<StevenK> We do not copy the SPPH.
<StevenK> We copy the SPR, which is linked to a new SPPH.
<wgrant> Well.
<wgrant> We don't copy the SPR.
<wgrant> We reuse the same SPR in a new SPPH.
<jtv> Right, that's the part I'm somewhat familiar withâbut then what is it that triggers the build?
<StevenK> The packagecopier
<jtv> I'm looking at that code, but not really finding how the information gets from A to B.
<wgrant> It calls SPPH.createMissingBuilds
<jtv> Ah!
<jtv> And that figures out what still needs building?
<StevenK> Yeah
<StevenK> Via disgusting code
<wgrant> jtv: Didn't you change that code just a couple of weeks ago?
<wgrant> getBuildByArch or something.
<jtv> One cog, yes.  But unfortunately that doesn't make me familiar with how it fits into this machine.
<jtv> Things are so nice and clear-cut when you've got existing, tested code and all you have to do is make it run faster.  :)
<jtv> I did see createMissingBuilds while working on that, but I mostly looked at what was relevant to me right then.
<wgrant> Hahahah getBuildByArchs tested?
<wgrant> A bit, yes.
<jtv> The painful reality is that for me, soyuz and the build system have too many concepts for the "ah, I know where this bit goes" to happen spontaneously.
<jtv> Actually I think it was called directly quite a few times in a doctest.
<jtv> Most of the coverage was indirect though.
<jtv> wgrant: I guess then that, to reproduce the bug about unwanted notifications to Debian maintainers, I can proceed as follows:
<jtv> Publish a package, with a maintainer.
<jtv> Sync it to another distro.
<jtv> (Well, archive I suppose)
<jtv> Run createMissingBuilds on the new SPPH.
<jtv> Thenâ¦ straight to the upload processor, and sabotage it somehow so that the binary package gets rejected?
<wgrant> Right.
<jtv> So the BPB is an input to the upload processor?
<wgrant> Yes.
<wgrant> It usually takes it as an argument.
<jtv> Ahhh that's where I was lost.  Thanks for explaining that.
<jtv> It actually sounds pretty simple now, but I was stuck for ages.
<jtv> (The test we _thought_ I could just twiddle a bit to get the same result didn't seem to, and it relied too much on implicit data creation to make it easy to figure out why)
<wgrant> Is the Julian back today?
<jtv> I think I'll prepare an NDT rollout request so it's ready to go on LPS as soon as the Q/A blockage clears.
<bigjools> morning all
<lifeless> zomg its alive
<bigjools> it will die under 3k5 emails
<lifeless> poolie:  https://code.launchpad.net/~python-fixtures/python-zope-fixtures/trunk probably has the fixture you wanted the other day, now.
<adeuring> good morning
<lifeless> poolie: I have a suggestion for another split out if you like
<lifeless> poolie: ec2*
<lifeless> poolie: possibly into lp-dev-tools (or whatever it is called - it exists already)
<lifeless> poolie: [I don't mean lptools]
 * wgrant deletes vostok
<lifeless> \o/
<lifeless> if it had templates and functionality, I'd be sad.
<StevenK> wgrant: Can you delete delayed copies while you're at it?
<wgrant> Sadly not.
<StevenK> Some good perms for copyPackages() would be a nice start.
<bigjools> I am working on PCJs for PPAs
<bigjools> then we can remove delayed copies
<wgrant> Well.
<wgrant> We also need to sort out permissions for ubuntu-security.
<wgrant> But yes.
<mrevell> Hello
<nigelb> Morning mrevell :)
* rvba changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: rvba | Critical bugtasks: 3*10^2
<allenap> mrevell: Re. danhg's email, are you happy to sort out the notifications for a translations-related downtime tomorrow at 1000 UTC?
<mrevell> allenap, Yeah, I'll do that now.
<mrevell> allenap, Same deal as before?
<allenap> mrevell: Yep. The only thing you might want to add is that this has been rescheduled from last week.
<mrevell> allenap, Sho thang. Okay, I shall dent, Tweet, email and bloggerise right this instant.
<allenap> mrevell: Ta dude.
<lifeless> argh ilbqtwebkit weekly dbg builds of 200MB
<lifeless> my disk, my disk, my kingdom for some disk
<bigjools> bad time to need disk :)
<wgrant> rvba: Thanks.
<rvba> welcome
<rvba> wgrant: in your branch distroseries-translations-admin, I'm not sure I understand why you removed the layer declaration in lib/lp/translations/browser/configure.zcml â¦Â ? Can you explain why you did that? (line 10 of the mp)
<wgrant> rvba: Ah, that's to finish off https://code.launchpad.net/~wgrant/launchpad/delayer-views
<rvba> wgrant: ok, thanks for explaining.
* benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: rvba, benji | Critical bugtasks: 3*10^2
<rvba> Morning benji.
<benji> mornin' rvba
<rick_h_> morning all
<rvba> benji: in the queue: 3 branches. The first one (jtv's fix for 717969) will be reviewed by Julian, the second one (Martin's) is really WIP because we need to figure out if we really want that.
<benji> rvba: cool, thanks for the info
<rvba> About the third one: there is a discussion ongoing between Rob and Jeroen so I did not touch itâ¦
<mrevell> Any Orange squadders around?
<mrevell> rick_h_, hey
<rick_h_> mrevell: yes
<mrevell> rick_h_, Hey, do you know when the spinner for custom bug listings might go live on production? If you don't know, no worries.
<rick_h_> mrevell: no, abentley is waiting to get some feedback on the goal I think from deryck, but that's a bit delayed atm.
<rick_h_> speak of the devil
<mrevell> aha :)
<rick_h_> I think there might be more more huw involvement
<mrevell> Thanks rick_h_
<rick_h_> requested as well, checking the email that went around this weekend
<nigelb> g37
<nigelb> urgh
<abentley> rick_h_: missed the context.
<mrevell> rick_h_, Yeah, Deryck asked huw for some help with aligning items vertically.
<rick_h_> abentley: mrevell is asking about the spinner for the custom bug listings
<mrevell> abentley, I was just wondering if we had an ETA on the spinner.
<abentley> mrevell: We have our normal spinners ready for QA.  They'll be replaced with deryck's spinner, hopefully this week.
<mrevell> Great!
<mrevell> Thanks abentley
<abentley> mrevell: np
<rvba> benji: could you please review https://code.launchpad.net/~rvb/launchpad/builders-timeout-bug-887078-eager-load3/+merge/83576
<benji> rvba: sure
<mrevell> abentley, In the comments to bug 894736 bryceh suggests that milestones, people, etc should be clickable and that clicking one of them should narrow the scope of the search. So, if I see abentley as a reporter, and then click abentley, I'd get a revised listing of bugs that met my original search criteria, kept the ordering I chosen but were limited to those where abentley was the reporter. That's out of scope for
<mrevell>  this project but do you have a rough idea of how much work it'd take to implement that?
<_mup_> Bug #894736: People not linked on bug listing <bug-columns> <Launchpad itself:Triaged> < https://launchpad.net/bugs/894736 >
<abentley> mrevell: I'd guess a week, but it depends on whether we'd have to do any work on the backend, or only support filtering by fields already supported in advanced search.
<mrevell> Great, thanks abentley.
<mrevell> Firmly out of scope :)
<abentley> mrevell: I'm also not sure that's the best way to implement such filtering.  It could be quite inconvenient to find the first instance of a milestone in order to filter by it, for example.
<mrevell> abentley, Hmm, we can offer other ways to filter by milestone. Of course, this is firmly in the territory of reworking advanced search. I'm also not sure it chimes with my experience of using Amazon search. I'll reply on the bug.
<abentley> rick_h_: standup?
<rick_h_> abentley: ping, heads up, issues with headset for stand up
<rick_h_> ugh, I can record with audacity, but can't get mumble to use it
<abentley> rick_h_: ack
<abentley> rick_h_: and ick
<rick_h_> think  I've entered pulse black hole somewhere
<abentley> rick_h_: bzr+ssh://bazaar.launchpad.net/~deryck/launchpad/buglists-loading-885272/
<rick_h_> ty
<rvba> Thanks for the review benji.
<benji> rvba: my pleasure
<rick_h_> abentley: hah, portable mic has a mute hardware button doh!
<rick_h_> will be better tomorrow
<abentley> rick_h_: hehe.  cool.
<sinzui> jcsackett, does allhands.canonical.com show that I modified your objectives last week? Can your accept them?
<abentley> gary_poster: If I want to find out the name of the current view, should I iterate through getGlobalSiteManager().registeredAdapters() ?
<jcsackett> sinzui: it does; i have two tasks showing modified objectives. one shows "registry" competency, one has "code" competency. i confirmed on the task with "code" since registry wasn't something we discussed.
<jcsackett> i'm not sure what to do about the now open task with the (seemingly?) wrong objectives in it.
<sinzui> jcsackett, do you still have bug domain? maybe I pasted over it?
<jcsackett> sinzui: in the task with objectives i accepted, there was bugs, code, and ui stuff--all things we agreed on recently. the other had bugs, registry, and ui stuff.
<sinzui> I see them now.
<gary_poster> abentley, that would do the trick, and an obvious was to do it does not leap to mind.  (I thought of and discarded another approach)
<abentley> gary_poster: Cool, thanks.
<sinzui> jcsackett, I think I accepted your acceptance. Do you need to confirm that you same my acceptance?
 * sinzui never knows if allhands is messing with his head
<jcsackett> sinzui: i have countersigned.
<jcsackett> sinzui: is there a way to delete a task on allhands? because i still have the redundant "check objectives" task.
<sinzui> Maybe there is a delay. I did not see your acceptance at first
<sinzui> jcsackett, I am confused my the status of https://bugs.launchpad.net/launchpad/+bug/893982 Are all the parts in place to QA it?
<_mup_> Bug #893982: loggerhead privacy ribbon doesn't have pad lock icon <branches> <disclosure> <exploratory-testing> <privacy> <ui> <Launchpad itself:In Progress by jcsackett> < https://launchpad.net/bugs/893982 >
<jcsackett> sinzui: yes, i had a note to qa it this morning. doing so now.
<jcsackett> sinzui: so, the bug is fix committed, but there's some weirdness as far as how to track it.
<jcsackett> sinzui: another branch (not one of mine) bumped the version of loggerhead that lp is using, which gets my loggerhead changes.
<jcsackett> should i just link that branch to this bug as well?
<jcsackett> kind of messes up links, since said branch is entirely incidentally grabbing my changes.
<sinzui> jcsackett, only if that branch is being landed. Did that branch cause a version confusion?
<jcsackett> sinzui: the branch that bumped loggerhead has landed and is also qa-ok. it is 14383. it has not caused any confusion, my loggerhead-bump branch didn't land, and then flacoste noticed the other one had landed and rejected mine to avoid any such confusion.
<sinzui> jcsackett, I do not think the branch needs linking unless you need the bug reset to qa-needstesting when the branch is on qastaging.
<jcsackett> sinzui: ok. that's what i thought as well.
<jcsackett> sinzui: then, all is well, bug is qa-ok fix-committed. i'll update kanban.
<sinzui> thank you
<rvba> benji: Can you please have a look at https://code.launchpad.net/~rvb/launchpad/bugs-timeout-bug-892820/+merge/83631 ?
<benji> rvba: sure
<rvba> thx
<gary_poster> abentley (since deryck is out today), dunno if this is helpful to highlight here, but I just triaged 897242 and 897277 as "bug-columns" bugs.
<abentley> gary_poster: thanks.
<gary_poster> welcome
<abentley> gary_poster: registeredAdapters iterates through AdapterRegistration, but AdapterRegistration doesn't seem to provide the adapter class, only a factory function, so I don't see how I can look up the view name.
<abentley> gary_poster: nm, the problem is that it's obfuscated behind SimpleViewClass.
<gary_poster> abentley, yeah, I wondered if I should warn you about that :-/
* rvba changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: benji | Critical bugtasks: 3*10^2
<nigelb> 300 critical bugs? or is it 900? :P
<rick_h_> abentley: adeuring going in search of food, bbiab
<abentley> rick_h_: ack
<adeuring> rick_h_: enjoy lunch
<rick_h_> abentley back fyi, the spinner stuff makes some sense, though now sure how he means to "use" it in the actually ajax calling code.
<abentley> rick_h_: cool.
<abentley> rick_h_: Since we have spinner support in stable, you should be able to merge that and hook it up pretty easily.
<rick_h_> cool, will merge this up with devel then and see if I can make it work out
<abentley> rick_h_: I think you'll just need to replace the implementation of ListingNavigator.set_pending.
<rick_h_> abentley: ok, looking
<rick_h_> abentley: I don't suppose there's any way around creating a bunch of fake bugs in my dev checkout locally to see/tests out this spinner stuff?
<abentley> rick_h_: there are already fake bugs, and you can force the batch length unnaturally low by specifying the "batch" query parameter (an int).
<rick_h_> ah, ok cool. that helps
<abentley> I created a bunch of fake bugs automatically.  I can show you later.
<rick_h_> abentley: yea, wasn't sure if there was a helper or a specific project with more bugs that would page
<rick_h_> setting batch=2 breaks the JS for me. get_current_batch returns undefined. Looking
<gary_poster> hey lifeless.  Are you already aware of some internal service errors on the oops tool (https://lp-oops.canonical.com/oops.py/?oopsid=6e5e984a017534cb393e47e50b93fe95 for example)?  I don't know the current log/dev story for the oops tools and am happy to leave you to it since you are working in that neck of the woods--OTOH I'm also willing to dig in if desired
<abentley> matsubara: Are you sure navigation is working for you on chromium, but the spinner is not working?
<abentley> matsubara: the symptom I see on chromium is that navigation is not working at all, which of course means the spinner never starts.
<matsubara> abentley, I meant the spinner is not working on chromium. If I click any of the sort wdigets, the list is sorted but I get no feedback
<matsubara> abentley, and you're correct, the batch navigation doesn't work on chromium either
<rick_h_> abentley: nothing works for me on chrome
<rick_h_> I'm debugging now, I've gotno next state
<abentley> matsubara: are you sure the list is sorted on chromium?  It's not for me.
<rick_h_> since chrome pops history on page load, checking to see if the model gets backed off or osmething
<matsubara> abentley, yes, it takes awhile but it's sorted
<matsubara> abentley, Chromium 15.0.874.106 (Developer Build 107270 Linux) Ubuntu 11.10
<matsubara> it seems batching works too, just takes some time to update (no spinner though)
<matsubara> oh, scratch that. I'm looking at the wrong place
<rick_h_> matsubara: ok, that matches me then
<matsubara> ok, on qastaging with listing pre-fetching and dynamic bug listing enabled I get no batch nav, no sorting and no spinner
<abentley> matsubara: So I suspect this is not a bug in the spinner at all, but a bug in the new History-based model code.
<rick_h_> abentley: right, the way the batches are indexed isn't coming out in chrome it looks like
<rick_h_> abentley: the ["-importance", xxx] as a key to an object?
<abentley> rick_h_: right.
<rick_h_> so get_current_batch ends up returning undefined and blowing up getting anything from there
<abentley> rick_h_: Yes, that's how it looks.
<wgrant> Is this broken on prod now?
<rick_h_> wgrant: no, doesn't look like it
<abentley> wgrant: my theory is it's broken by r14385
<wgrant> abentley: Have you tried reverting it locally?
<abentley> wgrant: doing that now.
<abentley> wgrant: Yes, reverting it locally seems to work.
<wgrant> OK. Do we turn the beta off for a few hours and deploy, or add a second revert and deploy tomorrow?
<abentley> wgrant: I was going to do a rollback.
<wgrant> We need to roll it back regardless, but do we care enough about preserving the beta to delay deployments for another day...
<abentley> wgrant: Not sure I agree.  If we turn off the beta, there isn't the same urgency to do a rollback.
<wgrant> I guess.
<abentley> wgrant: I don't know which deployments are key here.  We could deploy 14384 or we could turn off dynamic bug listings and deploy 4395.
<abentley> s/4395/14395
<abentley> flacoste: We have bugs in the feature that prevent us from deploying anything after r14384.  wgrant suggests we might turn off the feature in order to deploy more stuff.  Thoughts?
<jcsackett> benji: could i get a review of https://code.launchpad.net/~jcsackett/launchpad/fancy-filebug/+merge/83636
<benji> jcsackett: sure
<jcsackett> thanks!
<flacoste> abentley: what's the issue with 14385?
<abentley> flacoste: It breaks navigation in Chromium.
<flacoste> would be nice to have a browser scope selector :-)
<flacoste> browser:chromium 0
<flacoste> anyway, let's turn the beta off for a day then
<lifeless> bah, missed gary
<abentley> flacoste: Cool.
<flacoste> abentley: 14392 is also bad?
<abentley> flacoste: Yes.  Not my proudest day :-).  It breaks navigation on the alternate listings for a Person, such as Commented Bugs.  I landed a rollback this morning, but I don't think it's in stable yet.
<flacoste> abentley: we should put out a comment on the blog post and identi.ca that the beta is disabled for a day or two though
<abentley> flacoste: I do have an actual fix for 14392 nearly completed, but I just found out about 14385.
<flacoste> abentley: another possibility would be to simply leave the feature on
<flacoste> and deploy anyway
<flacoste> if it was easier to turn the feature off, that would probably the best thing to do
<wgrant> abentley: Do you know what your squad's plans for +bugs-index are?
<abentley> flacoste: I think we should try to avoid breaking all our beta users who use Chromium.
<wgrant> (that's the fairly pointless page with the top-10-hot-bugs listing on eg. https://bugs.launchpad.net/launchpad)
<flacoste> wgrant: they are replacing it with a real listing
<flacoste> i think
<flacoste> iirc
<abentley> wgrant: It seems likely we'll make it the same as the productgroup index page, but we haven't investigated the specifics yet.
<flacoste> abentley: right
<wgrant> abentley: Right, which is just +bugs. That would make my life much easier.
<abentley> wgrant: It will have the same bug listings as the other pages.  Details to come.
<wgrant> abentley, flacoste: If there was a checkbox for people to opt-out of the beta easily, I'd say leave it on. Since there isn't, I think we should turn it off until it's fixed.
<flacoste> agreed
<abentley> wgrant: Such a checkbox was originally part of the plan, but we didn't have time.
<wgrant> abentley: We need a general solution, anyway.
<wgrant> poolie and I have discussed it vaguely.
<wgrant> abentley: I suspect the easiest way to fix +bugs-index is just to port a few things across to +bugs (mostly just displaying a warning when the product doesn't have bugs enabled, I believe) and then setting +bugs as the default.
<benji> jcsackett: the notification_array.join(' ') bit is kinda funny; how about something like http://paste.ubuntu.com/752973/
<jcsackett> benji: sure.
<jcsackett> benji: that's pushed up now.
<benji> jcsackett: looks good
<wgrant> abentley: So, we're OK to deploy despite the two qa-bads?
<abentley> wgrant: Yes.
<wgrant> abentley: Great, thanks.
<abentley> flacoste: I am trying to update the blog, but I cannot log in.  Every time I try, it just prompt me to log in again.
<flacoste> abentley: weird, if you have the text, i can past put it in
<jcsackett> benji: i have another review, if you have time. very short: https://code.launchpad.net/~jcsackett/launchpad/404s-also-private/+merge/83639
<benji> jcsackett: sure
<abentley> flacoste: "Update: we have temporarily suspended the beta, but we'll have it back in the next day or so."
<benji> jcsackett: done, I included a -- possibly nonsensical -- thought that ocurred to me as well
<jcsackett> benji: it would make sense, but in this context we actually know the branch exists, or we would have encountered earlier errors.
<benji> k
<Noldorin> hi poolie
<Noldorin> has Markdown support landed on LP yet?
<abentley> flacoste, wgrant: rollback is on devel, as r14404
<wgrant> abentley: Thanks.
<lifeless> sinzui: sorry for rabbiting on :)
<sinzui> lifeless, your explanation was informative. I am glad you have thought about it
<flacoste> abentley: updated
<abentley> flacoste: thanks.
<Noldorin> anyone else besides poolie know?
<flacoste> Noldorin: it's landed, but i don't think it's deployed yet
<flacoste> and if it's deployed it's not enabled for sure
<Noldorin> flacoste, ah ok. i'm not really familar with the launchpad deployment process. are "merged", "landed", "deployed", "enabled" all separate steps in that order?
<flacoste> Noldorin: sorry, about the jargon
<flacoste> merged and landed are synonymous
<flacoste> the branch was merged in trunk
<flacoste> we deploy from trunk several times a week
<Noldorin> that's what i thought
<Noldorin> ah, got it
<Noldorin> flacoste, and what does it take for the feature to get enabled?
<flacoste> "enabled" refers to some features that we "feature flag"
<flacoste> basically, they have a control switch
<flacoste> for example, the new bugs listing
<flacoste> the switch is on only for certain users (members of the beta team)
<Noldorin> ah okay, that's what i was wondering about
<Noldorin> so it gets beta-tested live first
<flacoste> and earlier, we turned it off because of problems related to the code that is being deployed
<Noldorin> then enabled for the public once stable
<Noldorin> i guess
<flacoste> yes, that's the idea
<Noldorin> flacoste, sounds good. thanks for explaining :-)
<flacoste> but it really depends on the feature
<flacoste> for example, given the limited markdown functinality
<Noldorin> i'm supposing that for a simple(ish) feature like Markdown support, it should only be a week or two before enabling
<flacoste> we might just turn it on for everybody
<Noldorin> cool
<flacoste> and turn it off if any problems arise
<lifeless> Noldorin: this particular case needs a security audit of the markdown library first
<Noldorin> lifeless, hasn't that already happened?
<lifeless> no
<Noldorin> does it take long?
<lifeless> depends on the code size
<lifeless> time etc
<lifeless> not really a question that can be answered ;0
<Noldorin> lifeless, well, not unless you're the person doing the review :-P
<lifeless> even if you are
<Noldorin> well i debate that
 * lifeless shrugs
<Noldorin> if you are, then you ought to have a good idea
<Noldorin> of what it should take
<lifeless> I'll be doing part of it
<lifeless> I don't know how long it will take.
<Noldorin> not a perfect idea
<Noldorin> lifeless, well have you looked at the code for the lib yet?
<lifeless> you can tell me I should know, but I assert that I don't.
<lifeless> no, I haven't.
<Noldorin> don't be so defensive, i'm not telling you that you should know :-P
<Noldorin> i'm speaking in hypothetical terms here
<Noldorin> well if you haven't read the code yet, i certainly wouldn't expect it
<lifeless> I'm not being defensive :)
<Noldorin> if you had on the other hand, then yes i *might*...
<Noldorin> it seemed like it, but okay sure
* benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugtasks: 3*10^2
<poolie> hi all
<poolie> flacoste, adding a 'user-agent' feature scope should be pretty easy
<poolie> or perhaps one that just generically matches the request header
<flacoste> yeah, that was my guess
<StevenK> sinzui, wallyworld_, jcsackett: https://code.launchpad.net/~stevenk/launchpad/better-error-open-owner/+merge/83550
<StevenK> Rargh, rf-get should run make clean before updating just to stop all these conflicts.
<StevenK> poolie: 71 packages upgraded, 3 newly installed, 0 to remove and 0 not upgraded.
<StevenK> poolie: I think we should freshen the image.
<poolie> already?
<poolie> which image did you use?
<StevenK> 522
<poolie> hm, current is supposed to be 523
<poolie> i wonder why you don't see it
<StevenK> Probably because I didn't merge devel. This may lead to test failures.
<poolie> ...
<poolie> it shouldn't be necessary
<poolie> it is looked up on the network
<poolie> what does 'ec2 images' show, run from devel
<StevenK> 522, wgrant
<poolie> ok
<poolie> it was private
<poolie> despite i thought giving the option to make it public
<poolie> StevenK, try again?
<poolie> even just 'ec2 test --trunk' to see what happens
<StevenK> Now 523 shows up
<StevenK> However, that description is massive.
<poolie> 50 bytes!
<poolie> do you know how much those things cost?
<StevenK>   523  ami-f721e99e  957911449157  mbp          launchpad ec2test created 2011-11-23 08:32:11 UTC by 'mbp@canonical.com' on joy
<StevenK>   522  ami-7fa56916  873925794399  wgrant       Created 2011-10-08 00:52:32 UTC
<StevenK> It wraps and means it is harder to parse.
<poolie> but you know who to blame :)
<poolie> anyhow, patches welcome
<poolie> lifeless, good idea about splitting out ec2
<StevenK> I knew who to blame before! :-P
<lifeless> StevenK: doesn't wrap for me :P
<poolie> wish you'd told me before i went through pqm to land changes to it :)
<StevenK> lifeless: I run in 80x24 like we're SUPPOSED to.
<StevenK> :-P
<lifeless> StevenK: if god intended you to have a crippled terminal, he would have given you an amber screen
<lifeless> :)
<StevenK> I was waiting for "The 80s called, they want their terminal dimensions back."
<lifeless> The 80s called, they want their catchphrase back.
 * lifeless goes meta
<StevenK> Anyway, TERM says 'xterm', so it should act like one.
<lifeless> StevenK: indeed, 200x50 or so :)
<lifeless> no you have me doing history lookups for DEC VS100's
<StevenK> Heh heh
<lifeless> VS100 - original target for xterm
<lifeless> 19" 1088x864 pixels
<lifeless> https://docs.google.com/viewer?a=v&q=cache:3At_IEDM3YQJ:www.bitsavers.org/pdf/dec/graphics/VS100_Engineering_Specification_Jun83.pdf+&hl=en&pid=bl&srcid=ADGEEShzKk3TN13uySW6boAVQV9NfXdE7yBdoz9k77EtaSSpAJEZlhwNp-AlqRFIyLwRHnX04fWx1m-S7z6rTZmpMBP2QOIUen3NyC-946yC_hrybqELz0lOuJvrXJ5ENEfXbGj7dlp-&sig=AHIEtbR5d6Vm7ElUXQOozYGhpMEhr64iOA
<lifeless> so, > 80x24 :)
<lifeless> poolie: does https://code.launchpad.net/~python-fixtures/python-zope-fixtures/trunk have anything you need ?
<lifeless> poolie: if it does, I can cut a release now, otherwise I'm planning on waiting a little for additional bits to land
<lifeless> anyone want to have a go parsing https://bugs.launchpad.net/launchpad/+bug/897442 ? My brain glitches
<_mup_> Bug #897442: maintained packages page not up to date after several weeks <Launchpad itself:New> < https://launchpad.net/bugs/897442 >
<poolie> lifeless, um
<poolie> i added a new fixture to launchpad's own fixtures.py
<poolie> you could copy that out to the separate project
<poolie> in r14372 of devel
<poolie> i'm not blocking on getting anything else
<StevenK> poolie, jelmer: Did you guys have a plan of attack to QA r14397?
<poolie> sigh
<poolie> iirc that is mostly about an update to the webapp, not the builders?
<poolie> so we ought to be able to test it by adding such a recipe on qas ?
<StevenK> [r=mbp][ui=none][bug=891928] Update bzr-builder to the latest upstream revision.
<StevenK> Fixes:
<StevenK>     Bug:891928
<StevenK> Since bzr-builder is used on the buildds to build recipes ....
<poolie> but that's separate
<poolie> that's one of the things that is confusing
<poolie> StevenK, i will qa it now
<poolie> StevenK, 'qa' is so confused
<poolie> 'is it fixed' vs 'is it safe'
<wgrant> It is "is it safe"
<wgrant> It just has a bad name.
<poolie> so this fix doesn't work
<poolie> but as long as it doesn't obviously regress anything else
<poolie> it is probably ok to deploy
#launchpad-dev 2011-11-29
<StevenK> Right
<poolie> done
<StevenK> We have to wait for abentley's rollback to hit qas too
<wgrant> Nope.
<wgrant> We've turned the feature off on production.
<StevenK> So we can deploy to 14399, or would you prefer to wait?
<wgrant> I'd prefer to deploy ASAP.
<wgrant> The two rollbacks are qa-ok if they only affect the disabled feature.
<StevenK> There's two rollbacks? :-(
<wgrant> Yes.
<poolie> is it just me or is there something strange on lp where the font size varies on some pages?
<poolie> maybe it's just my browser
<poolie> hm
<poolie> i'm going to add a timeout to bzrlib tests
<poolie> it's pretty generous, 300s
<poolie> i wonder if this will hurt lp
<poolie> 300s per test
<poolie> *600s
<lifeless> how are you going to do this?
<poolie> https://code.launchpad.net/~mbp/bzr/test-timeout/+merge/83559
<poolie> it crashes the whole test run
<poolie> :/
<poolie> probably better than hanging
<poolie> what do you think/
<poolie> lp could turn it off
<poolie> or we could have it off by default
<lifeless> interesting
<lifeless> I have a few different angles here
<lifeless> firstly lp - you can check the longest test run by introspecting a subunit stream from a failed ec2 run
<lifeless> longest duration of a single test, I mean
<poolie> true
<lifeless> add 50% or so and we're safe.
<lifeless> secondly, I think this would be good to put into testtools and/or fixtures
<lifeless> it has no need to be bzrlib specific
<lifeless> and its really quite nice
<poolie> also true, i thought about that too
<poolie> it is a bit abrupt
<lifeless> e.g. fixtures would happily accept a timeout fixture which alarms if not turn down in time
<poolie> but i don't see any better options
<poolie> yep
<lifeless> you can use signal.signal to inject an exception when SIGALARM is received, I suspect
<poolie> we don't currently depend on fixtures
<poolie> we could
<poolie> probably yes
<poolie> that could make it tidier
<poolie> at the expense of perhaps not terminating in some cases
<lifeless> so the contract could be TimeoutFixture(duration, callback)
<poolie> eg if you are stuck in C code
<lifeless> perhaps yes
<poolie> as we actually were in the case that motivated me to write this
<poolie> mm
<poolie> it might still get it unjammed
<poolie> it could be an option
<poolie> if you review https://code.launchpad.net/~mbp/testscenarios/module-scenarios/+merge/80287 i will be more likely to send you more patches :)
<lifeless> anyhow those are my thoughts - share the niceness more widely, and check the longest run LP
<lifeless> has
<lifeless> poolie: its right up on my list actually, I nearly got to it yesterday.
<poolie> "you're boring me... BANG"
<poolie> lifeless, ok https://code.launchpad.net/~mbp/python-fixtures/timeout/+merge/83721
 * lifeless cues the on hold music
<poolie> for the diff?
<poolie> what's up with the longpoll stuff for that?
<lifeless> no, for you :P
<poolie> i see a lot of noise about it
<poolie> oh ok
<lifeless> we're about to go live
<lifeless> anyhow, immediate feedback is that this is much more than test specific; (haven't seen the diff yet)
<poolie> oh?
<lifeless> well its a good generic 'I'm doing something and I might not come back' facility, wrapped up in a context manager
<lifeless> so I'd like to drop the focus on tests in the prose around it
<poolie> mm
<poolie> i see your point
<poolie> i think it is really for emergency use
<lifeless> your readme says TimeoutFixture
<poolie> i wouldn't encourage using it widely
<lifeless> your body tests TestTimeout
<poolie> until for instance it copes properly with several things happening at once
<poolie> you'reright
<lifeless> I prefer the former (and jml is arguing for just Timeout, for this case)
<lifeless> poolie: I think its fine to have something with caveats
<lifeless> poolie: the caveats are real, and devs can make informed choices.
<poolie> you're right that aside from that it is not test specific
<lifeless> 211+ import pdb;pdb.set_trace()
<poolie> just 'Timeout' is ok with me
<poolie> heh
<lifeless> is possibly not intended to be merged
<lifeless> its amazing how loudly cats can snore
<poolie> i don't really want to fix that in this branch :)
<poolie> or indeed at all
<lifeless> cat snoring? Indeed :)
<poolie> so given all this, perhaps the sample data does not need to be tests but just functions
<lifeless> make sense to me
<poolie> that's quite nice they'renot test specific
<poolie> actually that's more testable not going through the double-testing layer too
<poolie> ok re-pushed
<lifeless> poolie: I wonder whether a callback is more useful than an exception
<lifeless> poolie: e.g. a trivial callback can raise, a different one could log a marker about how far through (whatever task) things were when the alarm triggered
<lifeless> perhaps I'm overthinking this
<wallyworld_> huwshimi: i've updated the demo to use local storage to persist changes between page refreshes, and added in editing on the details page. let me know if there's anything else
<huwshimi> wallyworld_: Awesome, will do.
<poolie> lifeless, maybe so
<poolie> perhaps that can wait until someone has a need for it
<lifeless> wgrant: pop quiz; lp_serve and codehosting sections in schema.conf, both run as same uesr right ?
<wgrant> lifeless: They should, yes.
<wgrant> lifeless: Check the owner of their existing OOPSes?
<huwshimi> Does anyone want to help me think about bug statuses/colours/grouping? Voice, or IRC if multiple people are keen.
<huwshimi> wgrant, poolie: I think both of you have discussed colours etc on IRC in the past
<huwshimi> wait, that should have been bug status colours and status grouping
<wgrant> huwshimi: Sure.
<wgrant> Hopefully poolie's around too.
<huwshimi> wgrant: OK thanks, let's give it a minute and see if poolie wants to join in
<huwshimi> wgrant: Or I can just ramble here
<lifeless> here is good
<huwshimi> sure
<huwshimi> So, we're going to try out moving the status into the top line of the new bug listings. We'll sit the status next to the importance and style it similarly to the status. We need to figure out some colours for statuses. So...
<huwshimi> The colours for the importance will probably stay the same (at least at this point). Colours do a good job of signifying different types of importance.
<huwshimi> Even though importances are really one type of "thing"
<huwshimi> We don't want the status colours to fight with importances so we could make them tones of one colour, or we could group statuses so that there are only a few colours
<huwshimi> The status groups I came up with were the following:
<huwshimi> Initial (New, Incomplete, Opinion, Confirmed, Triaged)
<huwshimi> Rejected: (Invalid, Won't Fix)
<wgrant> Opinion is Rejected
<huwshimi> wgrant: Ah thanks
<wgrant> Triaged is not Initial.
<huwshimi> The other two groups were: working (In Progress) and done (Fix Committed and Fix Released)
<wgrant> Right.
<huwshimi> wgrant: Would you break up that inital group? I'm mostly trying to identify a group of statuses that are essentially the same thing
<wgrant> Triaged certainly needs to be separate.
<wgrant> I don't know about the others.
<huwshimi> wgrant: So in a workflow it probably wouldn't move between statuses in the same group
<wgrant> huwshimi: Fix Committed goes to Fix Released, New to Incomplete.
<huwshimi> wgrant: Ah you're right
<wgrant> Fix Committed, Won't Fix, Opinion and Confirmed have no good reason to exist.
<lifeless> huwshimi: 'same thing'
<lifeless> huwshimi: have you seen the issue tracker draft spec ?
<huwshimi> lifeless: Yeah, I was just looking at that
<huwshimi> wgrant: In a sense could you say that In Progress and Fix Committed are kind of the same thing?
<lifeless> huwshimi: there is indeed an effort ongoing to reduce the number of statuses
<huwshimi> wgrant: They're both a "this bug is being worked on" statuses, just with a different tense
<lifeless> huwshimi: it is perhaps tricky to do just now- because of the ongoing debate about the value of the nuance
<huwshimi> lifeless: Yeah, in some ways this might be a step towards that (having similar colours)
<lifeless> huwshimi: we'd expect many 'new->incomplete', 'new->opinion|invalid|wontfix', 'new->triaged' transitions
<lifeless> many projects don't use in-progress /fix committed at all AFAICT
<lifeless> etc
<wgrant> I think most projects would use In Progress.
<wgrant> Fix Committed is probably rare.
<lifeless> wgrant: in progress is bit twiddling overhead
<huwshimi> lifeless: I guess the thing here is that the colours won't force anyone into any kind of workflow, but might (if people notice the subtleties) help people see what stage the bug is at.
<lifeless> so for a quick segue
<lifeless> there there a few interesting dimensions around a bug and lifecycle
<lifeless> there is the progress bar dimension
<lifeless> there is the what-is-it-waiting-for dimension
<lifeless> there is the 'when will it be usable' dimension
<lifeless> to illustrate the difference, consider an LP bug
<lifeless> its 'done' when its released and switched on via flags
<lifeless> to get there it may wait on user input many times (some of which may use the INCOMPLETE state)
<lifeless> it may wait on dev, design, qa inputs
<lifeless> it may wait on deployment pipeline
<lifeless> and it may be usable for a user once its enabled for beta users
<lifeless> so, I think its important to decide *which* dimension you are trying to quantise when you think about this
<lifeless> otherwise you'll tie yourself, and our users, up in knots :)
<lifeless> (and note that 'status' as its currently defined doesn't map directly to -any- of these dimensions)
<lifeless> I wish I had some helpful insight here - sorry !
<huwshimi> lifeless: That is helpful. I still think we can provide some hints to what part the cycle of the bug is at. There's a group of bugs that need some extra info/management/discussion before they can be worked on (New, Triaged, Incomplete, Confirmed). There's the bugs that have been rejected (Invalid, Won't Fix, Opinion). There's bugs that are being worked on (In Progress, Fix Committed). And there are bugs that are do
<huwshimi> ne (Fix Released).
<huwshimi> I know that doesn't necessarily map to everyone's workflows
<huwshimi> but I think it's correct in terms of what we have in Launchpad right now
<huwshimi> And I
<huwshimi> The result of those categories is just chosing a colour for each status. Do Invalid, Won't Fix and Opinion really deserve an individual colour or can they be lumped together in a "rejected" colour
 * micahg wonders why opinion still exists
<wgrant> micahg: I think we all do.
<huwshimi> haha
<wgrant> huwshimi: Triaged bugs are ready to be worked on.
<lifeless> huwshimi: 'triaged' bugs can be worked on'
<huwshimi> ah, yes
<lifeless> huwshimi: as can confirmed [they just don't have an importance set]
<lifeless> huwshimi: I think the 'reject' and 'being worked on' sets are good
<lifeless> huwshimi: arguably rejected + fix-released is one grouping (bugs that are finished with)
<huwshimi> lifeless: I thought the same about bugs that are finished with, but I think it would be good to distinguish between them as they have pretty opposite outcomes
<lifeless> huwshimi: sure, I buy that; was being a bit socratic :P
<huwshimi> :)
<huwshimi> So do you think I would get away with having 5 status colours? For each of: Unconfirmed (New, Incomplete), Ready (Triaged, Confirmed), Rejected (Invalid, Won't Fix, Opinion), Being worked on (In Progress, Fix Committed), Done (Fix Released)
<huwshimi> (with unconfirmed, ready etc being labels I'm only using to help explain right now)
<wgrant> That sounds reasonable.
<lifeless> +1
<huwshimi> wgrant, lifeless: Awesome, thanks for the help :)
<lifeless> anytime
<StevenK> Is doctest an actual upstream Python module?
<lifeless> yes
<lifeless> its in the stdlib
<StevenK> Oh, ew.
<StevenK> Ah ha! poolie!
<StevenK> 14337.1.1   mbp@can | from testtools.testresult.real import _details_to_str
 * StevenK tries to remember the magic he used to get Person:+index working in a test harness.
<StevenK> Stupid RDF.
<wgrant> Not RDF, but XRDS.
<jtv> wgrant: are you reviewing today?  If so: ooh, pick me, pick me!  https://code.launchpad.net/~jtv/launchpad/bug-849683-patchup/+merge/83729
<wgrant> jtv: The cloner is used ~twice per release.
<wgrant> To perform archive rebuilds.
<jtv> That explains.  Well, no more problem anyway, with this branch.
<wgrant> Actually, IDS uses it too.
<wgrant> So three times a release :)
<wgrant> jtv: What changed in lib/lp/soyuz/doc/buildd-queuebuilder-lookup.txt?
<wgrant> There's a big block of changes where I can't actually see any differences.
<jtv> Lint.
<wgrant> Yeah, but I can't see what.
<wgrant> Oh.
<jtv> Got indented too far (I guess by the doctest formatting tool), and thus overran the 78th column.
<wgrant> Indended by 4 instead of 5.
<wgrant> Yeah.
<StevenK> AH
<jtv> Was it 5?  Ah yes.  Then it wasn't even the formatting tool, I guess.
<jtv> StevenK: âAHâ?  are you okay?
<StevenK> jtv: I was wondering too
<wgrant> jtv: Approved, thanks for the fixes and cleanups.
<jtv> Thank you.
<lifeless> allenap: wgrant: teamparticipation updates are racy; thats why it gets out of sync I suspect.
<lifeless> allenap: or at least, may explain. Requires select for update to fix.
<poolie> hi all
<wgrant> lifeless: That's possible, but pretty unlikely.
<poolie> StevenK, yes that was me
<poolie> i even saw the lint warning
<poolie> exporting the symbol seemed likely to overflow my stack
<poolie> i guess i could have suppressed it?
<lifeless> its a rather noddy warning TBH
<lifeless> we have it disabled for most of LP
<poolie> you could more reasonably warn it is an underscored symbol...
<poolie> but
<poolie> it is more of a warning of "may break in the future" rather than "is broken now"
<poolie> let the future fix its own bugs, we have enough of our own
<poolie> huwshimi, lifeless, like robert said there are a bunch of different dimensions there
<poolie> ah
<poolie> i agree with a lot of that stuff
<poolie> i will suggest one more thing
<poolie> which is that some statuses are inherently more interesting than others, most of the time
<poolie> specifically the 'in play' statuses of new, inprogress, fixreleased
<poolie> *fixcommitted, i mean
<poolie> and maybe incomplete
<poolie> they are more likely to be something some one wants to work on next
<poolie> perhaps they should be more prominent
<StevenK> poolie: It annoys me that bin/test prints it every time
<poolie> does it?
<poolie> the import warning?
<StevenK> Yup
<poolie> guys, help me decide about the tarball download feature
<poolie> it does not appear on the default lh home page only on per-revision pages
<poolie> i could fix it and update
<wgrant> bug
<poolie> or, i could fix a bug saying it's not in the lp main ui
<poolie> which is ultimately more interesting
<poolie> bug 240580
<_mup_> Bug #240580: Ability to download a tarball for a revision <qa-ok> <Launchpad itself:In Progress by mbp> <loggerhead:In Progress by mbp> < https://launchpad.net/bugs/240580 >
<poolie> you just closed it wgrant :)
<wgrant> Both? :)
<poolie> or, no, maybe that was something else
<huwshimi> poolie: Thanks. I'm a little hesitant to start placing importance on certain statuses, just because I don't understand enough about the ramifications on some people's workflows (I know it'd just be a small UI thing, but it's one less issue with the UI I want to possibly introduce right now).
<lifeless> EWHAT: database/schema/tst.dot
<poolie> that also sounds reasonable
<poolie> what do you specifically want to do next?
<poolie> huwshimi, i'd be curious about your thoughts on my mail about a Thanks button
<poolie> more from a ux than implementation perspective
<poolie> not likely to happen soon though
<lifeless> poolie: doing it the lh main ui would be nice; doing it in the LP main ui is also nice. I don't see them as the same bug, and whichever you choose to work on first is up to you :)
<poolie> i think i will mark this bug complete and file followons
<huwshimi> poolie: My initial reaction is that it would be a nice, friendly, community building touch. I think it would add a nice rewarding experience to using Launchpad.
<poolie> :)
<lifeless> jtv: hey, is there an existing collection of scripts for doing db schema evolution w/slony the way we do ?
<jtv> lifeless: no idea â haven't followed how our schema stuff interacts with slony tbh
<lifeless> the scripts get wrapped in slonik
<lifeless> so things that want to use ORMs etc to generate db patches - well they can't ;)
<lifeless> they could if they emit the schema changes to a stream, but not if they depend on commit/rollback etc
<lifeless> or on processing data mid-migration
<lifeless> however there are a raft on pypi
<lifeless> random link of the day
<lifeless> http://bulbflow.com/overview/
<jtv> Maybe once we're on 9.x we can look into streaming replication.
<lifeless> yeah
<lifeless> stub wants to wait for a bit for 9.x, some slony  issues that concern him
<jtv> Yes.  I do still water at the mouth with new postgres release summaries though.
<lifeless> :)
<poolie> lifeless, nice
<jtv> lifeless: the staging oops report contains an entry that makes me wonder if you've set the timeout threshold too lowâ¦
<jtv>     -1.00s  OOPS-fead2780a43b01845f361d2acbb79f14  Unknown
<lifeless> jtv: hah
<lifeless> :)
<jtv> It took minus one second and _still_ it times out.  That's going to be hard to optimize.
<lifeless> jtv: its all about i
<jtv> Are we squaring our times then?
<poolie> call cern
<jtv> poolie: good point.
<lifeless> jtv: oops tools applies a heuristic
<lifeless> jtv: it appears to be glitched - https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-fead2780a43b01845f361d2acbb79f14
<lifeless> jtv: note the oops actually doesn't report any timing data whatsoever
<lifeless> jtv: and by glitched I mean '-1 to avoid confusion'
<jtv> GAHHH sorry I have to focus on something else right now.
<jtv> Grrr all these notify() functions and all this time the test wasn't calling the right one.
<lifeless> ugh
<lifeless> database/schema depends on canonical.*
<lifeless> twitch
<StevenK> How?
<lifeless> from canonical.launchpad.scripts import db_options, logger_options, logger
<lifeless> from canonical.database.sqlbase import connect, ISOLATION_LEVEL_AUTOCOMMIT
<lifeless> from canonical.database.postgresql import fqn
<StevenK> Ah
<lifeless> yes
<lifeless> so, making a separate tool will involve some way to bridge our global magical config settings and the separate tool
<StevenK> You want to pull database/schema out entirely?
<lifeless> the driving logic, yes.
<lifeless> microservices need DB's and slony.
<lifeless> I need a name though
<lifeless> lazr_postgresql I guess
<lifeless> (refusing to namespace on grounds of being sane)
<jtv> wgrant: if Ubuntu syncs a source package from Debian and then builds it, would SPR's dsc_signing_key and dsc_maintainer_rfc822 be the Debian maintainer's?
<wgrant> jtv: signingkey is probably empty. I'm not sure what maintainer is... it depends what gina does.
 * wgrant hunts.
<wgrant>         spr = SourcePackageRelease(
<wgrant>             section=sectionID,
<wgrant>             creator=maintainer.id,
<wgrant>             component=componentID,
<wgrant>             sourcepackagename=name.id,
<wgrant>             maintainer=maintainer.id,
<wgrant> So the maintainer should be the Maintainer field from the imported source package.
<bigjools> morning guys
<jtv> Good morning
<jelmer> Goeiemorgen jtv
<jtv> Goedenmorgen!
<bigjools> double dutch
<danhg> Morning
<mrevell> Hey
<adeuring> good morning
<poolie> hi all
<poolie> bigjools, is rabbit appropriate for storing thing indefinitely?
<poolie> maybe it is, i just had the idea it was for in-flight messages
<bigjools> poolie: not at all, it's just a transport mechanism
<poolie> so it's rabbit as a path to a service storing stuff in psql or whatever
<bigjools> it's just a very useful way of firing and forgetting
<poolie> memories of Dallas :)
<bigjools> I only shot Osama, no rabbits :)
<lifeless> bigjools: would need a query API too for it; so if doing that might use http for the store as well (given that we probably want to know the audit was committed :P)
<bigjools> lifeless: rabbit has a reliable commit mechanism IIRC
<lifeless> bigjools: if you use it in rpc mode its equivalent to an http call
<bigjools> why not just stick to Rabbit then?
<lifeless> bigjools: if you use it in persistent message fire-and-forget-but-tell-me-that-there-is-at-least-one-subscriber then its potentially lossy unless you have an HA story for it
<lifeless> bigjools: and we don't have an HA story for it
<bigjools> and do we have HA story for some future storage mechanism?
<lifeless> slony
<lifeless> (thats yes, if we use postgresql, or yes if we use cassandra, etc)
<bigjools> do slaves stay up if the master goes?
<bigjools> can you store data in a HA fashion?
<lifeless> for slony, yes and no - slaves stay up, master being down prevents writes
<bigjools> which is my point
<lifeless> this is different to rabbit in the fire and forget mode where messages can disappear entirely
<lifeless> so I don't see why its your point
<bigjools> my point is that nothing has HA for storage
<bigjools> so why not just use a single API over rabbit instead of an API for this, another API for that ...
<lifeless> uhm, cassandra does
<lifeless> if we need HA writes
<bigjools> we don't use cassandra
<lifeless> this discussion is maddening
<lifeless> I'll chat with you another day
<bigjools> why?
<bigjools> you don;t like my questions?
<lifeless> I don't mind questions, but the goal posts seem to be moving
<lifeless> I'm talking about being sure the audit item is logged
<bigjools> what goal posts?
<lifeless> you're talking about being up all the time
<lifeless> they are different things
<bigjools> umm, you started with the HA thing
<bigjools> not me
<lifeless> I started with lossiness
<lifeless> not with HA; to avoid loss of messages with rabbit you need to either be using it in RPC mode, which is rather pointless, or you need to an HA story for it
<bigjools> we might as well just use a separate Storm store to talk to an auditing PG
<lifeless> well
<lifeless> that wouldn't be reusable by other services and wouldn't have higher integrity due to needing direct access from the main appservers etc
<lifeless> what we have with rabbit today is a mostly-reliable messaging syste,
<lifeless> which is great
<lifeless> we don't have guaranteed delivery except in rpc mode where you wait for a reply from the consumer
<lifeless> the main impact this has is that when we really care about a message being delivered, we need to either use rpc mode or not pass it via rabbit; either is ok
<stub> Technically, our writes to PostgreSQL can be lost. Slaves are lagged, and if we lose the master we lose the lag time's amount of data changes.
<lifeless> true, though if we lose the change we made, and keep the audit trail for it, thats arguably less of a problem than keeping the change and losing the audit trail
<stub> I'd consider looking at celery, which seems to allow you to use various things as the back end such as rabbit or postgresql
<stub> I had a skim the other day since abently keeps mentioning it and it looks nice. I like projects that take their documentation seriously.
<lifeless> bigjools: sorry for the maddening comment; ELONGDAY + we are/were crossing wires
<lifeless> bigjools: I usually value teasing things apart and getting more clarity :(
 * lifeless heads to be
<lifeless> d
<lifeless> (PS: cynthia had a bad night last night.... so we all did)
<bigjools> lifeless: I have jetlag, you have a baby.  Great combo :)
<lifeless> indeed!
<lifeless> have a good day
 * lifeless goes
* gmb changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: gmb | Critical bugtasks: 3*10^2
<jml> lifeless: fwiw, re "Fixture" suffix: my main concern is that I use TempDir and EnvironmentVariableFixture the most, and either the former should be TemporaryDirectoryFixture or the latter should be EnvVar (or at least EnvironmentVariable). Too much to remember otherwise.
<allenap> wgrant: Regarding your comment on https://bugs.launchpad.net/bugs/897269 about check-teamparticipation.py. I don't want to have to manually fix TP ever, so I think it's worth fixing it automatically as well as having the report.
<_mup_> Bug #897269: check-teamparticipation.py does not fix spurious participation records <teams> <Launchpad itself:In Progress by allenap> < https://launchpad.net/bugs/897269 >
<mpt> "This membership is going to expire in 7 days from now"
<allenap> vila: You have put a big smile on my face :) https://bugs.launchpad.net/bzr/+bug/843211
<_mup_> Bug #843211: With pipelines I cannot set push_location and public_branch generally <config> <Bazaar:Fix Released by vila> < https://launchpad.net/bugs/843211 >
<bigjools> bug 872112
<_mup_> Bug #872112: Buildd-manager combines DB write transactions and asynchronous calls <soyuz-build> <Launchpad itself:Triaged> < https://launchpad.net/bugs/872112 >
<vila> allenap: glad you like it ! Other kind of feedback also welcome :)
<StevenK> I think everyone who uses pipelines will buy vila a beer ...
<allenap> Indeed!
<StevenK> bzr push errors with can't do it / branch doesn't exist, so I sigh and run sync-pipeline
 * vila coughs
<vila> make sure it works as you expect, there may still be some shaloow glitches
<vila> shallow
<StevenK> Since I've stopped landing so many Soyuz branches, I've stopped using pipelines so much. I wonder why that is ...
<allenap> Does anyone know where I can find (in order to modify) the oops-tools-django-dependencies or oops-tools-django-testsuite-dependencies packages? I am a total newb to this stuff.
<mrevell> matsubara, ^ ?
<matsubara> allenap, bzr+ssh://bazaar.launchpad.net/%2Bbranch/lazr-source-dependencies/
<allenap> matsubara: Thanks :)
<matsubara> hmm that came from bzr info and looks wrong. the right branch is lazr-source-dependencis
<matsubara> allenap, https://launchpad.net/lazr-source-dependencies
<allenap> matsubara: Cool.
 * allenap contemplates filing a bug about not finding that package from site-wide search.
<flacoste> allenap, matsubara-lunch: lazr-source-dependencies is deprecated, use lp-source-dependencies
<allenap> flacoste: I was about to ask matsubara-lunch about that, but for another reason. I was looking for the oops-tools-django-dependencies and oops-tools-django-testsuite-dependencies packages, but they're not in either lazr- or lp-source-dependencies. Do you know anything about them?
<flacoste> allenap: aren't they in the meta-lp-deps project?
 * allenap looks
<abentley> gary_poster: How I manually get the SimpleViewClass-munged version of a view class?
<flacoste> allenap: where did you get these packages from? apt-cache search doesn't find them over here?
<allenap> flacoste: They're in the python-oops-tools PQM chroot, so it's conceivable that they were made by a LOSA.
<flacoste> allenap: yeah, i don't know where these packages are maintained? did you look on the oops-tools deployment wiki page if there is any info on them?%
<allenap> flacoste: I'll have a look.
<flacoste> https://dev.launchpad.net/QA/OopsToolsSetup
<flacoste> allenap: i don't see anything there
<allenap> Ta.
<flacoste> if you find the answer, please update that page :-)
<allenap> flacoste: Will do :)
<allenap> mthaddon: Do you know what the lineage of the oops-tools-django-dependencies and oops-tools-django-testsuite-dependencies is? I've just seen that the former is installed on carob. It's also installed in the python-oops-tools PQM chroot (wrt. https://rt.admin.canonical.com//Ticket/Display.html?id=49246).
<mthaddon> allenap: I think they've been losa-maintained up til now and are just in lucid-cat (should be able to download the source from any server in the DC)
<rick_h_> abentley: no, I've muted everyhting, just listening
<abentley> rick_h_: no, you haven't.  Your lips are red.
<rick_h_> yea, but you can't hear me right? abentley
<abentley> I'm not hearing you saying anything.  I'm just hearing echo.
<rick_h_> ok, well I've muted all inputs at pulse
<abentley> rick_h_: You were still outputting to Mumble.  I guess it's possible to do that without using Pulseaudio, but it seems unlikely.
<allenap> mthaddon: Ta, thanks.
<rick_h_> abentley: adeuring http://www.youtube.com/watch?v=_zhQIfT7g58&feature=youtube_gdata_player is the video I mentioned, good stuff
<adeuring> thanks
<abentley> rick_h_: cool, thanks.
<rvba> Hi gary_poster, I'm investigating a timeout (bug 890927) and it turns out the problem is the repeated security check queries (line 2234 in lib/canonical/launchpad/security.py).  Jeroen said I should talk to you about this because it's a matter you've worked on previouslyâ¦ so here I am :)
<_mup_> Bug #890927: private PPA Archive:+index timeouts <timeout> <Launchpad itself:In Progress by rvb> < https://launchpad.net/bugs/890927 >
<bigjools> I thought they were cached these days?
<rvba> I think they are *somewhat* cached.  But each package on the page generates a security check query.
<rvba> In this case, the exact same query is repeated.
<bigjools> that is a harder problem to solve
<bigjools> I guess this is the "published" check?
<rvba> Right, hence Jeroen's advice to talk to Gary about it.
<bigjools> if you can already see the page, there's little point doing those checks
<bigjools> since all the necessary security has been passed already
<rvba> It's the  ViewArchive launchpad.View check.
<rvba> True.
<bigjools> right, so the view check is done for the archive, and then the same archive again and again for each package
<rvba> Exactly
<bigjools> a dirty fix would be rSP :)
<rvba> Dirty indeed :), a more generic caching mecanism would be nice here.
<rvba> mechanism*
<rvba> Besides, the call comes from a place that I think is called from many other places which might need the security checks.
<rvba> Working around that might be possible but even more dirty.
<bigjools> rvba: are you thinking of a cache of (obj, permission_name) ?
<rvba> Also, solving this elegantly might bring more performance improvements elsewere.
<bigjools> true
<rvba> bigjools: right now, honestly I don't know if that's really possible but yeah, I was thinking about having a LRU in the request for that.
<rvba> bigjools: but I need to understand how the caching in place (which does not work in this case for some reason) works first.
<gary_poster> rvba, I'm sorry I missed this
<gary_poster> rvba, I'm trying to use Empathy as my IRC client and not having success with notifications yet
<gary_poster> rvba, should I still look at the backlog and try to figure out what is going on :-)
<rvba> gary_poster: no worriesâ¦ maybe you can give me a few hints to understand how perm check caching works and what you think can be done in this case.
<gary_poster> ok lemme read backlog
<rvba> thanks gary_poster.
<abentley> gary_poster: How I manually get the SimpleViewClass-munged version of a view class?
 * gary_poster tries to remember these details...
<gary_poster> abentley, if I understad you correctly, that is what you will get from the adapter lookup
<gary_poster> You can get the adapter registry itself
<gary_poster> and do a lookup there without actually instantiating the class
<gary_poster> if that helps
<abentley> gary_poster: I have been trying to do the lookup manually, but I don't know exactly what to look up.  Here's what I'm trying: https://pastebin.canonical.com/56497/
<gary_poster> rvba, so, I'm not ignoring you, but abentley's question is easier :-P
<rvba> gary_poster: I figured :)
<gary_poster> abentley, I'd suggest getting the site manager.  that has an adapter registry on it for adapter lookups (and a separate one for utilities)
<rvba> gary_poster: I'm not expecting a solution but hints btw
<gary_poster> ack
<gary_poster> abentley, that gives you an interface that's closer to metal
<gary_poster> and I think it will be easier for you to use
<gary_poster> the interface is described in zope.interface IIRC
<gary_poster> abentley, as to the traceback, I don't know enough about what the cls is and so on to have an opinion as to what the problem is
<abentley> gary_poster: the cls is defined at the top of the pastebin.
<gary_poster> but I suggest dropping that and working with an interface that lets you specify the interfaces.
<gary_poster> abentley: duh, sorry.  in that case, I'd guess the problem is that +base-layout-macros is registered for something else.  The adapter registry will give you the raw materials to anwer that question directly though.
<gary_poster> I'm more than happy to dig in with you, abentley, but I'm trying to re-set up my LP environment with LXC, so I don't have a working environment atm
<gary_poster> and I should try to help rvba
<abentley> gary_poster: undertood.  I used +base-layout-macros because it's registered for *, which I thought meant anything.
<gary_poster> yes abentley, you are right.  I don't have an immediate answer; I would have to dig too.  :-/
<gary_poster> rvba, so, IIRC, we cache our security lookups in the security policy.
<rvba> gary_poster: I see that in lib/canonical/launchpad/webapp/authorization.py.LaunchpadSecurityPolicy
<gary_poster> right rvba.  Maybe if I look at that I will better understand the problem.
<gary_poster> Because ATM I don't see what isn't working
<rvba> gary_poster: me neither but maybe I'll try debugging it to see what's going on â¦ then I'll come back to you. How does that sound?
<rvba> gary_poster: the thing that I don't get is where the cache is exactly. I see a local object named cache being manipulated in checkPermission (same file).
<gary_poster> rvba, cool.  I'm interested in the IApplicationRequest check, because that's the only thing that jumps out at me as a way for the cache to be ignored.  Although...these objects are in fact different, right?
<rvba> I suspect my problem is that I've plenty of different objects being checkedâ¦ and it turns out the check is the same because the objects are all related to the same archive.
<bigjools> gmb, you need to thank me.  It took *me* 5 hours to review jtv's buildd-manager branch.
<gary_poster> rvba, right.  So, that would lead to a few sorts of solutions
<bigjools> the _underlying_ check is common
<gmb> bigjools: You mean the 1200 line one that StevenK appeared to have started reviewing?
<gary_poster> rvba, am I right that actually instantiating the IAuthorization adapter is the expensive bit here?
<rvba> bigjools: exactly.
<bigjools> gmb: that is the badger
<gmb> bigjools: Thank you for taking care of it.
<gary_poster> rvba, can we state categorically that these objects get their permissions from some other object?
<gary_poster> I believe we have an existing mechanism for that sort of thing
<bigjools> gmb: :)  You didn't actually need to thank me, I am just feeling relieved I finished it
<gmb> bigjools: (I daresay that it would have needed input from you or your fellow Soyuz experts either way)
<gary_poster> I do not know how efiicient it is
<bigjools> gmb: actually it was more of a Twisted domain change
<rvba> gary_poster: yes, SourcePackagePublishinghistory -> Archive.
<gmb> bigjools: I was really glad when I saw StevenK's name attached to it; I guess that made me overlook the fact that he hadn't actually reviewed it...
<bigjools> lol
<gary_poster> rvba, ok, what is their IAuthorization now?  I think bac recently did some work on IAuthorizations that can explicitly pass their work off to another object.  We should make sure that this takes advantage of the cache
<rvba> gary_poster: the expensive part is the query, triggered by the call to getUtility(IArchiveSubscriberSet).getBySubscriber in checkAuthenticated of ViewArchive (lib/canonical/launchpad/security.py)
<gary_poster> rvba, do you happen to have a stack trace for that?
<rvba> gary_poster: that sounds like a solution.
<rvba> gary_poster: yes, hang on.
<rvba> gary_poster: http://paste.ubuntu.com/753787/
<flacoste> rvba: are you qa-ing Bug:887078?
<flacoste> once that's clear, we can deploy and re-enable custom bugs listing
<rvba> flacoste: I'm on it.
<gary_poster> rvba, ok cool.  That looks like we could take advantage of the cache pretty easily.  Lemme see if I can find the magic authorization adapter...
<gary_poster> (I see you are doing something else, np)
<rvba> flacoste: rarg, in fact I QAed it this morning, I simply forgot to change the tag. Sorry about that.
<rvba> Qa-ok.
<rvba> gary_poster: cool!
<gary_poster> I'm not sure which objects we are talking about here, but I suspect that we are already using DelegatedAuthorization, which I bet is what I was looking for.  I'm slowed down by having to install stuff like ctags, sorry
<bigjools> gary_poster: not using pycharm? :)
<gary_poster> bigjools, with my new-ish Clojure (lisp-y) interest, emacs has been getting my attention these days :-)
<bigjools> run away!
<gary_poster> :-)
<gary_poster> rvba, ok yeah, this is interesting
<gary_poster> lib/lp/app/security.py has DelegatedAuthorization
<gary_poster> it delegates to the raw authorization adapters
<gary_poster> thereby skipping our cache entirely
<rvba> Makes sense.
<gary_poster> so...
<gary_poster> we can think about this a few different ways
<rvba> ViewSourcePackagePublishingHistory inherits from DelegatedAuthorization and delegates the checks to obj.archive.
<gary_poster> right
<gary_poster> my first coherent thought is that this abstraction bay be in the wrong place
<gary_poster> may
<gary_poster> Because it feels like the security policy ought to be in charge of its cache
<gary_poster> so *it* ought to be able to see "oh look, it is delegated"
<gary_poster> and *it* can look in the cache
<gary_poster> and if it doesn't exist, do the adapter dance
<gary_poster> that's less flexible than what we have now
<gary_poster> but it ought to be a whole heck of a lot faster for cases like this
<gary_poster> So let's think about an alternate case for a sec though
<gary_poster> let's say we wanted to keep DelegatedAuthorization as is
<gary_poster> in order to do that...
<rvba> Maybe instead of calling checkAuthenticated in there we could use the checking method from LaunchpadSecurityPolicy
<gary_poster> not only would it need to know about the cache, but it would need to be able to convert the cache answer to a real answer
<gary_poster> because the cache answer is for a gievn participation
<gary_poster> which means a given user
<gary_poster> checkAuthenticated would need to look at the participation
<gary_poster> (the request)
<gary_poster> and compare the participation's user with the user it has
<gary_poster> if they are the same, then we can use the cache
<gary_poster> if not, we have to do our usual checks
<gary_poster> that would be relatively easy to code, but it feels icky to me
<rvba> this might be good enough for my problem :)
<gary_poster> yeah
<gary_poster> I don't love it, but it feels practical
<gary_poster> let's talk about what feels nicer to me for just a sec though :-)
<gary_poster> and see if something can fall out there
<rvba> sure :)
<rvba> I'm fuzzy about this participation thing thoughâ¦
<gary_poster> well, first, do you see what I mean about why it would be nicer, or am I on crack
<rvba> What does it means exactly?
<gary_poster> oh sure
<gary_poster> so, the zope security system has this idea that multiple participants (think "principals," users or groups) can be involved in a security interaction.  This allows modeling people granting permissions without escalating permissions.  If I give you the ability to do something, and you try to do it, the security system should consider my permissions and your permissions, or else we are potentially in a privilege escala
<gary_poster> tion scenario
<gary_poster> that is unimprtant to us
<gary_poster> we only allo w a single participation
<gary_poster> and that is a request
<gary_poster> which represents a security interaction between a user and the system
<rvba> ok, hence the whole len(participations) > 1: raise RuntimeError("More than one principal participating.")
<gary_poster> typically for the length of a transaction
<gary_poster> precisely
<gary_poster> so that's a participation
<rvba> ok, thanks gary_poster.
<gary_poster> is that clear-ish now, or should I try again? :-)
<gary_poster> ok cool
<gary_poster> so, my concern is that the cache on the participation is an artifact of the security policy
<gary_poster> We could expose the cache in some way explicitly I suppose
<gary_poster> Although...
<gary_poster> Well, I have an old dislike of grabbing the request from the air
<gary_poster> (thread local)
<gary_poster> and I'd like to do that in as few places as possible
<gary_poster> but anyway
<rvba> gary_poster: why would we need the request? I suppose you mean because we need to get the user, but the user is provided when checkAuthenticated is called right.
<rvba> ?
<rvba> So if we expose the cache like you said, we could use it in DelegatedAuthorization.checkAuthenticated
<gary_poster> rvba, we need the request, because it is the participation, and it has the cache that the security policy sets up.  So if the authorization itself needs to check the cache, it needs to pull the request up
<gary_poster> right rvba.  It woud ideally be some kind of utility, I guess, if we did it that way
<rvba> Ah ok, the security policy is on the request itself.
<gary_poster> rvba, not quite, the cache is.  Let's look back at authorization.py
<rvba> wd = participation.annotations.setdefault(LAUNCHPAD_SECURITY_POLICY_CACHE_KEY,weakref.WeakKeyDictionary())
<rvba> cache = wd.setdefault(objecttoauthorize, {})
<gary_poster> LaunchpadSecurityPolicy is our security policy, obviously, Less obviously, instantiating it creates the interaction.
<gary_poster> And yeah, there's the participation.
<gary_poster> == request
<gary_poster> with the cache
<gary_poster> rvba, fwiw, I'm fine with a call of some sort too if you like, if you think it would go faster.  Fine with irc too :-)
<rvba> gary_poster: I was about to suggest the same thing :)
<gary_poster> :-)
 * rvba fires up skype.
<rvba> skype ok?
<gary_poster> yeah, cool rvba
<gary_poster> I'm garyposter on Skype, rvba
<rvba> gary_poster: I'm trying to find how to add a new contact, hang on :)
* gmb changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugtasks: 3*10^2
<sinzui> matsubara, Can you do exploratory testing of bug 885692. I think this bug fixes bug 603732 and bug 375331 too
<_mup_> Bug #885692: bug supervisors have more power than maintainers and admins <bugs> <disclosure> <qa-ok> <Launchpad itself:Fix Released by stevenk> < https://launchpad.net/bugs/885692 >
<_mup_> Bug #603732: If there is no driver a maintainer should be able to accept a bug nomination <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/603732 >
<_mup_> Bug #375331: Some extra filebug options don't show up for projects not having a bug supervisor <bugs> <projects> <Launchpad itself:Triaged> < https://launchpad.net/bugs/375331 >
<matsubara> sinzui, yes
<rick_h_> abentley: adeuring heads up, running out for lunch, bbl.
<abentley> rick_h_: okay
<poolie> hi all
<rick_h_> howdy poolie
<poolie> hi there
<poolie> what are you up to now rich_h_?
<rick_h_> poolie: chilling in a conference room in AL while beating my head on a wall trying to figure out why this one tests fails
<poolie> oh you're with gary?
<rick_h_> hate that "hmm, one last test and all done..." the last ones know how to poke you
<poolie> or deryck?
<rick_h_> not atm
<poolie> yeah
<rick_h_> but hopefully later this wek
<rick_h_> week that is
<poolie> or spurious failures after a long test run
<abentley> gary_poster: The ultimate solution to my problem: http://pastebin.ubuntu.com/754062/  (I did not realize that view were multi-adapters).
<poolie> o/ abentley
<gary_poster> abentley: ah, sorry, I should have spotted that.  Glad you found it.
<abentley> poolie: \o
<dobey> who do i bug exactly, to fix problems i see with vcs imports? or should i just bug someone to add me to that team so i can fix them?
<gary_poster> abentley, fwiw, I still think you will like the end result of your code better if you use the raw adapter registry
<poolie> fix in the sense of failures, or they are just misconfigured, dobey?
<poolie> if you want to help garden them i'm sure that would be welcome
<gary_poster> dobey, have you already mentioned something in Launchpad questions?  If so, I apologize, we're trying to work through the list.
<dobey> poolie: well in this case it's misconfigured
<abentley> gary_poster: This is test code, and it's passing, so I don't understand what more I'd want.
<gary_poster> abentley: oh ok.  I thought you were making a tool.  What you'd want more: not to have to create a throw-away object.
<poolie> jelmer, how could we do that for dobey?
<dobey> gary_poster: no, i just spent ~30 minutes or so looking at 2.5 year old code that i thought was new :(
<gary_poster> dobey :-( gotcha
<dobey> gary_poster: and my understanding is that ~vcs-imports is a place most people don't want to be
<abentley> gary_poster: Oh, I thought the API was the same whether I did it directly or on getGlobalSiteManager().
<jelmer> poolie: it looks like ~canonical-bazaar is admin for ~vcs-imports, so we should be able to add another member :-)
<dobey> and it doesn't help that i can't just configure a new import if someone else already owns a branch which i would like to set up as an import in the proper place
<gary_poster> dobey, it's certainly true that we don't have experts on that code any more. :-/
<jelmer> poolie: I wonder if we should make package imports and code imports more similar in this regard. code imports would benefit from out-of-date warnings as well.
<dobey> so i guess even if i was on that team, i am not sure i would be able to fix this specific import
<gary_poster> abentley: the API on the site manager is similar, but as I said, the site manager has adapter registries on it
<gary_poster> that API is different
<gary_poster> (the API of the adapter registries)
<jelmer> dobey: the main disadvantage of being in ~vcs-imports is that you get a dozen extra emails per day, with some spikes
<jelmer> dobey: what's the import?
<dobey> https://code.launchpad.net/glib
<dobey> jelmer: lp:glib, but it looks like someone set up an lp:glib/2.26 pointing at the git master
<dobey> which is really annoying, because git master is not 2.26
<dobey> and the series are all messed up on that project
<dobey> and other projects seem to have some imports set up, but don't have the series bits set up, so lp:foo doesn't work on them
<jelmer> dobey: ~vcs-imports won't really help for that, vcs import admins can only really change the source URL of code imports and remove them
<jelmer> dobey: ~registry can change a large set of projects that aren't hosted on Launchpad, including glib
<rick_h_> abentley: ping
<abentley> rick_h_: pong
<dobey> hmm
<rick_h_> abentley: know you're busy, but wanted to see if you get a sec if you could try out my branch with the indicator js plugin for the loading graphic?
<abentley> rick_h_: sure, where is it?
<rick_h_> abentley: it works here, but it feels fragile with css and curious if it's what was intended from the discussions
<rick_h_> https://code.launchpad.net/~rharding/launchpad/buglists-loading-885272
<rick_h_> abentley: I had to update the css combine template
<rick_h_> so make sure you regen that or add in the indicator assets css file
<rick_h_> I'm not 100% sure how that gets "updated" for people
<dobey> jelmer: and i have a feeling ~registry is not a team i can be in :)
<jelmer> dobey: so, in general, it seems like we are a bit too strict on who can change products that aren't hosted on Launchpad but merely tracked
<jelmer> s/on/about/
<dobey> right
<dobey> it seems there are a lot of defunct teams set up to deal with these, too
<jelmer> dobey: I think it's just ~registry :)
<sinzui> dobey, any canonical employee can be in the team if they needs to manage the 1100 projects that other projects like Ubuntu rely on
<dobey> https://launchpad.net/~gnome-python-maintainers is the "maintainer" for pygobject it seems, but the team has no administrator
<jelmer> dobey: I do think you could probably be in ~registry but you should probably check with one of the launchpad folks. Curtis is particularly active in ~registry
<sinzui> dobey, ~launchpad currently dominates membership, but a few years ago, it was Ubuntu staff
<dobey> ah
<sinzui> dobey, you are now a member of ~registry. You will see lots of edit icons on projects now
<dobey> sinzui: oh, ok; thanks
<abentley> rick_h_: the makefiles updates combo.css.  yui3-overlayindicator is present for me.
<dobey> is there any way to deal with the issue of having the same branch of git being imported by multiple people (or rather, that it can't be)?
<rick_h_> abentley: right, but combo.css is generated from bin/combine-css which I had to manually add a path to
<rick_h_> abentley: and combin-css comes from the buildout templates
<abentley> rick_h_: bin/combine-css is also generated by the makefile, AIUI
<jelmer> dobey: euhm, sortof
<jelmer> dobey: launchpad's URL checking is pretty naive, so if there are multiple URLs for a repository they can each be registered
<rick_h_> abentley: ok, cool, sounds like you're good
<abentley> rick_h_: I'm not seeing the spinner in firefox, and chromium looks busted.
<rick_h_> abentley: any errors? I'm seeing them in both.
<rick_h_> abentley: this branch doesn't have the backport for the chrome bug
<dobey> jelmer: hrmm
<rick_h_> so to view it I had to just manually change the html to disable the -disabled css
<abentley> rick_h_: Uncaught TypeError: Cannot read property 'next' of undefined.
<rick_h_> abentley: right, that's the chrome issue from yesterday
<rick_h_> try to merge with devel I guess to correct that part
<abentley> rick_h_: If you merge stable, that should go away.
<rick_h_> abentley: I just cheated and hand changed the css rule hiding the overlay
<rick_h_> abentley: right, just didn't do that today yet. Thanks
<rick_h_> abentley: the image is in the center of the overlay'd div
<rick_h_> so make sure to check for it in the center of the table
<dobey> jelmer: i guess i also need to be a member of vcs-imports to be able to configure a branch as owned by vcs-imports?
<dobey> jelmer: or should i just make ~registry the owner?
<abentley> rick_h_: It should be visible from the top and the bottom, so if being vertically centred makes it hard to see, we probably shouldn't do that.
<rick_h_> abentley: ok, so the stuff that this was working on was creating a single div overlaying the updating content with the spinner
<rick_h_> abentley: we've got a larger spinner, 32px that's used with the grey'ing out of the content to show it's going
<jelmer> dobey: ~registry is fine as a branch owner as well
<dobey> ok
<abentley> rick_h_: I don't know what the answer is.  Maybe two spinners.
<wgrant> rick_h_: Did you change the main spinner recently?
<rick_h_> wgrant: yes, I needed one that would work on a grey background
<rick_h_> wgrant: I guess Huw said to pick one until he got around to making one
<wgrant> rick_h_: Aha
<wgrant> Is it slightly larger than the old one?
<rick_h_> wgrant: we've got two versions now, a 16 and 32px versions of the same gui
<rick_h_> wgrant: shouldn't be, the px size is exact
<wgrant> The old one was already too big (it caused the page to jump around), but this one seems larger.
<wgrant> hmm.
<rick_h_> it might push to the edge of the px more than the last, but it's exactly at 16 like the old
<rick_h_> abentley: ok, does it show up for you though?
<rick_h_> abentley: I guess more "does it work" for you and then we can go after the asthetics some
<abentley> rick_h_: Unclear.  I merged stable and now I'm dealing with the aftermath.
<rick_h_> abentley: lol, ok. I'm running on juice with my mifi, so going to head to a coffee shop I think.
<rick_h_> abentley: so back online in a bit
<abentley> rick_h_: cool.
<dobey> sinzui: hrmm; i can't seem to change the development focus on glib.
<sinzui> dobey, ~registry does not maintain it
<sinzui> dobey, Robert Ancell is in ~upstream, and he is also in ~registry
<dobey> right, ok
<sinzui> dobey, ~jjardon can add you to the team
<sinzui> dobey, we made him the maintainer of several gnome projects. He will happily add people who can help fix the series.
<dobey> is "release url pattern" on a series a regex, or just a glob?
<dobey> i guess a glob, otherwise .* could have different results i suspect
<lifeless> its a glob
<dobey> is there some way to specify a git branch to import now, rather than it implicitly always being master?
<wgrant> Add ',branch=foo' to the end of the URL
<dobey> ah ok
<rick_h_> bah, coffee shops need more than 6 tables. Walk out there and no space
<dobey> hrmm, imports failing
<dobey> http://launchpadlibrarian.net/86214321/registry-glib-trunk.log
<jelmer> dobey: the URL is incorrect, the /git/ bit shouldn't be there for git.gnome.org IIRC
<dobey> jelmer: oh, hrmm; it's there for the git+ssh versions
<jelmer> the standard git server doesn't error out when a repo doesn't exist, it just hangs up, hence that error message
<jelmer> yeah, git://git.gnome.org/glib.git seems to work, git://git.gnome.org/git/glib.git doesn't
<dobey> jelmer: did you update the URLs somehow? i didn't see how to do it, so i just deleted the glib branch and recreated it, but i see the gobject-introspection ones are working now
<jelmer> dobey: there should be a link down the page "Edit import source or review import"
<jelmer> dobey: maybe you have to be in ~vcs-imports to see it
<dobey> jelmer: i don't see those.
<mwhudson> yeah, that's ~vcs-imports only i think
<jelmer> dobey: I don't think it's a problem for us to add you to ~vcs-imports
<abentley> rick_h_: So, it looks nice, and the grey overlay helps a lot.  But it seems to activate only on the first load, in both Firefox and Chromium.
<dobey> jelmer: ok :)
<mwhudson> unless you are allergic to email :)
<rick_h_> abentley: ok, will look
<dobey> mwhudson: i have procmail :)
<mwhudson> dobey: lucky you get any email at all then!
 * mwhudson doesn't trust procmail
<mwhudson> well, my ability to drive it, really
<dobey> mwhudson: well, i get the mail i care about :)
<abentley> rick_h_: easiest way to trigger a load is probably the sort buttons.
<rick_h_> abentley: ok thanks. Yea it looks like the show/hide is triggered. Not sure if it's too fast or just not showing
<rick_h_> abentley: ah, nvm, I see it. Thanks for the heads up
<abentley> rick_h_: np.
<rick_h_> abentley: ok, pushed fixes, the indicator spinner dom element was removed via the ajax response :)
<rick_h_> abentley: if you get a sec to see if that looks better for you, we can even but huwshimi and see if this is what we wanted and if I should try to land it
<poolie> huwshimi, hi, i'd like to have a link to download a tarball of branch contents, within the main lp ui
<poolie> can you help me work out where that ought to go (both which page and where on the page?)
<huwshimi> poolie: Sure
<poolie> there is a bug...
<poolie> bug 897537
<_mup_> Bug #897537: no link to download branch tarball within the main lp ui <branches> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/897537 >
<poolie> huwshimi, so...
<poolie> i can imagine this would lead in to a much larger redesign
<poolie> perhaps without end
<poolie> actually you know all this
<poolie> do you want to talk about it now or just reply on the bug later?
<huwshimi> poolie: Let's have a quick chat now
<huwshimi> poolie: So this link is to download the source of any branch at any revision?
<poolie> yes
<poolie> with obviously an important special case being getting the source of the tip of the branch, and an important case of that being getting the tip of trunk
<huwshimi> poolie: And can we only show links only on loggerhead or can we have them in our main ui as well?
<poolie> perhaps, until lp has good main-webapp per-revision pages, we're better off focussing on just getting the tip, and people can click through to loggerhead to get past revisions
<poolie> not quite sure what you  mean
<poolie> the point of the bug is that we ought to show them in the main ui
<poolie> the urls are predictable so the main ui can just make them up
<huwshimi> poolie: Right, I'm just checking we can actually do that
<huwshimi> ah awesome :)
<poolie> i think :)
<huwshimi> hehe
<huwshimi> So I imagine it would be useful to display a "download the source" link for on https://code.launchpad.net/loggerhead and https://launchpad.net/loggerhead/+download and https://code.launchpad.net/~debian-bazaar/loggerhead/experimental
<huwshimi> And probably http://bazaar.launchpad.net/~debian-bazaar/loggerhead/experimental/files
<poolie> yep
<poolie> that last one, bazaar.l.n is loggerhead, so that's a separate issue
<poolie> specifically bug 897535
<_mup_> Bug #897535: tarball download link only appears on per-revision page <loggerhead:Triaged> < https://launchpad.net/bugs/897535 >
<huwshimi> yeah
<huwshimi> I would be a little hesitant to put source download links on project overview pages (https://launchpad.net/loggerhead). Mostly as it might create confusion as to how to get the code properly (most of the time we don't want people to just download the code, we want them to grab a bzr branch).
<poolie> yep
<poolie> <https://code.launchpad.net/~debian-bazaar/loggerhead/experimental> is already a bit "omg what do i do now"
<poolie> :/
<poolie> perhaps part of the problem there is that the headings are not very visually distinct?
<poolie> the font is barely any larger and the whitespace is not helping
<huwshimi> actually now that I think about it if we add a get the source link on https://launchpad.net/loggerhead/+download it should probably be a link to https://code.launchpad.net/loggerhead so people can decide whether they want a branch or a tar ball
<poolie> yes
<poolie> https://code.launchpad.net/loggerhead has a link back
<poolie> i think both of them should try to say "or perhaps you want a branch/a tarball instead"
<poolie> it is worthwihle i suppose
<poolie> it feels a bit like a kludge for people ending up on the wrong page :)
<poolie> but i suppose it's natural people will go to a 'wrong' page that as they explore
<huwshimi> I'd be inclined to be fairly specific with the wording too. On a branch page I would write:
<huwshimi> Get this branch:
<huwshimi> bzr branch lp:~debian-bazaar/loggerhead/experimental
<huwshimi> or download .tar.gz
<huwshimi> or something along those lines
<poolie> yeah
<poolie> so
<poolie> often, people should either get an actual release tarball, or they should get a proper branch
<poolie> but if they do want a tarball, it ought to be possible to find how to do that
<poolie> so +download links to the branch list
<poolie> the branch page offers you a tarball with the wording you gave above
<poolie> and... should the list-of-branches page do anything?
<huwshimi> Do you mean this page: https://code.launchpad.net/bzr ?
<huwshimi> If so I would think it would be useful to have a .tar.gz link on that page too. I imagine some people will be using branches to host content that people would want to get at without using bzr (if it was a js library or some design files etc.)
<huwshimi> The wording would be a bit trickier as there is already "There are download files available for Bazaar."
<poolie> so as a special case, it offers you a tarball of the development focus?
<poolie> wfm
<poolie> the text on that page is an interesting case
<poolie> the freeform text above the table
<poolie> because it mixes together
<poolie> stats
<poolie> things you're supposed to take action on, if you're a developer
<poolie> "perhaps instead you want... download files"
<huwshimi> poolie: Yeah, we already have the special case of "bzr branch lp:bzr" so this would fit with the download of the same branch
<poolie> instructions, and navigation
<huwshimi> poolie: I know I was just trying to decipher everything going on there
<poolie> so i think it all blurs together
<poolie> :/
<poolie> perhaps we should just delete/hide the stats
<huwshimi> Active reviews probably should be a link in the sidebar. It is a "related content" link
<abentley> rick_h_: sorry, EOD.  Let's chat about it tomorrow.
<lifeless> poolie: I'm consdering coming to the xmas part
<abentley> rick_h_: (End Of Day)
<rick_h_> abentley: sure thing, I'll send an email in case huwshimi gets any time to look at it UI wise
<poolie> lifeless, awesome
<lifeless> poolie: wondering about couch camping or something
<poolie> i will ask
<poolie> lifeless, you're welcome to
<poolie> stay here
<poolie> no warranty as to comfort :}
<StevenK> lifeless: Sarah and I can probably find somewhere for you too, if you wish
<rick_h_> huwshimi: email on the way, if you get a sec to check it out. I'll be in/out online tonight so feel free to ping in irc if you want to chat
<rick_h_> huwshimi: and I'll catch you when I get back (going to hit dinner and such in a bit)
<huwshimi> rick_h_: Cheers, will do.
<poolie> huwshimi, ok this sounds like a plan: on the branch list page
<poolie> get rid of the counters
<poolie> active reviews and download files into the portlets
<poolie> ...
<poolie> can we get rid of the first sentence?
<poolie> maybe....
<poolie> add 'browse', 'tarball' links to each row?
<poolie> or
<huwshimi> poolie: It might be nice to make the format of that first sentence like it is on a branch page, with a few wording changes to make it clear this is just the dev focus
<poolie> maybe add tarball items only for series branches?
<huwshimi> "get a copy of the branch using the command" is a nice way to put it, but maybe a little redundant
<poolie> huwshimi, ok that sounds like a plan
<poolie> anything else?
<huwshimi> poolie: I think that all sounds pretty good
<lifeless> flacoste: around ?
<lifeless> flacoste: quick question if so
#launchpad-dev 2011-11-30
<huwshimi> I really, really, really, really wish we had an A/B testing framework in Launchpad
<rick_h_> heh, do we have the metrics to be able to put A/B testing to use?
<huwshimi> rick_h_: That's what I mean
<huwshimi> we have feature flags that could be used to set up the A and B, but we need random user assignment and result reporting to make it useful
<rick_h_> yea, it's a tough product to define what metric we'd want for each A/B case
<rick_h_> what's the current use case?
 * rick_h_ is bored and curious
<huwshimi> rick_h_: A proper a/b testing framework allows you to hook in tests for results. The test could be "number of people who get to a certain page" or "number of people who click a link" etc.
<rick_h_> huwshimi: yea, but LP is so different depending on who visits/use case that it's a bit fun to think of what would be better
<huwshimi> rick_h_: Isn't that the point of doing an A/B test?
<huwshimi> poolie: Could the feature flags system be modified so that instead of showing to a group it could pick users at random?
<poolie> sure could
<poolie> i would like that very much
<poolie> there is a bug asking for it
<huwshimi> poolie: Ooh
<poolie> i am happy to either help or to be bribed/distracted in to doing it
<poolie> but i need to go out and do some trip prep stuff so probably not today
<huwshimi> poolie: Wow, ok, this makes getting an A/B testing framework possible. It would be farily easy to write the javascript data collection side of things
<poolie> the other thing we need is to log it somehow
<poolie> either from the http server or something else
<poolie> maybe js
<huwshimi> poolie: Yes, there would need to be an AJAX call to log the resaults
<huwshimi> *results
<huwshimi> rick_h_: Are you still around?
<huwshimi> rick_h_: If you are, I have a conflict when I merge your changes for the loading notification. Any ideas? http://paste.ubuntu.com/754337/
<rick_h_> huwshimi: looking
<huwshimi> rick_h_: Cheers
<rick_h_> huwshimi: bah, yea I moved a couple things that it's not liking, sec
<huwshimi> np
<rick_h_> huwshimi: http://paste.mitechie.com/show/455/ is what it should be. That's merged with rocketfuel-get this evening so should be up to date
<huwshimi> rick_h_: Cheers :)
<huwshimi> rick_h_: Hmm... I lose the order/config bar completely
<rick_h_> huwshimi: make sure that your css combo script in bin gets updated.
<rick_h_> huwshimi: yea, make sure the css combo script is updated and you've got "indicator" css in the combo
<huwshimi> ah I see
<huwshimi> hmmmm....
<huwshimi> rick_h_: Still nothing
<huwshimi> rick_h_: The indicator css is there and working
<rick_h_> huwshimi: any hints?
<huwshimi> rick_h_: But still no order bar
<huwshimi> rick_h_: The order bar div is completely empty
<huwshimi> oh wait, I have js errors
<StevenK> poolie: I'm a little unsure about the branch you've put up
<rick_h_> huwshimi: ok, cool. Maybe typo in cleaning up the merge?
<StevenK> poolie: It looks mostly UI related, so I'd prefer huwshimi look it over.
<poolie> StevenK, ok with me, huwshimi do you mind?
<poolie> he reviewed the predecessor anyhow
<poolie> btw i'm happy to say google is now apparently enjoying the results of the microformat work
<huwshimi> poolie: Sure, mind linking me to the mp?
<StevenK> huwshimi: https://code.launchpad.net/~mbp/launchpad/888353-microformats-2/+merge/83856
<huwshimi> StevenK: Thanks
<huwshimi> I'll take a look soon
<poolie> thanks
<poolie> ok i'm out for a bit, see you later
<huwshimi> rick_h_: on this line "this.set('model', new namespace.BugListingModel({" it says undefined is not a function""
<rick_h_> huwshimi: in chrome?
<huwshimi> rick_h_: yea
<rick_h_> try on firefox in case you're hitting the chrome issue
<rick_h_> there's a known chrome issue that is in progress of rollback/fix
<rick_h_> unrelated to the spinner
<rick_h_> but BugListingModel is defined in line 37, and this sholdn't be undefined, so not sure what it's not liking there
<huwshimi> rick_h_: Firefox says: "namespace.BugListingModel is not a constructor"
<rick_h_> huwshimi: can you pastebin your whole file then? Something is up. It's defined back in 52
<rick_h_> huwshimi: http://paste.mitechie.com/show/456/ if you want to rnu diff locally
<huwshimi> rick_h_: Oh, something's very wrong with my file. It's about 70 lines shorter :)
<rick_h_> huwshimi: ah ok. Yea, that'll do it
<huwshimi> oh, there we go.
<huwshimi> rick_h_: just pasted over your new file and it works now. Thanks :)
<rick_h_> huwshimi: re-pushing my branch is case I missed somehting
<rick_h_> huwshimi: ah ok
<huwshimi> rick_h_: Thanks, I might grab your new branch. This is still not working for me.
<huwshimi> (in other ways now)
<rick_h_> huwshimi: ok, yea should just be able to  bzr branch bzr+ssh://bazaar.launchpad.net/~rharding/launchpad/buglists-loading-885272/
<rick_h_> into your lp-branches dir, update the links, and then make run
<huwshimi> rick_h_: Have you just remerged the branch with devel?
<rick_h_> huwshimi: sec I will
<rick_h_> did it this afternoon, but will redo a rocketfuel-get on it and repush
<huwshimi> cheers
 * rick_h_ is still figuring out how best to work around stuff
<StevenK> rick_h_: rf-get in your devel branch, and then bzr merge ../devel in your other branch
<rick_h_> huwshimi: ok, updated and pushed
<huwshimi> rick_h_: Cheers mate
<rick_h_> StevenK: ok, I've been doing some rf-get right in my branch itself, bad side effects?
<StevenK> rick_h_: I'm not sure -- that ^ is how I work
<rick_h_> StevenK: ok cool, thanks
* jtv changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jtv | Critical bugtasks: 3*10^2
<huwshimi> rick_h_: You did push the changes? It might just be taking a whiel
<huwshimi> *while
<rick_h_> huwshimi: yes, I pushed but "No new revisions or tags to push."
<rick_h_> so shouldn't be anything new there if you're waiting for new files
<huwshimi> rick_h_: And you did an update of devel (or rocketfuel-get) before you merged devel with your branch?
<huwshimi> rick_h_: Only checking cause when I merge your changes with a fresh branch I get a conflict
<rick_h_> huwshimi: yes, on rev 14404
<StevenK> jtv: O hai! https://code.launchpad.net/~stevenk/launchpad/death-to-lazr-testing-tales/+merge/83889
<jtv> hi
<jtv> Does that come with coffee?
<StevenK> Pft, it's +1/-19, like it needs any
<jtv> I don't care what it needs.  It's what I need that counts.
<rick_h_> huwshimi: ah, sec
<wgrant> lifeless: As long as we link to the final page.
<wgrant> I guess.
<wgrant> We probably need to redirect.
<lifeless> wgrant: ECONTEXT
<wgrant> ML archives.
<wgrant> We can't just show the oldest message by default.
<StevenK> jtv: Haha
<wgrant> We would have to default to the last page.
<lifeless> we can show the list of indices.
<jtv> StevenK: it's not funny.
<wgrant> lifeless: Hm?
<lifeless> like $every $other $mhonarc $site $out $there
<wgrant> That's no argument.
<rick_h_> StevenK: what make do I need to do db updates?
<wgrant> mhonarc is pretty terrible. Basing arguments on what its other users do is illy.
<wgrant> s
<lifeless> its other users are using it as it was intended
<lifeless> I notice now that we don't have monthly groupings
<rick_h_> huwshimi: looks like I'm learning new stuff. rocketfuel-get only updates devel, no matter if it's run from another branch. So merging and resolving now
<lifeless> so its all messages ever that get reindexed
<StevenK> rick_h_: To get them rolled out, or to make them?
<lifeless> wgrant: so most static archiver sites have a page that says
<lifeless> 'jan 2001'
<rick_h_> roll out to my local db, I've gotten changes from devel and getting missing column oops
<lifeless> 'feb 2001'
<lifeless> etc
<rick_h_> StevenK: ^^ sorry
<lifeless> I'm proposing we in the interim, link to that
<StevenK> rick_h_: Oh, 'make schema'
<rick_h_> StevenK: ty
<huwshimi> rick_h_: Ah, yes it does. Sometimes you'll get pqm errors if you have conflicts. If you have branches that have been hanging around for a while it's good to do a devel merge just to resolve any issues before you submit for testing
<wgrant> lifeless: Well, we don't have such a page, but OK.
<lifeless> wgrant: I'm fairly sure mhonarc knows how to make one
<rick_h_> huwshimi: yea, and this was pulled from deryck's branch so no idea how old it is to start out
<huwshimi> rick_h_: Yeah, that's where it gets hairy. You can always do a devel merge to be sure.
<rick_h_> oh boo, reset my db so lost my feature flags
<huwshimi> rick_h_: Don't you hate that :)
<rick_h_> ok, wtf...all my code is gone now
<rick_h_> bah, this is probably due ot the rollback or osmething arggg
<wgrant> StevenK: Your sorting in the canonical.lazr branch is wrong.
<wgrant> subunit, lazr, testools
<rick_h_> huwshimi: ok, pushing changes, gets rid of the js error
<rick_h_> working on getting records back in db so I can test the actual paging and such, but think that might do it
<rick_h_> huwshimi: ok, tests that it works now
<StevenK> wgrant: Oh sigh
<StevenK> wgrant: Fixed.
<huwshimi> rick_h_: perfect, all working now :)
<rick_h_> huwshimi: awesome, thanks for your patience withme
<huwshimi> rick_h_: No, no, all good!
<StevenK> rick_h_: Write a script to populate the db.
<StevenK> rick_h_: Or 'make iharness' at a pinch
<rick_h_> StevenK: yea, on my todo
<rick_h_> haven't peeked at the db much yet
<StevenK> Don't look too close, your eyes will bleed.
<rick_h_> StevenK: was actually thinking I need to look at the api and see if I can script stuff using that instead
<rick_h_> StevenK: heh yea, most dbs, and with this one being a bit old, I bet there's some scary in there
<StevenK> rick_h_: So if you're populating objects to test and play, I'd suggest you just write a script that makes use of LaunchpadObjectFactory.
<huwshimi> rick_h_: Just going to eat some lunch and then send through some feedback for you
<StevenK> rick_h_: For example, I want a product, I don't care what it is: product = factory.makeProduct()
<rick_h_> StevenK: ok cool, will look it up, taking notes
<StevenK> rick_h_: LaunchpadObjectFactory is what is used in tests when you see self.factory...
<rick_h_> ah makes sense, I've really only done much on the JS side of things so far.
<StevenK> Haha, I've tried to stay away from the JS side of things.
<wallyworld_> lifeless: we show private teams in the bug subscriptions portal (it used to oops but now fixed). but we explicitly filter out private teams from the branch subscription portal. should we be consistent here?
<wallyworld_> nb we filterout private teams the user cannot see
<wallyworld_> but we now have a mechanism for rendering private teams even if the user cannot see them
<lifeless> so, AIUI the new rules we are saying that subscribing to an object discloses the team
<lifeless> so yes, they should be shown; OTOH have we fixed up any erroneous (e.g. to public branches) subscriptions of private teams yet
<wallyworld_> no, not afaik
<lifeless> we probably want to address that before making the teams visible to any viewer of public artifacts
<wallyworld_> i'd have to check - do we allow public bugs to have private subscribers?
<wgrant> lalalalala
<wallyworld_> since if we do allow it for public bugs, shouldn't we allow it for public branches also?
<lifeless> I think wgrant means 'we have not implemented the UI/business rules around this area, nor around that of handlings privacy transitions of objects
<wgrant> s/implemented/devised/
<wallyworld_> so, what's the damage in allowing a viewer of a public branch to see that a private team is subscribed to the branch?
<lifeless> there is none; as long as the private team was told they would disclose their existence
<wallyworld_> sure. that's how it is now for bugs
<wallyworld_> with the recent changes
<wallyworld_> lifeless:  if we want to not expose any private subscribers, then i should stop work on this branch entirely, since we will be exposing the existence of private branch owners, reviewers, committers also
<lifeless> wallyworld_: well we do, subject to the previously discussed caveats
<lifeless> wallyworld_: which is that if someone doesn't want to be disclosed, they can't interact with the branch
<wallyworld_> yes. that point never came up though in our standup when my work on this branch was discussed
<lifeless> we probably need to refine the definition of interact too, to acknowledge that there are subscribers we don't know about (as anyone we notify could be a multiplier), and anyone that can see a page can poll it
<lifeless> I will nab sinzui tomorrow about that
<wallyworld_> so i guess i can complete the work andbefore landing we may need to do some other work
<lifeless> you can just ff it perhaps
<wallyworld_> could do, but it's already out there for bugs
<wallyworld_> we just went ahead and deployed it
<lifeless> heh
<wallyworld_> not sure if any private teams have complained or noticed
<lifeless> you probably want to check with sinzui
<lifeless> I'm not in any position to second guess him - I haven't been tracking the detail on how you've been approaching this
<wallyworld_> i sort of thought the policy was that it is now ok to expose a team's name / existence
<wallyworld_> i'll check tomorrow
<lifeless> my understanding is that we settled on 'when the team interacts with something' and 'team interactions will prompt to permit the interaction'
<lifeless> its the second bit I suspect is missing
<wallyworld_> yes
<wgrant> The second bit requires an API break.
<wgrant> In dozens of places.
<wallyworld_> maybe we decided against it then. i can't recall
<lifeless> I'd worry a little if we had
<lifeless> because we have a number of policies intended to prevent accidental discovery of private team names
<lifeless> and not doing the second bit will deeply devalue them
<wallyworld_> my memory is fuzzy, so i'm not sure one way or the other
<wgrant> I thought we were going to skip that, because of the enormous and ever-expanding scope creep it would entail.
<wgrant> Given recent happenings, I don't think we need any more of that...
<wallyworld_> i seem to recall that the discussion was that private team names could be exposed
<wallyworld_> and that people would have to given them cryptic names
<wallyworld_> if they didn;t want others to guess things
<lifeless> noone told me :)
<wgrant> We don't have a compelling, pretty, feasible private team design.
<wgrant> We no longer have a compelling, pretty, feasible private artifact design.
<wallyworld_> lifeless: perhaps we can all have a chat with sinzui tomorrow to clear it up. in case i/we have misunderstood anything
<lifeless> sure
<lifeless> I'm here all week :P
<wgrant> Wait a sec.
<wgrant> It's Wednesday.
<wgrant> Why *are* you here?
<lifeless> lynne told me off for frittering leave away :P
<wgrant> Heh
<wallyworld_> but you're here when you are on leave too
<StevenK> wallyworld_: Multiple people have told lifeless at one time or another that he sucks at leave.
<wallyworld_> i just think he is so attached to his nick that he would hate to not live up to it
<lifeless> other way around
<lifeless> the nick has to live up to me
 * wallyworld_ wonders if lifeless is the chicken or the egg
<StevenK> I can't use = to compare dates in postgres?
<StevenK> (I don't care about the time)
<lifeless> you can, but you'll need to cast to date first
<StevenK> WHERE datecreated = '2011-11-30'::date; == 0 rows
<StevenK> But max(datecreated) shows 2011-11-30
<lifeless> datecreated isn't a date
<lifeless> WHERE datecreated::date == '2011-11-30';
<StevenK> Ah
<lifeless> or (for more efficiently
<lifeless> WHERE datecreated BETWEEN 2011-11-30 00:00 and 2011-11-30 23:59:59.99
<lifeless> or something
<StevenK> lifeless: I added a few things into the db using the factory and iharness and forgot to grab their names -- I just wanted the quickest way to grab the details
<StevenK> Hm. members=[foo] has invited them
 * StevenK wishes alert worked better with zsh
<lifeless> so
<lifeless> lazr_dbschema or lazr_postgresql ?
<StevenK> The former
<lifeless> or lazr_slony
<StevenK> With a ., I hope
<lifeless> nooo
<lifeless> epic pain and suffering apply to namespaces
<StevenK> lazr.restful, lazr.restfulclient ?
<lifeless> terrible
<lifeless> bugs open for years about import issues in their debs
<lifeless> heinous tree layout to boot
<StevenK> The _ offends me
<StevenK> But fair point
<rick_h_> it could be worse lazrPostgreSQL
<StevenK> rick_h_: Don't make me send you to bed with no dinner.
<rick_h_> heh, too late, had dinner, but I should go to bed
<wgrant> lazr.restful* aren't as problematic as lazr.uri.
<wgrant> But they're still not good.
<StevenK> Sigh, 31degC makes me sad.
<micahg> StevenK: beats 33degF
<wgrant> Heh, it's like 14 down here.
<wgrant> micahg: Cold is easy to solve. Heat is not.
<StevenK> micahg: I'll take it
<wgrant> while cold: apply clothing
<wgrant> while hot: raise ImMelting()
 * micahg has this magical thing called A/C
<StevenK> Haha
<StevenK> No A/C here, and no option for it.
<rick_h_> what?!
<StevenK> Rented apartment
<rick_h_> A/C is why I went to college and got a good job. So i can just keep that running all summer and manage to pay the bill
<jtv> StevenK: it's a bit over 32Â°C here, but turns out a fan helps.
<jtv> Maybe that's just because it's winter here, but it feels nice & cool.
<lifeless> jtv: you're in europe atm ?
<jtv> Nope.
<lifeless> ah, 'winter'
<jtv> You caught me.  We don't _really_ have winter here.
<StevenK> Australia has summer and lack-of-summer.
<jtv> That's what I thought, until an Australian friend moved back there and emailed me a map, making me guess where he lived.
<jtv> Thinking it was always hot in Australia, I said "what maniac called that place Sunnyvale?"
<jtv> The answer: wow, you guessed our location!  We don't understand the name either â it's never sunny here.
<spm> StevenK: one of my Uncles was in Mt Hagen in PNG for many many years. He said they had two seasons. The Wet Season, and the Wetter Season.
<StevenK> Haha
<spm> and canberra has all four seasons. so nyah.
<huwshimi> spm: What, all the way from grey to misty?
<spm> huwshimi: um. no. that's melbourne. you don't get grey and misty on 600mm average rainful.
<huwshimi> spm: My first experience of Canberra involved not seeing the sun the whole time I was there.
<spm> gees. my inlaws in mackay had one 24 hour period that copped more rain then we get in an average year.
<huwshimi> spm: granted, it was only  a little over 24 hours :)
<spm> huwshimi: heh, very unlucky then. a rubbish day here is when you can actually see a cloud in the sky. :-)
<huwshimi> overnight bus, get in a 6am, misty, all day, misty, practice, misty, play gig at night, still misty, bus out at 4am
<spm> ah yes. winter fogs. they can be special.
<StevenK> spm: At $LAST_JOB, we found a part of southern NT that recieved 3000mm of rain.
<StevenK> In 48 hours.
<spm> although, the year before I moved here. they had a running joke. What do you call the day after two days of rain? Monday. Apparently it rained 12+ consecutive weekends.
<spm> Ha!
<spm> Tully would be the good one to check. in FNQ.
<StevenK> A lake formed.
<StevenK> PrivatePersonLinkageError: Cannot link person (name=team-name-100003, visibility=PRIVATE) to <TeamMembership at 0xfeb68d0> (name=None)
<StevenK> ... so we can't have private subteams?
<wgrant> Correct. That would make the team's membership opaque.
<wgrant> In the current rules, the owner might not even be permitted to see that the member existed.
<StevenK> Right, so there is no point testing for the private subteam case in terms of MixedVisibility?
<wgrant> Not until we define and implement new rules.
<StevenK> So according to lifeless, private superteams should not be disclosed
<wgrant> Right, they're not important.
<lifeless> StevenK: depends on the viewer :) But yes, not automatically disclosed.
<StevenK> So I'm confused, since that is the major cause of MixedVisibilityError
<wgrant> Those need to be hidden.
<wgrant> When we allow private subteams, they must be shown.
<wgrant> There's no reason to show private superteams to everyone.
<wgrant> That benefits nobody.
<StevenK> Right, so I should fix Person:+index to only return private superteams if the user is a member of the subteam?
<wgrant> If the user has permission to view the subteam.
<wgrant> Er
<wgrant> To view the superteam.
<StevenK> I'm just worried that is expensive.
<wgrant> Which membership in the subteam would convey.
<wgrant> But it's not the only way.
<wgrant> Howso?
<lifeless> StevenK: try:except: around it
<wgrant> It's already shown as <hidden> because we know they don't have access.
<wgrant> So we know whether they have access or not.
<StevenK> But that's the TALES formatter, we aren't filtering the list
<wgrant> That's irrelevant.
<wgrant> We have the information.
<wgrant> So we can filter on it.
<StevenK> I'm just questioning the cost of calling checkPermission on each superteam
<wgrant> Clearly it's fine, because we already do it.
<StevenK> Oh, it's just done in the formatter.
<wgrant> Otherwise we wouldn't know to raise the soft oops.
<StevenK> Duh
<StevenK> Obviously, my brain is made out of silicon.
<StevenK> Ewww, Person.select()
<lifeless> argggh
<lifeless> stuck ec2 instance
<lifeless> 500dollar ec2 bill
<nigelb> wow
<StevenK> WIN
<wgrant> ow
<nigelb> Extra Large instance?
<nigelb> Wait, how long was it stuck?
<lifeless> $0.68 per High-CPU Extra Large Instance (c1.xlarge) instance-hour (or partial hour) 729 Hrs 495.72
<nigelb> wait, it remained stuck for 729 hours?...
<lifeless> apparently
<StevenK> No, that can't be right
<StevenK> Probably a few weeks, plus other usage
<lifeless> a whole month, yes.
<StevenK> Since a month is only 730 hours.
<nigelb> Oh another note.
<nigelb> If you use 1.5 hours, amazon chargse for 2 hours. (assuming the instance is killed after that)
<lifeless> we setup the teardown after it sets up
<lifeless> no, one instance, I'm pretty sure
<lifeless> my net has been shite
<StevenK> You don't use ec2 for anything else?
<lifeless> so I suspect a ec2 instance was kicked off and never had the shutdown timeout called
<lifeless> StevenK: no
<StevenK> Ah
<lifeless> StevenK: or rather, yes, I don't use it for anything else.
<nigelb> Well, lessson learnt at least :)
<StevenK> It is well known that lifeless' Internet is crap.
<nigelb> Maybe someone should check how many instances are around at any time and when it was procurred.
<lifeless> what lesson? don't use 'ec2 land' ?
<nigelb> hah
<nigelb> like a status page showing all currently running instances and when it was taken, should help track it down better.
<StevenK> lifeless: rvb has a cronjob that checks ec2 ls
<huwshimi> lifeless: Will stuck instances show up with 'ec2 list'
<StevenK> nigelb: We have that, 'bin/ec2 ls'
<lifeless> huwshimi: if you think to run it, yes
<nigelb> ahh
<wgrant> They show up in the numbers at the bottom.
<huwshimi> lifeless: Haha, ok :)
<wgrant> They don't necessarily show up in the list, however.
<lifeless> I normally have a widget - aws-status - that shows all my instances
<lifeless> however, it hasn't been updated to indicators
<lifeless> so I don't have it anymore
<nigelb> StevenK: heh, I got very excited and tried it out :P
<nigelb> and realized I don't have access ;)
<wgrant> You do have access :)
<StevenK> I think you can have access, you just have to pay for it
<nigelb> hah.
<nigelb> Oh, you guys use your own accounts and then expense it?
<wgrant> Yep
<StevenK> Right
<nigelb> AHHHH
<nigelb> ok, that makes sense. I thoguht y'all were sharing one account.
<StevenK> So if you run ec2 test twice a month it should cost you about $5.50
<nigelb> (that's what we do at work)
<nigelb> We have one account which is paid by corporate card and 3 of us get the keys to that
<spm> o/ nigelb
<nigelb> Hey spm :)
<StevenK> lifeless: I can't find the mail again, but I find newest first for mailman to be *backward* from every other mailman instance I play with.
<lifeless> I agree
<lifeless> as wgrant says thats not a good reason to do/not do something
<lifeless> but newest first is just terribly slow the way things are structured today
<StevenK> lifeless: It just gives me pause to think when looking. Since it performs like a utter dog too, I say change it.
<wgrant> Oldest-first would work if we didn't have a single archive for all-time.
<wgrant> s/would work/might be acceptable/
<StevenK> Right
<huwshimi> So, how do we upgrade our version of cssutils?
<huwshimi> The current version can't handle some css3 syntaxes
<wgrant> We're currently using 0.9.66
<wgrant> Er
<wgrant> 0.9.6
<wgrant> See versions.cfg
<huwshimi> wgrant: Yup and there's a (stable) version 0.9.7 that supports what I need
<wgrant> huwshimi: Grab a tarball, add it to download-cache alongside the old one, update versions.cfg.
<huwshimi> wgrant: Where does our config look to grab stuff from?
<huwshimi> wgrant: pypi?
<wgrant> huwshimi: No, it doesn't look online. It looks in download-cache/dist
<wgrant> You'll see lots of eggs and tarballs in there.
<huwshimi> wgrant: Oh right, so I need to commit a new tarball?
<wgrant> huwshimi: Right.
<huwshimi> ah
<wgrant> huwshimi: There's no PQM/reviews for that branch. You can commit/push directly.
<huwshimi> wgrant: But I need to push the change to lp for the versions.cfg change though right?
<wgrant> Right.
<huwshimi> wgrant: Do I need to do anything else to make launchpad start using the new version?
<huwshimi> wgrant: Do I need to get it to build a new egg or something
<wgrant> huwshimi: No. buildout will automatically create the egg from the tarball once you update versions.cfg
<huwshimi> wgrant: Oh, I guess I better run buildout :)
<huwshimi> oops, thanks :)
<wgrant> Ah, I see, yes.
<huwshimi> Oh, I guess it would help if I changed the versions.cfg of the branch I was using too
<huwshimi> Does anyone know how to enable the beta banner for a page locally?
<mrevell> Hi
<adeuring> good morning
* jtv changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugtasks: 3*10^2
<gmb> Oh, hey, leaving a partial DB patch lying around will break things. Who'da thunk?
<nigelb> stub: You might enjoy this - http://thedailywtf.com/Articles/Directive-595.aspx
<rick_h_> morning
<abentley> rick_h_: The updated spinner looks good to me.
<rick_h_> abentley: ok cool, Since deryck is going to be here soon I'll want to sanity check with him and if that works then will submit for review/land
<rick_h_> /want/wait
<abentley> rick_h_: cool.
<rick_h_> abentley: did you see the emails with huw from last night? Are we supposed to be jumping to the top of the page on next/prev?
<abentley> rick_h_: Yes.  In an email today, I explained that deryck was going to do that, but I don't think it should be associated with the spinner, just updating the batch.
<rick_h_> abentley: right, ok.
<abentley> bac: would you consider re-reviewing https://code.launchpad.net/~abentley/launchpad/person-bug-listings/+merge/83860?  I've made an important change.
<bac> abentley: ok
<abentley> bac: Thanks.  The change is that the view_name is now provided by LP and used by the javascript.
<bac> abentley: it looks good but i have to wonder if there isn't a better way to get the viewname?  i don't have any bright ideas but what you've done feels kludgy.
<dobey> jelmer: were you going to add me to ~vcs-imports?
<jelmer> hi dobey
<jelmer> dobey: sorry, I hadn't understood that you actually wanted to be in ~vcs-imports. One sec, I'll add you.
<abentley> bac: I don't know of a better way to get the view name.  I believe using getGlobalSiteManager().registeredAdapters() was gary_poster's suggestion.
<dobey> jelmer: oh sorry. yes please. thanks :)
<bac> abentley: ok
<jelmer> dobey: welcome :)
<bac> abentley: done.
<benji> Hey guys, I'm working with the LOSAs to get a small DB permission change for a script to fix a critical regression; in an email stub said that I "need someone to manually apply the permissions to production" but neither I nor the LOSA know what exactly that means.  He asks if "running ./security.py wouldn't work?"  Hints?
<deryck> abentley, adeuring -- standup in 4 ok?
<abentley> deryck: sure.
<adeuring> deryck: sure
<deryck> cool
<bac> hi jelmer
<jelmer> hi bac
<benji> flacoste: I am in need of someone I can ask a DB deployment question.
* abentley changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: abentley | Critical bugtasks: 3*10^2
<flacoste> benji: shoot!
<flacoste> i'll do my best to answer
<benji> flacoste: I'm working with the LOSAs to get a small DB permission change for a script to fix a critical regression; in an email stub said that I "need someone to manually apply the permissions to production" but neither I nor the LOSA know what exactly that means.  He asks if "running ./security.py wouldn't work?"
<flacoste> benji: yes, if we run it as we do as part of nodowntime, that should do the trick
<benji> flacoste: thanks
<gmb> sinzui: I have a question about bug 894390, if you've got a second.
<_mup_> Bug #894390: It's possible for Person.account and EmailAddress.account to be different for the same person <db> <merge-deactivate> <openid> <users> <Launchpad itself:In Progress by gmb> < https://launchpad.net/bugs/894390 >
<sinzui> I do
<gmb> sinzui: Okay. Quickly put, then: Will anything really bad happen if we move the code that changes email.account and email.person out of MergePeopleView._doMerge() and into the job code?
<gmb> I realise that we're then deferring the check for other email addresses on the dupe account
<sinzui> quick answer nothing bad
<sinzui> gmb, I wanted to do it
<gmb> But at the moment that check / change is preventing me from putting in a simple constraint to ensure that Person.account and EmailAddress.account point to the same place.
<sinzui> I think it is a bit tricky because of timing and rules checks
<gmb> Yeah.
<sinzui> gmb, that view code is using the removeSecurityProxy too? I think we really do want that code in the model level
<gmb> sinzui: Yup, I thought that too :)
<gmb> sinzui: Are we only caring about the user in doing the check in the view? I.e. do we mostly care that they get told if the merge can't happen?
<gmb> (Because we could conceivably do all that in a rabbity fashion, hand-wave, etcetera)
<sinzui> gmb: yes. We really want the user to know that they are done confirming *many* email addresses. The code is counting down to 0
<sinzui> Then we tell the user we are going to do it
<gmb> Hrrrm.
<sinzui> We can count down to 1 maybe
<gmb> Hmm, possibly.
<gmb> sinzui: Actually, if we did the check for emailset.getByPerson(self.dupe).count() > 1 (or whatever) early on in _doMerge(), wouldn't it have a similar effect?
<gmb> Oh, no.
<gmb> Bother.
<gmb> That doesn't actually solve the problem *at all*
<deryck> lunch time for rick_h_ and I.
<sinzui> abentley, Do you have time to review https://code.launchpad.net/~sinzui/launchpad/dsp-vocab-unknown-distro
<abentley> sinzui: I can get to it in a bit.
<abentley> sinzui: Is it intentional that test_getInputValue_package_spn_dsp_picker_feature_flag and test_getInputValue_package_dsp_dsp_picker_feature_flag have identical comments?
<sinzui> no :(
<abentley> sinzui: around 171 and 187, I think you can simplify getUserBrowser().open(canonical_url(bug)) to getViewBrowser(bug)
<sinzui> I certainly an
<sinzui> can
<abentley> sinzui: technically, test_setDistribution should have a comment, though I don't know what I'd say.
<abentley> sinzui: And there should be a docstring on setDistribution itself.
<sinzui> I would say it permits the callsite to set the vocab's distribution after the vocab was instantiated by another mechanism
<abentley> sinzui: Okay,  r=me with those text changes.
<sinzui> thank you
<abentley> sinzui: Could you review one for me?
<sinzui> yes I can
<abentley> sinzui: https://code.launchpad.net/~abentley/launchpad/history-model-fix/+merge/83995
<abentley> sinzui: I can provide some context if you like.
<sinzui> Does this fix my back button? I sometimes need to go back twice to go back one page?
<abentley> sinzui: On the new bug listings?
<sinzui> yes
<abentley> sinzui: It's not a bug I've observed, but I think it might fix that.
<sinzui> abentley, lint lists 6 files, the diff shows 1
<abentley> sinzui: I don't think lint takes prerequisite branches into account.
<sinzui> ahh there is a preqeq
<sinzui> r=me
<abentley> sinzui: thanks!
<lifeless> sinzui: hi
<lifeless> sinzui: mailman
<sinzui> yes mailman
<sinzui> never postwoman
<lifeless> is it easy to turn on monthly indexes vs all-time ?
<sinzui> lifeless, I think it is the same work as what I described in my reply to your email. I am not familiar with the switch/flags in the file, but I think it is simple to add and a few days to generate
<lifeless> coooool
<lifeless> do you feel like doing it yourself?
<lifeless> cause, you have like +10000 from me if you want to
<sinzui> I will consider it for Friday. I have committed to fixing the so-called "bug supervisor" label on the advanced bug search form for Friday morning
<lifeless> \o/
<lifeless> sinzui: separately
<lifeless> sinzui: wallyworld_ asked me yesterday about disclosing team names on branch pages; I said my understanding was that was fine as long as we'd cleaned up existing subscriptions to remove teams that don't want to be disclosed
<lifeless> this turned into a discussion about whether that was work we'd decided to do or not, or something
<lifeless> 16:25 < lifeless> my understanding is that we settled on 'when the team interacts with something' and 'team interactions will prompt to permit the interaction'
<lifeless> 16:25 < lifeless> its the second bit I suspect is missing
<lifeless> 16:25 < wallyworld_> yes
<lifeless> 16:26 < wgrant> The second bit requires an API break.
<lifeless> 16:26 < wgrant> In dozens of places.
<lifeless> 16:26 < wallyworld_> maybe we decided against it then. i can't recall
<lifeless> 16:27 < lifeless> I'd worry a little if we had
<lifeless> 16:28 < lifeless> because we have a number of policies intended to prevent accidental discovery of private team names
<lifeless> 16:28 < lifeless> and not doing the second bit will deeply devalue them
<lifeless> 16:28 < wallyworld_> my memory is fuzzy, so i'm not sure one way or the other
<lifeless> 16:28 < wgrant> I thought we were going to skip that, because of the enormous and ever-expanding scope creep it would entail.
<lifeless> 16:29 < wgrant> Given recent happenings, I don't think we need any more of that...
<lifeless> 16:29 < wallyworld_> i seem to recall that the discussion was that private team names could be exposed
<lifeless> 16:29 < wallyworld_> and that people would have to given them cryptic names
<lifeless> 16:29 < wallyworld_> if they didn;t want others to guess things
<lifeless> 16:32 -!- timrc [~tim@99-57-130-76.lightspeed.austtx.sbcglobal.net] has joined #launchpad-dev
<lifeless> 16:34 < lifeless> noone told me :)
<lifeless> 16:35 < wgrant> We don't have a compelling, pretty, feasible private team design.
<lifeless> 16:35 < wgrant> We no longer have a compelling, pretty, feasible private artifact design.
<lifeless> 16:36 < wallyworld_> lifeless: perhaps we can all have a chat with sinzui tomorrow to clear it up. in case i/we have misunderstood anything
<lifeless> -fin
<sinzui> I think my team was slowly understanding what I meant by traversal. wallyworld_ did produce a branch that seem to do the right thing to check if a user may access a subordinate object
<sinzui> we have not solved the important case yet to say we really know how to do this...eg stop avoiding the private team p3a subscription problem
<lifeless> anyhow, I believe you have a standup. Perhaps I can join at the end of that and we can have a brief chat? I don't want to be adding confusion to your team when they ask me things :)
<sinzui> lifeless, We do not yet know how to present pages when the user as restricted access
<sinzui> okay
<sinzui> lifeless, https://code.launchpad.net/~wallyworld/launchpad/view-private-branch-artifacts-605130/+merge/83915 shows the draft that wallyworld_ took after discovering the bug was about traversal
<timrc> Not that I'm going to add anything of value here, but most of our (Commercial Engineering)  privacy needs could be met if running own instance of Launchpad were more feasible
<timrc> let the firewall naturally provide most of the security for us
<timrc> doesn't solve the problem of privacy for external customers though (unless they too ran their own instances of Launchpad)
<lifeless> timrc: AIUI thats bogus :)
<lifeless> timrc: because you have multiple vendors you collaborate with, and you don't want mutual-disclosure amongst them
<timrc> if we could bring up separate launchpad instances for each, though?
<timrc> that would imply Launchpad were much lighter than it is today, though
<lifeless> it would be a totally different thing at that point
<timrc> there has been customer interest to put image building behind the firewall but what of the archive? on the face of things launchpad seems perfectly suited for that, but in practice, that would be a nightmare
<flacoste> more like a set of bugzilla instances :-)
<flacoste> which means, no collaboration between them
<flacoste> which isn't what LP is about
<timrc> flacoste, how do private projects within launchpad.net collaborate today?
<flacoste> timrc: well they use multiple bug tasks
<timrc> I thought that was going away :)
<flacoste> you asked today?
<timrc> lol ok
<flacoste> but you are right
<timrc> fair enough
<flacoste> that's changing to use linking between issues
<flacoste> so that it's easier to manage the conversations
<flacoste> and easier to understand who has access to what
<flacoste> but linked issues will still make it easier to collaborate than setting separate bugzilla instances
<flacoste> which really never talk to each other
<flacoste> as a back story
<flacoste> Canonical and Ubuntu started by using bugzilla itself
<flacoste> and then quickly saw that trying to talk to upstream would never scale with it
<flacoste> and that's when LP was born
<timrc> Federate your Launchpad instances so BigCo's launchpad instance is comprised of all the data from launchpad.net and launchpad.bigco.com
<flacoste> well, its bug tracker at least
<flacoste> timrc: yep, that would be another way of doing it
<flacoste> but more work
<flacoste> and you guys have waited long enough already ;-)
<timrc> lol
<lifeless> federation has the same issues FWIW
<lifeless> 'if I reply to this bug, who sees the response'
<lifeless> 'who is this bug accessible to'
<lifeless> 'who should control accessibility to this bug'
<timrc> depends on where the bug originates?
<lifeless> sure
<timrc> if the bug originates on launchpad.bigco.com than by default it's only visible to people with access to launchpad.bigco.com
<lifeless> but if you get notified on bugs that are just federated in
<lifeless> and you reply with proprietary info because you are confused...
<abentley> deryck: chat?
<deryck> abentley, deep in sprinting right now.
<abentley> deryck: okay.
<deryck> abentley, is this about something you're hacking on?
<abentley> deryck: Yes.
<timrc> lifeless, we have a nice banner that says "this bug is private"... I can envision the same here
<lifeless> timrc: right, but you see that its not something improved by federating
<deryck> abentley, is this something you need me for, or could someone else pre-imp you?
<lifeless> timrc: federating doesn't naturally solve it
<timrc> lifeless, it seems more intuitive, the separation seems clearer
<abentley> deryck: I wanted some policy direction, but I think I can get started without it.
<timrc> you don't set a privacy bit, it's implied by organizational structure
<lifeless> I think federating is a good way to think about the problem
<lifeless> but I don't think the act of federating itself solves much, not if you have it all automated
<deryck> abentley, ok, cool.  Thanks!
<timrc> okay, well thanks for humoring me :) i know your time is precious
<timrc> I am trying to find a critical bug I can work on in my spare time -- need you guys focused on privacy! ;)
<jcsackett> abentley: free for a review? https://code.launchpad.net/~jcsackett/launchpad/fancy-filebug-part2/+merge/84004
<abentley> jcsackett: I don't understand the difference between a private bug and a private-by-default bug.
<abentley> jcsackett: Do you know why these ribbons are implemented in JavaScript?
<jcsackett> abentley: private by default is projects where all reported bugs are automatically marked as private.
<abentley> jcsackett: I got confused because I didn't understand were were talking about a bug that doesn't exist yet.
<jcsackett> abentley: ah, yes, this is on +filebug, so as you're reporting it.
<jcsackett> sorry if my mp didn't make that clear.
<jcsackett> as to the javascript, abentley, it was written originally for existing bugs, so that it would dynamically display when a bug was marked as private or public once it already existed.
<jcsackett> it seemed better to just reuse that even in non-dynamic situations then create another display.
<jcsackett> and in the case of toggling the security related stuff, the dynamic part is actually desireable.
<abentley> jcsackett: It just seems bad not to provide these ribbons to non-JS users.
<abentley> jcsackett: Our new beta banner was based on these ones, so I've filed Bug 891697 about it.
<_mup_> Bug #891697: Beta banner is not shown if JavaScript is disabled. <Launchpad itself:Triaged> < https://launchpad.net/bugs/891697 >
<jcsackett> abentley: that's fair.
<jcsackett> abentley: i *believe* the old privacy display (the grey stripey edging) still shows when you're looking at a private bug if you don't have JS.
<jcsackett> but having something like the banner still work would be better.
<abentley> jcsackett: oh, that's good.
 * timrc tests that
<abentley> r=me.  I prefer "set up" rather than "setup" for the verb, but I guess both are used.
<timrc> jcsackett, it does indeed show the grey stripey edge
<jcsackett> abentley: thanks. i've updated your bug to explicitly include the privacy banner, b/c the same considerations apply.
<poolie> hi all
<sinzui> abentley, I share your observation about the privacy banner. It was interactive. We removed the hide button, so it is just structure and CSS now. in fact, we still load the old css. you will see the old background stripe on the left when the page loads :(
<lifeless> thats good if js is disabled ;)
<flacoste> sinzui: timrc disabled js and confirmed that the old background strip appears
<flacoste> hi poolie
<poolie> hi there
<flacoste> benji: are we ready to renable the translations jobs?
<flacoste> benji: a user complained again on #launchpad
<sinzui> lifeless, flacoste it is not good, it is only acceptable. The page could have used the pretty CSS by default!
<benji> flacoste: we are (well, they weren't jobs before, but now are)
<flacoste> sinzui: right
<flacoste> benji: care to file a RT for it? I'll put it at the top of the queue
<timrc> gray stripey edge is definitely subtle
<benji> flacoste: the only problem remains that there is a disagreement about how exactly that is done, or if anything needs to be done at all
<flacoste> benji: what's the disagreement about?
<timrc> I've grown to depend on the red bar to make sure I'm not about to do something regrettable
<timrc> if only I had it for other aspects of my life
<benji> flacoste: I wasn't around when it was last discussed but the way it was communicated to me was that nothing needed to be done and the jobs would simply begin running
<flacoste> benji: i doubt that
<benji> flacoste: you and me both :)
<flacoste> benji: right, we need a new cron job
<flacoste> there is a generic script to run these jobs
<flacoste> but the arguments to the script are job-specific
<flacoste> iow, the config section to use
<benji> right (cronscripts/run_jobs.py)
<benji> flacoste: I'll file an RT to add "cronscripts/run_jobs.py pofile_stats" to the crontab or equivalent.
<flacoste> benji: thx, let me know the # so that i can adjust the priority
<benji> flacoste: will do
<benji> flacoste: RT #49635
<_mup_> Bug #49635: adept is _really_ slow.  <adept (Ubuntu):Fix Released by mornfall> < https://launchpad.net/bugs/49635 >
<flacoste> benji: thx
<benji> np
<lifeless> sinzui: so what time is the clal ?
<sinzui> 19 minutes
<sinzui> we use mumble
<lifeless> kk
<sinzui> I suspect we are the last team to use mumble
<poolie> o/ sinzui
<sinzui> poolie, Last year, I had mumble on all the time and talked many times a day. Used mumble about 7 time a week now. :(
<poolie> that is more often than me
<poolie> if i had better audio hardware
<poolie> i would use it more
<poolie> not sure what that would look like though
<lifeless> if it worked better in .nz I would use it more ;)
<beuno> sinzui, in U1 we only use mumble, a lot of us are on it for the full working day and move between rooms to talk to people
<sinzui> beuno, That is what I remember the ~launchpad team was doing 12 months ago
<mwhudson> lifeless: it's been fine for me lately
<mwhudson> of course i have super internet in the office
<mwhudson> (by nz standards)
<lifeless> beuno: I can't work like that - too much distractions (see for instance the research into productivity losses in open plan offices)
<beuno> lifeless, right, so my work is basically interrupt-based, so if anyone is blocked on something, they can just jump in and talk
<beuno> when I need to focus, I close it
<lifeless> beuno: they can do that to me too; but when its not happening I am focused ;)
* abentley changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugtasks: 3*10^2
<beuno> lifeless, it's a bigger barrier if you need to ping someone on irc, get them to log in, adjust their mic, etc etc etc
<beuno> if I'm already there, it's cheap and quick
<abentley> sinzui: We never really did that on the Code team.
<lifeless> beuno: perhaps
<beuno> lifeless, then we also have a policy where developers always work in pairs, so they have their own channels
<beuno> I can get dragged into those
<beuno> or other developers can get pinged to join them
<beuno> it's been quite good for us
<beuno> OTOH, when it's hammer time, people even drop off of IRC   :)
<sinzui> StevenK, bug 603732
<_mup_> Bug #603732: If there is no driver a maintainer should be able to accept a bug nomination <bugnominations> <lp-bugs> <planning> <series> <Launchpad itself:Triaged> < https://launchpad.net/bugs/603732 >
<sinzui> wallyworld_, Do you want to talk about launchpad.LimitedView on mumble
<wallyworld_> sinzui: ack
<poolie> is it just me who sees ec2 host keys getting cached in my real keyring by ec2test?
#launchpad-dev 2011-12-01
 * StevenK kicks celebrity_logged_in()
<huwshimi> wgrant, wallyworld_: Have either of you seen user research from dan about the manage disclosure mockups (I guess from a while ago)? I'm just trying to track it down
<wallyworld_> huwshimi: perhaps. was it an email?
<wgrant> I don't think there's much point continuing those at the moment.
<huwshimi> wallyworld_: I assume so
<huwshimi> wgrant: How come?
 * wallyworld_ checks his email
<wgrant> huwshimi: Requirements have changed.
<huwshimi> wgrant: Really?
<wgrant> It's not clear what we're doing now.
<huwshimi> wgrant: So the whole manage disclosure pages might be different?
<wgrant> Things are sort of on ice.
<wgrant> Possibly, yes.
<StevenK> Can haz review? https://code.launchpad.net/~stevenk/launchpad/bugtask-userHasPrivileges-nomination/+merge/84041
<wgrant> StevenK: How does that affect performance?
<huwshimi> wgrant: When did this happen? I got asked over night to get them ready for testing tomorrow
<StevenK> wgrant: IBugNomination.cannAprove() is already a dog?
<wallyworld_> huwshimi: wgrant: the page will likely not be that different
<wallyworld_> the testing should still go ahead
<wgrant> huwshimi: A little over 24 hours ago.
<huwshimi> wgrant: Oh right
<wgrant> It may not be that different. But it quite possibly will be. I guess we might as well continue testing.
<huwshimi> ok
<wallyworld_> there's been an adjustment to the scope - security bugs need shared conversations, even private ones, hence multipillar private security bugs are allowed
<wgrant> Which means that the disclosure overview page can no longer feasibly show an overview of disclosure.
<StevenK> wgrant: We already had to look up the product/distribution and {product,distro}series in canApprove() anyway, we're just checking a few more columns in the same tables.
<wgrant> StevenK: k
<wgrant> StevenK: Hm
<wgrant> StevenK: Isn't this wrong?
<wgrant> It seems like it will permit bug supervisors to approve.
<wgrant> Which is wrong.
<StevenK> We don't want that?
<wallyworld_> wgrant: i think the md page can still show an overview
<wgrant> wallyworld_: It can't.
<wgrant> It can show a partial view.
<wallyworld_> huwshimi: sent you an email with the info
<wallyworld_> sure it can
<wgrant> StevenK: No.
<wgrant> wallyworld_: How?
<StevenK> Bah
<wallyworld_> how not?
 * StevenK fixes it to check owner
<huwshimi> wallyworld_: Awesome, just got it. Thanks for that
<StevenK> By hand
<wallyworld_> huwshimi: np
<wgrant> wallyworld_: We can't join through BugTask.
<wgrant> wallyworld_: So we'd have to flatten everything.
<wallyworld_> wgrant: why can't we do a union
<wgrant> wallyworld_: Hm?
<wallyworld_> instead of joining through a single bugtask, why not do the same thing across all the bug tasks (if there are now allowed to be > 1) and union the results
<wgrant> If I have a bug for project A shared with project B, project A's disclosure management page has to show all the observers for A and B.
<wallyworld_> yep
<wgrant> If the disclosure management page knows about bugtasks, ARIWEJOFUWEFIHWEFIUWEF
<wallyworld_> and?
<wgrant> Also, slow.
<wgrant> It's slow enough as it is.
<wallyworld_> so we denormalise
<wgrant> I went through this two months ago :)
<wgrant> It is not pretty.
<wgrant> Or usable.
<wgrant> At all.
<wallyworld_> bottom line is, we've been tasked with providing a solution for a problem our stakeholders want solved, we need to figure out a way to do it
<wallyworld_> we may need to compromise on how some things are delivered/reported
<wgrant> We don't necessarily need to solve this.
<wgrant> It's not clear whether we want disclosure reports for security bugs.
<wgrant> Because proprietary projects will probably be forbidden from sharing security bugs.
<wallyworld_> what about private security bugs
<wgrant> Or having security bugs at all.
<wallyworld_> so, if that's the case, then problem solved :-)
<poolie> wgrant, like i said, why not just delete the field?
<wgrant> poolie: Which field?
<poolie> security
<wgrant> Ha ha ha.
<wgrant> It's not that simple, unfortunately.
<poolie> mm
<wgrant> Security privacy is now defined to be completely different from normal privacy.
<wgrant> s/now/as of yesterday/
<wgrant> wallyworld_: Not problem solved.
<wgrant> wallyworld_: It still requires a rethink of the UI.
<wgrant> wallyworld_: Regardless.
<wallyworld_> sure, but shit happens
<wallyworld_> we'll come up with something
<wgrant> Right, I'm not saying we won't.
<wgrant> I'm saying it needs more thought.
<wgrant> And work.
<wallyworld_> the mockups can stil be tested
<wgrant> And that spending time testing mockups that we know won't work is probably silly.
<wallyworld_> we aren't spending any more time developing them, but it's still valid to test them
<wallyworld_> as is
<wallyworld_> since there's other useful feedback we can get
<wallyworld_> like the use of the tag paradigm etc
<wallyworld_> the location of the delete icon
<wallyworld_> etc
<StevenK> wgrant: Fixed, diff updated. Ignore the ADMIN_EMAIL addition, I've removed that already.
<wgrant> StevenK: Why not add a helper method somewhere that checks if driver privileges are possesed?
<wgrant> It would be very similar to userHasPrivileges or whatever the existing thing is.
<wgrant> Except without bug supervisor.
<wallyworld_> wgrant: StevenK: same me some greping, do either of you know the unit test for viewing a P3A owned by a private team where the use cannot see the team but is subscribed to the P3A?
<wallyworld_> /same/save
<StevenK> wgrant: Because it probably requires rewritting IBugTask.userHasPrivileges() for the third time in as many branches.
<wgrant> StevenK: Good.
<wgrant> StevenK: Rewritten but clean is better than dirty.
<wgrant> wallyworld_: No idea, sorry.
<wgrant> wallyworld_: You can probably find it by following the evil aura, though.
<wallyworld_> np. just an opportunistic question just in case
<StevenK> wgrant: userHasPrivilegesContextWithBugSupervisor()
<wgrant> StevenK: userHasBugSupervisorPrivileges?
<wgrant> userHasDriverPrivileges?
<poolie> huwshimi,  i have a thing for you
<poolie> https://code.launchpad.net/~mbp/launchpad/feature-modulus/+merge/84042
<poolie> review pls
<flacoste> wgrant, wallyworld_: the discussion between sinzui, mrevell and I was that we think it's fine that manage disclosure only shows a partial view of security issues
<flacoste> basically the view of the security policy for your project
<wgrant> flacoste: Right. But it gets confusing for projects like U1, which may have both proprietary and security bugs.
<flacoste> with a warning saying: some people might have access to some security bugs through other projects
<wallyworld_> flacoste: i think that's workable
<wgrant> We need to present that somehow that they're different.
<huwshimi> poolie: You are awesome :D
<poolie> now do something useful with it :)
<huwshimi> poolie: Sure will, I just need to build a logging/reporting tool for it :D
<wallyworld_> poolie: do you take bribes to allow users to "get in" on the feature :-)
<poolie> sounds like a business model to me
<flacoste> wgrant: is it really a problem? either these security bugs are really private and they won't share it, or they are security bugs (and say also affect Ubuntu) in which case the warning is good enough
<poolie> more seriously you can always have something turned on either by membership in a team or by being lucky with your uid
<poolie> or vice versa
<wgrant> flacoste: Well, we have to make it clear that it shows everything for one tag, and not everything for the other.
<wgrant> Which is confusing, because in the current designs they're shown identically..
<flacoste> wgrant: yes, but a simple notice is probably good enough
<wgrant> Heh, users don't notice notices.
<wgrant> Ever.
<flacoste> agreed
<wallyworld_> wgrant: you are in "glass half empty"mode today :-P
<poolie> today?
<StevenK> What poolie said.
<flacoste> but the assumption here, which should be tested, is that users have different assumptions between privacy and security
<wgrant> flacoste: At present they have the same assumptions, because they behave the same way.
<flacoste> so the fact that other people have access through the shared project security policy isn't surprising
<flacoste> lke it would be for private bugs
<wallyworld_> flacoste: there will be some adjustment to users' thinking when security and privacy an de-conflated, but the overall; result will be much easier to grok i think
<flacoste> wallyworld_: that's what sinzui though when I suggested that we keep multi-tenancy around security bugs
<flacoste> thought
<wallyworld_> flacoste: i liked your email. enum for privacy/visibility with orthogonal security flag
<wallyworld_> agree with security bugs being multi-tenanted so conversations can be shared
<poolie> huwshimi, let me know howmuch you care about slicing anonymous users
<poolie> my impression is they do not have very interesting interactions
<wgrant> I don't agree that multi-tenancy is the solution to shared conversations.
<poolie> mm
<wgrant> But it may be the easiest way out for now.
<poolie> i wonder how many non bot requests are anonymous
<wgrant> poolie: A quick grep should tell you...
<poolie> indeed, i will do that in a sec
<flacoste> wgrant: yeah, i can agree with that
<poolie> but the real question is do people ever want to run experiments on them
<poolie> wgrant, though i wonder too how many anonymous users are also blocking or ignoring cookies...
<wgrant> poolie: True.
<poolie> we can possibly find that too
<wallyworld_> even some non anonymous users block cookies
<wallyworld_> well, those who care about privacy
<poolie> uh
<wgrant> Well, if they're not anonymous to LP they're probably allowing at least one LP cookie :)
<poolie> not if they want to sit at the big people's table
<wallyworld_> "big people's table"?
<poolie> you cannot be both logged in and also blocking lp's own cookies
<wallyworld_> wgrant: yeah, sadly i had to add lp to my whitelist
<wgrant> wallyworld_: Yeah :(
<huwshimi> poolie: I guess we could always build that into into the reporting, there would certainly be some a/b test where we would be targeting logged out users.
<poolie> spm i have a thing for you too: https://code.launchpad.net/~mbp/launchpad/feature-admin-party/+merge/84044
<poolie> wgrant, would you mind doing a security review of that one?
<poolie> it is not very big but it is a little scary
<wgrant> Admin party?
<wgrant> Aha
<poolie> when you first start couchdb, it says "Admin party! Everyone is admin!"
<wgrant> There seems to be a false assumption th ere.
<wgrant> Why would anyone willingly start couchdb? :)
<poolie> *if
<wgrant> I'm not sure if we should really do this.
<wgrant> We have non-Canonicalers in ~registry, for example.
<poolie> istr robert approving it
<wgrant> And even Canonicalers aren't always going to be aware of what implications flags have.
<poolie> the concept that is
<poolie> perhaps ~registry is the wrong thing
<wgrant> (there are security-influencing flags)
<StevenK> wgrant: Diff updated.
<poolie> i think we should probably be careful about putting really dangerous things in there
<poolie> in the first place
<lifeless> mmm
<wgrant> StevenK: DriverPrivileges and PrivilegesContext?
<poolie> i guess making it a specific group would let us flip this at run time
<poolie> without a boostrapping problem
<wgrant> StevenK: We probably want BugSupervisorPrivileges(Context)? and DriverPrivileges(Context)?.
<poolie> lifeless, what do you think?
<poolie> i had the idea it had a preimp approval
<lifeless> poolie:  https://bugs.launchpad.net/launchpad/+bug/790025/comments/1 is all I had said
<_mup_> Bug #790025: provide LP developers full access to QAS/Staging feature flags <canonical-losa-lp> <feature> <feature-flags> <Launchpad itself:Triaged> < https://launchpad.net/bugs/790025 >
<spm> poolie: heh, scratching an itch there?
<poolie> i was going to do the userslice thing when i was offline yesterday and then i remembered this
<lifeless> poolie: I think there are two issues; A) there is a code structure issue, which I've reviewed. B) there is the how-to-enable-it
<poolie> and yeah, it does seem like somewhat unnecessary manual handoffs
<StevenK> wgrant: PrivilegesContext is the current name and is still called everywhere
<poolie> yeah
<poolie> there are some precedents for the inline permission check
<poolie> but not necessarily good ones
<poolie> lifeless, so if it's hooked in differently and it is enabled by adding people to eg ~admin-features-rules
<poolie> that would be acceptable?
<lifeless> poolie: are you proposing admin-feature-rules as a celebrity ?
<poolie> i guess effectively yes
<poolie> i understand some of the machinery around celebtrities is disliked?
<poolie> it depends a bit how much you expect to want to change it dynamically
<lifeless> so the thing about celebrities that is disliked is largely the fact that its a well known team.
<lifeless> so avoiding the rest of the machinery but blessing a team name will just get the bad stuff without mitigating it via the other supporting code
<poolie> ok
<poolie> so what is the actually preferred place for configuration?
<poolie> or rather policy
<lifeless> zcml or lazr.conf or feature flags
<poolie> i hear config is disliked, zcml is disliked, embedding it in the code is disliked
<lifeless> are the three generic places we have
<lifeless> or schema things hanging off of objets
<lifeless> theres no 'site' or 'instance' object in the db today
<poolie> hm
<poolie> what defines who has a particular permission on a particular object?
<lifeless> the security adapters
<poolie> hm
<poolie> so i see there are things that grant some permissions if user.in_launchpad_developers
<poolie> s/things/adapters
<poolie> perhaps i'll leave  it, it was more of a drive by attempt
<wgrant> StevenK: The callsites can and should be changed...
<wgrant> It's a pretty trivial refactoring across part of one codebase, which will result in cleaner, more maintainable code.
<wgrant> lifeless: https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-ce9f40e4e0a1507f99c1f0eda494ae39
<lifeless> wgrant: win!
<wgrant> Rather.
<lifeless> a unicode string
<lifeless> wtf
<lifeless> wgrant: care to file a bug for me? I'll add a codepath to type-handle this
<lifeless> also someone should shoot the zope machinery
<poolie> lifeless, is there enough value in  pulling out the DemoSite fixture?
<lifeless> afk -> bank visit, bbsoonish
<huwshimi> wallyworld_: In those prototypes is the "Add an observer" link for adding a privacy observer?
<huwshimi> or security as well?
<wallyworld_> huwshimi: it adds the person ad both atm
<wallyworld_> as
<wallyworld_> from there you can use the popups to change
<wallyworld_> but we need to enahnce the picker
<huwshimi> wallyworld_: That seems suboptimal.
<wallyworld_> yes, the picker needs to be enhanced
<wallyworld_> but we didn't want to do that for the mockup - ran out of time
<huwshimi> sure, just checking
<wallyworld_> np
<huwshimi> wallyworld_: Also the 10	users can view project information
<huwshimi> 6	users are observers
<huwshimi> 5	users are restricted observers
<huwshimi> block seems to be a little redundant as it doesn't cater for security/privacy
<huwshimi> Not sure what to do with that
<huwshimi> Is it actually useful for anything?
<wallyworld_> those numbers are now used to represent users who have any sort of permission (security or privacy or ...)
<huwshimi> wallyworld_: I guess that info can now be determined by using the filters
<huwshimi> wallyworld_: And jumbling the stats together as they are now doesn't seem useful (but tell me if I'm wrong)
<wallyworld_> yes, but the portlet is useful because it shows users with *any* observer access for example
<wallyworld_> but that's just my opinion
<wallyworld_> as a project owner, that's what i'd want to see
<huwshimi> wallyworld_: is that useful if you don't know if they can only see apport bugs or something?
<wallyworld_> of course we'd add that into the nav portlet in the final version
<wallyworld_> what's there now is based on the original screenshots before these new things were thought of
<huwshimi> wallyworld_: Right, that's why I'm trying to determine how this should look now that we have that new info
<wallyworld_> ok :-)
<huwshimi> (btw I'm doing a whole new set of mockups, so this isn't me asking you to do anything at this point).
<huwshimi> "of course we'd add that into the nav portlet in the final version" <- what is "that" in this sentence?
<wallyworld_> the summary  of number of apport bugs and "quicklink"
<wallyworld_> (and any other quicklinks deemed necessary)
<huwshimi> wallyworld_: So there would be a list of each "tag" with the number of observers/restricted observers?
<wallyworld_> perhaps. that's what i would want to see as a user
<wallyworld_> sort of a shortcut to using the search form
<wallyworld_> but that may not be feasible for the number of tags we have
<wallyworld_> it's really up to the users to ask for what they want in that portlet
<wallyworld_> ah, scratch that comment about number of tags
<wallyworld_> the portlet has roles (observer, restricted observer)
<wallyworld_> plus any other shortcuts we want to add
<wallyworld_> or are asked to add
<StevenK> wgrant: Have another look?
<wgrant> StevenK: It seems sort of odd to call it as a method on a bugtask, but still have to pass the context in manually.
<StevenK> wgrant: I have no choice -- the main call sites don't have a BugTask.
<wgrant> StevenK: That's why there was a separate *Context method before.
<StevenK> wgrant: You'd like me to duplicate the current methods to Context and context-less versions?
<wgrant> StevenK: Yes. The implicit context version of userHasBugSupervisorPrivileges seems to already exist, but it's called userHasPrivileges.
<lifeless> poolie: I think showing the evaluated flag values is ok; just the scopes are the issue.
<huwshimi> wgrant, wallyworld_: In the prototype where it says "Subscribed to 6 bugs and 4 branches" is "subscribed to" the right wording there?
<wgrant> No.
<wallyworld_> nope
<wgrant> "subscribed" is wrong.
<wgrant> It's not clear what the right wording is.
<wallyworld_> the wording is/was true to the originally supplied screenshots
<wgrant> I want to dump "observer" and say "access" or something instead.
<lifeless> private things shared with you
<wallyworld_> "access" sounds ok for now i think
<lifeless> isn't that the concept ?
<wallyworld_> yes, but we need a single word or two
<lifeless> 'shared with'
<wgrant> huwshimi: Where in the prototype does it say this?
<wgrant> I haven't looked for a while.
<lifeless> also why are we showing counts.
<huwshimi> wgrant: http://people.canonical.com/~ianb/disclosure/
<wallyworld_> 'shared with' sounds ok
<wallyworld_> counts? as per the original screenshots
<wallyworld_> i personally like them
<lifeless> which original screenshots ?
<wallyworld_> i would want to see them as a user
<wgrant> So, showing the counts outside the context of a particular policy probably doesn't make sense.
<wallyworld_> the balsalmiq ones from 18 months ago
<wallyworld_> not sure who did them
<wgrant> I still suggest [Proprietary: everything] [Security: 2 bugs, 4 branches] or something like that.
<huwshimi> I was thinking the heading could be changed to "Disclosed bugs and branches" and the the content could just be the count "6 bugs and 4 branches"
<wgrant> Kills off the column, and destroys the probably deliberately obscure "observer" terminology.
<wallyworld_> that works
<lifeless> rule of thumb: showing precise counts is expensive. Unless it matters, don't.
<huwshimi> is that doing too much for a summary though?
<lifeless> in this case, what matters is that you can click to expand the user/team and see what they have access to.
<wallyworld_> i guess it's up to the users whether it matters
<lifeless> I don't see how the exact count matters.
<lifeless> (after all, its potentially wrong as soon as we render the page :))
<wallyworld_> as i user i'd want some indication, perhaps not an exact count
<wallyworld_> sure, and so are amazon stock counts on their order page
<wgrant> lifeless: As an OEM project manager, I need to grant a distro engineer access to one of my bugs, to get their expertise.
<wallyworld_> but they still show them
<lifeless> wgrant: then you need to know that their access is restricted.
<wgrant> On the disclosure management page, I want to be able to see that he doesn't have access to 2000 bugs.
<wallyworld_> actually, shopping sites normally show < 5 in stock or > 5 in stock or whatever
<huwshimi> wgrant: But how do we show that without an exact count?
<wallyworld_> so not exact counts
<lifeless> wallyworld_: right, because thats all that matters to the user: can they make a reasonable order and get instant delivery.
<wallyworld_> i think we need some indication, but not exact if it's expensive
<lifeless> wgrant: so do you want a whiteboard to say 'this engineer should only access branch X' ?
<lifeless> wgrant: how will the project managers coordinate things?
<lifeless> anyhow, if the schema can do it brilliantly-fast, thats cool. It just sets off alarm bells every time I see this.
<wgrant> lifeless: Note that "the schema" is now invalid :/
<wallyworld_> tales question - <div tal:repeat foo=somefoos tal:condition foo.somecheck/> - fails because foo is not yet defined for the condition check
<wallyworld_> what's the best way to get around this?
<wallyworld_> i guess i need to use a nested div
<lifeless> foo is a collection ?
<huwshimi> So, do we want the "Disclosed bugs and branches" column and if so what will the content be if it's not exact counts?
<lifeless> sorry, somefoos is ?. anyhow, the condition runs before the loop
<wallyworld_> somefoos is pseudo code. bugger about conditiob running before loop
<wallyworld_> i'll have to rework the tales
<wallyworld_> i want the condition on each item in the loop
<wallyworld_> huwshimi: we want the column with approx counts i reckon
<huwshimi> wallyworld_: How do you represent an approx count?
<wallyworld_> that's for you as ui designer to tell us :-P
<StevenK> "~ 4 branches" ?
<wallyworld_> or perhaps "< 10" or ">10"
<huwshimi> wallyworld_: Right, but I have no idea what an approx count is
<huwshimi> wallyworld_: Ok, thanks
<lifeless> one of the reasons I was challenging the column is that it will get long
<lifeless> bugs
<lifeless> branches
<wallyworld_> 10 is arbitrary
<lifeless> milestones
<lifeless> series
<lifeless> blueprints
<lifeless> answers
<huwshimi> wallyworld_: So we can get the database to return roughly what the user can see?
<lifeless> will all eventually be assets accounted for
<lifeless> huwshimi: we can ask it to estimate, it sometimes is spooky accurate, and sometimes heinously wrong
<wallyworld_> lifeless: can postgres return approx counts efficiently? i think so, based on previous irc conversations?
<huwshimi> lifeless: Ah great. Thanks
<lifeless> it can return the plan estimate
<wallyworld_> but it would be accurate for small numbers no? like > 10 or < 10 ?
<lifeless> sometimes
<huwshimi> wallyworld_: but would we no whether it was >10 or <10?
<huwshimi> wallyworld_: or just ~10
<wallyworld_> not sure
<huwshimi> *know
<wallyworld_> i don't think it matters
<wallyworld_> i would just want to know if my exposure to that user is big or small
<huwshimi> wallyworld_: Well it matters in how we represent it
<wallyworld_> to catch situations like wgrant described above
<wallyworld_> hmmm
<huwshimi> wallyworld_: 10, >10 and <10 are all very different things
<wallyworld_> yes. i don't think we need "10" though.
<wallyworld_> i would want to know < some number or > some number
<huwshimi> maybe we should kill this column and if people want to know what the user can see they have to view the person/team details
<huwshimi> cause >10 is really a meaningless number to represent
<wallyworld_> perhaps, especially if it expands to more than bug and branches
<wallyworld_> > 10 is not meaningless - it depends on what you want to know
<huwshimi> wallyworld_: well it is meaningless if the exact number is 2,000
<wallyworld_> ie have you accidentally allow the user to see more than just the 1 or 2 branches you assigned them to
<huwshimi> wallyworld_: I know in reality the estimation won't be that far off, but the user doesn't know that
<wgrant> Er
<lifeless> huwshimi: it may be that far off
<wgrant> The estimation may be orders of magnitude off.
<lifeless> huwshimi: approx *really is* approx
<huwshimi> lifeless: Oh
<lifeless> the accuracy depends a great deal on how complex the query is
<huwshimi> lifeless: OK, thanks that helps make the decision :)
<wallyworld_> lifeless: is it accurate for 0 or not 0?
<lifeless> wallyworld_: no
<wallyworld_> wtf?
<wgrant> It doesn't use indices.
<lifeless> wallyworld_: 0 is not special cased
<wgrant> It uses randomly gathered stats.
<wallyworld_> weird design then
<lifeless> wallyworld_: row count is just a vector, how many rows it thinks will be returned.
<lifeless> wallyworld_: it may think that 3 rows will be returned for a query that returns none
<wallyworld_> ok. i don't like postgres anymore then
<lifeless> wallyworld_: theres no possible way it can know a priori that nothing*can* be returned unless it executes the query
<lifeless> wgrant: did your previous schema report counts efficiently?
<wgrant> In my late schema it isn't denormed, but it's only through one join and row counts will be low unless people are doing stupid things.
<wgrant> I didn't try getting counts on Ubuntu-scale data.
<wgrant> But I could.
<wgrant> Not that it's going to be immensely useful, because the schema has to change.
<lifeless> why ?
<wgrant> Probably adding an intermediate table between APA and AP.
<wgrant> Because we need multiple policies per artifact.
<lifeless> ELOST
<wgrant> We must support multi-pillar private security bugs.
<wgrant> Therefore we cannot have a single policy per bug.
<wgrant> Therefore APA.policy cannot be.
<StevenK> wgrant: And another look?
<poolie> (back)
<poolie> lifeless, re your comment above, my branch does still show the active values, just not the scopes that caused them
<lifeless> cool
<StevenK> wgrant: Do you want to actually scribble on this MP, or shall I bug the OCR?
<wgrant> StevenK: Looking right now, actually.
<wgrant> Sorry, been distracted in -ops and stuff
<wgrant> Approved
<StevenK> wgrant: Thanks, tossing at ec2.
<StevenK> check_permission('launchpad.View', team) in a view magically looks up the user, or am I Doing It Wrong?
<poolie> stevenk i think it does use the current user
<wgrant> StevenK: It uses the user from the current request.
<poolie> lifeless, maybe we can make the 404 page just more obvious?
<poolie> that won't fix everything but it will help the common case
<poolie> nm actually
<lifeless> stub: so, catchup time ?
<stub> lifeless: sure
<bigjools> morning
<nigelb> Morning bigjools :)
<adeuring> good morning
* allenap changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: allenap | Critical bugtasks: 3*10^2
<nigelb> Hi, can I have the launchpad task removed from this bug? bug 884610
<_mup_> Bug #884610: Change the attendee in +temp-meeting-export <Launchpad itself:Triaged by nigelbabu> <Summit:New> < https://launchpad.net/bugs/884610 >
<nigelb> It seems we can solve this in summit itself and patching launchapd isn't necssary.
<rvba> allenap: I know you're busy right now but can you add this MP to your queue? https://code.launchpad.net/~rvb/launchpad/private-ppa-bug-890927-2/+merge/84083
<allenap> rvba: Sure can :)
<allenap> nigelb: I'll have a look
<bigjools> I already removed it
<nigelb> \o/
<nigelb> Thanks!
<allenap> bigjools: That would have been my first bugtask deletion ;-(
<bigjools> allenap: lol, sorry.  It was my 2nd :)
<allenap> :)
<rvba> allenap: I spoke with Gary about this so maybe he will be willing to have a look too.
<bigjools> I think I did the first one in prod when it went live
<nigelb> Thanks you guys *so* much for this feature :)
<nigelb> So many unnecessary bugtasks lying around :)
<rvba> allenap: I've been force to monkey patch a view in a testâ¦ I really don't know why that is because the view in question is created via an url that exists.
<rvba> forced even
<allenap> rvba: I am intrigued...
<gmb> Oh, how I love developing with trusted.sql
<nigelb> Somehow I think its infactuation and not *real* love ;)
<gmb> nigelb: Possibly. I use it in the same way as I would use the phrase "Oh, how I love slamming the car door on my fingers in the morning."
<nigelb> gmb: Heh
<bigjools> is there anyone around who knows anything about the code importer?
<jelmer> bigjools: somewhat
<jelmer> bigjools: what's up with the code importer?
<bigjools> jelmer: trying to find out why it ran out of memory on a certain date/time
<bigjools> the logs are somewhat obtuse
<bigjools> jelmer: a worker was killedby an admin is it was hitting swap, I am trying to find out which worker, which branch etc
<mhall119> can someone tell me the URL for the Launchpad XMLRPC service where I can call isTeamPublic?
<mhall119> I'm trying to test the public teams access controll for launchpad lists
* jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: allenap, jcsackett | Critical bugtasks: 3*10^2
<abentley> adeuring, rick_h_: with deryck away, do you want to do the standup in 22 minutes or 1:22?
<rick_h_> abentley: either works for me
<adeuring> abentley, rick_h_: let's wait for deryck
<abentley> adeuring: In other words, you want to hold the stand-up in the North American afternoon?
<adeuring> abentley: it's still afternoon for me too ;)
<abentley> adeuring: Even the morning is after the previous day's noon :-)
<adeuring> abentley: right ;) But 4:30 pm local time is still pretty early for me
<abentley> adeuring: cool.
<abentley> adeuring: Did you want any more help with the work you're doing?
<adeuring> abentley: no, thanks, I'm making some slow progress
<abentley> adeuring: okay.
<sladen> bzr log -r 14418..14419
<sladen> is that doing the opposite to what what poolie was working on with Like!/+1 etc?
<rvba> sladen: indeed. Our technical architect has spoken: external js/css is now verboten.
 * nigelb hugs james_w 
<sladen> rvba: sur gut
<abentley> mrevell: I have some questions about getting rid of the hot bug listings.  Do you have time for a chat?
<allenap> jcsackett: Thanks for the Approved.
<jcsackett> allenap: you're welcome.
<mrevell> abentley, I do at around 17.00 UTC, if that's any good.
<abentley> mrevell: sure, let's do that.
<mrevell> Great.
 * mrevell has been on the Launchpad team five years today :)
<rick_h_> mrevell: congrats?
<rick_h_> or hould we be sending bottles to a certain address?
<mrevell> https://launchpad.net/~launchpad-dev/+mailing-list-subscribers
<mrevell> oops
<mrevell> danhg, ^^^
<mrevell> rick_h_, Yeah, congrats definitely :) And bottles would be lovely, haha
<Ursinha> mrevell: congrats! :)
<mrevell> Thanks Ursinha :)
 * bigjools shakes mrevell's hand
<abentley> mrevell: chat?
<abentley> mrevell: sure.
<mrevell> thanks
<mrevell> abentley, Does Skype work for you?
<abentley> mrevell: sure.
<mrevell> abentley, https://dev.launchpad.net/Projects/CustomBugListings/Design
<lifeless> morning
<deryck> abentley, ping.
<abentley> deryck: pong
<deryck> abentley, hey, don't have long to chat since rick_h_ and I are sharing a sprint room, but he's lunching.  shall we mumble?
<abentley> deryck: let's.
<abentley> deryck: https://dev.launchpad.net/Projects/CustomBugListings/Design
<abentley> deryck: I figured it out: registry/browser/configure.zcml sets +bug-index as the default, but bugs/browser/configure.zcml actually defines +bug-index.  Never seen those done in separate place before.
<deryck> abentley, ah!  great, thanks.
<sinzui> abentley, bug is all messed up by bugtask
<abentley> sinzui: I don't understand.
<sinzui> abentley, I favour deleting all bug domain entries from registry
<abentley> sinzui: me too.  I'll do what I can.
<sinzui> abentley, sorry I misread the location of the configure.zcml
<abentley> sinzui: Does moving the browser:defaultView entries for BugsLayer out of registry/browser/configure.zcml sound good to you?
<sinzui> yes please
<abentley> sinzui: great.
<abentley> deryck: there's no way of feature-flagging this change, but I think even non-beta users will be pleased by it.
<abentley> mrevell, deryck: Another point in favour of a single display rather than configuring between two-row/one-row is that widescreen users may alternate between half-width and full-width browsers.
<wallyworld_> sinzui: i have to drive my wife to work today and will be absent at standup. could we have a chat now?
<sinzui> yes
<wallyworld_> jcsackett: hi, i just saw the email about the mockups. i can make the changes today if you haven't already
 * wallyworld_ runs away to drive wife to work. back soon
<rick_h_> deryck: http://uploads.mitechie.com/lp/lp_spinner_indicator.png
<jcsackett> wallyworld_: indeed, i didn't get to said mockup changes. however, if you don't manage to get much done on that front today I should be able to tackle them tomorrow.
<wallyworld_> jcsackett: shouldn't take too long. i'll let you know if i don't get it done
<jcsackett> wallyworld_: dig.
* jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugtasks: 3*10^2
* wallyworld_ changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: wallyworld | Critical bugtasks: 3*10^2
<wallyworld_> sinzui: there's currently no protection around icon in the team tales formatter; the icon is always rendered. so i don't think we need a test for that
<wallyworld_> same with logo from what i can tell
<wgrant> lifeless: Any objection to letting team admins promote other admins?
<wgrant> (with a view to abolishing team owners eventually)
<lifeless> given that delegated operation of teams is a feature, yes.
<lifeless> letting team admins promote other admins I have no intrinsic objection to.
<wgrant> We can't abolish ownership without separating member/admin, right.
<lifeless> perhaps I wasn't clear
<lifeless> abolishing ownership is undesirable
<wgrant> Why?
<lifeless> 'delegated operation of teams is a feature'
<wgrant> Assuming you can be an admin without being a member, why do we need an owner?
<lifeless> are you saying 'there are no differences between owners and admins if admins can promote themselves' ?
<wgrant> No.
<wgrant> If we change the current fact that admin privileges imply membership, the only benefit ownership provides is that it can't be revoked by the other admins.
<wgrant> (as well as confusing people to death, and introducing obscure special cases in our and others' code)
#launchpad-dev 2011-12-02
<lifeless> wgrant: whats driving this; I thought we were happy with the owner/admin/member system
<wgrant> lifeless: Owners are awkward, because they're an easily forgotten special case and they need to be queried for specially when reporting on access.
<wgrant> And admins-can't-promote-admins confuses everyone, so needs to be abolished. And conflating admin+member is a bit silly and hard to display. Which means that ownership doesn't really have a purpose.
<lifeless> wgrant: but owners don't get access, so they don't need special handling
<wgrant> lifeless: Huh?
<lifeless> wgrant: we stomped that out 6 months ago
<wgrant> lifeless: My OEM project grants access to a project team.
<lifeless> owners no longer get benefits granted to the team
<wgrant> The project team's owner has resigned :(
<lifeless> the team owner doesn't get access to the project
<wgrant> But my disclosure reporting views don't show the owner.
<lifeless> and they don't need to.
<wgrant> Despite they fact that they can add themselves or others to the team.
<wgrant> The reports are pretty damn useless if they don't tell me that Joe Random Former Employee can do whatever he wants.
<lifeless> Joe Random could be in a private subteam with a legit sounding name and wouldn't be shown either
<wgrant> Sure, and that's the price we pay for supporting unstructured private teams.
<wgrant> But that can at least be detected.
<wgrant> If you hide the owner, it can't be.
<lifeless> so, if admin doesn't imply membership you need the same special case for admins
<wgrant> If you have an invisible private team in your access list, you have to trust the team. We know that.
<wgrant> Yes.
<lifeless> so, I don't think that removing owner is a good idea at the moment
<lifeless> I acknowledge it may make sense
<lifeless> but it needs some thought, reasoning and consultatoin
<lifeless> right now, my brain is full of u1 stuff, and its late for the folk I'm talking to about that, so I'm going to stay focused on that for nw.
<wgrant> What have they broken now?
<lifeless> as far as admin promoting admins; I think we need to check with our users there as well; particularly losa/gsa etc
<lifeless> wgrant: they have a growing userbase
<huwshimi> wallyworld_, wgrant: Have you guys managed to have a chat about whether the manage-disclosure pages will need to be changed again due to the changes you were talking about yestereday?
<wgrant> We probably won't know for weeks.
<huwshimi> wgrant: Thanks, I won't worry about it unless I hear something then
<StevenK> Bleh. I think these failing tests assume that nominations created by admins are not automatically approved.
<sladen> poolie: no idea if that's the way to go:  lp:~sladen/launchpad/launchpad-microdata-bug-person  the way in which the data is dumped out doesn't immediately match up;  ask you've got eg.  _makeLink used around .unique_displayname
<wallyworld_> huwshimi: looking at your mp now. been having some stupid router issues
<huwshimi> wallyworld_: Ah, awesome :)
<lifeless> wallyworld_: how hard do you think it would be to adapt https://github.com/marcello3d/node-buffalo to run (just the serializer) in chrome+ff
<lifeless> ?
<StevenK> loltpg
 * wallyworld_ looks
<wallyworld_> StevenK: why loltpg? it's been hardware issues at my end :-(
<wgrant> lifeless: You haven't been turned off BSON yet?
<StevenK> wallyworld_: I saw 'router issues' and assumed.
<wallyworld_> StevenK: you know what they say about assuming :-)
<lifeless> wgrant: a bug in one implementation isn't sufficient to scare me off :P
<wgrant> Does anybody else use BSON?
<wgrant> At all?
<wgrant> Apart from in MongoDB?
<wallyworld_> we had a power outage last night and the router has been flakey ever since :-(
<rick_h_> mongo folks :)
<rick_h_> sourceforge uses it a bunch with their mongo stuff
<wallyworld_> lifeless: i've not seen that project before. i can't give you an immediate answer
<rick_h_> lifeless: what breaks? should just be able to inline the bson.js with the extern/xx files for their long support
<lifeless> rick_h_: cool
<wgrant> The only benefits it seems to provide are obfuscation and storing binary data so our applications can crash.
<lifeless> rick_h_: I haven't tried yet
<rick_h_> the only node parts are the require/packaging stuff
<rick_h_> wgrant: I'm with you, clarity/debuggable ftw
<wgrant> We've reinvented about 20 square wheels already, and used another 15 or so triangular wheels from other places.
<wgrant> Let's not add more...
<lifeless> wgrant: speaking of crashes, did you file a bug for that oops you ofund yesterday in +filebug ?
<lifeless> .
<lifeless> adfreakingsl
<lifeless> ...
<wgrant> lifeless: No, but matsubara id overnight.
<wgrant> did
<rick_h_> wtf, this thing will serialize code?
<rick_h_> heh, that's got to be interesting
<lifeless> rick_h_: I'd like to hook up an oops like thing in our js
<lifeless> rick_h_: to catch errors and report them immediately
<rick_h_> lifeless: cool
<rick_h_> I had something like that once for ajax requests in an app
<rick_h_> glboal error handler that would then fire off an api request to log and notify of the error with a load of client info
<wallyworld_> huwshimi: on the new screenshot, the affected project is not shown. where is it displayed now?
<huwshimi> wallyworld_: It's not
<wallyworld_> is that an issue?
<huwshimi> wallyworld_: Well, my branch doesn't address that. The intention is that we revert to not showing the affected branch/project like we had previously except on pages that previously displayed it
<wallyworld_> ah ok. i didn't recall that we previously did not show it
<huwshimi> wallyworld_: https://bugs.launchpad.net/launchpad
<wallyworld_> out of curiousity, if the user selects to show extra data like the project, it's in two lines then, right?
<huwshimi> lifeless: Can we really catch all our js errors while we have in page scripts?
<huwshimi> wallyworld_: Yeah, that's right
<wallyworld_> cool
<huwshimi> wallyworld_: There's some helpful comments from Matthew on the bug about that
<wallyworld_> ok
<huwshimi> wallyworld_: Pages like this show a package: https://bugs.launchpad.net/ubuntu'
<wallyworld_> so they do
<huwshimi> wallyworld_: So those will hopefully still show a package (although I doubt the functionality exists yet as I believe the customisation of the columns is site wide)
<wallyworld_> it's hard looking at those pages when the new bug listings look sooo much nicer
<lifeless> rick_h_: exactly
<lifeless> huwshimi: all of? maybe not. More than we do today? Definitely.
<rick_h_> lifeless: ok, I lied. It uses node buffers, which now needs node assert
<rick_h_> wheee, dependency chaining here we go
<lifeless> rick_h_: yah :(
<lifeless> rick_h_: so I imagine there is some porting needed
<rick_h_> lifeless: yea, and that needs util to do some inheritence binding
<huwshimi> wallyworld__: Thanks heaps for the review
<huwshimi> wallyworld__: I've been having router issues this week too, so I was gone for a sec then
<huwshimi> lifeless: Will the js errors be logged in the oops database? I'm just asking cause I want to set up a service for logging a/b test results (via ajax) and am interested to know if the architecture would be similar
<lifeless> huwshimi: yes
<huwshimi> lifeless: Ah ok
<huwshimi> lifeless: I was hoping to steal some code :)
<huwshimi> but this will have to be different
<lifeless> huwshimi: generic wsgi microservice that listens on http, sanitises some fields and forwards over amqp to the oops db
<lifeless> huwshimi: AIUI most a/b test frameworks use log files rather than actual services, but I am probably wrong
<lifeless> huwshimi: what do you want to have happen?
<huwshimi> lifeless: To record the results of the test we'll need some js to test that a link has been clicked etc. The results need to be stored somewhere. I had imagined we would send off an ajax request to some kind of service and it would dump it in a db.
<huwshimi> lifeless: And a tool that returns the results would be needed too
<lifeless> well, when the link is clicked, LP gets the request doesn't it ?
<huwshimi> lifeless: We won't necessarily just be testing links, it could be that someone uses a bit of js interaction, or scrolls to the bottom of a page, the possibilities are endless
<lifeless> sure
<lifeless> the easiest to deploy thing is just to make a GET request to <current-site>/+abtests/test/value with no referrer
<lifeless> this will go into our logs, and we can grep for it very easily
<lifeless> you can certainly do a service if you want
<huwshimi> lifeless: OK, that seems simple
<rick_h_> yea, just make things small enough to json/add to request
<lifeless> the no referrer will stop it oopsing
<lifeless> it will just be a random foreign request the appserveres and haproxy etc ignore
<rick_h_> could almost just do something like dynamically add an image to the page and go through apache logs or something
<rick_h_> are there real request logs out there?
<lifeless> yes, but restricted access (ip address is personal information)
<lifeless> however getting a script etc run on them is easy
<huwshimi> Anyone know how to turn on the beta banner locally?
<rick_h_> huwshimi: you need a feature flag with it limited in some way
<rick_h_> deryck showed me today
<rick_h_> set the buglisting flag for just your user, for instance
<rick_h_> and that would trigger the beta banner
<huwshimi> rick_h_: Ah right!
<lifeless> or just default 0 on
<lifeless> show it everywhere
<rick_h_> right, but if it's default I think it doesn't show the banner right?
<lifeless> it should, it just checks on some pages/templates for the flag evaluating
<lifeless> to on/off
<huwshimi> rick_h_: That worked
<rick_h_> huwshimi: cool
<huwshimi> lifeless: Wasn't working with default btw
<lifeless> ahm interesting
<huwshimi> lifeless: Which means we can't have features in beta for everyone, or at least have a notification for it
<lifeless> hmm, it just means there is a small bug somewhere :)
<lifeless> -or
<lifeless> IIRC the theory is that 'when feature X is enabled for everyone it is out of beta'
<rick_h_> yea, that's what I figured
<huwshimi> hmmm
<huwshimi> wgrant: In regards to the download-cache stuff we were talking about the other day, how do I actually commit changes? Do I do a bzr checkout of lp:lp-source-dependencies and just push the changes?
<wgrant> huwshimi: ~/launchpad/lp-sourcedeps/download-cache should be a bound checkout by default. You can commit straight to there, and it will push automatically.
<huwshimi> wgrant: Oh right, thanks :)
<huwshimi> A couple of reviews if someone wants them:
<huwshimi> https://code.launchpad.net/~huwshimi/launchpad/update-cssutils-version/+merge/84199
<huwshimi> https://code.launchpad.net/~huwshimi/launchpad/beta-banner-design/+merge/84198
<wgrant> huwshimi: Where'd you get that tarball?
<wgrant> They don't seem to distribute one, AFAICT.
<huwshimi> wgrant: you're right
<huwshimi> wgrant: I grabbed the tarball from the tagged branch
<lifeless> fwiw lp-land is the bomb
<wgrant> lifeless: Oh?
<huwshimi> wgrant: Is there a problem with that tarball?
<lifeless> wgrant: handles prodconfigs ok
<lifeless> wgrant: I'd forgotten to use it :P
<wgrant> huwshimi: Doesn't match the 0.9.7 tag tarball that I got from bitbucket. We might be better off using the official 0.9.7 zip.
<wgrant> There's nothing wrong with it, besides not being official to being unreproducable and thoroughly confusing.
<wgrant> s/to being/so being/
<huwshimi> wgrant: Oh, I couldn't find an offical one. I got mine from bitbucket, but the only place I could find was from the tagged branches
<wgrant> huwshimi: Yeah, odd that its hash doesn't match. There's no official tar.gz, but there is an official zip which we can use.
<huwshimi> oh and I removed the double containing folder
<wgrant> Ah
<huwshimi> wgrant: Oh right, should we just use the zip?
<wgrant> That would do it.
<wgrant> Probably, yes.
<huwshimi> wgrant: I actually didn't see the zip either
<wgrant> modifying upstream tarballs without labeling them as modified is a good way to get people upset :)
<wgrant> http://code.google.com/p/cssutils/downloads/detail?name=cssutils-0.9.7.zip&can=2&q= is one
<wgrant> I thought there was one on pypi too.
<wgrant> Although pypi makes it rather challenging to obtain a stable version...
<huwshimi> wgrant: Oh right, I didn't look at the google code page
<wgrant> Looks like they moved to bitbucket recently, and didn't bring across the old downloads.
<lifeless> .
<huwshimi> wgrant: Yeah ok, do you want me to replace it with the zip?
<wgrant> huwshimi: That would be cleaner, I think.
<wgrant> And sort of trivial.
<huwshimi> wgrant: No problems
<wgrant> Thanks.
<wgrant> huwshimi: Ahhhhh CSS3 gradients.
<wgrant> That is wonderful.
<huwshimi> :D
<huwshimi> wgrant: OK that file should be replaced
<wgrant> huwshimi: Indeed, thanks.
<huwshimi> wgrant: Thanks for the approve
<huwshimi> wgrant: Is there any reason to not just lp-land the change to versions.cfg?
<huwshimi> wgrant: Or should I be just as paranoid as I normally am
<wgrant> huwshimi: It's late on Friday, so we can't deploy for three days anyway... might as well just land.
<wgrant> It works fine here.
<huwshimi> wgrant: OK :)
<huwshimi> argh!
<wgrant> Uhoh.
<huwshimi> wgrant: Any ideas? http://paste.ubuntu.com/756731/
<wgrant> Hmm, that's pretty awesome.
<wgrant> Not one I've seen before.
<huwshimi> wgrant: I seem to have a knack of coming across pretty awesome bugs
<huwshimi> wgrant: Usually when it's friday afternoon and I want to land a few things :)
<wgrant> Hmm.
<wgrant> The beta banner doesn't seem to appear at all in IE9.
<wgrant> The page just sits there loading eternally.
<wgrant> Despite apparently having completely rendered otherwise.
<StevenK> Do we care?
<huwshimi> wgrant: Is that a js thing then?
<wgrant> huwshimi: Probably. Other pages finish loading.
<wgrant> It's the same on prod.
<huwshimi> wgrant: Strange
<wgrant> Unsurprisingly.
<wgrant> Yeash
<wgrant> yeah
<huwshimi> oh, I have no idea about this lp-land thing
<wgrant> pqm-submit, I guess.
<rick_h_> wgrant: https://bugs.launchpad.net/launchpad/+bug/894797
<_mup_> Bug #894797: bug portlet ajax calls break in IE and break js for other features <bugs> <ie> <javascript> <markup> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/894797 >
<rick_h_> wgrant: I tried to test some IE stuff and hit that
<wgrant> rick_h_: Aha.
<wgrant> Thanks.
<rick_h_> as soon as it hits an error it stops all other JS and the portlet stuff is pretty up there
<rick_h_> might be what you're hitting, so other pages without the portlets are ok
<wgrant> StevenK, wallyworld__: Could one of you review https://code.launchpad.net/~wgrant/launchpad/bug-728673/+merge/84205?
<huwshimi> wgrant: oh, does that do a similar thing?
<wallyworld__> wgrant: sure
<wgrant> huwshimi: lp-land is a wrapper around pqm-submit.
<wgrant> huwshimi: lp-land grabs the details from the MP and invokes pqm-submit the right way.
<huwshimi> wgrant: oh right
<wgrant> But normally if you're submitting to devel you can just say 'bzr pqm-submit -m '[r=wgrant] Blah blah blah I am a commit message'
<huwshimi> wgrant: Should I just ec2 land this? :)
<wgrant> That might fail the same way as lp-land.
<wgrant> But maybe not.
<huwshimi> wgrant: I'll give it a go
<rick_h_> ec2 land will run ec2 test first
<rick_h_> so beware it'll run for a bit if you were trying to avoid the test run
<huwshimi> rick_h_: Yeah, but I'd rather that than accidentally screw something up :)
<wallyworld__> wgrant: why remove the sprb formatter instead of just adding the permission check?
<wgrant> wallyworld__: Because the only other difference between the two is that in one the archive was linked, while in the other it wasn't.
<wgrant> wallyworld__: Which I've wanted to fix for a while anyway.
<wgrant> wallyworld__: There's no reason for the SPRB one to exist separately.
<wallyworld__> but the sprb formatter displays different text
<wgrant> Does it?
<wallyworld__> it appears to from my reading of the code
<wgrant> If you look at the tests I changed, only the closing tag is differnt.
<wallyworld__> ah, i think you are right
<wallyworld__> i've misread the code i think
<wgrant> It's easy to do with some of the formatters we have :/
<wallyworld__> yeah. thanks for humouring my question. i thought it best to double check
<wallyworld__> wgrant: r=me. also, did you see my email in reply to sinzui's email?
<wgrant> wallyworld__: I replied to it like 2 minutes later.
<wgrant> Thanks.
 * wallyworld__ hits refresh on thunderbird
<wallyworld__> wgrant: ah, i see the problem now.
<wgrant> The private team implementation looks to have been a little lazy :)
<wgrant> That's interesting, actually.
<wallyworld__> wgrant: i think we have a hole in that the only protection is via travsrsal, no?
<wgrant> Person searches may be a little revealing.
<wgrant> Yes.
<StevenK> wgrant: So how will SPRBs show up with your change?
<wallyworld__> since my dicking with a test allowed me to see attrs
<wallyworld__> StevenK: the same as now afaiui
<wgrant> StevenK: Same as BPBs. Building _$title_ [$person/$archive]
<wgrant> Or 'Building private job'
<StevenK> Right, then the formatter was buggy and useless
<wgrant> Rather than the current 'Building _$title [$person/$archive]_' and crash.
<wallyworld__> wgrant: so if i fix this security issue, it will restore the current traversal protection and plug an access hole
<wgrant> wallyworld__: Well.
<wgrant> wallyworld__: You need to restrict all attributes to launchpad.View or launchpad.LimitedView rather than zope.Public.
<wgrant> This may cause a lot of fallout.
<wgrant> It remains to be seen.
<wallyworld__> it does. but in theory, moving the IpersonPublic stuff behind a security adaptor which allows full access for public teams and delegates to the limitedview security adaptor for private teams should be the right thing to do
<wgrant> It's certainly the right thing to do.
<wgrant> I just think it may well break stuff.
<wgrant> But we have to do it.
<wallyworld__> yes. i'll do it and see what breaks in ec2 :-)
<wgrant> Unlikely to catch everything, but not much more we can do.
<wallyworld__> IPersonPublic is not really the correct name anymore either
<wgrant> Indeed.
<wgrant> It hasn't been for a while.
<wgrant> Move stuff onto IPersonView/IPersonLimitedView
<wallyworld__> not limited view - we only want access to name, url, displayname for linited view
<wallyworld__> should be moved to IPersonRestrictedView perhaps
<wgrant> Well, yes, clearly only move the appropriate stuff onto LimitedView.
<wallyworld__> IPersonRestrictedView required launchpad.View permissions
<wallyworld__> which is what we want
<wgrant> Right.
<wallyworld__> and the security adaptor for that permission allows public teams unfetted access
<wallyworld__> which is what we want
<wallyworld__> here's hoping.... that not too much breaks
<micahg> is it known the distro selection widget is broken?
<micahg> on bugs that is
<micahg> it shows the first one in the list as selected when trying to modify multiple values at once
<jtv> micahg: I'll have a look in the open bugs.
<micahg> jtv: thanks, I can file a bug if one doesn't exist
<jtv> micahg: doesn't look like we have one open for this.  So yes, please do file one!
<micahg> jtv: stale page cache on my end, everything's fine :)
<jtv> Heh
<jtv> Unless the next person is going to have exactly the same problem, of course, in which case it doesn't really matter that things are fine in theory.  :-)
<micahg> well, my use case was old bugs open in my session for quite a while, refresh still showed the issue, forced refresh shift+f5 fixed it
* wallyworld__ changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugtasks: 3*10^2
<micahg> wallyworld__: trying to scare people :)
<wallyworld__> micahg: no?
<wallyworld__> you mean the bug count?
<jtv> hi wallyworld__ â on your way to the weekend?
<micahg> wallyworld__: heh, yeah :)
<wallyworld__> jtv: i wish :-( I have to finish some coding
<jtv> Oh well
<wallyworld__> micahg: it was already like that. i just changed the reviewer :-)
<micahg> wallyworld__: ah, ok, I must have missed that :)
<wallyworld__> jtv: but i plan on popping a cork to help :-)
<wallyworld__> micahg: but when it's written in that notation, it is scary for sure
<micahg> wallyworld__: sorry, it's been like that for a while and I missed it :), yeah, I did a double take
<wallyworld__> np :-)
<bigjools> morning
<adeuring> good morning
<jtv> bigjools: I have two callback chains that I need to interleave just so for my test.  I would expect to have to orchestrate the events somehow.
<jtv> hi adeuring
<adeuring> hi jtv
<jtv> adeuring: are you reviewing today?  If so, could you review one for me?  It's https://code.launchpad.net/~jtv/launchpad/bug-849683-cloner/+merge/84222
<bigjools> jtv: let me just check the API
<adeuring> jtv: I'll look
<jtv> thanks
* adeuring changed the topic of #launchpad-dev to: : https://dev.launchpad.net/ | On call reviewer: adeuring | Critical bugtasks: 3*10^2
<bigjools> jtv: so you can call d.callback() to manually run the callbacks
<jtv> And that will stop at the point where the callback chain makes another async request, I take it.
<jtv> Do I then proceed with the new callback returned from the original callback?
<bigjools> if there are new Deferreds created then it gets harder, yes
<bigjools> jtv: I can't remember if Deferreds are processed in the order they are created in the reactor.  It's probably best not to depend on that anyway
<jtv> Right. ISTM any way I'm going to do this will be brittle: the interleaving will happen at the wrong point, and the test will either break in a mysterious way or become meaningless.
<bigjools> jtv: we can avoid that, we just need to make sure we fire Deferreds in a certain order
<jtv> But the order will be tightly coupled to internals that may change.
<bigjools> jtv: no, I mean that you have a 2 deferreds in the test, the rest are incidental
<jtv> The problem is that nothing interesting happens in the original deferreds.  They return new deferreds, and so on.
<bigjools> but it would help if we could construct it such that there are no other deferreds created
<jtv> I think we'd have to un-nest some functions.
<jtv> A bit annoying given how they rely on local variables of the surrounding functions.
<bigjools> jtv: the other option is to just keep calling the Deferred until None is returned
<bigjools> see http://twistedmatrix.com/documents/current/api/twisted.internet.defer.Deferred.html
<jtv> That's fine for one, but I need to get the interleaving right.
<bigjools> runCallbacks()
<jtv> Thanks.
<bigjools> it will be file
<bigjools> fine
<bigjools> the chain of Deferreds only applies to a single builder
<jtv> _runCallbacks?
<bigjools> yes
<bigjools> it stops when you get to a Deferred with no result, you'd just need to call _runCallbacks() on it
<jtv> I guess we could follow the chain with a single builder, but it'll be less like the integration test you were hoping for.
<bigjools> jtv: slightly less, but still valid
<bigjools> the interleaving we want is still there
<jtv> Then what exactly do we test for?  Abort after every step and see that the changes are still there?
<jtv> Or conversely, commit after every step, trigger a failure, and see that the changes are aborted?
<bigjools> we are testing the scenario I explained
<bigjools> storeBuildInfo() defers to a communication with the builder
<bigjools> in the meantime, another Deferred fires and aborts the txn
<jtv> But how do we get it to just that stage?
<bigjools> where "another" is in a different builder chain
<bigjools> we make a fake builder than doesn't return the results that the storeBuildInfo() wants
<bigjools> we can make it do nothing so the deferred does not return right away
<bigjools> hmmm do we even need to do that
<jtv> That's the question.  The problem is, we have to dig through at least some closures to get to a state where anything like the bug can happen.
<bigjools> yes yes, I have it
<jtv> then give it to me!
<bigjools> we use a fake builder that will sleep for a while when asked for the build info
<jtv> Argh!
<bigjools> not a real sleep
<bigjools> it's a callback-based one
<bigjools> bear with me
<bigjools> then we set up the other builder that will abort
<bigjools> we can use a special reactor were you can wind the clock forwards but less than the sleep interval
<bigjools> it'll make the 2nd builder abort
<bigjools> but the first is still "waiting"
<bigjools> ok?
<bigjools> I can show you an example
<jtv> Actually the part where the other builder fails can be simpler: we can call _scanFailed.
<bigjools> or that
<bigjools> it's easier to poke in a BrokenSlave
<jtv> Already have that, but it saves some reactoring.
<jtv> If we can just make the reactor return to us while storeBuildInfo is waiting to run, the rest should be easy.
<bigjools> look in lib/lp/buildmaster/tests/test_manager.py
<bigjools> search for Clock
<bigjools> it's a reactor that you can use instead of the testing one
<bigjools> let's you control how Deferreds are fired
<bigjools> by winding time forwards
<jtv> task.Clock?
<bigjools> yes
<jtv> I guess I need to adapt SlaveScanner.__init__ to accept a clock argument for testing, similar to the other scanners?
<bigjools> yes
<jtv> Hmmâ¦ actually that only exists on NewBuildersScanner, which passes it to its LoopingCall.
<bigjools> it's to override the LoopingCall
<jtv> The SlaveScanner has some equivalent to that?
<bigjools> yes
<jtv> Ah, it has one too
<bigjools> it sets the clock on the LoopingCall
<bigjools> this allows you to wind forwards one scanner but not another
<jtv> bigjools: I can see how the clock would let me force the LoopingCall into action, but why not just do that by calling scan()?  The trouble is getting a simulated chain of interaction going and stopping it at the right point.
<adeuring> jtv: the changes look good, and thanks a lot for the nice reformatting. just one question: what about a test?
<jtv> adeuring: good point â we know that the column is initialized, but it'd be nice to check it for correctness.  Let me just see what the existing tests do.
<adeuring> jtv: great, thanks!
<bigjools> jtv: you need to set up a FakeBuilder so it pauses in the right place
<bigjools> then you advance it to the pause point
<bigjools> then run the other builder scan to completion
<bigjools> jtv: you need a callLater() in the FakeBuilder to make it wait a bit
<jtv> That means that the _runCallbacks will stop at that point?
<bigjools> don't use runCallbacks
<jtv> Oh
<bigjools> use the Clock() as the reactor
<jtv> And I guess I need a FakeSlave?
<bigjools> yes
<jtv> Not a FakeBuilder?
<bigjools> sorry, yes
 * gmb just realised that there's no TLA equivalent of "OTP" for "Having a conversation with another human being in real life"
<jtv> How on Earth did we get by all this time?
<nigelb> full sentences I guess ;)
 * gmb -> real life
<jtv> gmr irl omg
<allenap> bigjools, anyone: Do you think mbp will get miffed if I change the trunk branch of txfixtures (to one owned by ~launchpad instead of ~mbp)?
<wgrant> ~canonical-launchpad-branches, not ~launchpad
<jtv> allenap: knowing him, no, as long as it's (1) well thought-out and (2) clearly documented and notified.
<bigjools> I would not think so
<bigjools> probably just an oversight
<allenap> wgrant, jtv, bigjools: Cool, thanks all :)
<jtv> bigjools: meanwhile, back on the broad and twisted path to hell, I just discovered that the triggering event just before the simulated problem comes from the Librarian, not from the slave.
<wgrant> allenap: https://dev.launchpad.net/CreatingNewProjects has setup instructions
<allenap> Ta.
<bigjools> jtv: really?  I thought it was "        d = build.getLogFromSlave(build)"
<jtv> bigjools: and you were right.  But, and this is exactly why I went with those narrow read-write policies, that call hides async calls to both slave and librarian.
<jtv> Or maybe that's synchronousâ¦  I'm not really seeing things clearly.
<jtv> Ahhh no, I think that's synchronous.
<bigjools> jtv: it calls getFile() on the slave first and then does a blocking upload to the librarian.  Which is a bit crap.
<jtv> But in this case another welcome simplification.
<jtv> Even though I imagine eventually we would change it, and thus break the test.
<mrevell> Welcome danhg_
<jtv> This is all so much SA-80 and so little AK-47â¦
<jtv> hi mrevell
<mrevell> Hey there jtv
<jtv> bigjools: when I say SA-80 I'm referring mainly to how you're free to fire it left-handed, but it'll spit hot brass at you from the left and smash your nose from the front if you do.
<jtv> Actually surprisingly reasonable design choices when you look at the tradeoffs, but not foolproof.
<bigjools> jtv: you have a way with words
<jtv> bigjools: not implying you're a fool.
<bigjools> jtv: but are these footguns?
<jtv> If you like.  But the SA-80 probably more so, since it's a bullpup.
 * jtv has a particular thing against people who don't mind where they point their assault rifles, especially when he ends up in front of them.
<jtv> That's enough ramblings from my overheated brain.  I must go.
<bigjools> jtv: sounds like you need a good dose of paracetamol and bedrest
<jtv> bigjools: unfortunately I have a train to catch.  Promised to take care of my friends' house.
<jtv> On the bright side, I sleep better on those night trains.
<bigjools> I remember thoes
<bigjools> those
<bigjools> no station stop announcements!
<jtv> I think they do announce them _sometimes_â¦
<jtv> â¦probably to lull you into a false sense of security.
<nigelb> heh
<bigjools> well it lulled me into a sense of no sleep
<nigelb> bigjools: You've visited India yet?
<bigjools> I have no plans
<nigelb> aww :(
<bigjools> wgrant: yay you are fixing bug 728673
<_mup_> Bug #728673: BuilderSet:+index crash (/builders) <oops> <Launchpad itself:In Progress by wgrant> < https://launchpad.net/bugs/728673 >
<stub> jtv: Good luck with the train. Its a long weekend so will be packed!
<jtv> stub: I booked ahead, thanks
<jtv> good weekend everyone
* bac changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: adeuring, bac | Critical bugtasks: 3*10^2
<rick_h_> morning
<bac> morning rick_h_
<jcsackett> morning, all.
<rick_h_> morning jcsackett
<sinzui> gmb, deryck, allenap: Do either of you have time to talk about renaming BugTaskSearchParams.bug_supervisor or structural_subscriber ?
<deryck> sinzui, I'm sprinting and have two calls already this morning....
<deryck> sinzui, but if no one else can help, I can try.
<gmb> sinzui: I have a little time. What do you need to know?
<sinzui> gmb, I think I cannot remove  BugTaskSearchParams.bug_supervisor because it is in the api. I probably need to add structural_subscriber to support older api
 * gmb refreshes his memory
<sinzui> gmb. search is actually looking at structural subscriptions, not bug supervisor in *most* cases
<sinzui> gmb. The only case that may be interesting to search bug_supervisor is if you search from  team/user advanced search page and want to see all the bugs in all the projects the person supervises...
<gmb> sinzui: Right, I'm with you.
<gmb> I think you're right; I don't think bug_supervisor can be removed.
<gmb> (At least until version 2.0)
<sinzui> gmb: I can add a new param, and then fall back to the bug_supervisor is it was provided
<sinzui> *if* it is provided
<gmb> sinzui: Yes, I think that would work.
<jml> random fact: 1.2GB of space on my system was being taken up by postgresql logs in a hardy chroot that I made for LP hacking and then forgot about
<sinzui> gmb: bug_supervisor search will fail in many current cases because the default supervisor is the maintainer; the supervisor field is null. I think we should drop this case. We want to search for structural subscriptions on projects instead
<gmb> sinzui: I agree.
<sinzui> gmb: dropping the bug_supervisor column clause will make most searches faster because we already search ubuntu.bug_supervisor when the user is only looking for package subscriptions
<gmb> Nice
<sinzui> okay. I am going to try this and hope the api tests like me
<gmb> sinzui: Best of luck to you :)....
<sinzui> thank you very much
 * gmb can't take the feeling that he's managed to let sinzui talk himself into a footcannon
<sinzui> s/footcannon/disclosure/
<rvba> adeuring, bac, could one of you guys please have a look at https://code.launchpad.net/~rvb/launchpad/authorize-bug-898237/+merge/84269 ?
<adeuring> rvba: sure, i'll look
<rvba> ta
<deryck> abentley, standup time now ok?
<deryck> adeuring, can you hear me on mumble?
<adeuring> deryck: seems that mmle does use the tright sound output. just a second...
<deryck> adeuring, still no sound luck?
<deryck> adeuring, we hear you fine, FWIW.
<adeuring> deryck: right...
<adeuring> deryck: let me try another machine...,
<deryck> adeuring, ok
<deryck> adeuring, now we don't hear you at all.
<adeuring> deryck: wierd... on this machine, all sounds goes to the headset -- only mumble does not use it...
<deryck> adeuring, we need to start, so we'll go ahead.  we can chat with you via irc.  sorry.
<adeuring> deryck: ok, sorry for the mess...
<deryck> adeuring, we did hear you that time though.
<deryck> adeuring, right.
<deryck> adeuring, is mumble audio going to the right device?
<adeuring> deryck: yes...
<deryck> adeuring, I guess we lost you again?
<deryck> mrevell, my entire team will join for the check point.  will we skype or conf call in?
<mrevell> deryck, Hey, I'm happy with Skype but I can also do the conference call. Is Skype okay for the Orange squad?
<deryck> mrevell, yup, skype works for us.
<mrevell> Great
<dobey> deryck: i'm surprised you're not meeting on PSN Home ;)
<deryck> dobey, dude, if I could! :)
<mrevell> Okay, calling now. I don't have adeuring or rick_h_ in my Skype contacts.
<adeuring> mrevell: try "adeuring"
<rick_h_> mrevell: mitechie
<bigjools> HALP.  Running tests here and it gets as far as "Set up canonical.testing.layers.AppServerLayer" and then hangs.... anyone else seen that?
<mrevell> Sorry jcsackett, accident.
<jcsackett> mrevell: what was an accident?
<mrevell> jcsackett, I thought I added you to a Skype conference.
<jcsackett> mrevell: huh; i'm not on skype right now, so now worries. :-)
<jcsackett> s/now worries/no worries/
<mrevell> ok :)
<mrevell> deryck, abentley flacoste danhg rick_h_ adeuring matsubara https://launchpadlibrarian.net/86370773/single_line_bugs.png
<rvba> Hi gary_poster, the branch that refactors LaunchpadSecurityPolicy to have it cache sub checks performed by classes inheriting from DelegatedAuthorization is up for review! (that's the one we talked about a few days ago).  Gavin is reviewing it but maybe you will be willing to have a look, out of curiosity if nothing else ;).
<rvba> https://code.launchpad.net/~rvb/launchpad/private-ppa-bug-890927-2/+merge/84243
<mrevell> matsubara, "A way to specify the information displayed and ordering via the URL (bug 124342)"
<_mup_> Bug #124342: It should be possible to specify bug list sort order via the URL <bug-columns> <lp-bugs> <Launchpad itself:Fix Released> < https://launchpad.net/bugs/124342 >
<matsubara> thanks mer
<matsubara> thanks mrevell
<mrevell> deryck, https://dev.launchpad.net/Projects/CustomBugListings/Design
<adeuring> rvba: r=me (sorry for the delay)
<rvba> no worries, thanks for the review adeuring.
<bigjools> can I bug one of you guys for a review please
<gary_poster> rvba, awesome! what kind of performance improvement to you expect?
<rvba> gary_poster: well, the page ~canonical-isd-hackers/+archive/ppa/+index spends 8 sec doing the checks that will be cached nowâ¦
<rvba> So I expect it the performance improvement to be pretty serious in this case, and hopefully it will help else were as well.
<rvba> s/expect it/expect/
 * bigjools hi-fives rvba
<rvba> bigjools: you should wait till this passes QA ;)
<gary_poster> rvba, cool, me too.  I was hoping for hard (& exciting) numbers and I will look forward to them soon :-) ... have you timed this on qastaging so that you can get a relatively accurate comparison?
<bigjools> I can merge it on DF right now if you want to test
<rvba> gary_poster: well, this will fix repeated statements so it's pretty easy to guess what the gain will be (that is if everything does well).
<gary_poster> right
<gary_poster> I just like hard numbers :-)
<gary_poster> (so bigjools' offer sounds good to me, but I'm a bystander)
<rvba> gary_poster: hard numbers: https://launchpad.net/~canonical-isd-hackers/+archive/ppa spends ~6000 doing the repeated checks that will now hopefully be cached.
<rvba> 6000ms that is.
<rvba> bigjools: why notâ¦ but I'll have to go in ~30 minutes.
<bigjools> ok, doing a load test on DF pre-patch
<bigjools> rvba: which branch?
<rvba> bigjools: lp:~rvb/launchpad/private-ppa-bug-890927-2
<bigjools> that private PPA you posted *cough* takes 1.5 seconds pre-patch
<rvba> bigjools: on df yes.
<rvba> bigjools: https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-d84aaa754e954254c44f803f45871daf  https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1baf5eff75609f641e53992c0023acb6
<rvba> bigjools: can you access https://launchpad.net/~canonical-isd-hackers/+archive/ppa/+index ?
<rvba> (Not on df that is)
<bigjools> yes
<bigjools> v quick, because I am commercial admin
<rvba> bigjools: hum, also, on DF, we are admins so unless I'm mistaken the security checks are by passedâ¦
<rvba> Am I right?
<bigjools> rvba: not bypassed, just different
<rvba> bigjools: right, so it's not a good test.
<gary_poster> https://qastaging.launchpad.net/~canonical-isd-hackers/+archive/ppa takes 8 secs once it loads at all
<gary_poster> s 7.63 seconds
<rvba> Right.
<bigjools> gary_poster: can you try https://dogfood.launchpad.net/~canonical-isd-hackers/+archive/ppa
<gary_poster> trying
<gary_poster> bigjools, 1.57 s
<bigjools> woo :)
<gary_poster> so that is post-patch bigjools?
 * bigjools hi-fives rvba
<rvba> That's without the patch?
<bigjools> with patch
<bigjools> I can remove it to compare
<rvba> I'd like to see how it goes without the patch
<gary_poster> yeah
<bigjools> ok try now
<gary_poster> k
<bigjools> unfortunately puppet is hammering DF
<gary_poster> 2.77 s, 1.71 s, 1.51 s
<gary_poster> so no proof yet
<bigjools> there must be something else going on here
<bigjools> need to find a slow one
<rvba> Any private ppa with lots of different packages in it.
<bigjools> that you are a member of
 * deryck lunches with rick_h_ 
<tumbleweed> bigjools: re sponsor-syncs-bug-827555: I assume "UI changes" includes visibility in source_package_publishing_history API objects?
<bigjools> tumbleweed: no, web UI
<bigjools> the API will be good
<tumbleweed> Laney had a look at it and thought this merge wouldn't cover API visibility
<bigjools> oh bugger actually I forgot one thing
<bigjools> well then he's wrong :)
<tumbleweed> :)
<bigjools> I forgot to make the syncer known in the spph... darn.  Will fix that Monday.
<bigjools> good night all, have a nice weekend
<tumbleweed> bigjools: thanks
<jcsackett> bac: you free to review https://code.launchpad.net/~jcsackett/launchpad/redundant-message-is-redundant/+merge/84308 by any chance?
<bac> jcsackett: sure
* adeuring changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: bac | Critical bugtasks: 3*10^2
<bac> jcsackett: done
<cjwatson> real    3m23.752s
<cjwatson> ^- first cut at replacement for most of cron.germinate
<cjwatson> running on my laptop with some stuff involving networking, so I expect cocoplum will be able to do it faster
<jcsackett> thanks, bac.
<cjwatson> 3m9s once I ensure everything's mirrored locally
<cjwatson> that's with the major structural optimisations; there's surely still room for profiling
<deryck> bac, hi.  rick_h_ and I have a branch for review if you have the time.
<bac> deryck: sure do
<deryck> bac, Thanks!  https://code.launchpad.net/~deryck/launchpad/buglists-loading-885272-final/+merge/84183
<deryck> bac, and sorry for the length, but it's new js code, so not as much there as the line number indicates.
<bac> yowzer!  :)
<deryck> heh :)
<deryck> rick_h_, https://dev.launchpad.net/Orange/NewStarter
<deryck> rick_h_, https://dev.launchpad.net/MaintenanceRotationSchedule
<bac> rick_h_, deryck: i don't understand this sentence:
<bac> This is using the YUI concept of class extensions to get us min-in like
<bac> 225	+     * features from the Widget classes.
<bac> "min-in like" ??
<rick_h_> max-in
<rick_h_> mix
<rick_h_> that's me, meant mix-in
<rick_h_> typo, but we actually ripped that out
<bac> oh, ok
<rick_h_> so that comment shold go away
<rick_h_> should...if I could type
<bac> :)
<abentley> rick_h_: Nobody's prefect.
<abentley> deryck: AFAICT person/+portlet-otherpackages is completely unused, but would be accessible via URL hacking.  Kill it with fire?
<deryck> abentley, death die dying dead kill it :)
<bac> rick_h_: Did Deryck write this or are you speaking Southern now?  "// Mess with the position of target div."
<deryck> heh
<rick_h_> bac: lol, I don't know, it is late in the week and I've been hearing too much twang this week
<deryck> I think I wrote that, bac
<flacoste> bac: there is https://code.launchpad.net/~flacoste/launchpad/self-hosted-3rd-party-js/+merge/84319 in needs of a reviewer when you are free
<bac> flacoste: ok.
<bac> deryck: I see this a few times:  Assert.isFalse(this.indicator.get('visible'), 'visible is not set');
<bac> Isn't the error message backwards?  The assert fails because visible is set.
<rick_h_> the message was more a helper for us to see in the log that it came back wrong
<rick_h_> because that onlyshows when it failed
<bac> rick_h_: right, but it seems the message is backwards
<rick_h_> so "what we did wrong" vs "what is the actual check"
<deryck> bac, yeah, what rick_h_ says. I'd prefer to drop the message.  It is backwards, but it was meant more has "did the visible check fail or the other assert fail"
<deryck> s/has/as/
<bac> flacoste: is the diff correct?  1921 lines?  (i suspect it is)
<flacoste> bac: probably, i haven't checked the number of lines, but most is ignorable, unless you want to review obfuscate Google GS code ;-)
<flacoste> JS
<bac> GoogleScript -- its coming
<jcsackett> bac: isn't it called Dart? :-P
<bac> flacoste: what is this doing, in the CSS?
<bac> src: local('Ubuntu Italic'), local('Ubuntu-Italic'), url('https://themes.googleusercontent.com/static/fonts/ubuntu/v3/kbP_6ONYVgE-bLa9ZRbvvvesZW2xOQ-xsNqO47m55DA.woff') format('woff');
<bac> flacoste: it is still loading from google, no?
<flacoste> bac: is is, but that's the web font specification
<bac> but just the font file, so it is ok?
<flacoste> which shouldn't be a problem
<flacoste> right
<flacoste> only the font file
<bac> ok, that was the only suspect thing i saw
<flacoste> CSS can be used to load JS extensions
<flacoste> so that's why we removed the external CSS loading
<bac> right
<flacoste> but i'm not aware of any such vulnerability in woff
<flacoste> so there have been a bunch of vulnerability in woff decoders over the years
<flacoste> but that's usually in the form of a local exploit, rather than an attack against the site linking to it
<flacoste> although one could say that if I root your browser, it's as good as if I could execute JS in it
<flacoste> but i don't think we should care about that
<rick_h_> flacoste: did we look at caja or adsafe for running 3rd party JS safely?
<rick_h_> flacoste: or just mandate it's not friendly regardless of method of inclusion?
<flacoste> rick_h_: we haven't
<nigelb> jml: Can you turn off the logging to errors only?
<nigelb> jml: Nevermind, stale scrollback :/
<flacoste> rick_h_: caja looks nice
<rick_h_> flacoste: yea, watched a yui video from crockford that mentioned those last night
<rick_h_> and wondered if they might help us keep some 3rd party js in a safe manner
<flacoste> i think it does
<flacoste> since you can deny the contained JS access to the user cookies
<flacoste> which is what we are protecting for
<flacoste> because if you have the user's cookie, you can basically do anything you can over the API or web as that user
<flacoste> which is a lot
<sinzui> If we had a charm to setup caja scripts, we could call it cajajuju. I would put a Kajagoogoo easter egg on the page. The "Too shy" song would play when your mouse crossed the 3rd party cookies text the description
<abentley> bac: could you please review https://code.launchpad.net/~abentley/launchpad/packagebugs/+merge/84329 ?
<bac> abentley: yes
<abentley> bac: thanks.
<bac> abentley: done,  thanks.
<deryck> bac, FWIW img is a naturally self closing tag. ;)
<bac> deryck: :)
<deryck> bac, but I changed div to match.  I'm actually not that strongly opinionated about that, despite any past statements of mine. :)
<bac> deryck: it is not something to get too worked up about!
<deryck> indeed
<deryck> YUI itself uses both forms.
<abentley> bac: thanks.
<abentley> bac: I extracted that code with minimal changes, but I'm happy to tweak its handling of "advanced".
<bac> abentley: your call.
<abentley> bac: I think it actually makes more sense to do that by doing params['advanced'] = '1'
<bac> abentley: +1
* bac changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugtasks: 3*10^2
<wgrant> flacoste: I don't believe you can do that.
<wgrant> flacoste: The font names are not meant to be staitc.
<wgrant> I'm glad we're going to send all our private URLs and other data to Google again :)
<lifeless> wgrant: you know we have google dcs etc
<lifeless> *docs*
<lifeless> wgrant: ga getting our data wasn't ever an issue; the risks around site compromise were
<wgrant> Sure, but I still don't like it at all :)
#launchpad-dev 2011-12-03
<wgrant> Hmm
<wgrant> Apart from the importance colour scheme, the latest iteration of the new bug listings (on qastaging) is pretty damn nice.
<rick_h_> wgrant: cool
<wgrant> Bug #899477, too
<_mup_> Bug #899477: New bug listings have no style for Expired status <bug-columns> <trivial> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/899477 >
<maxb> hmm... weird
<maxb> apparently /people/+requestmerge has allowed someone to submit a request to merge a team into a person?!
<jelmer> maxb: it wasn't a team at the time
<jelmer> just an unclaimed contribution
<jelmer> I quickly registered it as a team because the link was visible in the team archives
#launchpad-dev 2011-12-04
<Noldorin_> hi folks
<Noldorin_> any windows devs around herE?
<rick_h_> Noldorin_: that'll be a bit hard to come by I think
<Noldorin_> rick_h_, not really. some of the launchpad devs do stuff on windows
<rick_h_> Noldorin_: ok, so rare then perhaps :) I don't think anyone on my squad uses windows and my experience in asking for IE testers has run into a little challenge from time to time
<Noldorin_> rick_h_, i know mgz does.
<Noldorin_> but fair enough
<wgrant_> Ah, he's bzr, not LP.
<Noldorin_> i admit it's a lot rarer than unix devs ;-)
<Noldorin_> wgrant_, oh right
<wgrant_> A couple of bzr people might use Windows regularly.
<wgrant_> But no LPer does AFAIK.
<Noldorin_> yeah, fair point
<rick_h_> ah, ok makes sense
<Noldorin_> wgrant_, know of any other bzr guys who do windows stuff regularly btw?
<cjwatson> How hard would it be to get mawson synced up to current reality and have some kind of publisher run done that includes cron.germinate?  I'd like to have a control point before a germinate upgrade.
<cjwatson> This is for bug 899972.
<_mup_> Bug #899972: cron.germinate is very slow <Launchpad itself:New for cjwatson> < https://launchpad.net/bugs/899972 >
<lifeless> mrning
<rick_h_> afternoon lifeless
<lifeless> cjwatson: that will take ~48 hours probably, maybe more.
<lifeless> hey rick_h_
<lifeless> rick_h_: how was your induction sprint ?
<rick_h_> lifeless: heh, a bit crazy
<rick_h_> but some good stuff
<wgrant> lifeless, cjwatson: Indeed, takes well over 24 hours to restore the DB. And DB restores are in an undefined, broken state at the moment, so it would take some massaging to fix.
<wgrant> removed: lib/canonical/launchpad/pagetitles.py
<wgrant> omg
<wgrant> sinzui wins.
#launchpad-dev 2012-11-26
<nigelb> mrs_poolie: lol. epic nick there :)
<lifeless> mrs_poolie: \o/
<mrs_poolie> hi lifeless
<lifeless> mrs_poolie: back in Sydney ?
<mrs_poolie> lifeless: yes, we have been for about a week. just doing my normal rounds as fire-putter-outter this week
<lifeless> cool
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/archive-edit-dependencies-oops/+merge/136075
<wgrant> StevenK: I'd suggest not hardcoding those components
<wgrant> eg. I can probably set contrib without an API OOPS
<wgrant> It seems like you could create the SimpleTerm at runtime given the current component.
<StevenK> Ah, right
<StevenK> wgrant: Do you object to the wording?
<wgrant> Yes, but I'm not sure there's much better, so it'll do
 * StevenK tosses up between "if default_value not in (None, multiverse):" or "if default_value and default_value != multiverse:"
<StevenK> wgrant: Diff updated
<bigjools> halp: I have a recipe for raring that's ending up in quantal: https://code.launchpad.net/~julian-edwards/+recipe/maas-daily
<bigjools> oh hang on, I bet it's matsubara's stuff
<StevenK> Bloody hell, what is the point of mentioning OOPSes in bugs if they get pruned anyway.
<wgrant> StevenK: Which?
<StevenK> wgrant: https://bugs.launchpad.net/launchpad/+bug/1007131 comment 1
<_mup_> Bug #1007131: MalformedHunkHeader: Malformed hunk header.  Does not match format. '<!--\r <bzr> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1007131 >
<wgrant> StevenK: Hm, the ones from 2012-11-05 got pruned?
<StevenK> wgrant: Yup
<StevenK> Not that they're incredibly helpful
<StevenK> I have this feeling that it was a short read or something, but I'm still struggling to figure out how it was called
<StevenK> wgrant: Can haz review again?
<wgrant> StevenK: Looking
<wgrant> StevenK: Done
<StevenK> wgrant: I changed the docstring for the method -- I can move that down to be just before the if block if you wish?
<wgrant> Ah, I guess the docstring isn't too far away, so it's probably fine
<StevenK> ~ 20 lines
<StevenK> or so
<StevenK> Bleh, I can't work out how bzrlib.patches works
<StevenK> I have a diff that gets to raise MalformedHunkHeader, because of a bug that is setting a header from the file to be '<!--'
<StevenK> OH
<StevenK> It looks like it does not love dos line endings, and manages to skip most of the file
<adeuring> good morning
<czajkowski> Aloha
* benji changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On-call reviewer: benji | Firefighting: - | Critical bugs: ~170
<deryck> jcsackett, ping for standup
<rick_h_> adeuring: what branch did you need looked at?
<adeuring> rick_h_: justr a second; need to write the mp...
<rick_h_> ah ok cool
<adeuring> rick_h_:  https://code.launchpad.net/~adeuring/launchpad/bug-1056881-2/+merge/136194
<rick_h_> adeuring: r=me
<adeuring> rick_h_: thanks!
<abentley> rick_h_:
<sinzui> benji, , do you have time to review https://code.launchpad.net/~sinzui/launchpad/redirect-201/+merge/136198
<benji> sinzui: I should in the next hour or so; on a call now
<jcsackett> benji: do you have time to review https://code.launchpad.net/~jcsackett/launchpad/blueprints-in-ui-not-specification-2/+merge/136213 after sinzui's?
<benji> jcsackett: sure
<jcsackett> benji: thanks.
<benji> np
<benji> Why did I recieve a notice about a juju-gui mailing list that is (now) disactivated?
<benji> Why am I talking about it in this channel?
<czajkowski> rick_h_: wow you're busy :)
<rick_h_> czajkowski: bwuhahaha, it's why I do deploys. I look so busy when I flood the bug mails
<czajkowski> lol
<sinzui> benji, Do you have time to review https://code.launchpad.net/~sinzui/launchpad/team-without-admin/+merge/136274
<benji> sinzui: sure; I'll look at it now.
<benji> I know understand why people like inline commenting in code review apps.  After using it for a few months, doing reviews without it is irritating.
* benji changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On-call reviewer: - | Firefighting: - | Critical bugs: ~170
<wgrant> StevenK, sinzui: Using a team CTE gets that person search down from 2100 to 220 on DF
<wgrant> Most of the remaining time is in the email search, as expected, but it's probably not worth eliminating
#launchpad-dev 2012-11-27
<StevenK> wgrant: http://pastebin.ubuntu.com/1390509/
<wgrant> StevenK: searchable_names, maybe? Not sure
<wgrant> That otherwise sounds reasonable
<wgrant> Hmm
<wgrant> Though hmm
<wgrant> This will sometimes end up too long for pg_trgm to cope with, I suspect :(
<StevenK> Oh? pg_trgm has a limit?
<wgrant> Indices in general are sort of limited. Try inserting a very large string and see what happens
<wgrant> It will probably object
<StevenK> INSERT 0 1
<StevenK> That's inserting a row into packageupload with distroseries = 1, pocket = 0, archive = 1 and search_text = 'a'*50000
<wgrant> Huh
<wgrant> Interesting.
<wgrant> ALthough it may optimise because it's all a
<StevenK> I'm not sure about typing out 50000 random characters
<StevenK> :-)
<wgrant> Anyway, I must be off for a few hours
<wgrant> If it works, then great ;)
<wgrant> :)
<StevenK> wgrant: Well, how about we cook up a query to fetch the BPNs for a large source and set search_text to that and see what happens
<adeuring> good morning
* frankban changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On-call reviewer: frankban | Firefighting: - | Critical bugs: ~170
<adeuring> frankban: could you pease review this MP: https://code.launchpad.net/~adeuring/launchpad/lp-view-for-timelneproductseries/+merge/136374 ?
<frankban> adeuring: sure, on it in a minute
<adeuring> frankban: thanks, no hurry (enjoy lunch!)
* rick_h_ changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On-call reviewer: frankban, rick_h | Firefighting: - | Critical bugs: ~170
<rick_h_> adeuring: r=me on the review, I grabbed it frankban
<adeuring> rick_h_: thanks!
<frankban> rick_h_: thanks
<czajkowski> jtv1: there are two of you :) jtv
<rick_h_> abentley: ping, I'm about to get my car but while I pay/head home do you have time to look over my storm-ization of this https://pastebin.canonical.com/79162/ ?
<abentley> rick_h_: certainly:
<rick_h_> abentley: I've still got to get tests going against it, but I wanted to do a side vs side write and wonder if anything jumps out at you on it
<rick_h_> ok, going to be afk for a little bit. Will catch up.
<rick_h_> thanks!
<jcsackett> frankban, rick_h_: can one of you take a look at https://code.launchpad.net/~jcsackett/launchpad/404-not-403-for-private-blueprints/+merge/136442
<frankban> jcsackett: on it
<jtv> czajkowski: setting up  a new system...
<czajkowski> jtv: ahh I see
<czajkowski> jtv: I had a user of lp translations who was unsure how to progress on an issue so asked them to mail lp-users
<jtv> That's going to be a bit broad.  :/
<abentley> jtv: For private projects, we need to ensure there are no translations.  It seems we need to ensure translations are disabled, there are no POTemplates, no ProductSeries has automatic imports.  Am I missing anything?
<jtv> abentley: that almost covers it... ideally you don't want anything on its translations import queue either.
<abentley> jtv: Okay, I'll include that.
<czajkowski> jtv: which part is going to be broad...
<jtv> czajkowski: it's going to cover a lot of people.
<czajkowski> ah I see
<jcsackett> frankban: thanks!
<frankban> jcsackett: my pleasure
<rick_h_> ok, back yay for new tires
<abentley> rick_h_: So, the first thing is that I recommend using lp.blueprints.model.specification.get_specification_filters
<abentley> rick_h_: If for some reason you can't, I still recommend using fti_search rather than specifying it manually.
<rick_h_> abentley: ah ok, I thought I had seen something like that but when I searched a couple of other models found it done that way
<abentley> rick_h_: It's only a month or so old.
<rick_h_> ah, makes sense then
<rick_h_> and yea, get_specification_filters looks good as well. Thanks
<abentley> rick_h_: Also, using ProductSet.getProductPrivacyFilter does not address your bug, because that doesn't ensure the user can view the specification.  Use visible_specification_query
<rick_h_> ok, I was going to ask about that as it seemed a bunch of magic for storm to auto join/etc to get those filters to work
<abentley> rick_h_: Storm doesn't have a particularly clean way of doing composition with left joins, but using left joins was by far the most efficient way, so visible_specification_query returns both a list of tables and a list of clauses.
<rick_h_> abentley: right, so the tables goes into a using() and looks like it's got specifications so should be complete there
<rick_h_> is that prejoin right? It was the same api as used before, but that seems to match what I could find in the api docs for storm
<abentley> rick_h_: AFAIK, Storm does not support prejoins, so I was surprised to see it there.  Have you tested it?
<rick_h_> no, not yet.
<rick_h_> abentley: ah, it's on the api under SQLObjectResultSet :/
<abentley> rick_h_: Okay, so I would expect that to fail.  There are workarounds you can steal, though, if you need to support prejoins.
<rick_h_> abentley: ok yea. I'll look for some more examples then.
<rick_h_> thanks for the quick sanity check and pointers to the helper methods
<abentley> rick_h_: See blueprints.model.specification.HasSpecificationsMixin._preload_specifications_people
<abentley> rick_h_: But see if you need 'em first.  A lot of prejoins are premature optimizations.
<rick_h_> abentley: well right now I"m just trying to port over all the old code to storm and it was there as a prejoin if the argument was passed in
<rick_h_> I can look for uses and go from there
<rick_h_> bah, it's default True so it just might be left out of the arg clause
<rick_h_> man...that preload_specifications_people is fugly
* frankban changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On-call reviewer: rick_h | Firefighting: - | Critical bugs: ~170
<czajkowski> rick_h_: you there?
<rick_h_> czajkowski: yepper
<czajkowski> rick_h_: whats the link to the bugs ye have opened for P bp ?
<rick_h_> translation: links to bugs we're working on around private blueprints?
<czajkowski> yes sorry
<czajkowski> :/
<rick_h_> czajkowski: can you see our kanban board?
<czajkowski> really have to stop shortening words
<rick_h_> single letter variable names are bad :P
<czajkowski> yes
<czajkowski> and yes I can
<czajkowski> so just look there
<rick_h_> yea, that'll be the most up to date.
<czajkowski> grand job
<rick_h_> if it's not there then we're not aware/it snuck in so let us know
<czajkowski> thanks
<rick_h_> abentley: do you recall the use of Compile to dump a resultset query?
<abentley> rick_h_: You should use Store.compile, but you should really configure postgres to emit queries in the log, because that shows the substituted variables.
<rick_h_> yea, you're right
<abentley> rick_h_: log_statement='all' in /etc/postgresql/9.1/main/postgresql.conf will do the trick.
<rick_h_> abentley: rgr, updating now. thanks for the reminder to quick doing it the hard way
<abentley> rick_h_: np.
* rick_h_ changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On-call reviewer: - | Firefighting: - | Critical bugs: ~170
<wgrant> rick_h_: Do you know about the LP_DEBUG_SQL=1 environment variable, and the convert_storm_clause_to_string function?
<sinzui> wgrant, https://code.qastaging.launchpad.net/~sinzui/impressive/lirc-0/+merge/29161/++oops++
<sinzui> wgrant, StevenK, does this relate to bug 62976
<_mup_> Bug #62976: Soyuz should not allow duplicated packages in NEW/UNAPPROVED queue <boobytrap> <lp-soyuz> <oops> <rfwtad> <soyuz-upload> <Launchpad itself:Triaged> < https://launchpad.net/bugs/62976 >
<StevenK> wgrant: http://pastebin.ubuntu.com/1393012/
<cjwatson> ah, well that might make my long-stalled approved branch more sensible ...
<StevenK> cjwatson: That was the plan
<StevenK> If it's searchable_names you're talking about.
<StevenK> wgrant and I came up with a plan yesterday
<StevenK> Not quite as good as The Brilliant Disclosure Plan, but still a good one
<wgrant> cjwatson: Yeah, we had a +queue search timeout bug, so I decided we might as well fix that and yours once and for all
<wgrant> StevenK: https://bugs.launchpad.net/bugs/163997
<_mup_> Bug #163997: Subject of Soyuz binaryful acceptance emails, emails, emails, emails, emails, emails stutter <lp-soyuz> <Launchpad itself:Won't Fix> < https://launchpad.net/bugs/163997 >
<wgrant> sinzui, StevenK https://www.youtube.com/watch?v=H3OJvbnTnE8
<StevenK> wgrant, sinzui: http://sphotos-c.ak.fbcdn.net/hphotos-ak-prn1/16056_378137315607151_2018832644_n.jpg
<cjwatson> StevenK,wgrant: Right.  Every so often I've been looking at that branch and contemplating inserting a feature flag, and then giving up in despair at how horrible it is.
<StevenK> I am looking forward to making IPackageUploadSet.getAll() a shadow of its former self.
<StevenK> wgrant: The query is still baking?
<wgrant> StevenK: Yeah
<wgrant> StevenK: Something like http://paste.ubuntu.com/1393183/
<wgrant> Although we also need to do something with PCJs, I guess
<StevenK> I don't think you need array_agg for the source, there will be at most one source
<wgrant> Oh, there's actually a UNIQUE constraint for that, indeed
<wgrant> I know we only ever have one, and other stuff will break if there's more than one, but thought I might as well cope :)
<wgrant> But there's indeed a constraint
#launchpad-dev 2012-11-28
<StevenK> And yeah, we probably need to pull PCJ into it
<wgrant> StevenK: Of course, you don't have to do all this in SQL, but you'll want to use at least fragments of this to not make several queries per binary
<StevenK> wgrant: The package_name I'm already doing handles source and PCJs, so I might leave that, but yes, looping over the binaries is bad
<wgrant> StevenK: And you really want to be avoiding doing even one query per upload, ideally.
<StevenK> wgrant: I have to do at least one?
<wgrant> StevenK: Why?
<wgrant> You can probably get away with <5 queries per batch
<StevenK> Right
<wgrant> Even for batch sizes larger than 5 :)
 * StevenK stabs Storm
<StevenK> TypeError: Expected int, found <type 'tuple'>: (1,)
<StevenK> 2012-11-28 01:22:14 DEBUG2  [PopulatePackageUploadSearchableNames] Iteration 7 (size 532.2): 2.258 seconds
<StevenK> wgrant: ^
<wgrant> StevenK: Much better :)
<wgrant> Dare I look at the code...
<wgrant> Heh
<StevenK> No, you daren't
<wgrant> I recognise that SQL...
<StevenK> wgrant: I was trying to avoid spending four hours converting it to Storm
<wgrant> Huh, did that actually work? I would expect the 500 updates to take more than 2 seconds
<wgrant> I guess because the DB is local it works
<StevenK> I wasn't sure how to phrase it as one update
<wgrant> There's only one bulk update in the codebase
<wgrant> and it's on line 637
<StevenK> wgrant: I wasn't even clear on how to phrase it as SQL
<StevenK> Oh god, that monstrous 34 line SQL has to go into BulkUpdate, doesn't it?
<wgrant> Well, you don't have to do the string calculation in SQL
<wgrant> It's probably awkward to
<wgrant> So you can end up just passing the final strings into BulkUpdate
<StevenK> Yes
<StevenK> update_columns turns into PackageUpload.searchable_names, values=list of the names from results, where=list of ids from results?
<StevenK> Reading BulkUpdate to figure out how it works has left me more confused.
<wgrant> Sort of. The Values expression is a subquery table, so it will want to be a list of [(id, searchable_names)]
<StevenK> So list(results), then
<StevenK> By happy coinience
<wgrant> The where= will want to match the Values' id with PackageUpload.id, so the rows get matched with each other.
<wgrant> And then updated_columns wants to set PackageUpload.searchable_names to the Values' searchable_names
<StevenK> How do I reference Values'?
<StevenK> Do I need to ClassAlias?
<wgrant> (in this case, the Values virtual table is represented to Storm as the cache_data ClassAlias. I did that since the layout of the table is similar to the real LatestPersonSourcePackageReleaseCache table, but in your case it might be best to just define a class there with two cols
<wgrant> Or you could alias PackageUpload
<wgrant> Since it will have the two columns that you need
<StevenK> wgrant: http://pastebin.ubuntu.com/1393383/
<wgrant> StevenK: Are you deliberately shadowing the Values import there?
<wgrant> Values is a storm expression class
<StevenK> Nope, renamed to PUValues
<wgrant> That's still not a valid variable name
<wgrant> Oh, but I guess it's a classalias
<wgrant> So eeh
<StevenK> UpdateUpload ?
<StevenK> Trying to be short and descriptive, and failing
<StevenK> wgrant: Wait, so updated_columns is a list of the updated names, or a Storm expr?
<wgrant> StevenK: Updated columns is the SET dict.
<wgrant> {some_col_to_set: the_value_expression, another_col_to_set: some_other_value_expression}
<wgrant> In the existing example, the value expressions just directly reference columns from the Values expression.
<wgrant> Which is probably sufficient for your needs too
<wgrant> But they could be any Storm expression
<StevenK> ProgrammingError: syntax error at or near "1"
<StevenK> LINE 1: ...d SET searchable_names="_3".searchable_names FROM 1, mozilla...
<StevenK> Haha
<wgrant> sounds like you didn't actually use Values
<StevenK> I took "The Values expression is a subquery table, so it will want to be a list of [(id, searchable_names)]" to mean values=list(results) where results is the RS from the SQL o' doom
<wgrant> StevenK: It's not quite that simple. Look at how the LPSPRC thing does it
<StevenK> wgrant: My attempted simplification of LPSRPCs thing is not going well
<wgrant> StevenK: Oh?
<StevenK> TypeError: zip argument #1 must support iteration
<StevenK> Which falls out of compile_bulkupdate
<wgrant> StevenK: Diff?
<StevenK> wgrant: http://pastebin.ubuntu.com/1393537/
<wgrant> StevenK: Well, I'm not sure if it's the cause of this error, but your Values expression doesn't seem to contain the ID
<wgrant> Which isn't going to be helpful when you try to match the searchable_names up to their corresponding PackageUploads
<wgrant> Oh
<wgrant> I see why it's crashing
<wgrant> Your list of rows is just a list of values
<wgrant> You dropped a layer of list comprehensions
<StevenK> Yes, see 'attempted simplification'
<wgrant> Well, you simplified the table to be a single column, which isn't supported because it makes no sense whatsoever :)
<wgrant> (unless you want to choose a random row -- there's no other key)
<StevenK> WCPGW
<StevenK> Then I don't need the for data in ... second list comprehsion?
<StevenK> Since I have a list of tuples already?
<wgrant> StevenK: Since it's just an int and a string you might not need the list comps
<wgrant> It was required in the LPSPRC job to cope with the Reference and DateTime fields
<StevenK> It doesn't love that either
<StevenK> I've not used zip very often
<StevenK> In fact, It's clearly not working at all
<wgrant> What's the error now, and what's the code?
<StevenK> Well, zip((PackageUpload.id, PackageUpload.searchable_names), results)
<StevenK> => [(<storm.properties.PropertyColumn object at 0x6cb3600>, (1, u'mozilla-firefox'))]
<StevenK> Which is clearly why dbify_value is choking
<wgrant> dbify_value is for values
<wgrant> Not columns
<wgrant> Oh
<wgrant> I think you might be nested one level too deep now
<StevenK> .. I am?
<StevenK> And here I was thinking I needed another loop
<wgrant> Where did you get that zip() invocation from?
<StevenK> Trying to munge it from LPSRPC
<wgrant> Maybe you're still missing a list comprehension
<wgrant> If you look at LPSPRC, the values list comp iterates over the rows of the resultset. It zips each row with the columns, then dbifys each value.
<wgrant> Whereas you seem to be zipping the entire resultset with the columns.
<StevenK> 0-1@SQL-main-master UPDATE PackageUpload SET searchable_names=cache_data.searchable_names FROM (VALUES (7::integer, E'netapplet-1.0.0.tar.gz'::text), (8, E'cnews'), (9, E'cnews'), (10, E'netapplet-1.0.0.tar.gz'), (11, E'mozilla-firefox'), (12, E'language-pack-de'), (13, E'commercialpackage'), (14, E'pmount'), (15, E'iceweasel')) AS cache_data(id, searchable_names) WHERE PackageUpload.id = cache_data.id
 * StevenK declares victory
<StevenK> wgrant: Can I have DF's database back?
<wgrant> Hmm?
<wgrant> I hold no locks of significance
<StevenK> 2012-11-28 03:52:52 INFO    [PopulatePackageUploadSearchableNames] Blocked on 0:40:49.687301 old xact launchpad_main@launchpad_dogfood/16564 - <IDLE> in transaction.
<wgrant> Ah
<wgrant> That was probably my iharness
<wgrant> fg
 * StevenK waits for PPUSN to ramp up
<StevenK> It's only doing ~ 200 per iteration
<wgrant> Where's the time going?
<StevenK> 2012-11-28 03:57:07 DEBUG2  [PopulatePackageUploadSearchableNames] Iteration 39 (size 334.5): 1.334 seconds
<StevenK> 2012-11-28 03:57:24 DEBUG2  [PopulatePackageUploadSearchableNames] Iteration 43 (size 769.7): 3.574 seconds
<wgrant> Sure, but which queries are being slow?
<wgrant> (LP_DEBUG_SQL will be helpful here)
<StevenK> Argh
<StevenK> LP_DEBUG_SQL=1 is not helpful
<wgrant> Why not?
<StevenK> The queries are massive
<wgrant> Ah, true, there might be a bit of scrolling :)
<StevenK> A bit?
<StevenK> 2012-11-28 04:06:19 DEBUG2  [PopulatePackageUploadSearchableNames] Iteration 8 (size 1019.7): 2.791 seconds
<StevenK> Seems the slowest bit is the UPDATE
<StevenK> Or building it
<StevenK> wgrant: Perhaps we're being slowed down by tgrm
<wgrant> Ah, that's likely indeed
<wgrant> The strings are large
<StevenK> However, we're hovering around 850 per iteration
<wgrant> As long as we have a reasonable number of queries (ie. <10) and the UPDATE is the slowest bit, we are probably doing well
<StevenK> w
<StevenK> Bleh
<StevenK> 4 queries per batch
<StevenK> done is doing a SELECT 1 FROM (...), then __call__ is getting a list of ids, calculating the data, and then running the bulk update
<wgrant> Sounds good :)
<StevenK> wgrant: So should I put up the DB patch?
<wgrant> Might as well
<StevenK> wgrant: Do you want to review the DB patch, or wait for stub?
<wgrant> I imagine he'll be around soon
<StevenK> wgrant: I can't rip out most of IPackageUploadSet.getAll() due to the API :-(
<wgrant> StevenK: What do you mean?
<StevenK> wgrant: version and exact_match are exported via IDistroSeries.getPackageUploads()
<wgrant> Oh my god
<wgrant> I'd forgotten what getAll looked like
<StevenK> Heh heh
<wgrant> StevenK: So, exact_match=True is fairly easily doable
<wgrant> You just need to match LIKE 'term %' OR LIKE '% term %' OR LIKE '% term'
<wgrant> As for version, um.
<wgrant> We could check that Ubuntu doesn't use it and then entirely unexport it
<wgrant> Or we could denorm that too
<wgrant> I'd suggest the former
<StevenK> I was wondering about that
<StevenK> Wondering I care enough to grep appserver logs to check for getPackageUploads.*version=
<StevenK> We could be naughty, make version a no-op and see if anyone notices :-P
<wgrant> The PPR is probably running around now, so it's not a good time to be grepping
<wgrant> But I guess it might be worth doing
<wgrant> grep a week of logs or so, I guess
<wgrant> Not sure if -security use it
<wgrant> StevenK: So it turns out all the vocabs and person searching in general are sort of completely terrible
<wgrant> My performance fix comes out to -150 or so
<StevenK> Haha
<wgrant> Are you still abusing DF?
<StevenK> Yeah
<StevenK> Well, garbo-hourly is on its second run
<StevenK> wgrant: zcat -f *access*112[0-7]* | grep -E 'getPackageUploads.*version=' turned up nothing for all four appservers.
<wgrant> StevenK: Most of the logs are bzip2ed, IIRC
<StevenK> They are not
<wgrant> Also, make sure they're actually syncd
<wgrant> Oh, maybe that finally got fixed
<wgrant> That would be handy
<StevenK> All but gac were gzip'd until I got annoyed and filed an RT
<wgrant> so, remove it, I suppose :)
<StevenK> wgrant: garbo-hourly is done again if you want me to stop touching DF
<StevenK> It's roughly half done with two runs
<wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/bug-1075767/+merge/136572
<StevenK> w
<StevenK> Bleh
<StevenK> wgrant: r=me
<wgrant> StevenK: Thanks
<adeuring> good morning
<StevenK> stub: https://code.launchpad.net/~stevenk/launchpad/db-searchablenames-for-pu/+merge/136568 would love a review
<stub> StevenK: How are you going to be searching that btw?
<stub> Do you expect searches to return few results, or are we going to have lots of results and we need to order them by similarity or similar?
<StevenK> stub: contains_string
<StevenK> And like
<StevenK> stub: And both, sadly
<StevenK> Depends on if it's via a view, or the API
<stub> So if a search matches 100,000 results we need to return them all?
<StevenK> I doubt it's 100,000 results
<StevenK> Perhaps 400-500
<StevenK> Usually nowhere near that many
<stub> Ok, that sounds sanish
<stub> StevenK: If we needed to LIMIT, ordering my similarity, we might need a GiST index instead of GIN or perhaps an alternative data structure
<stub> c/my/by
<StevenK> stub: Can I get a +1 on that MP?
<stub> StevenK: Done, but index creation needs to be in a separate patch to apply live (took > 5s on staging)
<StevenK> Ah, right
<StevenK> I'll split it out tonight and land just ALTER TABLE tomorrow
<cjwatson> wgrant,StevenK: We use the version parameter in the event that somebody uses the 'queue {accept,reject} PACKAGE/VERSION' syntax, which is rather handy
<cjwatson> I'm sure we could do it otherwise by getting all the uploads for a given version and filtering manually, but I'm not sure that's an improvement (especially in the case of accepting from the rejected queue)
* abentley changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On-call reviewer: abentley | Firefighting: - | Critical bugs: ~170
<cgoldberg> )
<rick_h_> party
<czajkowski> rick_h_: you bringin the cake!
<rick_h_> czajkowski: sure, my wife makes a mean homemade cake, or maybe pie
<czajkowski> mmmm cake
<rick_h_> abentley: a review for you if you get a chance please https://code.launchpad.net/~rharding/launchpad/nonpublic_1052659/+merge/136479
<abentley> rick_h_: Sure.
<abentley> rick_h_: visible_specification_query already removes inactive products.  Line 1299.
<rick_h_> looking
<rick_h_> abentley: ah ok
<abentley> rick_h_: Could you explain why moving _cache_people is helpful?  I can see why making it static is good, but I don't see why you want to move it to SpecificationSet.
<rick_h_> abentley: because it's used in two places
<rick_h_> it was orignally inside of the method  _preload_specifications_people
<rick_h_> so for me to access it outside of that context I needed to move it to its own method
<rick_h_> I can't just use _preload_specification_people since I need to do things like limit and order the original result set
<abentley> rick_h_: You need to limit and order the original, not the decorated one?
<rick_h_> abentley: right
<abentley> rick_h_: Why?
<rick_h_> so if I'm understanding how the iter hook stuff works it first grabs the results of the original query, then generates a list of people ids, then does a query for all those people, and attaches them to the original result set
<abentley> rick_h_: Sorry, I've got to go now.  I'll ping you when I'm back.
<rick_h_> if you only want 10 specs (the home page stuff only lists 5 or so) then you want to get that result set limited first, then run it through the person fetcher method
<rick_h_> abentley: rgr, np
<rick_h_> thanks
<czajkowski> rick_h_: abentley is deryck about today ?
<rick_h_> czajkowski: no, he's sick today
<czajkowski> :(
<czajkowski> rick_h_: any idea if/when documentation will be out to explain the private bugs, properetary stuff, as am getting a lot of users very confused atm
<rick_h_> czajkowski: not at the moment. deryck was going to pull together some notes and go over the material already out there
<rick_h_> but with a friday deadline we've been cranking on the bugs vs docs so far
<czajkowski> nods
<rick_h_> czajkowski: so the *if* is we will have some docs. Then *when* might not be until next week but don't quote me on it. I guess call it 'in progress'
<czajkowski> no I understand
<czajkowski> just trying to point peopleat stuff to explain it
<rick_h_> if you can do us a favor though, keep tabs on the questions/issues that come up
<czajkowski> np
<rick_h_> it'll help direct us to make sure we hit points in the FAQish section or what not
<rick_h_> and potential future bugs to make things easier/less crazy
<czajkowski> the terminalogy seems to be the confusing
<rick_h_> heh, yea...
<abentley> rick_h_: Back.
<rick_h_> abentley: awesome
<abentley> rick_h_: The iter_hook_stuff doesn't fire until you actually start to access the results.  So you don't need the resultset in final form before you create the DecoratedResultSet.
<rick_h_> abentley: ok right, but I moved it from something defined inside the  _preload_specifications_people method
<abentley> rick_h_: The other call sites for _preload_specifications_people do ordering and sorting afterward, so if that was a performance issue, we'd know already.
<rick_h_> to it's own method on the HasSpecificationsMixin
<rick_h_> abentley: ok, gotcha, but the reason to move it was more to keep the _preload_specifications_people as is.
<abentley> rick_h_: Right.  I said why, you said because I need the resultset in its final form, I'm saying no you don't.
<rick_h_> I guess what you're saying is that I could use that and append my order and limit onto the returned DecoratedResultSet?
<abentley> rick_h_: Why can't you use _preload_specifications_people as is?
<rick_h_> abentley: ok, I gotcha. Because I didn't realize it could be used as it was. So let me update that then.
<abentley> rick_h_: Thanks.
<abentley> rick_h_: The other call sites are Product.specifications and Distribution.specifications, if you want to see how they do it.
<rick_h_> abentley: will do
<abentley> rick_h_: Most implementations of IHasSpecification.specifications are not well-tested, so I would add tests for the old implementation, make sure they passed, then do the stormification.
<abentley> rick_h_: I'd like to see something similar for SpecificationSet.  You can just copy the tests from test_product.TestSpecifications, but please make sure they pass against both implementations.
<czajkowski> abentley: can you see -ops just running out the door and there is an issue
<czajkowski> please
<jcsackett> abentley: do you have time to review https://code.launchpad.net/~jcsackett/launchpad/private-blueprint-dependencies/+merge/136756 ?
<abentley> jcsackett: sure.
<jcsackett> thanks.
<abentley> jcsackett: I don't think this is a view issue.  I think we want to forbid this kind of change at the model level using a storm validator.  We can catch the exception at the view to set the field error.
<jcsackett> abentley: if we do that, we end up having to set the error and abort outside of the validation, right? that seems awkward. i can see an argument for having the model forbid it as well, but i think validate's the right place for this check.
<abentley> jcsackett: It's not awkward.  I do it all the time.
<jcsackett> abentley: can you point me to an example? i've only dealt with form handling within validate.
<abentley> jcsackett: lp.code.browser.BranchEditFormView.change_action
<abentley> jcsackett: The bit where it handles BranchTargetError
<jcsackett> abentley: huh, that's it?
<jcsackett> ok, i'll make that change.
<abentley> jcsackett: Yes.
<jcsackett> i'll ping you when it's updated.
<abentley> jcsackett: Cool.
<abentley> jcsackett: The one thing you could run into is I believe self.next_url must not be set for this to work.
<jcsackett> abentley: changes have been made; worked out quite nicely, thanks.
<abentley> jcsackett: Thanks.  Looking.
<abentley> jcsackett: Nice.  r=me.
<jcsackett> abentley: thanks.
<wgrant> abentley, jcsackett: It's a bit strange to use a validator there, due to artifact-specific grants. You can't assume that just because two blueprints are private the permissions are equal.
<jcsackett> wgrant: not sure i follow you; we're not testing for equality between two blueprints. just ensuring that if the root one is public, it doesn't have a private dependency.
<abentley> wgrant: That's a good point.
<abentley> jcsackett: There is also the case where both are proprietary, but one is visible to the user and the other is not.
<jcsackett> aaah.
<jcsackett> abentley: i think though that's a situation handled by the follow up work. the strict "no dependency" rule only really applies in the public/non-public situation.
<abentley> jcsackett: But do we want that rule if it won't prevent oopses?
<jcsackett> abentley: we do, because it makes the excised dependency graph still valid information.
<abentley> jcsackett: Could you explain what you mean by that?
<wgrant> It's perfectly valid for a public blueprint to depend on a private one
<jcsackett> abentley: i was writing out what i meant, and i may be changing my mind. gimme a sec to think through this. :-P
<wgrant> Unlike branch stacking, a blueprint is useful without its deps
<jcsackett> perhaps we just cease to show the dependency graph if you don't have permission to see the various parts of it.
<jcsackett> b/c i'm not comfortable for the possibility of information leakage that anything else implies.
<abentley> jcsackett: Cease to show the graph completely?
<wgrant> As long as you don't traverse through invisible blueprints there's no possible leakage
<jcsackett> abentley: that's becoming my preference, yes.
<wgrant> But your solution is easiest
<wgrant> Hm
<wgrant> On the other hand, hiding the graph entirely reveals that there are private things :)
<jcsackett> wgrant: yes, and time is a factor, since we're operating under a hard deadline.
<abentley> jcsackett: I am okay with that.  Of course, I'm not a stakeholder, so I recommend getting deryck's buy-in.
<wgrant> You're well aware of how little respect I have for that deadline :)
<jcsackett> wgrant: don'
<jcsackett> wgrant: don't i know it. :-P
<jcsackett> wgrant: but i don't think it's budging.
<wgrant> Yeah
<jcsackett> abentley: i'm going to kill the ec2 land run for the branch, and we'll bring it up at standup tomorrow, i guess?
<abentley> jcsackett: Sounds good.
<jcsackett> done.
* abentley changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On-call reviewer: - | Firefighting: - | Critical bugs: ~170
<jcsackett> wgrant: how bad is doing check_permission over a sequence of items? I seem to recall that being a performance horror.
<wgrant> jcsackett: For blueprints? It'll issue a query per object, unless you've precached a positive or negative assertion for each
<jcsackett> so, gets bad with scale.
<jcsackett> the easiest way to deal with not navigating the bad nodes in dep graphs is to just filter with check_permission. as a proof of concept it works, but i think the performance question could become bad.
<wgrant> jcsackett: Well, I'd do the filter in the initial DB query
<wgrant> IIRC it's a WITH RECURSIVE, to which you could just add the usual access filter
<jcsackett> 're referring to modifying those attrs?
<jcsackett> ...something strange just happened...
<jcsackett> trying again... so the graph stuff doesn't do any querying directly. it uses the spec's "dependencies" and "blocked_dep" attrs. but i guess those might be what you're referring to.
<wgrant> Right, Specification.all_deps and Specification.all_blocked use a WITH RECURSIVE query to determine the dependency tree in two queries.
<jcsackett> wgrant: ok. i'll take a look at modifying those tomorrow morning. thanks.
<wgrant> It should be easy enough
<rick_h_> wgrant: I'd not played with the db setting you'd mentioned there
<rick_h_> I mainly just got stuck because a bad db query will come out as some sort of .pt 'location error'
<rick_h_> so got really confused for a bit until I pdb'd through it and got at the real error
<wgrant> rick_h_: Setting LP_DEBUG_SQL makes the appserver and bin/harness etc. spew DB queries (including times) to stderr, which can be handy
<rick_h_> very cool
<wgrant> LocationErrors are usually an issue with the Python expression, not the actual SQL
<wgrant> Since TAL has a bad habit of suppressing AttributeErrors
<wgrant> Who needs tracebacks anyway :)
<rick_h_> well, it was that I did .using(tables) vs .using(*tables( so the sql generated was comma seperated through the join section :/
<wgrant> Hm, I wouldn't expect that to be a LocationError, but who knows
<rick_h_> and that came out to the app/traceback  level as a locationerror
<rick_h_> heh, well I do have a way of breaking things in fun ways :)
<wgrant> Heh
<rick_h_> I did want to say that I feel like an LP dev now. Back to back deploys in my name wooo! :)
#launchpad-dev 2012-11-29
<StevenK> wgrant: r16321 is only a testfix or a testfix and a rollback?
<CyberJacob> Hi
<wgrant> StevenK: Just a testfix
<wgrant> sinzui: Hm, this same MP diff technique could/should probably be applied to the front page blog feed relatively easily, I guess
<StevenK> reference = u'unique-from-factory-py-line3339-102754 binarypackage-102763'
<StevenK> actual    = u'binarypackage-102763'
<StevenK> Success
 * StevenK peers at the garbo query of doom
<CyberJacob> Hi Guys
<CyberJacob> I'm trying to look up a user, but I keep getting a ComponentLookupError
<CyberJacob> any ideas?
<StevenK> What's the traceback?
<CyberJacob> http://pastebin.ubuntu.com/1395890/
<wgrant> CyberJacob: Where are you running that?
<wgrant> Looks like bin/py, when you might want bin/harness
<wgrant> bin/py just starts a Python interpreter, without setting up any of the infrastructure
<CyberJacob> oh, that might be the problem then
<CyberJacob> is there a binary for the infrastructure?
<StevenK> bin/harness
<StevenK> Or 'make harness'
<StevenK> wgrant: I've added the joins for BPB, SPR and SPN, but I can't work out how to get it into the output of the SELECT array_agg
<wgrant> Howso?
<StevenK> wgrant: Everywhere I try and put sourcepackagename.name, postgres complains
<wgrant> Complains?
<StevenK> HINT:  No aggregate function matches the given name and argument types. Perhaps you misplaced ORDER BY; ORDER BY must appear after all regular arguments of the aggregate.
<wgrant> What's your query?
<CyberJacob> StephenK: works perfectly with bin/harness, thanks
<StevenK> It's a V, damn it!
<StevenK> wgrant: http://pastebin.ubuntu.com/1395899/ is what I'm testing on DF with
<StevenK> With some variants
<wgrant> StevenK: What does that mean?
<wgrant> You're trying to create an array of rows?
<StevenK> wgrant: I'm trying to get 4749089 | debhelper hello-debhelper-df
<wgrant> StevenK: Right, but you seem to be trying to create an array of [(spn, bpn)] there, which is a bit odd
<StevenK> wgrant: I've been trying various things, that was the latest attempt
<wgrant> StevenK: I'd just do a separate join to get the SPN
<wgrant> Bring it out in a separate array
<wgrant> As we do with the source name
<StevenK> Right
<StevenK> That works nicely
<bigjools> any regex experts want to do a maas pre-imp with me?
<wgrant> A pre-imp for a regex... this does not bode well :)
<wgrant> I'm about to have lunch but could talk after
<bigjools> well it's more about how to deal with some postinst processing on a config
<StevenK> wgrant: Right, that's the garbo and IPackageUpload.addBuild working with adding the SPR
<StevenK> bigjools: In a word, "don't"
<bigjools> heh
<bigjools> I *know(
<StevenK> If you can possibly get away with a .d directory, do that
<bigjools> but we are stuck with it
<wgrant> bigjools: What are you trying to do?
<StevenK> If it isn't your config file, it's against policy, too ...
<bigjools> fixing someone else's crap^Wwork
<bigjools> a dpkg reconfigure is supposed to take an input value and insert it in various configs
<StevenK> wgrant: My vague plan was to have the garbo job done and out of the tree by Friday, too ...
<bigjools> this particular config may or may not have a commented line that we need to update
<bigjools> it may also have a commented and a non commented line
<wgrant> StevenK: I generally suggest getting all the work done before landing any of it (other than an obviously safe DB patch)
<wgrant> It's far easier to fix the garbo job than it is to fix several million rows
<CyberJacob> Is there a way of starting the Salesforce (voucher) server locally?
<CyberJacob> or is that a non-GPL part of launchpad?
<wgrant> CyberJacob: It's not part of Launchpad at all. I've never seen the source, and I'm not sure how much of it we actually have (it's just a thin wrapper around SalesForce itself, AIUI)
<CyberJacob> what about forcing `make run` to use the test proxy transport
<CyberJacob> that should get it talking properly
<wgrant> What are you trying to do?
<wgrant> If you just want to extend the complimentary commercial subscription, we use an API script like http://bazaar.launchpad.net/~canonical-launchpad-branches/lp-dev-utils/trunk/view/head:/extend_subscription.py
<wgrant> There's little point going through the voucher system unless you need to
<CyberJacob> that's basically what I was looking for
<CyberJacob> (I didn't realise SalesForce was a full piece of software, just looked like another module to me)
<wgrant> The SalesForce referred to here is the service provided by salesforce.com
<CyberJacob> which explains why there's just an RPCXML connecter in LaunchPad
<wgrant> Yep
<CyberJacob> wgrant: I'm getting a 404 error with that script...
<CyberJacob> Looks like it's trying to load NotFound: Object: <lp.services.webapp.publisher.RootObject object at 0x41d7d50>, name: u'devel'
<CyberJacob> any ideas?
<StevenK> wgrant: So the garbo job and the population now sets source name as well, and I've changed IPackageUploadSet.getAll() to use searchable_names in a different branch. That's all of it, isn't it?
<wgrant> CyberJacob: Have you told it to connect to the right URL?
<wgrant> StevenK: How are you handling version?
<CyberJacob> wgrant: I'm using ./extend_subscription.py -s "https://launchpad.dev/" -d 90 caveman
<wgrant> CyberJacob: api.launchpad.dev, not just launchpad.dev
<StevenK> wgrant: So far, I'm not.
<wgrant> StevenK: You'd best
<wgrant> CyberJacob: (also, for internal projects we normally just force it to extend for 10 years or so)
<CyberJacob> wgrant: ah, perfect!
<StevenK> 40-1 turns into ADD COLUMN searchable_versions ?
<wgrant> StevenK: It depends on whether there's a performance issue or not
<StevenK> wgrant: With the existing queries?
<wgrant> StevenK: With name denormed, and version not
<StevenK> Yeah
<wgrant> Optimise things that need optimising
<wgrant> Don't optimise things that don't :)
<StevenK> The garbo job on DF has 740k rows to go, but will need to start again
<StevenK> Anyway, I'll add the trigram, and test a few queries
<StevenK> Come on, DF, it's an index, not solving pi to 1 trillion places.
<wgrant> It'll take a while to build
<wgrant> They're longish string fields, and trigram indices aren't exactly efficient
<StevenK> It's been at it for 5 minutes already
<wgrant> And there's millions of rows
<wgrant> This will be one of our largest non-FTI indices
<StevenK> CREATE INDEX
<StevenK> Time: 621691.915 ms
<StevenK> wgrant: How can I check how large it is?
<wgrant> It's 656MB
<wgrant> Which places it just between the bug and bugtaskflat FTI indices, IIRC
<wgrant> The only indices likely to be bigger than those are message FTI, and branchrevision + *downloadcount indices
<wgrant> :)
<StevenK> SELECT max(char_length(searchable_names)) FROM packageupload; => 11908
<wgrant> Ouch
<wgrant> I wonder if that's langpacks on a delayed copy
<StevenK> I'm just trying to figure out how to check that
<StevenK> Not quite
<StevenK> ... libghc6-vte-dev libghc6-vte-doc libghc6-vty-dev libghc6-vty-doc libghc6-vty-prof ...
<wgrant> Ah
 * StevenK looks for a good victim
<StevenK> Actually, that Haskell one looks good
<StevenK> Just searching for a version is pathetically slow
<StevenK> wgrant: The way this query is going, it is clearly searching all BPR and SPR rows
<wgrant> Well, what does EXPLAIN say?
<StevenK> It hasn't finished yet
<wgrant> without ANALYZE
<StevenK>                      ->  Seq Scan on sourcepackagerelease  (cost=0.00..321829.18 rows=1627518 width=25)
<StevenK>                                  ->  Seq Scan on packageuploadbuild  (cost=0.00..49832.41 rows=3137541 width=8)
<wgrant> That's less than ideal
<StevenK>                      ->  Seq Scan on binarypackagerelease  (cost=0.00..1618381.64 rows=11488564 width=28)
<lifeless> nice
<StevenK> Let's look over what, 17 million rows?
<bigjools> sigh, how can linking a branch to a blueprint time out
<wgrant> bigjools: When the branch row is locked by the scanner, usually
<bigjools> it's an old landed branch
<wgrant> What does the OOPS show?
<lifeless> ur face?
<lifeless> </troll>
<bigjools> haha
<bigjools> https://oops.canonical.com/oops/?oopsid=OOPS-cf507af414fb03b447b2658c8c964a72
<StevenK> lifeless: Fail.
<StevenK> lifeless: This is you, you never close that tag
<lifeless> StevenK: closing it is itself a troll
<lifeless> StevenK: thus YHBT
<wgrant> bigjools: Ah, you probably typoed the branch name.
<wgrant> Yes
<wgrant> You gave the URL, not the name, so it did a search
<bigjools> it has SPACES on the end
<wgrant> and the search algorithm is insane
<bigjools> that is all
<bigjools> WTAF
<wgrant> Bug #793830
<_mup_> Bug #793830: Branch:+register-merge / Specification:+linkbranch time out due to substring matching many tables <critical-analysis> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/793830 >
<bigjools> you gotta love the optimism in the code around user input
<wgrant> Well
<wgrant> It'd be all good if the search mechanism wasn't completely stupid
<StevenK> bigjools: I'm in the middle of a fixing a critical around IPackageUploadSet.getAll(). That code is just as good
<bigjools> trailing spaces being very stupid
<StevenK> Let's JOIN against SPR, SPN, BPB, PCJ and LFA and do a substring match
<StevenK> Oh, I'm forgetting BPR and BPN
<StevenK> wgrant: So, I guess denorming version is a go?
<StevenK> wgrant: Oh, and that trigram index isn't even fully populated
<wgrant> StevenK: And you forgot PUS, PUB and PUC
<wgrant> But anyway
<wgrant> StevenK: Right, denorming version would make sense here, but we probably don't care about substring matches on it
<wgrant> We support it today, but that's fairly insane and probably not useful
<StevenK> I have a garbo query, too
<StevenK> wgrant: http://pastebin.ubuntu.com/1396146/
<wgrant> Right, but there's not much reason to use a string here
<wgrant> We don't care about substring matching
<StevenK> What do you suggest?
<wgrant> A plain array of strings with a GIN index
<wgrant> Much faster
<StevenK> Hmmm, don't know how to craft that one
<wgrant> Just don't turn the array into a string
<StevenK>  2873221 | {NULL,1:4}
<StevenK> Given we don't have mixed uploads, I don't think an array is useful
<wgrant> Binaries can have different versions
<StevenK> For the same upload, for the same source, from the same BPB?
<wgrant> StevenK: Yes
<wgrant> It's not hugely common, but some fairly significant packages like gcc do it
<StevenK>  3857156 | {NULL,4.4.6-15ubuntu1,1:4.4.6-15ubuntu1}
<StevenK> Well, damn
<StevenK> wgrant: ADD COLUMN searchable_versions TEXT[]; ?
<wgrant> StevenK: It's technically a debversion, but that's probably unimportant and possibly detrimental here
<StevenK> Right
<StevenK> I'm not sure I care enough to test both
<wgrant> I should really try some multi-column GIN btf indices at some point.
<wgrant> Hm
<wgrant> It actually works pretty well
<wgrant> Potentially useful for translations, too
 * wgrant plots
<StevenK> wgrant: How can I declare searchable_versions as an array in the model code so I can avoid SQL('') everywhere in the garbo job?
<wgrant> StevenK: A List, probably
<stub> StevenK: Today's search column is an array, yesterday's wasn't. Is that a foot gun for one or t'other?
<StevenK> Neither
<wgrant> "TODO: Test IE compatibility. StuartBishop 20041118"
<stub> haha
<stub> Does it work in Ireland now?
<wgrant> Heh
<czajkowski> morning folks
<adeuring> good morning
<StevenK> stub: So, no, it isn't a fail. searchable_names is a string that will be substring matched via a GIN index. searchable_versions is an array that will not be substring matched, so it made sense to store it as an array
<stub> StevenK: How does a package upload end up with multiple versions btw?
<StevenK> stub: Some packages do it, like gcc.
<StevenK> It's rare, but does happen
<deryck> Morning, all.
<rick_h_> morning
<cjwatson> stub: Binary package generation is controlled by the source package's debian/rules; by default it produces binaries with versions matching the source, but it's occasionally useful to use the -v option to dpkg-gencontrol (or its wrappers) to change that
<stub> It sounds very quantum.
<cjwatson> Entirely deterministic :)
<czajkowski> deryck: hope you're feeling better
<deryck> czajkowski, thanks! I am. Little worn down by being sick yesterday, but overall, much better.
<czajkowski> :(
<czajkowski> pleanty of tea!
<czajkowski> hot tea now, none of that ice cold stuff
<stub> Are Debian version numbers Turing complete?
<cjwatson> No. :-P
<cjwatson> Well, except that they can be arbitrarily long so I suppose you could encode any program in them.
<cjwatson> Same as you could encode any program in YOUR MUM.
 * cjwatson runs
<stub> I don't think my mum is Turing complete
* jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On-call reviewer: jcsackett | Firefighting: - | Critical bugs: ~170
<rick_h_> abentley: pushed updated changes with tests to my MP. Did you want to go over it or should I ask jcsackett? https://code.launchpad.net/~rharding/launchpad/nonpublic_1052659/+merge/136479
<abentley> rick_h_: I'll look.
<rick_h_> abentley: ty much
<cjwatson> http://www.rogerharford.com/content/img/blog-images/database-design-breakdown.png
<abentley> rick_h_: Looks great.  Thanks for making all those changes.
<abentley> rick_h_: r=me.
<rick_h_> abentley: thanks for the review and the push to 'do the right thing' by adding in the tests
<rick_h_> sometimes need a nudge :)
<abentley> rick_h_: np.
<czajkowski> rvba: is jtv on today ?
<abentley> rick_h_: What's the status of your rollback for _getTranslatables ?
<rick_h_> abentley: done and landed
<rick_h_> deployed as part of yesterday
<abentley> rick_h_: Shouldn't test_getTranslatables_filters_private_products no longer exist?  I still see that in stable and devel.
<sinzui> jcsackett, do you have time to review https://code.launchpad.net/~sinzui/launchpad/fast-contact-via-web/+merge/136986
<rick_h_> abentley: looking right now.
<jcsackett> sinzui: yes.
<rick_h_> abentley: that looks to have come from somewhere else. All I did was revert my branch in rev 16313
<rick_h_> which was a rollback of rev 16295
<rick_h_> neither of which hit test_product
<abentley> rick_h_: Okay.  It looks like abel did something similar to what you did, and I thought it was the same thing.
<rvba> czajkowski: he was this morning, but he's gone now.
<czajkowski> rvba: cheers
<adeuring> jcsackett: could you please review this MP: https://code.launchpad.net/~adeuring/launchpad/bug-1079116/+merge/136991 ?
<jcsackett> adeuring: right after i'm done with sinzui's.
<adeuring> jcsackett: thanks
<jcsackett> sinzui: r=me. adeuring: looking at yours now.
<sinzui> thank you jcsackett
<jcsackett> adeuring: r=me. thanks for addressing the possible performance point in the proposal. i agree with your logic there.
<adeuring> jcsackett: thanks!
<rick_h_> abentley: so in looking, the branch sharing policy method alone has a check for if branch_sharing_policy is embargoed_or_proprietary
<rick_h_>        # If the branch sharing policy is EMBARGOED_OR_PROPRIETARY, then we
<rick_h_>         # do not allow any other policies.
<rick_h_> do you know what branches might be different than bugs/blueprints for in this way?
<abentley> rick_h_: So you're saying once the policy has been set to embargoed_or_proprietary, it can never be changed?
<rick_h_> abentley: that's what the current code/comment seem to be doing
<abentley> rick_h_: Weird.
<rick_h_> but because we never allowed the option for embaroed_or_proprietary to be displayed it was never chosen
<rick_h_> so trying to think through a use case for this/issue
<abentley> rick_h_: This is in the browser code?
<rick_h_> no, sharingservice getBranchSharingPolicies
<abentley> rick_h_: I don't know why that's there, and I had discussions with Curtis about changing policies and he never mentioned it.
<rick_h_> abentley: ok, tracking down with our annotate suggestions
<rick_h_> if I don't see any reason thinking of removing that and seeing what tests blow up
<abentley> rick_h_: It's from ian.booth@canonical.com-20120927011545-3l2bzqe7qv5q7py9
<rick_h_> abentley: yea, looking
<abentley> rick_h_: Here's the merge proposal: https://code.launchpad.net/~wallyworld/launchpad/embargoed-sharing-policy-ui-1055617/+merge/126585
<abentley> rick_h_: "My primary motivation for requesting this is that it would be most unfortunate if someone accidentally changed the branch privacy policy and was not aware that you had to set it via API and then was not able to change it back to the proper value."
<rick_h_> ah ok, so this was part of keeping embargoed from the list
<abentley> rick_h_: (from the bug).
<rick_h_> abentley: right
<rick_h_> so they went the route of preventing one from accidentally changing it instead of adding the option in
<abentley> rick_h_: But you were already planning to add the option in, so I say drop the restriction.
<rick_h_> abentley: right, sounds like a plan thanks
<sinzui> jcsackett, Do you have time to review https://code.launchpad.net/~sinzui/launchpad/librarian-merge-proposal/+merge/137039
<jcsackett> sinzui: yes.
<sinzui> ^ There is a mess in there where I fought import fascist
<jcsackett> sinzui: line 20 of your diff. should that be "return self.preview_diff_text" or is that supposed to be a function call?
<rick_h_> jcsackett: for your queue if you have time before EOD please https://code.launchpad.net/~rharding/launchpad/limit_sharing_infotype_1083761/+merge/137038
<sinzui> jcsackett, I think I need a comment there to explain that the diff is being precached by the call
 * sinzui wrote it yesterday and boggled at the line with fresh eyes
<jcsackett> sinzui: dig. a comment would be appreciated.
<jcsackett> i sort of feared it might be a "do something by side effect" case.
<jcsackett> sinzui: 87 looks like a typo. view = view = create_initialized_view
<sinzui> oh yes.
<sinzui> my apologies. I did not read the diff properly before asking you to review it
<jcsackett> sinzui: all good.
<jcsackett> otherwise, r=me.
<sinzui> jcsackett, thank you.
<sinzui> abentley, I am was looking at oops reports and see that you dominate all the oopses for bug 1074385. I don't see it affect anyone else. Any idea why you are special?
<_mup_> Bug #1074385: Timeout from getMergeProposals <api> <branches> <privacy> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1074385 >
<abentley> sinzui: No one else uses it?
<sinzui> oh, you might be right.
<sinzui> I should write a script and verify it happens to me too. In any case, I think the queries need optimisation
<abentley> sinzui: No need to write a script, just use bzr lp-find-proposal.
<sinzui> noted, thanks
<rick_h_> jcsackett: going to be afk for a few. Let me know if there's any questions and such. Will check back in.
<abentley> sinzui: Ideally, we'd change BranchMergeProposal.merged_revno to BranchMergeProposal.merged_revevision_id.
<sinzui> ah, we are joining though BranchRevision. We need to rethink that table or purge the duplication in it Lp's branches are  os 75%of the table
<abentley> sinzui: Yes, it's long been an intention of the Code team to purge that table.  The last plan involved using bzr-history-db, IIRC.
<sinzui> wgrant is on leave, be he did some analysis about removing the duplication.
<james_w> jcsackett, would you take a look at https://code.launchpad.net/~james-w/python-oops-tools/recent-oopses/+merge/137055 please?
<jcsackett> james_w: certainly, but it will be a moment while i finish another review.
<james_w> jcsackett, no problem
<jcsackett> james_w: looks good. r=me.
<james_w> jcsackett, thanks,Â do you know if you can land it?
<rick_h_> thanks for the review jcsackett
<jcsackett> james_w: i can tomorrow, i'm running low on time for it now. is that ok?
<james_w> jcsackett, sure
<jcsackett> cool. i'll make a note of it.
<james_w> jcsackett, oh, seems there is tarmac, so it will land itself when I fix that test failure
<james_w> thanks though
<jcsackett> james_w: ah, excellent.
<lifeless> james_w: cool
<james_w> hi lifeless
<lifeless> hi :)
* jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On-call reviewer: - | Firefighting: - | Critical bugs: ~170
#launchpad-dev 2012-11-30
<SpamapS> So, before I run off experimenting with things.. say I want to use the django-openid-auth thing just for openid, but then I want to check for membership in a private team by doing LP calls on login. Has somebody already done this?
<mwhudson> SpamapS: pretty sure that's doable
<mwhudson> SpamapS: you'll need to get your RP specifically whitelisted to be able to ask about private teams
<rick_h_> StevenK: you know what's up with build-not?
<StevenK> rick_h_: Huh?
<rick_h_> StevenK: buildbot is in a strange place with the exception stuff?
<rick_h_> ah looks like someone did another force
<rick_h_> I hadn't seen buildbot blow up in that way so was curious if you had any idea what's up.
<rick_h_> both my ec2 lands bounced due to pqm buildbot fail
<StevenK> rick_h_: subunit stream corruption
<StevenK> rick_h_: And yes, I forced
<rick_h_> ok cool, thanks
<adeuring> good morning
<czajkowski> adeuring: ello
<adeuring> hi czajkowski!
<czajkowski> adeuring: is it cold in your neck of the woods today ?
<adeuring> czajkowski: depends on how you define cold ;) 5C and fotunatley no rain
<czajkowski> well no rain is good!
<czajkowski> it's 3C here and cold!
<stub> come on over, the weather is lovely
<czajkowski> stub: are you in AU or Nz?
<stub> Thailand
<czajkowski> oh even further
<stub> no, closer
<czajkowski> stub: ah yes but ye have the weird rain weather over there
<czajkowski> I want to avoid places with rain and snow. clearly moving to UK from ireland I didn't think this one through :)
<stub> monsoon storms are awesome
<czajkowski> http://www.irishpressreleases.ie/content/flooded%20house.6885.jpg  is what happens to irelan when it floods!
<czajkowski> or http://news.bbcimg.co.uk/media/images/61218000/jpg/_61218905_jex_1449008_de27-1.jpg
<stub> When Thailand floods, we cause worldwide harddrive shortages :)
<czajkowski> lol
<stub> Anyone here on maintenance?
<czajkowski> it's ok we still ge tour cider and guinness exported, we have our priorities! :)
<czajkowski> stub: StevenK if you can catch him
<czajkowski> stub: and we're down to one person for next week
<czajkowski> in USA timezone
<stub> who is that?
<czajkowski> stub: sinzui
<deryck> Morning
<rick_h_> morn
<sinzui> stub, I saw your remarks on bug 1050191. I agree with your suggestions. I will make this my next issue to work on
<_mup_> Bug #1050191: allocate-revision-karma.py is too slow to complete <branches> <karma> <lp-code> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1050191 >
<stub> sinzui: ok. The code changes are trivial if I'm right. Test fallout likely worse.
<sinzui> indeed
<stub> moving to garbo would be nice, but that is more than maintenance
<sinzui> stub, I was thinking the same. So you agree that this process looks like a garbo candidate. I will consider it.
<stub> it is perfectly suited to be a garbo job. migrating tests will again be the issue :)
* bac changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On-call reviewer: bac | Firefighting: - | Critical bugs: ~170
<deryck> adeuring, here's the MP I need reviewed, https://code.launchpad.net/~deryck/launchpad/remove-product-info-typo-garbo/+merge/137210
<adeuring> deryck: already looking at it :)
<deryck> adeuring, ah thanks :)
<adeuring> deryck: r=me, one nitpick
<deryck> adeuring, thanks
<deryck> adeuring, good catch, thanks
<sinzui> rick_h_, https://bugs.launchpad.net/launchpad/+bug/1079678 is still a private-project issue because Lp has to prevent the object from being created...other wise Lp's db has a integrity issue. Question cannot be deleted.
<_mup_> Bug #1079678: You're able to create new questions on Launchpad even when questions is not avilable to a project <distributions> <lp-answers> <projects> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1079678 >
<sinzui> I think someone also needs to verify that questions cannot be retargeted to a proprietary project
<rick_h_> sinzui: ah, that makes sense.
<rick_h_> sinzui: hmm, I thought that was checked though. I remember some work around the sprint stuff before UDS around that
<rick_h_> hmm, never mind, mixing up blueprints and questinos
<rick_h_> sinzui: so ok. I'll add a card/ticket around validating the question so that it can't be targetted around a non-public project
<deryck> rick_h_: so I agree with sinzui on that issue.  but retargeting is separate from people URL hacking, which can be done on any kind of project....
<rick_h_> deryck: right, so I'm filing a second bug that we'll address
<rick_h_> and then we can decide if the right fix for the url hack is to catch it at the view when the form is loaded, or as a validation error like this retargetting issue later
<deryck> rick_h_, sinzui -- and I still stand by my thinking that the first issue is nothing special because of privacy.  it's just a bug, and not dangerous for privacy.
<sinzui> rick_h_, maybe the solution is to ensure the vocab for question targets only contains Public information type projects. That would allow the existing validation processes to co unchanged
<deryck> i.e. yes, it creates bad data, but it doesn't risk a leak, so we don't care.  it also creates bad data for public projects.
<rick_h_> sinzui: yea, I've not delt with the questions to know the best implementation for it, but would assume it would be a simple fix in the model perhaps.
<sinzui> deryck, It was agreed that questions don't make sense and they DO contain confidential information that we do not have infrastructure to clean
<sinzui> deryck, That is why we ask commercial enquiries to use an email address instead of Lp answers. We could not delete the confidential information in the question
<deryck> sinzui: I agree they don't make sense.  but I don't see how this bug relates to private projects.  what confidential info are you thinking?
<deryck> sinzui: I'm sorry man, I'm lost.  How does using an email address and being able to url hack to file a question relate?
<sinzui> deryck, more than 100 hardening/transition bugs blocked Lp from implementing safe sharing. This is the same case, Once users discover that Lp is not trustable, you lost and cost someone and NDA
<rick_h_> I suppose there's the use case where someone submits a question on a non-public project via the url hack and it contains sensitive data
<rick_h_> if things ever do go public, and questions are turned on
<rick_h_> it'd be exposed
<sinzui> yes, that was what the issue was about
<rick_h_> sinzui: ok, from the comments it was more czajkowski wanted to make sure users didn't get ignored
<rick_h_> sinzui: so it wasn't 100% clear at first
<sinzui> Canonical has partners with NDA information spewing from their keyboards into projects that are working in secret
<rick_h_> however, if we fix the model so that you can't create/target to a non-public project the potential leak issue goes away
<deryck> but do those people have permission on private projects AND know how to url hack to file a question?
<rick_h_> so I think the *right* bug for us to fix and work on is #1079678 and #1079678 is a more general bad UX bug
<_mup_> Bug #1079678: You're able to create new questions on Launchpad even when questions is not avilable to a project <distributions> <lp-answers> <projects> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1079678 >
<_mup_> Bug #1079678: You're able to create new questions on Launchpad even when questions is not avilable to a project <distributions> <lp-answers> <projects> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1079678 >
<sinzui> Laura's issue one example of a root cause of bad data getting into a project
<deryck> I absolutely do not agree this warrants our precious time left until Monday, but I'm not going to fight about it.  I just don't see any real risk.
<czajkowski> re that bug I logged it not for a privacy issue, but more for the point people create answers on projects and they are not looked at. perhaps the people creating private projects and blueprints arent possibly going to be creating answers on lp
<deryck> That's my point, czajkowski.  It is absolutely a bug.  And it should be fixed.  I don't think it's risky or even related to privacy, and don't think we should agree to fix it before we release private projects from beta.
<czajkowski> deryck: that I don't know :) would there be a case if people didn't know it was turned off and if they were automatcally used to asking questions they could then.  I guess flacoste would have final say on it.  I just see it happening from time to time on projects on here
<czajkowski> which was why I logged it
<deryck> czajkowski: proprietary and embargoed projects are not the kind to file questions against anyway.  they're secret.  and I can see why anyone would accidentally do it because there's no use case for asking a question.
<deryck> but talking to rick_h_ in another channel, his fix for the retargeting issue will fix this as a by product, so no point arguing it anymore.
<deryck> rick_h_: what is the bug number of the new bug you filed?
<rick_h_> #1079678
<_mup_> Bug #1079678: You're able to create new questions on Launchpad even when questions is not avilable to a project <distributions> <lp-answers> <projects> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1079678 >
<rick_h_> sorry, wrong one
<rick_h_> #1079678
<_mup_> Bug #1079678: You're able to create new questions on Launchpad even when questions is not avilable to a project <distributions> <lp-answers> <projects> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1079678 >
<rick_h_> ugh, /me fail
<deryck> heh
<rick_h_> #1085102
<_mup_> Bug #1085102: A non-public project can currently be the target of a question. <lp-answers> <private-projects> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1085102 >
<rick_h_> there we go
<rick_h_> card updated on the board as well.
<deryck> rick_h_: thanks.  I reply in the original bug.
<sinzui> bac, do you have time to review https://code.launchpad.net/~sinzui/launchpad/system-bug-comments-api/+merge/137308
<bac> sinzui: i do
<bac> sinzui: done
<sinzui> thank you bac
<bac> de nada
<abentley> bac: Could you please review https://code.launchpad.net/~abentley/launchpad/no-private-translations/+merge/137316 ?
<bac> abentley: on it
<bac> abentley: in the error messages you alternately refer to "private projects" and "proprietary projects".  should you pick the one most common description -- or are you really trying to convey something different?
<abentley> bac: Okay.  (We don't really have a good umbrella term for Proprietary and Embargoed products, because "private" means something else in the context of branches and bugs.)
<abentley> bac: So that's my own dissatisfaction with either term showing up in the code.
<bac> abentley: gotcha
<bac> hi statik!
<statik> o/ bac
<lifeless> statik: o/
<lifeless> and o/ everyone here ;)
* bac changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On-call reviewer: - | Firefighting: - | Critical bugs: ~170
<jcsackett> deryck: you around? can i request a review of https://code.launchpad.net/~jcsackett/launchpad/blueprint-private-traversal/+merge/137330
<jcsackett> (or abentley or rick_h_) ^
<deryck> jcsackett: I can get it.
<jcsackett> awesome. thanks deryck.
<deryck> jcsackett: np
<deryck> jcsackett: am I reading the diff right, was there really a pdb in this code that your code removes?
 * jcsackett looks
<jcsackett> deryck: it's pipe madness. let me resubmit with the right target.
<deryck> jcsackett: ok, that's why I asked.  I didn't mind a missing pdb mind you. :)  But just wanted to make sure the diff was correct.
<jcsackett> deryck: if you want the fully proper diff: https://code.launchpad.net/~jcsackett/launchpad/blueprint-private-traversal/+merge/137333
<jcsackett> thought i had fully destroyed that pipe. clearly not. :-P
<deryck> jcsackett: heh, np.  thanks.
<deryck> jcsackett: looks great to me. r=me
<jcsackett> deryck: thanks!
<deryck> np
#launchpad-dev 2012-12-01
<jcsackett> rick_h_: are you actually around?
<jcsackett> or anyone in launchpad-reviewers? i have a branch i need looked at: https://code.launchpad.net/~jcsackett/launchpad/blueprint-private-traversal-deptree/+merge/137390
<rick_h_> jcsackett: yep, I'm in/out
<rick_h_> jcsackett: r=me with just one question/comment
<mordred> hey guys - I'm consistently getting an oops (and have been since yesterday) trying to access https://launchpad.net/~openstack
<rick_h_> mordred: looking
<rick_h_> mordred: do you have the oops id?
<rick_h_> mordred: loads here so if you can get me an oops id I can look but not seeing anything atm
<mordred> rick_h_: 76b293f762734f2a421cd3f3d4e8a160
<mordred> rick_h_: and I _LOVE_ it when it works for other people :)
<rick_h_> mordred: I *think* this looks a lot like an issue sinzui and czajkowski were working on friday. I don't have any handy trick for you atm. czajkowski might know a bug number to track as I think it was a recent regression sinzui was going to get a fix in for.
<mordred> rick_h_: that's fine - it's the weekend and I should perhaps be not working anyway :)
<mordred> rick_h_: if it's something that's in work, and if the page loads for some people, then I'll just leave it in your hands
<jcsackett> rick_h_: thanks for the review. i replied on the MP but the gist is that's covered by our standard access rules, as the root is the view context.
<rick_h_> jcsackett: ok cool. Assumed, but didn't want to assume
<jcsackett> rick_h_: dig. yeah, it's all good. thanks for the review; glad to clear this by monday. :-)
<rick_h_> jcsackett: yea, I've got to finish my branch tonight and try to get it reviewed asap monday :/
<jcsackett> rick_h_: i have to run a tool rental back to it's home by 8 tomorrow--i'll ping you when i get back from that and i you don't have a review by then i'll grab it. sound good?
<jcsackett> er, monday.
<jcsackett> not tomorrow. :-P
<rick_h_> jcsackett: cool, we'll get it
<jcsackett> rick_h_: damn right. :-P
#launchpad-dev 2012-12-02
<czajkowski> lifeless: sorry didnt run away, my screen well, went :) trying to reset up all the channels
#launchpad-dev 2013-11-25
<xnox> https://launchpad.net/~canonical-foundations/+archive/upstart-daily/+recipebuild/592052 "Missing build dependencies: python3:any" hm...
<cjwatson> wgrant: Have you had a chance to consider https://code.launchpad.net/~cjwatson/launchpad-buildd/livefs/+merge/193682 (and the request for your feedback in the associated bug)?
<wgrant> cjwatson: Sorry, hadn't noticed that was ready. Will look.
#launchpad-dev 2013-11-26
<lifeless> wgrant: hey
<lifeless> wgrant: is there a good thing around for generating comprehensive per-project stats of bugs?
<lifeless> wgrant: specifically # opened/day, # changed state/day # commented on by non-devs/day etc?
<wgrant> lifeless: No existing API script that I know of.
<lifeless> nuts
<lifeless> thanks
<stub> wgrant: So I think that @inlineCallbacks decorator is required because, despite having the result tossed away, the yields does let the reactor do its thing and all the processing completes before it returns and gets its result tossed
<stub> I seem to be able to hold twisted in my head for about 3 days before I forget how it fits together :-/
<wgrant> Hum, that shouldn't work :/
<wgrant> stub: Where does the exception show up if you raise one after a yield in resumeProcessing?
<stub> wgrant: https://pastebin.canonical.com/101029/
<stub> Its all so lovely and magical ;)
<stub> I thought I understood how it was working, but that might have just been me rationalising observed behaviour.
<stub> The request invokes resumeProcessing for the producer to actually produce something. It doesn't care what it returns, as it is responsible for spitting out the data. It can do this asynchronously if it wants.
<wgrant> Yeah, but normally in Twisted you have to return the deferred up to the reactor before it's executed.
<lifeless> wgrant: huh? no
<lifeless> wgrant: the reactor holds references to things it will fire like protocols and callLaters
<lifeless> wgrant: you return defereds so that your caller can attach a callback
<wgrant> lifeless: But a deferred won't fire until it has a callback attached, will it?
<wgrant> Hm, though I guess that doesn't matter here.
<lifeless> no
<lifeless> defereds can fire before the callback is attached
<wgrant> stub: Anyway, if resumeProducing is allowed to do nothing immediately I suppose that works.
<stub> yes, resumeProducing is invoked and expected to eventually produce the result. Its actual return value is ignored.
#launchpad-dev 2013-11-27
<lifeless> wgrant: offhand, how do you make searchTasks (HTTP API) order by bug id ascending ?
<lifeless> Unrecognized order_by: u'bug.id'
<wgrant> lifeless: id
<wgrant> orderby_expression = { "task": (BugTaskFlat.bugtask_id, []), "id": (BugTaskFlat.bug_id, []),
<lifeless> wgrant: ack, thanks.
<bigjools> why would I get signed MP mails occasionally rejected with "bad signature"?  It seems entirely random whether something will work or not.
<wgrant> bigjools: Have you tried verifying the signature manually?
<bigjools> wgrant: nope. I know Launchpad well enough to suspect a bug here :)
<wgrant> It's very unlikely.
<wgrant> More likely is that the email is being mangled slightly before it gets to Launchpad.
<bigjools> I love your optimism for a change, where's the normal LP pessimism? :)
<wgrant> I trust this code more than the rest, because I've audited it for security vulnerabilities :)
<bigjools> wait - does it deal with multipart/signed?
<wgrant> I haven't used stupid inline signatures in a good many years, so yeah.
<wgrant> Does it work if you resend the same email?
#launchpad-dev 2013-11-28
<lifeless> wgrant: is date_closed set when status -> Opinion?
 * lifeless is being lazt
<wgrant>         if ((old_status in DB_UNRESOLVED_BUGTASK_STATUSES) and
<wgrant>             (new_status in RESOLVED_BUGTASK_STATUSES)):
<wgrant>             self.date_closed = when
<wgrant> RESOLVED_BUGTASK_STATUSES = ( BugTaskStatus.FIXRELEASED, BugTaskStatus.OPINION,
<wgrant> lifeless: ^^
<lifeless> great, thanks
<lifeless> https://api.launchpad.net/1.0.html#bug_task is stale then
<lifeless> it claims "
<lifeless> Date Closed
<lifeless> The date on which this task was marked either Won't Fix, Invalid or Fix Released.
<pac1> A bug was marked as fixed in karmic, but it is also present in 13.10 as bug 1239912 The 13.10 bug was marked as a duplicate of the karmic bug.   #390421.  I removed the duplicate mark because the old bug was against karmic.  Was that the right thing to do?  If not I'll un-do it.
<lifeless> "
<_mup_> Bug #1239912: system-config-lvm fails to start citing missing /etc/init.d/lvm2 <amd64> <apport-bug> <patch> <saucy> <system-config-lvm (Ubuntu):Confirmed> <https://launchpad.net/bugs/1239912>
<_mup_> Bug #390421: system-config-lvm fails with invoke-rc.d: unknown initscript, /etc/init.d/lvm2 not found. <amd64> <apport-bug> <bitesize> <patch> <regression-potential> <system-config-lvm (Ubuntu):Fix Released> <https://launchpad.net/bugs/390421>
<wgrant> pac1: The new bug should be unduped if the old one was already fixed years ago.
<wgrant> (also, this sort of question about Ubuntu's bug management policies probably makes more sense in #ubuntu-bugs)
<wgrant> lifeless: Opinion shouldn't really exist, but Expired should be added to that list.
<pac1> ok thanks.  I'll go there next.
<lifeless> wgrant: agreed on should, but it does.
<lifeless> wgrant: the real bug is the duplication of definition
<wgrant> Sure
<lifeless> anyhow
<lifeless> just thought you'd like to know why I was asking a daft q :)
<wgrant> Yeah, will land a fix once buildbot is unbroken
<wgrant> I wonder if I can delete Opinion without Mark noticing :)
<lifeless> wgrant: https://review.openstack.org/#/c/58903/
<lifeless> (sample output in there now)
<wgrant> lifeless: Nice
<wgrant> (apidoc fix is in trunk)
<lifeless> cool, thanks
<lifeless> wgrant: obviously if you have suggestions about the bugstats thing, I'm all ears.
<wgrant> stub: https://code.launchpad.net/~wgrant/launchpad/multiproc-builders-db/+merge/197013
<stub> wgrant: r=stub
<wgrant> stub: Thanks.
<wgrant> stub: https://code.launchpad.net/~wgrant/launchpad/antibfjo-9-db-kill/+merge/196640
#launchpad-dev 2013-12-01
<daker> hi
<daker> the launchpad-developer-dependencies package doesn't exist on 13.10
<StevenK> daker: It is not in the Ubuntu archive, it is only in the launchpad PPA.
<wgrant> But it wasn't in trusty in the PPA until about 2 minutes ago :)
<daker> StevenK: i used the rocketfuel-setup script
<wgrant> Wait, 13.10 is saucy, I'm apparently asleep
<wgrant> daker: What was the error message?
<daker> wgrant: deps error wait a sec
<daker> wgrant: http://paste.ubuntu.com/6506826/
<wgrant> Ah, so it does exist, it's just not installable.
<daker> wgrant: also i am seeing http://ppa.launchpad.net/bzr/ppa/ubuntu/dists/saucy/main/binary-i386/Packages  404  Not Found
<wgrant> That shouldn't be a problem.
<wgrant> I'm looking at the lp-dev-deps uninstallability.
<StevenK> wgrant: pgbouncer in Saucy itself is a higher version than the pgbouncer in the PPA
<StevenK> I've had it pinned locally for a long while
<wgrant> Yeah, I know
<wgrant> Fortunately we don't use that as a build-dep
<wgrant> So a versioned alternative should be fine
<wgrant> (my patch made it upstream ages ago)
<wgrant> Hm, though it doesn't seem to have been in a release since then.
<wgrant> I'll upload a patched 1.5.4 everywhere.
#launchpad-dev 2014-11-25
<cjwatson> wgrant: Do we still need to keep lucid_lp_lxc and lucid_db_lp_lxc on the waterfall page?
#launchpad-dev 2014-11-28
<xnox> so ubuntu archive accepts uploads signed by sub-key, however ppa uploads don't?
<xnox> actually it does, got accept email now.
<xnox> probably i simply remember accepts to happen quicker. meh, all is good.
<wgrant> The same code processes both, just with slightly different rules. None of the signature verification differs.
<xnox> cool, thansk.
#launchpad-dev 2015-11-27
<dupingping> hi everyone.
<dupingping> How can i setup launchpad on my local server?
<cjwatson> dupingping: https://dev.launchpad.net/Running - that's mainly for test instances though, if you plan to expose it to the world then you will want to talk over the details with us first since that's much harder to get right
<dupingping> cjwatson: yes, let me try.
#launchpad-dev 2015-11-28
<samsruti> Hi everyone :)
#launchpad-dev 2016-11-30
<mgz> OOPS-31f7acbd51ace0fe4e163568bdd6660f
<mgz> hm, pretty oops bot gone?
