#launchpad-dev 2010-03-29
 * thumper twiddles his thumbs while the tests run
<spm> thumper: do you twiddle clockwise or anti? or switch from one t'other? enquiring minds wish to know.
<thumper> spm: I alternate
<thumper> like current, but not as fast
 * spm looks for a report on the intrawebsfont-of-all-knowledge as to what that says about thumpers mental state of being
<spm> sadly, reckon I could google for that and get a semi-credible answer.... :-)
<thumper> spm: an answer yes, a semi-credible one? now that is a question all in itself
<spm> ha
 * mwhudson lunches
 * thumper is having test failures with GpgmeError: (32, 176, 'Unknown error code')
<wgrant> thumper: There is a patch needed for Lucid.
<thumper> :(
<wgrant> I think it's in trunk now.
<wgrant> pygpgme trunk, that is.
<thumper> is it in LP code somewhere?
<thumper> I'm running the tests through ec2 now anyway
<wgrant> mwhudson actually approved the sourcedeps.conf change a few days ago.
<wgrant> It just hasn't been landed yet.
<thumper> ok
<thumper> I thought I had seen something about pygpgme fly past
<mwhudson> bad jelmer
<poolie> wgrant: re bug 549323, i did try to tell them so
<mup> Bug #549323: Discourages bug fixes <Launchpad Bugs:New> <https://launchpad.net/bugs/549323>
 * thumper has finally started writing the tests for iterReady
<lifeless> wgrant: ugh, is this deliberate or a bug ?
<thumper> mwhudson: happy with bug 532210?
<mwhudson> thumper: yes
<mwhudson> didn't i change the tag already?
<mwhudson> no
<thumper> not yet
<mwhudson> ok
<mwhudson> now i have :)
<thumper> ta
<thumper> mwhudson: do we have a reasonable branch to test https://bugs.edge.launchpad.net/launchpad-code/+bug/532402 on?
<mup> Bug #532402: record last_revision used to determine partial success before doing import <code-import> <qa-needstesting> <trivial> <Launchpad Bazaar Integration:Fix Committed by mwhudson> <https://launchpad.net/bugs/532402>
<mwhudson> thumper: not really
<thumper> mwhudson: I suppose we should just test the imports on staging and see that they work still
<mwhudson> thumper: i can think of some slightly contrived ways to test it by getting a losa to SIGSTOP the import processs
<mwhudson> thumper: yeah
<mwhudson> thumper: doing that already in fact :)
<spm> if only there was a friendly losa around....
<thumper> cool
<thumper> spm: yeah, know any?
<mwhudson> thumper:  https://code.staging.launchpad.net/~vcs-imports/gedit/head
<spm> thumper: not on the key criteria no. losa's around? yup. friendly? no.
<mwhudson> it's also a fiddle, the only public svn repo i can think of is codespeak, and bzr-svn tends to break on that
<mwhudson> (thanks to svn bugs)
<thumper> mwhudson: that subversion import is via cscvs, you knew that right?
 * thumper afk for a bit
<mwhudson> thumper: ah balls
<mwhudson> thumper: thanks
 * mwhudson stops
<wgrant> lifeless: Is which deliberate?
<wgrant> poolie: I decided not to. I've almost given up trying to convince people nicely.
<lifeless> wgrant: bug 549323
<mup> Bug #549323: confirmation dialog on bug status change discourages bug fixes <Launchpad Bugs:Confirmed> <https://launchpad.net/bugs/549323>
<wgrant> lifeless: It's quite deliberate.
<wgrant> If you're not a bug supervisor, you're not meant to change the status.
<poolie> lifeless: read the other bug linked from it
<poolie> wgrant :-(
<lifeless> wgrant: if we're doing that, we should delete 'in progress'
<poolie> ?
<lifeless> poolie: so that there is no bug state that contributors who aren't bug supervisors need to set the bug to
<poolie> oh i don't know
<lifeless> I think 'assignee' is sufficient to indicate in progress
<lifeless> confirmed|triaged + assigned
<poolie> i see
<poolie> this is a different issue
<poolie> fwiw i think you can say "some work has been done on this but nobody owns it at the moment"
<poolie> however
<poolie> the point here is not that people cannot set the state
<poolie> just an attempt to reduce erroneous settings
<poolie> and a confirmation dialog is a very poor way to accomplish that
<lifeless> poolie: arguably though, it won't progress when noone owns it.
<lifeless> poolie: yes, I can see the dialog being a poor way issue
<poolie> right, i don't think you should leave bugs in this state for long
<poolie> but they can be there
<poolie> it's like saying you shouldn't be allowed to have open critical unassigned bugs :)
<lifeless> true
<lifeless> I think it depends on what you take 'in progress' to mean.
<lifeless> if you mean 'started but not finished', then I agree that it can be in that state.
<wgrant> An alert() which users cannot bypass is a *particularly* poor way to accomplish it.
<lifeless> if you mean 'being worked on', then I don't agree.
<lifeless> And I think most users take 'in progress' to mean 'progressing' not 'started but not finished'
<poolie> i don't think "actually being worked on now" is modelled very well by a single field on the task
<poolie> it will be out of date
<lifeless> agreed
<poolie> wgrant: is it really 'bug supervisor' not, as promised in bug 531963, 'bug contributor'?
<mup> Bug #531963: Add a confirmation step when setting the bug status if the user is not a bug contributor <qa-ok> <Launchpad Bugs:Fix Committed by intellectronica> <https://launchpad.net/bugs/531963>
<wgrant> poolie: 'bug contributor' doesn't exist, except by karma.
<wgrant> poolie: The review I snooped at checked for importance privileges.
 * wgrant checks the source.
<wgrant> +            'status_requires_confirmation': not self.user_can_edit_importance,
<wgrant> poolie: ^^
<poolie> ok, thanks
<poolie> so there is no escape
<lifeless> perhaps it would be better if it was more sophisticated than that
<wgrant> Apart from Greasemonkey.
<lifeless> \o/
<wgrant> lifeless: Indeed.
<adeuring> good morning
<wgrant> noodles775: Morning.
<noodles775> Hi wgrant and adeuring :)
<adeuring> hi noodles775!
<wgrant> Hi adeuring.
<adeuring> hi wgrant!
<wgrant> noodles775: Since LP won't have emailed anybody, again, have a https://bugs.edge.launchpad.net/soyuz/+bug/550233. Also https://bugs.edge.launchpad.net/soyuz/+bug/550049, but I've already poked Julian about that.
 * noodles775 looks
<poolie> wgrant: urk, now i get that dialog about bzr in ubuntu
<wgrant> poolie: Ha ha ha.
<wgrant> Give them hell.
 * wgrant heads trainward.
 * thumper is having trouble stopping
<StevenK> thumper: Me too
<lifeless> thumper: switch to your other project
<bigjools> his wife?
<bigjools> lifeless: congrats BTW
<lifeless> bigjools: thanks :)
<bigjools> lifeless: is this a hacking honeymoon? :)
<lifeless> bigjools: honeymoon will be later this year
<lifeless> waaay to much on right now to do it atm
<mrevell> Morning
<fayce> Hi all ! I'd like to install Launchpad on a Ubuntu box, but I wonder how hard it would be to make it scalable
<lifeless> fayce: launchpad itself - depends on what you mean by scalable
<fayce> thank you lifeless, I meant by "scalable" the fact that I can load balance it trough different physical (or virtual) servers without too much work
<fayce> I guess that launchpad is designed that way... kind of stateless ?
<lifeless> its not stateless
<lifeless> but the state lives in a db
<wgrant> bigjools: What in IBuildFarmJob says that it is a DB object?
<wgrant> (I don't know how it manages to not live in the DB; that is Code magic.)
<fayce> ok, thanx. I'm about to get the source code and try to install it.
<jml> hello all
<wgrant> Morning jml.
<bigjools> wgrant: I don't understand your first comment, but the second one is exactly what bothers me
<bigjools> I don't want magic
<wgrant> bigjools: By calling Store.remove(some IBuildFarmJob), we are violating IBuildFarmJob.
<bigjools> how?
<wgrant> Nothing in the interface says that it can be removed from a store.
<bigjools> it's no different to destroySelf, except I don't want any more of the latter
<bigjools> IBuildFarmJob should have a concrete table behind it somewhere
<wgrant> IBFJ now defines destroySelf. verifyObject will correctly catch if something is missing it.
<bigjools> that's my point
<wgrant> It should.
<wgrant> And we should really work out a solution to that, true.
<bigjools> right - so I prefer that to hacking something in
<stub> create an IStorm interface and attach it to storm.base.Storm objects with ZCML.
<wgrant> stub: I thought we couldn't do that.
<stub> why not?
<stub> You could also adapt the object to IMasterObject, which would give you a Storm object or fail.
<wgrant> Didn't I see a comment suggesting that the I*Store adapters registered on Interface because you can't tell Zope that a class and all its derivatives implement an interface.
<wgrant> ?
<stub> maybe. I don't recall.
<wgrant> lib/canonical/launchpad/webapp/adapter.py, just before get_store.
<stub> Oh... that is classes.
<stub> We adapt classes to I*Store, which is the problem. This would be adapting instances which is fine. Just declare Storm implements IStorm and all Storm instances will provide that interface.
<wgrant> Oh, right.
<wgrant> That makes more sense ow.
<stub> But if a class provides an interface, subclasses don't automatically provide it too.
<stub> (Which I think is a bug? Not sure, but it isn't going to be fixed any time soon if it is)
<bigjools> sounds like a bug to me
<bigjools> wgrant: anyway as I said on the bug I'm of a mind to mark it invalid
<wgrant> bigjools: Well, when is the data model rework going to happen?
<bigjools> wgrant: noodles775 is working on it right now
<wgrant> Excellent.
<bigjools> we need it for builder history
<bigjools> and jtv needs to fix his stuff too :)
<wgrant> Yep.
<wgrant> But with that and my other Saturday fixes, translations jobs work perfectly.
<noodles775> wgrant, bigjools: *was* going to start working on it today, but other things came up ;)
<bigjools> :)
<bigjools> have you ever been to a blackhat wgrant? :)
<wgrant> bigjools: No.
<bigjools> anyway, UI should drive all other features, not the other way around.  jml would agree with me.
<wgrant> +inf
<bigjools> heh
<lifeless> bigjools: I would say user experience, not ui
<lifeless> I think its closer to the spirit of the issue
<bigjools> amounts to the same thing for me, but yeah
<deryck> Morning, all.
* deryck changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.03 | PQM is in RC mode | Release manager: deryck | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes |
<wgrant> deryck: Hi. Can I please have your blessing for https://code.edge.launchpad.net/~wgrant/launchpad/bug-540819-fix-builder-list-icons/+merge/22288?
<deryck> Hi wgrant.  Let me take a look....
<deryck> wgrant, can you get the normal reviews from someone else first please?  Then I'll consider it for an RC.  But my initial reaction is I have no qualms about this going in.
<deryck> I think we will also have someone from bugs back out the confirmation dialog for status change, until it has been correctly done.
<wgrant> deryck: Ah, OK.
<wgrant> deryck: +1 from me on backing out the confirmation dialog, clearly...
<stub> Has meliae become a new dependency, but not been added to buildout or -dependencies?
<stub> Ahh... forgot to rebuild my eggs.
<jml> gror backlight
<bac> hey salgado are you CHR today?
<salgado> I should be but had completely forgotten about it
<salgado> thanks for reminding me, bac
<bac> salgado: there are some questions in #launchpad
<adeuring> am I right that we now need a referer header for each POST request?
<leonardr> adeuring: right
<adeuring> leonardr: gah... I can't use firefox to edit any LP page...
<leonardr> mwhudson, if you're around i have a question about http://www.mail-archive.com/launchpad-dev@lists.launchpad.net/msg02999.html
<adeuring> but the important detail:
<adeuring> We don't get any longer new submissions to the HDWB
<adeuring> since when do we have this referer check?
<leonardr> adeuring: a couple weeks ago. it was a security measure
<adeuring> leonardr: sure, I understand that. Is it possible to switch this off for selected forms?
<leonardr> adeuring: yes, i'm finding the information for you now
<leonardr> adeuring: the bug is https://bugs.edge.launchpad.net/launchpad-foundations/+bug/529348
<adeuring> leonardr: thanks!
<leonardr> np
<adeuring> gary_poster: I just got a head-up from cr3 and jcastro that LP does not accept any longer ay HWDB submission. The reason is simple: The data is posted by checkbox, and the request does not contain a referer header. Can you give me a clue how to best fix this?
<adeuring> there is this exemption for "/+storeblob" and a few other URLs, should we extend that?
<gary_poster> adeuring: yes, I suppose so, with associated tests, and quite possibly with an associated bug or two against the webservice saying that whatever the HWDB is doing now should be doable with the webservice, at which point we can remove the exceptions.
<gary_poster> adeuring: what do you mean "The data is posted by checkbox"?
<adeuring> gary_poster: the program that collects the data on a user's machine is called checkbox.
<gary_poster> ah, ok
<adeuring> and it seems to POST without a referer
<gary_poster> I thought it was a form element :-)
<adeuring> gary_poster: ;)
<gary_poster> adeuring: so...actually, I also suggest that they change their script now to include a REFERER.  That way eventually legacy clients will "just work," and sooner than if they wait to be able to do whatever it is they need to do through the webservice API
<adeuring> gary_poster: yes, checkbox should do that. But it is installed by default on every Ubuntu system, and getting rid of old version will ned quite some time...
<gary_poster> adeuring: ack, so let's get started ;-)  getting the change into lucid would be a *big* win in that regard
<adeuring> right
<deryck> sinzui, ping
<sinzui> hi deryck
<deryck> sinzui, hi.  should Bug #550980 be a registry bug instead of malone?  (I realize anyone can fix it, just curious who normally owns that script.)
<mup> Bug #550980: Ordering of Bug importance is wrong <Launchpad Bugs:New> <https://launchpad.net/bugs/550980>
 * deryck fixes the title
<sinzui> It is a registry bug
<sinzui> It is also browser based. webkit shows things very wrong. firefox is kind of right
<deryck> ah, ok.  I'll retarget to you guys then.
<sinzui> already done
<deryck> and there's our friend the NotImplementedError. :-)  You did beat me
<gary_poster> jml, james_w: I'd really rather leonardr and I not consider the adapter question until we've dealt with other things on our schedule, and I'd really rather decisions assuming a given solution (which is what I--perhaps mistakenly--understand the current resolution of your mail thread) to be postponed.  I understand that you want changes, but pushing for too much at once will lead to flailing, I'm afraid.
<gary_poster> ah, disappointments: the local paneras no longer allows ssh connections apparently :-/
<gary_poster> leonardr: my concern expressed about the adapter thread (and the original question about exposing newCodeImport), if you do have any quick wisdom or ideas to share with james_w and jml, please do.
<sinzui> deryck: That bug was a duplicate of a foundations bug. I also found the same bug reported in malone. I duped them so that it was clear that there is one universal webkit js problem in launchpad
<deryck> sinzui, excellent, thanks!
<leonardr> gary, looking
<james_w> gary_poster: I don't think I'm rushing ahead, I'm just responding to the only replies I have had on the topic. If you want to defer exposing anything that uses adapters until there is a general solution in place then please tell me and I shall go work on something else.
<gary_poster> james_w: fair enough.  My concern is jml encouraging an approach when we don't have an agreement yet, and, as he said, we have (already too many) other things to focus on right now.  I don't know the API you are trying to expose.
<gary_poster> If your approach to exposing does not assume a particular general solution, and is something that would really be a big win for you, then I'll not feel comfortable discouraging you.  If it is a minor win, or if it involves changes to launchpadlib itself, then I'll be skeptical that this should be pursued until Leonard and I have bandwidth to discuss it.
<james_w> I can expose it anyway you like
<james_w> I definitely do not intend to change launchpadlib
<james_w> and the fact that I am working on LP rather than continue to wait for the change to bubble to the top of an LP developers stack should indicate that it is important for us
<leonardr> james_w, gary: i've responded via email
<james_w> thanks leonardr
<gary_poster> james_w: "expose any way you like": if it is within the range of the current standard tools to expose APIs, please consider me out of your way.  The code team (and probably Jono, with his experience there) are the ones to guide you.
<gary_poster> "do not intend to change launchpadlib": OK sorry, that was a question I had when I read that a change would be on the "client side."  A misunderstanding.
<gary_poster> not waiting for LP developers: understood.
<gary_poster> I don't want to be a hindrance.  It doesn't sound like I need to be one.  Sorry for the noise.
<james_w> gary_poster: thanks, let's continue discussion on the mailing list to decide what the best way to proceed is, at least in the short-term.
<gary_poster> james_w: cool.
<leonardr> is there an easy way to look up a member of an enum by token?
<mwhudson> leonardr: what do you mean by 'token' ?
<leonardr> mwhudson: i'm looking at a dbenum value and it has a number of fields: .token, .sort_key, etc
<leonardr> that's the token i mean
<mwhudson> um
<mwhudson> i don't see token on this dbitem
<thumper> morning
<thumper> leonardr: yes
<thumper> leonardr: I think
<mwhudson> leonardr: it looks like the class has a getTermByToken method though
<leonardr> ok, let me get precise about what i'm seeing
<leonardr> <DBEnumeratedType 'OAuthPermission'>
<leonardr> it has a .description and an __iter__
<leonardr> [x for x in OAuthPermission] is a bunch of lazr.enum._enum.TokenizedITem objects
<leonardr> each has a .title, a .token, and a .value
 * thumper nods
<leonardr> i don't see any lookup methods anywhere. i get the feeling this is on a different level of abstraction from DBItem
<thumper> leonardr: the item lookup is abstracted
<thumper> leonardr: for no good reason
<leonardr> i'm getting the feeling it's not worth figureing this out to save a couple lines of code
<thumper> what are you wanting?
<leonardr> here's my code
<leonardr>         for check_permission in OAuthPermission:
<leonardr>             if check_permission.token == permission:
<leonardr>                 permission = check_permission.value
<leonardr>                 break
<thumper> to get the enuerated value from the token name?
<leonardr> i'd like to write permission = OAuthPermission.getByToken(permission)
<thumper> permission = OAuthPermission.items[permission]
<thumper> assuming permission is the string
<thumper> >>> i
<thumper> <lazr.enum._enum.TokenizedItem object at 0x77526d0>
<thumper> >>> i.token
<thumper> 'WRITE_PRIVATE'
<thumper> >>> t
<thumper> <DBEnumeratedType 'OAuthPermission'>
<thumper> >>> t.items[i.token]
<thumper> <DBItem OAuthPermission.WRITE_PRIVATE, (50) Change Anything>
<thumper> leonardr: that work for you?
<leonardr> yes, that seems to work
<thumper> coolio
<leonardr> thanks
<lifeless> moin
 * mwhudson rebooting
<thumper> ââ¹
<thumper> that is for whoever decided that preferredemail was better than preferred_email
<thumper> and while we're at it
<thumper> displayname instead of display_name
<mwhudson> thumper: ancient history isn't it?
<thumper> probably
<mwhudson> for a while _ wasn't allowed in column names in the db
<thumper> stupid
<thumper> every time I need to access preferredemail I get it wrong
<thumper> mwhudson: got a minute to talk through something twisted?
<thumper> hmm.. that could be a no
<mwhudson> well, that was err, fun
#launchpad-dev 2010-03-30
 * mwhudson lunches
<wgrant> poolie: Is it really that odd to support only OpenID? Most Canonical web services exclusively use Ubuntu SSO.
<wgrant> There's little point have a webapp-specific username and password.
<poolie> i meant among other web apps generally
<thumper> mwhudson: do you know anything about ampoule?
<poolie> they seem to at least present the impression of letting you enter the username/password directly
<poolie> through a web page or over a machine interface
<poolie> even if in the backend it's going to a generalized db
<poolie> don't you think so?
<wgrant> Right -- but what's the point?
<wgrant> Er, wait, are you talking about webapps in general, or Canonical webapps?
<poolie> generally
<wgrant> Ah.
<wgrant> Well, what benefit does this provide?
<wgrant> In this case there is a default OpenID provider that makes a little bit of sense but not really, so it makes sense in the minds of people who want to associate Launchpad even further with Ubuntu to delegate all authentication to that.
<wgrant> Hopefully LP will soon be able to auth against any provider, which might make it less completely unacceptable to most distro-agnostic upstreams.
<wgrant> Forcing upstream projects through an Ubuntu-branded page is *not* going to make people happy.
<wgrant> But I think dropping the internal authentication is a good move.
<poolie> (phone)
<poolie> i think it's useful to be able to do "curl https://user:pass@launchpad.net/something"
<wgrant> I haven't had to do that in ages, thanks to the API.
<poolie> or similarly for the api
<poolie> which is where we started
<wgrant> The only supported auth mechanism for API requests is OAuth. The workflow for getting an OAuth token can be easily streamlined.
 * thumper stabby
<thumper> :(
<mwhudson> thumper: only in general terms
<thumper> mwhudson: I'm stuck and would like help
<mwhudson> thumper: ok
<mwhudson> thumper: skype?
<thumper> please
<mwhudson> oh right, it's not running...
<thumper> lp:~thumper/launchpad/all-code-review-email-using-jobs
<NCommander> In general, how bad of an idea is it to count the number of rows in SRP?
 * NCommander is working on the importer now that I have some time
<NCommander> My postgresql is rusty, how do you select columns where an integer column is unset (but not null, the parser doesn't seem to like '')
<NCommander> bah,nm
<lifeless> NCommander: unset *is* NULL
<lifeless> NCommander: '' is not NULL
<wgrant> NCommander: SRP? You mean SPR?
<NCommander> wgrant: er, yeah
<NCommander> lifeless: yeah, I had a bad flashback to mysql, and forgot that postgresql doesn't suffer that much braindamage
<wgrant> NCommander: Why do you want to?
<NCommander> wgrant: to give a total and an overall progress (this is probably going to take days run), but I also found count(*) does a sequential scan thus, may not be the best to run on such a large table :-)
<wgrant> Possibly. Hard to say.
<wgrant> Just do it, I think. If it becomes a problem, it can be fixed.
<wgrant> Premature optimisation, and all that.
<NCommander> wgrant: fair enough, I'm thinking I can't do this as one giant transaction, but break it into batches
<wgrant> NCommander: stub would come and strangle you if you did it in one transaction.
<NCommander> wgrant: yes, well :-), I dunno how much ot run in a batch
<wgrant> There is DBLoopTuner, but I don't think that handles committing for you.
<NCommander> 21:15:08 < NCommander> wgrant: yes, well :-), I dunno how much ot run in a batch
<NCommander> 21:15:42 < wgrant> There is DBLoopTuner, but I don't think that handles committing for you.
<NCommander> argh
<NCommander> wgrant: what's DBLoopTuner
<wgrant> It handles batching and automatically tunes the batch size to meet a target time.
<wgrant> Not sure how applicable it would be to this case, though.
<NCommander> wgrant: ah
<thumper> mwhudson: skype fail
<mwhudson> thumper: wifi fail
<thumper> http://pastebin.ubuntu.com/406280/
<mwhudson> anyone know how to debug postgres deadlocks?
<mwhudson> using pg_locks and all that
<lifeless> mwhudson: I've done so in the past
<lifeless> mwhudson: still need help ?
<mwhudson> lifeless: nah, we found it
<mwhudson> ta though
<NCommander> Stupid question, but is there a good example on how to retrieve a file via librarian?
 * NCommander is trying to figure out how to sanely download all the bits of a source package giving a SPR record
<mwhudson> NCommander: via the api or ... ?
<NCommander> mwhudson: I'm writing a script to handle migration for raw_source_changelog but my inexperience with LP's internal APIs are making it difficult, i.e., how to get a file from librarian, or getting all the bits of a source package, etc.
<mwhudson> NCommander: getting a file from the librarian is quite easy
<mwhudson> NCommander: it's just librarianfilealias.open()
<mwhudson> source packages i'm less sure of
<NCommander> mwhudson: well, I already have a function that spits out all the id numbers and filenames of the source package :-)
<NCommander> mwhudson: I'm not sure I can determine that given a specific LFA though if its in the restricted librarian or in the primary one
<mwhudson> NCommander: i think lfa.open() dtrt
<NCommander> dtrt?
<mwhudson> NCommander: does the right thing
<NCommander> mwhudson: ahhhhhh
 * thumper afk for urgent shopping for dinner
<wgrant> NCommander: LFA.restricted should tell you whether it's restricted.
<NCommander> wgrant: can I just lfa.open() or do I have to do something special if its restricted?
<wgrant> NCommander: LFA.open() should work fine.
<NCommander> wgrant: perfect!
<poolie> is hackberry the staging db or the real one?
<mwhudson> poolie: it's a slave not one of the masters
<poolie> but a slave of the production db?
<poolie> iow scripts running there see up-to-date data?
<mwhudson> poolie: looks like a slave of production, yeah
<poolie> thanks
<spm> poolie: https://devpad.canonical.com/~spm/launchpad-sec-levels.png hackberry is a slave; is < 5 minutes behind in replication or I get sms'd.
<poolie> nice
<spm> hrm. dated too. missing soybean for 1.
<spm> bleh. and pear.
 * thumper back later to finish submitting proposals
<wgrant> henninge: Morning.
<henninge> Hi wgrant! ;)
<wgrant> henninge: With https://code.edge.launchpad.net/~wgrant/launchpad/get-translations-jobs-working, translation template jobs work completely.
<wgrant> Just needed a few tweaks in untestable code.
<henninge> wgrant: Great to hear!
<henninge> let me look at it
<stub> Is there a PPA with basic Python 2.4 for Lucid? Lost 2.4 during the upgrade.
<StevenK> stub: You should be able to drag it in from Karmic
<wgrant> henninge: It has a few prereqs, one of which been rejected, so it will have to wait for the data model rework.
<wgrant> stub: Why would you want 2.4?
<stub> Because I build pytz for Python 2.3+
<wgrant> Ah.
<wgrant> henninge: But it was encouraging to see that it actually pretty much works.
<wgrant> And doesn't break the UI any more.
<wgrant> (and also gave me a crash course in Rosetta, which I'd not used before!)
<henninge> wgrant: yes, that is an important step forward.
<henninge> ;-)
<henninge> welcome!
<wgrant> henninge: What's going to happen during an update when new POs are scraped from the branch and uploaded before the new POT finishes building?
<henninge> wgrant: POs without a template will not be imported until the template has been imported.
<wgrant> henninge: Things aren't going to break strangely if POs with new strings are uploaded before the POT with those strings, though?
<henninge> wgrant: let me look at this but I think not
<adeuring> good morning
<noodles775> Hallo.
<henninge> wgrant: yes, all messages are stored in the database, even those that don't have a matching template entry (yet).
<wgrant> henninge: Ah, great.
<henninge> wgrant: they are marked as obsolete.
<henninge> but will be un-obsolleted once the template comes in.
<henninge> noodles775: moin!
<noodles775> Hi henninge
<wgrant> henninge: Just thought it might be a bit of a new situation, since existing bzr imports are synchronous.
<henninge> wgrant: well, we have always had manual uploads and they would come in any order the uploader chooses .... ;)
<wgrant> henninge: True.
 * wgrant looks for a reviewer for the RC candidate https://code.edge.launchpad.net/~wgrant/launchpad/bug-540819-fix-builder-list-icons/+merge/22288
 * thumper EODs
<mrevell> Morning
 * wgrant ponders removing ViewByLoggedInUser (the default IAuthorization adapter) and seeing what breaks.
<wgrant> It is dangerous.
<bigjools> g'day
<wgrant> Morning bigjools.
<bigjools> what's broken today?
<wgrant> SPRB permissions are bogus.
<wgrant> But not an actual problem yet, of course.
<jml> gary_poster,
<jml> good morning
<jml> wgrant, there's a default adapter? I think I knew that.
<jml> wgrant, I'm surprised the default is something like that though.
<wgrant> jml: Yes :(
<wgrant> It is perhaps the most silly default possible.
<wgrant> Except perhaps for allowing unauthenticated users and forbidding authenticated ones.
<wgrant> bigjools: Thanks. I've filed a bug and added it to the XXX.
<bigjools> ta
<bigjools> wgrant: mind if I mark bug 549340 as invalid?
<mup> Bug #549340: BuildQueue.destroySelf erroneously assumes that all jobs are Storm objects <buildfarm> <Soyuz:In Progress by wgrant> <https://launchpad.net/bugs/549340>
<wgrant> bigjools: Go ahead.
<bigjools> wgrant: thanks
<wgrant> Although I still believe that it's an interface violation, we'll see what happens once the model changes are shaken out.
<bigjools> yep
<bigjools> stub: dogfood has got a load of pg backends running that we can't cancel (mainly connecting to sso_main and launchpad_auth), what's the best thing to do?
<stub> bounce postgres probably.
<stub> pg_ctlcluster -force 8.3 main restart
<stub> Need a losa I guess
<mthaddon> actually, that'd need a GSA, losas don't have perms on dogfood (because we don't manage it)
<bigjools> stub: also, lazr.config.interfaces.ConfigErrors: ConfigErrors: database does not have a main_slave key.
<bigjools> database does not have a main_master key.
<bigjools> wtf?
<stub> bigjools: Perhaps you should request access as the postgres user? You can escalate your privs if you try hard anyway so it doesn't make much sense from a security point of view.
<bigjools> yeah sounds sane
<bigjools> well - I can do some stuff as postgres already
<stub> bigjools: It changed to rw_main_master I believe for read only mode stuff.
<bigjools> stub: does this mean we need a database refresh or what?
<stub> Its a config file change. I don't think you need to rebuild your database for that.
<bigjools> thought we'd changed that, hmm
<bigjools> yes we had, it got reverted
<bigjools> stub: there was a bin/run knocking around for some reason, killing that removed all the backends
<deryck> Morning, all.
<mwhudson> deryck: gosh, if you're online it must be time to go to bed
<deryck> mwhudson, heh.  yup.  I have some more western lying Europeans that serve as that marker for me. :-)
<allenap> soyuz: Are you aware of a failure in buildd-slave.txt <http://paste.ubuntu.com/406449/>? About 20 minutes ago I also saw it break with a "Connection reset by peer" instead of "Connection refused", but I can't seem to replicate that now.
<wgrant> It's started failing spuriously in various ways lately :(
<allenap> wgrant: Ah, well, at least you know, and it's not something I've broken for you.
<wgrant> bigjools: I don't think it's GPL-safe to remove sources that are still referenced by binaries.
<wgrant> As for the stay of execution... I'm not sure of the purpose of that, unless it's to leave old indices working for a while.
<wgrant> GPL2 is rather handwavy about it, but GPL3 is much more specific.
<bigjools> wgrant: I wasn't suggesting to do that
<bigjools> I'm talking about getting rid of "stay of execution"
<wgrant> It is handy if you don't want to apt-get update every publishing cycle.
<bigjools> I don't think we remove sources referenced by binaries right now do we?
<bigjools> wgrant: how does it make any difference to apt-get?
<bigjools> the index changes, that's what apt-cares about
<bigjools> s/-/ /
<wgrant> bigjools: We deindex them, but they are not removed from disk.
<bigjools> yes, and how does that affect apt-get if we remove them from disk at the same time?
<wgrant> bigjools: If I have a machine that updates its indices once a day (the default) and a package gets superseded some time between updates, I will get a 404 and installations and upgrades will fail. That is ugly.
<bigjools> I'd say that is desirable behaviour
<bigjools> your index is out of date
<wgrant> We have that behaviour for deletions, when removal is obviously desired.
<wgrant> But 404s look bad. And they will happen in released versions. To lots of users.
<wgrant> The behaviour is different (the stay of execution is set at 24 hours) for PPAs; maybe the PPA spec has some rationale?
<bigjools> hmmm
<bigjools> oh fuck sake, email crash, I hate that
<bigjools> wgrant: I guess we have to suck it up then, updating indices hours before downloading the files is crack IMO but there you go :/
<wgrant> bigjools: Indices are huge.
<bigjools> unfortunately so
<wgrant> Ideally we'd have something like (or better than) pdiffs: publishing could take less than a second rather than a couple of minutes, and updating before downloading would be easy.
<StevenK> But pdiffs just sucks
<bigjools> StevenK: what's better?
<wgrant> pdiffs suck, yes.
<wgrant> It would be much better to just allow additional files which provide additional entries.
<wgrant> Much quicker to generate, and probably to apply too.
<StevenK> bigjools: When you have a large delta, just redownloading the file. If the delta is small, rsync?
<StevenK> But that is hard to determine, and most of the smarts are on the client and it means the server must provide rsync, so that sucks more.
<StevenK> I'd rather we didn't provide pdiffs, they annoy me.
<bigjools> it would be nice to shake this all up a bit
 * jml shuts down email
<wgrant> bigjools: What's going to happen with the apt-ftparchive test failures?
<wgrant> We are getting close to Lucid.
<ricotz> hello, how often is there synchronization between different translation templates of a project? So how long does it take that a change is visible in another template?
<bigjools> wgrant: we either get them fixed, or block the lucid upgrade in the DC
<bigjools> I prefer the former of course :)
<wgrant> bigjools: Or we could accept them as better behaviour, of course...
<jml> my dns is down
<bigjools> wgrant: fixing can include that definition, yes
<wgrant> True.
<bigjools> is there a bug filed?
<bigjools> I need to get it scheduled for the next cycle so we don't miss it
<wgrant> Bug #526826
<mup> Bug #526826: test_ftparchive failing on Lucid <soyuz-publish> <tech-debt> <Soyuz:Triaged> <https://launchpad.net/bugs/526826>
<bigjools> fanks
<wgrant> Although it's also affecting soyuz-upload.txt now that I've reenabled it.
<bigjools> bleh
 * jml does the modprobe dance, maybe that'll help.
<jml> it does.
<jml> bigjools, I'm going to get back on that poppy stuff now
<bigjools> \o/
<bigjools> people will send you chocolate
<wgrant> As in SFTP poppy?
<jml> wgrant, yeah.
<jml> wgrant, although this is more "get the juicy non-codehosting ssh bits out of codehosting so I can think straight about sftp poppy"
<wgrant> jml: Ah.
<noodles775> jml: did you see https://dev.launchpad.net/LEP/GeneralBuildHistories ? Later, when you want a break from poppy, can you take a look?
<jml> noodles775, no, I didn't.
<jml> noodles775, I'll check it out later.
<noodles775> jml: yeah, do some coding for a change :D
<jml> I wonder when I'll be able to land the zope.testing change.
<noodles775> wgrant: /w 5
<noodles775> woops
<jml> I really need to configure my shell to make a noise when long-running commands complete
<jml> I wonder if that's at all possible
<wgrant> jml: Some terminal emulators have a "wait for silence" option.
<wgrant> GNOME has probably removed it, though.
<wgrant> Maybe I'm think of Konsole.
<bigjools> Konsole has "monitor for silence"
<bigjools> it also has "monitor for activity"
<wgrant> Yeah, that's the one.
<bigjools> isn't KDE cool :)
<jml> terminator doesn't :)
<wgrant> Is it just me, or does make take an awful long time these days?
<bigjools> it's not just you
<bigjools> see all the junk it keeps doing over and over for each branch, when it could do it once in a common area
<wgrant> With all this mailman building and WADL generation and apidoc compilation and JS combination.
<wgrant> Some of which is done on every single build; not just in the 5-minute initial tree build.
<bigjools> and that
<maxb> It feels like all the JS stuff is what's got slower recently
<maxb> Maybe we should have a test which asserts on "make; time make" :-)
<wgrant> See, I just ran 'time make' in devel.
<wgrant> It spends a few seconds bootstrapping for no good reason.
<wgrant> Then rebuilds mailma.
<wgrant> The jsbuild/jssize/combine-css take a while.
<wgrant> But it seems that rebuilds only take around 45 seconds.
<wgrant> Ah, no apidoc in that run.
<jml> 45s for a rebuild with no changes?
<bigjools> the make target dependencies suck donkey balls
<wgrant> jml: That was with a cold cache. 20s with it hot.
<jml> wgrant, that's still quite a long time spent achieving nothing.
<wgrant> Yes.
<wgrant> Now let's see if I can get it to trigger an apidoc build...
<wgrant> Hm, maybe it does only do that on clean builds.
<bigjools> yes the dependency is set up right for that
<jml> fixing the make stuff wouldn't be too hard, I bet.
<jml> I mean, the full on right thing to do is make a list of standard bits of a developer's cycle and optimise the crap out of all of them
<jml> but just getting 'make' to do less for the 90% case and then putting some strongly-worded documentation to stop it from creeping again wouldn't be that hard.
 * jml accidentally pissed some time away apt-get upgrading when apt-get install python-apt was all that was necessary
<bigjools> jml: + [a lot]
<bigjools> can you make it happen?
<jml> bigjools, I'm not going to make the change myself, most likely.
<bigjools> jml: well that's not quite what I meant :)
<jml> bigjools, and I've already been trying really really hard to let everyone know that I want them to stop what they are doing and make development faster
<jml> bigjools, but maybe I'm trying in the wrong way, or something.
<wgrant> jml: Bonus points if you convince people to make Launchpad faster, too.
<wgrant> real	4m38.942s
<wgrant> Awesome. That is a great way to encourage branching.
<jml> wgrant, yeah, that's a separate problem that I'm tackling.
<jml> wgrant, or rather, gary_poster's team are already making LP faster.
<bigjools> jml: at our upcoming Epic, I want us to schedule time to make development better.  Stuff like this.
<wgrant> jml: I made the mistake of using Github for a while earlier today.
<bigjools> and I think you expressed that is a good idea in the past
<wgrant> it is so fast, and the code browser is part of the application and doesn't time out constantly.
<jml> bigjools, I think that would be a good idea. but I don't really see why we need to wait until July.
<bigjools> jml: we don't, but there's nothing like a concerted group effort to effect real change
<jml> bigjools, as long as it's organized to be _doing_ and not talking.
<jml> bigjools, I also advocated that we have someone work full time on making development faster. The build engineer role hasn't been as successful as I hoped.
<kfogel> bigjools: should bug 551579 be moved to launchpad-buildd or to soyuz?
<mup> Bug #551579: Unable to build a PPA package with locked down permissions <Launchpad itself:New> <https://launchpad.net/bugs/551579>
<bigjools> kfogel: let me lookl
<bigjools> jml: yes - I think our time is being squeezed too much between different things
<wgrant> kfogel: It's a bug in the package.
<wgrant> But otherwise launchpad-buildd.
<jml> bigjools, the list of developer bugs hasn't really moved much https://bugs.edge.launchpad.net/launchpad-foundations/+bugs?field.tag=build-infrastructure
<kfogel> wgrant: oh!  (but thanks, that info is helpful for next time)
<wgrant> (it's the same bug that is being discussed in #launchpad at the moment)
<wgrant> s/bug/thing/
<kfogel> wgrant: actually, it should still be moved to launchpad-buildd then (though I'll wait for bigjools to come back)
<bigjools> kfogel, wgrant: hmmm
<bigjools> so basically it wants to be run as root
<bigjools> I don't like that
<kfogel> bigjools: while I care somewhat about the big picture, I care mainly about getting it out of the "Launchpad Itself" queue (I'm CHR today) :-).
<StevenK> I think it's a package bug
<wgrant> No, it's a bug in the package that it breaks fakeroot.
<bigjools> me too
<bigjools> kfogel: it's invalid
<kfogel> bigjools: *nod*  Want me to mark it as such, and quote this IRC conversation for why?
<bigjools> kfogel: yep
<bigjools> thanks
<jml> bigjools, how does "too many different things" apply to the build engineer role?
<bigjools> jml: I suspect that role has not been given the attention it deserves
<jml> bigjools, yeah, I'd agree with that.
<bigjools> jml: what I mean is, the pressure to get the team work done has probably usurped it
<jml> bigjools, ahh, I see. Do you have any ideas as to what we can do about that?
<bigjools> jml: schedule less work? :)
<jml> bigjools, you mean, like we did already?
<bigjools> I don;t see that personally
<jml> bigjools, well, I don't know what scheduling less would look like in that case
<jml> we went from each team having a list of 10+ things to do for 3.0
<jml> to having less than ten things in total
<bigjools> jml: well I'm still guessing here - it would be better to talk to the build engineers
<henninge> ricotz: did you get an answer to your question?
<jml> bigjools, that's certainly a good thing to do :)
 * henninge is too lazy to read all the backscroll ... ;-)
 * jml goes off to perform the "is toast food" experiment
<jml> I am highly confident of a successful result.
<bigjools> jml: anyway in soyuz world we get a lot of emergencies
<bigjools> hence my feeling of lots of work still
<jml> fail. cleaner is still occupying all the kitchen
<ricotz> henninge, not yet
<henninge> ricotz: there is no synchronization between templates within a series, if you are referring to that.
<henninge> ricotz: templates of the same *name* in different series of a project will share translations.
<henninge> And that instantly, no snychroniation time.
<henninge> ricotz: can you point me to where it is you are expecting the synchronisation to happen?
<ricotz> henninge, ok, i see it now they are in sync, but the colors arent right
<ricotz> https://translations.edge.launchpad.net/docky/trunk/+pots/docky
<ricotz> https://translations.edge.launchpad.net/docky/2.0/+pots/docky
<ricotz> henninge,  hm, they are not in sync ;-), have a look at Icelandic
 * henninge looks
<henninge> ricotz: how different (or similar) are the templates, though?
<henninge> ricotz: oh, are you just looking at the numbers on the overview page?
<henninge> ricotz: they may be outdated.
<ricotz> henninge, i am relying on the graphics
 * ricotz brb
<ricotz> henninge, all strings which are included in series 2.0 are also in series trunk
 * jml makes another attempt at toast
<henninge> ricotz: comparing the Icelandic files gives this: http://paste.ubuntu.com/406575/
<henninge> ricotz: only additions in the trunk file, just like in the template. So the files are in sync.
<henninge> ricotz: looks like the PO file statistics are still messed up, sorry about that.
<henninge> ricotz: we have the daily script disabled because of performance issues. The weekly script should fix them, hopefully.
<kfogel> adeuring: in my branch's Makefile, it still says "PYTHON_VERSION=2.5", but of course Lucid has 2.6.  What's the "clean" way to update that (i.e., I assume editing it directly and having a local mod is not how this is done?)
<kfogel> adeuring: https://dev.launchpad.net/Running/Lucid doesn't seem to address this point
<adeuring> kfogel: I haven't yet upgraded.
<adeuring> last week I was sprinting, and now we are quite closed to the rollout ;)
<adeuring> so, no idea...
<kfogel> adeuring: np
<kfogel> barry: have you upgraded to lucid?  If so, please see my above question to adeuring :-).
<kfogel> Oh, maybe answer is (re)install Launchpad PPA.
<kfogel> wgrant: ping
<kfogel> gary_poster: are you developing on Lucid?  I've run into a small prob.
<gary_poster> kfogel: I have various attempts half-made, which effectively means "no".  It's conceivable I can still help.  What's up.
<kfogel> gary_poster: my /etc/apt/sources.list has "deb http://ppa.launchpad.net/chromium-daily/ppa/ubuntu lucid main", and I have done 'sudo aptitude update'.  Nevertheless, 'sudo aptitude install launchpad-dependencies' just says "no such package".  Yet https://dev.launchpad.net/Running/Lucid implies this is the way I should get Python 2.5 back on my system (Lucid has 2.6) so I can develop Launchpad.
<gary_poster> kfogel: I don't think chromium has anything to do with it.  Try
<gary_poster> deb http://ppa.launchpad.net/launchpad/ppa/ubuntu lucid main
<gary_poster> deb-src http://ppa.launchpad.net/launchpad/ppa/ubuntu lucid main
<kfogel> gary_poster: hmm, that may be the wrong ppa
<kfogel> gary_poster: thanks, yeah, just got there at the same time I think.
<gary_poster> :-)
<jml> man I've written some abysmal code in my time
<kfogel> jml: so, uh, what prompted you to say that? :-)
<jml> kfogel, lib/lp/codehosting/sshserver/accesslog.py
<kfogel> jml: (though I'll bet I can win that contest: http://red-bean.com/cvs2cl/cvs2cl.pl)
 * kfogel looks
<jml> kfogel, that URL says basically everything I need to know.
<kfogel> jml: wow, that's a lot of classes
<kfogel> :-)
<jml> oh right, you are looking at the "old" version
<kfogel> jml: (responding to accesslog.py, not cvs2cl.pl, about which your comment is very sensible)
<jml> mostly I'm looking at LoggingManager, since I've put all of the other classes in a box
<kfogel> jml: hunh.  weird that setUp() isn't __init__(); and something similar with tearDown().  Or is that your point?
<jml> kfogel, not specifically, since both setUp and tearDown mutate global state and I hate it when objects do that on construction
<kfogel> jml: ah, gotcha
<kfogel> Hmmm: "ImportError: No module named tickcount"  looks like lp may still have some lucid issues
<jml> kfogel, it's getting stuff from config that should be parametrized, it mixes up codehosting bits and generic logging bits, the oops stuff in there is pointless, the mangle_stdout parameter is unused and unnecessary
<jml> the docstring is wrong
<jml> a thing that's core to what the class does is a top-level function elsewhere in the module
<jml> and I still don't really know what the code is doing (even though I kind of do because I wrote it)
<kfogel> deryck: if you go to https://bugs.edge.launchpad.net/launchpad right now, it says: "There are currently no hot bugs filed against Launchpad itself."  While that's true, it's true because there are no bugs at all.  What has heat got to do with it?
<deryck> kfogel, actually, it is true, thank you very much. :-)
<deryck> incoming
<deryck> https://bugs.edge.launchpad.net/launchpad/+bugs?field.searchtext=&orderby=-importance&field.status%3Alist=NEW&field.status%3Alist=INCOMPLETE_WITH_RESPONSE&field.status%3Alist=INCOMPLETE_WITHOUT_RESPONSE&field.status%3Alist=INVALID&field.status%3Alist=WONTFIX&field.status%3Alist=CONFIRMED&field.status%3Alist=TRIAGED&field.status%3Alist=INPROGRESS&field.status%3Alist=FIXCOMMITTED&field.status%3Alist=FIXRELEASED&assignee_option=any&field.assignee=&f
<deryck> ield.bug_reporter=&field.bug_supervisor=&field.bug_commenter=&field.subscriber=&field.tag=&field.tags_combinator=ANY&field.has_cve.used=&field.omit_dupes.used=&field.affects_me.used=&field.has_patch.used=&field.has_branches.used=&field.has_branches=on&field.has_no_branches.used=&field.has_no_branches=on&search=Search
<deryck> well, that didn't work.
<deryck> kfogel, all the other bugs are closed.  See:  http://tinyurl.com/y8jc8rs
<kfogel> deryck: ah.  IOW, going to that page now no longer shows a list of "recently filed" bugs or whatever.  It shows hot bugs, and if there are no bugs with heat, it shows none.
<kfogel> deryck: s/recently filed/open and recently filed/
<deryck> hmmm, actually....
<kfogel> deryck: I basically assumed it's saying "hot" where it means "open", but apparently not?
<deryck> kfogel, hot does mean open.  launchpad is not the same as launchpad-project.  yes, this makes 0 sense.
<kfogel> deryck: wait a sec.  We're using "hot" to mean "open"?  This is going to cause, er, a lot of confusion :-).
<kfogel> (aside from the launchpad vs launchpad-project distinction, which already causes enough confusion all on its own)
<deryck> kfogel, no, sorry.  hot bugs list only has open bugs.  it doesn't mean open. :-)
<deryck> Here are the bugs across the project group:  https://bugs.edge.launchpad.net/launchpad-project
<kfogel> deryck: let me ask this as unambiguously as I possibly can :-)  : if there were any open bugs at all in 'launchpad', even if they had no heat whatsoever, would they show up at https://bugs.edge.launchpad.net/launchpad ?
<deryck> kfogel, if you had 10 bugs against a project with 0 heat, and they were the only 10 bugs, they would show up in the hot bugs list.
<kfogel> deryck: IOW, in order for a bug to have any heat at all, it must be open; so if that page is showing no bugs, it always means both there are no bugs open, and no bugs with heat.
<kfogel> deryck: this makes sense, BUT
<kfogel> deryck: the message at the top of that page is somewhat confusing, because it implies that there might be some open bugs, just none of them are "hot" enough to show here.
<kfogel> deryck: it takes some subtle thinking and deep knowledge of the launchpad bug tracker to understand that "no hot bugs" in this circumstance implies no open bugs either.
<deryck> kfogel, your first line after me is not correct.
<kfogel> deryck: ok, thx.  Back up and tell me where I went astray :-).
<deryck> kfogel, no hot bugs, doesn't always imply no open bugs.
<deryck> kfogel, in this case, it's true.
<kfogel> deryck: but...
<deryck> and I'm abusing the comma there, but you get the idea.
<kfogel> deryck: let me re-read two things you just said and decide if I want to accuse you of flagrant self-contradiction.  One sec.
<kfogel> :-)
<kfogel> deryck: "if you had 10 bugs against a project with 0 heat, and they were the only 10 bugs, they would show up in the hot bugs list"
<deryck> true :-)
<kfogel> vs "no hot bugs, doesn't always imply no open bugs"
<kfogel> deryck: if you have no hot bugs, but you have open bugs, then... how is it that some of those open bugs are not hot bugs?
<deryck> kfogel, those 10 bugs would be the hottest for *that* project (i.e. a project with only 10 bugs all of which have no heat)
 * deryck processes
<kfogel> deryck: first: can a bug ever be hot if it's not open?
<deryck> kfogel, yes.  until the heat get re-caculated.
<deryck> the time between fix released and the next update of heat.  a small window, but still, could happen.
<kfogel> deryck: so, aside from this small window (implementation-caused, not design-caused), a hot bug is always an open bug, right?
<kfogel> (whether it's hot for a particular project or hot in general -- point is, no context will display it as hot if it's not open)
 * deryck thinks....
<deryck> I'm not sure I'm comfortable with the "hot bug is *always* an open bug" but in practice I think this is true.
<deryck> kfogel, ^^
<deryck> kfogel, why does this matter? :-)  What problem do you want to solve here?
<kfogel> deryck: the message at the top of https://bugs.edge.launchpad.net/launchpad is confusing.  The fact that I'm having trouble explaining to you why it's confusing may be considered a secondary bug.
<kfogel> deryck: It's confusing because someone visiting that page is liable to think "Okay, so there are no _hot_ bugs here, but there might still be some open bugs that I'm just not seeing."  When in fact that's not true -- if there were any open bugs, the user would be seeing them.
<jml> wow, that rendering is off
<kfogel> deryck: Attempting to prove that latter assertion is what's got us tangled up, I think.
<jml> hmm, but only in webkit
<deryck> kfogel, it opens so rarely I don't think its worth worrying about.  something like /launchpad is the only place you would ever see this.
<deryck> s/opens/happens/
<jml> and new projects, right?
<deryck> jml, no because initially you get a "no bugs filed" message.  once you have one bug you have a hot bugs list.
<deryck> jml, the only condition that leads to this is someone getting all there bugs closed.  so maybe a very small project could see this too
<jml> deryck, or a very successful one :)
<deryck> jml, come on. :-)
<jml> deryck, we should embed a flash game! if you get to this screen, you deserve to muck around a little!
<jml> :P
<deryck> jml, now that I like! :-)
<deryck> I'm not trying to be a dick about it.  if the language is bad, it's a trivial fix.  but I'm not sure it's as complicated as we're making it.
<deryck> find another project on launchpad other than "launchpad" where this applies and I'll change it myself later today. :-)
<jml> my 2c, just change the text to say "no open bugs" rather than "no hot bugs".
<deryck> I'm not sure that would always be accurate either.
<deryck> now you're forcing me to go consult code.
<jml> anything but that :)
<kfogel> deryck: I have to admit, the fact that we don't know which phrasing is more accurate is kind of disturbing in its own right.  But no, I don't have any other example pages right now :-).
<james_w> kfogel: tickcount: install python-support from the LP ppa and reinstall python-tickcount, cf abentley's email to launchpad-dev a couple of days ago.
<jml> ok, now I'm angry at Python
<kfogel> james_w: ah, thanks!  missed that one
<kfogel> jml: why, because it's not Lisp?
<jml> kfogel, because it has bad errors
<deryck> kfogel, jml -- I think it's fine to make it "no open bugs." it's just a searchTasks for open bugs ordered by heat desc.  but it's not an explicit count, which is why it makes me nervous to change the language.
<deryck> it's hand wavey to say "no open bugs" but it probably is more accurate most times.
<jml> hmm. error understood, now I can go back to hating our test suite
<deryck> kfogel, there's a hot bug on /launchpad now :-)
<kfogel> deryck: not for long...
<deryck> kfogel, I think that means it's better to have "no open bugs" :-)
<jml> (there's an smptd server that runs while the puller tests are running, just cos)
<deryck> oh crap, need to run
<kfogel> deryck: Well, I agree.  I mean, the message disappeared when a bug -- any bug at all -- showed up, and the message's disappearance was independent of the bug's heat.  And now that I've made the bug go away, the message is back :-).  So it's clearly about openness, not heat.
<kfogel> deryck: bye!
<kfogel> missed him
<gary_poster> sinzui: hey.  should I move https://bugs.edge.launchpad.net/launchpad-foundations/+bug/488399 to 10.04?
 * gary_poster expected mup
<gary_poster> "automate yui unit tests"
<sinzui> yes
<mup> Bug #488399: automate yui unit tests <build-infrastructure> <javascript> <Launchpad Foundations:In Progress by sinzui> <https://launchpad.net/bugs/488399>
<kfogel> jml: request for Captain Obvious: what am I missing here in my failure to get this test to run?  http://paste.ubuntu.com/406652/
<jml> kfogel, zope does exact string matching on the path
<kfogel> jml: so I thought I tried some exact paths
<kfogel> jml: does it need to be *absolute*?
<jml> kfogel, more accurately, the exact string of the path used to open the doctest is stored in the test object and zope.testing does regex matching on that
<jml> kfogel, lemme try to get this
<kfogel> jml: ./bin/test -vvct lib/lp/bugs/stories/bugattachments/10-add-bug-attachment.txt  fails too
<kfogel> jml: s/fails/succeeds by running 0 tests/
<jml> try "-cvvt stories/.\*/bugattachements"
<jml> actually, just "-cvvt stories/bugattachments"
<jml> kfogel, you'll notice that the name of the tests are like "lib/lp/bugs/tests/../stories/bugattachments/xx-attachments-to-bug-report.txt"
<kfogel> jml: yeah, I assumed those were run either all first or all last
<jml> kfogel, no, look more closely, "tests/../stories"
<jml> kfogel, how is the regex "lib/lp/bugs/stories/bugattachments/10-add-bug-attachment.txt" ever going to match the string "lib/lp/bugs/tests/../stories/bugattachments/10-add-bug-attachment.txt"?
<kfogel> jml: huh, wait, where are you getting the path with the ".." form from?
<jml> kfogel, it's what zope.testing prints when I manage to find the test successfully by running "./bin/test -cvvt stories/bugattachments"
<kfogel> jml: yes, that ran it for me too.  But my output doesn't have any "../" in it
<jml> kfogel, can you paste the command and output please?
<kfogel> jml: yes. And actually I should say, it didn't run the test, but it got an error that looked like it happened because it was sincerely trying to run the test.
<kfogel> jml: http://paste.ubuntu.com/406658/
<jml> kfogel, oh right, it didn't even get close to running the test
<kfogel> jml: so "no such file or directory" -- but every other cmdline just seems to "succeed" with 0 tests run
<kfogel> i.e., why don't I always get that no-such-file-or-dir error?
<jml> kfogel, that's a different problem
<kfogel> OH
 * kfogel carefully duplicates his brain, seeing he's going to need a spare
<jml> kfogel, when you run ./bin/test with "-t" and it finds no tests that match the regex, it never tries to run any of the layers
<jml> kfogel, when you run it and it *does* find tests, it runs the layers for the tests that it found
<kfogel> jml: *nod*
<kfogel> jml: so it found the test, and then died trying to set up layers to run it?
<jml> kfogel, correct
<kfogel> jml: hey, sounds to me like it DID get closer to running the test :-).  You know, closer than "infinitely far away".
<jml> kfogel, specifically, you have a problem with memcached layer that I have no clue about.
<kfogel> jml: ok, thanks
<jml> kfogel, anyway, I guess now that I've got one patch into zope.testing, I could try to fix the abomination of doctest naming
<jml> although that might be a bug in Python
<jml> in which case I'll write a patch and try to convince Michael Foord that it's uncontroversial
<jml> and then we'll be able to make use of it in three years time when we run LP on Python 2.7
<kfogel> jml: ooooouch
<jml> sweet
<jml> I think I've got an SSH server in lp.services.sshserver that does all of the neat logging, timeout, oops reporting and authentication-against-LP that codehosting does, but without actually knowing anything about hosting code
<jml> it's 4210 lines of diff though :(
<jml> g'night all
<ricotz> henninge, thank you for your help earlier today
<henninge> ricotz: np, glad I could help
<james_w> gary_poster: for these 502 errors is there any use you want to make of us logging them?
<gary_poster> james_w: oh, this is in regard to my comment about kees' patch, right?  Or is in regard to my general desire to get logs of the 502 errors so we can see if we can determine root causes?
<james_w> gary_poster: in that bug you said you would like to know when it happens. In what form would you like that info to come?
<gary_poster> james_w: my current feeling is that attachments to the bug would be good.  I was expecting logs of the sort that leonard requested earlier in the bug report.
<james_w> for instance I got a 502 at 2010-03-30 19:10:20.474341 UTC from production
<james_w> ok, gathering logs all the time is excessive for this process, I'll set something dedicated up if you like
<gary_poster> james_w: thank you.  if you are going to be organized about it, then let us be as well.  I'll get details of what we think might help to you (via the bug comments).  I intend to get this done by tomorrow.
<james_w> thanks
<thumper> morning
<didrocks> mwhudson: hey o/ I was just wondering when do you think the ssh key code will be exposed in the api in https://edge.launchpad.net/+apidoc/devel.html (not sure about the publishing process)
<mwhudson> didrocks: i think your branch landed on db-devel, so after the rollout in a couple of days
<didrocks> mwhudson: ok, I was expecting it to be published automatically in devel (like being a trunk). So no worry :) thanks!
 * deryck broadcasts from daughter's dance studio wifi
<maxb> For some reason launchpadlib 1.5.6 is missing its bzr tag. Please could someone who is a member of ~lazr-developers add it:
<maxb> bzr tag -d lp:launchpadlib -r revid:leonard.richardson@canonical.com-20100304202437-68ykm4fpbio8jdid 1.5.6
<maxb> I've checked the branch against the tarball to verify the revision
<mwhudson> no what's going on
#launchpad-dev 2010-03-31
<wgrant> How is the discussion about community committers/reviewers going?
<mwhudson> it's petered out
<thumper> mwhudson: had lunch yet?
<mwhudson> thumper: no
<thumper> you should grab something to eat, then we can talk about reviews :)
<mwhudson> thumper: just finished with kiko
<thumper> :)
<thumper> ok
<mwhudson> thumper: so yeah, i need to eat, and then we should talk
 * mwhudson lunches
<mwhudson> huh, x has gone odd and i need to restart anyway so...
<mwhudson> argh
<mwhudson> i've managed to uninstall gnome-panel somehow :/
<thumper> :(
<thumper> mwhudson: ping
<mwhudson> thumper: pong
<mwhudson> (sorry, no notifications with no panel!)
<thumper> heh
<thumper> mwhudson: when do you want to do a call?
<mwhudson> thumper: now is good
<wgrant> I pushed a couple of new revs to a branch 12 minutes ago... still awaiting scanning.
<spm> looking....
<wgrant> It's scanned now.
<wgrant> Just took ages.
<wgrant> Not quite 2006-ages, but getting close.
<spm> hrm...
<spm> :-)
<wgrant> Part of that could be slave lag too, I guess.
<spm> slave lag? as in DB? shouldn't be that bad!
<wgrant> I'd hope not.
<lifeless> thumper: mwhudson: have you guys seen beanstalkd ?
<mwhudson> lifeless: no
<StevenK> Bah, registering an MP just gave me a timeout error
<mwhudson> StevenK: the branch was probably still being scanned
<lifeless> mwhudson: apparently designed 'to make websites faster by doing slow tasks asynchronously'
<lifeless> http://flapjack-project.com/ references it
<mwhudson> StevenK: wait a minute or so and resubmit
<mwhudson> lifeless: on the face of it, that sentence doesn't make sense :)
<mwhudson> but i guess i know what they're driving at
<StevenK> mwhudson: Aye, will do
<lifeless> i paraphrased an equally not-quite-sane quote
<lifeless> anyhow, might be worth a gander
<mwhudson> lifeless: it seems a bit like gearman, maybe?
<lifeless> maybe, more lightweight and message bussy I think
<StevenK> mwhudson: Worked, thanks
<mwhudson> StevenK: np, it's just another reason to kill the branchrevision table...
<mwhudson> lifeless: it does have a "simple enough to possibly work" sort of feel to it
<lifeless> auxesis told me about it a year or so back
<lifeless> and I blanked on it when the whole reinvent wheels was going on
<lifeless> just got reminded, so - saying;)
<adeuring> good morning
<mrevell> Morning
<thumper> lifeless: I don't understand what beanstalked is
<wgrant> noodles775: So, apart from the state thing, how does the proposal look?
<noodles775> wgrant: It looks great so far... I usually find some things come out in the wash, hence putting together the schema change. But it allows everything we discussed earlier (common shared data, shared data for package builds etc.)
<noodles775> wgrant: I'll create a WIP mp when it's ready and add you to it to get your feedback/thoughts.
<wgrant> noodles775: Great, thanks.
* mthaddon changed the topic of #launchpad-dev to: Launchpad will be down/in read-only from 11:00 UTC until 13:00 UTC for a code update | Launchpad Development Channel | Week 4 of 10.03 | PQM is in RC mode | Release manager: deryck | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
<wgrant> noodles775: http://people.ubuntu.com/~wgrant/launchpad/buildfarm/new-build-model-again.png <- now with fewer missing attributes, more compliant names, and proper lp.* division
<noodles775> wgrant: thanks.
<wgrant> The layout seems quite a bit busier, but it is hopefully more accurate.
<wgrant> Hm, early release?
<noodles775> wgrant: how do you feel about s/job/build_farm_job so we don't have confusion over the job table?
<wgrant> noodles775: Good point.
<noodles775> wgrant: also, am I right that there's a bunch of stuff missing atm? (like IBuild.package_upload -> PackageBuild?, IBuild.can_be_retried/rescored -> BuildFarmJob?)
<noodles775> I'll do an initial change, and then check for missing fields in the new schema, probably easier.
<noodles775> erm, forget the last two... properties.
<noodles775> actually, forget I spoke at all :)
<wgrant> Yeah, all properties.
<wgrant> They should probably be included, but that would get very big.
<noodles775> Yeah, we can include them when updating the iface/model code.
<wgrant> Yay, Sphinx.
 * wgrant loves Sphinx.
<wgrant> Evening jtv!
<jtv> hi wgrant!  Sorry to have been ignoring you... half a world of travel plus tonsilitis.
<jtv> But I suggest you visit Thailand someday and we'll fix that.  I'll personally guarantee that you have a good time.  :)
<wgrant> Ew, tonsilitis.
<jtv> (I'm gushing a bit because of the "oh btw I fixed the bugs" thing :-)
 * wgrant is still not quite over the Wellington whooping cough.
<jtv> whoa
<wgrant> Yeah, it likes to stick around for a couple of months.
<jtv> yes, it sort of contradicted for me the whole theory that January is midsummer on that hemisphere...
<wgrant> Heh.
<jtv> So, what were the remaining bugs?
<wgrant> So, yeah, template builds and uploads work with a couple of my branches. 'tis encouraging.
 * wgrant digs up the branches.
<wgrant> https://code.edge.launchpad.net/~wgrant/launchpad/bug-549340-fix-buildqueue-destruction <- this one has been rejected, but should be resolved by noodles' model redesign.
<wgrant> And the last three revs on https://code.edge.launchpad.net/~wgrant/launchpad/get-translations-jobs-working
 * jtv looks
<jtv1> BTW thanks for unifying the slaveStatus... for our use case it was painfully obvious that something like that was needed, but I just didn't have enough overview to be sure about the other use-cases.
<jtv1> okay I shouldn't have touched that cable...
<jtv> bit awkward here with a main machine that won't boot and a router that throws up packet storms... means I had to plug the iffy wall cable straight into my laptop.
<wgrant> Hah.
<wgrant> Yeah, that slaveStatus branch should be landed right after PQM opens.
<wgrant> Then I'll get the others in soon after, although things won't work completely until the modle rework.
<wgrant> So we might actually have three working build farm jobs in 10.04... maybe.
<jtv> We're certainly hoping we can manage that.
<jtv> By the model rework you mean the simplification with build attempts etc. that you sketched out two weeks or so ago?
<wgrant> No attempts any more. I believe noodles is currently attempting to realise something like http://people.ubuntu.com/~wgrant/launchpad/buildfarm/new-build-model-again.png
<noodles775> jtv: yep, I'll have a WIP MP with a very early schema change in a few mins.
<jtv> cool!
<jtv> noodles775: meanwhile, we've been talking about replacing getName/verifySlaveBuildID with a single generateSlaveCookie method that produces a hashed cookie.  Verification consists of re-generating it and seeing that it matches what the slave says.
<wgrant> jtv: I've another branch which rips the slave build ID out of everywhere except rescueBuilderIfLost, so you will be able to completely replace it without breaking anything else.
<noodles775> jtv: yeah I saw. Is that related to wgrant's branch that removes the builder dependency on ...
<jtv> it gets better and better :)
<noodles775> wgrant, jtv, bigjools: If you want to take an early look, grab the branch from https://code.edge.launchpad.net/~michael.nelson/launchpad/db-build-generalisation-db-changes/+merge/22527 before LP goes down in 10mins :)
<bigjools> good call :)
 * wgrant grabs.
<jtv> noodles775: thanks
<noodles775> There are a lot of notes/todos in there, but it'll give you a chance to let me know if it's heading in the right direction.
<wgrant> I think they might've turned off the cron jobs off already.
<noodles775> OK, I'll paste a diff.
<jtv> that'd make sense, let things quiet down.
<wgrant> There's no mirrored copy of the branch, nor diff. :(
<bigjools> gah
<bigjools> branching failed
<noodles775> wgrant, jtv, bigjools: http://pastebin.ubuntu.com/406980/
<wgrant> noodles775: Thanks.
<bigjools> ta
<wgrant> I was thinking that it was rather simple. But of course there's no non-binary job data, so it can be...
<wgrant> Convenient.
<noodles775> wgrant: Yeah, if we don't have to populate with any future SPRBuild data, it will be convenient.
<wgrant> noodles775: I'd like to make BFJ.virtualized NOT NULL while we're at it.
<noodles775> Noted.
<jtv> noodles775: what's the stuff with the sequence for?
<wgrant> Virtualisation is very much something that you can't care about here.
<wgrant> Er, can't *not* care about.
<noodles775> jtv: It's just because I created the tables using CREATE TABLE blah AS (as I think it's a bit more readable, enabling the table creation based on existing data/types etc.)
<noodles775> But the only way I could see to have a primary key as the first column using this method was to explicitly create the sequence.
<jtv> noodles775: that's true... the other way to do it would be a conventional CREATE TABLE followed by INSERT INTO
<noodles775> (adding a primary key afterwards works fine, but it ends at the end of the table).
<noodles775> jtv: Yeah, I guess I thought the advantage of inheriting all the column defs from the existing tables was nice... but maybe it's not worth it?
<noodles775> It's really just the value+type.
<jtv> noodles775: it's not going to be all that important (barring mistakes), but personally I prefer to have all the constraints and indexes in there from the start.  Also makes it easier to catch little things like fixed-size columns following text columns.
<noodles775> jtv: yep. OK, I'll switch it over.
<noodles775> jtv1: yep. OK, I'll switch it over.
<jtv1> cool, thanks for accommodating my OCD.  :)
<jtv> Can anyone (esp. Registry I guess) shed some light on this shortlist exception?  OOPS-1550O2497
<wgrant> noodles775: I wondered about build_warnings. I guess we should just drop it.
<noodles775> wgrant: yeah, i can only see it in the schema.
<wgrant> noodles775: Are you going to leave buildduration in the model as an abstraction? We'll hopefully be able to fill it in properly soon.
<noodles775> wgrant: as a property on the model, sure, but I'm not sure why we'd want to fill it in (ie. for it to be a db column?).
<noodles775> Or that's what you meant?
<wgrant> noodles775: It has been thought for a while that we should eventually retrieve the real duration from the slave.
<noodles775> wgrant: ah I see. Well, I'm guessing rather than "We'll be using it real soon", we'll just have to add it later then ;)
<wgrant> noodles775: Yep, just leave it as a property for now.
<sinzui> I just saw an interesting test failure in ec2. object-milestones failed because the test data used 2010-04-01 and 2010-04-02 and the test is verifying those dates...but the UI shows the dates as "in 22 hours" as the moment counts down.
<sinzui> jpds: you can ignore the error in the test, there is nothing for you to do.
<jpds> sinzui: I figured so. :)
<wgrant> Hah.
<wgrant> We are in the future now.
<sinzui> We were promised jet packs in the future, where is mine?
* mthaddon changed the topic of #launchpad-dev to: Launchpad will be down/in read-only from 11:00 UTC until 14:00 UTC for a code update | Launchpad Development Channel | Week 4 of 10.03 | PQM is in RC mode | Release manager: deryck | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
<deryck> another hour on release time, for those who missed the channel change or the chatter on #launchpad
<kfogel> adeuring: did you have a chance to review my bug #538219 branch before LP went read-only?  (I'm having trouble reaching the branch right now, or I'd just look.)
<adeuring> kfogel: sorry -- I missed completely your review request (
<kfogel> adeuring: np, just when you have a chance (and lp is back) take a look
<adeuring> kfogel: to avoid such a mess in the future, please ping me on IRC and I'll do the review ASP
<kfogel> adeuring: I think it's a pretty small change.  I didn't refactor some duplicated in the tests because it didn't seem worth it yet.
<kfogel> adeuring: oh, you were definitely asleep when I made the MP :-)
<adeuring> kfogel: well, oh, right, that was around midnight my time... But anyway, somehow I do not notice the review requests in my inbox properly...
<kfogel> adeuring: it's really no mess, don't worry.  Considering that the next release is a month away, it's okay if there's a day delay in reviewing this change!  (I just wish I'd finished it sooner.)
<noodles775> wgrant: around? I keep finding my self saying, BuildFarmJob, PackageJob, BinaryPackageJob, insteada of PackageBuild, BinaryPackageBuild.
<noodles775> I'm keen for the Build ending, but wondering if we shoud update BuildFarmJob to SomethingFarmBuild?
<leonardr> flacoste, before we can put launchpadlib 1.5.8 into lucid (that's the one that uses web service 1.0 by default)
<leonardr> we need to port all the packages that use launchpadlib to api version 1.0
<leonardr> jamesw identified 9 such packages
<leonardr> apport
<leonardr> bughugger
<leonardr> bzr-builddeb
<leonardr> bzr-pqm
<leonardr> groundcontrol
<leonardr> lptools
<leonardr> quickly
<leonardr> ubuntu-dev-tools
<leonardr> ubuntu-qa-tools
<leonardr> pitti ported apport yesterday
<leonardr> some of these probably don't need to be ported, but we need to at least check
<flacoste> oh shit
<flacoste> that's a lot of work :-/
<flacoste> when the beta2 is coming out in a week
<leonardr> yeah
<leonardr> i'm happy to help but we need to coordinate with the owners of those packages
<leonardr> the porting is easy (assuming these apps have unit tests)
<leonardr> the coordination is a time sink
<james_w> we don't really have owners of packages in Ubuntu
<james_w> I can help you make the changes easily
<leonardr> ok, that makes things look better
<leonardr> flacoste, gary, should i go for it? (gary: this would mean putting off the launchpadlib pagetest thing)
<leonardr> gary: or i can finish the pagetest branch first
<flacoste> leonardr, gary_poster: since that item has a Due date, i think it makes sense to push that forward first
<gary_poster> leonardr: Yeah, I'm +1 on working on that
<flacoste> gary_poster, leonardr: especially if it means we don't need to support the /beta webservice for 5 years!
<leonardr> all right
<gary_poster> right :-)
<leonardr> james_w, do you have time?
<leonardr> if not i can send you recommendations via email
<james_w> right now?
<leonardr> james_w: i'm going to start right now, but like i said, if you're not free, you can come in late
<james_w> I'll be with you in 20
<leonardr> all right
<james_w> leonardr: ok, I'll take bzr-builddeb obviously, and ubuntu-qa-tools
<leonardr> james_w: there don't seem to be any changes necessary to ubuntu-qa-tools, but look it over to double check
<james_w> oh, ok
<james_w> should we make it explicit anyway?
* mthaddon changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.03 | PQM is in RC mode | Release manager: deryck | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
<james_w> leonardr: are we changing the constructor call to make it explicit, or just allowing them to pick up the default?
<leonardr> james_w: allowing them to pick up the default
<james_w> ok
<leonardr> i'm concerned about 1) uses of named operations not present in 1.0
<leonardr> 2) explicit references to "beta"
<james_w> I agree that ubuntu-qa-tools is fine, bughugger is done
<leonardr> because of 2) i just realized that ubuntu-qa-tools is not fine
<james_w> ok
<james_w> bzr-builddeb is fine
<leonardr> all right
<james_w> bzr-pqm is broken
<leonardr> there are several calls to launchpad.load() which include a "beta" url
<james_w> yeah
<james_w> what should they use there?
<james_w> lp.load(SERVICE_ROOT + "bugs/" ...
<james_w> or does it need the version in there too?
<leonardr> the version does need to be in there
<leonardr> let me try to find a better solution
<james_w> urg, we would have to use lp._root_uri
<leonardr> yeah
<leonardr> but, that's better than having a hard-coded version
<james_w> yeah
<james_w> would be great if lp.load could interpret a relative URL as being relative to the service root
<gary_poster> deryck, did you decide on a "consider opening up PQM" time?
<leonardr> james_w: yeah, that's bug 524775
<mup> Bug #524775: Launchpad.load doesn't like relative urls <launchpadlib :New> <https://launchpad.net/bugs/524775>
<deryck> gary_poster, thinking about that now actually.
<gary_poster> cool
<leonardr> james_w: i could fix that easily, but releasing a 1.5.7.1 launchpadlib would be a bit of a pain. do you think it's worth it?
<james_w> we can just use lp._root_uri +
<leonardr> yeah
<gary_poster> how much of a pain leonardr?  and is this the only app that does this?
<leonardr> gary: no, it's all over
<gary_poster> ...
<leonardr> lifeless even put a special hack into lptools/launchpad.py to fix bug 524775
<mup> Bug #524775: Launchpad.load doesn't like relative urls <Launchpad Foundations:Triaged> <launchpadlib :Triaged> <https://launchpad.net/bugs/524775>
<gary_poster> james_w, leonardr, how tight is time?
<gary_poster> and how long will this take?
<leonardr> gary: james_w and i agree it's not worth it right now
<leonardr> we'll just use _root_uri
<gary_poster> ...don't like it, but ok.
<james_w> we can take a list of the ones we do it to and followup with a correct fix once it is fixed in lplib
<leonardr> yeah
<gary_poster> ok
<gary_poster> might as well dump that info in the bug report, once you have it
<leonardr> james_w: my proposed changes to ubuntu-qa-tools
<leonardr> http://paste.ubuntu.com/407091/
<leonardr> gary: so far all the changes are "now it has to support multiple versions" changes, not 1.0-specific changes
<james_w> my bughugger change: https://code.edge.launchpad.net/~james-w/bughugger/fix-api-usage/+merge/22536
<gary_poster> leonardr: good, and not surprising
<james_w> leonardr: looks good to me
<leonardr> apport was the only app that actually used those features
<leonardr> james_w: i'm pushing a branch
<james_w> bzr-pqm: https://code.edge.launchpad.net/~james-w/bzr-pqm/fix-service-roots/+merge/22540
<leonardr> https://code.edge.launchpad.net/~leonardr/ubuntu-qa-tools/fix-api-usage
<leonardr> james_w, taking a look at yours
<leonardr> james_w: i have a bad feeling about the get_bazaar_host calls in bzr-pqm
<leonardr> investigating
<leonardr> eh, i guess theres' tests for it, they'l'l fail if something's wrong
<leonardr> james_w: i'm pretty sure quickly doesn't need any changes
<james_w> good
<leonardr> lptools does need changes, working on them now
<leonardr> https://code.edge.launchpad.net/~leonardr/lptools/multiversion-support/+merge/22544
 * leonardr does not look forward to searching for "1.0" in 5 years when we're trying to deprecate 1.0
<leonardr> perhaps we should give the api versions ubuntu release-like names that do not conflict with ubuntu release names
<james_w> lptools looks good to me
<leonardr> i don't see any problems with groundcontrol, i'm confirming with doctormo
<leonardr> james_w: the only one i don't have information for is bzr_builddeb. what does that look like?
<james_w> I'll fix it
<james_w> it's got an lp.load
<leonardr> all right
<leonardr> confirmed that the lp.load is the only problem
<abentley> bigjools, re: http://pastebin.ubuntu.com/407124/ can self.can_copy_to_context_ppa ever evaluate false in this loop?
<bigjools> looking
<abentley> bigjools, (and if so, would you really want to present it as a potential copy target to the user?)
<bigjools> abentley: it could do in some weird situations
<abentley> bigjools, so in those situations, it would make sense as a copy target?
<bigjools> no, it doesn't
<bigjools> that's what the loop's doing isn't it?
<abentley> bigjools, the loop says it will be included as a possible target if the user can't upload to it.
<abentley> If it evaluates false, the and expression is false, and so it will be included in the list of terms.
<jtv1> sinzui: thanks for the help with the shortlist issue.  I now remember it was adeuring who came across before, and I reviewed the branch myself.  Is anyone fixing it or shall I?
<bigjools> abentley: ah no it's excluding the context  PPA
<jtv1> abentley: hi!  Did you claim a review for me and not get around to finishing it?
<abentley> jtv, oh, maybe.
<bigjools> abentley: it's just doing it in a very odd way
<jtv> abentley: hope "maybe" is not your vote on the MP.  ;-)
<abentley> jtv, yes, I forgot about that.  Sorry.  I'll do it next.
<jtv> abentley: cool, thanks.  I wasn't going to force you, but would be grateful.
<abentley> bigjools, if the context ppa is included in self.ppas_for_user, and self.can_copy_to_context_ppa evaluates False, the context PPA will be included in the list of target PPAs.
<bigjools> abentley: yeah that's what I mean by that code being weird!
<abentley> bigjools, because the "and" expression will evaluate False, which means we won't continue.
<bigjools> indeed
<bigjools> abentley: I've no idea why the first condition is there
<abentley> bigjools, my guess is that the context PPA should be excluded all the time.  But "required" should only be set False if both conditions are true.
<bigjools> abentley: the archive in question is not always a PPA
<bigjools> it can be a copy archive
<bigjools> which uses the PPA views/templates
<bigjools> I think that's why it's there
<abentley> bigjools, The context may not be a PPA or the "ppa" variable may not be a PPA?
<bigjools> abentley: potentially both but the context usually
<bigjools> the ppa variable is 99% a ppa
<bigjools> unless there's a bug somewhere
<abentley> bigjools, that suggests that 1% of the time, getPPAsForUser will return archives that are not PPAs.  Is that what you mean?
<abentley> bigjools, and if so, how to I force it to give me only ppas?
<bigjools> abentley: it can happen if the user owns a non-PPA archive.  This is very rare.
<bigjools> abentley: actually let me check the code again
<abentley> bigjools, but in both selects, it's requiring the ArchivePurpose==PPA.
<bigjools> abentley: yes it does, my mistake, I was making a stupid assumption
<abentley> bigjools, so let's rewind.  I said "bigjools, my guess is that the context PPA should be excluded all the time.  But "required" should only be set False if both conditions are true."
<abentley> bigjools, do you agree with that now?
 * jml afk
<bigjools> abentley: I agree
<abentley> bigjools, cool.  I'll tweak that as I refactor it, then.
<bigjools> abentley: thanks - are you making a vocabulary out of it?
<abentley> bigjools, I'll at least extract a function that turns a list of archives into a SimpleVocabulary.
<bigjools> cool
<abentley> bigjools, haven't made vocabularies before, so it's a bit new.
<abentley> Because my context won't be an archive, I don't think the copy packages page can share an archive with the request-a-build page.
<abentley> Because my context won't be an archive, I don't think the copy packages page can share *a vocabulary* with the request-a-build page.
<bigjools> the vocab is dependent on the user
<abentley> bigjools, but the vocab on the copy packages page is also dependent on the context ppa, and this won't be true for my case.
<bigjools> that is true
<bigjools> but it's a bit of a hack
<abentley> bigjools, I haven't seen the UI, but it seems like another option would be to include the context PPA in the vocabulary, and select it using the "initial_values".
<sinzui_> join #launchpad
<jml> g'night all
<james_w> leonardr: oops _root_uri isn't going to work: https://code.edge.launchpad.net/~leonardr/ubuntu-qa-tools/fix-api-usage/+merge/22541
<leonardr> james_w: we can convert that uri to a string easily enough
<james_w> yep
<sinzui> gary_poster, ping
<gary_poster> sinzui: on call pong you later?
<sinzui> gary_poster, sure. barry and I want to know if there is a python 2.6 list of issues
<gary_poster> no; that's my next task tho
<barry> gary_poster, sinzui that's the #1 thing holding up my work on integrating lp w/ mm3 (which requires py2.6)
<gary_poster> barry: it will be tackled this cycle
<barry> gary_poster: fab
<leonardr> james_w: any progress on bzr-builddeb? that's the only one i don't know about
<james_w> leonardr: nope, I've been working on other things.
<james_w> it's an easy job though
<leonardr> james_w: i'll take a look if you want
<james_w> no, it's fine
<leonardr> all right, just let me know when you've got something so i can send another email
<kfogel> sinzui: not sure if this is something you can answer, but the definition of lib/lp/bugs/browser/bugcomment.py:BugComment.can_be_shown() puzzles me and I'd like to ask someone about it.  I expected it to contain code determining if getUtility(ILaunchBag).user can view this comment or not -- but instead, it's just about bugwatches.
<kfogel> sinzui: Is there some other method that does what I expected can_be_shown() to do?  I don't see any... (this is all for bug #546943, by the way).
<mup> Bug #546943: Hidden bug comments still have a direct link/page that is visible <easy> <spam> <Launchpad Bugs:In Progress by kfogel> <https://launchpad.net/bugs/546943>
<sinzui> kfogel, the rule may be vestigial. we hid imported bug comments for many months
<kfogel> sinzui: I guess I'm looking for the condition that says "this comment is hidden", in the way meant by bug #546943.  I thought this method was it, but it looks like not.
<mup> Bug #546943: Hidden bug comments still have a direct link/page that is visible <easy> <spam> <Launchpad Bugs:In Progress by kfogel> <https://launchpad.net/bugs/546943>
<kfogel> oh, maybe self.visible
<sinzui> ah
<sinzui> kfogel, .hidden means private. that is how we remove spam
<kfogel> sinzui: i.e., it's nothing to do with authn perms that might differ by user -- "hidden" in this context means "hidden from everyone"
<sinzui> kfogel, admins need to see them, maybe chr too
<kfogel> ?
<kfogel> sinzui: hmm, this prop might be on IMessage instead
 * kfogel pokes around more
<sinzui> can_be_shown is from comment imports. .visible is from spam control. In this case losas need to see them so that they can find what needs to be reenabled
<sinzui> kfogel, is the comment view guarded by a security checker? one that asks of the comment is .hidden and decides if the user can see it?
<kfogel> sinzui: yes -- that was how I got to can_be_shown() in the first place
 * sinzui thinks the comment and the link to it should have the same security rule.
<kfogel> sinzui: i.e., lib/lp/bugs/templates/bugtask-index.pt
<sinzui> okay. I think I see
<kfogel> sinzui: although bugcomment-index.pt also has that rule, ...
<kfogel> but I think that may be a red herring
<sinzui> We need up update the security rule on the view to do something like return user.is_admin or context.hidden
<sinzui> we want the link to reuse the rule
<kfogel> sinzui: thaht would be the clean fix, absolutely
<kfogel> sinzui: I started my journey by trying to find out what the rule ought to be :-)
<kfogel> sinzui:  which is how I ended up here in the Emerald City
<mwhudson> morning
<kfogel> mwhudson: morning
<kfogel> sinzui: I think I've got it now; thanks for the help.
<sinzui> fab
<kfogel> sinzui: hmm.  I'm trying to test this on launchpad.dev now.  Do you have any idea *how* the admins set a comment as hidden?
<kfogel> I don't see any UI for it.
<sinzui> kfogel, there is no ui :(
<kfogel> sinzui: db tweak?
<sinzui> they use an api script
<kfogel> aaaah
<kfogel> sinzui: ok, on it, thx
<sinzui> I was thinking that we could provide an form for anything that provides IMessage that allows the admin to edit or hide the message.
<sinzui> one day we will support this: /bugs/1/message/2/+edit
<mup> Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Invalid> <Computer Science Ubuntu:Invalid by compscibuntu-bugs> <EasyPeasy Overview:Invalid by ramvi> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <OpenOffice:Invalid by lh-maviya> <Tabuntu:Invalid by tinarussell> <Tivion:Invalid by shakaran> <Ubuntu:In Progress by sabdfl> <ubuntu-express (Ubuntu):Invalid by jahyire2006> <Ubuntu Jaunty:In Progress> <ubuntu-expre
<kfogel> sinzui: mup bit you :-)
<sinzui> mup is a luser
<kfogel> mthaddon: ping
<kfogel> mthaddon: Do you have the script that admins use to set a bug comment as "not visible"?  I'd like to see that script, if you do.
<sinzui> I want re re-enable bug expiration and expire it
<kfogel> losas, see above question I addressed to mthaddon
<kfogel> sinzui: Victory!  I think: I used psql to set the 'visible' boolean column to False for a comment I just added; now when I'm logged in as Foo Bar (i.e., launchpad.dev admin user), I still see that comment on the bug's page, but it's highlighted yellow now.  Do you know if this is the expected behavior?
<kfogel> ah, yes, and as anon user, I cannot see the comment.
<sinzui> I do not know about a highlight rule
<sinzui> I think gmb implemented the hid bug feature
<kfogel> sinzui: that helps.  I'm pretty sure what I'm seeing is the intended behavior, though (I mean, modulo the bug I'm trying to fix)
<wgrant> Hmm. Does anyone know if the PPA download counts script is running yet?
<thumper> I've started getting: /usr/lib/pymodules/python2.6/lazr/uri/__init__.py:19: UserWarning: Module launchpadlib was already imported from /usr/lib/pymodules/python2.6/launchpadlib/__init__.py, but /usr/lib/pymodules/python2.6 is being added to sys.path
<thumper> when I use bzr
<thumper> anyone seen this?
<thumper> or know how to fix it?
<mwhudson> i see it
<james_w> leonardr: lptools, ubuntu-qa-tools, bughugger and bzr-pqm all fixed in Ubuntu, bzr-builddeb will uploaded tomorrow
<leonardr> great, i'll send an update
<kfogel> sinzui: wouldn't mind a little Captain Obvious, if you have a sec: http://paste.ubuntu.com/407277/
<kfogel> I'm pretty sure I've got some kind of wrong-type-of-object issue going on, but so far haven't been able to determine how to get to the object I need.
<wgrant> kfogel: That's a fake BugComment.
<wgrant> The real object is BugMessage. BugComment is a browser-specific wrapper.
<wgrant> And it probably doesn't expose .visible yet.
<wgrant> Hm, but it looks like it is. Odd.
<kfogel> wgrant: looks like it is what?
<wgrant> Oh, it's just not in IBugComment.
<wgrant> kfogel: .visible is on BugComment, but not IBugComment.
<kfogel> wgrant: you'd think I'd have all these laysers straight by now, but noooooooo.
<wgrant> So it's accessible internally, but not externally.
<kfogel> ah
<kfogel> hmmm
<kfogel> I wonder if exposing it is the right approach.
<kfogel> What I really want is for the comment to 404 if visited via direct URL (e.g., bugs/54/comments/37)
<wgrant> Right, I was about to ask why you were doing it this way.
<kfogel> right now, the effect that code would have (if it were working) would be to show a bug comment page, but with an empty space where the comment text should be
<kfogel> wgrant: I'm not persuaded I should be doing it this way.
<kfogel> wgrant: do you know a better way, a way to make the page pretend not to exist at all?
<wgrant> kfogel: Change BugTaskNavigation.traverse_comments
<kfogel> wgrant: ?  hmrm.  Let me look, bbiab
<wgrant> Although you should probably 403 it instead.
<wgrant> Rather than denying its existence.
<wgrant> But I wonder how much other stuff that would break.
<kfogel> wgrant:     def traverse_comments(self, name):
<kfogel>         """Traverse to a comment by id."""
<kfogel> wgrant: I have to admit, that's a mismatch between method name and documentation :-)
<wgrant> kfogel: 'name' is a universal idiom.
<kfogel> wgrant: it's the "traverse_comments" plural part that bugs me.  The method is named for its implementation strategy rather than its effect.  What it's really doing is taking you to a comment; how it gets there is its own business, IMHO.
<wgrant> kfogel: Ah, right.
<kfogel> wgrant: anyway, I'm going to try a patch based on your idea.
<wgrant> kfogel: For an example of a similar 404 thing, check LaunchpadRootNavigation.traverse's person privacy handling.
<kfogel> wgrant: thanks!
<wgrant> But I really think it would be better to just revoke launchpad.View for hidden comments and deal with fallout. That would be the proper solution.
<kfogel> wgrant: how does one "revoke launchpad.View" ?
<wgrant> kfogel: Alter the ViewBugMessage security adapter to return True only if the comment is visible or the user is an admin.
<kfogel> wgrant: ooooh, btw, so far this patch has exactly the effect I wanted: http://paste.ubuntu.com/407283/
<kfogel> wgrant: trying your view-revokation method now
<wgrant> kfogel: Except that patch makes it impossible to show again without DB access.
<kfogel> wgrant: ??
<kfogel> wgrant: (I mean, I still have to add the user == admin condition in there, but that's not related to your comment just now I think)
<wgrant> kfogel: Ah, right, once you add the user == admin thing it will be fine.
<kfogel> wgrant: toggling visibility of comments is done through DB at the moment, true, but the lack of UI for it is not related to this change.
<wgrant> kfogel: Isn't there an API to do it?
<kfogel> wgrant: that is, even if the admin could still see the comment, they'd still need to use the DB to change the visibility
<kfogel> wgrant: nope, no UI yet
<kfogel> wgrant: now, it's true that an admin will *see* the comment (it'll have a yellow background, too, to highlight that it's not visible to the rest of the world)
<wgrant> kfogel: Admins may set visibility through the API with bug.setCommentVisibility.
<kfogel> wgrant: sorry, I confused API with UI, yes.
<wgrant> Though since it's on bug, not bug_message, it doesn't require the comment to be traversable.
<kfogel> wgrant: yes they can do it via API -- but that would be true either way
<kfogel> right
<kfogel> wgrant: http://paste.ubuntu.com/407285/  seems to have all the desired effects; care to have a look?
<kfogel> (I guess there's a style question of whether those parens are really needed, but whatevs.)
<wgrant> kfogel: That looks about right.
<kfogel> wgrant: ok, now I'm going to learn about "bzr shelve" and try your other proposed way :-)
<wgrant> Except that this whole visibility thing is implemented by working around the Zope security machinery.
 * wgrant cannot live without shelve.
<kfogel> wgrant: yeah... I have the feeling that if I had a wider/deeper picture of how Launchpad fits into Zope, I'd be more dissatisfied with this solution.  Ignorance truly is bliss sometimes.
<wgrant> kfogel: Do you know what the status of checkwatches is? see #launchpad
<kfogel> wgrant: btw, I don't think the LaunchpadRootNavigation.traverse() solution is actually better here.  It does 301 redirects, not 404s, and I think we actually want a 404 here (which my patch gets by simply returning None).  I guess I could do the "raise NotFound(self.context, name)" trick, but what's the advantage?
<wgrant> kfogel: Right, the NotFound case is what I was thinking of. But it's not really comparable, since you can just return None from a normal traversal method and get the same effect.
<kfogel> exactly
<maxb> For some reason launchpadlib 1.5.6 is missing its bzr tag. Please could someone who is a member of ~lazr-developers add it:
<maxb> bzr tag -d lp:launchpadlib -r revid:leonard.richardson@canonical.com-20100304202437-68ykm4fpbio8jdid 1.5.6
<wgrant> (I didn't actually look at the method first, sorry)
<maxb> I've checked the branch against the tarball to verify the revision
<wgrant> spm: How do I get the PPA log parser script set up?
<spm> this'd be similar ot the Q&D checks we did the other week?
<wgrant> I don't think I know about those checks.
<spm> wild stab in the dark guess that...
<wgrant> I guess I'm not even sure if the prod config has been fixed yet. And Julian won't be around for several days.
<spm> wgrant: in any event it's a two step. 1. it's in the code tree cronscripts/<file>.py 2. a request is made - RT is preferred cc'ing team lead/whome ever to get the nod that this is sane etc
<spm> some are part of a release cycle - code* are the most notable here - so they get added to the Production Status page as a special rollout request.
<spm> some are an *integral* part of ....
<wgrant> spm: Do you know if there's an RT about it already?
<spm> I don't - I'll check....
<wgrant> Julian suggested that he would organise it last week, but I can't see LPS or RT or anything at all to check its status.
<thumper> woo.. just slammed my big branch into ec2 test
#launchpad-dev 2010-04-01
<spm> wgrant: what's the script name? (sorry getting pulled via about 3-4 diff conversations, + phone, + doorbell....)
<wgrant> spm: parse-ppa-apache-access-logs.py
 * mwhudson splorfs at the test failure
<wgrant> mwhudson: Which? The one that will hit only today and tomorrow?
<wgrant> Or a real failure?
<mwhudson> registry/tests/../stories/milestone/object-milestones.txt
<wgrant> Yeah, that one.
<mwhudson> i think it'll fail from today onwards won't it?
<spm> wgrant: out of curiosity; how many guess do I get to work out what that script does?
<wgrant> spm: One.
<sinzui> I think it will only fail for the next two days
<spm> Oooo. tricky...
<wgrant> mwhudson: Won't it start working once the date passes?
<mwhudson> wgrant: maybe
<wgrant> Or once somebody changes it to year 3000. Whichever is sooner.
<sinzui> mwhudson, None-the-less, someone should ellipsis the dates
<mwhudson> yeah
<mwhudson> sinzui: want to be that person?
<mwhudson> or i can, i guess
<wgrant> If the release had been a few hours later, this could have been very inconvenient.
<mwhudson> hooray for easter being when it is this year?
<sinzui> If this had been the usual release with last minute landings this would have been a disaster
<wgrant> Yep.
<sinzui> mwhudson, I can submit branch for the test. I wasn't planning on landing anything until PQM opens though
<spm> wgrant: I can't find any ref to a script of that name; so i suspect it hasn't been requested yet. Or! it hasn't yet been sorted into an RT queue we have access to. But that's likely only for stuff in the past few hours max.
<mwhudson> oh right yeah, we're not in 10.04 yet
<wgrant> sinzui: Doesn't production-devel need it?
<wgrant> Or we won't be able to CP critical stuff like the checkwatches fix for days.
<sinzui> wgrant, right.
<wgrant> spm: Damn, I guess we'll be waiting a week then. Thanks.
<sinzui> I think the solution is obvious. The next person that has to land a branch verifies that the tests passes, and if not makes a trivial fix. ec2 will certainly tell us, and the test fails is dev so there should be no surprise
 * mwhudson stabby
<thumper> mwhudson: ?
<mwhudson> thumper: oh, test environment fun
<mwhudson> also, the tests concerned are the codehosting acceptance tests so very very slow
<wgrant> spm: AARNet agreed!?
<wgrant> Yay.
<spm> wgrant: to be AU offical? technically they asked; not agreed. :-)
<thumper> mwhudson: yeah they are
<mwhudson> are tests allowed to depend on the hostnames being set up?
<mwhudson> i.e. on bazaar.launchpad.dev being in /etc/hosts ?>
<spm> wgrant: The guy who is their tech manager, for want of a better description, is a good mate (as well as of mbp too); and they get impressive benefits to their network costs by doing all this mirroring.
<wgrant> spm: Did they also agree to be a push mirror?
<wgrant> Very convenient.
<spm> not sure tbh; but they are *very* keen to look into ways to better handle mirroring. so win win.
<wgrant> Excellent.
 * mwhudson would like a pushConfig that works cross-process
<wgrant> Can I whip up a branch that replaces the near-constant "Development"
<wgrant> ... status in branch listings with the MP status, if present?
<ctrlsoft> wgrant: doesn't that assume there's at most one mp status?
<wgrant> ctrlsoft: Yes.
<wgrant> Which is a false assumption, but a rare case in which I'm tempted to just revert to the branch status.
<ctrlsoft> wgrant: It's possible for a branch to be proposed for merging into multiple other branches
<ctrlsoft> wgrant: Hmm
<wgrant> Not optimal, but still much better than the current situation which is completely useless.
<ctrlsoft> wgrant: I agree the current status is a bit pointless, but I'm not convinced this is the best fix
<wgrant> If I have a couple of WIP MPs, https://code.edge.launchpad.net/~wgrant/launchpad does not help me.
<wgrant> I have to go through all of the branches with MPs to find them.
<ctrlsoft> wgrant: my branch listing page is useless anyway, I tend to only use +activereviews
<wgrant> ctrlsoft: +activereviews doesn't help with inactive reviews, though :(
<ctrlsoft> (although it's a bit better now that destroySelf() is exposed over the API :-))
<ctrlsoft> wgrant: yeah, we need +reviews :-)
<ctrlsoft> I'd also be nice to see reviews related to teams I'm in on that page
<wgrant> ctrlsoft: https://code.edge.launchpad.net/~jelmer/launchpad isn't that bad.
<wgrant> But I guess since you care about lots of other projects, it's not as useful.
<ctrlsoft> wgrant: true, but that's mainly because I've done quite a bit of housekeeping lately
<ctrlsoft> wgrant: I had about 3.5k branches registered, all set to development
<wgrant> ctrlsoft: ...
<wgrant> Whhhhhhy?
<thumper> mwhudson: you up for a 15 min max pre-mid-impl call?
<thumper> ctrlsoft: trying to hide?
<ctrlsoft> thumper: Now that Branch.selfDestroy() is available over the web API I've deleted most of them :-)
<thumper> ctrlsoft: cool
<thumper> ctrlsoft: james_w is exposing code import registration
<thumper> ctrlsoft: so you could create git imports :)
<ctrlsoft> thumper: (that should also be a significant part of your remote branches gone)
<thumper> \o/
<ctrlsoft> thumper: (-:
<wgrant> Wasn't the remote branch UI going to die at some point?
<ctrlsoft> thumper: It'd be interesting to use the Vcs-* headers in Debian packages to set up automatic imports for all Debian packages once Launchpad starts supporting code imports for packaging branches
<lifeless> ctrlsoft: they aren't sufficient to do packaging improts
<ctrlsoft> lifeless: not as the canonical branch perhaps, but it'd be interesting to have them around and be able to merge from them
<wgrant> It would be nice if there was some consistency in package version control.
<wgrant> Nobody in Debian seems sure whether they are meant to be versioning debian/ or everything.
<wgrant> Or whether they are based on the upstream branch or not.
<mwhudson> thumper: sure
<thumper> wgrant: how broken is the build branch into source package job?
<wgrant> thumper: It works perfectly.
<thumper> mwhudson: although I'm about to run and get a coffee before collecting the girls from school
<wgrant> We've run it on DF.
<thumper> wgrant: I heard that there were bugs
<mwhudson> thumper: ok, ping me when you get back i guess
<thumper> when bigjools tried recently
<thumper> mwhudson: ok
<wgrant> thumper: There were issues testing it on DF because of firewalls and broken slaves.
<wgrant> Oh, right, there are two sorta-bugs.
<thumper> ah, but on the whole it is good?
<wgrant> 1) Scoring isn't implemented.
<wgrant> 2) Things crash if you set it to non-virtualized.
<wgrant> But the latter should never happen.
<wgrant> Right, apart from those couple of minor things it works fine.
<wgrant> The second one may well be magically fixed in the model rework.
<wgrant> The first we have no algorithm for.
 * thumper really afk for coffee run
<wgrant> So, cron.publish-ftpmaster now timestamps all of its steps nicely. Can I get hold of the log for a couple of runs to work out where I can cut out more time?
<james_w> ctrlsoft: that's exactly my plan for doing the work
<ctrlsoft> james_w: great :-)
<ctrlsoft> gmta ;-)
 * james_w is wondering how to get Debian to adopt Vcs-*-upstream:
<ctrlsoft> james_w: wasn't there some talk about half a year ago about having that in debian/watch ?
<ctrlsoft> (I prefer a control header as well, though)
<james_w> yeah, I think so
<james_w> but every time you propose a new control header people complain about it bloating the size of the Sources file etc.
<ctrlsoft> james_w: I wouldn't mind debian/watch if it didn't have such an obscure syntax
<james_w> yeah
<james_w> we'd need a new watch file format to do it properly
<thumper> mwhudson: ping
<wgrant> thumper: Also, bug #552976
<mup> Bug #552976: SourcePackageRecipeBuild security needs work <Launchpad Bazaar Integration:New> <https://launchpad.net/bugs/552976>
<thumper> mwhudson: yeah, was just reading that in my inbox
<mwhudson> thumper: hi
<thumper> mwhudson: skype?
<mwhudson> thumper: unless you want to try mumble, sure
<thumper> mwhudson: haven't tried mumble
<thumper> what do I need to do?
<mwhudson> ah you need to get an account from IS
<mwhudson> skype it is then i guess
<wgrant> Why not use Jabber?
<wgrant> Empathy XMPP video chat works fine these days.
<wgrant> s/video/audio/
<thumper> ETOOMANYOPTIONS
<wgrant> What's the official plural of 'series' these days?
<wgrant> I thougt it was recently changed to 'serieses'
<wgrant> But I don't see that used anywhere.
<thumper> wgrant: series
<thumper> wgrant: all the serieses were changed to series
<wgrant> Oh.
<mwhudson> i think if you use anything else mpt kills you in your sleep
 * wgrant goes back to 'for das in archseries:'
<thumper> Total: 26259 tests, 54 failures, 6 errors in 224 minutes 52.774 seconds. ;-(
<thumper> best fix them then
<thumper> ah
<thumper> I wondered if there'd be much fallout from that
<thumper> glock tweak
<thumper> hmm, lots of breaks
<mwhudson> ah yeah
 * thumper afk for pizza
<thumper> man some doc tests are slooooow
 * thumper is still fixing tests
<thumper> what was the solution to the GpgmeError ?
<thumper> GpgmeError: (32, 176, 'Unknown error code')
 * thumper tries make clean build
<wgrant> thumper: Patch pygpgme.
<thumper> ?!?
<thumper> why do I have to?
<wgrant> Because the fix hasn't landed yet.
 * thumper sighs
<wgrant> It's an incompatibility with the new gpg in Lucid.
<thumper> where is the reference to the fix?
<wgrant> https://code.edge.launchpad.net/~jelmer/launchpad/new-gpgme/+merge/22093
<wgrant> http://bazaar.launchpad.net/~launchpad-pqm/pygpgme/devel/revision/46.1.23 is the actual diff.
<thumper> ta
 * thumper just merges jelmers branch
<thumper> ok, now it should pass
 * thumper runs the test again
<thumper> :(
<thumper> ImportError: No module named _gpgme
<thumper> gah
<wgrant> make?
<thumper> yeah, doing that now
<thumper> this branch has become the one that won't die
<thumper> or perhaps
<thumper> the branch that won't pass the freaking tests
<thumper> \o/ passing now
<thumper> at least that one
<wgrant> Which branch?
<thumper> https://code.edge.launchpad.net/~thumper/launchpad/all-code-review-email-using-jobs/+merge/22439
<wgrant> Ah, yay.
 * thumper stops for now
<jtv> wgrant: coming across one problem when simulating build slaves...  I think python-asn1 just dropped into universe.
<wgrant> jtv: What is that and why is it important?
<jtv> wgrant: it seems to be indirectly required by launchpad-buildd.
<jtv> And the slave's sources.list is set up for main only.
<StevenK> jtv: I can't find a package with that name at all
<wgrant> jtv: The slave's sources.list doesn't matter, except for translations jobs which don't override it.
<wgrant> StevenK: Same here.
<jtv> StevenK: sorry, python-pyasn1
<jtv> wgrant: ahhh
<StevenK> It's been universe since .
<wgrant> Ah, right.
<wgrant> It's been in universe forever, yeah.
<wgrant> conch requires it.
<wgrant> and twisted requires conch.
<wgrant> Hmmm.
<wgrant> jtv: Do you know of any further changes that will be necessary for the translations slave? lamont wants to push out a new lp-buildd tomorrow.
<wgrant> I would also appreciate a review of my get-translations-jobs-working, so the buildd hunk can be included in tomorrow's buildd rollout.
<noodles775> wgrant: did bigjools re-paste that buildd crash? Apparently I can't access the paste either ;) If not, I'll repaste it once I get access.
<wgrant> noodles775: bigjools was gone. lamont poked me a few hours ago and we worked it out.
<noodles775> wgrant: ah, great.
<jtv> wgrant: I can review that for you today, sure.  With pleasure, in fact.
<wgrant> jtv: Thanks. It's tiny.
<jtv> Okay, make that lots of pleasure.
<noodles775> wgrant: what was the issue?
<wgrant> noodles775: Running post-Wellington code with a pre-Wellington buildd-slave.tac.
<noodles775> Ah.
<wgrant> Easy to do, because buildd-slave.tac is copied in by a non-standard target.
<jtv> wgrant: I don't have any further changes for launchpad-buildd in the pipeline right now, but this one would be important.  I can file a bug and work on it today.
<wgrant> jtv: While I remember: the branch that you create on the wiki page caused an odd crash in the slave, so I ended up using a real source tree, which worked fine.
<adeuring> good morning
<noodles775> morning adeuring
<wgrant> Morning adeuring.
<adeuring> hi noodles775, wgrant!
<jtv> wgrant: the branch that I create on the wiki page?
<wgrant> jtv: The wiki page gives a sequence of commands to create a branch to pass to the slave.
<noodles775> wgrant: I spoke with bigjools yesterday about the schema change, and updated the MP with his comments if you're interested. https://code.edge.launchpad.net/~michael.nelson/launchpad/db-build-generalisation-db-changes/+merge/22527
<jtv> wgrant: ah... it's a pretty minimal setup, probably not sufficient.  It's never mattered in the past because things would break before it could.
<wgrant> noodles775: Hmm, I don't like BuildBase.
<noodles775> wgrant: why not?
<noodles775> (There's already a interface/class for it, it's just not yet a db-table)?
<jtv> wgrant: if we pick a sample branch from LP that people could just pull and then push locally, that'd work too.  Do you know of a conveniently small one?
<wgrant> jtv: No. I ended up hacking configure.ac or similar in hello to contain the GETTEXT_PACKAGE line that pottery looks for.
<wgrant> I couldn't find one that worked OOTB.
<wgrant> Though I didn't look hard.
<wgrant> noodles775: BuildBase seems a lot less obvious that BuildFarmJob.
<wgrant> s/that/than/
<wgrant> I've always regretted calling it that.
<wgrant> It is ugly.
<noodles775> wgrant: It is, but only because it should really be called simply "Build" right?
<jtv> wgrant: there has to be at least one that works out of the box, or I'll feel terribly pointless.  :)
<noodles775> And the only reason we don't call it Build right from the start is because that table already exists and will need to exist until we've got all the changes finished (in the pipeline of branches).
<noodles775> So we could rename it to simply Build at the end?
<wgrant> noodles775: A SourcePackageRecipeBuild shouldn't have a Build.
<wgrant> That's going to be terribly confusing.
<noodles775> wgrant: huh? It needs to have a build doesn't it? Now I'm confused. If a SPRecipeBuild is to have a date_started etc., that info is coming from the underlying build? Hrm, do you want to call about this and explain? or prefer IRC.
<wgrant> BuildFarmJob makes it abundantly clear what it is, and where it runs.
<noodles775> Yes, which is why I initiall suggested ServerFarmBuild (but from what you're saying, that's not nice either).
<jtv> wgrant: bug 553077
<mup> Bug #553077: Build slave needs universe package <buildfarm> <Launchpad Translations:In Progress by jtv> <https://launchpad.net/bugs/553077>
<wgrant> noodles775: These jobs needn't all be builds. If we call them jobs, we are more generic *and* we avoid confusion when talking about the concrete build types.
<wgrant> jtv: I am confused
<wgrant> jtv: Why does the slave's sources.list matter?
<wgrant> One doesn't install the slave in the slave.
<jtv> wgrant: ah, of course!  We're not emulating the chroot inside the chroot!
<jtv> This all gets so confusing
<jtv> No wait
<noodles775> wgrant: OK, so some jobs are builds, others may not be. OK.
 * noodles775 remembers the discussion at the sprint now.
<jtv> wgrant: we _are_ running launchpad-buildd on the slave, and that needs universe.
<wgrant> jtv: But the slave host's sources.list is out of our control.
<wgrant> We only control the one in the chroot, where we don't need to install launchpad-buildd.
<jtv> wgrant: okay, but launchpad-buildd started breaking for me.  It's gotta be pretty recent.
<wgrant> jtv: What's it breaking with?
<jtv> wgrant: no installable candidate
<jtv> referenced by another package but not available
<jtv> so I guess one of the packages we depend on in lucid sprouted a universe dependency in the past few weeks.
<wgrant> It's possible that python-twisted's dependencies have been strengthened lately.
<wgrant> I have to wonder if lp-buildd really needs more than python-twisted-core.
<jtv> yes... iirc it pulls in something like python-twisted-conch which in turn depends on python-pyasn1
<noodles775> wgrant: is this the same crash as earlier? bug 553068
<mup> Bug #553068: buildd-retry-depwait.py crashing <Launchpad itself:New> <https://launchpad.net/bugs/553068>
<wgrant> noodles775: No, completely unrelated. It's code I've touched recently... but it hasn't landed yet.
 * wgrant looks.
<wgrant> But... what.
<wgrant> Oh, I think I recall that code changing around the middle of 10.03.
 * wgrant checks.
<wgrant> It shuffled in there from another private method.
<wgrant> Come on, annotate...
<StevenK> blame!
<wgrant> qannotate, actually.
<StevenK> I wonder if gblame works too
<StevenK> Er, qblame
<wgrant> Yay it crashed.
<jtv> wgrant: looking at your branch... these are changes I looked at a bit yesterday, in fact.  Tip for this kind of MP: explain why each of the problems can't be integration-tested (or if you can't, well...  ;-)
<jtv> Saves lots of potential back-and-forth on the review, and sometimes leads to testing breakthroughs
<wgrant> Good point.
<wgrant> ctrlsoft: /win 5
<wgrant> Oops.
<wgrant> StevenK: Ahahaha
<wgrant> You are to blame.
<wgrant> Nice of you to suggest it :P
<StevenK> Oh crap, I am?
<wgrant> Yeah.
<wgrant> 10565
<wgrant> er, 10566
<wgrant> It was the one I thought.
<wgrant> You probably meant selectFirstBy. selectOneBy raises an exception if there's more than one.
<wgrant> StevenK: But why did you change from the list comprehension at all?
 * StevenK is looking for the branch locally
<wgrant> StevenK: http://pastebin.ubuntu.com/407455/
<wgrant> The test which used to catch that sort of thing is... gone.
<StevenK> wgrant: I thought ArchiveDependency only had one thing in it, though?
<wgrant> StevenK: It maps an archive to zero or more dependency archives.
<StevenK> I saw all through the ArchiveDependency code that only 1 depends is supported
<wgrant> One dependency per dependency archive.
<wgrant> That is, UNIQUE (dependent, dependency)
<wgrant> Not just UNIQUE (dependent)
<StevenK> Getting that test back would be nice, or was that I removed because it used expanded_archive_dependencies?
<wgrant> I'm not entirely sure why you removed it.
<wgrant> Why you removed expanded_archive_dependencies, that is.
<StevenK> There was a bug about it
<StevenK> cprov said it was only used by one bit and should die
<wgrant> Bug #300004
<mup> Bug #300004: Archive.expanded_archive_dependencies() is obsolete and should be removed <qa-untestable> <tech-debt> <trivial> <Soyuz:Fix Committed by stevenk> <https://launchpad.net/bugs/300004>
<wgrant> It seems odd that it's not used by the sources.list code.
<StevenK> Right, I have a diff. But I'm unhappy that the tests didn't pick it up
<wgrant> Well.. you removed them :P
<StevenK> The test was calling a method I removed, so ...
<wgrant> Indeed.
<jtv> oh damn, I just realized: we're not getting any launchpad-buildd fixes in today unless it's RC.
<wgrant> jtv: lamont already has a change on top of trunk.
<wgrant> As long as they're reviewed and noodles is happy, he is happy.
 * jtv likes happy
<jtv> hi henninge
<henninge> Hi jtv! ;)
<jtv> henninge: we're a lot closer to working buildfarm jobs... wgrant is working on integration-testing the tarball uploads, and I'm working on an accidental new dependency of launchpad-buildd on a universe package.
<wgrant> jtv: So, I think just try relaxing the dep to twisted-core, remove the rest, and see what happens.
<henninge> jtv: great! what can I work on?
<henninge> oh, the auto-approval ...
<henninge> ;-)
<jtv> henninge: enjoy :-P
 * wgrant wondered why it wasn't auto-approved.
<jtv> wgrant: that's what I'm trying... except: what is "the rest"?
<wgrant> jtv: python-twisted*
<wgrant> Anything that python-twisted depends upon.
<jtv> gah:
<jtv>   File "/usr/share/launchpad-buildd/canonical/buildd/slave.py", line 20, in <module>
<jtv>     from twisted.web import xmlrpc
<jtv> exceptions.ImportError: No module named web
<wgrant> Depend upon python-twisted-web.
<jtv> So we may have to add some twisted "sub-packages" as well
<henninge> Hi wgrant! ;)
<wgrant> Morning henninge.
<jtv> wgrant: there wasn't any other python-twisted* AFAICS
<wgrant> jtv: python-twisted-web has existed since at least Dapper.
<jtv> No I mean, I saw no other python-twisted dependencies in launchpad-buildd besides python-twisted.
<wgrant> jtv: Right, because python-twisted depends upon python-twisted-*
<jtv> And that was where I got confused about the "remove the rest."  :-)
<wgrant> Hmm.
<wgrant> But.
<wgrant> Actually, this may not be a problem.
<jtv> yay, launchpad-buildd is running!
<wgrant> jtv: I meant to actually remove those packages from your system.
<jtv> oh
<jtv> I'm working in a chroot
<wgrant> jtv: grep around for any more twisted imports.
<jtv> And for this I set up a fresh one
<jtv> good idea.
<wgrant> It turns out that this wasn't actually our bug at all; it's archive brekage.
<wgrant> But we should do this dependency minimalisation anyway.
<wgrant> (python-pyasn1 is a new dependency, and hasn't been promoted yet. it probably will be on Monday)
<mrevell> Morning.
<wgrant> StevenK: Is that known ^^?
<jtv> twisted.application, twisted.python, twisted.internet
<jtv> hi mrevell!
<wgrant> jtv: All of those are core.
<wgrant> Morning mrevell.
<jtv> great.
<jtv> And from my despot days I seem to remember a rule: keep your slaves lean.  If you find yourself with a fat slave, unless it's your food taster, you have a problem.
<wgrant> jtv: Heh.
<wgrant> So, yes, this was unnecessary, but a good idea anyway.
<StevenK> wgrant: I can't recall seeing it. Does it have an MIR?
<jtv> wgrant: I don't think it's unnecessary per se...  transient breakage in dev test setups is still a nuisance.
<wgrant> StevenK: Ah, yeah. Bug #552863
<mup> Bug #552863: [MIR] please promote pyasn1 <pyasn1 (Ubuntu):New> <https://launchpad.net/bugs/552863>
<StevenK> wgrant: Right, so the person to bug next is pitti.
<wgrant> StevenK: Well, the longer it stays broken the more likely we are to fix the excessive deps...
<StevenK> wgrant: I can promote before the MIR is approved, I just don't like doing it
<wgrant> StevenK: I think my last statement was in support of making it take as long as possible.
<StevenK> Sorry, I misread it
<jtv> wgrant: just found one more twisted import in buildd... twisted.mail.  Is that in core?
<wgrant> jtv: It's not, but isn't that only used by sequencer.py?
<jtv> Yes.
<jtv> It wasn't installed with the rest of the stuff on the slave, which is why I didn't see it before.
<wgrant> Yeah, that is part of a very ancient master.
<wgrant> It probably hasn't been run in years.
<jtv> I don't see it being used anywhere...
<wgrant> And isn't part of the buildd, so it doesn't matter.
<jtv> Maybe we should kill the file at some point then?
<wgrant> I have a buildd cleanup campaign planned. And partly implemented.
<jtv> yay
<wgrant> At the moment it's a mess of slave code, non-slave code, obsolete slave docs, obsolete non-slave docs...
<jtv> btw please let me know when that last test is pushed.  I'm itching to approve that MP!
<wgrant> jtv: Yeah, got distracted again. Just committed it now.
<wgrant> noodles775: Ah, I just realised why you had to ask about the buildd crash earlier. I sent an email several hours ago explaining that lamont had poked me, but it was still sitting in my outbox for no obvious reason.
 * jtv just had his fs go read-only again and is about to disappear for a while.
<noodles775> wgrant: heh, yep, the last email I had was you saying you didn't have access to the paste :)
<wgrant> jtv: That started happening to me again yesterday.
<jtv> noodles775: I have to leave to deal with various broken systems, but would very much appreciate review: https://code.edge.launchpad.net/~jtv/launchpad/bug-553077/+merge/22599
<noodles775> jtv: yep, one of us will do it asap :)
<wgrant> It happened twice, around a failed suspend. I hadn't had a suspend failure in years before then, so I'm not really sure which is the cause and which is the effect.
<jtv> wgrant: for a while I thought it was just the temperatures...  I had one HD operating in a room that was about 40Â°C and it broke, and another broke while in a room that must have been around 10Â°C.  But now it's happening more randomly AFAICS
<jtv> noodles775: thanks
<wgrant> jtv: Yeah, it's been happening to me since mid-December. But it only happened a couple of times between Wellington and yesterday.
<wgrant> And it's fortunately only /, so /home doesn't get horribly corrupted.
<wgrant> Also, those new tests are pushing... slowly...
<jtv> "only a couple of times"?  Ted T'so was sitting right in front of me at the bzr session, I could easily have whacked him over the head.  He even gave me an excuse.
<jtv> Spilt milk I guess.
<wgrant> Hahaha.
<jtv> yes, bzr has slowed down a lot it seems.
<wgrant> It's a combination of LP and bzr.
<wgrant> And crappy upstream bandwidth.
<jtv> that reminds me to go buy that new switch
<jtv> ...and fsck
<wgrant> Plymouth has this really nice feature where it breaks if your filesystem needs a manual fsck, like mine tends to after one of those.
<wgrant> s/Plymouth/mountall/, I guess.
<jtv> Guess what I'm about to do again...
<wgrant> jtv: It's not actually an FS problem, AFAICT.
<wgrant> The actual block device is read-only.
<wgrant> I am on LVM - are you?
<jtv> wgrant: your diff just updated; thanks for extending FakeMethod.  I had this feature in an early iteration but no use-case for it yet.  One tiny thing more: a full stop at the end of your assertion string.  :)
<jtv> No, not on LVM AFAIK
<wgrant> Hm. Damn.
<jtv> (I haven't used LVM for a while... never really liked ti)
<jtv> *it
<wgrant> jtv: Full stop added.
<jtv> wgrant: and approved.  Thanks again!
<jtv> And now, off to deal with everything being broken
<jtv> NO CARRIER
<wgrant> noodles775, jtv: Thanks for the reviews.
<jtv> pleasure
<wgrant> noodles775: Can you please mark https://code.edge.launchpad.net/~wgrant/launchpad/bug-549340-fix-buildqueue-destruction/+merge/22286 as Rejected?
<wgrant> Apparently I cannot.
<deryck> Morning, all.
<noodles775> wgrant: no problem, and done.
<wgrant> noodles775: Thanks.
<henninge> What could be the reason for the test framework to fail with "no module named meliae" and "No module named setproctitle" ?
<wgrant> henninge: bin/buildout
<wgrant> 'make' might do it too, but didn't in a couple of branches here.
<henninge> oic
<henninge> wgrant: yes, that's what I would have assumed.
<henninge> now this:
<henninge> Error: There is a version conflict.
<henninge> We already have: zope.interface 3.5.3
<wgrant> henninge: Lucid?
<henninge> yup
<wgrant> henninge: I resolved that by installing Karmic's 3.5.2.
<wgrant> It probably needs some buildout god to fix it.
<wgrant> Or we just upgrade Launchpad to use 3.5.3.
<henninge> wgrant: thanks.
<ctrlsoft> is there a particular reason the buildd slave package lives in the main launchpad branch ?
<ctrlsoft> it seems like it would be quite easy to split off
<wgrant> ctrlsoft: You are the third person to say that today.
<wgrant> It depends on the rest of Launchpad for two things: testing infrastructure, and tachandler.
<wgrant> I believe jml was looking at that. Although that may not have ended up happening.
<jml> everything I look at withers under my gaze
<wgrant> Hm, is USSO meant to be read-only?
<jml> wgrant, I didn't do anything about it, I just got pissed off and wrote an angry note in my todo lists. I've got two branches on my plate now, which is basically my limit.
<wgrant> jml: Heh.
<jml> ctrlsoft, but I think it would be quite easy to split off.
<jml> the toughest bit is deciding whether to extract the shared dependencies between it and LP, or whether to just duplicate them
<jml> and if the former, deciding what to do.
<ctrlsoft> jml, wgrant: ah, ok
<jml> so, in case you weren't following on #twisted
<jml> they think that tachandler is a bad idea, but that something like it could be achieved by giving twistd more of an API
<jml> (something which has been a goal for twistd for ages)
<jml> if I were doing the buildd split, I would just copy tachandler.
<wgrant> jml: Note that Translations' odd obsession with not having tonnes of untested code means that lp-buildd now has test dependencies on LP.
<jml> wgrant, which bits?
<wgrant> jml: TestCase, FakeMethod and TestCaseWithFactory from lp.testing.
<wgrant> The last sounds awkward.
<jml> wgrant, I'm saying this in total ignorance of the code, but...
<jml> wgrant, I reckon we could come up with a better testing strategy.
<wgrant> Almost definitely.
<leonardr> is anyone else having time-sensitive test failures for registry/stories/milestone/object-milestone.txt?
<sinzui> yes
<salgado> leonardr, it's been fixed on db-devel already
<gary_poster> noodles775: does your release-critical landing fix the existing failure in the branch that is running?
<gary_poster> well, disable :-)
<noodles775> gary_poster: yes, it disables the test and I created bug 553259.
<mup> Bug #553259: buildd-slave.txt has a timing issue <Soyuz:Confirmed> <https://launchpad.net/bugs/553259>
<gary_poster> thank you noodles775, great
<sinzui> bac: ping
<bac> yes, sinzui?
<sinzui> bac: I am looking at the end of tests/product-views and the start of product-portlet-packages-views ...
<sinzui> product-views doesn't seem to be doing much, and what it does is already tested in product-portlet-packages-views
<sinzui> bac: do you agree?
<bac> let me look
 * sinzui was looking for a test that the form creates a Packaging object
<bac> sinzui: yes that looks like a c-n-p error where the cut didn't happen
<sinzui> as I thought. I will remove it now
<Ursinha> sinzui, hi, do you know what's this oops about? https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1551O2750
<sinzui> Ursinha, someone using wget is trying to automate launchpad. He is not passing a referer so we do not trust him
<sinzui> Ursinha, gary_poster know the issue well. The user should be using API
<Ursinha> sidnei, hmm, I see
<gary_poster> Ursinha: yes, that should not be an OOPS
<Ursinha> gary_poster, want me to file a bug for that?
<sidnei> Ursinha, good to know! *wink*
<Ursinha> sidnei, sorry :)
<gary_poster> Ursinha: yes, thank you
<Ursinha> gary_poster, I have another oops here, do you know what's happening?  https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1552C764
<salgado> Ursinha, looks like someone attempting to login while login.lp.net was down.  (it was scheduled to go down for a few minutes at around 9:00UTC today)
<salgado> Ursinha, care to file a bug asking for better error handling when the OP is down?
<Ursinha> salgado, I see
<Ursinha> salgado, sure, will do that
<gary_poster> thanks salgado
<Ursinha> salgado, thanks
<salgado> np
<Ursinha> sinzui, have you seen this oops before: https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1552A1029
<sinzui> Ursinha, maybe. I think this is cause by a deactivated product that is linked to a package...this should not happen, but I recall seeing this once in production and I reactivated the product
<sinzui> Ursinha, yes! that is the case. I will fix activate the project, or remove the link and report a bug that this is happening.
<Ursinha> thanks sinzui
<sinzui> Ursinha, I can get a report of this cases in production from the staging db on Sunday,  can fix the data issues. We will need a code change to ensure projects linked to packages cannot be deactivated.
<Ursinha> right
<Ursinha> sinzui, want me to file the bug
<Ursinha> ?
<sinzui> I am doing that now
<Ursinha> ok
<Ursinha> Chex, gary_poster, rockstar, bigjools, danilos, sinzui, allenap, production meeting @ Freenode #launchpad-meeting in 5 minutes
<gary_poster> thanks
<sinzui> me
<maxb> leonardr: Can I trouble you to add a missing tag to the launchpadlib branch? I've checked the branch against the tarball to verify the revision. command: bzr tag -d lp:launchpadlib -r revid:leonard.richardson@canonical.com-20100304202437-68ykm4fpbio8jdid 1.5.6
<leonardr> max, sure
<leonardr> i now have a scirpt that tags releases as part of the release process btw
<leonardr> done
<Ursinha> stub, are you joining us in the meeting? I'm not expecting that :)
<stub> I just emailed my bit :)
<Ursinha> stub, thanks
<stub> I am supposed to be enjoying myself somewhere right about now...
<Ursinha> stub, go for it then :)
<Ursinha> Chex, gary_poster, rockstar, bigjools, danilos, sinzui, allenap, production meeting @ Freenode #launchpad-meeting now :)
<Ursinha> al-maisan, hi, want to join us in prod. meeting in behalf of bigjools? :)
<al-maisan> Ursinha: sorry but I am not part of the launchpad team any more.
<Ursinha> al-maisan, oh oh sorry, it's true
<maxb> thanks leonardr
<leonardr> np
<al-maisan> Ursinha: :)
<kfogel> adeuring: got a sec?
<kfogel> intellectronica: test suite question for you, got a minute?
 * kfogel rotates around the members of his team
<kfogel> gmb: test suite question for you, got a minute?
<sinzui> did someone add distroseries data to sampledata?
 * sinzui is seeing a tremendous amount of insane packaging data now. This crack cannot be created using the UI
<gary-lunch> Ursinha: fix for https://launchpad.net/bugs/547054 ("AssertionError: Don't try to render this page when the user is logged in.") was not in roll-out; will be in CP
<mup> Bug #547054: AssertionError: Don't try to render this page when the user is logged in. <oops> <Launchpad Foundations:Fix Committed by salgado> <https://launchpad.net/bugs/547054>
<Ursinha> gary-lunch, ah, cool, thanks!
<gary-lunch> :-)
<adeuring> kfogel: sorry, i was away. how can i help?
<kfogel> adeuring: ah, well, I think I've found a workaround, but in a minute I'll present you with a paste that contains the question
<kfogel> adeuring: (just need to run a test first)
<adeuring> kfogel: ok, no hurry ;)
<kfogel> adeuring: ok, so http://paste.ubuntu.com/407667/ works (note the branch has code changes already committed, this is just about the test suite).
<kfogel> adeuring: but http://paste.ubuntu.com/407669/ does NOT work
<kfogel> adeuring: I'm wondering why I can't reuse bug_11.messages.  I realize its value might be out-of-date by now and I'd need to refresh it, but I shouldn't be seeing the failure I am seeing, AFAIK.
<adeuring> kfogel: without having had a look at the complete test file: Did you perhaps call logout(9 before the failing line?
<kfogel> adeuring: yep
<kfogel> adeuring: I mean, I didn't call it, but pre-existing code does, to be preice
<kfogel> precise
<adeuring> kfogel: OK, so, a login() might help :)
<kfogel> adeuring: thanks.  Now that I understand what was going on, I think I'll stick with my existing method for finding the last index, which works fine too and doesn't require a login.
<adeuring> kfogel: yes, makes sense
<adeuring> kfogel: but: I think the test would be even better if you showed that the admin user _has_ accesss to the URL you are opening in user_browser. The "not Found" message might have completely different causes than what you want to test.
<kfogel> adeuring: can do
<kfogel> adeuring: why do our tests no longer seem to show +actual output?
<kfogel> adeuring: or am I doing something wrong?
<adeuring> kfogel: I didn't notice this problem yet...
<kfogel> adeuring: I run bin/test -vvct foo, with dummy expected output, but when the test fails, it just shows the "-" lines, no "+" lines
<kfogel> adeuring: you see + lines?
<kfogel> adeuring: for example http://paste.ubuntu.com/407679/
<adeuring> kfogel: admittedly, I did not ran failing doc tests todays
<adeuring> let me try...
<kfogel> adeuring: oh, I think I might be making a braino, hold on
 * kfogel hates on the way every experiment costs 30 seconds
<adeuring> ;)
<kfogel> yeah, braino
<kfogel> adeuring: okay, I'm baffled
<kfogel> I'm trying lots of ways to match the output, but I can't.
<kfogel> http://paste.ubuntu.com/407683/
<kfogel> adeuring: you can see the failure, with expected vs actual, and at the very end you can see the relevant part of the diff
<jml> g'night all
<adeuring> kfogel: admin_browser.contents contains the enitre HTML page. You want to use extract_text(...)
<kfogel> adeuring: erp
<kfogel> adeuring: thank you.  It's been great having you help me today, my first day ever programming Launchpad, or Python for that matter.
<kfogel> Where's the on/off switch on this thing, now...?
<adeuring> kfogel: there are these days, where everything fails :) Usually a sgin that you need some holidays ;)
<kfogel> adeuring: "Holiday".  I've heard people use that word a few times in the last few days.  Is it defined in the internal wiki somewhere?
<kfogel> Oor should I just google it?
<adeuring> kfogel: probably ;)
<kfogel> deryck: how did bug #544843 get filed in Launchpad Bugs?  It seems like a change in Apport's behavior to me (and I'm not sure whether it's a bug or not, though the fact that Martin Pitt thinks it is is persuasive).
<mup_> Bug #544843: Bug reports with +storeblob now attach the raw file <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/544843>
<deryck> kfogel, it is our bug.
<deryck> kfogel, something gmb did with the recent storeblob work for processing large data files offline has changed so that we store the raw file when previously we just did the attachments
<kfogel> deryck: it sounds like there's some structure to these blobs that I don't know about :-)
<deryck> kfogel, yeah.  was this on the easy bugs backlog?
<kfogel> deryck: yep, it's next in line
<deryck> kfogel, I think I was thinking this should be simple for gmb who will know exactly what he did wrong.
<kfogel> deryck: ok.  I'll look at something else, in the interests of efficiency.  I can watch for gmb's change later to see what it was, anyway.
<deryck> kfogel, yeah, good call.  might be a bit much for an afternoon's work.
<deryck> sorry to mislead you there earlier.
<kfogel> deryck: I know it was intentional, you don't have to pretend.
<deryck> kfogel, getting you back for your collusion in this god-awful april fools holiday the web takes.
<kfogel> deryck: btw, I sent this review to abel, but if you want to do it it'd probably be quick (he's off now I think):
<kfogel> https://code.edge.launchpad.net/~kfogel/launchpad/546943-really-hide-hidden-bug-comments/+merge/22647
<kfogel> deryck: then I can move my card along :-)
<deryck> ah, you get me with my secret passion.
<deryck> kfogel, sure.  looking now.
<kfogel> deryck: oh urgh diff still updating even now -- taking a while
<kfogel> there, done
<deryck> kfogel, done.  r=me.
<kfogel> deryck: thx
<deryck> np
<deryck> kfogel, could you help me test a theory please?  (visiting a page on lp to subscribe me to something)
<deryck> kfogel, (when you see this) never mind.  Found what I needed.
* mbarnett changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.03 | PQM is open | Release manager: deryck | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
<kfogel> deryck: sorry, was afk
<deryck> kfogel, no worries.
<deryck> later on, everyone...
<kfogel> barry: still on?
<kfogel> http://paste.ubuntu.com/407748/   <<--  sinzui, if you're on: I seem to have h0rked my devel tree (I discovered this while trying to link-external-sourcecode for another very old tree that I hadn't updated in a while)
<kfogel> sinzui: trying to figure out how to get back to buildability
<kfogel> bfiller: know much about launchpad development?
<bfiller> kfogel: played with launchpadlib api in python a bit
<kfogel> bfiller: ah, cool.  Okay, I won't hit you up for advice on why my build is broken right now, then :-)
<bfiller> kfogel: yeah, I wouldn't know about that :)
<salgado> kfogel, just call link-ext-s passing the path to the parent branch to it.  after that you can 'make build'
<kfogel> salgado: except if you're in devel, and you pass devel, you lose
<kfogel> salgado: search://"infinite binary loop"/ :-)
<salgado> kfogel, oh, in that case you have to pass /path/to/lp-sourcedeps I think
<kfogel> salgado: let me see
<james_w> how do I change the person making the requests in webservice doctests?
<kfogel> salgado: it worked!
<kfogel> salgado: thank you.
<salgado> np
<kfogel> james_w: show me which file
<kfogel> james_w: (not that I will necessarily know the answer, but I can try...)
<james_w> lib/lp/code/stories/webservice/xx-code-import.txt, but I don't know if it will be in your tree yet
<james_w> but anything in that dir should be similar
<james_w> I want to do a webservice.named_post() from someone other that 'sal<PING AVOIDANCE>gado'
<kfogel> james_w: so login() and logout() are usually the way...
<james_w> ok
<james_w> and I can pass a person object?
<kfogel> james_w: oh, I may be misunderstanding -- I don't know what named_post is
<kfogel> james_w: IOW, you're not doing this through direct URL requests?
<salgado> james_w, that's not easy, unfortunately
<james_w> damn
 * kfogel listens to what salgado says to james_w
<james_w> I guess I can dig into sampledata to find a team that the default user is and isn't part of then
<salgado> james_w, there are only a couple users setup to use the webservice
<james_w> or add them to a team I create?
<salgado>     test.globs['webservice'] = LaunchpadWebServiceCaller(
<salgado>         'launchpad-library', 'salgado-change-anything')
<salgado>     test.globs['public_webservice'] = LaunchpadWebServiceCaller(
<salgado>         'foobar123451432', 'salgado-read-nonprivate')
<salgado>     test.globs['user_webservice'] = LaunchpadWebServiceCaller(
<salgado>         'launchpad-library', 'nopriv-read-nonprivate')
<salgado> actually
<salgado> I think I lied to you
<salgado> webservice_for_person() in lib/canonical/launchpad/testing/pages.py seems to be what you want, no?
<salgado> yeah, it does the oauth dance for you
<james_w> looks like it, thanks
<kfogel> barry: in a launchpad.dev environment, how does one test bug email notification?
<kfogel> barry: I've instrumented BugNotificationBuilder.build(), but it doesn't even seem like it's ever invoked when I make a change to a bug.
<kfogel> maxb: ^^ if you know answer to my above question to barry, let me know
<maxb> kfogel: sorry, no. I've barely got any insight into how LP code actually works... too much time spent poking karmic/lucid upgrade issues instead :-)
<james_w> has anyone used webservice_error?
<james_w> is there a knack to it?
<james_w> my tests are reporting it not to work as I thought
<james_w> and in fact there's a code test asserting that it has the behaviour that I don't expect
<james_w> BranchMergeProposalExists has webservice_error(400), but when it is raised in the webservice tests the response code is 500
<wgrant> There's no knack to it.
<wgrant> It just works.
<wgrant> It's not inherited or anything?
<james_w> aha!
<james_w> I think I know what it is
<james_w> said exceptions aren't imported by canonical.launchpad.interfaces
<wgrant> Ah. Of course. Import them into lp.code.interfaces.webservice
#launchpad-dev 2010-04-02
<james_w> wgrant: any comments on https://code.edge.launchpad.net/~james-w/launchpad/register-code-import/+merge/22668 would be welcomed
<james_w> there are some spurious changes in the diff from prerequisite branches
<james_w> I clearly need to land some of them to avoid that issue
<wgrant> james_w: MEGADIFF
 * wgrant looks.
<james_w> Hopefully that will fix it
<wgrant> I used to run into this problem a lot. Now I just block until the first chunk is reviewed and landed -- it saves a lot of mess.
<wgrant> james_w:
<wgrant> + cvs_root=TextLine(title=_('CVS roor URL')),
<wgrant> roor?
<lifeless> phwoar
<wgrant> james_w: Also, newCodeImport should probably use export_factory_operation
<wgrant> That mixin business isn't nice, but I guess you have to.
<wgrant> Anybody around who can land three approved branches for me?
<wgrant> Morning jtv.
<jtv> hi wgrant!
<wgrant> jtv: Can you please land that branch of mine that you approved a couple of days ago?
<wgrant> https://code.edge.launchpad.net/~wgrant/launchpad/get-translations-jobs-working/+merge/22584
<jtv> wgrant: I may not be the best person to askâmy lucid laptop is stuck in an unbootable state and I need to rip open my other machine in hopes that a new network card will fix the problems that a new switch didn't.
<wgrant> (there are also another four approved branches which would be nice to get landed soonish...)
<wgrant> Ah. Ewww.
<wgrant> The read-only filesystem killed it?
<jtv> Sort of.
<jtv> The automatic fsck on boot bails out, telling me *on a text console that's hidden by the boot screen* to run fsck manually.
<wgrant> Heh, yes, that's the behaviour I described.
<wgrant> So it's not just me...
<jtv> Not that it matters much, since my usual trick for booting into a shell no longer works.
<wgrant> KEYBUK!!!!!!!
<wgrant> jtv: What did you try?
<jtv> init=/bin/sh
<wgrant> Hmmm. Do break=mount or break=top work?
<jtv> didn't know about those... trying.
<wgrant> They should drop you quickly into an initramfs shell.
<wgrant> But I'm not entirely sure how the mountall transition will have affected them.
<jtv> But I add them to the kernel command line, right?  Not the initrd one?
<wgrant> Right.
<jtv> Yay, this looks like a shell of a sort
<wgrant> Excellent.
<wgrant> Hope you have a fsck at your disposal.
<jtv> Doesn't look good.  :(
<jtv> I never liked initrds
<jtv> Now here we have something that really *should* be in there, and it isn't
<wgrant> No fsck.ext4?
<jtv> Nothing with fsck in the name.
<wgrant> It must be in there somewhere, since mountall is able to run it before mounting /
<jtv> (initramfs) pwd
<jtv> //
<jtv> (initramfs) find -name \*fsck\*
<jtv> (initramfs)
<wgrant> Hm, true.
<wgrant> How stupid.
<wgrant> Live CD time :(
* mthaddon changed the topic of #launchpad-dev to: LP in readonly mode 09:00 - 09:30 UTC | Launchpad Development Channel | Week 4 of 10.03 | PQM is open | Release manager: deryck | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
<jtv> I don't suppose shipit has same-day delivery... :)
<wgrant> Heh.
<jtv> A Hardy CD isn't going to be a good bet for fsck'ing ext4, is it?
<wgrant> Eeeeh no.
* mthaddon changed the topic of #launchpad-dev to: LP in readonly mode 09:00 - 10:00 UTC | Launchpad Development Channel | Week 4 of 10.03 | PQM is open | Release manager: deryck | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
* mthaddon changed the topic of #launchpad-dev to: LP in readonly mode 09:00 - 11:00 UTC | Launchpad Development Channel | Week 4 of 10.03 | PQM is open | Release manager: deryck | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
<deryck> Morning, all.
 * wgrant looks for someone who can land five branches
<ctrlsoft> wgrant: happy to
<wgrant> ctrlsoft: Thanks! https://code.edge.launchpad.net/~wgrant/launchpad/refactor-slavestatus/+merge/22285, https://code.edge.launchpad.net/~wgrant/launchpad/move-lots-of-builddmaster-to-queuebuilder/+merge/22009, https://code.edge.launchpad.net/~wgrant/launchpad/bug-540819-fix-builder-list-icons/+merge/22288, https://code.edge.launchpad.net/~wgrant/launchpad/filter-apt_pkg-deprecationwarnings/+merge/22317
<wgrant> But I guess they're un-EC2-able until LP comes back after its enormous downtime.
<ctrlsoft> wgrant: I count only 4 :-)
<wgrant> ctrlsoft: https://code.edge.launchpad.net/~wgrant/launchpad/get-translations-jobs-working/+merge/22584 is the other one, but I excluded that since it needs to be set to Approved, which requires LP to be back in reasonable time.
<wgrant> But it turns out they all need LP to be up, since Codehosting is inexplicably down.
* mthaddon changed the topic of #launchpad-dev to: LP in readonly mode 09:00 - 12:00 UTC | Launchpad Development Channel | Week 4 of 10.03 | PQM is open | Release manager: deryck | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
<wgrant> Ew.
<lifeless> complications
 * wgrant thought it was just meant to be a master swap.
<lifeless> wgrant: yes
* mthaddon changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.03 | PQM is open | Release manager: deryck | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
<wgrant> Thanks mthaddon.
<wgrant> ctrlsoft: So, since Codehosting is back, do you have a moment to fire off those test runs?
<mthaddon> wgrant: sorry we were down/readonly for so long
<ctrlsoft> wgrant: I was about to get some lunch, I'll submit them when I get back
<wgrant> it seems all too frequent these days :(
<wgrant> ctrlsoft: Thanks.
<ctrlsoft> wgrant: still there?
<barry> hi folks.  bug 375313 is a duplicate of bug 341503, and i'd like to mark it as such, but lp prevents me from doing this because the former has a slew of duplicates.  does anybody have a lplib script that rehomes all of a bug's duplicates to a different bug?  if not, i'll write it, but i didn't want to reinvent the wheel.  (maybe i should ask on the ml)
<james_w> barry: in ubuntu-qa-tools I think
<james_w> no, that doesn't seem to be it
<barry> james_w: maybe this would make a good addition to that package?
<james_w> yes
<james_w> the script is somewhere but I can't remember where
<barry> james_w: thanks, i'll ask on the ml.  i was hoping for instant gratification :)
<james_w> http://bazaar.launchpad.net/~ubuntu-bugcontrol/ubuntu-qa-tools/master/annotate/head:/responses/is-duplicate I think
<barry> i think that's the script that bdmurray uses to mark bugs as duplicates, but i don't think it'll work for a bug that itself has duplicates.  i could be wrong though.  bdmurray do you know?
<bdmurray> barry: what's that?
<barry> bdmurray: hi.  "bug 375313 is a duplicate of bug 341503, and i'd like to
<barry>         mark it as such, but lp prevents me from doing this because the former
<barry>         has a slew of duplicates.  does anybody have a lplib script that
<barry>         rehomes all of a bug's duplicates to a different bug?"
<barry> bdmurray: oh, apologies for the ugly paste
<barry> bdmurray: james_w pointed me to is-duplicate in ubuntu-qa-tools, but i'm wondering if it will do that rehoming of dups
<bdmurray> barry: lp-set-dup in ubuntu-dev-tools should do it I think but keep in mind all the mailings people will get
<barry> bdmurray: oh dang, good point. :(
<bdmurray> barry: usually the benefit of merging duplicates in mass quantities isn't worth the cost in my opinion
<barry> bdmurray: it would probably just be better if we could dup 375313 to 341503 even when the latter has dups of its own
<magcius> Is there any chance we can clean up the URLs in Loggerhead? They're really starting to bug me
<magcius> Also, would you mind adding a left margin to where the code starts, or indentation guides? It's really hard to judge indentation with Python code
<sinzui> I know someone meant well by creating Ubuntu series in sampledata, but I can see that the dev site is now in an invalid state because publishing, caching, and packaging information is now missing for all series and project. :(
<james_w> what does the librarian actually do when it starts?
<wgrant> sinzui: That's why we do the series mangling in a separate non-mandatory script.
<sinzui> wgrant, I do not understand your question.
<wgrant> sinzui: The only major Ubuntu-related sample data changes lately are done by the soyuz-sampledata-setup script, which is optional. Or are you talking about long-standing series issues?
<sinzui> I like seeing lucid in dev. but the binary and source counts are 0
<sinzui> wgrant, This is very painful for me because sourcepackage and product UIs reply on sane data.
<wgrant> sinzui: Then don't run the script. We need the data in that state for Soyuz testing, but it doesn't work for Registry stuff.
<sinzui> wgrant, I have reconstructed spph so that there is a count and that the objects again show they are linked.
<wgrant> That's why I rejected its inclusion in current-dev.sql.
<sinzui> wgrant, try developing a UI. You need to see which packages are published. firefox had nothing
<wgrant> sinzui: If your sample data has Lucid, you must have run the Soyuz script after loading the sample data.
<sinzui> wgrant, no, someone else did.
<sinzui> The data was created in March
<wgrant> sinzui: Ew, really?
 * wgrant looks.
<wgrant> That was not intentional.
<sinzui> you cannot create lucid without first creating the series
<wgrant> Argh, you're right.
<sinzui> wgrant, I just committed my spph fix. I would like to make the bpph. I do not know how to do that beyond crafting the sql that I read is required.
 * wgrant finds the rev that did it, replays the DB patches, and regenerates the sample data.
<wgrant> sinzui: No, the correct fix is to revert the mistaken sample data update.
<sinzui> that work for me
<wgrant> It was only two merges ago, fortunately.
<wgrant> ctrlsoft: Thanks for landing all of those.
<wgrant> NCommander: When you updated the current-dev.sql sample data for your changelog branch, you included a whole lot of other changes from the soyuz-sampledata-setup script. Since we need to revert that, how hard is it going to be to reconstruct that sample data update? Did you just apply the DB patch, or did you change other data as well (eg. importing real changelogs)?
<wgrant> And we really should land this to devel :/
<wgrant> Anyone around who can land https://code.edge.launchpad.net/~wgrant/launchpad/filter-apt_pkg-deprecationwarnings/+merge/22317 for me? OCR seems to have vanished.
<ctrlsoft> wgrant: anytime
<NCommander> wgrant: Ugh, I checked to make sure no garbage data went in
<NCommander> wgrant: *sigh*, it should just be a matter of reverting out current-dev.sql to before my commit, then applying all the SQL patches up to that point, then regenerating sample data
<wgrant> NCommander: Great. Doing so now.
<NCommander> wgrant: great, I broke launchpad ;.;
 * NCommander still has to find time to finish the converter
<ctrlsoft> wgrant: did you get 5 success emails ?
<ctrlsoft> wgrant: I only got 4 while I submitted 6 branches in parallel
<wgrant> ctrlsoft: I got four.
<wgrant> ctrlsoft: Did you try to land filter-apt_pkg-deprecationwarnings too?
<ctrlsoft> wgrant: just sent it off to ec2
<wgrant> ctrlsoft: Thanks.
#launchpad-dev 2010-04-03
<wgrant> What's buildbot's problem?
<lifeless> existence?
<wgrant> Well, I had something like four branches in that run, so I'm reasonably interested in its precise failure mode.
<wgrant> I have had a branch pending scanning for more than 14 minutes now.
<wgrant> https://code.edge.launchpad.net/api/devel/~wgrant/launchpad/revert-soyuz-sample-data-changes
<wgrant> This seems like a bad thing.
<wgrant> (https://api.edge.launchpad.net/devel/~wgrant/launchpad/revert-soyuz-sample-data-changes?oauth_consumer_key=nobody&oauth_token= suggests that it isn't DB lag, either)
<wgrant> Finally!
<wgrant> last_mirrored
<wgrant>  
<wgrant>     2010-04-02 23:29:31.302992+00:00
<wgrant>  
<wgrant> last_scanned
<wgrant>  
<wgrant>     2010-04-02 23:45:03.022906+00:00
<wgrant> ctrlsoft: Still around?
<wgrant> I have a testfix branch for the current buildbot failure at https://code.edge.launchpad.net/~wgrant/launchpad/ttbb-slavestatus-testfix/+merge/22733, if anyone is interested.
#launchpad-dev 2011-03-28
<lifeless> thumper: hoot
<lifeless> thumper: *shoot*, or did you want voice? (can do that too)
<thumper> you suggest using closures rather than functors
<thumper> I'm not sure I get the subtleties
<lifeless> mwhudson: I'll get someone else to review; need deeper restructuring to make the concept work
<lifeless> thumper: oh, less code
<thumper> lifeless: and here I was thinking you were hooting about my link in -ops
<lifeless> uses more of the language facilities, which tends to feel more natural
<lifeless> thumper: I linked that here last week :P
<thumper> lifeless: can you give me an example
<thumper> ah... ok
<thumper> well it was new to me :)
<mwhudson> lifeless: ok, thanks for letting me know
<lifeless> so - a class with only __call__ is equivalent to a function
<lifeless> mwhudson: (the symmetric vs asymmetric difference is biting here)
<lifeless> mwhudson: e.g. product can be symmetric, branch.owner can't be, not for merge proposals
<mwhudson> lifeless: oh right yes, duh
<lifeless> mwhudson: kindof reflects some limits in the collections approach
<lifeless> conceptually we might be better off with a merge proposals collection taking two branch collections as constraints, but then the constant folding will become *hard*
<thumper> lifeless: I think I get your suggestion now...
<lifeless> so, it makes me wonder more about whether ditching the layer would just be a net win.
<lifeless> thumper: right - rather than implement N classes + factories, implement N functions, and for some of them use currying (or partial()) to capture whatever constraints you need
<lifeless> wgrant: are you deploying us up ?
<wgrant> lifeless: I shall.
<lifeless> kk
 * mwhudson gets ready to shuffle off
<lifeless> http://pastebin.com/yVLXtDse
<poolie> hi all
* StevenK changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<wgrant> spiv: The package importer was only changed to specify a reviewer three weeks ago, and that may have been deployed even more recently.
<wgrant> lifeless: Actually, we can't really delete the code. Makes it impossible to test things without real DNS.
<wgrant> Like, say, a development environment.
<wgrant> Although I guess we could.
<spiv> wgrant: well, the other curious part is it appears to only or at least mainly affect imports that had been affected by bug 726584, which was a bzr issue
<_mup_> Bug #726584: flash-kernel import fails apparently mismatching maverick and natty branches <hpss> <udd> <Bazaar:Fix Released by spiv> <Ubuntu Distributed Development:New> < https://launchpad.net/bugs/726584 >
<spiv> wgrant: but perhaps it's just more likely to trigger that situation with relatively stale imports
<wgrant> spiv: It only creates MPs when there is a conflict.
<wgrant> And it will affect all of them.
<wgrant> The API usage is wrong.
<spiv> What do you mean by "all of them"?  I don't see the number of import failures rising, so I presume most of them have been passing.
<wgrant> spiv: It will affect anything with an import conflict.
<lifeless> wgrant: how so? we can inject stuff into the client
<lifeless> wgrant: and/or select the specific names we need a priori
<wgrant> lifeless: I guess.
<lifeless> can has review? https://code.launchpad.net/~lifeless/launchpad/bug-736008/+merge/55028
<poolie> is it just my flakey network or is lp refusing ssh connections?
<lifeless> poolie: works for me
<lifeless> (just tested)
<poolie> thanks
<lifeless> spm: he's noticed, you can take the special case for poolie out now
<spm> hmm?
<spm> oh. right. yes. will do.
<lifeless> StevenK: https://code.launchpad.net/~lifeless/launchpad/bug-736008/+merge/55028
<lifeless> StevenK: I know you are around :)
 * StevenK grumbles at lifeless 
<StevenK> lifeless: I note you're adding two methods getCandidateBranchesWith(). There is a little bit of duplicated code there, are you able to up-call from the more explicit one to the anonymous one?
<lifeless> StevenK: no
<lifeless> StevenK: anonymous(generic)  and visible(generic)
<lifeless> the generic class does no filtering by user; the anonoymous one filters as though an anonymous user, the visible as though a user is logged in
<lifeless> I think it should be refactored away from subclassing and into a strategy helper.
<lifeless> nice 477 Time Outs
<StevenK> lifeless: Sounds good, do that.
<StevenK> :-P
<StevenK> lifeless: If you want the refactoring to be done in another iteration, I'm happy enough to say r=me
<lifeless> StevenK: I think it should be done, I have no plans to do it.
<StevenK> lifeless: Okay, then I'd suggest you add an XXX to that effect.
<lifeless> StevenK: I don't think thats useful to do:
<lifeless>  - such XXX become stale all the time
<lifeless>  - its not a 'bug'
<lifeless>  - the refactoring might be wrong, until someone tries, who knows
<StevenK> lifeless: I might just be overly grumpy, but you asked for a review, not an argument.
<lifeless> StevenK: yes, and I appreciate that - you've mentioned two things; one that the *current code structure* isn't optimal and two that I record *one approach* to fixing it in the code base.
<lifeless> for the former, I'm agreeing with you vigorously, but the thing I'm trying to achieve is orthogonal to the current code structure
<StevenK> Then I withdraw my objection, r=me
<lifeless> for the latter, I'm saying that encoding a guess about what might be better in the code base isn't a useful thing to do
<lifeless> StevenK: thanks!
<lifeless> StevenK: http://bazaar.launchpad.net/~lifeless/storm/with/revision/387 - is the change to storm I'm bundling
<StevenK> I think I've read that before
<lifeless> yeah, i pastebinned it, when it was in-tree in lp
<huwshimi> Is it just me or do other people find that a lot of AJAX requests hang in Launchpad? Probably 60% of the time I do something it doesn't work and I have to refresh and do it again.
<wgrant> huwshimi: I've not seen that.
<huwshimi> wgrant: Really?
<huwshimi> wgrant: What browser are you using?
<wgrant> huwshimi: Chromium 11, mostly.
<huwshimi> wgrant: Ah right. I thought it might be a chrome thing, but I guess not
<huwshimi> It
<huwshimi> It's really annoying
<wgrant> huwshimi: Bug task actions?
<huwshimi> wgrant: Yeah
<wgrant> Er.
<wgrant> Is something wrong with OOPS reporting?
<wgrant> Our top two OOPSes have disappeared.
<wgrant> Maybe the broken remote branch is fixed, and the package importer has unbroken...
<wgrant> But hmm.
<spiv> wgrant: the package importer has stopped retrying that failure
<wgrant> spiv: Ah.
<spiv> http://package-import.ubuntu.com/status/: â51 packages failed to many times to retry with key lazr.restfulclient.errors.HTTPErrorâ
<spiv> I can kick it over and over today if you'd like lots more OOPSes :)
<wgrant> Yeah, the 717 critical OOPSes is a bit low. The extra 1400 will do us some good :)
<lifeless> huwshimi: possibly you're hitting lots of timeouts + the poor reporting we have of those class of errors
<wgrant>         1 /    2  BugTask:EntryResource
<wgrant> Not likely.
<poolie> is it just me or are there no longer tick/cross buttons when editing mp descriptions?
<poolie> nm, just me, they probably failed to load
<lifeless> huwshimi: I guess you need to track the requests and see what happens
<huwshimi> lifeless: Yeah next time it happens I'll take more notice :)
<lifeless> wow
<lifeless> https://lp-oops.canonical.com/oops.py/?oopsid=1912O253#statementlog is special
<wgrant> I was looking at that before.
<lifeless> https://bugs.launchpad.net/launchpad/+bug/743970
<_mup_> Bug #743970: ProductSet:+all timeouts <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/743970 >
<wgrant> I don't understand what it's trying to do.
<wgrant> There were some pretty slow queries for what should be a fairly trivial operation.
<lifeless> so the first clause is 'direct teams for person X with membership status 3'
<lifeless> the second clause is 'all active team memberships
<lifeless> the first clause can probably be radically simplified
<wgrant> Well, I didn't look at the queries in depth. I mostly just scanned the log and saw some 800ms queries that didn't make sense for listing products.
<wgrant> So I decided that I might come back to it later.
<huwshimi> My apologies to whoever is getting notifications for all the bug changes I'm making today.
<wgrant> How dare you try to fix our UI.
<lifeless> huwshimi: I'm not sure easy-ui is a sensible tag though :(
<lifeless> huwshimi: if you see js error bugs - per the list thread - they should be criticaled
<huwshimi> lifeless: sure.
<huwshimi> lifeless: It is a bit of a misnomer, but I couldn't think of a better one
<lifeless> huwshimi: what is the intent you want to convey?
<lifeless> wallyworld: hey, web_service_request_to_notification_request - seems to be entirely redundant. Can't you just use the INotificationRequest directly ?
<wallyworld> lifeless: probably. i'm a bit clueless when it comes to zope and adaptors
<wallyworld> i'll give it a go
<huwshimi> lifeless: They should be fairly straight-forward, only take a few hours and not really require much discussion. The sort of thing that can be just picked up and done.
<lifeless> huwshimi: I suggest tagging them easy, ui
<lifeless> huwshimi: a search for easy & ui will bring them back trivially
<huwshimi> lifeless: The problem with that is that implies that they are actually easy. I think I need to pick another word instead of that
<lifeless> huwshimi: shallow trivial simple tech-debt friction polish
<wallyworld> lifeless: i can't use your suggestion because INotificationRequest is lp specific and the implementation code is in lazr restful
<lifeless> wallyworld: you have an adapter registration with a factory line
<lifeless> wallyworld: in the one merge proposal
<lifeless> wallyworld: why can't you put the factory as ....INotificationRequest
<wallyworld> lifeless:  ah. i think i understand now. you mean "factory=canonical.launcpad.webapp.interfaces.INotificationRequest"
<wallyworld> ?
<lifeless> wallyworld: that seems like a trivial inlining
<lifeless> wallyworld: yes
<lifeless> wallyworld: def foo(bar): return quux(bar)    is equivalent to foo=quux, after all
<lifeless> you may be able to do better, but I was just pointing out the direct simplification I thought I saw
<lifeless> lib/lp/registry/browser/product.py makes baby jebus sad
<wallyworld> lifeless: thanks. i'll make the zcml change
<lifeless> wgrant: check line 1014
<lifeless> wgrant: and ask yourself, why would this show up in projects/+all
<wgrant> lifeless: It surely is not invoking ProductView for every project.
<lifeless> wgrant: :) :) :) :)
<lifeless> configure.zcml 1434
<lifeless> +listing-detailed is the culprit
<wgrant> Yay
<wgrant> Hmm.
<wgrant> Damn.
<wgrant> I had merged MixedFileAliasView into LibraryFileAliasView.
<wgrant> But that's not safe.
<wgrant> Because we may leak restricted LFAs in various places, which is safe at the moment because the code needs to explicitly ask for the token view.
<lifeless> yeah
<lifeless> btw
<lifeless> I *thought* I saw lp asking for appserver verification of public bug attachments or something the other day
<lifeless> that could be streamlined
<wgrant> Right, ProxiedLibraryFileAlias always goes to the appserver first.
<lifeless> for public files, we should bypass that
<wgrant> What if they become private?
<lifeless> will make branding load faster
<lifeless> wgrant: they they will get a 404
<wgrant> What? Branding doesn't go through ProxiedLibraryFileAlias.
<lifeless> wgrant: mmm, perhaps, I could swear it was
<lifeless> anyhow, there are two classes of links
<lifeless> current and persistent
<lifeless> for bug mail using the appserver redirect is appropriate
<wgrant> Right.
<lifeless> for web pages rendered, its very unlikely that the bug will go private before the user clicks - and if it does, shrug.
<lifeless> the race exists regardless because we don't hand out tokens for public files
<lifeless> the bug might go private between asking for the real url and actually requesting the content from the librarian
<lifeless> wgrant: different thing than what you're doing
<wgrant> I know.
<lifeless> wgrant: I only mention it so you can bear it in mind in any refactoring you do.
<thumper> wallyworld: you realise that Javascript doesn't have method overloading?
<wallyworld> thumper: :-( i thought it did
<thumper> wallyworld: nope, it is more like python
<wallyworld> thumper: you mean the recipe text xss branch?
<thumper> yep
<wallyworld> ok. will do a fix
<thumper> wallyworld: just check for undefined
<thumper> wallyworld: if someone calls a function with one param when it takes two, the second is undefine
 * wallyworld nods
<thumper> d
<thumper> wallyworld: did you want a voice catchup?
<wallyworld> thumper: ok.
 * wallyworld plugs stuff in
<thumper> wallyworld: with no leonardr now, we can do it at a more convenient time for you
<wallyworld> team of 2
<wallyworld> :-)
<wallyworld> thumper: i can hear you
<thumper> I no hear you
<lifeless> and yay - prejoining fail again
<wallyworld> thumper: not muted but no sound
 * wallyworld restarts :-(
<thumper> mic plugged in?
<wallyworld> thumper: yes!
<thumper> wallyworld: time to try something different?
<thumper> wallyworld: mumble doesn't seem to be working for you, how about skype?
<wallyworld> thumper: just testing with skype now and it works fine
<lifeless> anyone know why cve:+index has edit widgets for the bug tasks?
<lifeless> can has review? https://code.launchpad.net/~lifeless/launchpad/bug-743970/+merge/55033
<lifeless> StevenK: ^ its tiny, if you feel like it.
 * lifeless will bbiab
<lifeless> StevenK: thanks!
<StevenK> wgrant: I think I've come up with a tiny fix for python-debian's permissiveness
<wgrant> StevenK: Oh?
<StevenK> wgrant: http://pastebin.ubuntu.com/586329/
<wgrant> StevenK: Looks reasonable.
<lifeless> ok, why can't I ssh to ec2...
<StevenK> wgrant: Bleh, it isn't quite that simple. :-(
<StevenK> lifeless: What does ssh -v say?
<lifeless> debug1: Connecting to ec2-75-101-221-93.compute-1.amazonaws.com [75.101.221.93] port 22.
<StevenK> I don't think your instance is up
<lifeless> Instance i-c201daad starting.......................................... started on ec2-75-101-221-93.compute-1.amazonaws.com
<lifeless> Started in 3 minutes 37 seconds
<StevenK> I think the default security group allows ICMP
<lifeless> couldn't ping it either, and it should be in the ec2test group
<StevenK> lifeless: Do you have ElasticFox?
<StevenK> That should allow you to see what Amazon thinks about this
<spiv> StevenK: "ka-ching", I expect
<StevenK> spiv: Two drums and a cymbal fall off a cliff
<huwshimi> StevenK: I thought it was two elephants
<StevenK> huwshimi: Not heard that one
<StevenK> (Okay, that was bad)
<lifeless> I think its a routing issu
<lifeless> *issue*
<lifeless> http://developer.amazonwebservices.com/connect/entry.jspa?externalID=609&categoryID=88 fails to connect too
<StevenK> lifeless: Hm, works for me
<lifeless> just came good, will see whats up
<lifeless> in the mean time
<lifeless> would someone like to ec2land https://code.launchpad.net/~lifeless/launchpad/bug-743970/+merge/55033 and https://code.launchpad.net/~lifeless/launchpad/bug-736008/+merge/55028 ?
<lifeless> ulp, nevermind
<lifeless> it really is happier now
<wgrant> Can I tell a testbrowser not to follow redirects?
<StevenK> wgrant: Sadly, it wasn't that simple. But http://pastebin.ubuntu.com/586340/ makes it work
<StevenK> wgrant: So, do you think resolved DSDs should lose the diff?
<wgrant> StevenK: I'm not sure.
<StevenK> wgrant: I was going to get DSD.update() to lose them if the status is NEEDS_ATTENTION so they can be re-requested
<StevenK> Has anyone seen http://pastebin.ubuntu.com/586348/ ? Or should I fix it?
<wgrant> Didn't I see that in one of your branches/diffs last week?
<StevenK> A diff
<StevenK> I'm just surprised it's gone unfixed
<wgrant> Could I convince someone to look at an oversized branch?
<wgrant> 948 lines (+108/-606)
<wgrant> https://code.launchpad.net/~wgrant/launchpad/delete-librarian-streaming/+merge/55039
<StevenK> wgrant: r=me, with a niggle
<StevenK> https://code.launchpad.net/~stevenk/launchpad/apihelpers-imports/+merge/55040
<wgrant> StevenK: Bah, yes, people fail at sorting imports.
<StevenK> Haha
<wgrant> The test fallout from this could be *amusing*.
<StevenK> Oh, NFS, DIALCF.
<wgrant> What's it doing?
<wgrant> It can't be as flaky as nfsv4+krb5.
<StevenK> % ls -lh /media | tail -n 1
<StevenK> d????????? ? ?      ?         ?                ? media
<wgrant> Well then.
<StevenK> I finally managed to find an operation that actually told me what was going on, which was Stale NFS file handle
<StevenK> Except now the client thinks it's mounted, but the server and client are playing no-speakies. Awesome.
<StevenK> % sudo start portmap
<StevenK> start: Job failed to start
<StevenK> Could I have a clue, Upstart? Please?
<lifeless> StevenK: they cost more
<StevenK> lifeless: Hmph.
<StevenK> lifeless: Can haz review?
<lifeless> sure
<StevenK> lifeless: https://code.launchpad.net/~stevenk/launchpad/apihelpers-imports/+merge/55040 if you missed it
* henninge changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: henninge | https://code.launchpad.net/launchpad-project/+activereviews
<henninge> Good morning!
<wgrant> Morning henninge.
<henninge> Hi wgrant! ;)
<StevenK> henninge: Hai!
<henninge> And hello StevenK! :)
<StevenK> henninge: You guys switched to daylight savings on the weekend?
<henninge> Yes we did.
<StevenK> Means we probably lose ours in about a week.
<henninge> oh
<stub> This should be fun - I've got a branch that consistently fails with KeyboardInterrupt
<StevenK> stub: Stop bashing Ctrl-C during the test, then? :-P
<stub> StevenK: The keyboard is in another continent ;)
 * wgrant stabs germanium.
<wgrant> Stop being so slow.
 * wgrant stabs mizuho too.
<StevenK> wgrant: Same reason, or a different one?
<wgrant> StevenK: Same one.
<StevenK> germanium has an excuse, I didn't think mizuho did.
<wgrant> Maybe the DC just hates me.
<wgrant> I can't get more than 100KB/s from mizuho.
<wgrant> (substantially less from germanium. and request latency is terribly)
<StevenK> What's your path to the DC?
<wgrant> Optus->L3(US)->Canonical
 * spm leaves the "throttle wgrant" block in place for another day
<wgrant> :(
<spm> wgrant: have you done any sniffs locally and seen the effect on a Time Sequence graph?
<wgrant> A hwhat?
<spm> woooo! I know something wgrant doesn't. /first!
<wgrant> :(
<spm> http://www.tcptrace.org/manual/node12_mn.html
<wgrant> Thanks.
<spm> my #1 go to tools for 1st step debuging funky network crap
<spm> ethereal (or whatever they call it this week) has a not as useful version of that
<StevenK> wireshark
<spm> that's it. ta.
<StevenK> And it hasn't been renamed for like 2 years?
<wgrant> More.
<StevenK> More proof spm is just old
<wgrant> almost 5 years
<wgrant> May 2006
<wgrant> Wow.
<StevenK> Crumbs
<wgrant> Doesn't seem like that long ago :/
<StevenK> Haha
<spm> if you give me a bit, I can probably even tell you it's earlier name. and find a binary.
<StevenK> For which Unix? :-P
<spm> heh, I was actually looking for the windows version... /blush
<StevenK> Hah, fail
<spm> wgrant: so the trick you're looking for - is your tcp window saturated; or are you getting lots of normal flow then big gaps; multiple acks can be bad; etc. it depends.
<spm> also, getting a similar view off a different spot can be helpful. I've used my VPS in the USA to debug server woes in Oz, completely different network path and that can eliminate some imponderables.
<wgrant> From my US VPS it's fine.
<spm> I'd blame your ISP then. ;-)
<StevenK> wgrant: Oh, you also have a Linode?
<wgrant> I guess I don't grab much other big stuff from Europe.
<wgrant> StevenK: Of course!
<StevenK> wgrant: Fremont?
<wgrant> Indeed.
<StevenK> wgrant: Win
 * StevenK looks up his host
<StevenK> fremont35
<wgrant> I seem to be on fremontreallyslowio
<StevenK> Haha
<wgrant> aka fremont87
<StevenK> Haha
<StevenK> wgrant: If it's a problem, open a ticket
<wgrant> It's not really an issue at the moment.
<StevenK> lifeless: SIGH! I've not been able to self-review before, but thanks.
<StevenK> lifeless: Should have realised
<wgrant> Huh, it looks like -backports is going to be usable in natty.
<jkakar> thumper: Still there?
<nigelb> Would a server editiion Ubuntu be alright for lp install?
<StevenK> henninge: I have one for you -- https://code.launchpad.net/~stevenk/launchpad/dsd-lose-diffs/+merge/55059
<wgrant> nigelb: No, it only runs on Windows Server.
<stub> bzrlib.errors.InvalidHttpResponse: Invalid http response for https://code.launchpad.net/~stub/launchpad/trivial/%2Bmerge/54692/.bzr/branch-format: Unable to handle http code 405: expected 200 or 404 for full response.
<nigelb> wgrant: Har har
<wgrant> stub: bazaar.launchpad.net, not code.launchpad.net?
<stub> wgrant: This is from ec2 land
<nigelb> wgrant: I didn't want to kill my processor with a full install
<stub> Gah... pebkac
<huwshimi> rvba: Morning
<rvba> huwshimi: morning
<huwshimi> rvba: I have some icons for you
<huwshimi> rvba: Well if they're ok
<rvba> huwshimi: nice!
<StevenK> henninge: I have one for you -- https://code.launchpad.net/~stevenk/launchpad/dsd-lose-diffs/+merge/55059
<nigelb> wgrant: In revenge, I'll pick your brains, when I finish the installation.
<henninge> StevenK: looking
<wgrant> nigelb: Sounds like a good idea.
<StevenK> nigelb: Feel free to ask anyone, rather than just picking on wgrant?
<wgrant> That could work too.
<nigelb> StevenK: Sure :)
<henninge> StevenK: Shouldn't you also test for parent_package_diff being None?
<StevenK> henninge: I can, if you wish
<StevenK> henninge: http://pastebin.ubuntu.com/586364/
<henninge> StevenK: Looks good. Why did you chagne to Equal?
<StevenK> henninge: Because it was bugging me before I asked for the review
<henninge> StevenK: but why only on the second assert?
<StevenK> henninge: Because that one was assertTrue(... is None) which is effectively Equal. The former is assertFalse(... is None)
<wgrant> StevenK: assertIs(None, foo)
<wgrant> Not assertEquals.
<StevenK> Fixed
<henninge> Yes, that's what I would have expected.
<wgrant> And assertIsNot
<StevenK> henninge, wgrant: http://pastebin.ubuntu.com/586365/
<henninge> StevenK: r=me
<StevenK> henninge: Thanks!
<henninge> thanks wgrant ;)
<StevenK> jelmer: Ping!
<jelmer> StevenK: goooood morning
<StevenK> jelmer: Hai!
<adeuring> good morning
<henninge> Moin adeuring!
<adeuring> hi henninge
<StevenK> jelmer: Did you know the Version class in debian.changelog isn't strict enough?
<jelmer> hi henninge, adeuring
<jelmer> StevenK: No, I didn't. In what regard is it not strict enough?
<henninge> Hey jelmer ;)
<adeuring> hi jelmer
<StevenK> jelmer: The regex allows : in the upstream character class -- that is only allowed if the epoch is not None. I have a patch at http://pastebin.ubuntu.com/586340/
<jelmer> StevenK: ah, nice catch
<StevenK> jelmer: Okay, so you like my patch. :-) What's the next step?
<jelmer> StevenK: a test would be nice, and sending it upstream
<StevenK> jelmer: Just file a bug in the Debian BTS?
<jelmer> StevenK: Yep; we should probably poke James or one of the other devs to have a look, my last patch was sent to the mailing list but hasn't seen much attention.
<jelmer> (I'm not upstream)
<StevenK> jelmer: Ah, I thought you were, sorry.
<StevenK> jelmer: How do I run the test suite?
<StevenK> Ah, there's even a Makefile
<jelmer> StevenK: Yep, that's specific to Launchpad's friendly fork though
<StevenK> It helped me check the test I wrote, so win
<bigjools> morning all
<wgrant> Morning bigjools.
<LPCIBot> Project windmill build #110: STILL FAILING in 1 hr 8 min: https://lpci.wedontsleep.org/job/windmill/110/
<wgrant> bigjools: Do you have an opinion on bug #740892?
<_mup_> Bug #740892: Ownership of package sets doesn't get preserved when a new series gets initialized <Launchpad itself:Triaged> < https://launchpad.net/bugs/740892 >
<bigjools> wgrant: I agree with your last bug comment.  Also, he's not stated exactly why changing the owner is a problem.
<wgrant> bigjools: Because the DMB is given several of the packagesets.
<wgrant> bigjools: So they can administer them without involving the TB for every single thing.
<bigjools> ok
<bigjools> is DMB anywhere on the distro data?
<wgrant> No.
<bigjools> hm
<wgrant> The TB should be, but the DMB probably shouldn't be.
<bigjools> in that case they need to set it manually
<wgrant> Why?
<bigjools> hmm actually
<bigjools> we can preserve it for the same distro, change it for new distros
<wgrant> That's what I was suggesting, yeah.
<wgrant> I think that should work.
<bigjools> cool
<wgrant> bigjools: Also, archivepublisher's tests suck.
<bigjools> wgrant: tell me something I don't know
<wgrant> bigjools: I was changing Release file generation, and it looks like it's only slightly tested in soyuz-upload.txt and nowhere else.
<wgrant> Sigh.
<bigjools> blame celso :)
<wgrant> Mmm. I think it's probably mostly just old, complex code. Other bits of LP were also pretty bad.
<deryck> Morning, all.
<henninge> Hi deryck!
<mrevell> Hello everyone. Has anyone here tried working through the getting started guide for daily builds?
<wallyworld> deryck: hi. you tried to run windmill tests with firefox 4? the Launchpaduser.ensure_login() method fails for me. "current_url = client.commands.execJS(code='windmill.testWin().location;')['result']['href']" fails
<deryck> wallyworld: ah, no, I haven't tried it with 4 yet.
<deryck> wallyworld: that sucks :(
<wallyworld> deryck: yeah :-( i don't want to reinstall ff3
<deryck> wallyworld: is the site actually broken in 4?  Or the test just fails?
<wallyworld> deryck: the site/page loads but the javascript inside the call to ensure_login() fails
<wallyworld> deryck:  client.commands.execJS(code='windmill.testWin().location;') returns a near empty dict
<wallyworld> i mean the 'result' value is a near empty dict
<deryck> wallyworld: ah.  hmmm, let me look more closely at this now....
<wallyworld> hence there is no 'href' key and it barfs
<wallyworld> deryck: thanks. i had a quick look at the windmill devs google group but nothing jumped out. if now is not convenient for you to look, that's fine. it's way past eod for me  and i was hoping to take a shortcut in asking you :-)
<deryck> wallyworld: yeah, I don't know without looking.  Sorry.  I don't mind digging in the innards of the windmill js objects today.
<deryck> wallyworld: it might also be better to see if we can get the url in an easier way.
<wallyworld> deryck: np. i'll look tomorrow also. thanks
<danilos> henninge, hi, I've got a very nice branch that requires your both reviewer and ui expertise :)
<danilos> henninge, https://code.launchpad.net/~danilo/launchpad/bug-740640/+merge/55123
<danilos> henninge, most of the UI visible in https://devpad.canonical.com/~danilo/screencasts/inline-validation-error.ogv
<henninge> danilos: I am watching it right now ;)
<henninge> cool
 * henninge replays
<danilos> heh :)
<deryck> wallyworld: np!
<danilos> henninge, fwiw, I am unsure about "select all" after re-focusing on the field with the problem
<jml> wgrant: have to say again that I really appreciate you making the PQM step much faster. It makes a big difference to landing branches.
<lifeless> jml: quick chat?
<jml> lifeless: sure.
<lifeless> skype? voip? mumble is still very crackly
<wgrant> jml: Indeed. It is pretty nice.
<danilos> henninge, and whoops, lint issue due to code I should have removed (an unneeded nested function definition)
<henninge> danilos: np, just push it
<danilos> henninge, done already :)
<lifeless> jml: ^
<henninge> danilos: wow, you're quick!
<jml> lifeless: skype is trying to connect as we speak
<lifeless> s/as we/so we can/ :P
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #497: FIXED in 5 hr 16 min: https://lpci.wedontsleep.org/job/db-devel/497/
* benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: henninge, benji | https://code.launchpad.net/launchpad-project/+activereviews
<jam> I have a question about config overlays, anyone feel expert enough to help out?
<jam> (basically, I'm trying to create a custom development config, without having to actually modify the source tree, so I can run LP in a VM)
<jml> I'm getting 'cat: lib/canonical/launchpad/icing/yui/dom/dom-style-ie.js: No such file or directory' errors when running 'make schema' in db-devel
<henninge> danilos: is the UI review for the whole dialog?
<jam> jml: rm -rf lazr-js/build ; make ?
<jam> I've had a lot of problems there
<danilos> henninge, nope
<jam> may not have anything to do with you
<danilos> henninge, just the validation bits :)
<henninge> ok
<jam> but the build script seems to assume the targets must exist
<jam> if the dir does
<jam> jml: though that specifically sounds like somebody forgot to add a file
<jml> jam: I'll run 'make clean', see what happens.
<jkakar> Sometimes I think bug descriptions should be optional... so many of the bugs I file have a descriptive enough subject so I just end up writing 'Yep, we need this' as the description.
<jam> jkakar: you must have really shallow bugs :)
<jkakar> jam: Sometimes, yes... For example, I just filed one like 'Need a way to migrate foo's from the old schema to the new one'.
<jkakar> jam: There isn't really anything more worth describing... I guess the issue is that I'm using bugs, in this and possibly the common case, to track work as opposed to for describing a problem with symptoms, workarounds, etc.
<jam> you could still put context
<jam> where it is useful
<jam> etc
<jam> but sure, I see your point
<jkakar> jam: Yes, for sure.  It's important to provide context when, er, it's important.  I just find I end up filing a lot of bugs with 'Yep, we do' as the description... maybe I'm just being too lazy.  Hmm.
<jam> jml: quick twisted question. If you call 'addCallbacks' on a deferred, it will trigger the call immediately if the data is already present, right?
<jml> jam: yes.
<jam> I'm trying to track through an lsprof, though twisted seems to destroy the call tracking
<jam> but 50% of the time is in "addcallback" I figure that is just actually running the function
<jml> jam: http://paste.ubuntu.com/586438/
<jam> jml: sure. In this case I'm dealing with Launchpad xmlrpc stuff, which I don't think is actually deferred
<jml> jam: tbh, I've never had to seriously profile Twisted code. #twisted will probably have someone who has.
<jml> jam: it uses Deferred, at least on the client side
<jam> jml: It is guaranteed to be exactly as synchronous as the passed-in  proxy.
<jam> but since the trigger is happening under the callback, I'm pretty sure it is synchronous
<jml> it's synchronous, but it uses Deferred
<jam> jml: right. But I think the point of abstracting via Deferred is because they wanted to allow async (at least, avoid blocking)
<jml> yeah.
<jam> ATM, I'm thinking XMLRPC might be killing some of the codehosting perf
<jam> given that the request is sync
<jam> and is *blocking* everyone else at the same time
<jml> the sftp server uses exactly the same code, but uses an async client
<jml> hmm
<jml> jam: I'd really doubt that
<jml> jam: the XMLRPC takes place in the server for auth, and that uses an async xmlrpc client
<jml> jam: then the spawned bzr processes use a sync client, and that wouldn't block anyone else.
<jam> jml: well, running "initialize_on_transport_ex" on my VM is 30ms, under launchpad's implementation, it is 500ms, on Prod, we saw it ~3s
<jam> jml: you're right about probably not blocking others
<jam> so far, I've only been able to track it to the twisted layer. It also looks like handling exceptions is extra expensive
<jam> because twisted opportunistically expands the current stack
<jam> to give 'nice tracebacks'
<jam> but --lsprof lies, too :)
<bac> allenap: sorry i didn't get to your big branch on friday
<jml> jam: interesting.
<jam> https://bugs.launchpad.net/launchpad/+bug/740759
<_mup_> Bug #740759: creating bzrdir on launchpad is slow (BzrDirFormat.initialize_ex_1.16) <codehosting-ssh> <hpss> <lp-code> <performance> <Launchpad itself:Triaged> < https://launchpad.net/bugs/740759 >
<jam>  29 0 0.4081 0.2940 twisted.python.failure:416(__getstate__)
<jam> out of ~0.6s for run_argv
<jam> so 400+ms spent in failure.__getstate__
<jam> again, lsprof sometimes lies, and makes things that are super fast in locals slow while traced
<jam> jml: but 29 calls to __getstate__ generated 35k calls to safe_repr
<jml> jam: that seems an unreasonably large number.
<jam> jml: it walks self.frames, and calls safe_repr() across all objects it finds
<jam> AFAICT
<jam> so, walk the current stack
<jam> and safe_repr every live object available
<jml> jam: as an experiment, you could patch out cleanFailure to do nothing, and then see what affect that has on the run time.
<jam> jml: something worth trying, I was going to try to just avoid exceptions-as-return-codes, but I'll start with your idea
<jam> of course, the flip side is that it probably starts leaking references to other objects
<jml> jam: well, yes.
<jam> since it no longer indirects via safe_repr
<jam> a lot better under lsprof, trying without
<LPCIBot> Project windmill build #111: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill/111/
<jam> jml: http://paste.ubuntu.com/586444/
<jam> not 100% definitive, but averaging <300ms vs >450ms
<jam> for just changing "cleanFailure: return"
<jml> jam: yeah, interesting.
<jml> jam: I'm not sure if we could tolerate the memory leak on production. On the one hand, the processes have a fairly short lifespan, on the other, bzr is already very memory hungry.
<jam> jml: yeah, I'm not thinking that is a production change. But finding some way to avoid calling it at all would be worthy
<jam> It seems like it is about "nice failure tracebacks"
<jam> which is great, but costing us half our runtime isn't so nice
<jml> jam: a version of Failure which didn't keep the traceback would be interesting and maybe even generally useful.
<jam> jml: how would you inject it, though?
<jam> (tell the Deferred it really wanted to use *this* class of failure)
<jml> jam: you would have to monkey-patch.
<jml> jam: i.e. it's not supported
<jam> jml: yipee !
<jml> jam: another option would be to have a flag on Failure like 'debug', except that it said "don't store tracebacks at all"
<allenap> bac: No worries, it is oversize and I wasn't able to stay around.
<abentley> adeuring: Have you made any further changes that I should merge?
<adeuring> abentley: not yet
<abentley> adeuring: cool.
<danilos> henninge, hi, how's the review going? anything I can help clarify perhaps?
<henninge> danilos: sorry for the delay, I got a bit distracted ...
<danilos> henninge, no worries, as long as you don't give up on me :)
<jam> jml: if I change Failure to not walk the stack frame and traceback frame, it drops all the way down to 30ms like I expected
<jam> Twisted Failure is just *way* more expensive than Exception
<jml> jam: :(
<jam> jml: Failure starts by walking the traceback and stack frames and copying all global state
<jam> well, the state leading to local
<jam> but that is locals.copy() and globals.copy()
<jml> jam: this sounds like something that ought to be raised on twisted-python@twistedmatrix.com.
<henninge> danilos: err, all those tests ...
<henninge> danilos: they are quite repetitive, aren't they?
<danilos> henninge, indeed they are, but I was lazy to factor them out more nicely :)
<danilos> henninge, if you really insist, though,... :)
<henninge> danilos: I think I do. :-P
<danilos> henninge, oh well, that's the life of a developer... I mean sure thing! :)
<henninge> danilos: also, this looks very likely to break:
<henninge>  196	+            field_container.get('parentNode').get('parentNode').setStyles(
<henninge> 197	+                {height: 'auto'});
<danilos> henninge, well, what I actually need there is to trigger a recalculation of height (it is already set to 'auto'): any suggestions on that front?
<jam> jml: so I was wrong it my measurements, it is only 183ms. So the big cost is cleanFailure but __init__ is a little expensive, too.
<danilos> henninge, when I add a new element, it doesn't properly resize
<henninge> that's bad :(
<danilos> henninge, yeah, it is, and this was the only solution I could come up with
<henninge> danilos: but my issue was more about finding the right element to apply the style to.
<danilos> henninge, I know, but I am using the opportunity to pick your brain as well :)
<danilos> henninge, anyway, how would you suggest to do it? actually get the ID for the element in question?
<henninge> or the class, maybe?
<henninge> you could have a clase "resizeme" or similar.
<henninge> and you could simply apply the style to all of them (because there are two in the overlay).
<henninge> It won't hurt where it is not needed.
<danilos> henninge, I believe CSS selectors don't have a nice "find me a parent which has 'resizeme' class on it", which is why I didn't bother
<danilos> henninge, as far as not hurting where it's not needed, so much is true with how it is already :)
<henninge> danilos: I did not mean to search for a parent of the input by class but for a child of the overlay.
<henninge> Y.all(overlay_id + ' class=sizeme')
<henninge> I am not sure that selector is correct, though.
<danilos> henninge, right, I don't like the implicit "it might be any 'resizeme' element in the overlay"
<henninge> hence the "won't hurt" comment from me ... ;-)
<danilos> henninge, well, it should be Y.all(overaly_id + ' .sizeme')
<henninge> rightg
<henninge> t
<danilos> henninge, and hence my reply on that :)
<henninge> just my suggestion
<danilos> henninge, anyway, if you think that's better, I am fine
<danilos> henninge, in general, I don't like any of the solutions too much, because it's basically a hack to work-around an observed problem
<henninge> it's "possible side effects" vs. "fragility"
<danilos> henninge, "pick one"? :)
<danilos> henninge, anyway, sure, I'll go with your suggestion (or something along those lines)
<henninge> cool
<jam> jml: I was blocked from posting to twisted. Should I really subscribe for this?
<jam> Or can I send you the email and you post it for me.
<jam> I went ahead and subscribed
<jml> jam: asking on #twisted, everyone seems to like the idea of a version of Failure that implemented the same API but minimally, perhaps based on whether debug is set.
<jml> jam: filing a ticket & including benchmarks would help, or maybe commenting on http://twistedmatrix.com/trac/ticket/2466
<jam> jml: of course, I think that requires an account as well :)
<jml> jam: yes. technology is terrible.
<jml> jam: oh right, just saw your email to twisted-python
<danilos> benji, hi, henninge was already looking at https://code.launchpad.net/~danilo/launchpad/bug-740640/+merge/55123 (I noticed that you claimed it)
<benji> danilos: ah, thanks for letting me know
<henninge> benji: sorry, forgot to claim it.
<benji> henninge: np, I've reassigned it
<jml> jam: also, dropping our sftp support would let us re-write the lpserve thing without Deferreds
<jml> jam: I'd be OK with that, I think.
<henninge> danilos: what's this for? I don't see it appearing when I use the dialog.
<henninge> > +        overlay.showError(
<henninge> > +            'Value for ' + error_text + ' is invalid.');
<danilos> henninge, when you try to save with the errors still present, it should show up at the bottom of the overlay
<danilos> henninge, save == "submit checkbox"
<henninge> duh
<henninge> thanks ;)
<henninge> danilos: code review done.
<henninge> danilos: both reviews done, r=me
* henninge changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: benji | https://code.launchpad.net/launchpad-project/+activereviews
<danilos> henninge, thanks
<jml> sinzui: hi
<sinzui> hi jml
<jml> sinzui: we said we were going to have a call.
<jml> sinzui: but I forgot. got time for a quick call now?
<sinzui> yes. I am on mumble now
<sinzui> I do
<LPCIBot> Project windmill build #112: STILL FAILING in 1 hr 1 min: https://lpci.wedontsleep.org/job/windmill/112/
<adeuring> abentley: I have new version of the branch with the "set branch links" for the translation sharing settings page
<abentley> adeuring: Cool.
<abentley> adeuring: by "new version", do you mean that it's a new branch, or just new revisions on the old one?
<adeuring> abentley: a new version of the existing branch
<adeuring> abentley: lp:~adeuring/launchpad/js-translation
<adeuring> abentley: The links are now quite often hidden
<adeuring> for anon users and non-privileged users
<adeuring> and when no packagaing links exists
<abentley> adeuring: I see.
<abentley> adeuring: It looks like you don't need to import getMultiAdapter in browser/sourcepackage.py.
<adeuring> abentley: gahh, right
<abentley> adeuring: same for ILink.
<adeuring> rigtt
<abentley> adeuring: Also, I think you don't mean "If a product ls linked", but "If a product is linked" in edit_branch_link docstring.
<adeuring> yes ;)
<abentley> adeuring: could you please submit this branch for review, but not land it?  I'll land it as part of this work.
<abentley> adeuring: actually, if you want to land it, that's fine, too.
<adeuring> abentley: ok, I'll submit it.
<abentley> adeuring: thanks.
<abentley> deryck[lunch]: chat (when you're back) ?
<jcsackett_> benji: can i request a review of https://code.launchpad.net/~jcsackett/launchpad/affects-before-bug-email/+merge/55184 ?
<benji> jcsackett_: sure
<jcsackett> benji: thanks!
<benji> jcsackett: done
<jcsackett> benji: my, but you're speedy. thanks again. :-)
<benji> :)
<deryck> abentley: have call with flacoste now, but can chat after that.
<abentley> deryck: cool.
<lifeless> moin
<lifeless> gary_poster: hi
<gary_poster> lifeless, hi.  thank you for email.  was very good.  I've been doing allhands review stuff today so have not gotten to that except for reading it.  I am going to review what I've done in light of what you said also
<lifeless> gary_poster: kk
* lifeless changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: benji | https://code.launchpad.net/launchpad-project/+activereviews
<lifeless> statik: ping
<deryck> hi abentley.  I can mumble now if you wants to.
<abentley> deryck: cool.
<lifeless> flacoste: yo
<flacoste> hi lifeless
<flacoste> skype time?
<lifeless> yarp
<nigelb> 3% rocketfuel
<nigelb> I guess its a good idea to run it overnight
<LPCIBot> Project windmill build #113: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill/113/
<wallyworld> thumper: mumble?
<thumper> wallyworld: aye
<lifeless> deryck: http://people.canonical.com/~huwshimi/ajax_time_log.ogv
* benji changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<deryck> Later on everyone.
<thumper> sinzui: still around?
<sinzui> thumper: yes
<thumper> mumble?
<sinzui> thumper: https://bugs.launchpad.net/launchpad/+bug/417100
<_mup_> Bug #417100: ajax picker for distro source packages needs to display better info <lp-registry> <packages> <search> <Launchpad itself:Triaged> < https://launchpad.net/bugs/417100 >
<sinzui> thumper: bug 618366 and  bug 736014
<_mup_> Bug #618366: Question:+huge-vocabulary is often slow (10% of requests) <lp-answers> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/618366 >
<_mup_> Bug #736014: BugTask:+huge-vocabulary timeout <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/736014 >
#launchpad-dev 2011-03-29
<lifeless>  wallyworld ping
<wallyworld> lifeless: hello
<lifeless> wallyworld: we're blocked on qa for Revision 12671 can not be deployed: needstesting -
<wallyworld> lifeless: thanks. i'll look now
<lifeless> wallyworld: thanks!
<lifeless> StevenK: you have one as well - or is it behind a featureflag?
<wallyworld> lifeless: on qastaging it has a problem which is makes deployment a no go (not a blocker locally). it should be fixed by a branch already in ec2. can we wait till it goes though?
<lifeless> wallyworld: so it would break prod ?
<wallyworld> lifeless: yes
<lifeless> tag it bad-commit-NNNN
<wallyworld> ok
<lifeless> I've fine with waiting for the followup branch thats already playing
<benji> bac: I think the test name changed; to run all the tests I've been using just "xvfb-run ./bin/test --layer=RegistryWindmillLayer -c"
<benji> oops, wrong chan
<huwshimi> lifeless: It appears we can't get oops info for javascript at this point either. Would you be happy with me trying to land the ajax log branch as is, and then we can add the url and oops as an improvement when they become available?
<lifeless> huwshimi: I love incremental improvements
<lifeless> huwshimi: we don't get the http headers?
<huwshimi> lifeless: You mean for the url or the oops?
<lifeless> huwshimi: the oops - its a response header
<huwshimi> lifeless: We get the headers, but I didn't see anything about an oops...
<lifeless> huwshimi: its an X header when an oops occurs
<huwshimi> lifeless: Let me take another look
<lifeless> I'm just digging up the header id
<lifeless> lib/lp/app/javascript/client.js:690:                if (o.getResponseHeader('X-Lazr-OopsId')) {
<lifeless> lib/lp/app/javascript/client.js:691:                    server_error = server_error + ' OOPS ID:' + o.getResponseHeader('X-Lazr-OOPSid');
<lifeless> huwshimi: ^
<huwshimi> lifeless: OK great
<huwshimi> lifeless: Ugh, ok you're right, it's there
<lifeless> for extra credit
<lifeless> clicking on the oops id could take one to pad.lv/OOPS-xxxxx
<lifeless> totally optional of course
<huwshimi> lifeless: It's trivial to add
<huwshimi> lifeless: I've added all this under the "visible_render_time" flag that you added. Is that ok or is it better to have a separate flag for this?
<lifeless> huwshimi: thats ideal
<lifeless> huwshimi: simplest thing, can always refactor
<huwshimi> lifeless: OK great. Do you want to review this branch?
<lifeless> huwshimi: I'm not a gun js guy, but I'll certainly give it a shot
<huwshimi> lifeless: Sure. Thanks
<lifeless> whiskers are so ticklish
<wgrant> lifeless: Let me translate: "feed me"
<lifeless> wgrant: more 'you make a nice bed'
<lifeless> face aimed at my face
<lifeless> wow
<lifeless> cve:+index templates are smoking something
<wgrant> lifeless: Most old templates are :/
<lifeless> no really
<wgrant> eg?
<lifeless> show the bug
<lifeless> format a link to it
<lifeless> then include the bugtasks-and-nominations-view for the bug
<huwshimi> lifeless: https://code.launchpad.net/~huwshimi/launchpad/ajax-time/+merge/55257 for when you get to it.
<lifeless> wgrant: ^ - I'll mentor you
<wgrant> lifeless: k
<wgrant> :( structure
<wgrant> huwshimi: You can't put that JS somewhere other than template, with a call in the template to activate it?
<wgrant> JS in templates makes me very sad; "structure" doubly so.
<huwshimi> lifeless: I'm not sure. I would like it if we had not javascript in templates (and I think we should move towards that). I just don't know where it should live.
<huwshimi> wgrant: I meant that for you ^
<wgrant> The log also highlights if the AJAX event takes more than 1 minute.
<wgrant> That's quite a long request!
<wgrant> However, this is really nice.
<wgrant> Particularly with the OOPS ID stuff.
<StevenK> james_w: Ping!
<lifeless> huwshimi: I was suggesting 1 second, not 1 minute, for slow ;)
<huwshimi> lifeless: Oh right
<huwshimi> lifeless: That's pretty short!
<lifeless> huwshimi: goal is 99% completing withing that time
<lifeless> huwshimi: perhaps start at 5s
<huwshimi> lifeless: I think it's great
<huwshimi> actually I think I did make it 1 second
<huwshimi> wgrant: Sorry the description is wrong. The code is for 1 second
<huwshimi> wgrant: My brain failed me when I was writing the description
<wgrant> huwshimi: Ah, that is less completely crazy. Excellent.
<wgrant> I think 2-3s is probably more sensible at the moment, but I guess we'll see.
<wgrant> Could FF it, but probably YAGNI...
<huwshimi> wgrant: Yeah, lifeless suggested 5 seconds. I don't mind changing it.
<huwshimi> wgrant: Also if you know of a better place for the javascript to live I can do that as well
<james_w> hi StevenK
<StevenK> james_w: Hi. :-)
<StevenK> james_w: I've been dealing with python-debian for the derived distros stuff -- did you know that the BaseVersion class in debian_support isn't strict enough with parsing versions?
<james_w> no, I did not know that
<StevenK> james_w: The regex allows a : in upstream version, but that's only allowed if there is an epoch
<james_w> erm, eh?
<StevenK> james_w: Heh, have I lost you?
<wgrant> huwshimi: Reviewed, with a few comments.
<james_w> I'm trying to work out if what you said is a tautology or not
<huwshimi> wgrant: Thanks, will take a look
<StevenK> james_w: 'foo:bar-1' isn't valid, but '1:foo:bar-1' is.
<james_w> ah, ok
<james_w> epoch can only be digits
<StevenK> james_w: Right. I have a patch: http://pastebin.ubuntu.com/586681/
<wgrant> huwshimi: The YUI IO events are more limiting than I thought :(
<StevenK> lifeless: Yes, r12677 is FF-protected.
<StevenK> lifeless: I can mark it qa-untestable, if you wish
<lifeless> StevenK: up to you
<lifeless> StevenK: we've a bad commit to rectify (its landing via ec2 now) before we can deploy
<StevenK> Once I've shaken out more of the stuff on dogfood, I'll be QAing it there, for the short-term
<lifeless> StevenK: long as your commit is either known-bad, or deployable in the next 4 hours, we should be fine
<StevenK> lifeless: Which is the bad commit, btw?
<wgrant> lifeless: Next 4 hours? That's not even enough for buildbot.
<lifeless> wgrant: fuzz
<lifeless> StevenK: wallyworlds security fix
<james_w> StevenK, I'm very glad you didn't try and fix that bug using regexes :-)
<james_w> StevenK, hunks 2 and 3 look good to me, but I don't think I like hunk 1
<StevenK> james_w: Which would have made the regex about 4 times more complex. I'm masochistic, but not *that* much.
<james_w> StevenK, also, I think the comment should be more explicit about what it is checking for, and the error could be clearer about the problem too
<james_w> StevenK, aren't you a perl guy though? ;-)
<StevenK> Haha
<StevenK> james_w: Hunk one is a seperate, but related problem -- the code currently throws a ValueError if the changelog you parsed contains an invalid version
<StevenK> james_w: And I've fixed the comment.
<StevenK> james_w: The meat of the patch is hunks 2 and 3. Hunk 1 is at best, a workaround, since I don't want invalid versions in my list of versions.
<james_w> StevenK, I think that the issue with hunk 1 is better solved at a different level, so I'd leave it out.
<james_w> StevenK, please send the patch as a bug report to Debian and hopefully someone will commit it
<StevenK> james_w: And solve it in a seperate patch?
<james_w> StevenK, what is the calling code that falls over a ValueError?
<james_w> presumably it involves a loop over .blocks?
<StevenK> ._blocks, yes
<StevenK> james_w: get_versions()
<james_w> ah, I forgot about that
<james_w> don't use that then I would say :-)
<StevenK> Haha
<StevenK> james_w: But it's ._blocks, it isn't mine to touch :-P
<james_w> currently the library is designed to give you good data or nothing
<james_w> I think it's public in newer versions
<StevenK> Right
<lifeless> what library are you discussing?
<StevenK> python-debian
<james_w> I was slightly naÃ¯ve in my estimation of how much crap was in all the changelogs in Debian/Ubuntu though
<james_w> in old versions etc.
 * StevenK is finding out
<StevenK> james_w: Crumbs, do you know how long it's been since I filed a bug on Debian :-)
<james_w> MIA!
<lifeless> whoa
<lifeless> 366 Time Outs
<lifeless> haproxy reconf FTW
<wgrant> The BugTask:+index change is striking.
 * lifeless blames making things more efficient
<wgrant> We've 25% fewer timeouts than Sunday. That is pretty nice.
<StevenK> james_w: Well, ... duh
<wgrant> Gaaaaah
<wgrant> ShipIt needs to die.
<lifeless> ?
<wgrant> EmailAddress handling is impossible to do correctly.
<wgrant> Because they are linked to both accounts and persons.
<wgrant> So you may have an email address without a Person because the user used ShipIt.
<wgrant> You can't then create a properly unactivated Person for that email address, since the email address is active.
<lifeless> well, we know the email is valid
<lifeless> whats the issue
<wgrant> We are creating an active Launchpad person for somebody who has never used Launchpad.
<lifeless> join the dots for me
<wgrant> lifeless: A user creates an SSO account and requests CDs from ShipIt. The parasite creates an Account and an EmailAddress in the LP DB.
<wgrant> The user performs some action visible to LP. This could be maintaining a package in Debian, commenting in an upstream bugtracker, etc.
<wgrant> LP tries to create an unactivated Person for that email address, so it can be referenced in the DB.
<wgrant> But the EmailAddress is already active, and just needs to be linked to the Person.
<wgrant> So the Person is now activated, and the user apparently uses Launchpad.
<lifeless> wgrant: in other words 'person activation is implicit not explicit'
<wgrant> lifeless: Yes.
<wgrant> sinzui disagrees that that is a bug.
<lifeless> wgrant: why ?
<wgrant> lifeless: I don't recall. It's been a while.
<lifeless> wgrant: also, if you want to rotflas - lib/lp/bugs/browser/buglinktarget.py - look for except Unauthorized
<wgrant> lifeless: :(
<lifeless> yes
<wgrant> Easy fixes, at least.
<lifeless> we were disclosing private bug *existance* and generating links
<wgrant> Existence and linkage.
<lifeless> content="structure batch_navigator/@@+table-view" -> I want to check precisely what this means
<lifeless> its 'the view called +table-view on the object batch_navigator' rendered into the contents of this element
<lifeless> right ?
<wgrant> Yes.
<lifeless> fingers crossed
<lifeless> wgrant: what are you hacking on today
<wgrant> lifeless: Bug #740594 at the moment.
<_mup_> Bug #740594: ProgrammingError: permission denied for relation emailaddress  <bugwatch> <oops> <Launchpad itself:In Progress by wgrant> < https://launchpad.net/bugs/740594 >
<lifeless> bah
<lifeless> how is the +table-view hooked up
<wgrant> lib/canonical/launchpad/zcml/batchnavigator.zcml?
<lifeless> thanks
<lifeless> grepping to  deep
<lifeless> however
<lifeless> its not rendering ><
<wgrant> Grar, ensurePerson is buggy.
<wgrant> If you call it with an EmailAddress with only an Account, it will create a Person, but forget to link the EmailAddress to it.
<wgrant> Call it with that EmailAddress a second time, and it will link the EmailAddress to the Person.
<lifeless> \o/
<lifeless> also
<lifeless> \o/
<wgrant> batchnavigator fixed?
<lifeless> tal:replace
<lifeless> can has review? https://code.launchpad.net/~lifeless/launchpad/bug-727023/+merge/55266
 * thumper looks
<lifeless> thanks
<thumper> lifeless: question, why create a BugListingBatchNavigator for each bug?
<lifeless> thumper: the page shows
<lifeless> bug X
<lifeless>   tasks
<lifeless> bug Y
<lifeless>   tasks
<lifeless> thumper: it was easy to split it up like this without triggering more queries, so I just kept the prior page layout
<thumper> ok
<thumper> r=me
<lifeless> its a bit noddy really given that there is a portlet showing the exact same stuff :)
<lifeless> thanks
<LPCIBot> Project windmill build #114: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill/114/
<nigelb> ah, started working when I put my lp keys into the .ssh folder of the virtual machine
<wgrant> lifeless: Thanks. https://code.launchpad.net/~wgrant/launchpad/bug-740594/+merge/55269 is a follow-up.
<StevenK> wgrant: r=me
<lifeless> StevenK: thanks
<wgrant> StevenK: Thanks.
 * StevenK ponders how one changes a branch in sourcecode
<wgrant> StevenK: Get the change into the branch, fix utilities/sourcedeps.conf.
<StevenK> wgrant: Ah, and then toss the branch that changes sourcedeps.conf through ec2 to make sure there isn't fallout?
<wgrant> StevenK: Yes
 * StevenK looks at updating python-debian to the latest
<nigelb> How long does the fetch source code stage take?
<nigelb> Its been running for an hour :|
<wgrant> What's it up to?
<StevenK> Depends on your connection, and if you have any of it, the phase of the moon
<lifeless> did we remove the special case of using http fo rlaunchpad clones?
<nigelb> Its branching something I guess
<wgrant> nigelb: But what?
<wgrant> lifeless: That's not update-sourcecode.
<nigelb> "Making local branch of Launchpad trunk, this may take a while..."
<wgrant> Oh.
<wgrant> Not update-sourcecode. I see.
<wgrant> lifeless: I don't see hot_products in lp-production-configs, so it seems it's gone.
<lifeless> nigelb: it should be about 200MB
<nigelb> lifeless: oh, ugh.  Its at something like 90% I think
<lifeless> nigelb: lots of code + lots of history
<lifeless> when we get around to shrinking the codebase
<lifeless> might do a fresh tree and massively compact it
<lifeless> but not for $quitesometime
<nigelb> I wonder if there's a way to grab it without history
<nigelb> or make it notshow history by default
<nigelb> Oh. I should get around to signing that agreement I guess.
<StevenK> wgrant: https://code.launchpad.net/~stevenk/python-debian/merge-and-stricter/+merge/55277
<wgrant> StevenK: Deleting changelog entries: wgrant says no.
<StevenK> wgrant: Think of it as a sync?
<StevenK> It was a cherry-pick from upstream anyway
<wgrant> k
<StevenK> Well, a sync + a small patch :-)
<StevenK> lifeless: You mind reviewing wgrant's review? https://code.launchpad.net/~stevenk/python-debian/merge-and-stricter/+merge/55277
<wgrant> StevenK: Actually, any reason the ".find(':') != -1" can't be "':' in "?
<StevenK> I'm not sure, does that work?
<wgrant> I'd hope so... group() returns a str.
<StevenK> Right, my string manupliation stuff is dated back to Linda.
<StevenK> wgrant: Good catch: http://pastebin.ubuntu.com/586728/
<wgrant> StevenK: Looks reasonable.
<StevenK> wgrant: Pushing
<lifeless> can has review? https://code.launchpad.net/~lifeless/launchpad/bug-741092/+merge/55278
<lifeless> huwshimi: fwiw I agree with wgrants review
<lifeless> huwshimi: if you need a hand with any of it just shout
<huwshimi> lifeless: Yeah, I think they were all good comments.
<wgrant> lifeless: Looking.
<wgrant> lifeless: Is this to fix the insane 7000 subscriber thing?
<StevenK> wgrant: I'm happy to review your review
<wgrant> I think we probably want to batch as well, but OK.
<wgrant> Ah, you mention that in the commit message.
<StevenK> lifeless: You drop an XXX, can the bug related to that be closed (or is it)?
<wgrant> It looks like it's the ancient Storm __nonzero__ bug.
<wgrant> I suspect.
<wgrant> By the age.
<wgrant> So it doesn't need closing.
<wgrant> Anyway, looks good.
<lifeless> StevenK: it was a bogus XXX
<lifeless> StevenK: never ever needed
<lifeless> wgrant: its to improve it
<wgrant> lifeless: Improve/destroy
<wgrant> Although I guess retrieving all of them should be fairly quick.
<wgrant> We'll see!
<lifeless> I had a glitch in there
<wgrant> A glitch?
<lifeless> I've just pushed a trivial tweak to cache the subscription list too
<lifeless> bah bah
<lifeless> gimme a sec, to undo my fail
<wgrant> Ah, good point.
<lifeless> right, pushed
<wgrant> StevenK: Want to mentor?
<StevenK> wgrant: Done
<wgrant> Thanks.
<lifeless> https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1914M186 is going to be fun to fix
<wgrant> lifeless: Delete BranchRevision, problem solved.
<lifeless> wgrant: how so?
<wgrant> lifeless: It's probably a scanner lock.
<lifeless> on branchmergeproposal ?
<wgrant> CONTEXT: SQL statement "SELECT 1 FROM ONLY "public"."branch" x WHERE "id" OPERATOR(pg_catalog.=) $1 FOR SHARE OF x"\\n',)
<lifeless> wgrant: where are you getting that ?
<wgrant> lifeless: End of the exception message.
<StevenK> Hm. Now what do I do ...
<StevenK> I doubt I can lp-land a python-debian PQM branch
<wgrant> StevenK: pqm-submit
<wgrant> lifeless: We're running with the old appserver configs, but a new single-threading haproxy one?
<lifeless> wgrant: partly
<lifeless> wgrant: depending on whether mbarnett did the tweak requested, we're running on 4 or 6 concurrency from haproxy, and the unaltered appserver configs
<wgrant> lifeless: 4 or 6 for the 12-thread servers, 1 for the 2-thread?
<lifeless> 4 or 6 for the 4-thread servers
<lifeless> 1 for the 2-thread
<wgrant> Oh, right, it was 12 for 4 before.
<lifeless> + xmlrpc at 1 per backend that is listening for xmlrpc
<StevenK> lifeless: First self-review \o/
<lifeless> \o/
<huwshimi> wgrant: I took a look at your comments and have pushed some changes. Your review were very helpful btw.
<wgrant> huwshimi: Great, will look in a sec once the diff is upgraded.
<wgrant> updated.
<huwshimi> wgrant: or downgraded, depending on the quality of the changes
<wgrant> lifeless: Could you please db https://code.launchpad.net/~wgrant/launchpad/backports-not-automatic/+merge/55279?
<StevenK> wgrant: I do not like the use of \ at line 151
<wgrant> I'm not sure whether \ or parentheses are more vile.
<StevenK> wgrant: And I don't think the commit() is needed at 187
<wgrant> StevenK: It's needed to invalidate the DistroSeries.
<wgrant> StevenK: So it pulls in the changes from the script.
<lifeless> StevenK: \ is fine
<StevenK> wgrant: But the script commits?
<lifeless> StevenK: its more cromulent than () just to do a newline
<lifeless> *I* would rather we stopped cutting at 80 characters for such things
<wgrant> StevenK: Right, but then we have to invalidate the local object to see the committed changes.
<StevenK> Fair enough
 * spiv waits for autopack of his launchpad repo to finish
<spiv> Hmm.  12 minutes, but it did take 30% off the space consumed.
<wgrant> huwshimi: Your branch hasn't scanned yet :/
<wgrant> Ah, there we go.
<wgrant> huwshimi:
<wgrant>  7    /* Can't use LPS here as it doesn't exist yet.
<wgrant>  8    */
<wgrant>  9    LPS.use('node', 'lazr.anim', function(Y) {
<StevenK> Heh
<huwshimi> wgrant: hahahaha
<huwshimi> wgrant: Guess I forgot to take that out when I refactored some things
<wgrant> lifeless: Thanks.
<wgrant> StevenK: Are you code reviewing my branch?
<wgrant> huwshimi: Bear with me here, I'm experimenting with YUI.
<StevenK> wgrant: I wasn't, but I can
<huwshimi> wgrant: There's no hurry. I'll be heading out soonish so I'll probably just take a look in the morning
<wgrant> StevenK: Ah, you were whinging about backslashes and other stuff, so I thought you might have been.
<wgrant> It would be great if you could.
<wgrant> 7/win 3
<wgrant> Grar.
<StevenK> wgrant: Done. You just need a number from stub
<wgrant> StevenK: Thanks.
<wgrant> huwshimi: A couple of suggestions, but it looks good.
<huwshimi> wgrant: Thanks for that. I'll take a proper look tomorrow
<LPCIBot> Project windmill build #115: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill/115/
<wgrant> stub: Hi.
<stub> wgrant: yo
<wgrant> stub: Could you DB-bless https://code.launchpad.net/~wgrant/launchpad/backports-not-automatic/+merge/55279?
<stub> Just done
<wgrant> Thanks.
<wgrant> stub: The apt flag is negative :/
<stub> ok
<stub> Just keep away from the double negatives :)
<lifeless> stub: ping
<stub> lifeless: pong
<lifeless> stub: two things I'd like your help evaluating
<lifeless> stub: first is the bug heat thing - I saw different results on the candidate index (but only /great/ for ubuntu)
<lifeless> stub: I'd like to debug why we saw different results; and evaluate denormalising heat down into bugtask
<lifeless> we might be able to squeeze that into this db cycle even
<lifeless> secondly, BranchRevision doesn't support range queries by date - but merge proposals (and other interesting queries) are all date based, so I'd like to evaluate the impact of denormalising the revision date into branch revision
<stub> I'm thinking denormalizing into a separate table would be better - I think the regular heat recalculation is contributing to bloat.
<lifeless> https://bugs.launchpad.net/launchpad/+bug/742916 is a bug showing the merge proposal query (for a merge proposal - old data, will be out of cache a lot)
<_mup_> Bug #742916: BranchMergeProposal:+index timeouts - slow query plan <dba> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/742916 >
<stub> I don't have a problem denormalizing the heat. It is a cache anyway - ideally it would be calculated on the fly but that is far to expensive.
<lifeless> stub: the concern I have with a different table is that the plan problem with sorting on a different table would still exist, unless that table had (heat, status, bug target)
<stub> For BranchRevision, this has been suggested in the past but not acted on (grand plans of removing the table alltogether and all that)
<stub> lifeless: True. Such a table is what we might end up with for bug search anyway - all the searchable fields denormalized into a BugSearch table, but that is a much larger task.
<lifeless> stub: can we add a datetime column, and an appropriate index to handle the query in https://bugs.launchpad.net/launchpad/+bug/742916 (e.g. (branch, sequence, revision_date)) on staging/qastaging
<_mup_> Bug #742916: BranchMergeProposal:+index timeouts - slow query plan <dba> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/742916 >
<lifeless> stub: so that we can determine the disk impact?
<lifeless> s/we/you/
<stub> lifeless: we can. It will take several hours.
<lifeless> stub: no time like the present (if we're going to do it, it would be good to add it (NULLable) this cycle
<stub> lifeless: There is work we need to test to drop the id column that should counteract the disk space impact.
<lifeless> stub: the column add should be instant, right? and the index able to be done CONCURRENT ?
<stub> Yes. A job to populate the column will take ages though.
<lifeless> stub: sure, I think I can make it tolerably efficient
<lifeless> stub: would like to parallelise the garbo perhaps to help it and othe migrations carry on a little faster
<stub> We need that job too - can't see the disk space impact until the rows have all been rewritten with data in that column.
<lifeless> stub: ah, well for the test we can do it with an update
<stub> Adding --threads= to garbo should be trivial.
<lifeless> update branchrevision set revision_date = revision.revision_date from branch_revision, revision where branch_revision.revision=revision.id;
<lifeless> ^ untested, but approximate
<stub> lifeless: The script will likely be faster, as it works in batches rather than attempting to do it all in one transaction. Also, we have to do it in chunks or we will double the space used by that table
<stub> (all the old rows and all the new rows will exist, and then vacuum will clean up all the old rows giving 50% bloat)
<lifeless> staging might run out of space
<lifeless> ok
<lifeless> I guess branchrevision should wait for next week then - I suspect we couldn't qa in time
<lifeless> bugtask.heat I'll whip up a patch for tonight.
<lifeless> stub: do you have any idea why you didn't see the performance benefits on the heat query I did?
<lifeless> stub: my only WAG is that your test query wasn't adjusted, or something
<stub> I was testing ordering by heat, id. You had a performance boost by dropping id from the sort.
<lifeless> stub: I tried on qastaging with a (heat, id desc) index and a query sorting by bug.heat desc, bug.id
<stub> I managed to get the existing query to use the new index once, and it saved a few seconds, but the query was still two orders of magnitude slower than your no id sort query.
<lifeless> ah
<lifeless> stub: so, I think then we weren't testing quite the same query - because I did adjust the query; I will try to be less ambiguous in future
<stub> No, I was expecting different results :-) It would have been nice to drop in an index and keep stable ordering, but I don't see that happening.
<lifeless> ah, ok - then *I* didn't understand  :)
<lifeless> anyhow, I think its clear to make it snappy we need heat in the same table
<lifeless> otherwise it has to traverse *tonnes* of bugs to find the first bug in a context with no really hot bugs
<lifeless> I can guess that having lots of bugs often equates to having some very hot ones
<lifeless> but the plan - I pasted it in - for the updated query + new index on launchpad was pretty expensive
<lifeless>  Limit (cost=0.00..146128.04 rows=40 width=1017) (actual time=1.302..500.965 rows=40 loops=1)
<lifeless> (this is actually 50% cheaper than the *current* plan, but still awful)
<lifeless> stub: would you like a bug about the garbo parallel processing ?
<lifeless> stub: I don't think we need a --threads option
<lifeless> stub: multiprocessing.cpu_count() will return the # of cpus
<lifeless> stub: we can just run that many threads
<stub> Garbo jobs are almost certainly database bound, so not sure if that is so useful.
<stub> I've considered adding options to only run specific tasks (but how to handle lock files is unclear), and the --threads.
<stub> I suspect we want a lock file per task
<stub> Rather than a single lock for the script.
<stub> Bug #744803
<_mup_> Bug #744803: Better garbo locking <Launchpad itself:Triaged> < https://launchpad.net/bugs/744803 >
<adeuring> good morning
<stub> Bug #744804
<_mup_> Bug #744804: Parallel garbo <Launchpad itself:Triaged> < https://launchpad.net/bugs/744804 >
<lifeless> stub: if we run N garbos we'll need per-tuneable locks yes; if we run one master that parallelises I think we can ignore the locking for now
<lifeless> stub: I think many garbo jobs are overhead bound - I mean, time to pull stuff out, mangle it, shove it back in; others are going to be db bound - but we should remember we have 8 available cores on master atm, and most garbo jobs are completely up to date, we'll only have a few migrations in flight at any one time
<lifeless> stub: I don't particularly care about implementation, the reason I suggested using cpu count is that it avoids manual selection (at the cost of not being tunable)
<stub> Sure. I'll probably add --threads and have it default to cpu count :)
<lifeless> stub: all that said, I'd love to see things like stevenk's migrator, and the branchrevision one next week, get more of a timeslice than they do at the moment - ideally 55minutes per hour
<stub> k
<StevenK> That sounds good.
<lifeless> stub: so I think that is the key goal: more time for migrations in the garbo
<lifeless> stub: everything else is tactics
<stub> I won't suggest to jtv that if looptuner became thread aware it could tune the number of instances dynamically to maximize throughput based on database load ;)
<lifeless> stub: another thing, probably EFUTURE, would be allowing the loops themselves to parallelise
<lifeless> stub: please don't :P
<jtv> stub: as if you'd need to!
<jtv> You think I don't think about these things?
<stub> I know you do :-)
<stub> Actually, garbo 2.0 is looking like a twisted system... but I think we are several years away from that.
<lifeless> bug 744808 is a browser/X bug right ?
<_mup_> Bug #744808: Redrawing of Bug View page very slow in Firefox <Launchpad itself:New> < https://launchpad.net/bugs/744808 >
<jtv> It does sound like one we had.
<lifeless> stub: can you do a quick timing check for me on staging ?
<lifeless> stub: add a heat column to bugtask and copy the heat over from bug
<lifeless> stub: bugtask is only 250MB in size
<lifeless> stub: so just a single sql statement should be fine - I want to see if its fast enough to do during downtime
<lifeless> stub: can I take 59 for this ?
<stub> gah... distracted
<stub> lifeless: 59... sure.
<lifeless> stub: pastbinnning a script
<lifeless> stub: http://pastebin.com/fUQN4JvF
<stub> The productseries index seems to have a bogus WHERE clause
<lifeless> you pass :P
<lifeless> [bad copypaste, thanks]
<lifeless> stub: so, did that work, how long to do the update?
<stub> lifeless: I'll shove that through slony now
<lifeless> stub: thanks
<lifeless> stub: can a trigger update columns in a related table?
<stub> lifeless: yes
<stub> lifeless: I'd rather just remove Bug.heat though
<lifeless> how do we update stuff in trusted.sql ?
<lifeless> stub: I'll keep that in mind as I look through this
<stub> You just edit it. And things can break if you remove things.
<lifeless> calculate_bug_heat is what I'm changing, for obvious reasons
<lifeless> # Bugs decay over time. Every day the bug isn't touched its heat  ---- needs to die
<stub> Just change it inplace then. You shouldn't need to alter the name or signature.
<lifeless> hah
<lifeless> its not even a trigger
<lifeless> its just an in-db function
<lifeless> which of course... headdesk
<lifeless> stub: might need a hand doing a trigger for this - we don't want to calculate the heat once per task, so keeping the bug.heat seems easier.... but we need to copy any changes to bug.heat to the bugs bugtasks.heat
<stub> We need to shutdown most of the staging systems - its too busy to apply the patch live.
<lifeless> ok; lets do it
* gmb changed the topic of #launchpad-dev to: erformance Tuesday | https://dev.launchpad.net/ | On call reviewer: gmb | https://code.launchpad.net/launchpad-project/+activereviews
<lifeless> stub: triggered up
<lifeless> http://pastebin.com/xMfWZWFE
<lifeless> gmb: you lost a P
<gmb> Ha.
<bigjools> the irony.  I'm just going for one.
<lifeless> TMI
* gmb changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: gmb | https://code.launchpad.net/launchpad-project/+activereviews
<bigjools> lifeless: you'll like this: http://1.bp.blogspot.com/-YhwtjjSfRf4/TZBOYuLTNVI/AAAAAAAAAP0/0P7KUdJ1RBI/s1600/James.png
<lifeless> bigjools: rotfl
<stub> lifeless: Do you want that trigger now? If so, we should not try and monkey patch this - just land it as normal and back things out if the timings suck
<lifeless> stub: don't need it for the timings
<lifeless> stub: Just want your eyeballs on the updated db side of the patch
<stub> lifeless: So the alternative of changing the UPDATE Bug SET heat=calculate_bug_heat(...) WHERE id=... to UPDATE BugTask SET heat=calculate_bug_heat WHERE bug=... and dropping Bug.heat sucks?
<lifeless> stub: some bugs have hundreds of tasks
<lifeless> stub: and bug heat calculation isn't, uhm, efficient
<stub> lifeless: How does this change that?
<lifeless> stub: oh, and I need a check for NEW.heat != OLD.heat
<lifeless> stub: uhm, it copies the one calculated value from the bug, rather than doing it once per row - or would pgsql optimise that out ?
<stub> UPDATE BugTask SET heat=calculate_bug_heat($bug) WHERE bug=$bug; calls the stored procedure once.
<stub> Well... assuming the function hasn't been declared to volatile
<stub> Its declared stable, so should only be invoked once.
<lifeless> stub: ok; I'm inclined to go inchwise for now anyway, simply because there are a few hundred references to heat.
<lifeless> and I'd like to get this in tomorrow, which a more aggressive branch is more likely to fail
<jml> is there still a MOTU liaison to Launchpad?
<lifeless> wgrant last I heard :P
<gmb> jtv: https://code.launchpad.net/~jtv/launchpad/db-bug-55798/+merge/55174 says "This is still a work in progress; expect a proper cover letter later when it goes up for review" yet it's marked "needs review". Should it be marked "work in progress"?
<jtv> gmb: I thought I did mark it WIPâ¦ thanks for noticing.
<gmb> jtv: I'll do it now.
<jtv> Thanks.
<lifeless> stub: updated with the if condition - http://pastebin.com/bYHpVBWH
 * gmb just marked it Approved for a second. Would've been funny if we'd been doing automated merges
<gmb> jtv: Done.
<lifeless> stub: if I'm correct, this is landable once we confirm the migration time
<lifeless> stub: and we can then do the better query post db deploy
<wgrant> jml: So, yeah, not really.
<jml> wgrant: I didn't think so.
<wgrant> jml: People basically all just know to poke me.
<lifeless> stub: I'm going to propose this for merge
<henninge> Hi!
<henninge> I get this when running "make lint": http://paste.ubuntu.com/586801/
<henninge> Updating my sourcedeps now ...
<henninge> no joy ;(
<stub> lifeless: I'd say land it anyway so we don't block on losas.
<stub> lifeless: We can rework it if the rollout time sucks
<stub> lifeless: The trigger should be INSERT OR UPDATE
<jml> henninge: there's nothing there that will come from sourcedeps
<lifeless> stub: why? INSERT never has anything to do
<wgrant> henninge: That class is new in 2.7 :(
<jml> henninge: what version of pocketlint do you have?
<lifeless> stub: because bugtask has a foreign key on bug
<henninge> jml: yes, I realised
<wgrant> henninge: Could you file a bug on pocketlint?
<wgrant> henninge: sinzui will fix it quickly I'm sure.
<wgrant> Otherwise downgrade python-pocketlint.
<jml> ahh, there you go.
<henninge> wgrant: Will do, thanks.
<StevenK> Oh, is that the upgraded python-pocketlint I haven't done yet? Bonus.
<wgrant> python-pocket-lint
<stub> lifeless: oic. yes.
<bigjools> allenap: in your review you said that stuff returned from canonical_url needs to be escaped.  Really?
<stub> lifeless: So when a bugtask is created, it should get the heat of the attached bug. Probably better having that in Python land rather than another trigger.
<lifeless> stub: as we only need to sort on the high heat tasks initially, as long as new tasks have heat 0, the heat garbo job will sort it out
<lifeless> stub: and yes, we can also sort it out in python; the column is non-null
<wgrant> Every problem can be solved with another layer of triggers.
<lifeless> wgrant: except having too many triggers
<stub> lifeless: So as the patch stands at the moment, creating a bugtask will fail because it will have a NULL heat.
<lifeless> stub: default 0 ?
<wgrant> lifeless: We can have a trigger to generate the triggers on demand and delete them when they're not needed any more!
<stub> Might want a DEFAULT 0 in there if you want to land this separate from code changes.
<lifeless> yeah, I'll just tweak the alter
<stub> lifeless: Set the default at the same time or after setting the NOT NULL - not in the initial ALTER TABLE
<lifeless> stub: https://code.launchpad.net/~lifeless/launchpad/bug-618406/+merge/55303
<lifeless> stub: yeah, I did :)
<stub> Its a non-obvious footgun :)
<lifeless> bah, I left 58 in the .sql file
<lifeless> changing to 59
<stub> lifeless: 58 is fine
<lifeless> stub: the .sql file is called 59
<stub> ok
<lifeless> stub: they need to match, don't they ?
<stub> yes
<lifeless> stub: I can rename both to 58, or leave - up to you
<stub> I don't care. I have many integers :)
<lifeless> \o/
<henninge> wgrant: bug 744873
<_mup_> Bug #744873: lint fails on Python 2.6 <pocket-lint:New> < https://launchpad.net/bugs/744873 >
<allenap> bigjools: Yes. Everything that goes into HTML/XML/XHTML should be escaped unless it has been escaped already.
<bigjools> allenap: ok
<wgrant> bigjools, allenap: canonical_url should always be safe, but if you don't escape it then you have a bad habit.
<lifeless> stub: https://code.launchpad.net/~lifeless/launchpad/bug-618406/+merge/55303 has its fidd
<wgrant> And if it's easier to not escape it than it is to escape it, then we have a systemic problem.
<lifeless> oh fail, EWRONGTARGET
<bigjools> wgrant: my point
<allenap> I think we do have a systemic problem then :)
<lifeless> take two
<lifeless> https://code.launchpad.net/~lifeless/launchpad/bug-618406/+merge/55306
<wgrant> We do :)
 * stub waits again for the diff
<allenap> wgrant: One of my suggestions for bigjools in the review was to use lxml.html to build fragments. For very small things it appears heavyweight, but imo it soon pays off, and the cssselect() method makes it similar in use to Y.Node.
<stub> wgrant: I wonder if we can hack structure to explode if it is passed something not known to be escaped? We already have monkey patches in place for the cache: stuff.
<bigjools> allenap: I liked that but I chickened out and used cgi.escape
<lifeless> bigjools: you know thats incomplete right ?
<wgrant> cgi.escape(foo) is fine for element contents.
<wgrant> It is not fine for attributes.
<allenap> I pointed out the quote argument too.
<wgrant> You need cgi.escape(foo, True) for attributes, and you cannot use single quotes around the attribute.;
<wgrant> stub: I am considering changing zope.tal to use markupsafe for escaping, which will allow us to just pass markupsafe objects into the template without structure.
<wgrant> stub: Then any use of structure that isn't a widget will be easily visible.
<lifeless> stub: diff up
<stub> looking
<lifeless> thanks
 * lifeless breaks the build
<lifeless> gnight all
<lifeless> hmm, wonder if wallyworld's fix landde
<lifeless> looks like it, we should be able to deploy if someone wants to coordinate
<lifeless> night all
<adeuring> gmb: could you please review this MP: https://code.launchpad.net/~adeuring/launchpad/js-translation/+merge/55309 ?
<deryck> Morning, all.
<adeuring> morning deryck
<adeuring> gmb, ping
<henninge> gmb: Hi!
<henninge> gmb: Can you please review my branch? I will be having lunch and switching locations now but I'll be happy to answer questions after that.
<henninge> https://code.launchpad.net/~henninge/launchpad/devel-737418-remove-sharing-links/+merge/55313
<wallyworld> lifeless: fix has landed
<wgrant> Fix is going to be deployed once the staging mailbox loads.
<LPCIBot> Project windmill build #116: STILL FAILING in 1 hr 8 min: https://lpci.wedontsleep.org/job/windmill/116/
<wgrant> Grar.
<wgrant> Stupid Pacific.
<wgrant> And/or Optus.
<wgrant> I would like more than 10KB/s to international hosts tonight, thanks...
<gmb> adeuring, henninge-lunch: Sure.
<adeuring> gmb: thanks
<jml> wgrant: on cable? maybe there's a popular show on the telly
<henninge> gmb: thanks a lot
 * henninge reloates
<henninge> relocates
<henninge> even
<jml> (no, I don't understand anything about how physical networks work)
<wgrant> jml: Sadly they don't work that way.
<wgrant> And it's only international.
<wgrant> cjwatson: The buildbot slaves are updated, and I've hopefully built a sufficient EC2 AMI, which I'm about to test your binary xz branch against. If it goes well I'll land it tomorrow.
<StevenK> wgrant: And added your ID to ec2 land?
<wgrant> StevenK: In this branch, yes.
<gmb> adeuring: r=me
<LPCIBot> Project windmill build #117: STILL FAILING in 45 min: https://lpci.wedontsleep.org/job/windmill/117/
<cjwatson> wgrant: excellent.  is that the data.tar.xz one?
<wgrant> cjwatson: Yeah.
<cjwatson> great.
<wgrant> jml: I don't know how I lived without ec2 list.
<wgrant> cjwatson: Should both be deployed on the 6th, I guess.
<jml> wgrant: :)
<cjwatson> I was just about to ask that.
<wgrant> cjwatson: We'll hopefully be able to get cocoplum into the daily nodowntime rollouts eventually, but it's more difficult than just about anything else.
<cjwatson> I can imagine.
<gmb> henninge: r=me
<jml> I'm not hopeful for an answer but it's worth asking...
<jml> is there a programmatic way to find out what, say, a distribution owner is allowed to do?
<bigjools> haha
<bigjools> security.py nicely obfuscates that :/
<wgrant> At least most Soyuz stuff is neatly encapsulated in security.py
<wgrant> For other stuff (bugs come to mind).. uh, good luck.
<bigjools> I'm not that sure it is
<bigjools> there's a few methods that take user args
<wgrant> Soyuz has an entirely separate permission system for it's complex stuff.
<wgrant> Mm, true.
<wgrant> But not many.
<bigjools> separate - as in "do what you like!"
<adeuring> gmb: thanks!
<bigjools> which I will endeavour to fix for some cases for DDs
<wgrant> bigjools: Oh?
<bigjools> yes, I want to make distro management done in the UI/API as much as possible
<wgrant> Oh, I was referring to ArchivePermission.
<wgrant> Not ssh cocoplum.
<bigjools> the coming year is going to be painful
<jml> wgrant: manual inspection hasn't proved too hard at this point
<jml> just... manual
<wgrant> bigjools: I was looking forward to working on some of that stuff, TBH :/
<jml> next time I hear someone say "for audit purposes", I'm going to ask them how they are going to audit.
<bigjools> wgrant: what's stopping you? :)
<wgrant> '
<wgrant> It's out of scope of the maintenance squads :(
<henninge> gmb: Thanks for the review! ;) I had already filed bug 744873 about pocketlint.
<_mup_> Bug #744873: lint fails on Python 2.6 <pocket-lint:New> < https://launchpad.net/bugs/744873 >
<gmb> henninge: Cool, thanks.
<deryck> henninge: ping for standup
<henninge> deryck: coming
<jml> why is https://blueprints.qastaging.launchpad.net/launchpad showing different data to https://blueprints.launchpad.net/launchpad ?
<jml> apparently because they are configured differently
<jml> sinzui: hey, you had a XXX analyzer, right?
<sinzui> jml: I wrote the script in utilities
<sinzui> I am uncertain of its value
<jml> sinzui: I was wondering which XXX comments refer to fixed bugs.
<jml> sinzui: your script helps
<sinzui> jml: :) I found one a few days ago actually
<bac> sinzui: when i run 'make lint' i get an import error for ParseError.  did you miss a dependency on pocketlint?
<sinzui> jml: utilities/xxxreport.py -f c ./xxx-report.csv
<jml> sinzui: ./utilities/xxxreport.py -f csv  . | cut -d, -f5 | sort -n | uniq
<jml> http://pastebin.ubuntu.com/586882/
<sinzui> jml: did did something similar to feed and api script that added tech-debt to each of those bugs
<jml> shame there's no way to get all of those bugs in one API request.
<henninge> bac: bug 744873 ;)
<sinzui> some most bugs are in the launchpad-project. A few are in Zope
<_mup_> Bug #744873: lint fails on Python 2.6 <pocket-lint:Fix Committed by sinzui> < https://launchpad.net/bugs/744873 >
<jml> sinzui: I imagine a few are in Bazaar or Twisted too.
<sinzui> oh, yes, some are in twisted
<bac> thanks henninge
<jml> how do you get a bug by id in the API?
<jml> huh, just add it to the end of the bugs collection
<sinzui> jml: something like this: http://pastebin.ubuntu.com/586883/
<jml> sinzui: ta
<abentley> henninge: if you'd like to discuss my changes, I'm free.
<henninge> abentley: your code page is timing out on me ...
<abentley> henninge: the loggerhead page?
<henninge> abentley: which branch should I look at?
<henninge> https://code.launchpad.net/~abentley
<abentley> henninge: the cumulative changes are in lp:~abentley/launchpad/js-translation-2
<henninge> abentley: thanks
<henninge> abentley: I'll play around with it a bit
<abentley> henninge: Cool
<jml> sinzui: http://paste.ubuntu.com/586884/ all of the bugs in XXX comments that have a closed task
<jml> sinzui: that's over a quarter of all bugs mentioned.
<sinzui> jml: :) Maybe we should have had an action item to followup on this
<sinzui> This is very good progress
<LPCIBot> Project db-devel build #503: FAILURE in 43 min: https://lpci.wedontsleep.org/job/db-devel/503/
<jml> sinzui: an action item for who to follow up on this, exactly.
<sinzui> jml: We concluded the report was not driving the XXX to zero, but it would have been nice to check the report every thunderdome to see if we are making progress
<jml> sinzui: ahh yes.
<jml> sinzui: well, that's probably a good idea for next thunderdome, especially if those XXXs for closed bugs are removed from the tree
<jml> well, that's enough play for me. back to the grindstone
<sinzui> henninge: pocket-lint is fixed for python 2.6  and published in launchpad's ppa
<henninge> sinzui: cool, thanks for the prompt fix
<sinzui> It was just as prompt as my fix for python 2.7...but this time both work. I was incompetent.
<bac> sinzui: i'm getting an ExpatError from pocketlint (the new one) http://pastebin.ubuntu.com/586887/
<sinzui> bugger
<bac> sinzui: you want my branch for testing?
<sinzui> no
<sinzui> bac: I did not see those changes in the diff :(. I made this change a long time ago
<sinzui> bac: henninge: you will need to wait another 60min for a fix. I see the second change in annotate and it is simple to treat both kinds of errors separately.
<LPCIBot> Project windmill build #118: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill/118/
<sinzui> I sense some irony in this mess. I was slow to release pocket-lint fixes in the past because I lost the test suite when I divorced the linter from gedit-developer-plugins. I rebuilt the test suite so I now release quickly. The test suite runs with the default python, which for my 2.7
<bac> sinzui: i made a wee fix to formatcheck.py that allows me to go forward
<bac> sinzui: it is also ironic that formatcheck.py has lint errors in it.  :)
<sinzui> bac: it did not until I had to backport :)
<sinzui> pyflakes hates import redefinitions
<jml> there are ways to avoid redefinitions
<jml> hey,
<jml> does "Contact this team" send emails to teams if the team-within-a-team has a contact address?
<jml> looks like it contacts members with preferred emails
<jml> can teams have preferred emails?
<jml> mrevell: do you know how this works?
 * mrevell reads up
<mrevell> jml, Teams can have preferred emails. They can choose from "Email every member individually", "email the team's list, if it exists", "email some other address that the admin has specified"
<mrevell> So, I assume the team-within-a-team thing will respect the team-within-a-team's choice.
<mrevell> But I don't know for sure.
<jml> thanks
<jml> frustrating that this is so hard to figure out.
<henninge> abentley: Shouldn't all the items of the check list be active already?
<abentley> No, that depends on the state of the sourcepackage.
<henninge> abentley: by 'active' I meant 'using ajax'
<abentley> henninge: You mean that the page where users select the target branch for a sourcepackage already uses ajax, for example?
<henninge> abentley: I can set the targe branch for the upstream project using ajax.
<henninge> abentley: and I can search for a upstream product series
<henninge> but the edit icons next to the other two items in the list take me to the edit pages for those (web 1.0)
<henninge> abentley: I guess I am just asking if you have pushed your latest code ... ;)
<abentley> henninge: Yes, I have pushed all my latest code, and the last two items still need to be done.
<henninge> ah!
<abentley> henninge: I am currently fixing up the windmill tests for selecting a productseries.
<henninge> abentley: np, I was just wondering because I see that there is already code for those in the js.
<abentley> henninge: There is model code, but I wan't sure what to do for the UI code until I spoke with deryck yesterday afternoon.
<LPCIBot> Project db-devel build #504: STILL FAILING in 42 min: https://lpci.wedontsleep.org/job/db-devel/504/
<henninge> abentley: thanks. I will EOD now and continue tomorrow.
<abentley> henninge: cool
<jml> looks like the build failure is a legit one
<jml> anyone addressing it?
<LPCIBot> Project windmill build #119: STILL FAILING in 35 min: https://lpci.wedontsleep.org/job/windmill/119/
<jml> quick review for testfix? http://pastebin.ubuntu.com/586935/
<jml> taking that as a "no"
<jml> landing w/out review
<bigjools> jml: r=me
<jml> bigjools: thanks.
 * bigjools pines for the r=trivial days
<jml> heh. this isn't trivial, per se
<bigjools> I actually think testfixes should be mandatory trivial - else back the revision out
<jml> hmm
<jml> bzr pqm-submit --submit-branch=bzr+ssh://bazaar.launchpad.net/~launchpad-pqm/launchpad/db-devel -m '[testfix][r=bigjools] Be explicit about which table we get heat from.'
<jml> what's wrong with that line?
<jml> it's acting as if it works but nothing happens on pqm.launchpad.net or lpbuildbot
<bigjools> [no-qa] missing?
<bigjools> or bug=
<jml> bigjools: seems to work now, thanks. maybe it was just a delay (or fast PQM processing + slow buildbot)
<jml> bigjools: interesting idea re mandatory trivial testfixes. certainly a good rule of thumb.
<nigelb> flacoste: thanks.
<bigjools> jml: indeed - the idea being that all the time it's not fixed, everyone is blocked, so we need to spend the minimum amount of time fixing it.
<jml> ...
<jml> there's a launchpad.Driver permission
<jml> for some reason, that makes me wish I could swear like a 19th century Parisian.
<bigjools> just look up the Merovingian's line in The Matrix
<jml> there's a thought.
<jml> although I doubt that whatever the real Merovingians spoke sounded like that.
<bigjools> "It's like wiping your arse with silk"
<jml> An analogy for which I lack direct experience
<jml> (since I've never sworn in French, of course)
<bigjools> of course
 * jml is filling in https://wiki.ubuntu.com/LaunchpadPermissions very very slowly
<jml> !!@#@!
<jml> ok. I am sick of surprises.
<bigjools> nytol
<jml> g'night all
<jml> ugh.
<jml> the build is failing again
<jml> looks like a hangover from the previous test failure. forcing a new build after the failure ought to fix it.
<jml> I know there are plenty of North Americans around to read this in a timely fashion, so I'm not going to send an email.
<lifeless> jml: thank you
<lifeless> sinzui: ping
<sinzui> hi lifeless
<lifeless> hiya
<lifeless> you helped me get an oops for https://bugs.launchpad.net/launchpad/+bug/741092
<_mup_> Bug #741092: Archive:+subscriptions times out with many subscribers <qa-needstesting> <timeout> <Launchpad itself:Fix Committed by lifeless> < https://launchpad.net/bugs/741092 >
<sinzui> \o/
<lifeless> could you perhaps help me qa it - see if it lets you view it cold, and after a few retries, on qastaging ?
<lifeless> I haven't batched it
 * sinzui types the url
<lifeless> but in theory we can handle 7K in one shot; if it doesn't render even after a few refreshes we probably are running out of storm cache or something and will need to batch :(
<sinzui> lifeless: I do not have permission to see that one, but I think the beta-font team has an equally pathological subscriber page. I will check production first
<sinzui> ;/
<lifeless> I found the design there a little confusing too, given it doesn't use teamparticipation, but presumably we need to permit new members to see automatically
<sinzui> lifeless: I asked achuni to help since he reported the bug. The cold cache does timeout: OOPS-1914QS128 and the second try does too: OOPS-1914QS129
<lifeless> sinzui: 'darn'
<lifeless> sinzui: can you get him to try 6 times
<lifeless> sinzui: (6 times because if its /just/ cold cache in the DB, we are only walking Person records 10 seconds worth at a time
<lifeless> I'm syncing qastaging to look at the oops
<sinzui> :) I explained the same to achuni
<lifeless> sinzui: \o/
<sinzui> lifeless: all oopes: oops ids are OOPS-1914QS133 to 139
<LPCIBot> Project devel build #588: FAILURE in 5 hr 0 min: https://lpci.wedontsleep.org/job/devel/588/
<lifeless> garh
<lifeless> oops ids are being reused on qastaging
<lifeless> garbo hourly is overwriting
<lifeless> ok, I got a trace https://lp-oops.canonical.com/oops.py/?oopsid=1914QS133
<lifeless> Statement Count: 1491 ><
<lifeless> 1467 person lookups
 * lifeless thinks the storm cache is too small
<lifeless> no, thats not it, its clearly doing single loads
 * lifeless failed 
<lifeless> the traceback is really weird
<lifeless> ids = set(map(attrgetter('subscriber_id'), result)) is triggering late evaluation. EWTF
<lifeless> grraaaaaaa
<lifeless> Module canonical.launchpad.webapp.authorization, line 218, in checkPermission
<lifeless>     principal.account)
<lifeless> sinzui: what do you think http://pastebin.com/70gVSwSt ?
<sinzui> lifeless: we were doing the expensive check first?
<lifeless> sinzui: we loaded the subscriptions
<lifeless> then in the line I pasted - the attrgetter triggered permission check via the security proxy trying o read the subscriber_id
<sinzui> :/
<lifeless> the person hasn't been loaded at that point
<lifeless> so it lazy evaluated the person
<lifeless> found that the user wasn't the subscriber, found the user was archive admin, and we go around again
<lifeless> I'm doing two changes
<lifeless> one to change the order - common case will be the archive admin managing subscriptions
<lifeless> and
<lifeless> I think we should allow subscriber_id to any code
<lifeless> I think http://pastebin.com/HTPWc93e will do it
<sinzui> that is sensible
<lifeless> but i don't know - because the _id is also in the interface
<lifeless> will the lp.View declaration override the allow declaration ?
<sinzui> lifeless: I do not have an issue with using the id. (I have some issues with is appearing in the ui/forms). I think the security.py change is good to land...
<lifeless> sinzui: the id shouldn't appear in any ui/forms
<LPCIBot> Project windmill build #120: STILL FAILING in 1 hr 8 min: https://lpci.wedontsleep.org/job/windmill/120/
<sinzui> lifeless:  I do not think id will be exposed anywhere. Do we need a test to verify access to subscriber_id? Is it implicitly tested?
<lifeless> sinzui: do you know if subscribers use this page as well ?
<lifeless> sinzui: we know it works via the security proxy; the question is whether it avoids triggering a security check at all
<lifeless> however its a bit of a stopgap
 * sinzui checks the alternate path
<lifeless> the basic problem is that when subscribers view this page we should /only/ load the subscribers subscription
<lifeless> not all
<lifeless> by which I mean, the security.py change fixes admins/archive owners
<lifeless> but won't fix 'archive subscriber' viewing the page
<lifeless> sinzui: so, I'll land the security.py thing
<lifeless> we might not have subscribers view the page
<lifeless> and if we do, we can do the root cause change rather than a bandaid
<sinzui> lifeless: subscribers cannot access that page/view. They have a separate view
<lifeless> sinzui: in which case, though its a little more expensive than idea, the change I've made is sufficient
<lifeless> s/idea/ideal/
<lifeless> sinzui: do you want to review, or should I self review?
<sinzui> You have my r=,me
<sinzui> I can mark it reviewed if you point me to an mp
<lifeless> just filing it
<lifeless> https://code.launchpad.net/~lifeless/launchpad/bug-741092/+merge/55429
<lifeless> sinzui: it has a diff now
<sinzui> approved
<lifeless> sinzui: wrong widget :P
<sinzui> It will come
<lifeless> man
<lifeless> I really want callbacks
<lifeless> would make our UI /so much better/
<lifeless> now to see what my db patch is up to
<lifeless> allenap: you've seen load_related, right ?
<thumper> how do I find out what files get installed by a package?
<wallyworld_> benji: thanks for the review. i run the tests using "bin/test" and assumed that would "Do The Right Thing"
<wallyworld_> benji: are you saying it is using the wrong python doing that?
<lifeless> benji: whats wrong with testtools as a dep ?
<wallyworld_> benji: btw, i only ran the one new test i added and it did pass for me locally
<benji> wallyworld_: it's more that it's the wrong Python to run the buildout with; you want a "clean" python, one without anything installed in it's site-packages
<wallyworld_> benji: ah ok. in that case yes i was using the system python
<wallyworld_> to do the buildout
<benji> lifeless: nothing, it just looked accidental so I did the easiest thing that came to mind to get past the test failures
<benji> wallyworld_: I wouldn't be surprised if the tests fail on the current trunk, if you can verify that for me then tomorrow I'll take a look at straightening them out
<wallyworld_> benji: ok. you want me to do a fully clean build not using the system python?
<benji> right; you can use a virtualenv to get a clean python
<wallyworld_> benji: will do. thanks again.
<benji> my pleasure
<lifeless> benji: theres lots of test goodies in there; perhaps we should just add it as a dep to everything
<benji> lifeless: that'd be fine with me; the problem was that it was an _undeclared_ dependency.  I don't have an opinion one way or the other on it being an actual dependency.  Well, I take that back; it wasn't really used for anything so I'd rather it wait until we actually want it to use it for something.  But that's a pretty low bar.
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #505: FIXED in 5 hr 6 min: https://lpci.wedontsleep.org/job/db-devel/505/
 * wallyworld_ starts a full windmill test run and runs off for a bit to see the accountant so the tax man doesn't come knocking
<sinzui> I know how to fix the spastic mhonarc message encoding
<sinzui> wgrant: can you see this https://code.launchpad.net/~sinzui/launchpad/mailing-list-archiving-0/+merge/55435
<sinzui> wgrant: mumble?
<wgrant> sinzui: I can.
<thumper> wgrant: how do I find out what files get installed by a package?
<lifeless> dpkg -L packagename
<thumper> ta
<allenap> lifeless: Yes, I saw load_related recently. Neat :) What about it?
<lifeless> allenap: seems like the load recipe you mentioned in a recent review would also be generalisable
<lifeless> :)
<wgrant> sinzui: Are there docs on running a local mailman? I recall I did it 18 months ago, but don't recall how.
<allenap> lifeless: I included a way to do it with load_related() too. Or are you saying there's scope for another helper? In that case the code turned out to be more concise just using load(), but maybe I missed a trick.
<lifeless> allenap: I think theres room to improve our tooling furhter
<lifeless> allenap: not quite sure how, its just a feeling
<allenap> lifeless: Yeah, I feel the same way.
<allenap> Right, properly off to bed now.
<wgrant> Night allenap.
* StevenK changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: StevenK | https://code.launchpad.net/launchpad-project/+activereviews
 * StevenK assumes it is no longer Tuesday or that gmb is actively reviewing. Pointers about both assumptions welcome.
#launchpad-dev 2011-03-30
<lifeless> thnaks
 * lifeless whines about BranchCollection more
<StevenK> lifeless: I'm chanelling thumper, but "Fix it, dear Elisa" ?
<lifeless> StevenK: https://bugs.launchpad.net/launchpad/+bug/745310
<_mup_> Bug #745310: Person:+branches timeout <regression> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/745310 >
<StevenK> wgrant: http://pastebin.ubuntu.com/587111/ makes me sad. It's obvious what it should be, but why was it the other ...
<wgrant> StevenK: blame
<StevenK> blame says jelmer and a 10k rev no
<StevenK> So it's been like that for a while ...
<wgrant> Oh, this is in archivepublisher, not python-debian.
<wgrant> Huh.
<StevenK> Yes
<wgrant> Well, the test docstring is reasonably clear..
<StevenK> wgrant: I should commit that change and re-ec2?
<lifeless> StevenK: its overly generic
<lifeless> StevenK: access patterns for product scope vs person scope are not efficiently reusabke
<wgrant> StevenK: It was changed as part of his port to python-debian. So just fix it.
<lifeless> but our code structure is so heavily geared for code reuse, I suspect i need to entirely ditch the layers to fix it.
<StevenK> wgrant: You think ec2 or it's safe to lp-land? (That was the only failure.)
 * StevenK glares at Tasque
<wgrant> StevenK: It's already been through ec2? lp-land.
<StevenK> Which just SEGV'd. Orsum.
<StevenK> wgrant: Yes, so doing so.
<wgrant> cjwatson: Your branch failed ec2. The test deb didn't have the necessary Pre-Depends (I've fixed that), but python-apt doesn't support data.tar.xz.
<StevenK> Or multiarch ...
<wgrant> huwshimi: Have you had a chance to look at my JS suggestions for your branch?
<huwshimi> wgrant: I had a quick look at it. I will have a look again in a minute. It is certainly cleaner than what I pushed. Generally this kind of node creation is very resource expensive (hence why people often do it with strings or arrays), but I'm more than happy to merge your changes.
<wgrant> huwshimi: Yeah, I imagine it could be slightly more expensive.
<huwshimi> wgrant: I'm not sure what it's like with YUI. It might be fine
<cjwatson> wgrant: oh blasted thing
<cjwatson> wgrant: Debian was going to work around that, IIRC
<cjwatson> wgrant: where's the relevant code?
<wgrant> cjwatson:                     apt_inst.debExtract(deb_file, tar_checker.callback, file)
<wgrant> nascentuploadfile.py, around like 734
<wgrant> *line
<cjwatson> IIRC that was blocked on adding xz to python
<cjwatson> or something joyous like that
<cjwatson> though IIRC there was going to be a workaround with a subprocess
 * cjwatson hunts down where he saw that
<cjwatson> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=556407
<wgrant> Oh, right, I remember something like this.
<wgrant> But it was a while ago.
<cjwatson> debExtract not working is lame, though.
<wgrant> It attempts to gunzip it instead.
<wgrant> gzip is the default, I guess.
<cjwatson> that really shouldn't be that desperately hard to fix.
 * cjwatson goes looking
<cjwatson> (though don't expect a solution tonight, it's late)
<StevenK> "Early"
<wgrant> Sure, thanks.
<StevenK> :-)
<cjwatson> oh, should be fairly trivial
<cjwatson>       else if(strcmp(".lzma", &Chunk[strlen(Chunk)-5]) == 0)
<cjwatson>               Comp = "lzma";
<cjwatson> that kind of thing
<wgrant> Right, but what handles Comp?
 * StevenK waits for devel to be repacked
<cjwatson> ExtractTar in apt-inst, and it's just a program name
<wgrant> Oh, that's easy, then.
<cjwatson> actually I think it might need a slight tweak in apt
<cjwatson> but still, nothing especially hard
<cjwatson> I might make mvo do it
<wgrant> Handy having Platform at your disposal :)
<cjwatson> he could probably do it twice as quick and once on Sundays
<wgrant> Yup.
<StevenK> The changes to apt and python-apt will need to be backported to Lucid though, right?
<wgrant> Ja.
<StevenK>         Updating python-debian to revision 186
<StevenK> Text conflict in lib/debian/debian_support.py8ing stream:Estimate 11/41
<StevenK> bzr fail.
<cjwatson> lucid-cat plus ppa:launchpad, right?
<StevenK> But I suspect it's my fault.
<StevenK> cjwatson: Yeah, it will need to hit both
<lifeless> thumper: btw, select expressions can be used with with now in storm
<thumper> oh cool
<lifeless> With('name', Select(..)) [or get it from a result set or whatever)
<StevenK> Damn it, db-devel, stop failing
<StevenK> shell_6 [test no test results failed] [42 seconds]
<StevenK> Wheeee
<StevenK> Ahh, xfvb, my old nemesis
<lifeless> Error 324 (net::ERR_EMPTY_RESPONSE): Unknown error on lpbuildbot :(
<StevenK> lifeless: For which page?
<lifeless> the build that died
<StevenK> Hm. It worked for me
<lifeless> >< 117 /   41  Person:+branches
<wgrant> lifeless: By the number of soft timeouts I presume it's real.
<wgrant> And a regression.
<lifeless> wgrant: yes, its the bug I'm bitching about
<wgrant> Ahh
<lifeless> we use the same code to query for merge proposals in one scope
<lifeless> and unscoped for a person
<lifeless>  /totally/ different criteria
<LPCIBot> Project devel build #589: STILL FAILING in 5 hr 0 min: https://lpci.wedontsleep.org/job/devel/589/
<lifeless> can has review https://code.launchpad.net/~lifeless/launchpad/bug-745310/+merge/55459
<StevenK> lifeless: It looks good, but is it possible to check in a test that it isn't going to generate a bong query for the general case?
<lifeless> StevenK: if we could define the general case, perhaps
<StevenK> Person:+branches sounds like a good one
<lifeless> its a special case
<lifeless> the schema is not optimised for querying
<StevenK> lifeless: I'm just concerned that you're fixing a fix, and there is no tests to confirm it isn't noddy.
<lifeless> we're going to have to touch this again and again and against
<lifeless> StevenK: I'm not sure that it isn't noddy
<StevenK> lifeless: Okay, so your general plan is for you or someone to iterate over it?
<lifeless> StevenK: without about 100GB of data, I can't prove it one way or another
<lifeless> StevenK: thats what we've been doing for all timeout fixes so far; checking that query X is issued is easy, being sure that query X is what we need is hard
<StevenK> lifeless: r=me
<lifeless> in fact, I can go futher
<lifeless> the query here *will* be noddy, just less noddy than the one optimised for product/package scopes
<lifeless> StevenK: thanks
<StevenK> Oh, sigh
<StevenK> I voted on my own MP, not yours
<lifeless> StevenK: -lol-
<lifeless> StevenK: its still noddy because it unions two querie together that probably are faster done directly.
<lifeless> StevenK: which speaks to the heart of why I don't like (having taken my time to think about this) the 'Collection' approach.
<lifeless> well, this part of the implementation
<lifeless> I'm  not ready to critique the entire thing yet, just speaking out of frustration
<StevenK> lifeless: Swap you: https://code.launchpad.net/~stevenk/launchpad/dsd-invalid-versions/+merge/55461
<StevenK> It's a little larger than yours, so not quite a fair swap :-)
<StevenK> lifeless: Thank you for the review.
<jtv> StevenK: which LP configs does cron.publish-ftpmaster run under?  dogfood, staging, qastaging, production, ftpmaster, ftpmaster-publish, â¦?  I need to add to those configs.
<jtv> Possibly only to some.
<StevenK> ftpmaster and ppa
<StevenK> I think
<StevenK> jtv: lp-production-crontabs will help
<jtv> Ah, of course.  Thanks.
<StevenK> jtv: But certainly dogfood, so that will need fixing too
<jtv> StevenK: BTW you mean lp-crontabs, right?
<jtv> Oh, it _contains_ a dir lp-production-crontabs.
<jtv> Ah no, I am fatally confused.
<wgrant> jtv: What do you need to add to those configs?
<jtv> The stuff I added in https://code.launchpad.net/~jtv/launchpad/db-bug-55798/+merge/55174
<wgrant> jtv: Only ftpmaster runs cron.publish-ftpmaster. Although dogfood *can*, we mostly maintain its config locally.
<wgrant> Er, ftpmaster-publish, not ftpmaster.
<jtv> Then we may want to set up, configure, and maintain a pair of run-parts directories for publish-ftpmaster runs there.
<wgrant> jtv: Those things really don't belong in the tree :/
<jtv> That's why it's configurable.
<wgrant> Why is one fo them publish-distro.d, and the other finalize.d?
<jtv> Better names welcome.
<jtv> It's just that I couldn't bring myself to call one pre-publication.d when it runs _after_ publish-distro.
<jtv> In fact I rather got the impression (also based on what you said) that that one might properly belong in the publish-distro script itself.
<wgrant> Also, PublishFTPMaster is terrible name, but I can't think of anything better since publish-distro is inappropriately taken.
<StevenK> PublishDistribution
<wgrant> No, too confusing until we fix publish-distro
<StevenK> ThePublisherOfDoomDeathAndDestruction is apt
<wgrant> Yes.
<cody-somerville> hahahahah, apt!
<jtv> This does explain why I had such trouble coming up with good names and docstrings, I think.
<jtv> StevenK: do I sense a bit of "See I am become Death, the destroyer of Windows" there?
<wgrant> jtv: You've a function in PublishFTPMaster to append -distscopy, but you don't seem to use it everywhere.
<StevenK> No, I was channeling something
<jtv> wgrant: oh, thanksâI'll check
<jtv> wgrant: I do see it used
<wgrant> jtv: But not everywhere.
<jtv> True.  Am fixing.
<jtv> Found 1 place that neglected to use it.
<wgrant> jtv: Also, ARCHIVE_SUFFIXES is surely duplicating something esle.
<jtv> Again, true.
<jtv> pocketsuffix, I think.
<jtv> Ah, no, that's different of course.
<jtv> But highly similar.
<StevenK> Wheee.
<StevenK> Now buildbot is failing like Jenkins is.
<StevenK> Difference: True != False: memcached is live but should not be.
<wgrant> Ah, lucid_lp this time.
<wgrant> It's because the slave was killed badly when the master was restarted.
<wgrant> spm: Hi, lucid_lp needs a memcached massacre.
<wgrant> spm: Might as well bounce all the slave bits there.
<wgrant> Since the build has failed.
<StevenK> lifeless: I note you're up for QA
<wgrant> StevenK: Could you review lamont's buildd changelog entry?
<wgrant> https://code.launchpad.net/~lamont/launchpad/lp-buildd-76/+merge/55378
<StevenK> Ah, thanks
<StevenK> Lookink
<StevenK> wgrant: All done.
<wgrant> Thanks.
<wgrant> lifeless/StevenK: Could one of you mentor https://code.launchpad.net/~huwshimi/launchpad/ajax-time/+merge/55257?
<poolie> wgrant, re your recent post, did you ever see http://haml-lang.com/
<wgrant> Huh, interesting.
<huwshimi> Noooooo
<poolie> huwshimi, ?
<poolie> i don't mean the ruby bit, just the syntax
<wgrant> It's an interesting idea.
<wgrant> I'm not sure if it's good or not.
<spm> wgrant: okidoki
<huwshimi> poolie: Oh, I've just had fun times with haml in the past
<poolie> fun or 'fun'? :)
<poolie> i'm not sure either
<poolie> getting away from the specific syntax of xml may be good
<StevenK> From The Register: MySQL.com hacked via... SQL injection vuln
<spiv> From the URL, I was hoping it would be a programming langauge that was a homage to Mark Hamill.
<wgrant> StevenK: Stones, glass houses.
<poolie> :)
<StevenK> :-(
<spiv> So I was already disappointed without trying it!
<poolie> :)
<spiv> (Not sure how you'd make such a language, but if people can manage to put nearly 3000 toothpicks in their beard and post the video to the internet then surely anything is possible)
<huwshimi> poolie: Yes, non-xml templating languages are good
<StevenK> Actually, this sounds useful: http://www.theregister.co.uk/2011/03/30/dynatrace_ajax_firefox4_ie/
<poolie> it reminds me a bit of latte, which i used years ago
<jtv> wgrant: had some network trouble earlierâ¦ I unified the ARCHIVE_SUFFIXES across two old modules that did it ad hoc (and removed it from mine which turned out not to need it any more).  Did you say anything after that?
<wgrant> jtv: No, that was all I saw with a quick glance.
<huwshimi> poolie: I think haml goes to far. It is actually really nice for writing HTML in, but I think it abstracts too far from HTML
<jtv> Also, it says "markup haiku."  Don't we normally like our formal languages to avoid the kind of linguistic ambiguity that's the whole point of haikus?
<wgrant> It's Ruby, what do you expect? :)
<poolie> for the real 60s retro flavour suggested by the theming it ought to be all caps :)
<spiv> jtv: it's poetry, so beauty is truth, truth beauty, etc.
<jtv> I'll leave that to the test suite, thank you very much :)
<spm> wgrant: the entire buildbot whatsit is stabbed. have deliberately not restarted the existing builds, so you'll need to force when ready.
<jtv> I'm hungry for Japanese food now.
<wgrant> spm: Both slaves are clean?
<spm> in that I shut down everything that was running. yes. afaik, we don't do any cleaning beyond that?
<wgrant> Well, there's non memcacheds or anything left on either slave?
<spm> if we have to, that's a pretty awesome bug given the purpose.
<wgrant> s/non/no/
<lifeless> spm: you used to have a script for pqm
<lifeless> spm: that would kill /every/ process running in a chroot
<lifeless> spm: that would be the thing to run
<spm> sure. either way, the processes are wiped and restarted.
<wgrant> Thanks.
<wgrant> Forcing.
<wgrant> Hopefully for the last time.
<wgrant> StevenK: Hi.
<StevenK> wgrant: Hm?
<wgrant> i-528A0905 needs killing for the same reason.
<wgrant> It was cursed by db-devel build 503.
<LPCIBot> Project devel build #590: ABORTED in 3 hr 46 min: https://lpci.wedontsleep.org/job/devel/590/
<wgrant> Thanks.
<StevenK> And slave killed.
<StevenK> Shall I request a build of devel, or something should come along soon?
<wgrant> Nothing will be around for at least 4 hours.
<wgrant> Please force.
<StevenK> Just devel or both it and db-devel?
<lifeless> wgrant: can you look at the soyuz ppa +subscribers on qastaging
<lifeless> qa for 741092 - I don't admin any private ppas
<wgrant> StevenK: I think db-devel is fine, so no need for a build there.
<StevenK> lifeless: We have these people called losas, who can fix that for you
<lifeless> StevenK: we we're also trying not to interrupt when we don't need to
<wgrant> StevenK: Also these people called wgrant, though.
<wgrant> 38 queries/external actions issued in 0.43 seconds â¢
<StevenK> wgrant: Okay, devel is waiting for Jenkins to start a build slave.
<lifeless> StevenK: because they have a backlog longer than ours per-head
<wgrant> 38 queries/external actions issued in 0.30 seconds â¢
<wgrant> StevenK: Thanks.
<lifeless> wgrant: nifty
<wgrant> lifeless: So, it is too fast.
<spm> StevenK: that's a pack of lies. we are not people, and resent being so labelled.
<lifeless> ohI'msorry
<StevenK> spm: How would you prefer to be addressed?
<wgrant> Demigod?
<StevenK> Haha
 * spm tears up another of wgrant's cake IOU's.
<wgrant> StevenK: Could you review https://code.launchpad.net/~wgrant/launchpad/packageset-owner-preservation/+merge/55466?
<StevenK> wgrant: r=me
<wgrant> StevenK: Thanks.
 * StevenK glares at patch-2208-59-0.sql, mawson and lifeless
<lifeless> StevenK: why?
<StevenK> Because it's taken seven minutes to apply so far
<wgrant> It might be a while.
<lifeless> is it doing stuff
<wgrant> As well as breaking the build several times :P
<lifeless> or is it deadlocked because you didn't shut everything down ?
<StevenK> Everything ought to be stopped
<lifeless> it has to rewrite 250MB
<lifeless> so it should be pretty snappy
<lifeless> check top, iotop, etc
<StevenK> mawson can not be described as snappy
<StevenK> It's still running UPDATE
<lifeless> which one
<StevenK> I have no idea, I'm going by ps output
<lifeless> select current_query from pg_stat_activity;
<lifeless> should give you exactly 2 rows
<lifeless> StevenK: speaking of qa -Revision 12689
<StevenK> lifeless: There is only one UPDATE in -59
<lifeless> StevenK: this will tell you if anything else is hanging around that could be blocking it
<StevenK> lifeless: https://pastebin.canonical.com/45423/
<lifeless> StevenK: cool
<StevenK> So I just have to wait?
<wgrant> And wait.
<StevenK> Haha
<stub> So the BugTask.heat update is too slow for a standard rollout?
<StevenK> stub: This is mawson, so the data means nothing
<stub> k
 * stub waits for the staging update
<lifeless> stub: I missed a bit, then we've had continual buildbot fail
<lifeless> and I mean continual
<stub> \o/
<wgrant> I think four or five db-devel builds have died in various ways so far today.
<lifeless> stub: should we try manually, expedite rollback if needed?
<lifeless> stub: as in , ask for staging to be quiesced, and you pump it through slon?
<stub> If we do it manually, we need two losa interrupts (one to apply, one to rollback). If we do it with the standard update, we may need one (to rollback, if necessary)
<lifeless> spm: ^ up for it?
<stub> Hmm.... what I *can* do is time how long it takes to do the UPDATE on a slave. I just need to roll it back before I commit and break replication.
<spm> yes, but no. currently mid a security update on an LP service and hoping to get the u1 migrtaion actually started in the next hour before I EOD
<lifeless> stub: can't do that with out the new column ?
<StevenK> 2011-03-30 05:25:26 INFO    Applying /srv/launchpad.net/codelines/trunk/database/schema/patch-2208-59-0.sql
<StevenK> 2011-03-30 05:58:54 DEBUG   Committing changes
<stub> lifeless: begin; add column; update; rollback
<StevenK> 2011-03-30 05:59:00 INFO    2208-59-0 applied just now in 0.0 seconds
<StevenK> FAIL
<lifeless> so, 25 minutes
<stub> on mawson - should be ok.
<StevenK> The fail is the 'in 0.0 seconds'
<huwshimi> Do we still have sparklines in Launchpad?
<lifeless> I doubt it
<stub> Dunno why you would get 0.0 on mawson - that code was tricky. Might be an edge case with a single db patch on unreplicated env?
<stub> We pulled sparklines
<lifeless> huwshimi: ...why
<StevenK> stub: Could be.
<huwshimi> lifeless: I just found a bug about them and was wondering if I should kill it
<huwshimi> lifeless: bug #383924
<_mup_> Bug #383924: Sparklines displayed over row below in Opera <lp-code> <ui> <Launchpad itself:Triaged by thumper> < https://launchpad.net/bugs/383924 >
<lifeless> kill it
<wgrant> They were pulled temporarily in July 2009 due to a bug.
<wgrant> They were to be returned within two months.
<spiv> wgrant: I guess no-one missed them then ;)
<huwshimi> lifeless: https://bugs.launchpad.net/launchpad/+bugs?field.searchtext=sparkline&orderby=-importance&search=Search&field.status%3Alist=NEW&field.status%3Alist=INCOMPLETE_WITH_RESPONSE&field.status%3Alist=INCOMPLETE_WITHOUT_RESPONSE&field.status%3Alist=CONFIRMED&field.status%3Alist=TRIAGED&field.status%3Alist=INPROGRESS&field.status%3Alist=FIXCOMMITTED&field.assignee=&field.bug_reporter=&field.omit_dupes=on&field.has_p
<huwshimi> atch=&field.has_no_package=
<huwshimi> lifeless: Kill them all?
<lifeless> bug 284400 seems like a reasonable defect
<_mup_> Bug #284400: "Package bugs" page has no indication of how numbers have changed <feature> <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/284400 >
<lifeless> the others seem to be implementation bugs about the sparklines we had
<lifeless> we might want sparklines for 284400, but I doubt it, context doesn't really fir
<lifeless> *fit*
<lifeless> huwshimi: did you end up getting the url?
<huwshimi> lifeless: Nope. I haven't landed the first lot yet
<lifeless> please file a bug or something if you need server side changes whipped up for you
<huwshimi> lifeless: It feels like getting the url from header is a big hack. I might ask around first
<wgrant> It is a big hack, but I don't think you have much choice.
<lifeless> http://stackoverflow.com/questions/4230876/re-associate-ajax-call-with-original-url-using-jquery
<lifeless> huwshimi: we might already set Location
<lifeless> though that can be legitimately different
<lifeless> http://forum.jquery.com/topic/jquery-ajax-response-url is relevant
<lifeless> 'If it was really important, you could do some voodoo with sending
<lifeless> along the current URL as a custom response header'
<lifeless> :P
<huwshimi> lifeless: OK you win :)
<huwshimi> lifeless: What exactly do I need to ask to be done? Is it unhelpful if I just ask for a new header?
<lifeless> that would be fine
<lifeless> you need 'a sensible url to show in the ajax timeline put in the response sent to every ajax request'
<lifeless> and we can probably sensibly say 'api requests on non api domains are ajax requests'
<lifeless> the submission url with parameters is probably sensible on all ajax requests
<wgrant> D:
<wgrant> stub: I see that you found zope.tal about as extensible as I did.
<stub> wgrant: I don't remember. I had that part of my brain removed.
<stub> Nah nah nah can't hear you...
<wgrant> I am glad there is precedent :)
<huwshimi> lifeless: Would this be low priority?
<wgrant> It's at least High, because it's hindering performance improvements.
<wgrant> And it's easy.
<huwshimi> wgrant: OK :)
<lifeless> huwshimi: I would say high
<wgrant> Huh, that monkeypatch worked, and stuff doesn't even OOPS.
<wgrant> Incredible.
<lifeless> and then suggest you bat your eyelids at wgrant :P
<wgrant> lifeless: That was the plan.
 * wgrant declares structure obsolete.
 * huwshimi bats eyelids at wgrant
 * huwshimi is not sure if he's doing it right
<huwshimi> OK people, how do I land a branch that has no bug it is addressing? "ec2: ERROR: Branch doesn't have linked bugs and doesn't have no-qa option set."
<wgrant> huwshimi: File a bug, or --no-qa
<StevenK> Add --no-qa
<StevenK> Or file a bug, as wgrant says
<wgrant> Depending on how qa-worthy it is.
<lifeless> if you want the ability to say 'oh fuck this broke things'
<lifeless> file a placeholder bug
<huwshimi> ok maybe I'll file a bug
<lifeless> if you reckon it doesn't need checking on qastaging, use --no-qa
<huwshimi> I don't really want to break things
<lifeless> huwshimi: breaking things is a little orthogonal :P
<lifeless> wow
<lifeless> some folk really do get confused
<huwshimi> lifeless: Well this way I can make sure it doesn't go live if it does break things right? Or did I misunderstand?
<lifeless> huwshimi: it gives you another set in the process
<lifeless> huwshimi: the question is, will you notice something on qastaging that you wouldn't notice on your machine
<huwshimi> lifeless: I guess probably not.
<huwshimi> lifeless: But if that was the case then what's the point in qastaging?
<lifeless> huwshimi: its entirely up to you; we try hard to be able to rollthings back easily
<lifeless> huwshimi: many things will only show up on qastaging
<lifeless> huwshimi: for instance, db permissions; scaling; icing changes
<wgrant> buildbot seems to be happy this time.
<wgrant> I guess there'll be a UPS failure in the DC soon.
<StevenK> Who's a little ray of sunshine today.
<wgrant> Shh
* StevenK changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<jelmer> Has the timeout been tightened again? I can no longer view my code page, it now consistently times out.
<StevenK> jelmer: Known regression.
<wgrant> Fix waiting for buildbot.
<jelmer> StevenK: Ah - thanks.
<stub> jtv: https://code.launchpad.net/~stub/launchpad/drop-branchrevision-id/+merge/55480 is where I got to. Unfortunately, after that patch I end up with a primary key on branchrevision that cannot be dropped because 'constraint does not exist', so I think there are some other links lurking in the schema we haven't tracked down yet.
<wgrant> lifeless: Are you checking other callsites before you add preloading?
<lifeless> wgrant: briefly yes
<wgrant> p-r-f is broken because ProductSet.all_active is trying to access emailaddresses.
<lifeless> wgrant: oh gawd
<wgrant> Yes.
<lifeless> wgrant: I didn't see that call site at all
<lifeless> wgrant: is it not under lib/ ?
<wgrant> lifeless: I bet it iterates over ProductSet.
<lifeless> wgrant: OTOH the +all page loads snappily now
<wgrant> Heh
<wgrant> Indeed.
<lifeless> uhm
<wgrant>         products = getUtility(IProductSet)
<wgrant>         for product in products:
<lifeless> i'm inclined to broaden its access
<lifeless> it should
<lifeless> bah
<lifeless> it should be moved to an API client eventually anyway
<wgrant> That will make it load lots, though.
<wgrant> Same issue as populate-arcdhive.
<lifeless> thats true
<lifeless> move it off of __iter__ ?
<wgrant> I think all this stuff needs looptunerising or deleting.
<lifeless> I'm past EOD and TL meeting is at awful-am
<stub> lifeless: How should we handle logging with parallel garbo? I could buffer the output from each task, and emit it when the task is complete. Or I could log to individual log files rather than stdout. Or what?
<lifeless> I can JFDI something raw
<lifeless> or perhaps you could do it ?
<wgrant> lifeless: It can be dead for a couple of days.
<wgrant> I am mostly EODed and attempting to stick to it this week.
<wgrant> (utterly failing, but attempting)
<lifeless> wgrant: ah true; file a bug for me and I'll pay the piper tomorrow
<lifeless> wgrant: welcome to canonical
<wgrant> Thanks.
<stub> You're at work until you shut down your IRC client :)
<lifeless> stub: logging is threadsafe; I think using a sublogger with the loop class name will be sufficient
<stub> lifeless: I'm talking about making the output readable
<lifeless> e.g. logger = somelogger.get(self.__class__.__name__)
<stub> Maybe it will be readable with the output interleaved...
<lifeless> stub: as long as there is a [prefix] blah goes here
<lifeless> it should be tolerable
<wgrant> lifeless: Bug #745512
<_mup_> Bug #745512: product-release-finder broken <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/745512 >
<lifeless> only matters when we actually read it after all
<lifeless> wgrant: thanks
<wgrant> And now I am gone for at least a while.
<stub> Hmm.... prefix could be a quick fix and good enough... yeah.
<lifeless> stub: thats what I was suggesting :) - use an outputter that outputs the logger name, and tweak the logger name for the loop when we construct/access the logger
<lifeless> stub: I failed to separate what and how though :P
<stub> I followed and was rephrasing in my agreement
<lifeless> \o/
<lifeless> stub: I totally broke code.l.n/~user/ yesterday
<lifeless> stub: my clever optimisation for projects with lots of private branches + open merge proposals caused 2xseq scans on Branch for per-user counts
<stub> eggs... omlettes...
<stub> Mmm.... omlettes....
 * stub drools
<lifeless> mmmm
<adeuring> good morning
<jtv> hi adeuring
<adeuring> hi jtv!
<lifeless> jml: ping
<jtv> StevenK: your DSD diff invalidation card is still under Review on the board, but the the branch is merged.
<jtv> Thank you :)
<jtv> In fact the bug is marked Fix Released.
<lifeless> jml: I'm wondering if we can set the hard timeout for pages feature squads are creating/changing to 5 seconds
<jtv> wgrant: any other notes about my publish-ftpmaster branch by the way?
<wgrant> jtv: I didn't end up having more than a glance through it. I will look at it in depth tomorrow.
<jtv> That'd be great, thanks.  I'm emotionally eager to move on to the next similar job, but I'd like to have confidence that my current branch is stable.
<wgrant> Indeed. It looks mostly sensible.
<rvba> I'm having a strange failure in the test suite: http://paste.ubuntu.com/587260/
<rvba> doest it look like a known spurious error?
<jtv> wgrant: Also, it seems like rather a lot of change to Q/A and deploy in one cycle.  But waiting won't help with that.
<jtv> rvba: hang on, looking
<wgrant> rvba: That's not a spurious failure.
<wgrant> jtv: Yeah, but I'm not sure we can do much to avoid that :/
<jtv> rvba: Did you use unittest.TestCase instead of lp.testing.TestCase maybe?
<jtv> wgrant: exactly.
<rvba> wgrant: DistroSeriesMissingPackagesFunctionalTestCase is indeed a new class so this looks like a genuine error but "./bin/test -cvv -t translationimportqueue.txt" passes
<wgrant> rvba: Could you pastebin/push your changes?
<jtv> Did you override setUp and neglect to call the super?
<wgrant> TestCase is normally good at warning about that, IIRC.
<rvba> jtv: no, just fixed that
<wgrant> But that would be the obvious thing.
<rvba> wgrant: hang on
<wgrant> Maybe only TestCaseWithFactory warns, though.
<lifeless> wgrant: the warning is in testtools.TestCase
<jtv> True, setUp warns about it.  rvba, I take it you didn't define an __init__ in the test case?
<stub> jtv: https://code.launchpad.net/~stub/launchpad/drop-branchrevision-id/+merge/55480 is where I got to. Unfortunately, after that patch I end up with a primary key on branchrevision that cannot be dropped because 'constraint does not exist', so I think there are some other links lurking in the schema we haven't tracked down yet.
<rvba> jtv: no
<jtv> stub: I wonder if the transplant I worked out is perhaps needlessly complex.
<jtv> Instead of using an existing unique index, I think it'd be simpler with an existing unique constraint.  Do we have one?
<stub> Yes - two. That was my next issue - I think we end up with a constraint that things it is both a UNIQUE constraint and a PRIMARY KEY constraint :)
<jtv> rvba: I haven't used self.oopses in tests yet, personally; did you track through the code to see that it's really supposed to be created?
<stub> So if we can swap the revision__revision__branch__key UNIQUE constraint easier, that would be better.
<rvba> jtv not sure what you mean ...
<jtv> stub: or an index that thinks it belongs to both a PRIMARY KEY constraint and a UNIQUE constraint.
<stub> yer. looks messier than I initially suspected anyway.
<rvba> wgrant: modifs to tests http://paste.ubuntu.com/587267/
<rvba> wgrant: the whole branch https://code.launchpad.net/~rvb/launchpad/dds-add-missingpackages-page
<jtv> stub: Yes.  I'd have to get back into the details but IIRC the transplant I documented ran into a dependency relationship that more or less needs to be inverted.  That's probably not the case if we use the unique constraint instead of the unique index.
<stub> That sounds promising. I haven't scoured the schema documentation yet, so I just followed your notes for the initial cut.
<jtv> stub: I'm on feature rotation nowâ¦ would it make sense for me to dive back into this after?
<stub> jtv: Whoever gets to it first. I've got other things for today. But we are looking at some denormalization and adding a timestamp to BranchRevision, so pulling id and unnecessary indexes would be good to do at the same time so we don't have a disk space problem.
<wgrant> +inf
<jtv> overflow trap
<wgrant> Damn.
<jtv> stub: I am somewhat puzzled by the notion of piggybacking the addition of a timestamp column onto the urgent removal of an int column to save disk space.  :)
<stub> Its not *that* urgent
<jml> lifeless: sure, sounds good, but can we excuse translations & bug subscription for now?
<lifeless> jml: yeahm wouldn't be fair to just dump it on existing work.
<jml> lifeless: cool.
<lifeless> jml: i'll mail the list
<jml> lifeless: ta.
<jml> wow, we haven't had a successful test run since I stopped last night
<lifeless> no, its been bad today
<wgrant> We're nearly there.
<rvba> wgrant: did you by any chance pick up some obvious mistake in my code?
<wgrant> rvba: Nothing obvious, sorry :/
<wgrant> Yay, 800/1800 structure keywords removed.
<rvba> wgrant: that's allright, I'll keep on investigating :-)
<jtv> rvba: did you say it failed intermittently?
<rvba> jtv: it's more bizarre than this:
<lifeless> night all
<jtv> night lifeless!
<rvba> it failed when I launched the whole testsuite on ec2
<rvba> it failed when I launched testr run --failing
<rvba> but ./bin/test -cvv -t translationimportqueue.txt passes
<jtv> Oh, I hate those.
<jtv> Heisenbug.
<rvba> sort of yes
<jtv> And this is a custom oops reporting setup, which we've sort of abandoned now AFAIK.  There may easily be something wrong with it.
<jtv> Layer setup or isolation issue maybe?
<jtv> What layer is the test in?
<rvba> LaunchpadFunctionalLayer
<jtv> Hmm can't really be anything missing there then can there?
<jtv> Drat.
<rvba> jtv: here is the diff for this test file http://paste.ubuntu.com/587267/
<jtv1> rvba: have to go otp now
<henninge> What's the easiest way to render some TAL into a string?
<jtv> henninge: don't you __call__ the view?
<jtv> rvba: I'm backâ¦ my connection went awol after I asked: is there no cleanup for the "set_derived_series_ui_feature_flag"?
<jtv> allenap: where do you retrieve the selection of packagesets that a derived distroseries is based on?
<rvba> I don't think so
<rvba> perhaps I know what's happening ... I'll relaunch the test suite ...
<bigjools> jtv, henninge: call .render()
<wgrant> bigjools: Is lp-buildd-76's commit message meant to be empty except for the tags?
<wgrant> Ah, just on the MP.... hm.
<wgrant> Landed with a commit message.
<bigjools> wgrant: bug in lp-land :/
<henninge> bigjools, jtv: I was wondering if I could do without a view.
<henninge> but maybe I can create a simple one for this purpose.
<bigjools> henninge: no, templates need views to work
<henninge> ok, thanks
<bigjools> in the same way they need a context
<henninge> well, the view provides the context, doesn't it?
<bigjools> henninge: grep dor create_view
<bigjools> for*
<wgrant> Mm, you could probably grab a PageTemplate manually. But that is bordering on more evil than the problem deserves.
<henninge> yes, I was coming to that next. Why is that evel?
<henninge> evil, even
<wgrant> I don't think it's exactly light-weight, and I'm not sure how well it will work to instantiate one manually.
<wgrant> Which bit of inline HTML are you replacing?
<henninge> I got the same thing in at least four places
<henninge> lp.translations.browser.potemplate.POTemplateUploadView
<henninge> around line 492
<henninge> It's creating a <ul> of file names.
<wgrant> Ugh. We don't use templates for notifications anywhere else, but that just about deserves one :/
<wgrant> Could you put them into the page's template?
<wgrant> No.
<wgrant> Because they need to be in the session.
<wgrant> Hmm.
<wgrant> Awkward.
<henninge> I know it is
<henninge> Acutally, I wrote that code in another life ...
<jml> allenap: thanks! I didn't know that we had multiple security.py files.
<wgrant> jml: Somebody is bound to come along and "s += untrusted_variable", and structured() won't stop that. MarkupSafe does, which is nice.
<spiv> jml: so you're more secure than you thought!
<jml> wgrant: so it's more that it makes it what variables are being used less obvious
<wgrant> jml: Right. And it won't work after next week if my plan goes as I hope.
<jml> wgrant: your plan... to use MarkupSafe?
<wgrant> jml: Something like that.
<wgrant> It remains to be seen how well this is going to work out, but I fixed all the FormatterAPIs up this evening and it seems to be OK.
<jml> wgrant: cool.
<wgrant> jml: MarkupSafe fixes that sort of thing by escaping any unsafe strings that are concatenated with it.
<wgrant> So Markup('<b>') + 'untrusted' + Markup('</b>') does the right thing, if you want to do things like that.
<jml> wgrant: as long as it's on the LHS of the operator, I guess.
<wgrant> jml: Seems to work either way.
<jml> wgrant: I skeptate.
<wgrant> >>> Markup('<foo>') + '<bar>'
<wgrant> Markup(u'<foo>&lt;bar&gt;')
<wgrant> >>> '<bar>' + Markup('<foo>')
<wgrant> Markup(u'&lt;bar&gt;<foo>')
<jml> Huh. I wonder how that works.
<wgrant> As do I.
<jml> (incidentally kids, that was the scientific method in action)
<wgrant> Ah.,
<wgrant> __radd__
<wgrant> (it's used by pylons and jinja2 and mako, so it seems to be a reasonable choice)
<jml> uh.
<wgrant> Oh?
<jml> that was a typo for 'huh'
<jml> I didn't know about __radd__
<wgrant> Ah.
<wgrant> I've not seen it used before :/
<wgrant> That wasn't meant to happen.
<jml> I'm on the conflict, btw.
<jml> got distracted waiting for make compile so I could run tests on the resolved code
<jml> anyone around who's familiar with test_sharing_information?
<bigjools> wow, 3 separate conflicts
<jml> yeah.
<jml> I guess we've done more work on db-devel recently.
 * jml famished
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #591: FIXED in 5 hr 25 min: https://lpci.wedontsleep.org/job/devel/591/
<bigjools> yeah mostly my squad I think
<deryck> Morning, folks.
<bigjools> howdy deryck
* danilos changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: danilos | https://code.launchpad.net/launchpad-project/+activereviews
<LPCIBot> Project windmill build #121: STILL FAILING in 1 hr 10 min: https://lpci.wedontsleep.org/job/windmill/121/
<deryck> adeuring, henninge, abentley -- I'll be 5 minutes late for standup, sorry.  Compiling notes from another conversation.
<abentley> deryck: okay, ping when ready.
<deryck> abentley, adeuring, henninge -- ok, ready.  jumping into mumble now.
<benji> gary_poster: I guess I'll move the "Get client branch reviewed" card back to one of the WIP spots... but I don't remember which one it was in, feature work 2?
<benji> I'll also add a card to move to the right channel.
<james_w>         # Create the credentials with no Consumer, then set its .consumer
<james_w>         # property directly.
<james_w>         credentials = Credentials(None)
<james_w>         credentials.consumer = consumer
<james_w> the comment should say why!
<deryck> henninge: http://developer.yahoo.com/yui/3/
<jml> james_w: +1
<allenap> danilos: Hello dude, fancy reviewing a few hundred lines of javascript? https://code.launchpad.net/~allenap/launchpad/dd-initseries-bug-727105-form-submission/+merge/55514
<danilos> allenap, why of course
<allenap> danilos: Thank you very much :)
<bigjools> why on earth does LP reject an email review to an MP with no subject line?
<danilos> allenap, fwiw, there's a conflict in lib/lp/soyuz/scripts/tests/test_initialise_distroseries.py with devel (it's simple, but just fyi)
<jml> umm
<jml> didn't that get fixed yesterday or something?
<jml> bigjools: don't know, actually.
<deryck> henninge: chat in 2?  Running late, as seems my habit lately. :-)
<henninge> deryck: ok
<danilos> allenap, also, what's the +feature-rules line I need for this particular feature?
<allenap> danilos: Oops, looking...
<allenap> danilos: soyuz.derived-series-ui.enabled	default	1	true
<danilos> allenap, thanks
<danilos> allenap, FormActionsWidget looks pretty neat, is this not something that should be used in many of our other forms? do you have plans to factor it out?
<allenap> danilos: Yeah, there's a few bits and pieces in that module that could be quite reusable. They're not up to standard for lazr-js I don't think. I intend to refactor as I need them in future branches for the derived distributions feature.
<danilos> allenap, sounds good, we've also been doing similar stuff with our feature development and could have benefited from the same work, so it'd be nice to share it on the mailing list as well :)
<allenap> danilos: Yeah, good idea. Okay, I'll write something.
<danilos> allenap, danke schÃ¶n :)
<danilos> allenap, I wonder about Camel Case On Buttons, is that the recommended UI practice?
<allenap> danilos: I don't know, I just carried on merrily :)
<jml> it's not.
<jml> I think.
<jml> gah, I don't know any more
<allenap> danilos: Ah, you just mean the words on the button, visible in the UI?
<allenap> Sorry, I got mixed up there.
<jml> there must be a guide *somewhere*...
<jml> https://dev.launchpad.net/UserInterfaceWording
<jml> Buttons and tabs should be Headline Case; the last word capitalized, and all other words capitalized except those three letters or fewer that are prepositions, articles, or conjunctions.
<allenap> jml: Does Launchpad have tabs in the UI?
<allenap> danilos: So, I should change the button wording.
<jml> allenap: I guess not any more.
<jml> allenap: apart from the component tabs.
<allenap> jml: Thanks for digging that page up.
<jml> allenap: np
* abentley changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: danilos, abentley | https://code.launchpad.net/launchpad-project/+activereviews
<allenap> danilos: Thanks for the review :)
<danilos> allenap, you are welcome
<danilos> allenap, yeah, jml is right, no need to change the button text it seems
<danilos> allenap, anyway, all good I'd say, a very nice, nice branch :)
<allenap> danilos: Ah, I did change it, to "Initialize Series".
<danilos> allenap, heh, fair enough :)
* danilos changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: abentley | https://code.launchpad.net/launchpad-project/+activereviews
<gmb> losa ping
<mbarnett> heya gmb
<gmb> Hey mbarnett. Can you run and paste the results of `SELECT * from BugWatch WHERE bug = '710652';` on production for me?
<adeuring> abentley: could you have a lok at this mp: https://code.launchpad.net/~adeuring/launchpad/js-translation-2/+merge/55575 ?
<abentley> adeuring: sure.
<adeuring> thanks
<mbarnett> gmb: http://pastebin.ubuntu.com/587408/
<gmb> Ta
<abentley> adeuring: Could you please use 'complete' rather than 'configured' in the ids, so that it matches the other entries?
<adeuring> abentley: ah, right
<adeuring> abentley: just noticed that i forgot a few tests, for the upstream sync link...
<abentley> adeuring: cool.
<jml> https://code.launchpad.net/~jml/launchpad/old-twisted-bugs/+merge/55569
<jml> abentley: can you please review that?
<abentley> adeuring: in the tests, did you consider using canonical_url to generate the exact expected URLs, rather than using regexes to match?
<abentley> jml: sure, I'll look at that next.
<adeuring> abentley: right, we can do that
<abentley> adeuring: did you consider writing the tests of translation_sync_link_configured etc as view tests rather than browser tests?
<abentley> adeuring: i.e. instead of getting the page and parsing it, you'd just inspect the return value of translation_sync_link_configured?
<adeuring> abentley: well, that's possible. I don't have a real opinion there...
<abentley> adeuring: Well, testing the output HTML is more of an integration test and testing the view is more of a unit test.
<adeuring> abentley: ok, i'll change them
<jml> abentley: thanks.
<abentley> adeuring: Cool.  When I ask questions like this, I'm not necessarily asking for a change, but I do want to understand whether there's a reason you went one way instead of another.
<LPCIBot> Project devel build #592: FAILURE in 5 hr 17 min: https://lpci.wedontsleep.org/job/devel/592/
<benji> I'm getting NotFound exceptions on icon-sprites; does that ring any bells for anyone?
<james_w> could someone help diagnose bug 745801 as it's frustrating testing of one of our projects?
<_mup_> Bug #745801: system-based authorization doesn't store useful credentials in gnome-keyring <amd64> <apport-bug> <natty> <launchpadlib :New> <python-launchpadlib (Ubuntu):New> < https://launchpad.net/bugs/745801 >
<bac> benji: re: icon sprites, look at rev 12697
<benji> bac: thanks
<benji> bac: 12697 of devel or something else?
<bac> devel
<bac> benji: https://code.launchpad.net/~huwshimi/launchpad/private-objects-298152/+merge/53195
<bac> of note you'll see that branch removed icon-sprites.  :(
<benji> oh!  I wasn't getting the connection between "Changed private bug notifications to be more obvious." and sprites.  So in other words "it's broke".
<lifeless> sinzui: bug 745791 is deliberate, no?
<_mup_> Bug #745791: participation links are all blue <Launchpad itself:New> < https://launchpad.net/bugs/745791 >
<sinzui> lifeless: yes. We have not concluded what they should look like
<sinzui> lifeless: I think the bug is legitimate. huwshimi needs to address it, or advice me
<lifeless> bigjools: why does queue acceptance read from the librarian?
<bigjools> lifeless: because it publishes files
<bac> benji: yes.  broken.
<bac> benji: i copied icon-sprites from another branch and it is fine
<bigjools> lifeless: or are you not talking about process-accepted?
<bac> benji: i'll land a fix that restores them
<lifeless> bigjools: https://bugs.launchpad.net/launchpad/+bug/745799
<_mup_> Bug #745799: Timeout accepting packages <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/745799 >
<benji> cool, as long as we know about it
<bigjools> lifeless: let me check
<lifeless> bigjools: bottom of the summary, or look at the oops directly
<bigjools> lifeless: ah it's checking filenames and checksums
<bigjools> lifeless: see lib/lp/soyuz/model/queue.py, setAccepted()
<bigjools> lifeless: it's also showing packagediffs
<jcsackett> anyone else chasing the merge conflicts pqm is nagging about?
<lifeless> jcsackett: I don't think so
<jcsackett> lifeless: dig. it's vexing me, so i'll kill it.
<lifeless> \o/
<lifeless> jcsackett: btw, did you see the bug on merge proposal diff popups ?
<jcsackett> lifeless: i saw it yes. not sure it's a regression, since technically we're still better than we have been since i started looking at the popup. :-)
<lifeless> jcsackett: :)
<jcsackett> i may pick that one up soon; depends how much i feel like cursing the javascript gods.
<lifeless> jcsackett: no worries, I only mention cause you are now familiar with it :P
<jcsackett> lifeless: for a very small value of "familiar" :-P
<jcsackett> and i'm not going to be able to easily resolve those merge conflicts after all. this appears non-trivial.
<jcsackett> sinzui: i just noticed it's slightly past three. mumble time still good?
<sinzui> jcsackett: can we postpone to 4:00. I have to leave early today
<jcsackett> sinzui: hm, any chance we can do 4:15?
<jcsackett> sinzui: if not, 4 can work, 415 would just be easier.
<sinzui> okay
<jcsackett> fantastic. thanks. :-)
<bac> sinzui: here are the changes i've made to the Makefile:  http://pastebin.ubuntu.com/587496/
<bac> unfortunately, the dependency i added to sprite_images is not working and the file is rebuilt whether it exists or not
<bac> i thought removing it from .phony would be the trick
<bac> my makefile fu is atrophied
<lifeless> .phony says 'do not look at disk timestamps', so its certainly a good first step
<sinzui> That is tricky. We only want to use create-image when the position file or sprites.css.in has changed.
<lifeless> however
<lifeless> its transitive
<lifeless> if foo depends on bar
<bac> lifeless: right, that's why removing it made sense
<lifeless> neither foo nor bar can be .phony
<bac> there are no phony dependencies
<lifeless> ok
<bac> i expect 'make sprite_image' to create the file and the next 'make sprite_image' to report nothing to be done
<lifeless> sinzui: db-devel conflicts
<lifeless> sinzui: you may know; is reassign a new menu entry for distributions?
<lifeless> sinzui: or one that was removed
<sinzui> lifeless: I believe it is new. We never admitted it could be done in the past
<lifeless> ok
<lifeless> I think I have a resolution then
<lifeless> http://pastebin.com/vBapxE6w
<lifeless> ^ can has sanity check
<lifeless> \o/ on huw's ajax thingy
<lifeless> deryck: ^ - btw the initial version is in devel so you can see what he's working towards with the need for url info
<deryck> lifeless: ah, ok.  I'll take a peak then.  Thanks
<lifeless> deryck: its controlled by the visible_render_time flag
<lifeless> so you can see it on https://bugs.qastaging.launchpad.net/launchpad
<deryck> ok
<deryck> this is really nice
* abentley changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<lifeless> yeah, its pretty cool
<lifeless> needs the url to be /really/ useful
<lifeless> sinzui: I'll land what I have, i think its no worse
<lifeless> sinzui: ping
<bac> lifeless, sinzui: i got the makefile to work: http://pastebin.ubuntu.com/587507/
<bac> the key is to separate the phony, easy-to-type target from the files
<lifeless> bac: looks fine to me
<lifeless> sinzui: is bug 162510 actually fix released ?
<_mup_> Bug #162510: Person:+delete timeouts : Person merging needs to be done asynchronously <canonical-losa-lp> <chr> <feature> <lp-registry> <merge-deactivate> <qa-untestable> <tech-debt> <timeout> <Launchpad itself:Fix Released by allenap> < https://launchpad.net/bugs/162510 >
<lifeless> ah, bug 728471
<_mup_> Bug #728471: Person:+delete timeouts : Add sanity checks to mergeAsync <merge-deactivate> <qa-ok> <timeout> <Launchpad itself:Fix Released by sinzui> < https://launchpad.net/bugs/728471 >
<jcsackett> can someone tell me if bug 151362 is something we care about? do we support/care about basic given oauth?
<_mup_> Bug #151362: Error: Incorrect padding (base64 error) during authentication <lp-foundations> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/151362 >
<lifeless> we still support basic auth in dev mode
<lifeless> we should either nuke it entirely
<lifeless> or have it robust
<lifeless> (the test suite depends on this at the moment)
<jcsackett> lifeless: dig.
<jcsackett> fixing the error seems like the easier to take on in afternoon solution.
<jcsackett> vs nuking and removing the need for it.
<lifeless> I think we care, but its trivial to fix: catch and return auth required
<lifeless> s/return/reraise
<jcsackett> lifeless: right on.
<lifeless> ah
<sinzui> lifeless: hell no. this is vesitigal from the merges of other branches
<lifeless> sinzui: context?
<sinzui> lifeless: I am pretty confident that bug will not be fixed this week
<lifeless> jcsackett: also there are two bugs conflated there
<jcsackett> sinzui: you're referring to the merge person stuff, right?
<lifeless> sinzui: its marked fix released is all; a user reported a new incident; I haven't duped because I was unsure o fthe status
<lifeless> jcsackett: the OOPS at the bottom of the bug you are looking at is for a different occurence of the same exception class
<jcsackett> lifeless: ah, thanks.
<lifeless> jcsackett: so I suggest you file a new bug for the OOPS (1188C1227)
<jcsackett> will do.
<sinzui> lifeless: allenap ran out of time (and scope) when working on that bug. we are traking the timeout in another bug because he claimed that one was fixed. His branch provided infrastructure to fix the issue
<lifeless> sinzui: yep. the other bug is also closed ;)
<jcsackett> sinzui: if you still wanted to chat at 4, the conflict i thought i had has disappeared, so i'm available at your convenience.
<sinzui> lifeless: bug 736421 i still open
<_mup_> Bug #736421: Person:+delete timeouts : Connect the UI the merge jobs <merge-deactivate> <Launchpad itself:In Progress by sinzui> < https://launchpad.net/bugs/736421 >
<sinzui> lifeless: I can finish that bug when I can return to bug 740559
<lifeless> sinzui: ahh, its not tagged timeout
<_mup_> Bug #740559: Update PersonNotification to support teams <merge-deactivate> <Launchpad itself:In Progress by sinzui> < https://launchpad.net/bugs/740559 >
 * lifeless tags it
<deryck> Later on, everyone.
<lifeless> -> allergy shot, BBIAB
<bac> sinzui: you see any reason to keep icon-sprites.positioning in the tree if we have sufficient makefile support to generate it on demand?
<sinzui> bac: I think removing that file will make our house of cards topple
<bac> i don't think so sinzui
<bac> i've removed it and updated the makefile rules and all seems well
<sinzui> icon-sprites.positionin + sprites.css.in = sprites.css
<bac> right, and as long as create_css dependson i-s.pos then it'll rebuild
<sinzui> bac: I stand down. you are correct
<bac> sinzui: looky at http://paste.ubuntu.com/587535/
<bac> sorry for the distracting boxes
<sinzui> that looks correct
<bac> great, i'll piggyback it on my next branch
<wallyworld> thumper: morning. standup?
<thumper> wallyworld: sure
<wallyworld> thumper: i can hear you
<wallyworld> thumper: my lips are red here
<wallyworld> thumper: ffs. try skype?
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #593: FIXED in 5 hr 7 min: https://lpci.wedontsleep.org/job/devel/593/
<jcsackett> apparently i have been disconnected for some time. hurrah.
<sinzui> jcsackett: mumble?
<jcsackett> sinzui: sure.
<wgrant> Morning.
<huwshimi> wgrant: Morning
<sinzui> wallyworld: thumper, mumble?
<thumper> sinzui: I can't hear you :(
<huwshimi> thumper: we can hear you
<thumper> damn, same thing happened with wallyworld and me earlier
 * wgrant fixes the conflict.
<sinzui> wgrant: I neglected to mention that the mhonarc rt was closed. We are running the latest version now
<wgrant> sinzui: Great!
<wallyworld> sinzui: sorry i missed your ping. i had already had a call with tim after which i was dropping the kid to school and then having breakfast
#launchpad-dev 2011-03-31
<LPCIBot> Project windmill build #122: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill/122/
<lifeless> wgrant: you've been nominated to coordinate killing poppy, if thats ok with you and sinzui
<wgrant> lifeless: Killing poppy? You mean moving to poppy-sftp before the rollout?
<wgrant> Or do you mean renaming poppy-sftp to something sensible, which I would also gladly do?
<lifeless> wgrant: moving to the new service before the db deploy
<lifeless> partial scheduled downtime; I've mailed mrevell for a slot on monday
<wgrant> lifeless: Yeah, saw that. I'll chat with mrevell tonight.
<wgrant> Thanks.
<lifeless> wgrant: cool, and *thank you*
<lifeless> wgrant: I don't understand why https://bugs.launchpad.net/launchpad/+bug/745799 shows librarian access
<_mup_> Bug #745799: DistroSeries:+queue Timeout accepting packages <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/745799 >
<wgrant> lifeless: It grabs the changes file to send emails.
<wgrant> It's on a POST, right?
<lifeless> yes
 * wgrant looks at the OOPS.
<lifeless> we should probably store that stuff in -db [massively more resourced] and clear it on completion
<wgrant> Possibly, although DDs will change where this is required.
<wgrant> Hmm.
 * wgrant stabs SPFP.
<lifeless> StevenK: Revision 12696
<wgrant> I'm not even sure why it's using SPFP... except maybe for conflict checks.
<wgrant> We really need to merge the copier and uploader and queue conflict check mechanisms.
<wgrant> Erm.
<wgrant> Why are the PackageDiff queries so slow?
<wgrant> Could you EA them?
<lifeless> sure
<lifeless> just qaing stuff first
<wgrant> Thanks.
<lifeless> so we cna fix person:+branches
<wgrant> Great!
<lifeless> wgrant: thoughts on https://bugs.launchpad.net/launchpad/+bug/740640 ?
<lifeless> qa wise
<wgrant> lifeless: Deploy it. The non-AJAX stuff looks safe, the AJAX stuff is bad but needs user interaction behind a flag.
<wgrant> (and it's already broken on prod)
<lifeless> wgrant: you've looked at subscriptions - -12704
<lifeless> bah
<lifeless> 12703
<wgrant> I'm even in the relevant team on qas now.
<wgrant> CHecking.
<wgrant> huwshimi: Ah! The privacy overlay!
<lifeless> anyone know the time dimension in staging restore logs ?
<lifeless> 725,778
<lifeless> e.g. is that milliseconds, seconds, minutes?
<wgrant> I presume ms... aren't there timestamps around it?
<huwshimi> wgrant: Indeed
<wgrant> huwshimi: Not quite pretty yet, but looks like a good start.
<lifeless> looks like redoing trusted.sql was /way/ more time than the heat migration
<wgrant> Hm, that's really unlikely.
<lifeless> wgrant: and yet...
<wgrant> So I think you're reading it wrong :)
<wgrant> lifeless: We are QAd.
<wgrant> Hm.
<wgrant> Except that we are bad.
<wgrant> r12697 still has the sprites breakage, right?
<wgrant> huwshimi: ^^
<lifeless> anyhow
<lifeless> 2011-03-31 00:44:24 INFO    2208-59-0 applied just now in 495.6 seconds
<lifeless> so, fine.
<wgrant> Yeah.
<wgrant> Not good, but fine.
<huwshimi> wgrant: That's right
<wgrant> huwshimi: I'll tag it.
<wgrant> Let's deploy 12696.
<wgrant> Unless the fix is coming RSN.
<wgrant> Hmm, will be on qas in three hours.
<wgrant> TYhere's only one interesting performance fix between 12696 and the fix, so I think we might as well deploy now.
<lifeless> +1
<wgrant> Requesting.
<lifeless> want to queue it? I'll get onto those explains
<lifeless> thanks!
<wgrant> Thanks. I don't see why they should be so slow.
 * lifeless hopes he got the tone right in his last email
<lifeless> does that include ajax times?
<lifeless> wgrant: you're talking about the 5x574ms queries ?
<wgrant> No.
<wgrant> lifeless: Yes, those queries.
<wgrant> AJAX log is 12699
<lifeless> wgrant: its one slow one
<lifeless> query 569
<lifeless> the data isn't on qas yet, will see about prod
<wgrant> Thanks.
<lifeless> oopses are looking good - product:+code-index is only a soft timeout now
<wgrant> Excellent.
<lifeless> and > 300 are +branches/+registeredbranhces
<wgrant> Your fix unbreaks Person:+registeredbranches too, right?
<lifeless> it should
<wgrant> Heh
<lifeless> I didn't explicitly test
<lifeless> but to have a different cause would be excitingly coincidental
 * wgrant growls at qa-tagger.
<wgrant> Don't add more items for me to the db-stable report 30s after I look at it.
<huwshimi> wgrant: Sorry I had a bit of a brain failure when I tagged that as qa-ok, I was forgetting about the sprites thing.
<wgrant> huwshimi: It happens. I always check the bugs manually before deploying.
<huwshimi> wgrant: Yeah thanks.
<lifeless> wgrant: you're doing 721591 ?
<wgrant> lifeless: Yes, just waiting for mawson to catch up.
<wgrant> Publishing is slow.
<lifeless> no worries
<lifeless> just seeing if the bugmessage.visibilty move regressed bugtask performance
<lifeless> OOPS-1916QS19 suggests it may have
<StevenK> wgrant: Ah ha, that explains why mawson's code was up-to-date when I updated it. Let me know when you're done and I'll play with it.
<wgrant> lifeless: It could have, but it should be fixable.
<lifeless> ah, hitting reload enough worked
<lifeless> so if it has regressed its tolerable
<lifeless> wow
<lifeless> 8 ajax requests for bugtask:+index
<lifeless> huwshimi: another tweak - show the count in the collapsed log
<lifeless> Ajax Requests (8 in 12s)
<lifeless> (just enumerate & sum)
<wgrant> StevenK: All yours.
<huwshimi> lifeless: Sure.
<lifeless> gnar, new bugtask heat indices don't want to be used
<StevenK> wgrant: Thanks, I'll be evil after lunch.
<poolie> hi all
<wgrant> StevenK: What DSD malice are you up to?
<StevenK> wgrant: The job runner.
<wgrant> Aha.
<poolie> lifeless, i like your downtime page
<lifeless> poolie: thanks!
<poolie> re bug 746486, couldn't you just remember a mapping from transaction-id to url?
<poolie> huwshimi, ^
<huwshimi> poolie: But how do we create that mapping?
<wgrant> I found a YUI bug report asking for this sort of thing.
<wgrant> There's no way to do it at the moment :/
<wgrant> Unless we patch Y.io, I guess.
<poolie> wow
<wgrant> Yes :/
<wgrant> It's a bit crap.
<poolie> it's not possible to dig the transaction out of the yui internals?
<poolie> i believe you, i'm just surprised
<poolie> lifeless, what did you mean by "blueprints linked bugs" has "inline editing of lists of bugs"
<poolie> when i look at a blueprint page with linked bugs it does not seem to have any inline edit controls
<wgrant> thumper: You've hacked lazr.restful a bit, haven't you?
<thumper> kinda
<wgrant> I need to change the JSON encoder that it uses. Do you have hints for using a non-released version in a dev LP instance, or should I just create a test egg?
<thumper> well... I don't personally have a preference
<thumper> to test you can easily just create a test egg
<thumper> and not add it to the branch
<wgrant> Sure.
<thumper> if I have a resultset designed to bring branch Branch objects
<thumper> how can I change the resultset to just give me the Branch.id column?
<wgrant> thumper: Look at .values
<wgrant> resultset.values(Branch.id)
<thumper> wgrant: cool, ta
<wgrant> wallyworld: You can't do the escaping inside update_field?
<thumper> wgrant: I mentioned that in one of my review replies
<wgrant> It was my initial suggestion, and I still think it's the right place to do it.
<wgrant> The latest version is less bad, but it's still the wrong layer.
<wallyworld> wgrant: sorry, i misunderstood. i'll rework it
<wgrant> wallyworld: Thanks. While your previous versions will work, I want it to be hard to misuse, or people are going to misuse it.
<wallyworld> wgrant: are we sure update_field is used universally and will capture all cases?
<wgrant> wallyworld: No.
<wgrant> But it should be.
<thumper> I know it isn't used universally yet
<thumper> it is still a recent (ish) change
<wgrant> thumper added the event recently, though...
<wgrant> So update_field may be the only consumer of that event.
<thumper> I think it is
<thumper> we may however need to tweak it
<wgrant> I want these abstractions to be neat, so people don't have to care about escaping.
<wgrant> They can just do the simple thing, and it will do the right thing for them.
<wallyworld> wgrant: agreed :-)
<thumper> as I think there are places where it expects a plain string
<thumper> and other places where it expects real HTML
<thumper> we may want two methods
<wallyworld> thumper: that's my worry too
<wgrant> thumper: Hm? Are there are more than two cases?
<thumper> two methods is fine
<thumper> wgrant: yes
<wgrant> 1) Given HTML. Needs not to be escaped.
<thumper> titles
<wgrant> 2) Given non-HTML. Always safely escapable, sometimes needs to be escaped.
<thumper> assignees which is a link to a person
<thumper> wgrant: the question becomes how to determine the HTMLness of the param
<thumper> I suppose we could have the new_value_html be a Node
<thumper> and check that
<wgrant> thumper: Exactly.
<thumper> then there is still one method
<thumper> wallyworld: make it so
<wgrant> My suggestion is to parse all HTML values into Nodes before giving them
<wgrant> to the callbacks. This lets update_field distinguish between safe and
<wgrant> unsafe values.
<thumper> that makes a certain amount of sense
<wgrant> This is similar to what I am doing to our templates at the moment.
 * wallyworld switches branches again
<wgrant> Unity, you were doing so well. You had not crashed for three days.
<StevenK> Hah
<huwshimi> A simple review for someone: https://code.launchpad.net/~huwshimi/launchpad/browse-links-746104/+merge/55667
<huwshimi> thumper? wgrant? ^
 * wgrant looks.
<wgrant> Difficult.
<wgrant> StevenK: Want a very difficult mentor review?
<StevenK> wgrant: No? :-)
<thumper> done
<wgrant> Thanks thumper.
<StevenK> 2011-03-31 03:19:20 INFO    Ran 48 DistroSeriesDifferenceJob jobs.
<StevenK> 2011-03-31 03:19:20 DEBUG   Removing lock file: /var/lock/launchpad-rundistroseriesdifferencejob.lock
<StevenK> \o/
<wgrant> But did they work?
<lifeless> do we have a bug for the loggerhead test fail ?
<wgrant> There is a loggerhead bug. Not sure about a launchpad one.
<lifeless> ah 745738
<StevenK> wgrant: That's the next question.
<StevenK> But first, tea.
 * StevenK tries to work out how to disable the annoying update-motd stuff
<LPCIBot> Project windmill build #123: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill/123/
<lifeless> ok, buildbot should succeed *next run*
 * StevenK finds another DSD bug
<StevenK> This is just a wart, nothing bad
<StevenK> wgrant: I'm having trouble finding the code that lists the differences for DSDs, can I have a quick hand?
<StevenK> wgrant: I'd like to hide the child difference line if dsd.base_version == dsd.source_version
<wgrant> StevenK: You want to hide the child diff, you mean?
<StevenK>         * 3.6.2.0-3 to Maverick version: 3.6.2.0-3
<StevenK>         * 3.6.2.0-3 to Sid version: 3.6.3.0-2
<StevenK> I'd like to not display the first line, since they're the same and the diff will be empty
<wgrant> StevenK: Grep around in lib/lp/registry/templates, I guess.
<wgrant> I think Unity just survived me pressing Ctrl+Alt!
<wgrant> Yes, I can press Ctrl+Alt reliably now.
<wgrant> This is a good start.
<wgrant> Ah no, broken again.
<StevenK> wgrant: I can't see anything obvious in lib/lp/registry/templates/distroseries-localdifferences.pt
<wgrant> StevenK: distroseriesdifference-listing-extra.pt
<StevenK> wgrant: Ah ha! Thanks.
<wgrant> thumper: Still around?
<wgrant> thumper: Do you recall why we wanted hybrid JSON/XHTML representations in the client cache? We can restore them now, if we want.
<poolie> huwshimi, while you're doing css-easy bugs, it would be nice if the textareas for entering bug and mp comments used the same monospace font as the comments are finally shown in
<poolie> i think there's a bug for this
<huwshimi> poolie: I think I've seen that bug
<poolie> just if you're running low :)
<wgrant> In https://code.launchpad.net/~wgrant/lazr.restful/bug-684430 I've changed lazr.restful to give XML-escape-safe JSON in most cases. But this will somewhat bloat XHTML representations that are delivered inside application/json... do we care?
<wgrant> (<, >, & get translated to \uXXXX)
<huwshimi> poolie: Are you thinking of this bug: 713366?
<wgrant> If we care, then I can shuffle things so it only affects the TALES function, but that seems overcomplicated.
<poolie> wgrant i doubt that would ever impact performance
<poolie> compared to all the round trips
<wgrant> poolie: Right.
<poolie> is there any chance any client libraries will fail to understand it?
<wgrant> No.
<wgrant> Even cjson understands this, and cjson is pretty bad at complying with the spec :)
<poolie> i can't think of any other consequences then; can you?
<wgrant> The only negative consequences are ugliness and slight bloat.
<wgrant> It's still perfectly correct.
<poolie> mm
<poolie> it's not really optimized for human reading now
<wgrant> Indeed.
<poolie> so a weak approval from me
<wgrant> (ugliness also in terms of escaping where we don't need to. but it's not like HTML escaping, because the strings are exactly equivalent once the JSON is parsed)
<poolie> or rather enthusiastic but non-authoritative
<wgrant> Heh
<wgrant> Thanks.
<wgrant> https://code.launchpad.net/~wgrant/lazr.restful/bug-684430/+merge/55678
<LPCIBot> Project db-devel build #507: FAILURE in 5 hr 21 min: https://lpci.wedontsleep.org/job/db-devel/507/
<wallyworld> wgrant: i think my xss fix may be ok now?
<wgrant> wallyworld: Let's see.
<wgrant> wallyworld: You use get('innerHTML'), but can't you just set the Node as the content directly? I don't really know YUI, but from my experimentation that seems like it should be possible.
<wgrant> Indeed, you already do that in multicheckbox.js
<wallyworld> wgrant: which line nr?
<wgrant> lp.ui.js, 81 in the diff.
<wgrant> In picker.js you've lost a containing <span>. That doesn't matter at all?
<wallyworld> wgrant: yes, i missed that one. as you say i changed in elsewhere. i'll fix
<wgrant> Thanks. This looks really good.
<wallyworld> wgrant: also, your lazr restful mp - you need to update news.txt and other stuff. i've already got a mp almost ready to land. i can merge in your stuff once approved and it can be done in one go if you like
<wgrant> wallyworld: I've already added a NEWS.txt entry. But yes, we should probably release both as 0.18.1.
<wgrant> What's your change?
<wallyworld> wgrant: https://code.edge.launchpad.net/~wallyworld/lazr.restful/propogate-notifications/+merge/54690
<wgrant> Ah, right.
<wgrant> that one scares me :)
<wallyworld> why? :-)
<wgrant> Very handy, but scary.
<wallyworld> it replicates what we already do
<wallyworld> for page loads
<wallyworld> but also now via xhr calls
<wgrant> Yeah, but it's really racy.
<wgrant> I am very wary of adding 10x the opportunities for races.
<wallyworld> i can see what you mean, but given how we use ajax atm, it should be ok
<wgrant> I would prefer it if the lazr.restful operations would use a separate notification system that didn't use the session, just returning them within the request.
<wgrant> s/system/provider/
<wgrant> eg. if you comment on a bug then quickly load another one, the notification comes up on the wrong page.
<wgrant> This is going to make that a whole lot worse.
<wgrant> However, I guess that fix is on the LP side, not in lazr.restful.
<wgrant> So this change is probably OK to land. Do you have the LP changes yet?
<wallyworld> wgrant: yes - currently being reviewed. i have a pipe. 1st one for the core infrastructure changes, next one for the actual end functionality
<wallyworld> wgrant: https://code.edge.launchpad.net/~wallyworld/launchpad/show-ajax-notifications/+merge/54342
<lifeless> stub: so I'm thinking a -1 and -2 patch *in case* qa is stalled and my updating indices patch doesn't get through in time for the deploy
<wallyworld> wgrant: re your earlier comment. the notifications are returned with the response to a given request
<lifeless> stub: that way we /can/ do it live if we need to
<stub> lifeless: sure.
<wgrant> wallyworld: I'm not sure how well that's going to interact with my markupsafe changes, but I guess I'll see when I get to that.
<lifeless> (-1 to drop the index that is superceded, -2 to add the two new ones). We may need to change the other indices, but I'd like to wait for performance data before doing that.
<lifeless> stub: ok, I'll prep
<wgrant> wallyworld: But the normal LP notification infrastructure uses the session, so they can leak cross-request if timing is bad.
<wgrant> wallyworld: We may later need to fix this for API requests, but it seems well layered so that should be easy.
<wallyworld> wgrant: atm, i'm not changing the backend implementation wrt content/escaping or mechanics of generating notifications. i'm just providing the transport so that xhr calls can get notifications pushed through
<wgrant> wallyworld: In the bit around line 32 of the blueprint-title-fix diff, any reason you're escaping and setContent instead of using default_value.set('text', default_value)?
<wgrant> Um, obviously without using the same variable name twice.
<wallyworld> wgrant: cause i didn't know about set('text', ...). my js/yui foo is improving but i'm not an expert clearly :-)
<wallyworld> wgrant: so does set('text'...) escape correctly etc?
<wgrant> wallyworld: Yeah, me neither... I know a bit of jQuery, but my only real YUI experience is this week.
<wgrant> wallyworld: It doesn't need to escape, since it is using the DOM.
<wgrant> But effectively, yes.
<wallyworld> wgrant: ok, so under the covers it will render correctly i guess i what i'm asking. i'll change it and retest
<wgrant> lifeless: Thanks.
<wgrant> wallyworld: I'm going to try to break it locally, since I don't quite understand what a couple of the bits do.
<wgrant> I need to learn more about our JS :/
<wallyworld> wgrant: break away :-)
<wgrant> wallyworld: Does the new object from the event get stored somewhere that I can get to in the console?
<thumper> wgrant: ask me again tomorrow when I'll have time to give you a full answer
<wallyworld> wgrant: you mean the new_value key of the dict?
<wgrant> wallyworld: I ideally want to see the whole object, I guess.
<wgrant> Hmm, something's still wrong.
<wallyworld> wgrant: you may need to set a breakpoint as it's just passed around and not stored anywhere
<wgrant> wallyworld: If you load a blueprint, change the title to have '<boo>' in it, refresh the page, then change another field (eg. Approver), the breadcrumb flashes green on the first change.
<wgrant> So we're still doing something slightly wrong.
<lifeless> ugh
<lifeless> why does launchpad forget I merge into db-devel every day now ?
<wgrant> lifeless: Oh, so it's not just me.
<wallyworld> wgrant: you mean the first change after refreshing the page?
<wgrant> wallyworld: The first change after refreshing the page makes the breadcrumb flash, yes.
<wgrant> wallyworld: So the cache value is wrong, somehow.
<lifeless> stub: can has tick? https://code.launchpad.net/~lifeless/launchpad/bug-618406/+merge/55680
<wgrant> Or the new value is just different, despite being the same.
<wallyworld> wgrant: could be the stupid <p> wrappers
<wallyworld> i'll have a look
<wgrant> wallyworld: Yeah, I'm guessing so.
<wgrant> Thanks.
<wgrant> wallyworld: It's possibly related to the fact that there is no HTML cache to compare against.
<wgrant> This can be fixed now, though, since we are escaping JSON properly.
<wallyworld> wgrant: well, the html is just presentation. it's the underlying values which are compared
<wgrant> wallyworld: Ah, OK.
<wallyworld> so the <p> comment i made is likely wrong
<wallyworld> wgrant: for example, an assignee attribute may be fred but the html property will be <a> href=...~fred...</a> etc
<stub> lifeless: tick
<wgrant> wallyworld: Right, but I wasn't sure if it compared the HTML values too.
<wallyworld> wgrant: no, it doesn't
<wgrant> stub: Ah, thanks, forgot to reply to that email, but you said what I wanted to :)
<stub> I hereby apologise for INotificationRequest. My only excuse is I didn't know better, many years ago when I wrote it :)
<lifeless> stub: hahaha
<lifeless> stub: do you mean about the session aspect, or some other aspect ?
<stub> re: session, it was written correctly first off with state being passed in the URL. But that was deemed too ugly so I had to break it.
<lifeless> thats right
<stub> I'm talking about notifications being HTML strings
<lifeless> ah!
<stub> Think Curtis fixed that
<lifeless> righto, that makes more sense (I distinctly remembered discussing the session aspect with you at the time)
<lifeless> I have an idea for fixing the urls.
<wgrant> stub: I'm hopefully getting to fixing that stuff tomorrow.
<lifeless> if we :
<lifeless>  - put notifications in their own place (perhaps in the session db still)
<stub> IIRC, the fundamental problem is it is impossible to differentiate one browser window from another reliably.
<lifeless>  - with an association to a user (as only logged in folk can trigger notifications)
<wgrant> stub: Yes :/
<wgrant> The easiest way to fix this is probably to just continue AJAXifying everything.
<lifeless> then we could use a very short short (primary key in the table) reference to select the notification to show
<poolie> lifeless, hi
<poolie> can you give me a url showing inline bug list editing?
<lifeless> poolie: hi, sorry, edistracted
<poolie> np
<lifeless> there are some in 727023
<poolie> i just had no idea that was actually implemented so i was curious to see it
<lifeless> the blueprint thing I'm going off an escalated bug
<lifeless> which I think was fixed
<lifeless> that you couldn't change the status of linked bugtasks on a blueprint
<lifeless> I may be misremembering
<stub> lifeless: I suspect the existing mechanism is good enough rather than polluting the URLs with a query parameter. Cure seems worse than the problem (which is rare to trigger (?), and harmless).
<lifeless> anyhow, I'm basically massively skeptical of any per-bug rules until we have /any/ bug stuff working really well
<poolie> separately, i just got a timeout again  OOPS-1916L440 on my code homepage
<lifeless> stub: its certainly low priority
<poolie> is that a regression of bug 745310
<_mup_> Bug #745310: Person:+branches timeout <qa-ok> <regression> <timeout> <Launchpad itself:Fix Released by lifeless> < https://launchpad.net/bugs/745310 >
<poolie> (if known, np)
<lifeless> stub: I can reproduce it really easily; I have  ascreen shot around somewhere with 8 or so notifications on one page
<lifeless> poolie: I'll look at the OOPS. my WAG is that the page was poor before the regression, and thus is just timing out naturally because ithasn't been optimised (just restored to its prior behaviour)
<stub> yer, but you have to try hard. People are less likely to be submitting 8 tabs simultaneously now that Launchpad performance sucks so much less.
<poolie> i wonder
<wgrant> stub: And we do lots of stuff by AJAX now, which can do notifications without the session evil.
<stub> wgrant: If you pull in the notifications asynchronously, you still have to somehow identify *this* window's set of notifications vs. *that* window's set of notifications.
<wallyworld> wgrant: the issue was that when a page first loads, the cache contains the escaped value, but then the update_cahced_object() function stuffs the unescaped new value into the cache. this is how it always has been so i'll fix it
<wgrant> Oh.
<wgrant> wallyworld: My lazr.restful branch fixes that.
<wgrant> wallyworld: The cache now contains the unescaped value.
<wgrant> How did I not think of that :/
<wgrant> Let's see if it works now.
<wallyworld> wgrant: ok, then i won't change it on my side
<wgrant> wallyworld: Yeah, new lazr.restful fixes it.
<wgrant> Great.
<wallyworld> \o/
<wgrant> So, I'm not quite done yet, but I think this infrastructure may just about be correct.
<wgrant> Then we can deploy it eeeeeverywhere.
<wallyworld> well it's certainly a big improvement on what it was.
<wallyworld> did you head spin around when you said "eeeeeeverywhere"?
<wallyworld> wgrant: can you let me know when your lazr-resful mp is approved so i can roll it into a single 18.1 release? thanks.
<wgrant> I need to work out how to land it.
<lifeless> lp-land
<wallyworld> i was just going to merge it into mine directly
<wgrant> Might as well land separately, or it's going to be confusing to people looking through the history.
<wgrant> lifeless: You haven't by any chance added a long-poll change notification service to LP while I wasn't watching, have you?
<wallyworld> ok
<lifeless> wgrant: sorry no
<lifeless> wgrant: I know where one is we can pick up and use
<lifeless> wgrant: or I have some ideas on a variant
<wgrant> Damn.
<lifeless> wgrant: I think this is fine
<lifeless> the race is lower on faster requests
<wgrant> Sure.
<lifeless> wgrant: I agree we should improve it, but perfect enemy of the good
<wgrant> Oh, lazr.restful isn't PQM.
<wgrant> Handy.
<wgrant> wallyworld: Mine's landed.
<wgrant> Now, let me finish up reviewing your client fixes.
<wallyworld> wgrant: thanks. will grab it. hopefully my mp will get approved tonight.
<wgrant> wallyworld: One thing: testtools is only a test dep, so it shouldn't be in requires.
<wallyworld> wgrant: ah ok. i was just going by what i though the review comments said i had to do. i actually agree with you that it needent be listed as a dep. i'll have another look.
<poolie> lifeless, sorry, i'm tired and i'm probably just being thick, but i can't see why a lock bit would have any performance impact
<wgrant> wallyworld: I didn't read the comments at all.
<wgrant> wallyworld: Do we need getHTMLOrDefault? I can't think when we'd need a fake HTML representation for something that doesn't have an HTML representation.
<wallyworld> wgrant: i put it there as a "failsafe" so that something sensible would still be displayed if there were no explicit html included
<wgrant> wallyworld: I think we probably want to crash in that case, no?
<wallyworld> wgrant: not sure. i can see both sides.
<wgrant> Because that function does some slightly questionable stuff, so it would be nice to demolish it if it's not really useful.
<rvba> Hi lifeless. A question about qa-ing changes. If you look at 737165 ... you can see I've qa-ed this (on dogfood, with the help of bigjools because qa-ing this required admin rights to set a feature flag) but it got flagged qa-needstesting again. Not sure what to do at this point.
<rvba> https://bugs.launchpad.net/launchpad/+bug/737165
<_mup_> Bug #737165: Add search option for higher versions of local packages <derivation> <qa-needstesting> <Launchpad itself:Fix Committed by rvb> < https://launchpad.net/bugs/737165 >
<rvba> I suppose I should just tag it again qa-ok ... but I prefer to check with you first ...
<wgrant> rvba: Yes, just tag it again.
<wallyworld> wgrant: i'll look at removing it. leave it with me
<wgrant> qa-tagger ignores old tags, since bugs can be reused.
<rvba> all right
<wgrant> wallyworld: Thanks. Apologies for being so nitpicky, but this stuff will hopefully be used everywhere eventually, so I want to get it *really* right now.
<wallyworld> wgrant: np at all. i agree with the need to get it as right as possible.
<lifeless> rvba: things get tagged every time that they show up on the landing branch
<lifeless> rvba: tagging them ok orotherwise before the taggers get to them just means you'll need to do it again
<lifeless> rvba: ah, as wgrant explained ;)
<rvba> lifeless: I get it now ;-). I'll make sure to check that every time I receive a mail from the buildbot.
<wallyworld> wgrant: tis done
<wgrant> Woo.
<wgrant> This change has made the code 15 lines shorter, which can only be a good thing.
<wgrant> wallyworld: As I said on the MP, _defaultFormatter is a bit odd and I don't understand its purpose.
<wgrant> It's still using innerHTML, so it smells bad.
<wgrant> It might be right, but I don't know.
<wgrant> 161	+        // If there is no html representation, the raw cache value should be
<wgrant> 162	+        // properly wrapped inside a node.
<wgrant> That test comment's wrong now, right?
<wgrant> The test itself seems to be correct.
<wallyworld> wgrant: bugger sorry. comment is borked
<wallyworld> wgrant: _defaultFormatter is required to return text, and now getHTML wraps stuff inside a node, so get get('innerHTML') is necessary
<lifeless> poolie: all software sucks.
<lifeless> poolie: some software sucks more
<wgrant> wallyworld: Hm, so it will return HTML or unescaped text, right?
<wallyworld> wgrant: although the null case needs to be checked now
<wgrant> Anything that calls it is probably broken, then.
<lifeless> poolie: for performance in launchpad the question isn't 'can it have a performance impact', it is 'is the performace impact low enough'
<lifeless> poolie: we have, you might say, a target rich environment
<lifeless> poolie: there are practical limits beyond which theoretical models just fail - like in python profiling, actual measurement is the key things
<lifeless> poolie: I suggest that if you want a better metric than my hunch as expressed, that it needs coding up and scale testing
<wallyworld> wgrant: yes, I think it returns html (when asked to do so) but that html is expected to be safe
<wgrant> wallyworld: Hm, so the caller sets use_html?
<wgrant> As I said, this might well be fine, it just seems odd to have a function return both trusted and untrusted strings.
<wgrant> Depending on the situation.
<wallyworld> wgrant: yes, see text-area-editor.pt
<lifeless> principle of least suprise
<wallyworld> that's the only place that sets it to true
<wgrant> Ah, I see.
<wgrant> I might file a bug for that.
<wgrant> Since it's mildly insane.
<wgrant> Anyway, approved.
<wgrant> lifeless: Still around to mentor?
<wallyworld> wgrant: thanks
<lifeless> in a few minutes
<LPCIBot> Project windmill build #124: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill/124/
<wgrant> lifeless: Thanks.
<lifeless> grah
<lifeless> another compiler layering bug
<adeuring> good morning
<wgrant> More conflicts :(
<lifeless> can has review https://code.launchpad.net/~lifeless/launchpad/bug-618406/+merge/55703 ?
<lifeless> stub: one more if you have time https://code.launchpad.net/~lifeless/launchpad/bug-618406/+merge/55703
<LPCIBot> Project devel build #595: FAILURE in 5 hr 5 min: https://lpci.wedontsleep.org/job/devel/595/
<StevenK> allenap: https://code.launchpad.net/~stevenk/launchpad/dsd-hide-child-diff/+merge/55685
<allenap> lifeless: I'll take it.
* allenap changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: allenap | https://code.launchpad.net/launchpad-project/+activereviews
<stub> lifeless: done
<lifeless> stub: thanks
<lifeless> allenap: Thanks, but stub got there ;)
<allenap> lifeless: No worries, I'm happy :)
<stub> db-devel qa is all green (sorry about those patches! I need some better reminder system for qa todo)
<stub> Hmm... so my ec2 instance is terminated, but no results :-(
<stub> oh... there they are
<stub> Do we have any scripts that still need LibrarianLogger? Its a pain and I think the noise problems it solved no longer exist.
<lifeless> dunno; fine to nuke IMO
<lifeless> stub: I check both deployment reports as part of 'starting work' and 'wrapping up
<lifeless> '
<lifeless> daily
<lifeless> night all
<deryck> Morning, all.
<bigjools> allenap: did you say you were fixing the db-devel conflict ?
 * jml has spent the morning defacing sheets of creativity-sized paper.
 * henninge lunches
<bigjools> jml: is that a euphemism?
<jml> bigjools: heh. no.
<bigjools> jml: guess where I do most of my thinking :)
<jml> bigjools: :D
<danilos> allenap, hi, care to give me a quick review of: https://code.launchpad.net/~danilo/launchpad/revert-xss-workaround/+merge/55738 (all removal of something we landed earlier :)
<wgrant> danilos: Yay, thanks.
<danilos> wgrant, heh, you are welcome :)
<wgrant> danilos: If you run into any more escaping issues like that, *please* whine at me until I fix them before working around them like that :)
<danilos> wgrant, heh, if all it takes is whining at someone, I'd be happy to :)
<danilos> wgrant, fwiw, I do agree with your position, and I brought it up multiple times (in pre-imp, MP, ... :), but we are also working against a deadline
<danilos> wgrant, now that I know the proper address to whine at, all the better :)
<jml> oh yeah
<jml> colo
<wgrant> danilos: Maintenance squads are handy for this sort of thing.
<jml> ...
<wgrant> Oh dammit.
<wgrant> The new superlong buildd connection timeout sucks.
<jml> whycome
<wgrant> One could previously kill a build from a hung builder by disabling the builder.
<wgrant> The connection would time out after a reasonable interval.
<wgrant> And the next scan would see that the builder was disabled.
<wgrant> But now the connection timeout is something like a day.
<wgrant> So that doesn't work so well.
<bigjools> s/builder/virtual builder/ :-)
<wgrant> Yeah.
<wgrant> But I like to pretend that armel doesn't exist, and the others don't hang, so...
<bac> hi allenap, have time for a review?
<jml> ffs.
<jml> I should release testtools already.
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #508: FIXED in 5 hr 27 min: https://lpci.wedontsleep.org/job/db-devel/508/
<jelmer> jml: you should.
<allenap> bac: Certainly :)
<allenap> danilos: But I'll do yours first.
<bac> allenap: great: https://code.edge.launchpad.net/~bac/launchpad/bug-745660/+merge/55636  -- most of the 700+ lines are a file deletion
<allenap> bigjools: I think I fixed the db-devel conflict... I have my fingers crossed.
<allenap> bigjools: No I didn't.
<bigjools> allenap: yeah :(
<bigjools> nagbot is still nagging
<allenap> PQM swallowed my branch and defecated it into /dev/null.
<wgrant> bigjools: I didn't think of this consequence when I sped up PQM :/
<allenap> I'll have another go.
<wgrant> (buildbot-poller only tries to merge again if there's not already a merge in the queue, so the OMG-so-slow PQM slowed down the spam too)
<rvba> allenap: a small MP to review if you have the time https://code.launchpad.net/~rvb/launchpad/dds-fix-localpackagediffs-745776/+merge/55704
<allenap> rvba: Should do.
<rvba> allenap: great
<deryck> henninge: ping for standup
<adeuring> allenap: could you please have a look at this mp: https://code.launchpad.net/~adeuring/launchpad/js-translation-2/+merge/55575 ?
<allenap> adeuring: I'll probably get to it, sure.
<adeuring> allenap: thanks!
<danilos> allenap, btw, thanks for the review :)
* jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: allenap, jcsackett | https://code.launchpad.net/launchpad-project/+activereviews
<allenap> danilos: No worries. It was an easy one!
<gary_poster> hey allenap, jcsackett, would love a review of https://code.launchpad.net/~gary/launchpad/bug741684/+merge/55656 and follow-on branch https://code.launchpad.net/~gary/launchpad/bug741684-2/+merge/55657 when you have a moment.
<jcsackett> gary_poster: looking at the first of those now.
<gary_poster> awesome thank you jcsackett
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #596: FIXED in 5 hr 4 min: https://lpci.wedontsleep.org/job/devel/596/
<jcsackett> gary_poster: i don't think you have any reason to worry about problems with caching initial_message; to my knowledge, one of the reasons we access initial_message is b/c it *doesn't* change, so we can use it as a reference.
<gary_poster> cool, thanks
<jcsackett> so it's a pretty excellent candidate for cache.
<gary_poster> awesome
<jcsackett> gary_poster: what's the reason for LeftJoining on BugActivity from BugNotification? can you have a situation where a notification has no corresponding activity?
<gary_poster> yes, jcsackett.
<jcsackett> huh.
 * jcsackett makes note that he misunderstands what bugactivity must be.
<jml> I want to run a process from a Python script. I want to collect the stdout and the stderr. I want to be able to get the "pure" stdout, unmixed with stderr, but I would also like to be able to have a combined stdout/stderr that looks a lot like hat you would get if stdout & stderr went to the same pipe
<jml> s/hat/what/
<gary_poster> jml, ec2 test does something like that IIRC.  It's not easy nor perfect
<james_w> jml, you're not going to like it, but there is code at http://bazaar.launchpad.net/~james-w/ubuild/trunk/view/head:/ubuild/background_runner.py that works
<jml> gary_poster: actually, it doesn't. I'm fixing ec2 test to do that :)
<jml> james_w: your powers of prediction are amazing :)
<jml> james_w: I think I could justifiably switch to using Twisted rather than do that.
<jml> hmm.
<jml> maybe I can do something where I set up a couple of forking streams and replace p.stdout & p.stderr with them
<jml> something roughly equivalent to p.stdout = MultiStream(pure_stdout, mixed_output); p.stderr = mixed_output
<jml> ugh, no, wrong direction
<deryck> henninge: did you want to chat about js at all?  Or you're good?
<henninge> deryck: at the moment I am good.
<henninge> thanks for asking
<deryck> henninge: ok.  I'm stepping out of the office for a lunch outside the house.  In about an hour.
<deryck> adeuring: ^^  just fyi
<adeuring> ok
<henninge> ok
<gary_poster> jcsackett, on calls, so sorry for "succinct" message
<gary_poster> bug activity is not always generated
<gary_poster> IIRC, comments don't always have one, for instance
<gary_poster> I don't remember the details, but I do remember that it is not always there
<jcsackett> gary_poster: dig. no worries for the succinctness. :-)
<gary_poster> :-)
<gary_poster> thanks
<jcsackett> i am surprised that creating a comment could not count as activity; seems to break my mental model of what should be happening. :-)
<jcsackett> but then, that happens all the time.
<gary_poster> :-)
<LPCIBot> Project windmill build #125: STILL FAILING in 1 hr 10 min: https://lpci.wedontsleep.org/job/windmill/125/
<bac> allenap: thanks for the review and the suggestions!
<allenap> bac: You're welcome.
<bac> i wasn't aware about cmp.  good to know.
<jcsackett> gary_poster: r=me on part 1. looking at the second bit in just a few.
<gary_poster> great, thanks again jcsackett
<jml> Ursinha: I'm looking at https://bugs.launchpad.net/launchpad/+bug/715832
<_mup_> Bug #715832: ec2 land adds duplicate tags to the merge proposal's commit message <build-infrastructure> <ec2land> <ec2test> <Launchpad itself:Triaged> < https://launchpad.net/bugs/715832 >
<jml> Ursinha: I think you originally changed autoland to edit the commit message
<jml> Ursinha: what was the motivation behind that?
<gary_poster> jcsackett, while it is still in your head, I'd love to get your ideas on reducing the churn in that for loop.  mumble would be fine--I could take notes for a third branch, or comments, or future work, or something.  It would be a shame to lose your idea though.
<jcsackett> gary_poster: what does the ProxyFactory decorator do? is that the one that throws a security proxy on returned data?
<gary_poster> jcsackett, exactly
<jcsackett> gary_poster: my notions on it are still churning--i'm not certain i *have* a solution, just i see that the DB call is happening at roughly a similar O() as before, so could still be an issue.
<gary_poster> jcsackett, gotcha.  If there are no teams without preferred emails, and only one notification at a time, you are right, the changes in the loop are small.
<gary_poster> I know we have multiple notifications at a time, though; and I actually ought to see how many teams we have with and without a preferred email--I bet that's a relatively simple SQL select I can make on staging.
<gary_poster> Sadly, I don't see a way to simplify this further and maintain the desired functionality.  That isn't to say that there isn't onem of course.  If there are more problems, it will be interesting to see what we can do.
<jcsackett> gary_poster: yeah, i think if we reach a point where further changes are needed, we're going to have an interesting time of it.
<jcsackett> that said, and as i said in the review, the actual DB stuff itself is cheaper than it was thanks to your changes, which is a huge help. and the situation in which your other changes doesn't help should be an edge case.
<gary_poster> yeah, hope so. Thanks.
<jcsackett> gary_poster: on part 2, you the goal of _get_recipients_for_team is to get emails for every Person (who is not a Team) that has membership in the team, whether directly or via another team, right?
<gary_poster> jcsackett, not precisely.  The logic is similar to that described in get_recipients:
<gary_poster> a team with a preferred email is its own recipient, with no transitive walk down.
<jcsackett> aaah, missed that bit.
<gary_poster> if a team does not have a recipient, then you need to do a transitive walk down to find what does
<gary_poster> (which can be a person or a team)
<jcsackett> right, now it all clicks together.
<gary_poster> cool
<jcsackett> i was about to suggest a single query via teamparticipation, but the filtering to then get rid of people you don't need would be just as expensive as the current iterations.
<gary_poster> yeah.  I also considered a recursive with statement, but it didn't seem to be a match: I need to walk down the things I don't return.
<gary_poster> (where with helps you walk down the things you return)
<gary_poster> I could have done it with "with" but it would have made more loops in the DB, even though there might be fewer in Python.
<gary_poster> At least the way I conceived of it
<jcsackett> i think you're right, as i'm thinking through this. and we would much rather have the loops be in python, given db speed was the problem. :-P
<gmb> allenap, jcsackett: Could one of you review the code portion of https://code.launchpad.net/~gmb/launchpad/team-subscription-opt-out/+merge/55764 please?
<gary_poster> :-) yeah.  Well, the speed is honestly the first problem.  The DB usage is a scalability issue, and a secondary problem.  But yeah, ideally we'd fix both. :-/
<allenap> gmb: Sure, I'll do it next.
<gmb> Cool, thanks.
<jcsackett> gary_poster: r=me on part 2. i have no problems with it being a function, rather than a method. :-)
<gary_poster> jcsackett, cool :-) .  Thanks!
<jcsackett> you're welcome.
<allenap> gmb, apparently you have largely reimplemented Launchpad: Diff against target: 345455 lines (+151809/-75208) 3004 files modified
<gmb> WAT
<allenap> I'm not going to be able to review that today.
<gmb> gary_poster: So, that bzr problem you mentioned...
<gmb> ^^
<gmb> Oo-kay.
<gmb> allenap: Let me take a look at that, then.
<gmb> Oh.
<gmb> Hahaha.
<gmb> Interesting.
<gmb> allenap:  Ready for review for merging into lp:~gmb/launchpad/db-devel
<gmb> E_RONG_TARGET
<allenap> Hehe :)
<gmb> allenap: https://code.launchpad.net/~gmb/launchpad/team-subscription-opt-out/+merge/55779 instead.
<rvba> allenap: jcsackett I've 2 small bugfix MP for you to review:
<rvba> https://code.launchpad.net/~rvb/launchpad/dds-req-packagediff-fix-flashing/+merge/55775
<rvba> https://code.launchpad.net/~rvb/launchpad/dds-fix-localpackagediffs-745776/+merge/55704
<allenap> gmb: Okeydoke.
<gmb> Ta
<allenap> rvba: I'll see if I can squeeze one of those in after Graham's.
<rvba> great
<jcsackett> rvba: i'm looking at the first of the two now.
<rvba> jcsackett: thx
<jcsackett> rvba: why the name change from "retry_handler" to "recompute"? was it just to be clearer, or was there anything else?
<rvba> jcsackett: yes, just to be more clear
<jcsackett> cool.
<jcsackett> rvba: to summarize, the big change here is that you're now attaching events to that new span, which is replaced by a placeholder without events on error. is that correct?
<rvba> jcsackett: right
<jcsackett> rvba: last question. :-) ui wise, i see that you are removing the red flash animation. this isn't something we want on error anymore? or is it provided by somethine else now?
<rvba> jcsackett: I removed the flash anim because, although I like js animation as much as the next guy, you'll be instant feedback right where you've just clicked
<rvba> hence no need to flash
<jcsackett> rvba: okay. r=me.
<rvba> s/you'll be/you'll get/
<jcsackett> rvba: it might be worth it to run that flash change by a ui guy, but i see no problems with it.
<rvba> jcsackett: we plan to get feedback for the ui (and the rest) asap. All of this is related to the new derived distro series ... thing.
<jcsackett> rvba: sounds good to me. :-)
<jcsackett> rvba: as i said, r=me. :-)
<rvba> jcsackett: great, thx.
<jcsackett> if allenap hasn't hit your other branch in a few, i'll take a look at it as well.
<jml> https://code.launchpad.net/~jml/launchpad/various-ec2-fixes/+merge/55763 genuinely needs a review now
<jml> sorry for the noise
<gmb> allenap: I'm doing quite a bit of work in a dependant branch, so I'll add the tests you suggested there.
<allenap> gmb: Cool :)
<jcsackett> rvba: i am looking at your second branch now.
<jcsackett> jml: if allenap hasn't gotten to your branch in a bit, you're up next for me.
<allenap> It's a race!
<allenap> jcsackett: Actually, it's all yours; I have to go and collect the kids now.
<jcsackett> allenap: righto. :-)
<rvba> jcsackett: looks like allenap took my second branch first :-)
<jml> jcsackett: thanks.
<jml> man.
<jml> web browsers suck.
<jml> can we stop using them please?
 * jcsackett laughs.
<jml> turns out the svgs on http://lpqateam.canonical.com/doc/vision.html look fine in firefox but tiny in chrome
<jcsackett> rvba: r=me, with a small change listed in the MP comment.
<jcsackett> jml: i'll be looking at yours in just a few.
<rvba> jcsackett: thanks.
<jml> jcsackett: thanks.
<rvba> allenap: thanks also for your review
<jcsackett> did we both hit that branch?
 * jcsackett laughs.
<jelmer> jcsackett, allenap: I've got a MP for a project which doesn't appear to be listed on the launchpad-project page
* allenap changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jcsackett | https://code.launchpad.net/launchpad-project/+activereviews
<jelmer> allenap: ah :)
<jcsackett> jelmer: which project?
<jelmer> jcsackett: lpreview-body
<jcsackett> huh. there is definitely a strong argument for that being an lp project. :-P
<jcsackett> MP link? i'll look when i'm done with jml's MP.
<jelmer> jcsackett: yeah, indeed :)
<jelmer> The MP is @ https://code.launchpad.net/~jelmer/lpreview-body/lazy-hooks/+merge/50471
<jelmer> thanks :)
<jml> mrevell: could you have a look at https://code.launchpad.net/~jml/launchpad/vision-doc-split/+merge/55789 ?
 * mrevell looks
<mrevell> jml No problem
<mrevell> jml, Is there a package with sphinx-build or do I need to go hunt it down from somewhere?
<jml> mrevell: python-sphinx
<mrevell> ah, thanks
<mrevell> jml, I'll complete my review of that branch in the morning.
<jcsackett> jml: looks like a good grab bag of fixes. r=me.
<jml> jcsackett: thanks.
<jcsackett> jml: are there bugs filed about the coverage of tests on ec2? your MP mentioned it is not well tested, in general.
<jml> jcsackett: not really.
<jml> jcsackett: there's such a great deal of stuff that's not tested I'm not sure that it's possible to file a closable bug.
<jcsackett> jml: yeah, i could see that being an issue.
<jcsackett> it's sort of funny that our testing infrastructure is sufficiently complicated it's hard to test itself.
<jcsackett> jelmer: r=me. i love short branches. :-)
<jelmer> jcsackett: Thanks :)
<jml> jcsackett: yeah. it's ec2 and Launchpad's own deployment/installation complexity that really does it.
<jml> jcsackett: ec2 test was originally a fairly organic script that (AIUI) was written as we were learning about how best to do this. Because it wasn't written with testing in mind, there are a lot of sticky bits that need to be unstuck.
<jcsackett> jml: makes sense. refactoring for testability can be a major pain, too.
<jml> jcsackett: yes. how long has it been since I last advocated TDD in this forum?
<jcsackett> your talk was last week, wasn't it?
<jcsackett> does that count? :-)
<jkakar> ++TDD!
<jkakar> :)
<gary_poster> As the original source of the ec2 script and its lack of tests, and someone who has written *lots* of tests, I personally think that exploratory programming has its place.  And personally still feel a tad defensive, obviously ;-) .
<gary_poster> When the exploration takes much more time than expected, and what you have works, and you are supposed to go off and do something else now thank you very much, you're left with something like what ec2 was before jml started his magic on it.
<jml> gary_poster: I think that doing TDD with ec2test would have been prohibitively difficult
<gary_poster> I certainly had no freaking idea what I was doing when I was starting :-)
<jml> :)
<jml> and once something is working, it's much easier for someone else to come along & refactor, because "use it and see if it works" becomes a valid & effective test
<jkakar> gary_poster: I agree.  Sometimes you just need to type really fast and see what comes out the other end.  There's nothing wrong with that.
<gary_poster> jkakar, cool.  congrats on new position!  Looks really interesting
<jkakar> gary_poster: And you shouldn't feel defensive--you solved an important problem.
<gary_poster> :-) thanks
<jkakar> gary_poster: Thanks!  It's turning out to be a lot of fun. :)
<jml> gary_poster: and I honestly don't mean to attack. I couldn't fault your approach with ec2test, and ... what Jamu said :)
<gary_poster> I bet
<jkakar> That's actually an interesting thing...
<gary_poster> heh, thanks guys.  when I said I felt defensive, I was trying to admit that I was not entirely rational in feeling that way :-)
<jkakar> :)
<jml> g'night all.
<jcsackett> night.
<gary_poster> night
<deryck> allenap: thanks for the review!  Was away and nice to see it done when I returned. :-)
<deryck> Wasn't expecting it. :-)
<LPCIBot> Project windmill build #126: STILL FAILING in 1 hr 12 min: https://lpci.wedontsleep.org/job/windmill/126/
<jcsackett> is raising Unauthorized the best way to send back a 401?
<jcsackett> i mean, i assume so, but i don't want to get tripped up by something i'm not aware of, and status codes in launchpad have tripped me up before.
<jcsackett> sinzui:  did you still want to mumble about bug 676988?
<_mup_> Bug #676988: Relax-NG validity error : Element hardware failed to validate  <hwdb> <lp-bugs> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/676988 >
<sinzui> yes
<jcsackett> when would you like? now?
<jcsackett> sinzui: https://lp-oops.canonical.com/oops.py/?oopsid=1782C2286
<lifeless> jcsackett: wow, special oops
<james_w> is there a bug about editing blueprints changing the title in an odd way?
<james_w> <title> specifically
<lifeless> yes
<lifeless> erm possibly yes
<lifeless> I hink its pending deploy
<lifeless> does qastaging look better to you?
<wgrant> james_w: Yes, and the fix has landed.
<james_w> great, thanks
<wgrant> We can probably deploy it in a few minutes, too.
<wgrant> :(
<wgrant> IE breaks all my plans.
 * thumper gets coffee
<thumper> I'm natty now
<jcsackett> IE breaks everyone's plans.
<thumper> and unity is being a PITA
<wgrant> thumper: Oh?
<wgrant> It's pretty stable for me once I've been running for an hour or two.
<wgrant> But for the first hour just pressing Ctrl+Alt is enough to make it crash.
<thumper> Ctrl+Alt is fine here :)
<lifeless> wow, its conflict city atm
<lifeless> -> hospital, bbiab
<wgrant> *Another* one?
<lifeless> garbo
 * wgrant fixes.
<wgrant> Fairly non-trivial :/
<deryck> Later on, everyone.
 * thumper stabs unity in the face
 * thumper goes to a different shell
<wgrant> thumper: Is it being buggy, or you just don't like it?
<thumper> crashing on me all the time
<thumper> can't work like that
<thumper> it is different if it is my work
<thumper> but it isn'tyet
<wgrant> Heh
<wgrant> Which driver?
<thumper> intel
<wgrant> :(
<jcsackett> sinzui: got another few moments to mumble?
<thumper> hi guys
<thumper> I have to take caitlin to the dentist shorty
<thumper> so I'll miss the standup
<wgrant> james_w: Fixed now.
<james_w> thanks
<wallyworld> sinzui: are you guys having a standup now?
<huwshimi> wallyworld: Technically
<wgrant> I've seen no sinzui this morning, though.
<wallyworld> huwshimi: i hope my mumble works today. it's been flakey of late :-(
<huwshimi> wallyworld: It has been for a lot of people
<poolie> hi all
<wgrant> Morning poolie.
<LPCIBot> Project windmill build #127: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill/127/
<wgrant> wallyworld: Do you have a PyPI account? Might as well poke someone to give you or I upload access to lazr.restful.
<wallyworld> wgrant: no, i don't have one. i'm lp-landing now. i want to package it all up and get it into download-cache so i can get a downstream branch going
<wgrant> wallyworld: Great. You've fixed the date in NEWS.txt?
<wgrant> I think that's all that's left to do.
<wallyworld> wgrant: well, no :-) i guess i should
<wgrant> wallyworld: Also, lazr.restful isn't PQM-managed, so you just merge directly.
<wallyworld> ok
<wgrant> huwshimi: You have no outstanding Loggerhead CSS updates?
<huwshimi> wgrant: Nope. Should I?
<wgrant> huwshimi: Just checking that I wasn't going to conflict with you if I attacked it to remove the double line spacing.
<huwshimi> wgrant: Nah all good. Go for it
<wgrant> Thanks.
#launchpad-dev 2011-04-01
<wgrant> wallyworld: Hmm, you left the testtools addition in when you merged?
<wallyworld> wgrant: yes. bin/test failed to run properly without it
<wallyworld> wgrant: atm i'm waiting on launchpadlib to build so i can run the upload script
<wgrant> Great.
<bac> wgrant: can you connect to anything coming out of the data center?  it is dead to me.
<bac> spm: ^^
<wgrant> bac: WFM
<spm> wfm, what does mtr stay?
<wgrant> Terribly slow.
<wgrant> Like 600ms.
<wgrant> Hm. Lots of that is within Optus, though.
<wgrant> So it's fine.
<bac> lp.net, irc.c.com, imap -- all ead
<spm> seems about the same for me. you're on optus tho = so cause and effect
<bac> dead
<bac> wgrant: wfm?
<wgrant> bac: Works for me.
<spm> yeah. same for me. 330ms
<bac> ok.  so perhaps it is just the transatlantic pipe
<wgrant> Back down to 320ms now.
<spm> bac: we go via the USA, fwiw.
<wgrant> bac: Well, except that my route is via the US (L3)
<bac> even better, perhaps just my isp
<spm> sydney -> lax ->
<spm> wgrant: tho interestingly, I'm staying internode routers right thru to london. curious.
<spm> 9. gi0-0-1.bdr1.lon1.internode.on.net
<wgrant> I didn't know they had their own routers there.
<spm> neither did I. must be a new thing. might ping Mark and ask.
<wgrant> That's pretty nice.
<spm> I stick iwth internode for many reasons; knowing their router god, who's a damn nice guy; is but one of them :-)
<bac> i can only get as far as atlanta before traceroute shows it dying.  best to call it a day then.
<spm> maybe atlanta has been nuked?
<lifeless> wheee, I can see why https://bugs.launchpad.net/bugs/cve/2010-4080/+index was having trouble rendering
<wgrant> spm: I've wanted to switch to Internode for a long time, but DSL here is bad :/
<wgrant> lifeless: Uh, yeah.
<lifeless> 12 pages
<spm> wgrant: NBN will fix it
<wgrant> Indeed.
<spm> bac: looks like you were'nt alone. see a few folks have been impacted.
<wallyworld> gary_poster: are you able to give me permission to register a new lazr.restful release on pypi?
<wallyworld> wgrant: i'm not sure what to do with the natty source package side of things for the new release
<wgrant> wallyworld: Is lazr.restful packaged?
<wgrant> Huh, so it is.
<wgrant> But it seems to be prehistoric.
<wgrant> So ignore it.
<wallyworld> wgrant: ok. i had to upload the tarball by hand because i couldn't get the uplaod script to run properly. that's for another day
<wallyworld> wgrant: any idea why the 0.18.1 milestone is not showing up on the timeline?
<wgrant> wallyworld: Some of the timelines are cached.
<wallyworld> what's the cache refresh interval?
<wgrant> Huh.
<wgrant> Something strange is going on here.
* jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<wgrant> It's not showing up in the milestone listing either, but it is in the release listing...
<wgrant> https://launchpad.net/lazr.restful/+series
<wallyworld> wgrant:  look at the bottom :-)
<wallyworld> wgrant: it's because i entered the release date
<wgrant> Bottom of where?
<wallyworld> wgrant: sorry. wrong url. https://launchpad.net/lazr.restful/+milestones
<wallyworld> wgrant: but 0.18.1 is on the +series url for me
<wallyworld> wgrant: ah, no its not
<wgrant> Code time.
<wallyworld> wgrant: i think it's got the ordering wrong because of the release date being set for 0.18.1 and not the others
<wgrant> Ah.
<wgrant> Could be.
<wgrant> The release data is set on the rest, but the target date is not.
<wgrant> I removed the target date and now it's showing in the timeline.
<wgrant> And bypassing the cache now shows it everywhere else.
<StevenK> wgrant: Have you seen the stable->db-devel conflict, or shall I fix?
<wgrant> StevenK: I already fixed one this morning.
<wgrant> And there were two or three fixed yesterday.
<wgrant> Is there another?
<wgrant> No.
<StevenK> wgrant: Ah, you already handled the garbo one?
<wgrant> If it's not spamming more than every 20 minutes, it's fixed :)
<wgrant> Yes.
<poolie> does jc's recent proposal <https://code.launchpad.net/~jcsackett/launchpad/apparently-we-support-basic-auth/+merge/55845> mean lp actually will support basic auth?
<thumper> no
<thumper> I'm pretty sure basic auth is only supported in tests
<wgrant> It does support basic auth.
<thumper> in order to get windmill logged in without openid
<wgrant> But we need to delete it.
<thumper> the zcml to configure basic auth is only in the testrunner profile
<thumper> AFAIK
<poolie> ok
<wgrant> Are you sure? It is actively OOPSing on production and IIRC it works in the past...
<wgrant> But it won't for long :)
<poolie> :/
<thumper> really?
<thumper> ok
<wgrant> poolie: Why?
<thumper> I've not tried
<poolie> on the api it seems to just be ignored
<wgrant> poolie: Right, it's not supported by the API.
<wgrant> That uses different auth.
<poolie> wgrant, i think it's a good model for some api users
<poolie> it would shortcircuit some of the flying circus about getting credentials
<wgrant> poolie: Why?
<wgrant> Well, LP doesn't have passwords any more.
<poolie> specifically the text mode/server case
<wgrant> So it can't use basic auth.
<poolie> from the inside, no, you don't; from the outside obviously you do
<poolie> lp is tied 1:1 with a password database
<wgrant> Assuming that we make launchpadlib better, what's wrong with OAuth?
<wgrant> poolie: Yes, and that is stupid.
<wgrant> And an awful, awful hack.
<poolie> the main problem with oauth is that it pretty much requires an interactive graphical browser
<poolie> but this is an implementation problem not an essential problem
<wgrant> It doesn't.
<wgrant> It requires that you somehow navigate to that page.
<wgrant> It can be done on any host, but *should* work in text browsers anyway.
<poolie> you could implement something in lplib that takes a username and passowrd password and gives back a token
<wgrant> That's been rejected.
<wgrant> The policy is that your password must only be given to the SSO UI.
<wallyworld> wgrant: i'm coming in late to this discussion, but with firefox 4, you no longer get that annoying authorisation dialog pop up when running windmill tests
<wgrant> wallyworld: I saw your fixes.. does it actually work with 4 now?
<wgrant> It crashed terribly when I last looked, but that was some weeks ago.
<wallyworld> wgrant: *almost*. there's a few tests failing due to some keypress event handler weirdness
<wgrant> poolie: While LP doesn't maintain SSO any more, I think the policy is mostly sane.
<wgrant> Hmm. Except that there is now the desktop SSO client.
<wgrant> So maybe it has changed.
<wgrant> But still, I don't think launchpadlib is the place.
<poolie> that policy seems almost superstitious
<wgrant> If people are having issues opening the URL that it gives, then we should fix that.
<poolie> as if there is a difference between showing a form running in lynx, and showing the same forms in controls drawn by the application itself
<lifeless> wgrant: the oopses in prod fro base64 are bad filebug data attachments
<wgrant> lifeless: Ah.
<poolie> or in fact a captive webkit form is an even better example
<wgrant> poolie: Why would you use lynx instead of your local browser?
<wgrant> Unless you're using launchpadlib from a server console.
<poolie> yes, that
<wgrant> In which case you're more than likely doing it wrong.
<poolie> so, to back up
<wallyworld> thumper: how's natty going?
<poolie> how are people to get api credentials onho a server?
<poolie> *onto
<wgrant> poolie: launchpadlib gives a URL. They open the URL in their desktop browser.
<gary_poster> wallyworld, would you still like that pypi permission?
 * gary_poster just asked that question on #clojure.  oops.
<wallyworld> gary_poster: yes please. there's a lazr.restful 0.18.1 release to upload
<gary_poster> got it.  are you "wallyworld" there?
<thumper> wallyworld: stopped with unity as I couldn't get any work done
<wallyworld> gary_poster: yeah
<gary_poster> k
<thumper> may poke it with a stick again later
<lifeless> gary_poster: how do you like it?
<wgrant> thumper: Sounds like you have a big job in a couple of weeks :P
<wallyworld> thumper: ah, but i wouldn't use Unity. KDE Rulez!
<poolie> hm, i guess so
<poolie> so lplib should not try to open a browser in that case
<thumper> wgrant: yeah, seems so
<wgrant> poolie: Does it try to open a non-X browser? That's somewhat insane.
<gary_poster> lifeless, I like it quite a bit.  It's been four or five months since I actually played with it though. :-/
<lifeless> wgrant: w3m
<lifeless> wgrant: its *why* we fixed
<lifeless> (not saying thats sane, just history)
<poolie> i think it still does; imbw
<poolie> the other case i've hit this is when trying to file bug reports from a machine with no working k
<gary_poster> wallyworld you have the POWER
<poolie> *x
<poolie> but that might be too much of an edge case
<wgrant> So, I don't want to be giving my Ubuntu SSO password to arbitrary remote servers. (I also don't want to be giving them authenticated Launchpad tokenas at the moment, since they have unlimited scope, but that's a separate issue)
<lifeless> we explicitly support that; login with w3m
<wallyworld> gary_poster: awesome thanks. why am i now thinking of spiderman. "with great power comes great responsibility"
<gary_poster> wallyworld, :-) go save the world, man
<poolie> anyhow, this is fine
<poolie> i just wondered what the patch was about
<wallyworld> gary_poster: i'm saving the world one windows installation at a time :-)
<poolie> lifeless, so you're saying lplib should not launch w3m by default, but ... should give an option or something?
<poolie> or i should retype the url on a different console?
<gary_poster> wallyworld, lol, sounds like a good plan
<gary_poster> have a great day/night/etc., everyone!
<lifeless> poolie: I'm saying we support logging in using w3m so folk on server consoles can report bugs using apport
<lifeless> I think the launchpadlib story has gotten convoluted with different scenarios and use cases over time and could benefit from a ground up refresh; however its extremely low priority given our critical bug list
<poolie> i agree on both counts
<wgrant> Definitely.
<wallyworld> wgrant: lazr.restful up on pypi now :-)
<wgrant> wallyworld: Great! Shall I handle the upgrade?
<wallyworld> wgrant: you mean to lp versions.cfg?
<wgrant> Yeah.
<lifeless> which is of course a bit circular given that we choose what defines critical :P
<wgrant> Since I have immediate need for it.
<wallyworld> wgrant: i've already uploaded to download-cache and i've got a mp that should be ready to go but needs a final +1
<lifeless> poolie: btw you r oops on ~mbp/+branches
<wallyworld> https://code.launchpad.net/~wallyworld/launchpad/show-ajax-notifications/+merge/54342
<wgrant> wallyworld: Ah, great.
<lifeless> poolie: different bug; cold cache impact of a query examining every bug branch link you ever made
<lifeless> thumper: wgrant: which team is on CHR this week ?
<poolie> the one i mentioned yesterday?
<poolie> or maybe filed
<wgrant> wallyworld: Are the notifications always correctly escaped?
<wgrant> lifeless: thumper's, I think, but they aren't really their own any more...
<lifeless> also
<lifeless> wow
<lifeless> https://bugs.launchpad.net/launchpad/+filebug does an ajax request
 * thumper shrugs 
 * thumper looks at wallyworld
<lifeless> wgrant: so I ask because we have lots of untriaged bugs
<thumper> wallyworld: care to triage?
<wallyworld> wgrant: it depends. it pulls the notifications from the request so however they are put there....
<wallyworld> thumper: ok
<wgrant> wallyworld: OK, at the moment addNotification escapes them if necessary. So it's OK.
<wallyworld> wgrant: that's what i thought
<wallyworld> wgrant: but given all the holes lately, i wasn't confident to say "yes" to your question :-)
<wgrant> Heh
 * wgrant mauls IE.
<lifeless> wgrant: we need to rollback
<wgrant> lifeless: What's regressed?
<lifeless> wgrant: go to https://bugs.launchpad.net/launchpad/+bug/746866 and edit the description using ajax
<_mup_> Bug #746866: Person:+branches timeout: sometimes-slow bug-branch link query <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/746866 >
<lifeless> wgrant: make a trivial change - extra space at the end or something
<wgrant> Eep. wallyworld ^^
<wgrant> The saved data is OK, FWIW.
 * wallyworld looks
<wgrant> wallyworld: It uses 'undefined' as the new description representation.
<wgrant> Hmm. Is this the one widget which uses _defaultFormatter for HTML? :)
<wallyworld> oh bollocks
<wallyworld> it worked fine for blueprints etc on qastaging i think
<wallyworld> wgrant: yes, believe so
<wgrant> wallyworld: Yeah, I QAd all the blueprint widgets and a few things on bugs.
<wgrant> Blueprint description worked fine, but I mustn't have checked bug descriptions.
<wallyworld> wgrant: me too. i checked a few other places and it looked good
<wgrant> wallyworld: Can you see exactly what's going on here?
<wgrant> Ah, nevermind, we missed the buildbot run.
<wallyworld> wgrant: not yet. looking at qastaging
<poolie> lifeless, is "loses your data" enough to bump bug 735290 to critical?
<_mup_> Bug #735290: changing Project drop down in project group +filebug loses all your work <ajax> <bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/735290 >
<poolie> it's basically as bad as an oops
<poolie> indeed, worse, because you can retry an oops, but you can't retry this
<poolie> s//retry a POST
<lifeless> poolie: I don't think so
<poolie> :(
<poolie> fair enough
<poolie> it's awful when it happens though
<lifeless> it is
<wgrant> wallyworld: Huh, it seems to work OK locally.
<lifeless> unlike all the oopses and javascript fails, its not a complete blocker
<wallyworld> wgrant: yeah, i can't see in the code what's wrong
<lifeless> I suggest taking those keys off your keyboard :)
<wallyworld> wgrant: i was just about to run it up
<wgrant> Oh, it helps if my copy of devel is up to date.
<wgrant> stable was, but not devel.
<wgrant> Fail.
<wgrant> wallyworld: Is it because a Node doesn't have an innerHTML?
<wgrant> Still waiting for new devel to build.
<wallyworld> wgrant: nodes should all have that. which bit of code do you mean?
<wgrant> Hm, yeah, that's it.
<poolie> lifeless, lp may be getting to the point where fixing bugs like this is more important to user experience than making it faster
<wgrant> Now, how do I stringify a Node...
<wgrant> wallyworld: I have a hack that works.
<wgrant> I presume there's a more direct way.
<wgrant> -            return result.getHTML(attribute).get('innerHTML');
<wgrant> +            return Y.Node.create('<p/>').append(result.getHTML(attribute)).getContent();
<wgrant> I couldn't see a way to get a string representation directly.
<wallyworld> wgrant: i'm trying something atm. give me a minute
<wgrant> So, erm, bugs and blueprints use different AJAX textarea widgets?
<wgrant> I wasn't aware we had more than one :/
<lifeless> poolie: perhaps
<lifeless> poolie: I think there is great value in actually reaching a goal
<lifeless> poolie: it will take considerable time to turn around the perception that lp is slow
<lifeless> poolie: firstly we need to make it not slow, and then we need to keep it that way for a while ;)
<wgrant> (Windmill didn't catch this breakage, FWIW)
<lifeless> booyah
<lifeless> https://bugs.staging.launchpad.net/ubuntu
<lifeless> its not quite subsecond
<lifeless> but not terribly far off
<wgrant> I am disappoint.
<poolie> yeah, i do also agree with hammering it thoroughly flat
<poolie> just sayin
<wallyworld> wgrant: this is what we want
<wallyworld> -      return Y.Node.create(value);
<wallyworld> +      var result = Y.Node.create("<span/>");
<wallyworld> +      result.setContent(value);
<wallyworld> +      return result;
<wgrant> wallyworld: How is that different?
<wgrant> Apart from adding another layer, it looks like it should be the same.
<wgrant> Oh.
<wgrant> Does it get an innerHTML if it's a single element, whereas this is a sequence of <p>s?
<wallyworld> wgrant: it's done in the getHTML method, not the _defaultFormatter method
<wallyworld> wgrant: i think the getHTML method was constructing the node incorrectly
<wallyworld> with the above change, there's no need to change _defaultFormatter
<lifeless> I consider this timing a success: 03976-04169@SQL-launchpad-main-master SELECT BugTask.assignee, BugTask
<wgrant> wallyworld: A span is not the right thing, but we do need something like that.
<wgrant> Ugh, this is hard to do correctly.
<lifeless> good things often are
<wallyworld> wgrant: the span is just a container - it doesn't come out in the final html
<lifeless> *blink* 40 /   54  BugTask:+index
<thumper> that's good right?
<lifeless> yeah
<lifeless> was sitting at 180/* before
<lifeless> I'm wondering wtf changed
<lifeless> thumper: I really wouldn't tackle what you're working on by blocking branch renames.
<lifeless> thumper: Its up to you of course, but I fear it will just decrease usability and increase user frustration
<thumper> lifeless: I'm not
<lifeless> thumper: oh cool; what are you doing?
<thumper> lifeless: I'm trying to catch the object modified event for the branch and checking the unique name
<thumper> lifeless: creating a rename job if the unique name has changed
<thumper> and there are branches stacked on it
<thumper> however, since the unique name is a string col that is updated with a db trigger
<thumper> it isn't being reloaded on Store.flush
<lifeless> yeah
<thumper> I could easily use old skool logic
<thumper> which would work
<lifeless> thumper: _mark_autoreload
<lifeless> is related
<lifeless> obj_info.variables[column].set(AutoReload)
<thumper> looks private(ish) though
<lifeless> IIRC you do this:
<lifeless> obj.column = AutoReload
<lifeless> thumper: did you consider making the stacking location be id based rather than unique-name ?
<lifeless> thumper: and rewriting all the current branches Just Once to stack by id ?
<thumper> lifeless: I don't know how do work that well for HTTP access, or enough in the smart server to do it right
<lifeless> thumper: so for http its pretty easy - we already export branches by id in the dc; we just need to make a matching export to the world with the existing branch-privacy logic attached to it.
<lifeless> for the smart server, we have a hook already that mangles the stacked on location on branch open - a 95% solution would just decorate the set_stacked location in the backend
<thumper> actually that may well be a better solution than what I'm doing
<thumper> bugger!
<lifeless> I suspect its a little more fiddly, but much more robust - no race conditions, immune to any changes we make to branch names in the future etc
<thumper> yeah
<wallyworld> wgrant: https://code.launchpad.net/~wallyworld/launchpad/text-area-widget-html-fix/+merge/55860
 * lifeless forces himself away from coding 
<thumper> lifeless: I think what we need is a special traversal hook, like +branch-id
<wgrant> wallyworld: I don't quite understand what the root problem is. Is it that we can't get the innerHTML from a Node created from a fragment, instead of a single element?
<thumper> lifeless: so we could stack on /+branch-id/12345
<thumper> internally
<thumper> not sure what it would look like to the outside world
<wallyworld> wgrant: no exactly. innerhtml assumes that there is an element eg <div>, <span> etc wrapping the content
<lifeless> thumper: I think exporting that to the outside world would be fine
<wallyworld> wgrant: and so innerhtml returns what's inside that containing node
<lifeless> thumper: that is the heart of the solution (for working with http and sftp)
<thumper> yeah...
<thumper> unfortunately I think I'm probably the most qualified to actually implement this
<thumper> given jml and mwhudson not actively coding on LP now
<wgrant> wallyworld: This is probably a hack, but it seems to work.
<thumper> at least I understand much more of it now since my sloecode workl
<wallyworld> wgrant: not sure if it's a hack or not. seems to match what i've seen elsewhere. but i'm not expert
<LPCIBot> Project windmill build #128: STILL FAILING in 1 hr 11 min: https://lpci.wedontsleep.org/job/windmill/128/
<wgrant> wallyworld: You've tested a good variety of (potentially duplicated) widgets?
<wallyworld> wgrant: i tested bugs, blueprints and am about to double check recipes
<wgrant> Great.
<wgrant> We have more than two hours to confirm everything's really working.
<sinzui> huwshimi: have you formed an opinion of the my revision to the mailing list archive header?
<huwshimi> sinzui: ah sorry, yes, I think it's good
<huwshimi> sinzui: Again it would be nice to have this fully integrated, but let's go with this for now
<huwshimi> sinzui: At least it's consistent with Loggerhead
<sinzui> huwshimi: Then are the wiki's next? The Lp primary css is now usable by other Lp wsite
<wgrant> sinzui: The spacing under the "$TEAM team mailing list archive" seems to be a little small, but it looks good to me too.
<huwshimi> sinzui: I'm not sure what to do with the wikis. At the least the help wiki should probably look more like Launchpad, but it's difficult to know exactly what to do when the auth system etc. isn't integrated.
<huwshimi> sinzui: We have deeper issues about navigation etc. that need to be resolved to help us with that
<sinzui> wgrant: I thought so to. I suspect that lose of the image changed the positioning. I can ad some pixels
<sinzui> of white-space
<wgrant> sinzui: Is the title an h1?
<wallyworld> thumper: textareawidget - if hide_empty=True, then there's no way to enter text apart from using the html form. why do we have a hide_empty?
<wallyworld> eg sourcepackagerecipe index page
<wgrant> Was that for the recipe description?
<wallyworld> wgrant: yes
<wgrant> Most recipes don't have descriptions, so it took up too much room for too little gain, IIRC.
<wallyworld> but if you wanted to enter a description later...
<wallyworld> and it doesn't take up *that* much room
<wgrant> Then you're screwed, yeah.
<wgrant> It could be like the MP's commit message widget, I guess.
<wallyworld> i mean, in the context of what else is on the page, it's stuff all wasted space
<wgrant> That's just a single line when empty (an add button, not a normal textarea widget)
<thumper> yes, it is for the commit message and description
<thumper> it has been suggested that we integrate the menu items into the widgets themselves
<thumper> and have the code in one place for showing and hiding
<wgrant> That would be very sensible.
<thumper> but I didn't get that far
<wallyworld> thumper: and now you're desserting us
<wallyworld> or deserting, i can't spell
<thumper> wallyworld: I'll leave it in your capable hands :)
 * thumper is eating dessert
 * wallyworld lols
 * wallyworld needs food. dessert even
<wgrant> I could sit here and play with the blueprint implementation status widget all day. The way the other fields magically change...
<lifeless> bacj shortly, vet run
<wgrant> wallyworld: lifecycle_status was relying on getHTMLOrDefault?
<wallyworld> wgrant: yes. there is no html representation for that field
<wallyworld> thumper: bug 745230. it that really a bug or do they just need to edit their recipe when they rename a branch? or perhaps there was already a build pending when the branch was renamed, hence the error?
<_mup_> Bug #745230: recipe fails when branch gets new owner <Launchpad itself:New> < https://launchpad.net/bugs/745230 >
<wgrant> wallyworld: Ah, I didn't realise a bug was filed for that.
<wgrant> wallyworld: It is the stacked-on-a-renamed-branch bug.
<huwshimi> Is there any way to make my local launchpad dev server accessible over my network?
<wallyworld> wgrant: ah yes. i knew about that one. i didn't pick up that was at play here though. it looked like just a simple branch rename and a recipe referring to the old branch
<wgrant> huwshimi: https://dev.launchpad.net/Running/RemoteAccess should still mostly work.
<wgrant> huwshimi: You're not going to try and unbreak some of our JS in IE, are you?
<StevenK> huwshimi: The other thing you can do is 'utilites/ec2 demo'
<wallyworld> i reckon we shouldn't even attemp to support IE < 9 :-)
<huwshimi> wgrant: Would I dare? :P
<wgrant> StevenK: That was broken at one point. Not sure if it still is.
<wallyworld> why go to all the bother?
<wgrant> wallyworld: Sadly we have to :(
<wallyworld> why? sure our lp users are smarter than joe sixpack who swallows whatever dross ms dishes up with windows?
<wallyworld> s/sure/surely/
<wgrant> Ha ha ha.
<wgrant> It's not quite that simple.
<wallyworld> how so?
<wallyworld> especially with ie 9 out now
<huwshimi> wallyworld: Some people work in corporate environments that won't upgrade their browsers for various reasons. We have a significant number of stakeholders that fall into that category.
<wallyworld> huwshimi: how should i triage bug 744808? i'm thinking medium? or high?
<_mup_> Bug #744808: Redrawing of Bug View page very slow in Firefox <Launchpad itself:New> < https://launchpad.net/bugs/744808 >
<wallyworld> huwshimi: yes, i realise corporates still use ie 6 - i used to work at one. i didn;t realise there were that many of them
<wallyworld> or that they couldn;t install ff for lp use
<wallyworld> even at my old employer where ie 6 was the standard, we could still install ff etc. i went one step further and used linux :-)
<huwshimi> wallyworld: I don't know how you should classify that bug. We're going to need some more info about browser version etc.
<wgrant> huwshimi: It's an nvidia driver bug :/
<wgrant> huwshimi: Well, plus Firefox, I guess.
<wgrant> But all the other drivers do fine.
<wgrant> So I tend to blame nvidia.
<huwshimi> wgrant: If it's just one image that is cause it there are things we can do.
<wallyworld> huwshimi: i'll mark it as medium i think then
<wgrant> wallyworld: lifeless will hate you forever.
<huwshimi> wallyworld: I don't think we use medium
<wgrant> Medium doesn't exist :)
<huwshimi> wallyworld: It should probably be high
<wallyworld> high it i
<wallyworld> is
<wallyworld> oh, i forgot about not using medium
<lifeless> wallyworld: the rationale is this: high is nominally 6 months worth of work queued; with reviews to winnow it; anything below that is simply 'unscheduled'
<wallyworld> thanks for the reminder
<wallyworld> jtv: do you think 743913 is a duplicate as suggested in the bug report?
<jtv> wallyworld: looking
<wgrant> Bug #743913
 * wgrant pokes _mup_
<wallyworld> mup has been a bit slack of late
<jtv> wallyworld: that does look like it's probably the same issueâeven though it's not a simple one and the individual reporters may just as well have run into different sides of the problem.
<wallyworld> jtv: thanks. i'll mark it as a dup
<wallyworld> jtv: dpm poses an interesting question in this bug #743029 - when you have a moment it would be cool if you could take a look to provide an answer. regardless of the answer, it's a problem that needs fixing it appears
<_mup_> Bug #743029: Launchpad Translations only allows submitting 3 translation suggestions and discards the rest <Launchpad itself:New> < https://launchpad.net/bugs/743029 >
<lifeless> wallyworld: if you're doing triage; there are a couple of links in the BugTriage wiki page that jointly should be driven to 0
<wallyworld> ok
<huwshimi> I'm guessing there is no way to log into launchpad.dev without openid?
<wgrant> huwshimi: Basic auth might work.
<wgrant> admin@canonical.com:test
<huwshimi> wgrant: how do I get a prompt for that?
<lifeless> huwshimi: you don't
<wgrant> huwshimi: Put it in the URL.
<wgrant> Although I guess you'll need to escape the @...
<wgrant> Let's see if it works.
<lifeless> huwshimi: note that we don't support basic auth in prod
<huwshimi> lifeless: Yeah this is just to test over my network
<huwshimi> lifeless: From a vm running ie that doesn't like the openid stuff
<wgrant> Hm, doesn't seem to work.
<jtv> wallyworld: wow, that one is just insane.  But frankly it sounds as if it's more of an "only the latest suggestions are displayed" issue.  I'm not aware of anything like that having been done deliberately while we still had the Rosetta team, though I could well imagine someone deciding that if the same user enters successive suggestions for the same message in the same language, it's probably to improve on the older ones rather than to offer lots of alternati
<StevenK> huwshimi: How does it error?
<huwshimi> StevenK: It won't load https://testopenid.dev/+openid
<huwshimi> StevenK: I don't know how to get the header code
<StevenK> Can it even resolve it?
<wgrant> I think I ended up with SSL errors of some kind last time i tried... I don't remember details.
<wallyworld> jtv: yeah weird. thanks for looking
<huwshimi> StevenK: Possibly not. I assumed it was something to do with my network setup
<jtv> wallyworld: first I'd make sure that we understand what steps dpm went through.
<StevenK> Can haz review? https://code.launchpad.net/~stevenk/launchpad/dsd-dont-request-child-diff/+merge/55863
<lifeless> https://bugs.launchpad.net/launchpad/+bug/741092/comments/11 <- and they say python is slow
<_mup_> Bug #741092: Archive:+subscriptions times out with many subscribers <qa-ok> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/741092 >
<wgrant> wallyworld: It's almost buildbot time.
<jtv> wallyworld: that may sound strange given his description, but our UI doesn't actually support the steps he describes directly.  He'd have to submit the page every time, then page back to get to the message he was entering suggestions for.
<wallyworld> jtv: i'm going to try it on qastaging using the bug report as a guide
<wgrant> wallyworld: Your branch seems fine, so it should probably get into the next buildbot run.
<jtv> wallyworld: same here
<jtv> I'll try a different page then
<wallyworld> wgrant: i've submitted it to ec2 just after it was approved
<huwshimi> wgrant: Damn, doesn't like that either
<wgrant> wallyworld: Hmm, but we don't run windmill in ec2 yet, so will that actually test anything?
<wgrant> huwshimi: No :/
<wgrant> huwshimi: Can you see if the testopenid.dev request is making it to the appserver?
<jtv> wallyworld: by the way, be careful to do this in Translator mode.  Do it in Reviewer mode once, and the translation you enter will become official and your old ones will be counted as reviewed.
<huwshimi> wgrant: erm...
<wallyworld> wgrant: the tests are plain javascript ones. i thought windmill ones were separate again?
<wgrant> huwshimi: 'make run' prints all the requests it receives.
<huwshimi> wgrant: Ah
<wgrant> wallyworld: Ah, they're not still run in WindmillLayer?
<wgrant> test_yuitests lives in lib/lp/*/windmill....
<wallyworld> jtv: ok. on qastaging though i can mess up as much data as i want though?
<wgrant> class CodeYUIUnitTestCase(YUIUnitTestCase):
<wgrant> layer = CodeWindmillLayer
<jtv> wallyworld: yes, but:
<wallyworld> wgrant: i didn't appreciate that fact. i would have thought they were separate :-(
<jtv> wallyworld: dpm gave a specific link, and on staging you can tell that link doesn't do quite what he expected any more.  Be careful not to change the effect of the same link on qastagingâit can be a useful reference.
<wgrant> wallyworld: I guess they need similar infrastructure, to actually get them to run in the browser and such.
<wallyworld> jtv: yes, true
<huwshimi> wgrant: This is what I get after clicking on the login link: http://paste.ubuntu.com/588109/
<wallyworld> jtv: so to double check, the fact that it says "Your suggestions will be held for review ...." means I am in reviewer mode?
<jtv> No
<jtv> That means you're in translator mode.
<wallyworld> ah sorry, my fingers got mixed up
<wallyworld> i meant to say translator mode
<jtv> wallyworld: I just tried it and yes, it only shows my latest 3 suggestions for that one particular message.
<wallyworld> jtv: so the bug is legit
<wallyworld> ?
<wallyworld> or if it's be design we need to say that
<wallyworld> by
<wallyworld> and why
<jtv> In the sense that we can reproduce it, yes it's legit.  Another question is whether it's a bug.  To be honest I don't see why, if a single person suggests >3 different translations for the exact same message into the exact same language, we should expect all of those to be useful.  I think it makes sense to assume that the person is improving, rather than throwing a bunch of garbage the reviewer's way after making a good suggestion.
<jtv> Think about the spam or shelf-stuffing potential alone.
<wgrant> huwshimi: Hm, do other browsers work?
<wgrant> huwshimi: I'm able to log in from another machine OK.
<wgrant> huwshimi: Although it's not Windows.
<StevenK> huwshimi: Do you happen to have Firefox on that Windows machine?
<wallyworld> jtv: i agree, and it seems any change was likely done for that reason. so i can mark it as won't fix and explain why?
<huwshimi> StevenK: Just installing it
<jtv> wallyworld: to be blunt, "won't fix" may be too mild.
<jtv> More of a feature-not-bug.
<lifeless> opinion
<wallyworld> :-)
<huwshimi> wgrant: Firefox doesn't work either
<jtv> lifeless: I agree completely with that being an ideal resolution for this, except in this case the discussion hasn't played out to that point yet.
<jtv> wallyworld: BTW the messages are not actually gone.
<wgrant> huwshimi: What error does the browser show?
<jtv> wallyworld: The reviewer can still get to them by "zooming in" on the message.
<huwshimi> wgrant: Same thing. Can't find the server
 * wallyworld tries that
<jtv> wallyworld: They just don't clutter up the regular translation page which must show multiple translations.
<wgrant> huwshimi: You added testopenid.dev and launchpad.dev and bugs.launchpad.dev and anything else of interest to C:\Windows\System32\drivers\etc\hosts, or whatever it is these days?
<wgrant> http://pastebin.ubuntu.com/588111/ is the config diff I use.
<jtv> wallyworld: â¦which on the one hand makes perfect sense, though on the other the reviewer obviously has no sane reason to assume that the translator may still want them to see the suggestion they made 4 page submissions ago.
<huwshimi> wgrant: Yeah it's all there. The other urls work so it must be working
<StevenK> I thought testopenid.dev was a different IP
<wallyworld> jtv: all valid points.
<jtv> wallyworld: bear in mind that, thanks to the back-paging, the translator must go to considerable extra trouble to enter the redundant messages.  So it ends up being a bit of a "doctor it hurts when I punch myself in the eye."
<wgrant> StevenK: It's not really configured properly.
<wallyworld> jtv: i didn'h have to backpage at all
<wallyworld> perhaps there's an ajax version working for me
<jtv> wallyworld: no, that's probably because you happen to be on a view that only has one page to display then.
<jtv> Even so, it's not like the submit is instantaneous.  So this is a pretty rare case.
<jtv> Or should be.
<wallyworld> yeah
<jtv> If anyone's running into this, I think it's probably because of some tangentially-related misconception or misleading UI subtlety.
<wallyworld> jtv: i'm commenting on the bug now and we'll see what reply we get. did you want me to subscribe you to it?
<jtv> wallyworld: sure, thanks
<wallyworld> jtv: thanks for your help with this
<jtv> No worries.  Bear in mind by the way that this is David acting as a conduit, which is part of the Ubuntu Translations Coordinator role he's been fulfilling wonderfully.  It's great for all of us to establish contact with him about these things; he lost a Rosetta team as a support point so it'd be good to know he's gaining a Launchpad team in its place.
<huwshimi> wgrant: Any other ideas :)
<wgrant> huwshimi: How does your Apache config look compared to the diff I provided?
<wallyworld> jtv: agreed. the past few times i've come across input from him i've tried to act promptly on it.
<wgrant> I don't *think* I changed anything else to get it to work.
<wgrant> But I remember it didn't initially.
<jtv> That's great.  He represents a lot of people's needs!
<huwshimi> wgrant: http://paste.ubuntu.com/588114/
<huwshimi> wgrant: Not a diff
<wgrant> huwshimi: Try http://pastebin.ubuntu.com/588115/ (you might need to fix the branch-rewrite path, but I don't think Apache will complain too much if it's wrong)
 * wallyworld does the school run
<huwshimi> wgrant: No change
<wgrant> huwshimi: Do you even get to the login prompt?
<huwshimi> nope. I think it's just 404ing
<huwshimi> but I don't know the actual status code cause the browsers are trying to be nice and hiding that info from me
<wgrant> Firefox at least should be able to tell you.
<wgrant> Otherwise maybe tail /var/log/apache2/other_vhosts_access.log?
<huwshimi> wgrant: Yeah pretty sure it's a 404
<huwshimi> wgrant: "Firefox can't find the server at testopenid.dev"
<StevenK> tcpdump ? :-)
<wgrant> That doesn't sound like a 404.
<wgrant> Can you ping testopenid.dev OK?
<wgrant> Ah hm.
<huwshimi> wgrant: It gives exactly the same error when I go to a non-existent url
<wgrant> Is the default-ssl site enabled in /etc/apache2/sites-enabled?
<wgrant> Mine isn't.
<wgrant> That may be what I did to fix it.
<wgrant> sudo a2dissite default-ssl
<huwshimi> wgrant: Pinging fails
<wgrant> That could also be a problem.
<wgrant> Sounds like your C:\Windows\System32\Drivers\etc\hosts is broken...
<huwshimi> wgrant: it's possible
<huwshimi> but it's weird that it works for some urls
<huwshimi> wgrant: This is what I have in my hosts: http://paste.ubuntu.com/588122/
<wgrant> Yet 'ping testopenid.dev' fails to resolve?
<huwshimi> wgrant: yep
<huwshimi> wgrant: "Ping requests could not find host testopenid.dev. Please check the name and try again."
<wgrant> And it's XP, so it's not UAC virtualisation...
<wgrant> I'm out of ideas :(
<huwshimi> wgrant: Ok thanks anyway :)
<huwshimi> I guess I'll leave IE broken for now, I can't think of anything else to try either.
<lifeless> 7M renders yesterday
<wgrant> lifeless: Nice.
<huwshimi> wgrant: If this helps all the base urls: bugs. translations. etc. work but anything else breaks: shipit. lists.
<lifeless> wgrant: after opstats etc
<wgrant> huwshimi: Interesting.
<huwshimi> wgrant: I think I must have some kind of port set up incorrectly or something
<wgrant> huwshimi: That doesn't explain why ping wouldn't work.
<huwshimi> wgrant: by breaks I mean I can't ping any of them
<wgrant> huwshimi: I wonder if Windows' hosts file implementation has a limited line length.
<wgrant> huwshimi: I would not put it past them. Try removing all except the hostnames you need.
<huwshimi> wgrant: I thought about that but they're spread out through that list
<wgrant> Bah
<huwshimi> wgrant: I actually wonder if it's the Allow from in the apache file
<wgrant> huwshimi: That doesn't affect name resolution.
<huwshimi> wgrant: Ah right
<wgrant> testopenid, lists and shipit are all in the last half of the line.
<huwshimi> wgrant: oh, maybe I was wrong before
<huwshimi> wgrant: Oh I am going to stab things. Hard.
<wgrant> huwshimi: Oh?
<huwshimi> wgrant: Moving it to the start of the line makes it work
<wgrant> Yeah.
<wgrant> When in doubt, suspect shoddy Microsoft code? :)
<huwshimi> wgrant: That really is aweful. I just don't understand.
<wgrant> Yes :/
<wgrant> huwshimi: So, it's all working now?
<huwshimi> wgrant: It exploded in an oops at first, but now it's working fine. Thanks heaps for your help :)
<huwshimi> wgrant: Now I probably owe you cake too
<wgrant> Haha.
<wgrant> I'm glad that it's working.
<wgrant> Maybe you can even make it work slightly in IE9 :)
<huwshimi> wgrant: Maybe I should just stab myself in the face with my desk?
<wgrant> IE will encourage you to sharpen your desk.
<wgrant> So maybe.
<StevenK> Haha
<wgrant> (if the hosts brokenness didn't already)
<lifeless> huwshimi: hosts files in windows have a maximum line length
<poolie> huwshimi, i would love it so much if you proceeded with your idea to remove the variable sizes from the bug tag cloud
<huwshimi> wgrant: I think I just blunted my desk from that
<huwshimi> lifeless: hahahahaha, thanks :D
<poolie> ever since i saw the mockup, the current ui hurts me even more
<huwshimi> poolie: Me too!
<wgrant> lifeless: Yes, we worked that out after an hour or so :P
<wgrant> Which mockup?
<poolie> so i think you're morally obligated
<lifeless> wgrant: I was writing a mammoth mail; sorry
<huwshimi> wgrant: https://bugs.launchpad.net/launchpad/+bug/709009
<_mup_> Bug #709009: Tag clouds are very hard to read <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/709009 >
<huwshimi> oh so *now* mup decides to work, just after I've given up and started pasting bug links into the channel
<wgrant> Bug #1234
<_mup_> Bug #1234: Gina is an unmaintainable mess of command line options, environment variables and shell scripts <lp-foundations> <Launchpad itself:Fix Released by debonzi> < https://launchpad.net/bugs/1234 >
<wgrant> Huh.
<wgrant> Indeed.
<wgrant> Bad _mup_
<huwshimi> wgrant: Specifically: https://launchpadlibrarian.net/62993362/tag_cloud.png
<wgrant> (nice mockup)
<huwshimi> wgrant: But I think I would like to revise that slightly again when I get to it
<wgrant> Yeah.
<wgrant> A lot of other things need revision, too :/
<poolie> i wonder now, given robert's point about performance
<poolie> if it would be better to just simply show all the official tags
<huwshimi> poolie: My brain no longer functions. I can not reply
<poolie> ! sorry
<LPCIBot> Project windmill build #129: STILL FAILING in 48 min: https://lpci.wedontsleep.org/job/windmill/129/
<poolie> wow, trying to get a sorted list of tagged bugs is awful
<wgrant> As awful as <script> tags in TAL?
<wgrant> Because fuuuu
<poolie> i want a list of bugs in bzr tagged doc, sorted newest first
<poolie> you'd think clicking the tag then clicking sort would get this, but no
<poolie> i know there's already a (bunch of) bugs for that
<wgrant> poolie: Yeah, easiest way to search is URL-hacking.
<wgrant> &orderby=-datecreated is handy
<bigjools> morning people
<poolie> hi jools
<poolie> mrevell, "we have a different definition of success" nice
<poolie> i wonder if we can articulate it?
<poolie> i guess it's something like "the success of our users, rather than the number of users"
<mrevell> poolie, And perhaps, "the success of free software, in particular as distributed by Ubuntu."
<poolie> yeah, but that's getting pretty diffuse
<mrevell> Yeah, it is but I worry that the word "users" is perhaps too narrow. People with LP accounts is probably what I'd first think of as a "user" in this context. So, we see their success as a good thing but our vision is for a success that helps Ubuntu in particular and free software projects in general. Maybe that doesn't all need to be said in that short paragraph.
<wgrant> bigjools: Yes, deprecate partner and delete most of IDistroSeries and IDistribution please :)
<poolie> mrevell, it seems like pinning it down precisely is a lot like hard work and doesn't obviously pay off
<bigjools> wgrant: :)
<poolie> or maybe it is super important
<poolie> understanding it is more important than saying it
<bigjools> wgrant: the soyuz stuff in those two model classes is all pretty awful old code, we need to move it out
<bigjools> or delete it
<wgrant> Yeah, but some of it has good reason for being there.
<bigjools> IArchive seems like the obvious place but it's already too bloated
<wgrant> It default to all_distro_archives if archive is not specified.
<wgrant> s/It/They/
<bigjools> h I missed that
<bigjools> still not right for us
<wgrant> Partner is not right for anyone :)
<bigjools> and I hate the word
<gmb> stub, lifeless: Could you take a look at https://code.launchpad.net/~gmb/launchpad/team-subscription-opt-out/+merge/55779 for me?
 * stub has a look
<stub> gmb: Do we have any future plans for people tweaking their subscription type, such as... umm... summary only, or digest modes, or selecting alternate email addresses?
<gmb> stub: Not that I'm aware of (though those are all good ideas and I'm going to pretend you haven't suggested them so that I don't have to think about them)
<stub> I thought they were sucky ideas but the best I could come up with on the spot
<stub> I'm just wondering if we want BugSubscriptionFilterPreferences instead that can hold more than just a single boolean state.
<stub> Its probably YAGNI
<gmb> Yeah, I suspect to.
<gmb> (Mainly because I don't think anyone's going to be touching subscriptions for a while once we're done with ti)
<stub> Nothing to stop us migrating to that with the suggested data model anyway
<gmb> True.
<stub> gmb: Do you have any objection to dropping the id column? We are starting to do that in places where it makes sense, and here is such a case (we just need to declare (filter, person) the primary key instead)
<gmb> stub: No, that's fine. In fact Gary suggested it and I forgot to make the change.
<stub> cool. The relevant code changes just look like removals.
<gmb> Right.
<stub> Missing a NOT NULL on the date_created.
<gmb> Whoops. Mea culpa; I cargo culted that from somewhere else.
<stub> gmb: So... how do we cope with people dropping out of teams (or teams dropping out of teams)?
<stub> gmb: Do those rows just rot?
<gmb> stub: The idea is that we'll drop those rows.
<stub> So they won't be matched when sending notifications, and there will be a garbo job to remove the noise?
<stub> (A case could be made for not having the garbo job - drop out of a team and rejoin and your settings are the same as they used to be. Or maybe that is suprising?)
<gmb> stub: Sorry, can you repeat your last two messages? My IRC client went all wibbly after my "drop those rows"
<stub> (17:01:29) stub: So they won't be matched when sending notifications, and there will be a garbo job to remove the noise?
<stub> (17:02:21) stub: (A case could be made for not having the garbo job - drop out of a team and rejoin and your settings are the same as they used to be. Or maybe that is suprising?)
<gmb> Ta
<gmb> I think the Garbo job would be the way to go. It seems more logical that your settings dissappear with your membership
<stub> With a garbo, your settings *might* disappear - depends if you rejoin before it kicks in :)
<stub> Anyway - that is irrelevant to the DB patch.
<wgrant> stub: Speaking of garbo, is there any specific QA for your parallelisation?
<stub> wgrant: If the garbo script runs without exploding, it is qa-ok
<stub> Maybe we should check it defaulted to > 1 thread too if the debug output is enabled
<stub> gmb: r=stub, approved version of script in the mp
<gmb> stub: Brilliant, thanks.
<gmb> stub: Off the top of your head, do you know how to declare the composite primary key in Python for Storm's sake?
<stub> __primary_key__ = 'person', 'filter'
<gmb> stub: Ah, thanks.
<stub> Sorry - __storm_primary__ = 'person', 'filter'
<stub> gmb: ^^
<gmb> Right, ta
<deryck> Morning, folks.
<danilos> mrevell, how can we tell if LP downtime announcement is not an April 1st prank? :)
<mrevell> danilos, Hah, I did wonder about putting something in there about that.
<henninge> Hi deryck!
<henninge> deryck: If you have some time to chat about the js code.
<danilos> mrevell, you should have announced it as "On April 6th, we are scheduling the final downtime for Launchpad: after this one, there won't be any more because we are not bringing it back"
<mrevell> heh
<henninge> s/to/we could/
<deryck> henninge: I can chat here certainly.  If you want to do voice, I try to hold voice calls until top of the next hour if possible....
<mrevell> danilos, bigjools suggested changing the font to comic sans
<deryck> henninge: children sleeping nearby :-)
<danilos> mrevell, heh, I'd prefer "dingbats" or something like that
<jml> mrevell: google beat us to it
<henninge> deryck: since it's just us two, we can do it at the regular time.
<jml> search for "Helvetica" in google
<deryck> henninge: standup time?  Sure.  That works for me.
<mrevell> oof
<henninge> deryck: yes
<deryck> henninge: ok, thanks.  Sorry to put you off about it.
<henninge> deryck: np at all!
<bigjools> mrevell, jml: searching for launchpad, we're only the 4th result :/
<henninge> ;-)
<henninge> bigjools: on German google we are first
<deryck> bigjools: first for me too.  Logged in or not.
<bigjools> I was using .co.uk
<bigjools> oddness
<deryck> yeah, weird.
<henninge> bigjools: even there
<elmo> it's pretty hard to get untainted results from google these days
<bigjools> I jsut did it again and we're 4th
<jml> bigjools: that's google UK, right?
<elmo> unless you're on a fresh IP (range) with no cookies etc.
<bigjools> yes
<jml> bigjools: yeah. I get the same as you then.
<henninge> bigjools: ah, now I get it, too.
<henninge> (needed to switch to English)
<jml> tbh, not sure I care too much about being #1 on everyone's google search
<jml> would rather be #1 for "project hosting" or something like that
<bigjools> jml: and for that we're not even on the first page
<jml> bigjools: yeah.
<bigjools> oddly, google is top.  fancy that
<jml> bigjools: turns out you get what you put in.
 * jml rebooting for upgrade
 * deryck really meant to take today off to avoid the Internet
<bigjools> deryck: +1
<wallyworld> henninge: hi. are you able to +1 on that notification mp? me and wgrant need the lazr restful upgrade to land
<henninge> wallyworld: oh sorry, forgot about that
<wallyworld> henninge: np.
<henninge> wallyworld: I also thought you had mentioned another reviewer ...
<henninge> going there now
<wallyworld> henninge: i didn't have anyone picked out. it's only a fairly small branch :-)
<henninge> ok
<henninge> ah, at the end of your comment "Of course! I've asked Benji to review it."
<henninge> but you were referring to the lazr-js branch
<henninge> my erro
<henninge> r
<wallyworld> henninge: i think i was referring to the lazr restful one
<henninge> what I meant ;)
<wallyworld> yeah, just messing with you
<henninge> wallyworld: how does "node.set('innerHTML', notification);" scale up to wgrant's mail about xss attacks?
<wgrant> It's evil but fine,.
<wgrant> Since our notifications are escaped by addNotification.
<wallyworld> henninge: what he said
<henninge> ok, cool
<henninge> wallyworld: approved although I have a further refactoring suggestion that I put in a branch. Have a look at my reply, please.
<henninge> wallyworld: thanks for bearing with me. ;-)
 * henninge is off to lunch
<jelmer> jml: Hi
<jelmer> jml: Do you if there was any discussion about the security implications of bfbip ? The LEP mentions it's something that should be considered but it doesn't have any further details.
<jml> jelmer: yes, there were lots
<jml> jelmer: on ubuntu-devel
<jelmer> jml: Ah, I'll have a look in the archives. Thanks.
<wgrant> The security issues are going to be a little interesting :/
<jelmer> wgrant: yeah
 * jelmer was thinking it would make sense to generate a manifest file that included the signed bzr testaments for the revisions involved
<jelmer> but I'll catch up on the ubuntu-devel thread first
<wgrant> jelmer: I think that's what we'll have to do.
<LPCIBot> Project windmill build #130: STILL FAILING in 1 hr 11 min: https://lpci.wedontsleep.org/job/windmill/130/
* bac changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: bac | https://code.launchpad.net/launchpad-project/+activereviews
<jml> gary_poster: we were going to have a meeting *next* week to talk about bug subscriptions, right?
<gary_poster> jml, right.  Monday?
<jml> gary_poster: Monday's perfect.
<henninge> Hi deryck! Sorry for the delay, I'll abe with you in a few minutes
<deryck> henninge: no worries.  I thought you april fooled me for the standup today. ;)
<gary_poster> jml, cool.  The big change from last time is that things have landed on qastaging.  So it can be a quick one probably.  I'll send you an email with details as before and you can propose a time for the meeting?
<jml> gary_poster: sounds great, thanks.
<henninge> deryck: gah, missed a chance ...
<henninge> ;-)
<gary_poster> cool
<deryck> henninge: I would have hated you deeply for it ;)
<wallyworld> henninge: thanks for the suggested changes. i've merged them in. i considered setting up a structure like that but with only 4 levels, it seemed ok without. but i like your changes
<deryck> henninge: http://javascript.crockford.com/private.html
<wallyworld> deryck: i'm landing the windmill prerequisit branch now. i've merged in your other changes into mine. will run that through ec2 over the w/e and keep my fingers crossed :-)
<deryck> wallyworld: awesome, thanks!
<wallyworld> np. i sure hope it passes ok :-)
<deryck> henninge: 2 minutes and I'm ready again
<deryck> henninge: http://developer.yahoo.com/yui/3/api/
<deryck> henninge: http://developer.yahoo.com/yui/3/api/YUI~oop.html#method_bind
<deryck> henninge: http://developer.yahoo.com/yui/3/api/module_widget.html
<jtv> allenap: did you say you reformed how our CSS is composed as well?  My "make" breaks on picker-skin.css not being where it's supposed to.
<allenap> jtv: No, I didn't change that. There have been some changes in that area recently though. Does it break after a make clean?
<jtv> Oh, the softlink there makes no sense at allâ¦
 * jtv tries that
<jtv> (May be why I've got the EC2 script hanging as wellâ¦ I hope the meter isn't ticking while it's ignoring my instructions to shut down)
<jtv> allenap: yup, "make" works.  Thanks.  Perhaps I rely in rocketfuel-get too much.
<allenap> jtv: Cool :)
<allenap> sinzui: Have you got a few minutes? In https://dev.launchpad.net/LEP/DerivativeDistributions, the mock-up for the amended +addseries page shows fields for name and displayname. There is no title field any more. However, for Ubuntu, displayname is always an init-capped name (see http://pastebin.ubuntu.com/588237/). I would prefer to keep the name and title fields, and auto-fill displayname. Do you have any thoughts about this?
<sinzui> allenap: we want to remove title from all our objects
<sinzui> We have spent about 4 years ignoring it, but we really do mean to do it
<sinzui> Title was only intended to be used for the title in the browser (which will not work any more) and on the index page, which causes confusion between what you see and what you get.
<allenap> sinzui: Okay. In the case of Ubuntu series, will people expect to put "The Natty Narwhal", for example, in as the displayname instead of title?
<sinzui> allenap: I think people are being senseless. "The" make no more sense than "The Mozilla Firefox". It does not work in a sentence or a title.
<sinzui> allenap: I want to look up one or two series bugs that are relevant to how we present names.
<allenap> sinzui: I agree, but I don't think there's another field that holds the full series codename: name and displayname have just "natty" and "Natty". Only title contains "Natty Narwhal".
<jml> oh hmm
<jml> this reminds me of a spec thing that mpt wrote
<jml> https://dev.launchpad.net/RegistrySimplifications
<mpt> I'm old enough to remember back in 2005 when "Launchpad" was called "The Launchpad".
<jml> "it's cleaner"
<sinzui> allenap, jml, mpt: bug 419788, bug 493024 and bug 547082
<_mup_> Bug #419788: Display name for distribution series doesn't follow vendor conventions <distributions> <lp-registry> <Launchpad itself:Triaged> < https://launchpad.net/bugs/419788 >
<_mup_> Bug #493024: Debian sid/experimental have meaningless numeric versions assigned for Launchpad's internal purposes only <lp-registry> <series> <Launchpad itself:Triaged> < https://launchpad.net/bugs/493024 >
<_mup_> Bug #547082: LTS series are not designated as such <distributions> <lp-registry> <Launchpad itself:Triaged> < https://launchpad.net/bugs/547082 >
<sinzui> allenap: I think we also need to backout the named_version attr I added. Users are confused by the version and display name. And we used them inconsistently. We may want to make displayname a smart property that know to show both
<gmb> Can anyone remind me how to export a property that returns a Boolean via the webservice? I can't export Attribute()... BooleanField maybe?
<allenap> sinzui: Would the codename field then contain "Natty" or "Natty Narwhal"? If the former, is there any place for the full codename or will we assume that it will be added to the summary?
<james_w> gmb, IArchive.isSourceUploadAllowed might give you inspiration
<gmb> james_w: Aha, ta.
<sinzui> allenap: *I think* code name should be "natty", isn't is used in URL? version is "11.04". displayname in the db is "Natty Narwhal"
<mpt> codename != name
<mpt> name = "natty", codename = "Natty" or "Natty Narwhal" (at the driver's discretion)
<sinzui> allenap: but please consider that while we develope, pages need to say Natty Narwhal, when it is release, we refer to it as Ubuntu 11.04
<jml> https://launchpad.net/ubuntu/oneiric â Apparently -1 packages need linking.
<allenap> At the moment we have name=natty, displayname=Natty and title="Natty Narwhal".
<mpt> My spec doesn't solve bug 493024, but I guess that would involve making version nullable -- and then deriving displayname from codename not just  before release date, but also after release date if there is no version.
<_mup_> Bug #493024: Debian sid/experimental have meaningless numeric versions assigned for Launchpad's internal purposes only <lp-registry> <series> <Launchpad itself:Triaged> < https://launchpad.net/bugs/493024 >
<sinzui> jml: the copying of package data took a few days for M and L.
<mpt> (I guess sid/experimental don't have release dates anyway, so it's moot.)
<sinzui> jml: I was terrified at first because I kept seeing my name appear for packages I linked months before
<allenap> So that would migrate to name=natty, codename="Natty"-or-"Natty Narwhal" and displayname=(... a property ...)
<sinzui> allenap: I think that gets us close to what we want
<sinzui> I am unsure of LTS. I think it needs its own field/flag so that we can show Ubuntu 10.04 LTS
<allenap> sinzui: Can that go in the version field?
<sinzui> allenap: no, version must be a debversion
<allenap> sinzui: I will just capture name and displayname for now then, auto-fill title, and discuss fixing bug 419788 with bigjools.
<_mup_> Bug #419788: Display name for distribution series doesn't follow vendor conventions <distributions> <lp-registry> <Launchpad itself:Triaged> < https://launchpad.net/bugs/419788 >
<allenap> sinzui: Ah, okay.
<sinzui> though I recall we have two definitions of debversion. While both will oops with if you try to add SPACE  " LTS", the oopses may be different
<jml> huh. Launchpad has a million revisions.
<sinzui> \o/
<sinzui> We have reached the end of development
 * sinzui gets out the django books
<mpt> Is debversion some rule set by Debian? If so, why do non-Debian distributions in Launchpad have to follow it?
<jml> please go to the main dais to collect your prize
<sinzui> mpt: you will find one or two bugs that strongly state Lp cannot support non-debian distros. those that we have work by accident, though I would hardly call them weorking
<sinzui> mpt: we cannot support non-debian packages names either
<mpt> By "non-Debian" I meant Ubuntu, actually
<bigjools> mpt: Debversion is not connected to Debian other than that they use it and they named it.  Ubuntu uses it too.
<bigjools> but also Launchpad uses it
<mpt> bigjools, well either "10.04 LTS" is a valid debversion (I'm assuming it isn't), or Ubuntu occasionally uses a version which isn't a debversion.
<bigjools> mpt: that looks more like a title :)
<mpt> heh
<mpt> If Windows had been Debian-based, "98 SE" would still be a very different version from "98".
<mpt> (yeah, I know, it was really version 4.10 underneath.)
<james_w> hi, is someone available to debug bug 745801?
<_mup_> Bug #745801: system-based authorization doesn't store useful credentials in gnome-keyring <amd64> <apport-bug> <natty> <launchpadlib :New> <python-launchpadlib (Ubuntu):New> < https://launchpad.net/bugs/745801 >
<james_w> it's not obvious to me, and may break launchpadlib entirely for some people with the move to system-wide credentials
<jtv> Does anyone have any ideas why my ec2 instance seems to be hanging where it's supposed to be branching off db-devel?  I can ssh into the instance and "bzr ls" to the same branch URL so I don't understand why the dev script isn't moving on from that point.
<jml> jtv: no.
<jml> jtv: tried strace?
<jtv> Noâ¦ but I'm wondering if maybe either ssh or bzr is waiting for input.
<jtv> It'd have to be bzr.
<jml> Wondering is a good start.
<jtv> I hit enter just now, and then I got a traceback saying the bzr command failed.
<jtv> It's going to be a noninteractive shell, so weird things happen if the remote command prompts for inputâthough normally I think that means it will fail rather than block.
<jtv> Nope, nope, "ssh cat" will happily take client-side input and forward it to server-side stdin.
<jtv> As will subprocess.call.
<jtv> Let's see whether the bzr command actually has a process on the instance.
<jtv> Holy cow!
<jtv> It's not hanging, it's busy.
<jtv> Or was.
<jtv> Then it falls idle, and when I hit enter in the terminal that's running the ec2 script, I get the traceback.
<jtv> So it looks as if bzr is sitting there forever waiting for input.
<jtv> What input does a "bzr branch" ever ask for?
<jml> jtv: none that I know of.
<jtv> I just got a "broken pipe" during instance startup, by the way.  That's unusual.
<jml> jtv: if you can reproduce the problem, use strace, then you'll see if it's writing a prompt, what it's writing, whether it's waiting for input etc.
<jtv> That's what I'm doing now, yes.
<jtv> It's completely reproducible so far.
<jtv> Oh private parts!
<jtv> The "bzr branch" seems to complete.
<jtv> write(2, "Branched 10381 revision(s).\n", 28) = 28
<jtv> ioctl(2, SNDCTL_TMR_TIMEBASE or TCGETS, 0x7fff18dd7ce0) = -1 EINVAL (Invalid argument)
<jml> jtv: what's going on locally?
<jtv> On my system?  Just a RunTimeError saying the "bzr branch" command failed.
<jtv> It happens in this order:
<jtv> "bzr branch" does its work, and prints its "Branched 10381 revision(s)" which normally means success.
<jtv> It then does that ioctl, which fails, and it exits.
<jtv> The ec2 script, which is doing a subprocess.call on an ssh command line that runs the "bzr branch," doesn't notice this until I hit enter on my client system.
<jtv> When I do, it reports failure without further details.
<jtv> Oh, here's more fun:
<jtv> ettimeofday({1301673535, 408435}, NULL) = 0
<jtv> write(3, "96.727  return code 0\n", 22) = 22
<jtv> exit_group(0)                           = ?
<jtv> No idea what that last write means, but it sort of suggests success as well.
<jtv> Maybe it's just a crappy connection.  On one connection I use a lot I've been completely unable to use ec2.
<jml> me neither. lsof could probably tell you what fd 3 means. what's the local traceback?
<jtv> And my IRC connections keep dropping in several places ("because TCP connections use port 80 and are short-lived," I guess)
<allenap> There is an inverse correlation between my patience with diff generation in merge proposals and my age.
<jml> allenap: nice patch. couple of questions.
<allenap> bac: Do you fancy taking a gander at a very short improvement for LayeredDocFileSuite? https://code.launchpad.net/~allenap/launchpad/sensible-test-ids-for-doctests/+merge/55959
<jml> allenap: have you tried it?
<jtv> jml: the local traceback just says that prepare_tests failed in run_with_ssh_agent, doing the "bzr branch <â¦> /var/launchpad/test"
<allenap> bac: Cancel; jml's seen it.
<allenap> Thanks though.
<allenap> jml: I've tried it only for a limited set of tests.
<jml> allenap: and, also, it might be nice to normpath the id, since the __repr__ tends to have "/../" in it.
<jml> allenap: I'll give it a go locally.
<allenap> jml: Okay, good idea. Do you think I should normpath(test._dt_test.filename) then, instead of using __repr__?
<jml> allenap: uhhh... pass.
<allenap> jml: I'll be brave and do it then ;)
<jml> wait, Henri II, no crap that's University Challenge
<jml> test: lib/lp/code/tests/../doc/branch-notifications.txt
<jml> successful: lib/lp/code/tests/../doc/branch-notifications.txt
<jml> \o/
<allenap> Woohoo :)
<jml> and it works for pagetests too
 * allenap is getting dizzy.
<benji> bac: incoming: https://code.edge.launchpad.net/~benji/launchpad/better-HTML-generation/+merge/55960
<jml>             test.id = lambda: os.path.normpath(repr(test))
<jml> that works.
<allenap> jml: But the right hand side of the lambda is references test from the local scope doesn't it, so all tests would have the same id as the last.
<allenap> jml: My solution is: test.id = partial(os.path.normpath, test._dt_test.filename)
<bigjools> jml: when is convenient to have a phone call about derived distros next week?
<jml> allenap: well spotted
<jml> allenap: if that works, do it. thanks so much for the patch. this has been bugging me for ages.
<allenap> jml: Cool, I'll send it off. Thanks for looking at it.
<jml> bigjools: any time in the morning.
<jtv> jml: got a bit further by setting ServerAliveInterval to 5!  Still failing, but it's progress
<bigjools> jtv: shouldn't you be down at the banana bar at your time of night?
<jtv> the what now?
<jtv> I've got a watermelonâ¦ does that count?
 * bigjools 's eyes water
<jtv> Bit out of my daylight cycle; had a lat-night call from the West Pole.
<jcsackett> hm. it would seem none of the hardwaredb unittests are hooked up to test discovery...
<jtv> jml: victory!  Lowered the ServerAliveInterval to 2 and now I'm failing with a regular merge conflict.
<jml> jcsackett: blink.
<bigjools> I've come across tests that were not in test discovery before, can't remember why though
<jcsackett> jml: not as dire as i thought, but there are tests in lib/lp/hardwaredb/scripts/tests that i cannot seem to fire off.
<jcsackett> perhaps they're hooked in some funny way so bin/test can't do its name matching magic.
<jml> jcsackett: could you please file a bug about this?
<jcsackett> jml: i fully intend, just as soon as i have some more information.
<jml> jcsackett: thanks.
<jcsackett> s/i fully intend/i fully intend to/
<jcsackett> and then i'll have to close that bug to close the one i'm working on. :-P
<jml> dig.
<jtv> This had me forgetting what day it was for a momentâ¦ http://chromeadblock.com/freedom/
<jtv> OMG my "ec2 land" detached successfully.  I'm keeping protocol keepalives in my ssh config.
<jcsackett> jml: so, just moving the tests from script/tests into the regular tests folder makes them discoverable, which works as a work around--should bin/test be able to find any tests in a "tests" folder?
<jcsackett> (regular here being "hardwaredb/tests")
<LPCIBot> Project windmill build #131: STILL FAILING in 1 hr 11 min: https://lpci.wedontsleep.org/job/windmill/131/
<bigjools> night everyone, have a nice weekend
<bac> benji: can you fix the conflicts in your branch and repush?
<benji> bac: sure
<bac> benji: "Ititilize"?
<bac> i'm not sure what that is meant to say
<benji> heh; "Initialize" probably
<bac> ah, that makes sense.  i was going for 'utilize'
<benji> bac: conflict fixed
<bac> benji: in add_recipient_picker you reverted to the old style string catenation method.  why is that?
<benji> bac: looking
<benji> bac: there is some string concatination, but it's for string constants, which I believe is kosher
<bac> benji: ok.  i just hope it isn't a vector for future mods to revert us to the bad way
<benji> yeah, I've given some thought as to how to validate or at least lint against bad string injection and I haven't had any great ideas; maybe something to research (I think Google has a JS to JS compiler somewhat along those lines)
<benji> gary_poster: Feature Work 2 is empty! (other than the over-arching card)
<gary_poster> benji, yay!  I'll close it down!
<gary_poster> bac, benji, yay & congrats!
<benji> gary_poster: I'm trying to find the next card I should start, but I've lost it.  Any hint as to where it went?
<bac> \0/
<gary_poster> benji, "Link bug target subscription edit page from target's main page and bug pillar main page." I think?
<gary_poster> bottom left of FW 1
<deryck> have a nice weekend, everyone!
<jcsackett> anyone fixing that merge error? and if not, can someone sanity check this fix: http://paste.ubuntu.com/588384/
 * sinzui looks
<sinzui> jcsackett: that looks good to land
 * jcsackett goes to land it.
<jcsackett> thanks.
* bac changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<sinzui> jcsackett: looks like answers were ignored this week :(
<sinzui> jcsackett: do we have any power to delete bug spam yet
<jcsackett> sinzui: yeah, we can kill bug spam, but i am still hunting down the hiding-comments script, so we have to do it the hard way.
<jcsackett> i can tell you how, or you can assign the answers to me and i'll go whack 'em.
<sinzui> jcsackett: I asked a losa for the script.
<jcsackett> sinzui: i emailed him about that this morning, since i think they are getting a tad hammered. have not yet heard back. which confirms some of the hammered.
<mbarnett> jcsackett: i saw your email earlier today but have been running about all day.  I'll shoot it over to sinzui momentarily
<mbarnett> sinzui: this staging lists thing is also going to be a bigger project than just flipping a bit in the vhost.  The relevant tools don't appear to even be installed, so this is going to take some doing to get fixed up.
<jcsackett> mbarnett: figured you were just really busy. :-)
<sinzui> mbarnett: noted. I will rethink this issue. either we convince ISD to restore the feature, or we introduce a hack that generates apache handler rules. I think we may want to remove the feature temporarily next week so that we can test. I will be in touch with the losa next week
<LPCIBot> Project windmill build #132: STILL FAILING in 1 hr 11 min: https://lpci.wedontsleep.org/job/windmill/132/
<mbarnett> sinzui: okay.  i'll revert the current changes.  Sorry it wasn't a quick fix.  Is there anything else (barring the entire archive!) that woudl help you qa right now?
<sinzui> mbarnett: the script I provided does not use the web, so we are not blocked form running it next week, we can choose to make the staging list server use the inline team setup (add admin and launchpad) so we can test. Actually that might be the correct hack until we have a long term fix
<mbarnett> sinzui: jcsackett: devpad:~mbarnett/tmp/hide-comments.py
<jcsackett> mbarnett: you are my new best friend.
<jcsackett> so it doesn't live in a branch anywhere?
<mbarnett> jcsackett: it used to live on asuka, but with some SSO changes, we have to now run it locally
<mbarnett> jcsackett: so... not really no.
<mbarnett> jcsackett: if you send me a patch, i'd check it back in on asuke
<mbarnett> asuka, so we have a current copy in the place it used to run from.
<mbarnett> that is the closest thing to a home the thing has.
<jcsackett> mbarnett: dig.
<jcsackett> mbarnett: i will send the patch to you as soon as it's ready.
<mbarnett> many thankses
<sinzui> jcsackett: I am hacking on the script now. it does not immediately work on my machine. It assumed old Lp and launchpadlib
<sinzui> My setup is current since I use lplib almost every day
<jcsackett> sinzui: on the spam front; we should have the power to kill question spam ourselves in the next day or two; branches all actually landed.
<sinzui> I saw
<jcsackett> *such* bad luck landing those things.
<sinzui> I have been distracted planing mailing list archive qa
 * jcsackett nods.
<jcsackett> that would be distracting.
<sinzui> jcsackett: I now have a working version\o/
<sinzui> I do not recall authorising it though
<sinzui> jcsackett: here is the version I used to hide the bug comments mentioned in the open questions: http://pastebin.ubuntu.com/588419/
<jcsackett> sinzui: i was just disconnected from my irc bouncer; i was alerted you messaged me, but didn't get the message.
<sinzui> jcsackett: here is the version I used to hide the bug comments mentioned in the open questions: http://pastebin.ubuntu.com/588419/
<sinzui> jcsackett: I will update the maintenance page on dev Monday. I on on Maintenance for the day (you being off), so I am sure to remember to do it
<jcsackett> sinzui: dig. i'll use your updated comment to add the question stuff too.
<LPCIBot> Project windmill build #133: STILL FAILING in 47 min: https://lpci.wedontsleep.org/job/windmill/133/
<jcsackett> sinzui: we're at EOD. did you want to do the mumbly standup thing?
<sinzui> jcsackett: No, we chatted. we both struggled to get things done
<jcsackett> sinzui: works for me. :-)
<jcsackett> have a good weekend!
#launchpad-dev 2011-04-02
<LPCIBot> Project windmill build #134: STILL FAILING in 1 hr 10 min: https://lpci.wedontsleep.org/job/windmill/134/
<LPCIBot> Project windmill build #135: STILL FAILING in 1 hr 10 min: https://lpci.wedontsleep.org/job/windmill/135/
<LPCIBot> Project devel build #604: FAILURE in 4 hr 59 min: https://lpci.wedontsleep.org/job/devel/604/
<LPCIBot> Project windmill build #136: STILL FAILING in 56 min: https://lpci.wedontsleep.org/job/windmill/136/
#launchpad-dev 2011-04-03
<wgrant> I think LP now feels only very slightly slow.
<lifeless> its getting there
<lifeless> still has that mainframe feel
<wgrant> Yeah. But bug pages and listings load in just a couple of seconds now.
<wgrant> Three years ago they freqently took >10.
<lifeless> we're making progress :)
<lifeless> +bug-index is going to be much happier on thursday
<wgrant> Yeah.
<lifeless> I think we'll need to do constraint based batch offsets soon
<lifeless> sadly storm doesn't seem to have editable queries
<lifeless> and given the contribution dynamic there, I don't fancy trying to do it upstream
<sidnei> lifeless, :D
<wgrant> What's the issue with contributions?
<lifeless> wgrant: perfect is the enemy of the good, or thats how it feels to me so far
<lifeless> wgrant: I think I've put up 4 patches and had none get in
<wgrant> Hmm.
<wgrant> We should fix that.
<lifeless> wgrant: 2 got rewritten, one got bounced [no tests, which is fair enough, though finding tests in the storm layout is idiosyncratic]
<lifeless> and one is stalled (the with patch)
<lifeless> sidnei: hai :)
<lifeless> wgrant: https://code.launchpad.net/storm/+activereviews
<sidnei> lifeless, maybe you'll get a suggestion to go look at sqlalchemy with your next patch
<lifeless> wgrant: its a lot healthier in absolute terms than LP; but in terms of reviews-per-dev, or reviews-per-loc, I think LP would measure better
<lifeless> sidnei: hah!
<lifeless> sidnei: so let me tell you about this idea
<lifeless> sidnei: you know we have big collections in LP
<lifeless> like ubuntu - 400K bugs
<lifeless> sidnei: and a batch navigator that lets users page through the collection
<sidnei> lifeless, im having dinner, but go on, i will read l8r
<lifeless> sidnei: this breaks down spectacularly when the user is paging through (say) page 30
<lifeless> or earlier (depends on the cost of calculating the batch)
<lifeless> because the way we implement it is to say LIMIT [batch size] OFFSET [page number * batch size]
<lifeless> LIMIT + constraint is a *much* better way to do it [plus reversing the sort order when walking backwards]
<lifeless> e.g when we show the first page [or the last], we remember the sort order key of the last row.
<lifeless> and rather than saying OFFSET [1 * batch size] to go to the next page, we say And(order_column1 > lastrow1, order_column2 > lastrow2, ...) LIMIT [batch size]
<lifeless> to walk back, flip the sort order; to skip a page use an OFFSET as well as this technique. To go to the end, flip the sortorder take the first page of results.
<lifeless> In LP we pass ResultSet objects to BatchNavigator, which then calculates # of pages, does the slicing etc.
<lifeless> so, we need some way for the batchnavigator to:
<lifeless>  - ask for the needed last-row-tuple
<lifeless>  - pass in a last-row-tuple and say 'after this row please'
<lifeless> just wrapping the query (using it as an inner query) is pretty much guaranteed to give poor query performance.
<lifeless> (and by pretty much, I mean 'its not even worth trying')
<lifeless> challenges I see are that storm doesn't understand the contents of its queries.
<lifeless> so we're likely to get redundant constraints and so on - which pgsql can optimise out, but that often results in odd plans even so
<lifeless> we may be best off redefining the contract
<lifeless> pass BN a factory to create a page result set as well as a resultset for 'entire collection'
<lifeless> wgrant: still around ?
<wgrant> lifeless: Mostly.
<lifeless> wgrant: where did you and wallyworld get to fixing that undefined thing
<wgrant> lifeless: It's on qas.
<wgrant> r12723
<lifeless> cool, so we can deploy?
<wgrant> Once we're QA'd past it, yeah.
<wgrant> That's blocking on fixing lists.staging though, which I might try to get done tomorrow morning.
<lifeless> staging or qastaging ?
<wgrant> Either.
<wgrant> But qastaging doesn't have one at all, so it's going to be more work.
<lifeless> ah
<lifeless> whatever you learn.... can you rt that towards a qastaging one in future ?
<wgrant> Sure.
<lifeless> thanks
<lifeless> 275 timeouts
<lifeless> hmm
<jelmer> moinmoin
<wgrant> Morning jelmer.
<wgrant> lifeless: Not quite below 500 critical OOPSes yet :(
<lifeless> wgrant: indeed
<lifeless> 1918HWDB1 will wipe out 150
<wgrant> Yeah.
<lifeless> and fixing OOPS-1918SMP10 will likely fix real issues (e.g. stale locks)
<StevenK> SMP? What is that prefix from?
<wgrant> Supermirror Puller
<wgrant> Not to be confused with SMPJ, which is the staging merge proposal job runner.
<lifeless> OOPS-1918CIW12 seems suspect too
<lifeless> OOPS-1918A602...
<lifeless> odule zope.publisher.http, line 73, in sane_environment
<lifeless>     dict['PATH_INFO'] = dict['PATH_INFO'].decode('utf-8')
<lifeless> wtf
<lifeless> another 20 a day there trivially fixed (its garbage; give a 400 bad request)
<lifeless> or a 404
<StevenK> Which could be bad bookmarks or something of that ilk?
<lifeless> could be
<lifeless> though how you get \x0a into a bookmark, I dunno
<wgrant> Not spammers?
<StevenK> It could be ...
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #605: FIXED in 5 hr 23 min: https://lpci.wedontsleep.org/job/devel/605/
<LPCIBot> Project windmill build #137: STILL FAILING in 1 hr 11 min: https://lpci.wedontsleep.org/job/windmill/137/
<lifeless> is json or simplejson the right one to use
<wgrant> It depends. json is the version of simplejson in the standard library, but simplejson is better at some things.
<lifeless> json will do then
<lifeless> wgrant: I've just pushed a better schema for oops-repository
<lifeless> gnight
<thumper> morning
<lifeless> moin
<lifeless> anyone know how we land changes to lazr.batchnavigator ?
<lifeless> hmm
<lifeless> I think we've broken subscribing to mailing lists via indirect team membership
<lifeless> false alarm, but horribly confusing ui
<wgrant> Confusing UI in LP? Surely not!
<lifeless> indeed
<lifeless> not to mention buildbout breaking for me again
<lifeless> :(
<lifeless> bug 749753
<_mup_> Bug #749753: Confusing missing 'subscribe to mailing list' on disabled list with indirect membership <confusing-ui> <easy-ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/749753 >
<wgrant> lifeless: buildbot, or buildout?
<lifeless> out
<lifeless> bug 749739
<_mup_> Bug #749739: fails to bootstrap on lucid <lazr.batchnavigator:Triaged> < https://launchpad.net/bugs/749739 >
<wgrant> Erm, were you running it in a virtualenv?
<wgrant> That eperm suggests not.
<lifeless> no
<lifeless> why would I ?
<lifeless> I have a VM for development for a reason
<lifeless> if it won't bootstrap in lucid, building debs would be hiiiiiilarious
<lifeless> wgrant: ^
<wgrant> Hmm.
<lifeless> separately
<lifeless> in just over two hours we're going to merge db-stab;e
<lifeless> Revision 10382 can not be deployed: needstesting - Adds a method via the API to hide question comments and 10383 are unqaed
<lifeless> I suspect both can be landed directly on devel after the merge, if they aren't qaed before
<wgrant> That is also my suspicion.
<wgrant> But let's see.
<lifeless> wgrant: I'm going to be pretty terrible today - was @ 24 hour medical centre till 3am last night w/Lynne
<wgrant> Ugh.
<wgrant> No chance in hell of QAing 10383.
<wgrant> Fortunately there's nothing useful behind it.
<wgrant> lifeless: Ugh, that's not ideal. Everything OK?
<lifeless> wgrant: yeah, something triggered asthma (she has no history of it), and with pregnancy you take everything seriously
<wgrant> Yup.
 * thumper is sending away for urgent passport
<thumper> bad planning on my part
#launchpad-dev 2012-03-26
<wgrant> StevenK: Indeed. I wonder if the removeSecurityProxy is really required, then.
<StevenK> wgrant: I think without it you get Unauthorized
<wgrant> StevenK: I'd like that rechecked -- if it's the case, please try to work out why.
<wgrant> That shouldn't be happening.
<StevenK> wgrant: http://pastebin.ubuntu.com/899838/
<wgrant> StevenK: I speak of lib/lp/bugs/tests/test_bug_mirror_access_triggers.py
<wgrant> StevenK: There was no removeSecurityProxy, now there is.
<StevenK> wgrant: Right. I think the rSP() is just so we don't have to make sure we have lp.Edit on the bug by faffing around with with {person,celebrity}_logged_in.
<wgrant> StevenK: Try removing it.
<wgrant> From that test.
<wgrant> Since it wasn't there before.
<StevenK> wgrant: Hmmm, right, so, I'm wrong.
<StevenK> wgrant: I guess I added the rSP thinking I was avoiding failures.
<lifeless> is there a test helper to run a job ?
<StevenK> "Personal receieved 10590 new messages."
 * StevenK stabs the messaging indicator for lying.
<StevenK> Machine id 525? When did that happen?
<StevenK> wgrant: You can kill AMI in 522
<wgrant> StevenK: Is gone.
<StevenK> Now to pester bac
<lifeless> hmm
<lifeless> what deletes executed jobs?
<StevenK> lifeless: So why are you looking at the job system when abentley is rewriting it?
<lifeless> I'm not
<lifeless> I'm making sure what I write works with the stuff in play today
<lifeless> StevenK: also, AIUI the job specific stuff in LP is staying there, so the new system will be DB backed. There may be little changes on that stuff.
<mwhudson> why might launchpad-dependencies not be installable on oneiric currently?
<StevenK> postgres 9.1 at a guess
<StevenK> I've not investigated yet
<wgrant> What does apt say?
<nigelb> Mornin.
<mwhudson> i think it might be pg 9.1 too
 * mwhudson tries to remember how to drive apt
<mwhudson> apt-get upgrade keeps things back if installing them is only possible by removing packages?
<lifeless> and the answer is, nothing
<lifeless> we have 1.7M jobs in our history
<lifeless> 350K answers jobs created and not pruned
<wgrant> mwhudson: That's the main cause, yes.
<wgrant> mwhudson: Try explicitly installing.
<wgrant> lifeless: Time: 0.72 seconds
<mwhudson> The following packages will be REMOVED:
<mwhudson>   postgresql-8.4-slony1 slony1-bin
<wgrant> lifeless: For a launchpad/+sharing batch on prod.
<lifeless> wgrant: congrats
<wgrant> So we can do sub-second.
<wgrant> Just.
<nigelb> Has +sharing landed? behind a pref?
<wgrant> nigelb: We'll hopefully enable it for beta testers early this week.
<wgrant> It's only on for ~launchpad on prod so far.
<nigelb> Aww, ok
<nigelb> I wish I could see on qastaging
<wgrant> Everyone can see it on qastaging, but only if you're a driver/owner of a project with something shared.
<StevenK> We can probably fiddle it on qas for a bit so nigelb can have a look.
<wgrant> It's been on for everyone on qas for weeks :)
<nigelb> Oh cool.
<nigelb> I'm seeing it.
<mwhudson> oh
<mwhudson> https://code.djangoproject.com/ticket/17062
<mwhudson> no
<mwhudson> The following NEW packages will be installed:
<mwhudson>   launchpad-database-dependencies-8.4 postgresql-8.4-slony1-2 slony1-2-bin
<mwhudson> so this is a package name change, basically?
<StevenK> mwhudson: Yup
<wgrant> mwhudson: New version of slony.
<nigelb> This is cool. Except hrm, probably slightly confusing.
<wgrant> And the pg version added to the package name.
<wgrant> nigelb: Oh?
<mwhudson> cool
 * mwhudson hits Y
<nigelb> wgrant: Even after reading the help text, it "feels" complicated. Maybe if we have enough explanation when we launch, it should be good.
<wgrant> nigelb: The UI is RO on production at the moment, and will be for at least a few weeks.
<wgrant> It'll hopefully become clearer as we rework that UI and integrate the new terminology throughout the site.
<nigelb> wgrant: Oh, that's cool then.
<StevenK> nigelb: Just wait until I change the bug UI
<lifeless> and thus bug 964935
<_mup_> Bug #964935: answerpersontransfer/etc jobs are accumulating without bound <jobs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/964935 >
<nigelb> StevenK: woah.
<StevenK> But that's like two weeks away. I'm dealing with the model and surrounding changes.
 * nigelb looks forward to the changes.
<StevenK> lifeless: O HAI
 * nigelb pins bigjools to IRC.
<nigelb> STICK!
 * StevenK hands nigelb a staple gun.
<bigjools> I disappear once for 10 mins and you want to pin me?
<bigjools> good lord!
<StevenK> bigjools: Obviously, nigelb missed you a lot.
<nigelb> heh
<wgrant> StevenK, wallyworld_: https://code.launchpad.net/~wgrant/launchpad/bug-962912/+merge/99252 (fixes the duplicate query on +sharing)
<StevenK> wgrant: Why are you removing lib/lp/services/database/tests/test_decoratedresultset.py ?
<wgrant> "    Drop test_decoratedresultset. It's just a wrapper around a doctest that's already run by test_doc.
<wgrant> "
<wallyworld_> wgrant: great. i'd also update the accesspolicy tests to decrement the query count by one
<bigjools> I feel loved
<wallyworld_> bigjools: fuck off
<wgrant> wallyworld_: This is why I generally use Equals for my query count tests.
<bigjools> I knew I could count on wallyworld_
<wgrant> wallyworld_: Otherwise someone else will halve the page's query count, and then the other optimisations can regress without anyone noticing :/
<wallyworld_> wgrant: doesn't work here. the product view uses one or 2 more queries
<wallyworld_> so less than is better
<wgrant> Ah, probably the session thing.
<wallyworld_> the commercial subscription thing
<wgrant> The first request in a new process will issue an extra query to get the session secret.
<wgrant> Ah
<wgrant> That suggests you're not invalidating caches properly.
 * StevenK stabs buildbot again.
<wgrant> But anyway.
<wallyworld_> wgrant: no, the product view does indeed use one more query
<wallyworld_> i use a helper method for both cases
<wgrant> Oh, right.
<wallyworld_> i guess i could have passed in the expected count
<wallyworld_> but i like lessthan because it's more robust
<wallyworld_> but does need updating if we reduce the query count, that's true
<wgrant> wallyworld_: I wonder if we should inhibit the rough_length EXPLAIN, since we're not showing total size in the UI at all.
<wgrant> Did you look at that?
<wgrant> Also, can one of you please approve my MP? :)
<StevenK> I thought wallyworld_ was reviewing.
<wallyworld_> StevenK: i will but have to pick up the kid from school first
<StevenK> Whee, memcache._ConnectionDeadError on ec2 too
<wgrant> StevenK: Yeah, saw that twice last week.
<wgrant> See my email to launchpad-dev.
<wgrant> I might revert the python-memcached upgrade.
<wgrant> I'm actually not sure if that alone is enough to fail buildbot.
<wgrant> I believe it is for ec2, but I don't think I've ever seen buildbot fail just on that.
<wgrant> So it's possible it's happening more often than we see.
<StevenK> wgrant: See the last db-devel failure -- it looked like just it failing.
<wgrant> StevenK: There was a rabbitmq failure up the top IIRC
<wgrant> Yeah
<wgrant>   File "/srv/buildbot/slaves/launchpad/lucid-db-devel/build/orig_sourcecode/eggs/rabbitfixture-0.3.2-py2.6.egg/rabbitfixture/server.py", line 260, in check_running
<wgrant>     raise Exception("RabbitMQ server is not running.")
<wgrant> Exception: RabbitMQ server is not running.
<StevenK> Ah
<StevenK> Bah, testr needs a way to pass options to the test command it runs.
<wallyworld_> wgrant: the doctest change doesn't really appear to test bot result sets, or?
<wgrant> eparse
<wgrant> wallyworld_: ^^
<StevenK> I think he means 'both'
<wallyworld_> wgrant: the doc test have been changed but it doesn't appear to test any of the next stuff
<wgrant> wallyworld_: The doctest tests DecoratedResultSet
<wgrant> Not StormRangeFactory.
<wallyworld_> sure, but we are changing rs.config and the doctest invokes that
<wallyworld_> so it should be tested
<wallyworld_> in the doc test
<wgrant> Confused
<wgrant> What's not tested?
<wallyworld_> calling config with return_both = true causes both rs to be returned
<wgrant> And return_both = False is the default.
<wgrant> So it's tested throughout the rest of the file.
<wgrant> It seems not particularly valuable to test that explicitly, but I guess I could.;
<wallyworld_> but the doc test explicitly makes a config call with return_both=true and then doesn't test anything related to that
<wallyworld_> it tests the same thing as if return_both = false
<wgrant> Huh?
<wallyworld_> so it's not really testing the new expected behaviour
<wgrant> Look at the results of the iteration.
<wgrant> It's returning both.
<wgrant> (.config doesn't copy -- it changes in place and returns self)
<wallyworld_> ah, right, i think i get it. it's hard to tell from a few lines of diff
<wgrant> Indeed. You can see what the decorated output is like in the context at the top of that hunk.
<wgrant> The decorator returns "Dist name is: %s" % result.name.
<wallyworld_> right. clear now.
<wgrant> wallyworld_: Thanks.
<wallyworld_> np. sorry for confusion
<wgrant> np
<wallyworld_> wonder how much benefit it will have
<wgrant> 600ms for Ubuntu
<wgrant> 20ms for everyone else.
<wallyworld_> what was the time for ubuntu before mod?
<wgrant> 1.2s
<wgrant> Maybe 1.3s
<wallyworld_> we it's now 1/2
<wgrant> It's roughly halving it, anyway.
<wallyworld_> yeah, cool
<wallyworld_> might make it to qas before tomorrow if bb plays nice
<StevenK> bb is currently running
<wallyworld_> for how long before erroneous error
<StevenK> Do not say that.
<wgrant> Does anybody have a good reason for me to not revert the python-memcached upgrade?
<StevenK> wgrant: Doeet
<StevenK> (As in, do the revert.)
<lifeless> StevenK: hi
<lifeless> wallyworld_: you can use Or(Equals(1), Equals(2))
<lifeless> wallyworld_: or you could write a simple Between(1,2)
<lifeless> wgrant: ^
<wallyworld_> lifeless: ECONTEXT
<lifeless> wallyworld_: LessThan vs Equals
<lifeless> wallyworld_: you're not limited to simple comparators
<StevenK> lifeless: Can you set launchpad-security as the bug supervisor and security contact on the auditor project?
<lifeless> sure. you can't ?
<wallyworld_> lifeless: handy. good to know
<lifeless> StevenK: also have set the answers option to lp
<wallyworld_> lifeless: pretty happy to write a test with query count < 6 where some of those are updating session data etc
<StevenK> lifeless: I can not, no, since I'm not an admin of -security.
<wgrant> wallyworld_: That's only because the caches are not invalidated.
<wgrant> wallyworld_: AFAICS
<lifeless> wallyworld_: I've no objection, I was merely noting an oversight in your 'cannot use Equals' conversation
<wallyworld_> sure, np
<wallyworld_> lifeless: so you had a car accident yet with the new road rules?
<lifeless> nope
<wallyworld_> i bet a few people do though
<lifeless> perhaps
<lifeless> should have less in the long run or thats the theory
<wallyworld_> well, it's only tourists i guess who would have been caught out
<wgrant> """Backport of Python 2.6 logging handlers.
<wgrant> Hmmm
<wgrant> I wonder if that can be dropped now.
<lifeless> StevenK: (thats done)
<StevenK> lifeless: Thanks.
<adeuring> good morning
<jml> hello
<jml> I updated my MP over the weekend: https://code.launchpad.net/~jml/lazr.restfulclient/multiple-instance-safe/+merge/98873
<jml> wgrant: IIRC, it can.
<wgrant> jml: It's in EC2 now, so we'll see.
<jml> loggerhead :(
<wgrant> jml: What about it?
<jml> it was 503ing before
<wgrant> jml: crowberry's Apache has been collapsing a bit the last couple of hours.
<wgrant> loggerhead is not the problem here, for once.
<jml> hurrah
<wgrant> FSVO
* benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: benji | Firefighting: - | Critical bugtasks: 4*10
<wgrant> jelmer: Heh, thanks, was about to do that myself.
<wgrant> It's caused us some problems before.
<jelmer> wgrant: :)
<jelmer> wgrant: the buildbot plugin for bzr is also using a few API calls that cause unnecessary traffic
<jelmer> (and slowness)
<wgrant> jelmer: See carob:~wgrant/shadowrobot for logs
<jelmer> thanks
<deryck> Morning, all.
<adeuring> abentley: could you have a look at this MP: https://code.launchpad.net/~adeuring/lazr.jobrunner/early-reference-to-job.fail-test/+merge/99312 ?
<abentley> adeuring: sure.
<adeuring> thanks
<abentley> adeuring: r=me.
<adeuring> abentley: thanks
<dobey> hey all; getting https://lp-oops.canonical.com/?oopsid=ae5111a6e4ae33b1e2c60a6dc72164e7 when trying to search for a person/team in the bug assignee dialog
<sinzui> wallyworld_, go to sleep
<jcsackett> sinzui: wallyworld_ never sleeps.
<czajkowski> neither does lifeless wgrant or sinzui or StevenK
<czajkowski> :)
<jcsackett> czajkowski: putting aside lifeless, that's everyone else on my team.
<jcsackett> i'm the only one that sleeps...no wonder i question if our team has a full ration of sanity. :-P
<czajkowski> hah
<sinzui> jcsackett, I can lp-land branches for people; they do not need to stay up waiting for the review
<sinzui> benji, do you have time to review https://code.launchpad.net/~sinzui/launchpad/front-page-layout/+merge/99121 ? It has pictures
<jcsackett> sinzui: I concur; certainly there's no need for wallyworld_ to stay up. But he always seems to. :-P
<benji> sinzui: well, if it has pictures how can I refuse?
<wallyworld_> sinzui: can you have a quick look at something for me
<wallyworld_> lp:sharing-picker-title-959417
<sinzui> wallyworld_, yes
<wallyworld_> in _display_step_two(data) the click callback refuses to use the latest value of data passed to the function
<wallyworld_> so there's a scoping issue i can't quite solve
<sinzui> oh dear
<wallyworld_> it always uses the value passed in when the function is first invoked
 * sinzui nods
<wallyworld_> it's for bug 965197
<_mup_> Bug #965197: sharee picker always uses first selected person <disclosure> <ui> <Launchpad itself:In Progress by wallyworld> < https://launchpad.net/bugs/965197 >
<wallyworld_> the click callback is the delegated one for .next
<wallyworld_> thanks in advance
 * wallyworld_ off to bed
 * wallyworld_ doesn't like javascript today
<sinzui> czajkowski, the the notes on the whiteboard were generated by Lp when the user set and OTHER license. The i don't know are a problem because they are not open source. We would disable them after 6 months if there was an automated way to do it
<sinzui> czajkowski, some of the messages you are seeing shows that the user was trying other opensource, then read the rules in the email and selected a different type of license
<sinzui> czajkowski, so no human left a not on the board...we leave out names.
<czajkowski> sinzui: ahh right re the mail I sent
<czajkowski> sinzui: well assumed it's you as you do a lot of the liceince reviews.
<sinzui> czajkowski, When you see such a note and a proper license, the user understood the instruction in the email and fixed the license
<sinzui> I only review other/open source and proprietary
<czajkowski> ok
<czajkowski> cheers
<sinzui> most I don't knows will fail on their own. they are not worth anyone's time to fix the license. We only care about them also being used for spam
<benji> ok, sinzui: I've looked at the pretty pictures.
<sinzui> thank you benji
<benji> any time
<salgado> benji, hi there.  if you have some time today, I could do with a review on https://code.launchpad.net/~linaro-infrastructure/launchpad/team-engineering-view/+merge/99342 :)
<benji> salgado: sure
<rick_h> deryck: got a sec to do a quick pre-impl?
<deryck> rick_h, on call now, ping when free
<rick_h> deryck: k
<abentley> adeuring: I've pushed the updates to download-cache.  Sorry for the delay.
<deryck> rick_h, I can chat now.  let me start a hang out.
<deryck> rick_h, https://plus.google.com/hangouts/extras/talk.google.com/go-for-deryck
<czajkowski> deryck: love the url!
<deryck> czajkowski, :)
<czajkowski> deryck: thanks for covering tomorrow as well
<deryck> czajkowski, yeah, no problem at all.
<cr3> hi folks, how can I request that a project (and all its bugs) become private?
<czajkowski> cr3: what is the project?
<cr3> czajkowski: hw-labs
<czajkowski> cr3: made bugs private
<czajkowski> as for making the project private hmm
<czajkowski> sinzui: is it possible to make the project private I dont seem to be able to do that, or is making the bugs private enough
<cr3> czajkowski: yeah, we noticed that checkbox after asking the question but we'll see if the project as a whole could also be marked as private
<czajkowski> I suspect it's something webops will have to do
<sinzui> czajkowski, there is no such thing as a private project
<czajkowski> hmm
<czajkowski> cr3: ^^
<sinzui> cr3, as the maintainer of a project with a commercial subscription, you can configure the bug tracker to have private-by-default bugs. all new bugs will be private
<sinzui> cr3, I can give /checkbox a commercial subscription, then you can enable private bugs
<cr3> sinzui: this is for the /hw-labs project though, not /checkbox. however, we'd like the project as a whole private if possible
<sinzui> cr3, as I said, no such thing
<sinzui> I have been working the private projects for 22 months. and we are still 6 months away from saying a project can be private
<cr3> sinzui: ah, misssed that part. thanks!
<cr3> czajkowski: I guess we're all good with private bugs then, thanks!
<sinzui> cr3, the project already has a commercial subscription...the maintain can enable private bugs
<cr3> sinzui: yeah, czajkowski helped with that. I was under the impression that the project as a whole could be private though, all clear now.
<sinzui> okay
<czajkowski> sinzui: cheers
<abentley> deryck: Would you like to review a very short lazr.jobrunner branch? https://code.launchpad.net/~abentley/lazr.jobrunner/fix-celeryd-tests/+merge/99378
<deryck> abentley, sure.
<deryck> abentley, easy peasy that one. :) r=me.
<abentley> deryck: Thanks.
<deryck> np!
<salgado> benji, would you be able to land that branch for me?  I think I remember last time your VM was not setup to run ec2-land or something so I thought I'd check
<benji> salgado: sure (I've gotten it straightened out since then)
<salgado> oh, cool. thanks!
<adeuring> abentley: thanks (no problem with the delay)
<abentley> adeuring: I've also landed a fix for those test failures.
<adeuring> abentley: cool
<lifeless> salgado: hi
<salgado> hi lifeless
<lifeless> salgado: I've just commented on your MP, I thought I'd ping you here to make sure you understand the comment ;)
<lifeless> benji: you may want to see that comment too - its a mis-pattern to watch for with storm use
 * benji looks.
<lifeless> this is the culprit I'm referring to:
<lifeless> +        results = store.using(*origin).find(
<lifeless> +            (WorkItem, Milestone, Specification, Person, Product,
<lifeless> +             Distribution),
<lifeless> a milestone with 10 workitems, each with different assignees, and the spec with an 11th assignee, will return 20 rows
<lifeless> and you'd parse and update storm metadata for the same distro or product 20 times, for the spec assignee 10 times, for the spec 20 times, for the milestone 20 times, for the workitem twice per workitem.
<lifeless> if the workitems had the same person a lot, which is much more common, the redundant work level will increase
<lifeless> benji: salgado: do you see why this happens?
<benji> lifeless: yep; it's not pretty
<benji> salgado: I'm going to kill the landing, if you don't object
<salgado> lifeless, I don't understand why in that scenario it'd return 20 rows
<lifeless> because of the coalesce on person
<lifeless> salgado: even without that its still an exmaple of the anti pattenr
<lifeless> salgado: DecoratedResultSet is designed for this - the pre_iter_hook can bulk load all the related objects, at one query per *type*, the collection is still slicable and still looks like it returns just Workitems, or whatever.
<lifeless> salgado: the issue is that sql flattens a graph to a table, which leads to redundancy, the more types you include in the result, the higher the waste
<lifeless> if it was nearly free to parse and assemble storm objects, it wouldn't matter, but its not.
<salgado> lifeless, ok, I can use a pre_iter_hook, but I'm curious as to how the coalesce on person does that?
<lifeless> the coalesce increases, its not the only cause
<lifeless> it says 'make a row for person where milestone.foo or spec.foo'
<lifeless> bah workitem
<salgado> I understood it's not the only cause
<lifeless> if the workitem person and the spec person are different, you'll get two rows, one with the workitem person and one with the spec person
<lifeless> all the other fields will be identical
<lifeless> brb
<lifeless> actually, I'm wrong about the coalesce
<lifeless> the rest I'm right on :)
<lifeless> thje coalesce will pick just one. Fuzzy morning thinking.
<salgado> ok, I understand what the issue is with storm, but the coalesce didn't make sense so I thought I was missing something
<lifeless> yeah, you're spot on :)
<lifeless> I was nutty there
<lifeless> benji: so yes, stopping the landing would be good; or salgado could do a follow up branch - I'm happy either way.
<salgado> just kill it and I'll fix it on this same branch
<benji> 'E's expired and gone to meet 'is maker!
<abentley> deryck: The cross-format stacked branches bug #828409 won't be fixed until the RT has been performed.
<_mup_> Bug #828409: cross-format stacked branches can't be accessed <branch-stacking> <codehosting> <Launchpad itself:In Progress by abentley> < https://launchpad.net/bugs/828409 >
<abentley> deryck: We're not fixing the bug by changing code, we're fixing it by ensuring there are no cross-format stacked branches.
<lifeless> flacoste: I can talk anytime that suites
<lifeless> s/e//
<rick_h> 984758
<deryck> abentley, ok, cool.  So I'll move the card all the way to done done for us.  since we're done coding....
<deryck> abentley, and do you mind commenting that the bug will be fixed with RT #XXX in the bug itself?
<abentley> deryck: sure.
<deryck> abentley, thanks!
<lifeless> flacoste: inviting you back in
<lifeless> flacoste: modem crashed AFAICT
<poolie> hi all
<salgado> lifeless, does http://paste.ubuntu.com/901105/ look ok to you?
<salgado> hi poolie!
<lifeless> morning poolie
<lifeless> salgado: nice
<lifeless> salgado: I think you can change your origin list now
<lifeless> salgado: as you're not constraining by Person etc
<lifeless> salgado: IMBW
<salgado> lifeless, I am: Person.id.is_in(self.participant_ids)
<lifeless> salgado: uhm, where is that MP again, so I can get more context?
<salgado> I can get rid of Product/Distribution from there, though
<salgado> lifeless, https://code.launchpad.net/~linaro-infrastructure/launchpad/team-engineering-view/+merge/99342
<salgado> now my origin joins only Specification, Person and Milestone
<lifeless> salgado: this is in getAssignedSpecificationWorkItemsDueBefore ?
<salgado> yes
<lifeless> yeah, you don't need Person :)
<lifeless> Or(WorkItem.assignee_id.is_in(self.participant_ids), Specification.assigneeID.is_in(self.participant_ids))
<lifeless> would be an exact translation
<salgado> oh, indeed
<lifeless> it may be that you want just WorkItem.assignee_id.is_in(self.participant_ids) though
<salgado> no, that's not what I want
<lifeless> [I wouldn't know :P]
<lifeless> so, milestone workitem and specification are the tables you need
<salgado> yep, and those are all I have in origin now :)
<lifeless> cool
<lifeless> you can inner join to workitem
<lifeless> because you are returning the workitems, a NULL workitem won't be useful for you
<salgado> lifeless, how can that return a NULL workitem, though, if that's the only thing I'm selecting on?
<lifeless> you're selecting on milestone & workitem|spec person
<lifeless> ah, I may have misread, one sec
<lifeless> you have workitem join spec, coalesce join milestone
<lifeless> so its all inner
<lifeless> cool, ignore me :)
 * lifeless puts that on hotkey
<salgado> haha
<lifeless> StevenK: have you added any more types to enterpriseid ?
* benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugtasks: 4*10
<StevenK> lifeless: Not yet.
<lifeless> cool
<sinzui> jcsackett, mumble?
<jcsackett> sinzui: having a little trouble getting it to launch.
<jcsackett> be there in a moment.
<wgrant> lifeless: Except that fixtures.TempDir doesn't support that.
<wgrant> (it probably should, but I wanted this code to go away first)
<lifeless> wgrant: it does
<lifeless> wgrant: rootdir=...
<wgrant> Not in LP's version.
<wgrant> Maybe we're out of date.
<wgrant> lifeless: Hah
<wgrant> lifeless: Support was added 3 revisions after our current version.
 * wgrant upgrades.
<wallyworld_> sinzui: is there a bug already for the existing sharee / picker issue?
<sinzui> wallyworld_, no, I was thinking to co-opting/editing  https://bugs.launchpad.net/launchpad/+bug/933828
<_mup_> Bug #933828: Cannot change what is shared with a user <disclosure> <javascript> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/933828 >
<wallyworld_> sinzui: yes, that bug is already fixed as written. so i will change the description
<sinzui> fab
<salgado> lifeless, hey, would mind having another look at https://code.launchpad.net/~linaro-infrastructure/launchpad/team-engineering-view/+merge/99342 to see if it the changes I did address the concerns you have?  both methods there are now using pre_iter_hook to do eager loading
<lifeless> sure, una momento
<lifeless> salgado: so search_bugs isn't really meant to be the consumer interface - BugTaskSet.search still is
<lifeless> salgado: if you can use that, that might be better
<wgrant> search_bugs is forbidden.
<lifeless> wgrant: you should make that clearer then
<salgado> hmm.  BugTaskSet.search() already returns me a decoratedresultset with a pre_iter_hook to eager load some stuff.  can I wrap that in another decoratedresultset with another pre_iter_hook?
<lifeless> well you're listifying the result already
<lifeless> so you can just bulk load in your function
<lifeless> no need for DRS stuff there
<salgado> indeed, but then when somebody changes BugTaskSet.search() to not do eager loading (or to eager load more stuff than it currently does), my method will either not work as expected (i.e. no eager loading of everything it should) or issue more queries than necessary because some of them were already executed by BTS.search()
<lifeless> salgado: equally when someone changes the schema your eager loader would need to change
<lifeless> salgado: I don't see 'changes need things changed with them' as a particular problem :)
<salgado> that's not my point but whatever
#launchpad-dev 2012-03-27
<wgrant> http://highscalability.com/blog/2012/3/26/7-years-of-youtube-scalability-lessons-in-30-minutes.html
<wgrant> "They wrote their own BSON implementation which is 10-15 time faster than the one you can download."
<StevenK> Nice of them to release it.
<lifeless> wgrant: nice post
<wgrant> Indeed.
<wgrant> wallyworld_: Your rev breaks 3 JS tests
<wgrant> They time out.
<wallyworld_> not locally, i checked
<wallyworld_> and the ones which do do not use any of my code
<wallyworld_> well, they might use my code indirectly
<wgrant> wallyworld_: I can reproduce the timeout new on 15018 locally.
<wallyworld_> so i wonder why they pass locally
<wallyworld_> for me
<wallyworld_> let me check
<wgrant> Different html5browser version?
<wgrant> Anyway, reverting.
<wallyworld_> could be, though i'm fairly up to date
<wgrant> Exactly
<wgrant> That's the problem.
<wgrant> You're running precise's.
<StevenK> Ahhh, I love fallout between lucid and precise.
<wallyworld_> i don't :-(
<wallyworld_> hopefully this will go away soon
<wgrant> wallyworld: http://paste.ubuntu.com/901543/ fixes the three broken tests
<wgrant> But probably breaks others.
<lifeless> well, there is our space usage:
<lifeless>  * 54474 Exceptions
<wgrant> Wrong
<lifeless> FDT
<wgrant> That's just normal fastdowntime
<wgrant> The space usage is the 25000 OOPSes per day from qastaging and staging
<wgrant> because gandwana doesn't have a memcached
<lifeless> ah
<wallyworld_> wgrant: hmm. i don't see why .prototype is needed if i am using yui inheritance
<lifeless> thanks for tracking that down, are ops on it?
<wgrant> lifeless: The immediate problem will go away once I unbreak the build.
<wgrant> lifeless: Since the garbo job that's whinging about memcached is about to be deleted.
<wgrant> But did ping webops about memcached.
<spm> oops. sorry, one sec
<lifeless> interesting, adsl modem isn't dropping link anymore, but apparently having routing issues nonetheles
<lifeless> s
<hloeung> tried moving to the motherland Australia? ;-)
<lifeless> been there done that
<StevenK> lifeless might have even got the t-shirt.
<lifeless> 'last ride on the big dipper'
 * StevenK missed out on the big dipper
<lifeless> wgrant: whats up with deleting the decoratedresultset doctest thunk
<wgrant> lifeless: It was already being run by test_doc
<wgrant> The registration got duplicated during the apocalypse.
<lifeless> kk
<lifeless> http://attractivechaos.github.com/HN-prog-lang-poll.png
<lifeless> flacoste: http://ceklog.kindel.com/2011/08/06/90-of-the-decisions-you-make-dont-matter/
<adeuring> good morning
<lifeless> stub: lib/lp/services/mail/sendmail.py line 9. Possibly the oldest TODO in the code base.
<stub> \o/
<stub> From before we had a working bug tracker to track that sort of thing :)
<cjwatson> So I have a trivial patch for bug 966092 that doesn't increase LoC in itself (and is arguably a slight simplification).  Before the new maintenance policy I'd have added a 20-line test for it without worrying.  What should I do now?
<cjwatson> I guess I could try to find some bits of those tests to compactify or something
<cjwatson> or I could not add a test on the (dubious) grounds that it's currently untested anyway
<cjwatson> I don't know what people would prefer
<lifeless> you could reduce duplication/waste in the tests, you could reduce it elsewhere to make room for your test
<cjwatson> that general approach then?  ok
<cjwatson> but in particular I guess you'd prefer not to see a branch where I hadn't written a test in order to comply with the maintenance policy?
<lifeless> the working theory is that we've had little/no pressure for compact code, and we now have one.
<lifeless> Right, that would be pathological, and reviewers would reject it anyway : if it wasn't ok to do that before, its not now.
<cjwatson> ok
<lifeless> An alternate strategy is to ask someone that has been deleting lots of code for some credit; we don't really track per-person or anything, but something of a barter system has arisen
<cjwatson> yeah, I should have saved my big deletions for after this policy was instituted ;-)
<lifeless> Push comes to shove, we'd rather have simpler-and-correct code in than not.
<cjwatson> I'll figure something out.  Thanks
<lifeless> the LoC thing is documented to be a proxy metric. If you do decide you need a waiver for more LoC, flacoste or I can provide one.
<wgrant> cjwatson: Ah. I've been considering fixing that for a while, but thought there must be some catch that had kept that horrid step in place for 6 years.
<cjwatson> wgrant: yes, it's called "inertia"
<wgrant> Heh
<salgado> benji, hi there.  so, I've done the changes that lifeless suggested to that branch (https://code.launchpad.net/~linaro-infrastructure/launchpad/team-engineering-view/+merge/99342).  would you have some time for another look? :)
<benji> salgado: sure, I can look at it now
<benji> salgado: looks great
<salgado> benji, cool, thanks!  would you mind ec2-landing it again? :)
<benji> salgado: sure
<nigelb> benji: Great post about tests! I didn't realize LP tests could be run in 45 minutes now. That is cool.
<benji> salgado: the ec2 wheels are turning
<salgado> benji, great, thanks!
<benji> nigelb: hopefully we can pull together a few more loose ends and get it deployed so buildbot will be a bit more responsive than it is now
<nigelb> \o/
<deryck> Morning, all.
<deryck> rick_h, ping for standup
<deryck> rick_h, so we need a bug describing the issue that led to:  https://code.launchpad.net/~rharding/convoy/add_listmod_header/+merge/94437
<deryck> rick_h, it's implied in the MP, but a proper bug describing the problem would be nice. ;)
<rick_h> deryck: ok
<deryck> rick_h, and to clarify, the reason U1 and Landscape didn't get hit by bug 966220 is because they run convoy in tree, rather than as a separate service?
<_mup_> Bug #966220: using the path info to route breaks in various deployment cases <Convoy:New> < https://launchpad.net/bugs/966220 >
<rick_h> deryck: well it's not clear they're running the updated convoy at all.
<deryck> rick_h, ah.
<rick_h> deryck: but it's that their not doing any routing, so as it works now, we couldn't get it to work either
<deryck> rick_h, what do you mean by routing?
<rick_h> deryck: so in the way it broke works toward how U1 and landscape work
<deryck> ah
<rick_h> so /combo/12345 would get eaten in apache to /12345 which convoy then ate the first part of and so no path hint to do extra path finding on disk
<rick_h> deryck: so what we need is /combo/12345 to make it to convoy as /12345 which tells convoy to look in /12345/build/js/...
<deryck> rick_h, so they're not using the new feature of revnos being significant?  right.  Their use only cares about the constructed query?
<rick_h> deryck: right, they only care about the ?yui-min...
<deryck> rick_h, and the current version of convoy is backwards compatible to work either way, i.e. even if they did upgrade, they wouldn't notice because of using convoy the way they do. is that right?
<rick_h> rigght
<rick_h> deryck: correct, so in order to really make things work 'as they should' would require being backwards incompatible
<deryck> rick_h, so why don't we get hit by this in local dev?
<rick_h> deryck: because we don't attempt to use the revno locally
<rick_h> so path is just /combo?
<rick_h> which gets cut to just /
<rick_h> and that doesn't effect our routing to the build location on disk at all
<deryck> rick_h, and by routing, you mean a script alias directive?  We're using that, not a proxy pass directive, right?
<rick_h> so currently it looks at what the server puts into thw wsgi PATH_INFO env variable
<rick_h> deryck: right, this is all seperate from any proxy pass/etc
<deryck> rick_h, i.e. it's the script alias directive that consumes the +combo fragment and hands off the rest to mod_wsgi?
<deryck> rick_h, sorry if I'm being overly specific here. just trying to think it through.
<rick_h> deryck: think so, but not 100% sure. let me pull up the apache config
<rick_h> right, the proxy pass says NOT to proxy /combo urls, but then it hits the alias, which passes the request to convoy and the url is different in there
<deryck> rick_h, ok, I see now.
 * deryck is playing locally
<rick_h> deryck: yea, I'm reading the pep docs to see just what PATH_INFO needs/should be
<deryck> rick_h, this feels to me like there ought to be some apache magic we can do to make sure the script gets passed everything it needs.
<rick_h> deryck: right so in testing out a different server there's a RAW_URI environment variable I could use, but that's not available in pure wsgiref
<rick_h> so it blows up in tests/etc
<rick_h> but really, I think the magic should be on the other end. If you want to ignore the url and not deal with extra paths, you should rewrite your urls to get rid of it
<rick_h> so if you want a magic ignored /12345 it should be on you to rewrite that to /
<rick_h> the problem is we're trying it do it backwards and it's causing things to break down
<deryck> rick_h, does convoy log anywhere locally?
<rick_h> deryck: no, it doesn't
<deryck> rick_h, how can I tell what URL it's getting handed?
<abentley> adeuring: Could you please review https://code.launchpad.net/~abentley/launchpad/run-via-celery/+merge/99099 ?
<adeuring> abentley: sure
<rick_h> :) playing with the source to print/log/pdb
<rick_h> deryck: ^
<deryck> ouch :)
<rick_h> deryck: yea, it's not the best thing ever writte, but that's why I think it's been wanting to move on
<rick_h> deryck: so in looking, if we wanted to keep doing the magic I've been doing, I'd need to combine the SCRIPT_NAME and PATH_INFO to keep the full url
<rick_h> and then keep doign the regex/cleaning I do now
<deryck> rick_h, yeah, exactly.  I was trying to see if mod_wsgi had some directive to include script_name in path_info.
<rick_h> but I think that would then still break things for the other users.
<rick_h> deryck: I don't think so, I don't think it's meant to be done. They have clear goals for each part of that
<deryck> rick_h, right.  so at the least, if convoy.wsgi gets something like environ['PATH_INFO'] = environ['SCRIPT_NAME'] + environ['PATH_INFO'] that should work, right?
<rick_h> deryck: right, so if I used that, it should work I think, but when I look at that, I think it'd still end up backwards incompat with U1/landscape.
<deryck> rick_h, how so?  if they're dropping the first part of path_info anyway?
<rick_h> deryck: looking, sec
<deryck> rick_h, np.
<rick_h> deryck: can we jump on a call?
<deryck> rick_h, was just about to ask you the same.  spin up a hang out and give me 2 minutes to warm coffee.
<rick_h> https://plus.google.com/hangouts/extras/talk.google.com/orange-one-on-one
<rick_h> deryck: ^^
<jcsackett> can someone point me to docs about landing lazr stuff? particularly lazr.restfulclient?
<adeuring> abentley: I'm wondering, if line 34 of the diff might result in a race condition: You call CeleryRunJob.delay() for an "absolutely fresh" job: What happens if celeryd tries to execute the job before txn.commit() is called in the transaction that creates the job?
<abentley> adeuring: I think you're right.  Is there a way to hook in so the next txn.commit executes CRD.delay()?
<adeuring> abentley: I have no idea. But if there no such mechanism, we could also let celeryd wait for a few seconds, I assume
<abentley> adeuring: We could, but we would lose the responsiveness we gain by using Celery.
<adeuring> abentley: right... And a hook for "after txn.commit() work" would anyway be much cleanr
<abentley> adeuring: transaction appears to support current.addAfterCommitHook
<adeuring> abentley: sounds good
<deryck> jcsackett, what kind of docs do you need?  I've always just branched, coded, put up for mp,done.  or do you mean how to update versions in lp after that?
<jcsackett> deryck: nothing happens post MP? just mark approved and it lands?
<deryck> rick_h, also, I'm going to drag that card  in progress for you on the board.
<rick_h> deryck: thanks
 * jcsackett hasn't landed lazr.restful stuff before
<deryck> jcsackett, same story as us.  submit to pqm like lp.
<jcsackett> deryck: ah.
<deryck> jcsackett, then you need to make a tarball and added to download-cache and update versions.cfg to use the new version.  for us.  not sure about packaging releases if you need to do that.
<adeuring> abentley: in class UniversalJobSource, you override the dbuser. That sounds fine for branch management job, but it will not be useful for example in translation imports.
<jcsackett> deryck: thanks.
<deryck> jcsackett, np, man!
<abentley> adeuring: Indeed.  I'm not worried yet, since BranchScanJob is the only supported job type in that branch.
<adeuring> abentley: well, ok, let's defer that to another branch
<abentley> adeuring: I want to talk to lifeless about using launchpad_main for all Jobs, because the alternative is pretty messy.
<adeuring> ok
<abentley> adeuring: i.e. you have to switch DB user after you've loaded the job, and then you have to re-load the job so it uses the new db user.
<adeuring> right
<abentley> adeuring: skip_celery should be True only in tests.
<adeuring> abentley: yes, that's how I understand it. Just worth a short explanation, Ithink ;)
<abentley> adeuring: Ah.  Your review said "...skip_celery should be False only in tests"
<deryck> abentley, adeuring, rick_h -- forgot to remind in standup, self reviews and peers selected due tomorrow.
<adeuring> abentley: gah, I like to mix up reft and light
<deryck> rick_h, you and I probably need to chat about this later, since you haven't done it yet.
<deryck> but for now, it's lunch and work out time ;)
<rick_h> deryck: convoy MP approved and pushed so hopefully this helps the deploy/test crew.
<rick_h> deryck: let me know when you want to chat re review stuff. I filled out the one, but looks like we need to chat re goals
<deryck> rick_h, yeah.  let's hangout once more.
<deryck> rick_h, https://plus.google.com/hangouts/extras/talk.google.com/orange-dudes
<salgado> benji, did you get any failure/success from that ec2 land you did for me?  it should be done by now, right?
<benji> salgado: I got a failure.  I thought that a message would be sent to you too.  If not, I will forward it.
<salgado> benji, no, I didn't get it.  probably because is not owned by myself but by a team
<benji> ah
<salgado> the branch, I mean
<benji> salgado: email forwarded
<salgado> thanks benji
<lifeless> morning
<benji> np
<salgado> benji, looks like I broke it when I replaced the Join+Coalesce on Person.  the filter I wrote didn't have the exact same semantics of coalesce
<salgado> benji, I've fixed it and pushed the change: http://paste.ubuntu.com/902724/
<lifeless> salgado: sounds like fallout from my suggestion... sorry@!
<benji> salgado: cool, I'll warm up the ec2 machinery
<salgado> lifeless, yeah, I shouldn't trust you when it comes to coalesce! (everything else I think I can, though ;)
<salgado> thanks again, benji
<lifeless> salgado: did it bring an unintended workitem,.. from a spec that the spec assignee was a participant
<salgado> lifeless, yep. luckily my tests caught it.  if only I hadn't forgotten to run my tests after that one final change, I'd have found this out earlier
<lifeless> cool
<lifeless> did you update to use the public bugtaskset.search API?
<salgado> yes
<lifeless> great
<lifeless> sorry that its not clear that its private
<lifeless> I'll nag wgrant (who extracted it for code hygiene only) to make that really clear
<lifeless> (that the other function is private I mean)
<StevenK> sinzui, wallyworld_, wgrant: http://pastebin.ubuntu.com/902944/
<sinzui> wallyworld_, wgrant, StevenK: http://people.canonical.com/~curtis/audit/
#launchpad-dev 2012-03-28
<wgrant> lifeless: Hi
<lifeless> hi
<wgrant> lifeless: Can you try to outsmart the planner for me? :)
<lifeless> there is no try
<lifeless> only do, or do not
<wgrant> lifeless: http://paste.ubuntu.com/903111/
<wgrant> With seqscans forbidden it takes 950ms, but when permitted it applies them liberally and takes 3500ms.
<wgrant> I can get it down to 1100ms by employing the perplex-a-planner strategy of pushing everything out into CTEs.
<lifeless> qastaging ok for this?
<wgrant> Ja
<wgrant> Ah
<lifeless> grah unity
<wgrant> You'll need different AccessPolicy IDs.
<lifeless> I *so* hate the 'raise all windows of an app' behaviour
<wgrant> SELECT id FROM accesspolicy WHERE distribution=1;
<lifeless> it breaks me every time I alt-tab
<lifeless> so much more fiddly than metacity
<lifeless> wgrant: two rows
<lifeless> 65 type 4
<wgrant> Those are the IDs that the query needs.
<lifeless> 66 type 3
<lifeless>  Total runtime: 1094.256 ms
<lifeless> (17 rows)
<lifeless> default behaviour
<lifeless> how fast do you need this?
<wgrant> orly
<wgrant> Um
<wgrant> As fast as possible :)
<lifeless> http://paste.ubuntu.com/903115/
<lifeless> thats with grantees AS (...
<wgrant> http://paste.ubuntu.com/903116/ is the perplex-a-planner strategy.
<wgrant> How does that perform?
<lifeless> 730ms hot (the first one I tried, looking at your new one now)
<wgrant> Hm, not bad.
<wgrant> That's almost acceptable, in fact.
<lifeless> 719ms for the perplex
<lifeless> 711ms hot
<wgrant> That's more sensible.
<wgrant> Because it's saving a lot of TP lookups.
<lifeless> 770ms
<lifeless> some variation
<wgrant> Can has explain analyze?
<lifeless> http://paste.ubuntu.com/903118/
<wgrant> And one of the hot original query?
<lifeless> http://paste.ubuntu.com/903120/
<wgrant> Thanks,.
<lifeless> this is showing the expanded out list of people right? The crazy thing.
 * lifeless thinks we're failing to make it usable
<wgrant> lifeless: Yeah, the filterable expanded audit view.
<lifeless> btw
<lifeless> person_sort_key -> 24 seconds
<wgrant> Er, really?
<wgrant> It wasn't that bad last I checked.
<lifeless> yup
<wgrant> 2500ms here
<wgrant> Still far worse than expected.
<wgrant> But not 24s...
<lifeless> 1440ms for the confuddle-the-planner query
<lifeless> http://paste.ubuntu.com/903125/
<lifeless> so I really think we shouldn't do this at all
<lifeless> Who do I need to speak to to steer this
<lifeless> Curtis?
<wgrant> He's probably the first point of contact.
<lifeless> pagination 180K rows of anything is pointless.
<lifeless> There is /no/ way its usable.
<wgrant> Oh yes.
<lifeless> sinzui: ^ hi. I'd like to arrange time to talk about this.
<wgrant> But for sensible projects it should be fine.
<wgrant> There might be a hundred in total.
 * StevenK tries to find an API method that takes an enum as a parameter.
<wgrant> StevenK: searchTasks
<lifeless> StevenK: searchTasks
<wgrant> StevenK: addMember
<wgrant> lifeless: THe auditing view is only meant to be useful for sensible projects.
<wgrant> lifeless: And potentially for auditing All sharing in Ubuntu.
<wgrant> Which will be much smaller sets.
<wgrant> However, I needed to find out whether it would vaguely work on the full Ubuntu set.
<wgrant> Or if we'd have to take evasive action.
<poolie> lifeless, your comment on 956655
<poolie> > It configures it to talk to the *host side address of the libvirt
<poolie> bridge*.
<poolie> what's the result of that?
<poolie> isn't it that the packets are passed through to the guest?
<poolie> o/ wgrant
<wgrant> Morning poolie.
<lifeless> poolie: no, libvirt runs a dnsmasq there
<wgrant> poolie: No, the host has an interface on the bridge.
<lifeless> poolie: as part of it running dhcp etc for the containers/vms
<poolie> ok
<lifeless> (well, I say libvirt, I'm not 100% sure *what* is responsible for starting that dnsmasq up, but thats orthogonal to it existing :))
<poolie> so i think the echo is actually between the host's general dnsmasq and this one
<poolie> so i was inaccurate to say 'in the guest'
<wgrant> lifeless: (also, there's only 20000, not 100000 :P)
<lifeless> wgrant: oh, read the wrong row out? Still. Its more than 200.
<wgrant> lifeless: Right. There's about 19000 explicit grants, expanding to 20000 when team members are included.
<StevenK> So paginating more than 200 items is wrong?
<wgrant> Er
<wgrant> s/explicit grants/distinct people and teams with explicit grants/
<wgrant> lifeless: http://people.canonical.com/~curtis/audit/ are mockups of the UI
<lifeless> StevenK: for usability, yes.
<lifeless> StevenK: some folk say more than 30 or 40
<StevenK> lifeless: Reading it that, that makes our critical bug list, and Ubuntu's bug listing is unusable.
<wgrant> Nobody would argue that they're usable.
<lifeless> StevenK: they are
<lifeless> StevenK: we do two things with them commonly: we data mine to extract patterns that are much shorter - human scale
<lifeless> StevenK: and we provide a search facility to let folk find things that interest them within the big pile
<lifeless> StevenK: how would you answer the question 'is ~somebodyinlotsofteams in ~ubuntu-members' ?
<lifeless> StevenK: and how would you *like* to be able to answer that?
<StevenK> Hmmmm.
<StevenK> lifeless: So to be fair, I was trying to troll you. Clearly, I failed epically. :-)
<lifeless> the key to a good troll is to say something thats totally outrageous in a way that sounds plausible, or something thats *nearly* plausible but really actually silly.
<lifeless> Speaking the truth, not a troll!
<StevenK> wallyworld_: So I think I've made bugs-remove-old-privacy safe enough to land whenever, but I'd like a second set of eyes.
<wallyworld_> StevenK: sure, can you remind me of the link?
<StevenK> It flips CreateBugParams() to using information_type and keying private and security_related off of it.
<StevenK> wallyworld_: It will be a few minutes, I need to push and wait for the diff.
<wallyworld_> ok, np
<StevenK> wallyworld_: https://code.launchpad.net/~stevenk/launchpad/bugs-remove-old-privacy/+merge/98974 -- diff updated, and I've rewritten parts of the description, so glance at that too.
 * wallyworld_ looks
<StevenK> wallyworld_: But you were happy with it before, so I'm okay with it if you want to 1) just read the diff for r14994, and 2) read the big diff to see how those changes fit into the big picture.
<wallyworld_> sure
<wallyworld_> StevenK: why are we adding the individual private/security attributes back? waiting for db patches to land?
<StevenK> wallyworld_: So it's safe to land no matter the state of the DB patches, and so I can base the UI branch off that work.
<wallyworld_> StevenK: and because we have allow multi-pillar private bugs for everyone via the feature flag, that extra exception won't matter
<wgrant> StevenK: Why do you unprivate the bug in lib/lp/bugs/doc/bug.txt?
<StevenK> wgrant: For the reason I gave in the description -- without the feature flag turned on, we will now always complain in transistionToInformationType()
<wallyworld_> StevenK: will the extra complaining be temporary?
<wgrant> StevenK: I don't see an explanation of that.
<StevenK> wgrant: "This slightly changes the behavior of the function, since it will now always complain about multi-pillar private bugs whereas it used to only care if you were flipping the privacy flag."
<wgrant> Ah.
<StevenK> wallyworld_: That is the 64-million dollar question, isn't it?
<StevenK> wallyworld_: I'm not sure about the timeline of multi-pillar private bugs complaining.
<wallyworld_> StevenK: so once all the legacy stuff is removed, we only want to complain if the info type transitions to embargoed sucurity or user data or proprietary?
<wallyworld_> or did we decide to allow such things now and remove the fflag and check entirely?
<StevenK> There was talk of a footgun.
<wallyworld_> so why can't we fix the errouneous complaining in the current branch?
<wgrant> The multi-pillar restriction only applies to PROPRIETARY
<wgrant> But the code has not yet been fixed.
<wallyworld_> ok. i was thinking it applied to user data etc also
<StevenK> I can do that in this branch if you like, but we haven't defined what the heck PROPRIETARY means for bugs.
<wgrant> userdata potentially.
<wgrant> But applying it to embargoedsecurity was vetoed.
<wgrant> It needs to be a separate branch.
<wallyworld_> let's land this then since it's moot given the current sdtate of the fflag
<StevenK> So you two are happy with it as it stands right now, and we can fiddle it later?
<wallyworld_> i think so
<wgrant> I'm only half done, but happyish so far.
<StevenK> wgrant: Anything I can do to flip you from happyish to happy?
<wgrant> StevenK: lib/lp/bugs/mail/commands.py is confusing and buggy
<wgrant> And it's using 'is' instead of '=='
<wgrant> What happens if I set a bug private when it's UNEMBARGOEDSECURITY?
<wgrant> Seems like it will become PUBLIC
<wgrant> Also if I try to make  USERDATA bug private, it will become public.
<StevenK> wgrant: I plan on reworking that in a later branch when I add a information_type flag.
<wgrant> Yes, but this is fail-open buggy.
<wgrant> cannot land.
<StevenK> wgrant: So currently you can't have UNEMBARGOEDSECURITY in the mail handler.
<StevenK> So we're only tracking three states.
<wgrant> StevenK: Ensure that.
<wgrant> s/else/elif blah/
<wgrant> And else: raise AssertionError("shit is broken")
<StevenK> wgrant: Yeah, I was reading over the diff and shaking my head a little at what I wrote in mail commands.
<wgrant> Setting it to public because it was proprietary sounds bad.
<wgrant> 393	- # BugSet.createBug() requires new security bugs to be private.
<wgrant> 394	- context.private = True
<wgrant> 395	+ context.information_type = InformationType.EMBARGOEDSECURITY
<wgrant> 396	+ else:
<wgrant> 397	+ context.information_type = InformationType.PUBLIC
<wgrant> Is also a bit odd
<wgrant> I suspect it defaults to PUBLIC
<wgrant> So it might be nice to avoid explicitly setting it.
<wgrant> For the same reason.
<StevenK> wgrant: 'security no' -> PUBLIC, unless private is true
<wgrant> Sure, but the old code didn't do anything in that case.
<wgrant> The old code purely overrode to security_related=True if security=True
<wgrant> s/security_related/private/
<wgrant> 621+ if self.information_type is information_type:
<wgrant> Should probably be ==, again
<wgrant> You can't spell 'privileged'
<wgrant> (line 741)
<StevenK> Which line number?
<lifeless> 741
<wgrant> A tonne more 'is'
<wgrant> 803+ bug_privacy_filter = 'AND Bug.information_type IN (1, 2)'
<wgrant> That shouldn't be literal.
<wgrant> And in bugtasksearch.py
<wgrant> 1134	where=And(
<wgrant> 1135	- Bug._private == True,
<wgrant> 1136	+ Bug.information_type in PRIVATE_INFORMATION_TYPES,
<wgrant> Always True
<wgrant> 1144	+ where=And(Bug.information_type in PRIVATE_INFORMATION_TYPES,
<wgrant> 1145	+ BugTask.assignee == self.id)),
<wgrant> Likewise
<wgrant> 1166	+ SELECT Bug.id FROM Bug WHERE Bug.information_type IN
<wgrant> 1167	+ (1, 2)
<wgrant> More literals where the enum should be.
<wgrant> wtf is there a createBug in lp.systemhomes
<wgrant> Otherwise looks good.
<StevenK> I have no re systemhomes
<StevenK> no idea
<StevenK> wgrant: I can't use Bug.information_type in PRIVATE_INFORMATION_TYPES in storm?
<wgrant> No
<wgrant> You might recall that I nuked lots of the Ubuntu archive like that once.
<StevenK> Haha
<StevenK> wgrant: What's the right pattern, then?
<wgrant> .is_in(blah), probably
<StevenK> Right
<lifeless> cjwatson: how did you go w/LoC?
<cjwatson> lifeless: Clawed it down to +56/-57.
<cjwatson> (I refactored the tests a bit.)
<lifeless> cool
<cjwatson> Also submitted a +0/-531 branch to store up some credit ;-)
<lifeless> hah :)
<cjwatson> (Is there a graph somewhere of the Launchpad line count over time?)
<lifeless> flacoste has muttered about making one
<lifeless> for every project we have
<spm> spads did a very nice bubbles of code/person over time viewy thingy of our puppet branch. rather amusing to see. I don't recall the proper name for those offhand; but he may have some advice to offer there?
<cjwatson> code_swarm?
<cjwatson> http://code.google.com/p/codeswarm/
<wgrant> cjwatson, lifeless: https://www.ohloh.net/p/launchpad
<wallyworld_> wgrant: what's the verdict on the sharing team participation query?
<wgrant> wallyworld_: http://paste.ubuntu.com/903111/ works well on qastaging
<cjwatson> wgrant: ah, cute
<wallyworld_> wgrant: thanks, will use that as a starting point
<lifeless> wgrant: as long as you don't use person_sort_key
<lifeless> wgrant: where it goes to 24 seconds
<wgrant> lifeless: I still don't believe that.
<wgrant> But we can't anyway due to SRF.
<wallyworld_> lifeless: at the moment, we can't use person_osrt_key anyway
<lifeless> check the explain analyze I gave you :)
<wgrant> what the postgres
<wgrant> How could it think that was a good idea?
<wgrant> Oh
<wgrant> The sort index
<wgrant> It uses the bloody sort index.
 * wgrant drop index and retries.
<wgrant> Magical, 1s.
<wgrant> Postgres, you are a moron.
<StevenK> wgrant: Do you want to recheck that MP? I think I've addressed all of your concerns.
<wgrant> StevenK: In a few, once I've lunched.
<StevenK> I'll start putting together a UI branch
<StevenK> Since I'm mostly happy with that one.
<lifeless> wgrant: so, I'll leave that happy thought with you
<wgrant> lifeless: You're so kind :)
<wgrant> Its estimates aren't even that far off :(
<lifeless> wgrant: I aim to please.
<lifeless> (time to do my allhands stuff. \i/.
<wgrant> That reminds me
<wgrant> I have more of that wondrous task to do
<lifeless> 'am I wonderful? Yes I am!'
<StevenK> cjwatson: If you're still around, I've approved all three of your MPs, do you want to toss them at ec2?
<StevenK> cjwatson: Or shall I?
<StevenK> Bah, I hate how I can figure out what I want the portlet to say, but don't know enough about the privacy portlet or TAL to express it :-(
<wgrant> StevenK: Sorry, looking at your MP again now.
<StevenK> Hah.
<wgrant> StevenK: I'd do sqlvalues(PUBLIC_INFORMATION_TYPES)
<wgrant> Rather than spelling out '(%s, %s)
<wgrant> ' % (InformationType.blah.value, blah, blah)
<StevenK> %s' % sqlvalues() is it, I guess?
<wgrant> Should work
<wgrant> Possibly needs manual parens, but I don't quite remember.
<wgrant> I use native Storm parameterisation where possible now, but it's a bit harder here.
<wgrant> lifeless: Can I set the 5s timeout on Product:+sharing and Distribution:+sharing now?
<StevenK> wgrant: I didn't change the tests, so they're still looking for "IN (1, 2)" in the text
<wgrant> StevenK: The bug search query changes should be feature-flagged.
<wgrant> StevenK: They have the potential to completely destroy bug search performance.
<StevenK> Haha, PUBLIC_INFORMATION_TYPES doesn't exist.
<wgrant> Correct.
<wgrant> But it can easily be brought into existence.
<StevenK> wgrant: I can revert the bug search query changes and we fix it with BugTaskFlat?
<wgrant> StevenK: That might be best.
<wgrant> It should be fairly safe.
<wgrant> But you never know, and there's hundreds of cases.
<StevenK> wgrant: I figured that I didn't want to play with bringing up a feature flag and playing with that function more when you're going to rip it to pieces in a week or two.
<StevenK> Hmmmm, it's taking a very very long time to save MP comments today.
<StevenK> wgrant: Perhaps something in the NDT we did today is to blame?
<StevenK> wallyworld_: Your two MPs are approved
<StevenK> wgrant: Do you have any other comments?
<wgrant> StevenK: Commenting is fast for me.
<wgrant> StevenK: commands.py is still unintelligible.
<StevenK> Perhaps it was the nine MP pages I had open, but approving one of wallyworld_'s MPs took 200 seconds.
<wgrant> And you can't spell 'Unknown'
<wgrant> That would be the nine MP pages, yes.
<wgrant> Longpoll connection exhaustion.
<StevenK> But approval doesn't require longpoll?
<wgrant> The limit is a general browser per-domain connection limit.
<StevenK> wgrant: What about commands.py is still unintelligible?
<wgrant> StevenK: What's wrong with:
<wgrant> if information_type == UNEMBARGOEDSECURITY: information_type = EMBARGOEDSECURITY
<wgrant> elif private and information_type == PUBLIC: information_type = USERDATA
<wgrant> else: wtf
<wgrant> (ideally the first branch wouldn't be required, but I suspect it might be)
<StevenK> wgrant: security_related just sets to EMBARGOEDSECURITY if it's set. But I don't want to just set it to USERDATA if the next command is private yes
<StevenK> Hmmm, the current tests trip that assert.
<wgrant> Ah, so this is only for creation.
<StevenK> Yes.
<StevenK> wgrant: If the bug exists, the mail handlers just call set{Private,SecurityRelated} and job done. If the context is CreateBugParams, it needs to be kept in a sane state since it's used to file the bug.
<StevenK> Ah ha.
<wgrant> StevenK: So, if it's PUBLIC or USERDATA, it goes to PUBLIC or USERDATA depending on value. If it's UNEMBARGEODSECURITY or EMBARGOEDSECURITY it goes to EMBARGOEDSECURITY
<wgrant> If it's proprietary, it crashes.
<StevenK> At the moment, it doesn't do that.
<wgrant> This is the behaviour that the command has in the new world.
<wgrant> It may not be how it's currently implemented.
<StevenK> What's going on is the test creates a params with EMBARGOEDSECURITY and then executes private no, which asserts
<wgrant> But that's how it is.
<wgrant> Actually, I guess on creation UNEMBARGOEDSECURITY should crash now.
<StevenK> I think that assert lives on.
<StevenK> Sadly.
<StevenK> wgrant: This is why I force to EMBARGOEDSECURITY with security_related, it's what we're currently expecting.
<wgrant> It should be impossible, but indeed, I guess that makes sense.
<wgrant> So, case information_type of (PUBLIC, USERDATA) -> {False: PUBLIC, True: USERDATA}[private]; (UNEMBARGOEDSECURITY, EMBARGOEDSECURITY) -> EMBARGOEDSECURITY; otherwise -> error
<lifeless> wgrant: please do
<wgrant> lifeless: Thankyou sir.
<wallyworld_> StevenK: thank you
<danhg> Morning
<adeuring> good morning
<lifeless> stub: hey
<lifeless> stub: reminder: hr stuff is due today
<stub> lifeless: I'm a few days ahead of you. All tasks done (although I didn't get requested for peer reviews? Maybe because I don't have any peers with the new structure, or is that next cycle?)
<lifeless> stub: oh excellent
<lifeless> stub: the system doesn't make it terribly clear what reports have/haven't done.
<lifeless> stub: thanks!
<lifeless> stub: peer reviews are done by doing 'select peer' in the UI
<lifeless> stub: if you haven't been asked to do any, noone has asked you is all.
<lifeless> stub: feel free to reach out and ask for reviews from e.g. folk you've helped, whatever team they are in.
<lifeless> stub: 2 or 3 is probably a good number
<lifeless> wgrant: so what did you think of the checkmarkable thing ?
<lifeless> wgrant: if we were for instance, to ditch some of our policy and process wiki pages for such checklists, would it be good, or bad, or meh?
<wgrant> lifeless: It looks like a good idea.
<wgrant> lifeless: Seems pretty incomplete and somewhat awkward at present.
<wgrant> But promising.
<lifeless> I'll show flacoste and statik and get their thoughts
<cjwatson> StevenK: I don't believe I have access, or at least if I do I don't have it set up; so please do lob them at EC2, thanks.
<StevenK> cjwatson: You could probably ask for PQM access -- but in either case, I'll land remove-archive-override-check directly and throw the other two at ec2.
<gmb> danhg, Morning. Just wanted to update you on the blog post I've written about Yellow + Juju: I'm redrafting it because it repeated large swathes of what Benji said; should have it to you by EOD though.
<gmb> (Completely forgot to send it to you on Monday; apologies)
<cjwatson> StevenK: yeah, Francis suggested that at one point.  Maybe I'll think about it around my next LP hacking spree :)
<czajkowski> aloha
 * StevenK kicks pqm-submit
<StevenK> Getting very close to just drafting a signed message
<danhg> Hey gmb, thanks so much for that - will ping you now...
<StevenK> cjwatson: Victory, finally.
<cjwatson> Cool.
<jtv> huwshimi: are you eod?
<huwshimi> jtv: Just on a call and then I'll be free
<jtv> Great, thanks
<huwshimi> jtv: I'm back
<jtv> Hi
<jtv> I'm trying to get a handle on the workings of nodes_chart.js.
<jtv> Whether I get to fix the bug or not, I though adding some nodes would be useful.
<huwshimi> nodes or notes?
<jtv> notes, sorry!
<huwshimi> :)
<jtv> My fingers form habits.
<huwshimi> yeah...
<huwshimi> jtv: I need to go back and refactor a bunch of that js
<jtv> The raised middle finger earlier was accidental as well.
<huwshimi> haha
<jtv> I figure I could do some of that.
<jtv> Oh, you didn't notice the middle finger?  It was, er, hypothetical.
<jtv> Yeah, that's the ticket.
<huwshimi> jtv: You're more than welcome to, but I'm happy to fix up what I started :)
<jtv> Okay, if you're motivated to do it, then I can leave it at this.
<jtv> With gratitude, even!
<huwshimi> jtv: There are already a bunch of notes on the MP from Raphael
<jtv> If you're not, well, I'm here to help.
<huwshimi> jtv: Yeah, really it's all good. I understand the code pretty well
<huwshimi> and I know you guys are busy!
<jtv> Then my advice is: write it all down before you forget!
<huwshimi> It's on my list of things for early next week
<jtv> OK, perfect.  Then I'll just drop this whole issue like a hot potato!
<jtv> Thanks.
<huwshimi> jtv: Great, no problems :)
 * jtv goes browse car pictures
<huwshimi> jtv: OK, I'm off too, catch you later
<cjwatson> StevenK: remove-archive-override-check didn't need a [no-qa] or something?
<jtv> nn!
<cjwatson> Or have the rules changed?
<cjwatson> StevenK: oh, never mind, I see the commit message that actually landed in devel had [no-qa].
* abentley changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: abentley | Firefighting: - | Critical bugtasks: 4*10
<deryck> rick_h, hey.  also.... it seems you took an action from the last TL call.  I'm assuming you didn't look at that yet?
<abentley> adeuring: In your regular expressions, ".*?" looks equivalent to ".*" to me.  Am I missing something?
<adeuring> abentley: it's just the non-greedy variant. SHould be slightly faster
<adeuring> I simply prefer this one ;)
<rick_h> deryck: ah right. let me pull that back up
<deryck> rick_h, don't loose any time over it today, since we've lost a bit to convoy lately.... but let's chat about it in our call later.
<abentley> adeuring: Oh, I didn't realize that was a thing.  I thought you were using ? to mean optional.
<rick_h> deryck: ok, trying to find the email on that. Honestly forgot all about that after I handed over notes
<deryck> rick_h, yeah, no worries.  it was new for you.  let's just chat about how to investigate those sorts of things, and then you can finish it this week, after you've gotten further on the email bug.
<adeuring> abentley: >>> re.search('(.*?)a', '123aabab').groups()
<adeuring> ('123',)
<adeuring> >>> re.search('(.*)a', '123aabab').groups()
<adeuring> ('123aab',)
<rick_h> deryck: ah I remember now. Yea, I got the bug number from StevenK. Let me find that bug. That's all I did with it was to follow up and find the bug.
<rick_h> deryck: https://bugs.launchpad.net/launchpad/+bug/724750 for your notes
<_mup_> Bug #724750: launchpad redirects pack files to https://launchpad.net (while repacking?) <Launchpad itself:Triaged> < https://launchpad.net/bugs/724750 >
<rick_h> ty giant irc logs
<deryck> rick_h, ok, we'll look together during our call.  Thanks!
<abentley> adeuring: r=me.
<adeuring> abentley: thanks!
<rick_h> is there a way in tests to specify a single test? using nosetests for instance I can do a --with-id xx to run a single test
<deryck> rick_h, couple ways....
<deryck> rick_h, either use the -t option and specify a very explicity regex, or drop the -t and full dotted path to test, i.e. lp.bugs.tests.test_xxx
<rick_h> deryck: right, so that's to the file, but not a test within the file
<rick_h> deryck: I think for now I just moved my test to its own file to speed things up
<rick_h> and will clean up once things are going ok, just curious if there was a way to get at a single test within the file
<deryck> rick_h, so this kind of thing works for me:
<deryck> ./bin/test -cvvt lp.bugs.browser.tests.test_bugtask.TestDistributionBugs.test_mustache_cache_none_for_feeds
<rick_h> deryck: ah, gotcha awesome
<deryck> rick_h, you can also just pass the final test name to -t, but if there's more than one test with that name it will run them all.
<rick_h> ok, I thought it was only file based, awesome to know it's more than that
<deryck> jcsackett, hey man.  are you able to qa bug 933797 today?
<_mup_> Bug #933797: Cannot see the Some things that are shared with a user <disclosure> <qa-needstesting> <ui> <Launchpad itself:In Progress by jcsackett> < https://launchpad.net/bugs/933797 >
<benji> rick_h: anyother little test runner tidbit: you can use -m to speed up test runs; it specifies the module(s) that should be scanned for tests, so if you're running lp.foo.bar.TestBaz.test_baz, you can add "-m lp.foo.bar" to the command line to tell the test runner to only look there for tests
<rick_h> benji: ooh, ty
<benji> on my box it makes test runs about 6 or 7 seconds faster
<abentley> rick_h: You can also omit the -m, but this gives substring matches rather than regular expression matches.
<jcsackett> deryck: would've replied earlier but didn't see the highlight.
<jcsackett> i'm about to start qa-ing it right now.
<barry> we have a user who is changing bug statuses against our will.  is there anything we can do to prevent this?
<barry> flacoste or sinzui ^^ perhaps?
<sinzui> barry, czajkowski, can contact the user and ask him to desist. She might choose to suspend the user first so that he understands we mean it
<sinzui> oh, but I think she EODed
<barry> sinzui: cool, i've done the former.  if he continues i'll ask czajkowski to do the latter
<barry> thanks!
<sinzui> barry, report the issue as a question at https://answers.launchpad.net/launchpad and provide the user's Lp Id and an example bug id
<sinzui> I will start action
<deryck> jcsackett, cool, thanks man!
<barry> sinzui: https://answers.launchpad.net/launchpad/+question/191955
<czajkowski> sinzui: wahts up sorry was just on bus home
<sinzui> czajkowski, I just sent a warning email to the user mentioned in barry's question
<sinzui> Looking at the user's Lp profile, I think he is confused about what Lp is
<czajkowski> sinzui: just ack the lp
<czajkowski> have seen that person at it already before
<barry> thanks guys!
<czajkowski> barry: np
<lifeless> sinzui: hola
<sinzui> hi lifeless
<lifeless> do you have time for a quick skype/hangout? I want to talk about a bit of the sharing UI & usability
<lifeless> you may have seen your name in scrollback :)
<lifeless> sinzui: ^
<sinzui> yes. I never promised that feature to Ubuntu. only oem wants it because they will murder us if we do not allow them to continue auditing their projects
<lifeless> flacoste: I've sent you an invite to a shared-checklist/process thing, would like to know what you think about it
<lifeless> sinzui: so, skype or hangout, or $other/
<sinzui> hangout
<sinzui> I am ready
<lifeless> invite sent
<sinzui> I got the invite right away, but the button will not show.
 * sinzui continues to kick google
<lifeless> https://plus.google.com/hangouts/a3baf4b8edd991e0d6fdd61b85d31012c81a2a69?lm1=1332957654772&authuser=0&hl=en#
<sinzui> interesting, google will show me notifications when it is not even certain I am logged in to interact with them
<lifeless> different service endpoints I suspect
<deryck> abentley, hi there.  I've got one for review if you are available.
<abentley> deryck: sure thing.
<deryck> abentley, thanks!  https://code.launchpad.net/~deryck/launchpad/person-latest-bugs-listings-921366/+merge/99800
<abentley> deryck: I think you could avoid re-indenting all that code by doing "if getFeatureFlag('bugs.dynamic_bug_listings.enabled') and not FeedsLayer.providedBy(self.request)"
<deryck> abentley, I thought the same, but I thought it was more obvious what the code would like look once the feature flag check goes away.  I'm not picky though.  I can change back to that if you think it's better.
<abentley> deryck: I guess I thought it was better because it makes it obvious that none of the contained code has changed, but it doesn't matter now.
<deryck> abentley, yeah, it's a fair point.
<abentley> deryck: Did you consider making getFeedViewCache a function?  That way you don't need to create another class.
<deryck> abentley, no, I didn't actually.  I'm okay with making it a function.
<abentley> deryck: Cool.
<abentley> deryck: I could swear we had a general-case version of _createView, but I'm having trouble remembering exactly where it is.
<deryck> abentley, yeah, I couldn't find it either.  Code feeds had basically what I added to bug feeds, so was following the pattern there.
<abentley> deryck: I think it is lp.testing.views.create_view / create_initialized_view
<deryck> abentley, do you mean to replace what I did in the test or what I did in the feed classes?
<abentley> deryck: Oh, I see.  I thought that was test code.
<deryck> abentley, ah, ok.  no, it's just meant to make the feed classes easier to test.
<abentley> deryck: Okay, never mind, then.
<abentley> deryck: r=me
<deryck> abentley, awesome, thanks.
<abentley> deryck: np, it
<abentley> 's what I'm here for.
<lifeless> -> my monthly allergy shot, back in a while (wil mis the TL meeting)
<salgado> hmm. the work items field has vanished
<salgado> abentley, do we deploy stable to production?
<abentley> salgado: yes.
<salgado> then the last deployment was a big fuckup, it seems: http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/14860 is what got deployed
<salgado> and that's a month old
<abentley> salgado: So, I don't recall the details, but it's plausible that FDT rollouts are from db-stable, or that they involve merging db-stable into stable.
<salgado> abentley, it magically went from r14860 to r15027 now
<abentley> salgado: That's what the last NDT rollout was.
<salgado> abentley, right, but for some time it was at r14860 and the workitems field had vanished: http://people.canonical.com/~salgado/lp-blueprints-without-workitems.png
<salgado> abentley, looks like some appservers somehow have r14860.  discussing that in lp-ops
<poolie> lifeless, hi
<lifeless> poolie: hi
#launchpad-dev 2012-03-29
<StevenK> wallyworld_: O hai
<wallyworld_> hello
<wallyworld_> did we need to mumble?
 * StevenK looks for mumble
<StevenK> wgrant: How is the second branch wrong? I don't want to set it to private if it's EMBARGOEDSECURITY for the moment.
<wgrant> StevenK: In commands.py?
<StevenK> wgrant: Yup.
<wgrant> StevenK: It will currently AssertionError if given EMBARGOEDSECURITY
<StevenK> Oh, I fixed that, just forgot to commit and push.
<poolie> wgrant, hi
<wgrant> poolie: Hi
<StevenK> wgrant: I removed the assert, I'm going to toss it at ec2.
<StevenK> wgrant: As to why createBug() is in systemhomes, I still have NFI
* abentley changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugtasks: 4*10
<lifeless> wgrant: got 5 for a quick chat ?
<wgrant> lifeless: Just got back from lunch. Still around?
<lifeless> yes
<lifeless> skype?
<wgrant> It hates PulseAudio. Let me see.
<lifeless> pavucontrol will let you fix that easily
<wgrant> No, it won't even connect to PA AFAICT
<lifeless> oh, thats new and special
<wgrant> Yeah
<wgrant> Haven't had this one before.
<wgrant> Let's see if the one from partner works.
<mwhudson> PA seems to have decided my headset is not an input device recently
<bigjools> it would be right? :)
<StevenK> Hmm, how do I tell firebug to keep executing JS?
<StevenK> The play button. Tricky.
<adeuring> good morning
<StevenK> cjwatson: Are you good to QA r15036 on DF today?
<cjwatson> StevenK: yep, I will be
<stub> wgrant: If your still there, does PG 9.1 make your life easier with bugtask search data structures? Things are so far on track to switch Production starting 23rd April.
<stub> (Minor things like the Precise release might affect that :) )
<stub> This merge proposal is making me thirsty.
<wgrant> stub: Being able to use GIN for a second iteration will be nice.
<stub> Saw your comment about array syntax mentioning 8.4
<wgrant> And its better support for text arrays might open up new avenues for tags.
<stub> But it isn't worth doing this as a 9.1 specific feature branch?
<wgrant> No. This is good enough for now, and we get it a month early.
<wgrant> Anything 9.1-specific will need more experimentation post-upgrade.
<wgrant> And this is deliberately designed so we can iterate cheaply by introducing new versions of the table.
<StevenK> I'm looking forward to the death of slony. Oh, and 5 seconds FDT.
* rick_h changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: rick_h* | Firefighting: - | Critical bugtasks: 4*10
<stub> wgrant: What do the indexes ending with the bugtask or bug columns get used for. Is it being used as a creation date (but with stable ordering)?
 * stub reads the mp blurb just in case this was explained :)
<wgrant> stub: Ah, sorry, forgot about the indices.
<wgrant> stub: Those are bug search sort orders.
<wgrant> So bugtask/bug are tiebreakers.
<stub> Just to ensure consistent ordering?
<wgrant> bug is used when the context is unambiguous (only a single task for each bug can appear, ie. most contexts), bugtask when it's ambiguous (eg. distributions, where you can have tasks for multiple packages on a single bug)
<wgrant> Yes, I believe.
<wgrant> And to make it logical.
<wgrant> eg. ordering by importance, the first 200000 bugs or so are a single importance.
<wgrant> And having them arbitrarily ordered would be odd.
<wgrant> Those indices are just a start, but they satisfy all the common bug search cases.
<stub> But no need to pull in product.name and friends
<wgrant> We may eventually want something more selective on status, duplicateof, etc.
<wgrant> No.
<StevenK> Is this the review of bugtaskflat?
<wgrant> There are some orders that I'm probably going to ban on large sets of bugs.
<wgrant> Assignee name, tag, location name, etc.
<wgrant> They're not useful, and very expensive on large sets.
<wgrant> And for small sets it's fine to do the sort as the last stage.
<stub> I think we should consider telling the user they are being nonsensical if the query results are nonsensical. If the first 200000 bugs are a single importance, the answer isn't important because it is rather useless.
<wgrant> stub: Indeed.
<wgrant> stub: But that's the default sort order :)
<stub> Yes, the principal remains :)
<wgrant> stub: Half these indices are to satisfy eg. https://launchpad.net/launchpad/+bugs
<stub> Just wrapping my head around what we have here
<stub> StevenK: Yes
<wgrant> With just a straight walk down the index.
<wgrant> In 2-3ms.
<wgrant> Person:+bugs is going to be interesting. It's not clear what indices are best there.
<wgrant> Fortunately we have feature flags to disable BugTaskFlat use where it doesn't work yet :)
<wgrant> s/doesn't work yet/turns out to be slow sometimes without further indices/
<wgrant> The indices *will* need further work, but this seems to be a good start for most things.
<stub> wgrant: So Celery can use PostgreSQL as a backend. Perhaps these triggers should just launch Celery jobs :-)
<wgrant> These are about 100x faster than bugsummary in some cases, and I am rewriting bugsummary to be less insane, so it will be a net speedup :)
 * cjwatson would love review of https://code.launchpad.net/~cjwatson/launchpad/proposed-as-staging-pocket/+merge/99911
<cjwatson> Anyone mind if I upgrade mawson?
<wgrant> cjwatson: Go ahead.
<wgrant> Hopefully the DB upgrade won't break.
<cjwatson> Anything I need to watch out for in particular?  Or is this the stuff that's been afflicting NDTs?
<wgrant> Nah, just mawson sometimes misses some of the migrations.
<wgrant> And occasionally gets manual BugTaskFlat patches that could conflict.
<wgrant> But I think it should be OK
<wgrant> If it breaks, yell and I'll fix.
<cjwatson> running
<StevenK> cjwatson: Only if you happen to 'upgrade' to something that's 100 revisions behind.
<wgrant> 150, TYVM
<StevenK> Indeed.
<StevenK> That was a fun morning.
<cjwatson> How did that happen?
<StevenK> cjwatson: Worse, it was on prod.
<wgrant> 'tis a mystery
<StevenK> And we're in the middle of two rather large migrations for disclosure.
<cjwatson> hmm, Text conflict in lib/lp/scripts/garbo.py
<wgrant> bzr resolve --take-other
<wgrant> That's from when I was manually running a single garbo job to complete a migraiton
<cjwatson> mkay
<cjwatson> mawson's bzr is too old for resolve --take-other, I used revert instead, hope that was ok
<wgrant> Yeah, that's fine.
<wgrant> cjwatson: You win the award for lowest SNR in a Launchpad merge proposal, but approved :)
<cjwatson> wgrant: I assumed everyone else's merges were like that nowadays too as they desperately scrabbled for LoC.  Maybe not
<wgrant> cjwatson: We tend to split branches when the get like that. It's not a strict rule that every *branch* must not be LOC-positive.
<cjwatson> OK, I'll try to remember that.  I guess I find it hard to keep track otherwise
<wgrant> cjwatson: Indeed, that's the problem :)
<wgrant> cjwatson: Can you set a commit message?
<cjwatson> done
<cjwatson> Ah, mawson at least thinks it's done now.
<cjwatson> http://people.canonical.com/~cjwatson/tmp/loc.png - I was curious what LoC delta per revision actually looks like since the new policy ...
<cjwatson> different view from ohloh's
<wgrant> I've been meaning to get around to repurposing community-contributions.py to provide some kind of analysis like that.
<cjwatson> net +1901 since new policy
<cjwatson> which is not really obvious from that graph
<bac> so, perhaps we should limit the length of a bug report title:  pad.lv/943974
<rick_h> hah, that's great.
<stub> I think the limit in the db is 2GB
<nigelb> wtf
<nigelb> that is a big title.
* jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: jcsackett, rick_h* | Firefighting: - | Critical bugtasks: 4*10
<deryck> I'd be happy to help get another deploy today, but I must admit I have no clue how to qa cjwatson's changes.
<deryck> I think all of our soyuz experts are australian now, too.  hmmmm
<cjwatson> deryck: I just finished QAing it.
<deryck> cjwatson, ah very good!  Thanks, man.
<deryck> adeuring, now we're just waiting on your revision.
<cjwatson> I'd like that deployed on ftpmaster at some point; I don't know what the deployment procedures for that look like these days.
<adeuring> deryck: argh, got distracted...
<deryck> adeuring, np. :)  Can you ping me when it's done, so I can put up a ndt deploy request?
<adeuring> deryck: sure
<deryck> cjwatson, I don't know either.  But I'll check into it when I put the deploy request up.
<cjwatson> ftpmaster (well, the bit of it that matters) is currently on 15032, so not that far back.
<cjwatson> OK, LPS says it's still a separate 5-minute window.
<deryck> ah yeah, just saw that too.
<deryck> so we'll have to schedule that.
<deryck> mrevell, we're doing a deploy later today that we need deployed to ftpmaster, and that needs 5 minute downtime.  How do we schedule that now?
<deryck> mrevell, do you or danhg set that up?
<mrevell> deryck, Hey, for five minutes just ask czajkowski to announce it to the status feed. Antu
<mrevell> deryck, Anything more than that and I can find out when would have least impact on our stakeholders and users generally.
<deryck> mrevell, ah ok.  I'll schedule with webops then and ping czajkowski when I know when they can do it.
<czajkowski> deryck: just let me know the details more than 5 mins before hand and then I can get it out
<deryck> czajkowski, will do, thanks
<cjwatson> wgrant: hm, I meant to ask, did you feed my branch to EC2?
<cjwatson> I guess I'll find out by the time you read this.
<cjwatson> wgrant: ... never mind. :-)
<sinzui> rick_h, jcsackett. Do you have time to review https://code.launchpad.net/~sinzui/launchpad/obsolete-js/+merge/99978
<czajkowski> deryck[lunch]: am heading offline for EOD but will be back later if you want an annoucement put out.
<rick_h> sinzui: will look at it now
<deryck> czajkowski, ack.  I think it will likely be done tomorrow now.  waiting on QA for a branch.
<czajkowski> deryck: ok well mail me and I can still annouce it in places for tomorrow
<deryck> czajkowski, ok, thanks.
<sinzui> rick_h, can you clarify the space issue now that I see it? I see two spaces...but I like code to be 4 spaces. Do you want me to make it 2 or 4 spaces
<rick_h> sinzui: sorry, mean the space between the keyword 'function' and the parens
<rick_h> sinzui: I think most LP people have no space between them, my background nad jslint are for a space
<rick_h> sinzui: complete nitpick thing
<sinzui> okay. I see that.
<rick_h> other than that, thanks for finding and cleaning that up. One less thing for my long term JS clean up todo list :)
<jcsackett> sinzui: i have followed up on rick_h, and have nothing to add. r=me.
<sinzui> thanks jcsackett. I just extracted the script and ran it though lint. There were some == issues that also needed fixing
<rick_h> doh, I should have caught those. Evil JS in .pt files...
<rick_h> sinzui: have a sec for a quick ? on the subscribers stuff?
<sinzui> sure
<rick_h> sinzui: I'm trying to see where/how you test that once you have the notification bits working, that the event is bound correctly/working in the actual app layer through your configure.zcml bits?
<rick_h> all the tests manually generate or call the event method that gets fired
<sinzui> sorry, thunderbird-aka-shitehawk was stealing focu
<sinzui> s
<sinzui> rick_h, we have a utility that subscribes to the event that allows us to see it what was passed.
 * sinzui looks for it in answers or registry
<sinzui> rick_h, sorry, this time I could not find. it We use TestEventListener
 * sinzui looks for real example of use
<rick_h> ok, will search for that thanks
<rick_h> that's ok, I can peek, just need a hint. Wasn't seeing it in the existing tests I was looking at
<sinzui> rick_h, http://pastebin.ubuntu.com/905984/
<sinzui> ^ the setup is odd because of the registration/deregistration of the listener
<rick_h> ty much sinzui
<wallyworld> sinzui: i *might* be a few minutes late to the standup. i have to drive my wife to work. hopefully traffic won't be too bad
<sinzui> okay
<sinzui> wgrant, wrong protocol on port -> http://pastebin.ubuntu.com/906352/
<wgrant> sinzui: Yeah, that's speaking HTTPS to an HTTP listener.
<sinzui> StevenK, I was thinking of bug #367533
<_mup_> Bug #367533: Collapsible fieldsets don't degrade gracefully when Javascript isn't available <css> <easy> <javascript> <lp-web> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/367533 >
<jelmer> g'd evenin' launchpadders
<wgrant> Morning jelmer.
<sinzui> StevenK, and bug #677339
<_mup_> Bug #677339: filebug per-project help is below the fold <confusing-ui> <easy> <lp-bugs> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/677339 >
<StevenK> wgrant, sinzui: https://www.destroyallsoftware.com/talks/wat
* wallyworld_ changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: wallyworld | Firefighting: - | Critical bugtasks: 4*10
#launchpad-dev 2012-03-30
<wgrant> lifeless: Do you want to look at https://code.launchpad.net/~wgrant/launchpad/bugtaskflat-db/+merge/96520 before it lands?
<bigjools> launchpad-developer-dependencies is still uninstallable for me on precise
<StevenK> bigjools: Pastebin the error?
<bigjools> http://pastebin.ubuntu.com/906456
<StevenK> bigjools: That's archive skew.
<StevenK> IE, nothing to do with the LP PPA
<bigjools> StevenK: it's been skewed for >24h then
<StevenK> Which mirror are you using?
<bigjools> .au
<StevenK> Hm. Then I think .au is lagged.
<bigjools> :(
<wgrant> Is it still aarnet?
<wgrant> yes
<StevenK> Yeah. See -ops
<lifeless> wgrant: if stub is happy, I'm happy. There is a tonne of code :P
<bigjools> lifeless: just saw your messages on #twisted.  PPA email is a total crock right now, it really needs an opt-in subscription mechanism.
<bigjools> (of varying email types)
<lifeless> bigjools: yes, I'd like to make it easy to do that
* wallyworld_ changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugtasks: 4*10
* kornbluth.freenode.net changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: wallyworld | Firefighting: - | Critical bugtasks: 4*10
<adeuring> good morning
<czajkowski> aloha
<jtv> mrevell: are you familiar with the password-form controversy?  https://bugs.launchpad.net/maas/+bug/944565
<_mup_> Bug #944565: User details and password are separate forms <MAAS:Triaged> < https://launchpad.net/bugs/944565 >
<jtv> You said a few weeks back that we ought to have a discussion about it.
<jtv> Now I'm looking to implement the UI for an admin changing a user's password, and so the same issue is relevant there as well.
<jtv> (And on a largely irrelevant sidenote, who came up with the password confirmation field that directly follows the original password input field?  Just right for your fingers to make the same mistake twice, or to send the whole thing in and forget about it before the choice of password has reached your brain)
 * cjwatson upgrades mawson for some more soyuz qa
<wgrant> cjwatson: Thanks.
<jml> jcsackett: thanks for getting that patch landed.
<jml> anyone interested in releasing lazr.restfulclient?
<gmb> danhg, There are now two Juju-related blog posts for you to take a look at. The split turned out to be pretty trivial, actually.
<czajkowski> gmb: oh nice looking forward to reading
<danhg> That's great gmb, thanks!
<czajkowski> gmb: got pics in them as well ::) helps when I add them to the fb/G+ pages :)
<gmb> czajkowski: Ta. And I made no mention of a certain breakfast cereal, either. No pics yet; we'll have to find some.
<gmb> (I did make a quark-related pun, though).
<danhg> K
<danhg> sorry, that wasn't meant to be anything
<czajkowski> gmb: hahah
<danhg> Puns are always welcome!
<wgrant> Shouldn't the old client do?[B[B
<wgrant> Bah
<wgrant> Lag
<gmb> czajkowski, I think bug 969024 is a dupe. That's a problem that's been around for a long, long, long time.
<_mup_> Bug #969024: Remote watch has two times the 'None, the status of the bug is updated manually' option <Launchpad itself:New> < https://launchpad.net/bugs/969024 >
<czajkowski> gmb: any idea of the original bug :)
<gmb> Looking for you now :)
<gmb> I'm not that much of a git :)
<gmb> czajkowski Well, it's certainly a dupe of bug 931573
<_mup_> Bug #931573: there are two radio buttons each offering "None, the status of the bug is updated manually" <Launchpad itself:Triaged> < https://launchpad.net/bugs/931573 >
<gmb> But I'm sure there's an older version...
<czajkowski> gmb: cheers
<gmb> czajkowski: Bug 516799 is the earliest one I can find. Let's make that the canonical one.
<_mup_> Bug #516799: BugTask +edit has duplicated option for tasks assigned to a BugWatch <confusing-ui> <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/516799 >
<czajkowski> what do you mean 'canonical one'
<gmb> Also, I will buy a beer for anyone willing to fix it (or at least paypal them some money with which to buy a beer).
<gmb> czajkowski: Canonical in the sense of "if we see this bug get filed again, we mark it as a dupe of Bug 516799."
<_mup_> Bug #516799: BugTask +edit has duplicated option for tasks assigned to a BugWatch <confusing-ui> <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/516799 >
<gmb> So bugs 969024 and 931573 are dupes of 516799
<czajkowski> grand job
<gmb> czajkowski: Should I take care of the duping, since I'm here already?
<czajkowski> nope
<czajkowski> done
<gmb>  Coolio.
<gmb> Ugh, not saying that again.
<czajkowski> now to tackle the other bugs this morning
<czajkowski> am rotating between my check list
<czajkowski> onto bugs now
<gmb> I'm still willing to bet Â£5 that bug #1000000 is from mpt and reads "Launchpad still has too many bugs"
<mpt> gmb, shush, it was supposed to be a surprise
<gmb> :)
<czajkowski> so gonna makr that one invalid :p
<czajkowski> *mark
<gmb> Haha.
<gmb> czajkowski: Mark it "opinion".
<gmb> Or did that get taken out yet?
<mpt> alas no
<czajkowski> have you reported a bug about opinion mpt :)
<mpt> czajkowski, no, Martin Pool did, bug 772954
<_mup_> Bug #772954: "Opinion" bug status causes user confusion <opinion-status> <Launchpad itself:Triaged> < https://launchpad.net/bugs/772954 >
<czajkowski> so tempting to change the status to opinion there
<czajkowski> :)
<mpt> I could provide citations to get it out of that state
<mpt> Most notably, the people who currently use opinion unwittingly disagree about what it means
<czajkowski> it's a better status than won't fix which royally gets on nerves.
<mpt> Some think it means "Well, that's your opinion, but we aren't going to do that" (a synonym of Won't Fix), while others are on record as saying they use it for "We'll come back to this issue in future"
<mpt> (a synonym of New)
<mpt> (or more precisely, a synonym of Unconfirmed, which Launchpad weirdly calls "New")
<wgrant> mpt: Didn't you preside over that particular renaming? :)
<wgrant> (also, I think one could safely s/Opinion/sabdfl/ :))
<mpt> wgrant, heck no, I've always complained about using New for years-old reports and Triaged for untriaged ones
<wgrant> I thought you were still around then.
<mpt> I don't actually remember who changed the names
<mpt> I think it was at the same time Triaged was introduced
<wgrant> I believe so.
<wgrant> And Needs Info -> Incomplete
<mpt> yes
<mpt> (which is now also confusing primarily because Unity misuses it)
<wgrant> What do they use it for? :(
<wgrant> Maybe we should enforce aggressive expiration just to prevent people from abusing it.
<mpt> They use it for "we're waiting for a designer to decide how to fix it"
<wgrant> Bah
<wgrant> thumper isn't here to harass about his team abusing Launchpad :(
<mpt> Yeah, I'd be in favor of always expiring Incomplete bugs
<wgrant> Bad thumper.
<mpt> If you don't want expiry, don't use Incomplete
<mpt> Bug statuses need a purpose, not just a meaning
<wgrant> Indeed.
<wgrant> And Opinion has neither :)
* bac changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: bac | Firefighting: - | Critical bugtasks: 4*10
<deryck> Morning, all.
<rick_h> party
<nigelb> rick_h: TGIF, eh?
<rick_h> nigelb: definitely
<deryck> rick_h, ping for standup.
<rick_h> deryck: ack, sec
<adeuring> abentley: mumble?
<deryck> do we not serve feeds off qastaging?
<sinzui> I was going to fix some IE bugs today, but it looks like I do not have enough disk to run  everything :( I really do not want to buy a hard drive to run a browser I would never use
<rick_h> sinzui: run off an ec2 windows machine and expense?
<deryck> ewww, yeah, that would suck to do.
<sinzui> rick_h, I need it to talk to my local dev instance...I am not putting my lovely computer on the naked net
<sinzui> MS provides ideal VMs for their browser, I just do not have enough disk space to accommodate their bloatware
<rick_h> sinzui: gotcha
<deryck> sinzui, wine?  I've run IE via wine before pretty easily.
<deryck> apt-get install wine and winetricks then run `winetricks ie9` or whatever version you want.
<sinzui> Dude! MS provides the real thing with real OS to get the real engine parts that users have
<deryck> sinzui, right, I understand that.  I was just thinking of how you might without being able to run the VM.
<sinzui> deryck, 90% of Lp's IE users are IE8
<sinzui> IE9 being an es6 engine mostly works...the moronic guard in our code are disabling IE9 with actually works
<sinzui> I tried a thumbdrive, but I think it is too slow. the same vm that runs on my harddrive will not run on the thumbdrive
<sinzui> IE9 being an es5 engine
<deryck> sinzui, do you know if feeds should work off qastaging.
<sinzui> deryck, they should, but I have never tested that they do. They have special configuration and it may not have ever been done
<sinzui> deryck, they worked in staging so if staging is unhappy, then feeds are really broken
<deryck> sinzui, hmm, I'm guessing that's it then.  http://feeds.qastaging.launchpad.net/~deryck/latest-bugs.atom is just completely unavailable.
<deryck> I guess I can mark it qa-untestable since it's flag guarded in production.
<sinzui> deryck, they are not configured to work :(
<sinzui> deryck, Is the change to staging yet. Feed has a https server setup for them there
<deryck> sinzui, let me see, but I doubt it....
<abentley> adeuring: I had a thought about how we could manage routing keys with fast-lane/slow-lane.  The fast lane could be the root name.  So "job.standard" or "job.branch_write".
<jcsackett> sinzui: re ie9, could you get the vm running on your older machine? then you would just need to have lp.dev setup for remote on your network.
<adeuring> abentley: sounds good
<abentley> adeuring: The slow lane's routing key would tack on ".slow".  So, "job.standard.slow" or "job.branch_write.slow".
<sinzui> jcsackett, My old computer cannot fit Ms's smallest installation
<abentley> adeuring: This is extensible for the number of slow lanes we have.  e.g. with 5 slow lanes, we get job.standard.slow.slow.slow.slow.slow :-)
<adeuring> abentley: ;)
<deryck> sinzui, ah it is caught up on staging.  awesome, thanks for the tip.
<czajkowski> gmb: nice post!
<sinzui> deryck, wine tricks informs me that it does not support 64bit
<deryck> sinzui, ah, I must have wget it from wine itself.  it's just a shell script.
<sinzui> deryck, but something is running. I guess I need to find an ichoicesource to see if it really breaks as I expect
 * deryck has spent a lot of time with wine lately trying to get swtor running
 * abentley likes the double meaning of "spent a lot of time with wine"
<sinzui> deryck, I cannot log into Ubuntu SSO using the wine trick browser. I could using the bloated vm :(
<deryck> ah bummer.
<deryck> abentley, heh! :)
<abentley> deryck: I'm sure when they named that project, they didn't mean it would drive you to drink.  But sometimes...
<deryck> most times! (if you're a gamer) :)
<deryck> ok, so I've finished qa and going to put together a deploy request now.
<sinzui> Ah, I see. ie8 did fail to install. This is ie6
<deryck> well, actually it is Friday. hmmm
<deryck> sinzui,  I think if you wget the winetricks from wine itself, all will work again.
<deryck> I forgot I had done this.
<sinzui> I will try that now
<sinzui> well I give up. I cannot get wine + ie8 to accept self=signed certificates
<czajkowski> sinzui: you should talk to YokoZar he is Mr. wine
<sinzui> I could. I want to run MS's vm which provides a real foundation to test. I just do not have enough disk space
<YokoZar> I have been summoned.
<czajkowski> YokoZar: meet sinzui win is not being nice
<czajkowski> perhaps you could help, maybe
<YokoZar> what do you mean
<sinzui> YokoZar, I am trying to test wine + ie8 against a development instance of Launchpad which uses a self-signed ssl certificate
<YokoZar> winetricks ie8 is a good start
<abentley> adeuring: It appears that only celeryd creates rabbitMQ queues, so be wary of that that in your tests.  I just had to add a sleep() to ensure that celeryd finishes starting up before I try to dispatch tasks.
<abentley> adeuring: However, you may not be affected, since the celeryd tests use the system rabbit, so the queues would be preserved for you.
<adeuring> abentley: ah, ok. thanks for the heads up!
<abentley> adeuring: np
<sinzui> YokoZar, I started with that. I can see the real Lp, but the development instance uses self-signed certificates. There is no Internet tools to walk through the menus to add my certs to the trusted certs
<YokoZar> sinzui: you mean the ie8 preferences windows and such don't work?
<YokoZar> Yeah I'm not sure
<deryck> sinzui, can you not open the internet settings and relax the permissions from the browser settings?
<sinzui> deryck, I do not see how to find the internet settings. Though the browser signature is IE8, It looks like Ie6 with no menus
<deryck> sinzui,  run: winecfg and see if the settings are in there.  if not, `wine control` will bring up the control panel and you can do it from there.
<sinzui> deryck, in real windows, relaxing the permission does not work (As I understand it), you must register the cert
<deryck> it sounds like ie8 didn't get installed correctly though
<deryck> sinzui, what do you use in real windows to do this?  I don't recall doing this step before, but maybe I only tested on production.
<sinzui> deryck, the browser has a link to internet options in the menu and the toolbar. It is absent in wine
<sinzui> deryck, I cannot login to lp/ubuntu SSO does not work with wine+ie8
<deryck> sinzui, I belive if you run `wine control` it should open a panel with "internet settings" showing as an option there.
<sinzui> I see it!
<rick_h> 016524
<rick_h> bah
<sinzui> deryck, I have installed Lp's certs and IE can see the lp dev instance.
<deryck> sinzui, nice!
<sinzui> deryck, but I think I need to remake the cert because I borked the login domain name testopenid.dev
<lifeless> 'morning
<sinzui> deryck, I now have a small consolation that the login failure I see on production servers is also see on dev. wine + IE8 is not sending the referrer header.
 * sinzui has an enormous head ache.
<deryck> ah interesting.
<deryck> morning lifeless
<rick_h> 043207
* bac changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugtasks: 4*10
#launchpad-dev 2012-03-31
<aputtu> Will there be comment field added to the translation process? I'm trying to find the proposed tasks for dev.launchpad.net/translations - I've read https://wiki.ubuntu.com/OgMaciel/blueprints/rosetta.
<aputtu> Also, I'd like to know where to file ideas to dev.launchpad.net/translations. For example as a reviewer I'd like a filter that filters out entries where I have made translation suggestions myself.
<aputtu> Regarding the comment field, I read that this is included in the user story, so it's more a question on if and when. I know that the Danish team avoids Launchpad in some projects, as the comment function is crucial to the process.
<aputtu> Well, okay - I've filed a bug on the comment field here: https://bugs.launchpad.net/launchpad/+bug/970414
<_mup_> Bug #970414: Launcpad/translation: Add comment during translation process <translation> <Launchpad itself:New> < https://launchpad.net/bugs/970414 >
<aputtu> I'll just file another one about the request for extended filtering.
<aputtu> As for the request for an extended filter, I've filed a bug report on this also: 970423
<cjwatson> you probably won't find too many developers around here at the weekend, fwiw ...
<cjwatson> but bug reports are generally the right place for feature requests, yes
<cjwatson> merge proposals get things done faster but are rather more work :-)
<aputtu> great, I've noticed that tags have been added already - I wasn't aware of the existing tags and I'll add "lp-translations" and "feature" as well.
#launchpad-dev 2012-04-01
<lifeless> mpt: hi
<mpt> hi ho
<lifeless> mpt: can you confirm, how do you believe Opinion operates vs WontFix ?
<mpt> lifeless, do you mean, how do I believe people use it, how do I believe people should use it, or how do I use it?
<lifeless> how does launchpad treat it
<lifeless> you say 'reports that are currently "Opinion" can't just migrate to "Won't Fix", because that would hide many reports that the developers considered to be valid. '
<lifeless> but Opinion bugs are already closed as far as Launchpad is concerned.
<lifeless> This leaves me a little confused.
<mpt> lifeless, I went solely on their comments when they set the status. Maybe they don't realize Launchpad is treating it as closed.
<lifeless> thanks
<lifeless> that lets me understand enough to reply sensibly :)
<mpt> lifeless, this is something I've been meaning to do ever since John Lea  explained in a UDS session that "Opinion" doesn't mean "Won't Fix", it means "we know this is an issue and we're going to look at it later".
<lifeless> yeah, it means the same as 'invalid' and 'wontfix' as far as Launchpad is concerned.
<mpt> Unfortunately I haven't found a citation for that, it's just my personal memory, but it turns out to match up with how the developers often use it.
<mpt> If Launchpad itself treats "Opinion" identically to "Won't Fix", that in itself is a reason they shouldn't both exist -- but then the same is true for "Invalid". :-)
<lifeless> exactly
<mpt> (I know BjornT explained once that there is a small difference in behavior between "Won't Fix" and "Invalid", but I don't remember what it is.)
<lifeless> thats why all the refresh-of-bugs proposals include consolidating those things
<lifeless> there is no such difference
<mpt> It wasn't to do with normal bug searching, it was something more esoteric
<lifeless> I believe the current consensus is that LP should let folk choose the behaviour they want and allow semi-arbitrary explication
<lifeless> e.g. want the bug not shown to developers by default - thats 'closed', and whether it was fixed/rejected/might come back later is data the bug tracking system doesn't need to really know about
<lifeless> this suggests two core states for bugs - open vs closed, and a separate workflow layer - 'next action by: [Triage/developer/architect|design/qa]'
<lifeless> mpt: there may be an ACL on whether a bug can be reopened, I'm not sure.
<lifeless> mpt: (but I'm fairly sure invalid and wontfix are identical there)
<mpt> lifeless, "Next action by:" = assignee
<mpt> There are seventy-odd bugs assigned to me right now, and all of them I'll reassign/deassign when done, because they're assigned to me only to do the design on them
<lifeless> mpt: there is handwaving involved :)
<mpt> When Google Code Hosting started out it had only "Open" vs "Closed", though it had better tags
<lifeless> mpt: consider the case when e.g. mvo is tasked with a bug, but decides he needs more input from the reporter; how does reporter know to hand it back to mvo? [And how do they get the privileges to do so]
<mpt>  I think it may have expanded since then
<lifeless> mpt: it would be nice to avoid losing datum (that mvo is tasked with the bug) when a temporary handoff is needed but mvo is *still responsible*.
<mpt> lifeless, https://dev.launchpad.net/IssueTracker#Delegation
<lifeless> yeah
<lifeless> just noting that this != assignee
<mpt> So you've just made me realize a flaw in my mental model
<lifeless> or at least, its not as trivial as saying that
<lifeless> mpt: cool!
<mpt> That one status == one behavior in Launchpad
<mpt> Let's say you find a bug report that was marked as "Closed" without comment. You download the new version and the bug still exists. Should you reopen it?
<lifeless> so my goal here is to identify the minimal set of primitives in the system that will let folk build robust, automatable processes on top of them. Less primitives => lower learning curve and smaller UI footprint.
<mpt> If it was marked as "Declined", no, the developers decided the behavior should stay as it is. If it was marked as "Fixed", yes you should, because they meant to fix it but didn't fix it properly.
<mpt> So that's a difference in behavior, just not inside Launchpad itself.
<mpt> The only way you could tell that from "Closed" was if the developer commented to explain which kind of Closed it is, and they shouldn't have to do that.
<lifeless> yeah. Some developers might prefer a separate bug report in those situations, because it might be a different root cause or something. But that seems outside LP's purview.
<lifeless> I think one key thing we need to do is get away from the model that a bug report is roughly equivalent to a mailing list
<lifeless> it frames things poorly
<mpt> All of that to say that I think it's more efficient to have "Fixed" (= Fix Committed + Fix Release) and "Declined" (= Invalid + Won't Fix + Opinion + Expired) as separate statuses
<mpt> +d
<lifeless> I note an assumption in your reasoning
<mpt> do tell
<lifeless> that status U ('open', 'closed') implies a comment is needed to differentiate between ('fixed', 'declined')
<lifeless> if you imagine the current status rows
<lifeless> which has a wide status field and a wide importance field
<mpt> lifeless, re. making it less like a mailing list, absolutely -- that's why I reported bug 1734, bug 520405, and bug 555293, for example
<_mup_> Bug #1734: Bug discussion is append-only and cannot be gardened <feature> <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1734 >
<_mup_> Bug #520405: Canned "this is a duplicate" response wastes time - launchpad itself could explain about the duplicate nature and process (and avoid storing the redundant copies of this message) <degreasing> <lp-bugs> <story-better-bug-notification> <Launchpad itself:Triaged> <Launchpad Greasemonkey Scripts:Triaged> < https://launchpad.net/bugs/520405 >
<_mup_> Bug #555293: Canned "This bug did not have a package" reponse wastes time <degreasing> <lp-bugs> <story-better-bug-notification> <Launchpad itself:Triaged> < https://launchpad.net/bugs/555293 >
<mpt> lifeless, I suppose there are other ways you could tell the difference between Fixed and Declined -- for example, whether the trunk branch or a release ended up associated with the report. But Launchpad would still need to do work to get from that to a clear answer to the question, "Should I reopen this if I still see the bug 47 days later?"
<_mup_> Bug #47: Add -F (Force) and -p (sourcePackage) flags to nicole <lp-foundations> <Launchpad itself:Invalid> < https://launchpad.net/bugs/47 >
 * mpt pats _mup_ on the head
<lifeless> mpt: do you think users will listen to such a hint? (where users could be users of LP or users of Ubuntu or whatever)
<lifeless> mpt: for instance, I tell folk that interact with my projects to never ever reopen a bug. I want them to always open a new one, no matter the resolution of the prior one, unless they know a-priori its the same root cause.
<lifeless> (Which is often impossible without being deeply into the code)
<lifeless> mpt: I do this because I can only merge bug reports effectively, not split.
<mpt> lifeless, oh, sorry, I'm using "reopen" as short-hand for "reopen if the existing report is wieldy, re-report if it isn't"
<lifeless> how much easier or useful would this make LP?
<mpt> For example, last week bug 949414 was marked fixed, but it still occurred for me, and so (for the reasons you just gave) I re-reported it as bug 969076
<_mup_> Bug #949414: Some GTK+3 events are not emitted when using a touchpad (but are with a mouse) <rls-mgr-p-tracking> <rls-p-tracking> <elementary OS:Triaged> <GTK+:Fix Released> <gtk+3.0 (Ubuntu):Fix Released> <gtk+3.0 (Ubuntu Precise):Fix Released> < https://launchpad.net/bugs/949414 >
<_mup_> Bug #969076: With touchpad, non-first menu items don't highlight and submenus don't open until clicked <gtk+3.0 (Ubuntu):Confirmed> < https://launchpad.net/bugs/969076 >
<lifeless> (this == telling folk whether to reopen or not via LP status data)
<mpt> lifeless, I think it's very useful, but maybe I work around the sort of reports where correct behavior is more debatable than it would be in other kinds of software, so it would be harder to tell whether a Closed report was fixed or not
<mpt> I don't know that you could easily measure how useful the distinction is, because if sequel bug reports reference the original, they do so as plain text
<lifeless> we will be able to model that in future
<mpt> (And even if you managed to measure it, you'd have a post hoc ergo propter hoc problem: you'd know only how often people reopened/re-reported a bug *after* it was marked fixed, it wouldn't tell you how often they did so *because* it was marked fixed.)
<lifeless> did you know that Fix Released bugs are not reopenable except by bug controllers
<lifeless> this was done because too many Fix Released bugs were reopened and developers wanted new bugs.
<lifeless> s/too many/sufficiently many/
<cjwatson> splitting bugs in LP wouldn't necessarily be horribly difficult
<cjwatson> doing it with total flexibility is hard, but fork-history-at-this-point is relatively simple
<mpt> lifeless, I have occasionally known, when I re-report a bug that was mistakenly marked fixed in a project I don't otherwise contribute to. But usually I am in a state of having forgotten that. :-)
<cjwatson> though ideally it'd be more like "take this set of comments and hive off to new bug"
<lifeless> mpt: so I think we have data that developers do care about reopen/open new - sufficiently strongly that LP implements a policy supporting 'always open new'
<lifeless> cjwatson: its not rocket science, as they say, but it does need work - and needs to handle dealing with things like bug 1.
<_mup_> Bug #1: Microsoft has a majority market share <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Confirmed for compscibuntu-bugs> <LibreOffice Productivity Suite:New> <dylan.NET.Reflection:Invalid> <dylan.NET:Invalid> <EasyPeasy Overview:Invalid by ramvi> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <LibreOffice:In Progress by bjoern-michaelsen> <Linux:New> <Linux Mint:In Progress> <The Linux OS Project:In Progress> <metacity:In Prog
<mpt> lifeless, okay, but that's still a side issue from the general point of how a tester can tell *whether* a bug was supposed to have been fixed or not.
<cjwatson> wontfix/invalid> I certainly implemented something that made ACLs on those different
<lifeless> mpt: I think its totally different
<cjwatson> which Contributions tells me was bug 294846
<_mup_> Bug #294846: Setting to Won't Fix is ACLed but unsetting it isn't <lp-bugs> <qa-ok> <ubuntu-qa> <Launchpad itself:Fix Released by cjwatson> < https://launchpad.net/bugs/294846 >
<mpt> lifeless, what is totally different?
<lifeless> mpt: there are two cases where someone other than the developer who closed the bug will encounter it: a) they go looking for it and b) they encounter it spontaneously.
<lifeless> mpt: case a) is software testers - whether crowd sourced or staffed. They *know* that it was meant to be fixed - that is why they are going looking for it.
<lifeless> mpt: case b) *sometimes* happens to people that are software testers but also happens to people that are not.
<lifeless> mpt: why should case (b) be treated differently depending on whether the person experiencing it is a software tester or not ?
<mpt> lifeless, I disagree that software testers necessarily know whether a bug was meant to be fixed.
<lifeless> mpt: in case (a) they always do, because its been handed to them to verify that it is fixed.
<mpt> lifeless, that's only one of the things testers do, though. Often they also report things that seem wrong to them. Often they're right. :-)
<lifeless> mpt: indeed, but that is case (b)
<mpt> I don't understand what you mean by "case (b) be treated differently depending on whether the person experiencing it is a software tester or not".
<lifeless> mpt: AIUI you are arguing that a software tester that spontaneously encounters a previously reported & closed bug needs to take different action depending on whether the bug was closed-fixed or closed-declined
<lifeless> mpt: clearly non-testers cannot take different actions, because they are prohibited from reopening closed-fixed bugs
<mpt> lifeless, I'm arguing that *anyone* in that situation should do that.
<lifeless> mpt: but developers don't want them to be able to
<mpt> What?
<lifeless> mpt: reopening fixed bugs is a privileged operation
<lifeless> s/fixed/'fixed'
<mpt> lifeless, so we're still side-tracked. Once again, this is nothing to do with whether they have permission to reopen the bug report or whether they make a new bug report.
<lifeless> I'm not sure I follow, but please continue :)
<mpt> lifeless, my point in wanting Fixed and Declined to be separate is that it tells someone, who comes across the bug report and currently experiences the behavior that it complained about, whether the developers thought the complained-about behavior was correct or not.
<mpt> If the developers thought the behavior was incorrect, and they did something and marked it Fixed, that means the person who comes across the same behavior later should go and report it again (assuming your never-reopen world).
<lifeless> thats certainly one way to guide users.
<mpt> If the developers thought the behavior should not be changed, and they marked it Declined, that means the person who comes across the same behavior later should just sigh and test something else.
<lifeless> there are other ways to provide that guidance - faqs, docs, editing the bug subject
<mpt> Yes, but those other ways uniformly involve more clicks and/or keypresses. :-)
<lifeless> mpt: I'm basically just ruminating on how minimal we can be without negative side effects.
<mpt> sure
<lifeless> mpt: I will note that there are folk that want to require comments on closing of bugs; e.g. folk see *value* in human commentary on bugs.
<mpt> and I'm rummaging for babies in this bathwater
<mpt> lifeless, yes, I commented on that too a couple of days ago, with data from bugzilla.mozilla.org
<lifeless> yah, I saw that go by
<lifeless> assertion: the only reports where number of clicks is a dominating component of the effort involved is declined bugs
<lifeless> actioned bugs, assuming automated stuff like Ubuntu has, have most of the effort happen in the code change.
<mpt> I have a counterexample for that assertion
<lifeless> mpt: cool!
<mpt> Every so often I will see bugmail from seb128 that looks something like this:
<lifeless> mpt: what is it?
<mpt> Importance: Undecided -> Wishlist
<mpt> Status: New -> Confirmed
<mpt> *** This bug has been marked as a duplicate of XYZ ***
<mpt> which makes no sense, until I realized he was using a greasemonkey script to make marking duplicates faster
<mpt> The problem is that if the duplicate was mistaken, and someone just reopens the bug report, it's at Confirmed and Wishlist when it hasn't actually been seriously triaged.
<lifeless> so, why is he marking it wishlist confirmed at all ?
<mpt> I don't know. :-)
<mpt> (Actually it might be Wishlist Invalid ... I'd need to search bugmail to be sure)
<lifeless> so, ongoing food for thought
<lifeless> Duplicates shouldn't be open or closed
<lifeless> they are indeterminate
<mpt> yes
<mpt> hence, greyed-out status table
<lifeless> mpt: have you seen my ruminations about moving to a 'symptoms' and 'fixes' model, rather than the simple 'bugs' we have today?
<mpt> (which was a tradeoff between being completely indeterminate, and showing enough info that you could tell whether it would save time to reverse the duplicate direction)
<mpt> lifeless, I have not ... That sounds like a bit like the very early days when there were separate pages (and numbers!) for bugs and tasks.
<lifeless> mmm, haven't sketched at that level
<lifeless> this comes out of the 'bug reports with imperatives are not bug reports' concept
<mpt> Hypothesis: Any bug report that has identical symptoms to another bug report is either a duplicate or incomplete.
<lifeless> yes
<lifeless> so the idea was to strip out all the stuff related to what changes are needed and what a fix/nonfix is from the things that users tell us about
<lifeless> rather than asking for a bug title we would ask for 'what happened', and the docs could guide folk towards reporting evidence in preference to speculation / desire
<mpt> As in have two fields, something like "Description of the problem" and "Possible solutions:"?
<lifeless> I think they are larger than fields
<lifeless> many symptoms -> many possible solutions
<lifeless> common case being one symptom one solution
<lifeless> the url we give users when they report something is pretty sticky, so we wouldn't want to futz with that
<mpt> That's one case where there is no great distinction between Fixed and Declined: when the solution for the symptoms described is "we removed this feature".
<lifeless> indeed!
<lifeless> we might want to give the cluster of symptoms+possible solutions a number
<lifeless> we might want to let possible solutions apply to multiple disjoint symptoms
<mpt> When I redesign a feature, I try to come up with solutions to as many of the bug reports about that feature as I can. So it's fairly common that a single code change will resolve multiple bugs -- fixing some, deciding not to fix others.
<mpt> (See also my propensity for saying "(See also...)" in bug reports, or "(It may save time to fix bug XYZ at the same time as this bug)", or "(It may save time to design the fix for bug XYZ at the same time as this bug)".)
<lifeless> indeed
<lifeless> consider symptoms A,B,C and possible solutions X,Y
<lifeless> X may fix A,B, Y may fix B,C
<lifeless> and X and Y may be incompatible
<mpt> yep
<mpt> (though I'm skeptical because I can't think of a real example of that, and I'd expect to be able to)
<lifeless> flip flop bugs
<lifeless> like the one with 'contact this team'
<mpt> I'm not familiar with that one
<mpt> Is it that some people expect "Contact this team" to send mail to every member, and others think that's a terrible idea?
<lifeless> they get very very close to this - if contact this team obeys the team contact address, we can't tell if everyone is contacted. If it does not obey the team contact address and instead mails each person, the list is bypassed and everyone has to respond or not separately.
<mpt> That seems like a false dilemma, but anyway, it's just an example
<lifeless> right
<mpt> If Launchpad did not track branches at all, I'd be more interested in this idea of distinguishing symptoms and solutions
<mpt> but so far, it seems a bit bureacratic
<lifeless> the LP project itself gets a lot of imperative bugs
<mpt> "This is branch X representing solution Y that fixes bug Z"
<lifeless> part of the issue is the definition of 'bug'
<lifeless> is it 'something that needs a code change' or is it 'the abstract thing that is wrong'
<lifeless> (consider a bad coding pattern - like using sprintf - is each occurence a separate bug, is the use in each discrete project a separate bug, or is the thing itself a single bug?)
<mpt> A theoretically perfect design can lead you to a system that's just too much bother to use in practice
<lifeless> hell yes :) thats happened to a lot of LP
<lifeless> wgrant holds the opinion that multi-pillar (product/distros etc) bugs are an example of this
<mpt> Perhaps a real-life example of what you're talking about -- certainly a real-life example of what wgrant is talking about -- is Python transitions
<mpt> which need to be done in dozens of packages near-simultaneously
<mpt> but I can't tell which you would classify it as, "something that needs a code change" or "the abstract thing that is wrong"
<mpt> They used to be done in a single bug report, but then the maintainer of package A was being spammed by status changes for packages B, C, D, E, F ... Z
<lifeless> does Ubuntu file one bug saying 'transition to 2.8'
<lifeless> or does it use one bug per package
<lifeless> exactly
<lifeless> if you look at it from a symptoms perspective
<lifeless> each package that doesn't work is a separate symptom
<mpt> yes
<lifeless> and each solution (change package X) fixes at most one symptom (but may be a pre-requisite for fixing others due to dependencies)
<lifeless> [note the handwave in the middle there]
<mpt> There are Ubuntu Software Center bug reports describing problems that need work on the client and on the server
<lifeless> if one wanted to, the rdepends could be used to mass create such symptoms + proposed fixes + link them all together. So you could then trivially see what work needs to be done to get say bzrlib working in 3.0
<lifeless> so, I don't have a full and clear view on what to do with all this yet
<mpt> They're tracked together, because the discoordination of tracking them separately would be greater than the noise from tracking them together.
<mpt> ("together" = as a single bug report)
<lifeless> but, I do think that LP as it stands is driving noise in the system
<lifeless> mpt: with bug dependencies you may find that gets better (we'll show dependent bug task status in a table at the top, so you can see at a glance where things are at)
<mpt> I do think it's an architectural problem that you can't change multiple packages in a single branch. For example, update ubuntu-docs to reflect the change you're making to the UI of something. Or, move a checkbox from jockey to gnome-control-center.
<lifeless> do you mean single branch, or single landing ?
<mpt> single branch
<mpt> with a single log
<mpt> (I realize that would be near-impossible)
<lifeless> well, bsd does it
<lifeless> its definitely doable
<mpt> I didn't know that
<lifeless> doesn't mean its a good idea ;)
<mpt> Do they have all packages in one repository?
<lifeless> the bsd core system - not ports - is all in one big source tree, for all the BSDs I know of.
<mpt> heh
<lifeless> the ports system has all the metadata for all the packages in one tree as well.
<lifeless> so you can, in a single commit, update both jockey and gnome-control-center, but the update, like debian packages, consists of either adding patches or grabbing different upstream tarballs.
<lifeless> gentoo does that too
<mpt> In itself, I still think dependencies are a waste of time ... they'd be worth it only as the cost of abolishing blueprints
<lifeless> mpt: funny you should mention that
<lifeless> mpt: I think dependencies are very useful myself, for complex multi-step changes.
<lifeless> mpt: but also the blueprints thing
<mpt> When I saw the announcement of the work items work in LP I did a little head-desking
<mpt> If a software engineer who had never heard of Launchpad went to their boss and said, "You know, I think we should have two work tracking systems. One for defects, and another for work in general, whether that's fixing defects or something else. In the tracker for work in general, each piece of work should be a line of text, starting with the ID of the person who should do it in square brackets. And then we should have a third system, that screen-scrapes th
<mpt> e second one, and makes burndown graphs" ... I'd hope they would be fired on the spot.
<wgrant> mpt: I feel that multitask bugs are the source of a lot that is wrong with LP
<wgrant> s/multitask/multipillar/, at least
<lifeless> mpt: I agree
<wgrant> And I don't like workitems one bit :)
<wgrant> I was somewhat disappointed that lifeless didn't veto them.
<lifeless> mpt: linaro however wanted to automate what has grown organically; we need to fix the underlying facilities before we can consolidate
<lifeless> wgrant: the time to veto them was 4 years ago
<lifeless> wgrant: when folk started inventing task tracking inside blueprints; the modelling of them in LP is neither here nor there
<wgrant> lifeless: You've retroactively vetoed a lot of stuff like that :)
<lifeless> wgrant: always with an eye on the impact and how folk will acclimatise
<lifeless> bah, spelling
<mpt> I think the way to get rid of something that has inertial stupidity is to make its replacement more attractive.
<lifeless> right
<lifeless> win by competing
<mpt> For example, integrate burndown charts in LP ... for bugs but not for blueprints.
<mpt> (You want something to show on the burndown chart? Nominate it for the series. That's what series are for.)
<lifeless> mpt: well, I want to remove non-series bugs, simplifying a bunch of model and code in LP and making it easier to reason about
<mpt> That works too (though I'm dubious about it in general).
<lifeless> there's also the tension between planning milestones and recording milestones to sort out; I think it would be nice to have just one way to plan things
<lifeless> (for deadline planning that is)
<mpt> Is "planning" there an adjective or a verb? :-)
<lifeless> I don't know :)
<mpt> wgrant, what are some examples of the wrongness caused by multi-task bugs?
<wgrant> I hope that the stakeholders who shall remain anonymous will be convinced that multitask bugs can go away once bug linking exists :)
<mpt> apart from bug 1
<_mup_> Bug #1: Microsoft has a majority market share <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Confirmed for compscibuntu-bugs> <LibreOffice Productivity Suite:New> <dylan.NET.Reflection:Invalid> <dylan.NET:Invalid> <EasyPeasy Overview:Invalid by ramvi> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <LibreOffice:In Progress by bjoern-michaelsen> <Linux:New> <Linux Mint:In Progress> <The Linux OS Project:In Progress> <metacity:In Prog
<wgrant> mpt: Every notable piece of data in Launchpad is owned by a project (except for a few things that are owned by people or teams).
<wgrant> mpt: Except bugs.
<wgrant> mpt: Which aren't owned by a person.
<wgrant> mpt: And aren't owned by a project.
<wgrant> They are cooperatively owned by projects.
<mpt> wgrant, that's theory. Give me practice.
<wgrant> mpt: Who controls access to a cross-pillar bug?
<mpt> ok, that's a good one
<wgrant> Nobody owns the data.
<wgrant> The project owner can't control access to their project's private bugs.
<mpt> Another one: There are multiple URLs to pages that are exactly the same except for the highlighting of a table row.
<wgrant> And there is the whole notifications mess.
<wgrant> mpt: Well
<wgrant> mpt: And some of the other links are silently context-sensitive.
<wgrant> But nobody realises this.
<wgrant> (eg. Target to series)
<mpt> yes
<wgrant> mpt: It also causes problems with the person pickers.
<wgrant> Because if I want to search for say someone to subscribe or a branch to link, it pretty much has to search all of Launchpad.;
<wgrant> (or prioritise people from all the related projects, but that's getting silly)
<wgrant> The notifications thing is part of why we have bug muting.
<wgrant> Ubuntu tried multi-task bugs for a couple of transitions.
<wgrant> With maybe 250 packages.
<wgrant> It doesn't result in usable UI, and the bugs just get huge and very noisy.
<mpt> Transitions were never what multi-task bugs were intended for anyway.
<wgrant> Even now you can't mute notifications from just some of the tasks.
<mpt> If people would have inevitably made that mistake, that's a good example, otherwise it's just misuse.
<wgrant> Even Launchpad abuses multi-task bugs in privileged ways that wouldn't be acceptable for other projects to do.
<wgrant> eg. bug in Loggerhead -> have a Loggerhead task, and a Launchpad task to track its deployment on Launchpad.
<wgrant> We consider that acceptable because we own both projects.
<wgrant> And we get noise from both anyway.
<wgrant> But that's a privileged position.
<wgrant> What if SourceForge added tasks on all Loggerhead bugs?
<mpt> Unity abuses multi-task bugs in a similar way ... I'm almost tempted to agree with you for that reason alone
<wgrant> Or, say, a distro named Baltix on all of Ubuntu's bugs.
<wgrant> It forces the upstream to know about the downstream.
<wgrant> And be notified about the downstream.
<lifeless> mpt: you cannot misuse someting that doesn't exist; the feature existing invites its [mis-]use.
<wgrant> When really they don't give a shit.
<wgrant> Except that they want the bloody noise to stop :)
<mpt> lifeless, you could say the same about Launchpad as a whole! I'm talking about misuse vs. productive use
<lifeless> mpt: I was addressing your 'inevitably misuse it' thing - we invite that by how we design and present the feature
<mpt> lifeless, right, I meant "if they would have made that mistake even if we presented it in the clearest possible way"
<mpt> wgrant, there is a deep irony here. Launchpad was intended to become what Github has become, i.e. the center of the open-source universe. If it had, multi-task bugs would have been more useful. But it didn't at least in small part because it was too complicated ... for example having multi-task bugs. :-)
<lifeless> mpt: one of the key insights I had on bugs a while back was that who gets notified maps to who will participate in the conversation
<wgrant> mpt: Oh yes
<lifeless> mpt: and it follows from that that the *size* of the conversation is deeply connected to how you partition data
<wgrant> mpt: The situations that multi-task bugs are good in are probably better solved in other ways. There are questions around how we share the conversation, but the current no-opt-out conversation sharing isn't working either.
<lifeless> mpt: both for private data (you can't combine multiple private-data bugs even if they have the same symptoms) and for related-things (just because something is related doesn't mean you want a large conversation)
<mpt> lifeless, I have the sneaking suspicion that the ideal bug tracker wouldn't send e-mail at all. I don't know what it would look like, though.
<mpt> You'd go ... somewhere and see all the issues that might benefit from your input.
<lifeless> mpt: heh
<mpt> and somewhere to see all the issues that were blocked on you.
<lifeless> I'm not sure there is a right answer
<mpt> (Maybe the same place for both.)
<lifeless> I think those are necessary features for a good bug tracker
<lifeless> Who wants notifications and when is more complex ;)
<mpt> If the purpose of a bug tracker is to help developers make better use of their time, then the ultimate is "Here is an ordered list of things on which your time would be best used"
<lifeless> I want one that covers multiple forges
<lifeless> :)
<mpt> At the risk of using a buzzword, that could be a game-changer
<mpt> at least for people who are at least partly volunteers
<lifeless> LP's federation facilities could do it, if we finished it
<wgrant> I thought LP was meant to do that.
<wgrant> Having global bug numbers basically prevents it.
<wgrant> But I thought that was a vague LPish goal originally.
<mpt> get out of the competition between trackers, and into "here's all the things I can do, everywhere"
<huwshimi> mpt: We've been talking about that (well at least I have). We currently use email to solve the whole problem of notifications instead of managing them in-house (and make users do all their personal prioritising outside LP).
<mpt> huwshimi, and inboxes suck as task lists :-)
<huwshimi> mpt: Exactly!
<huwshimi> mpt: One of our planned goals was to create personal dashboards
<huwshimi> mpt: But, we don't have the time for that now :)
<lifeless> I have a fair chunk of a notification system overhaul done.
<lifeless> but its not a top priority
<wgrant> Yes, but you're the only person who will be able to touch it for at least a year and probably ever Z:)
<mpt> Possibly the ideal bug tracker would still send notifications, but only by SMS, because that's how urgent they'd be
<huwshimi> haha
<mpt> huwshimi, there are plenty of ways Launchpad can be improved merely by removing stuff
<mpt> Ooh, I should show you my user style sheet
<wgrant> Um
<wgrant> The former LP UI designer uses a user stylesheet rather than filing bugs to get it fixed for everyone? :)
<lifeless> mpt: I'm glad you're getting somet time to think about LP again.
<wallyworld_> wgrant: why did you revert my audit sharing stuff? the owner of a project has a right to see who can access their project and the implementation was as agreed
<wgrant> wallyworld_: It wasn't restricted to the owner.
<wgrant> And that wasn't agreed with everyone who uses private teams.
<wallyworld_> it was agreed with us on the call
<wgrant> wallyworld_: eg. as an unprivileged user I was able to find a bug with Canonical subscribed, retarget it to my project, make it private, and read all of Canonical's memberships.
<wgrant> => inappropriate
<wallyworld_> the audit sharing view has launchpad.Edit permissions against it
<wgrant> Sure
<wgrant> But bugs can be retargeted.
<wgrant> So I can create a project, find an interesting bug, move it to my project, and have root :)
<wallyworld_> so that's not the fault of the audit sharing implementation
<wgrant> It's still a complete hole.
<wgrant> (and yes, it is)
<wgrant> Well
<wallyworld_> i implemented what we agreed to
<wgrant> It's a fault of the design.
<wgrant> And I disagreed with the design, and on the grounds that it completely violates the team security policy I will not let it persist in the tree until it is fixed.
<wallyworld_> that's not your call
<wgrant> Nope.
<wgrant> But I'll do it anyway.
<lifeless> the hole wgrant just described is one jane would be extremely unhappy with.
<wgrant> Precisely.
<wgrant> your team is private. loljk.
<wallyworld_> sure, but some collaboration would have been nice and curteous
<lifeless> she has put a mandate out there that the membership of ~canonical must-not-be-disclosed
<wgrant> wallyworld_: I tried to ping you.
<wgrant> wallyworld_: But you weren't around.
<wgrant> And I didn't want to comment in the bug while it was live on production data.
<wallyworld_> ok
<lifeless> wgrant: can we tell if anyone abused it while it was on qastaging ?
<wgrant> lifeless: We can make reasonable guesses from logs. Let's see if I can find them.
<lifeless> wallyworld_: wgrant: could you please check; if it looks like a nontrivial thing to determine, please open an incident report.
<wgrant> Of course this just *has* to be across a DST shift, doesn't it...
<mpt> huwshimi: http://i.imgur.com/Vr465.png
<mpt> Spot the differences
<poolie> hi all
<wgrant> mpt: Aw
<poolie> hi mpt, thanks for your cites on my Opinion bug
<wgrant> mpt: You didn't fix the subscription portlet as much as I expected :(
<mpt> wgrant, there's only so much I can do in CSS alone ... I just hid the first line of text
<wgrant> Ah :(
<mpt> poolie, you're welcome
<wallyworld_> wgrant: do you know why there's a gap in the log file dates on asuka?
<wgrant> wallyworld_: Of about 2 months? I hope it's just a partial rsync. I requested a full sync a few minutes ago.
<lifeless> wgrant: hows carob's disk space looking
<wgrant> Otherwise logrotate is terribly misconfigured and someone will be digging on aboa :)
<wgrant> lifeless: over 9000
<wgrant> Or 128GB, whichever.
<lifeless> wgrant: spm disabled full syncs till the oops were cleared
<wallyworld_> wgrant: it seems to be a few days from 26/3 till 1/4
<wgrant> lifeless: I know.
<lifeless> wgrant: kk
<wgrant> lifeless: We have another few tens of gigabytes of oopses to delete in the next day or two, I believe.
<wgrant> But might switch it on anyway since spm is off. :)
 * wgrant drowns in librarian logs.
<mpt> wgrant, I'm just experimenting at the moment ... bug reports come later
<wgrant> mpt: I'm not sure that's ideal. Last time someone proposed a change to the subscriptions portlet it became 10x larger and unintelligible :)
<wgrant> The next change will likely cause it to fill the entire sidebar.
<wgrant> With meaningless text :)
<wgrant> wallyworld_: Sync done
<mpt> "You are not directly subscribed to the notifications of the mail about the changes to the notifications, but you might be subscribed to the notifications of the notifications."
<wallyworld_> ok
<wgrant> launchpad.log.1 is now from 2012-01-23 to 2012-03-31
<wgrant> Which is I think a complete record.
<wgrant> The code was deployed around 2012-03-31T09:00
<wgrant> It was all me and wallyworld.
#launchpad-dev 2013-03-25
<StevenK> wgrant: How goes the rosetta-chainsawing?
<czajkowski> carving it up into chunks I hope
<wgrant> StevenK: rarrrgh
<wgrant> 2013-03-25 03:58:23 INFO    Importing: Italian (it) translation of ddtp-ubuntu-universe in ddtp-ubuntu trunk
<wgrant> (still going)
<StevenK> Hah
<wgrant> So testing is slow
<StevenK> Oh, you're kidding me.
<wgrant> ?
<StevenK> If you create a milestone with tag "WTF", it will be stored as 'wtf'. If you edit a milestone, and set the tag to 'WTF', it will be stored as that.
<czajkowski> StevenK: wgrant any idea how I get this looked at https://bugs.launchpad.net/launchpad/+bug/1158830
<_mup_> Bug #1158830: The string count in the po file and on Launchpad do no match <lp-translations> <Package Descriptions for Ubuntu:Confirmed> <Launchpad itself:Confirmed> < https://launchpad.net/bugs/1158830 >
<wgrant> StevenK: Heh
<czajkowski> https://bugs.launchpad.net/launchpad/+bug/1158854
<_mup_> Bug #1158854: String present in Launchpad is not exported <Package Descriptions for Ubuntu:Confirmed> <Launchpad itself:Confirmed> < https://launchpad.net/bugs/1158854 >
<wgrant> czajkowski: I'll hopefully look at that tomorrowish
<czajkowski> morning btw
<wgrant> I'm in rosetta atm
<wgrant> Also, go to sleep
<czajkowski> full of roses and fun
<czajkowski> cant sleep :/
<StevenK> Oh, I'm wrong, they're both lowercased
<StevenK> czajkowski: Read some autoconf macros?
<StevenK> According to Matthew Garrett, they induce comas.
<wgrant> 50000 message templates: Australia says no.
<StevenK> wgrant: http://pastebin.ubuntu.com/5645396/ is making me very sad.
<wgrant> StevenK: Why did you _ it?
<wgrant> devel is field_names, not _field_names
<StevenK> wgrant: http://pastebin.ubuntu.com/5645405/ is my diff, I'm refactoring
<wgrant> StevenK: Confused
<wgrant> StevenK: You rename an attribute from field_names to _field_names in one place
<wgrant> But no others
<StevenK> wgrant: You think it's unnecessary?
<wgrant> StevenK: I think renaming an attribute in one context but no others is probably not going to end particularly uncrashingly.
<StevenK> OH
<czajkowski> StevenK: did you say deleting a lp profile is now possible not just deactivating it
<StevenK> czajkowski: You can not delete a LP person. The person themselves can deactivate it now that it won't OOPS every time. Why?
<czajkowski> RT came in
<StevenK> Link me?
<czajkowski> StevenK: https://support.one.ubuntu.com/Ticket/Display.html?id=31156
<czajkowski> We do get some rather unusal mails into the RT - https://support.one.ubuntu.com/Ticket/Display.html?id=31124
<StevenK> We can not.
<StevenK> Moreover, we will not.
<czajkowski> StevenK: heh ok.
<StevenK> czajkowski: Just to trying to stop "Oh, yes you can. I know it's against policy, but surely you can escalate this request to someone who can just do it for me? Just this once?"
<czajkowski> nods
<czajkowski> StevenK: can you help me out on https://answers.launchpad.net/launchpad/+question/224961
<czajkowski> really should also attempt sleep rather than doing answers.. but still
<StevenK> czajkowski: His Build-Depends are bong
<StevenK> czajkowski: Answered.
<czajkowski> thanks
<czajkowski> bong I like that new description
<StevenK> Clearly, a Launchpad issue.
<czajkowski> StevenK: we do get unusal requsts..https://answers.launchpad.net/launchpad/+question/224934
<czajkowski> hmm I'm going to the petrol station and see if they've had bread delivery so I can make toast, bbiab
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/validate-milestone-tags/+merge/155171
<adeuring> good morning
#launchpad-dev 2013-03-26
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/validate-milestone-tags/+merge/155171 and https://code.launchpad.net/~stevenk/launchpad/fix-translations-opening-specify/+merge/155375
<wgrant> StevenK: Looking
<wgrant> StevenK: Is the translations fixer now sharing-safe?
<StevenK> wgrant: I doubt it, what's required for that?
<wgrant> StevenK: Was it written after the big message sharing migration?
<wgrant> We don't want to revive a one-off script if it's no longer safe.
<StevenK> wgrant: It was used for Quantal, at least
<wgrant> Was it?
<StevenK> Since it was hardcoded for that series
<wgrant> Oh, I guess so
<wgrant> So yeah
<StevenK> wgrant: I'm happy to add a check for series.status, too
<wgrant> StevenK: How does generalising that script fix that bug?
<wgrant> At best it is a semi-fix for bug #576822
<_mup_> Bug #576822: Add an option to clean up after a failed run of copy-translations-from-parent.py <lp-translations> <tech-debt> <Launchpad itself:Triaged> < https://launchpad.net/bugs/576822 >
<StevenK> Ah, that is probably the bug I was aiming for
<StevenK> wgrant: However, it sort of addresses it, but allowing ops to scorched earth translations for a series
<StevenK> But only if you squint
<wgrant> It probably makes it non-critical
<wgrant> StevenK: Have you tested that the script works?
<wgrant> Given that there are no tests.
<wgrant> AFAICT
<StevenK> scripts/fix-translations-opening -s warty deletes nothing against my dev instance, but it doesn't croak
<wgrant> Can you find a series with translations?
<StevenK> wgrant: http://pastebin.ubuntu.com/5648155/ :-(
<wgrant> StevenK: Yeah
<wgrant> So, that's because there are diverged messages
<wgrant> DELETE FROM translationmessage WHERE potemplate IS NOT NULL; on your dev DB, and try again
<StevenK> wgrant: http://pastebin.ubuntu.com/5648169/
<wgrant> Right.
<StevenK> wgrant: Can haz review, or do you still have concerns?
<wgrant> StevenK: I've approved one, got distracted by users for the other
<StevenK> wgrant: Ah, missed that. Landing the translations one now that I've corrected its bug link.
<wgrant> StevenK: Does bug tag validation also use valid_name?
<wgrant> StevenK: And what's _field_names for?
<StevenK> wgrant: _field_names is only used by +edit
<wgrant> StevenK: Is there a reason for it to be used?
<StevenK> I can't change the _field_names property to field_names, because then I can't set it in the superclass
<wgrant> StevenK: Also, you have two or so extendFields methods that do nothing but upcall. They can be deleted.
<StevenK> wgrant: Looks like bug tags do widget validation
<nigelb> Hrm, critical bugs are down to 130 now?
<nigelb> whoa.
<wgrant> Below 120, I think
<wgrant> Yeah, 118
<nigelb> Um, how big is the LP team now? Just the 3 of you?
<wgrant> 2
<nigelb> 33
<nigelb> (grr)
<nigelb> I counted laura as well.
<wgrant> Ah, yeah, but 2 engineers :)
<wgrant> And mysteriously we're still making better progress than the entire period that we weren't on maintenance.
<nigelb> *sadface*
<nigelb> haha
<nigelb> Yeah, I was about to say that.
<nigelb> I guess you aren't introducing new bugs now ;)
 * StevenK is about to make it be 117
<StevenK> And I'm hoping wgrant has a branch or two for me, but that might be too optimistic.
<nigelb> I still have a wip branch that I haven't gotten around to.
<StevenK> I need to figure out what to land next, I have the last 9 branches on devel
<nigelb> has spm left Canonical? Or just no longer idling here?
<StevenK> nigelb: Yeah, he's gone. :-/
<nigelb> aww, I miss the fun with him around :(
<czajkowski> adeuring: hows things?
<adeuring> czajkowski: work is fun, we have now even some sunshine -- can't complain :)
<czajkowski> I have snow.. still
<czajkowski> less than impressed, more due this weekend
<czajkowski> some easter it's gonna be
#launchpad-dev 2013-03-27
 * wgrant throws rocks at SKS
<StevenK> wgrant: Oh?
<wgrant> Its 404 message has changed
<wgrant> That's why we're getting oopses from it
<wgrant> But it doesn't actually crash the page
<StevenK> What's one of the OOPSes?
<wgrant> OOPS-7d16107fdc9d638d26333ce4668d524e
<StevenK> I thought we dealt with both 404 and 500?
<wgrant> We do
<wgrant> But if the error message isn't plausible, we raise a soft OOPS
<StevenK> Ah
<StevenK> And they changed it
<wgrant> Yes
<StevenK> wgrant: So I can't find any OOPSes for "[^:]DistroSeries:+localpackagediffs"
<wgrant> StevenK: Probably because nobody uses it, potentially because it doesn't work
<wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/sks-404-msg/+merge/155647
<StevenK> wgrant: No fair landing something
<StevenK> wgrant: Meh, that was probably fair enough to be a self-review. r=me
<wgrant> Probably
<wgrant> But the r=wgrant in the log was getting a bit boring
<StevenK> Haha
<StevenK> Only because the last 9 landings on the trot have been mine
<StevenK> wgrant: It does load, though
<StevenK> Of course, now it times out for me.
<wgrant> StevenK: What or who loads?
 * StevenK shoots Murphy in the face.
<wgrant> Oh, +localpackagediffs?
<StevenK> https://launchpad.net/ubuntu/raring/+localpackagediffs
<wgrant> 7.87s
<wgrant> Nice
<StevenK> 52 queries/external actions issued in 7.43 seconds
<StevenK> Yeah
<StevenK> It spends 300ms looking up the DSDs, and then 3.5seconds grabbing the publications
<wgrant> Yeah
<wgrant> Explaining now
<wgrant> It's using SPR.spn, which is probably the main problem
<wgrant> I missed another one :(
<StevenK> Can you also explain why it runs the query four times?
<StevenK> It's Launchpad, not Santa Claus.
<wgrant> Hey, you were on the team that wrote that page, not me :)
<wgrant> But I'd get an OOPS and check what calls it
<wgrant> Because that's hilarious
<wgrant>  Total runtime: 90667.543 ms
<wgrant> ooh yeah
<wgrant> 350ms->65ms by dropping the SPR join
<StevenK> wgrant: http://pastebin.ubuntu.com/5651202/
<wgrant> Oh right
<StevenK> wgrant: Just check that I'm not going insane, and it is
<wgrant> This is the derpquery
<wgrant> That has the completely strange anti-optimisation technique
<wgrant> To try to prevent it from using the index that is probably now optimal
<wgrant> AND SourcePackagePublishingHistory.archive+0 = Archive.id
<wgrant> 3ms
 * wgrant enbranchinates
<StevenK> Ah
<StevenK> I was going to do that
<StevenK> Beh eh
<wgrant> No'
<wgrant> I get the trivial ones today, to let me escape from translations crap :)_
<StevenK> And all four of them are different
<wgrant> Meh
<wgrant> I'll make them not suck
<StevenK> derived_series vs parent_series
<wgrant> Then we can separately eliminate the dupes
<wgrant> Ah
<wgrant> Anyway
<wgrant> Not so bad once they're a direct index scan
<wgrant> What's the traceback?
<wgrant> Not sure which func it is
<wgrant> I guess I could just grep for +0...
<StevenK> That +0 might be a red herring
<StevenK> I was pulling it from the query log
<wgrant> No
<wgrant> I remember the query
<StevenK> eager_load_dsds
<wgrant> It was deliberately used as an anti-optimisation technique
<wgrant> And the archive join is just pointless
<StevenK> Twitch
<StevenK> It's limited to purpose PRIMARY, though
<wgrant>         # XXX: GavinPanella 2011-06-23 bug=801097: The + 0 in the condition
<wgrant>         # below prevents PostgreSQL from using the (archive, status) index on
<wgrant>         # SourcePackagePublishingHistory, the use of which results in a
<wgrant> get the ID :)
<wgrant>         # terrible query plan.
<wgrant> Yeah
<wgrant> But you know what we can do instead...
<StevenK> Fetch self.context.main_archive.id ?
<StevenK> wgrant: Anyway, damn you for stealing my bug
<wgrant> Hmm
<StevenK> Now I have to find another.
<wgrant> May not be able to do this,a ctually
<wgrant> Because it cares about the parent
<StevenK> "#899003 deadlock detected when importing .po file" is likely going to die as a case of your chainsawing ?
<_mup_> Bug #899003: deadlock detected when importing .po file <lp-translations> <oops> <rosetta-imports> <Launchpad itself:Triaged> < https://launchpad.net/bugs/899003 >
<wgrant> Unlikely
<wgrant> Unless I rewrite rosetta
<StevenK> Pity
<StevenK> Maybe I should destroy the puller
<StevenK> That will keep me out of trouble for a long while
<wgrant> Heh
<StevenK> But I have no idea where jelmer got up to
<wgrant> A very very long while
<StevenK> We can't just transition the branches of type MIRRORED to code imports and destroy it?
<wgrant> You can get rid of mirrored branches just by migrating the existing ones, really, but that won't obviate the need for the puller.
<wgrant> The puller is still used for imports
<StevenK> I'm thinking of the broken netstrings critical
<StevenK> Migrating MIRRORED branches and dropping the type might close it, or at least drop it to Low
<wgrant> I don't think so
<wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/bug-1160461/+merge/155651
<StevenK> wgrant: v = ? Really?
<wgrant> Really!
<wgrant> I can change it if you want
<StevenK> I would prefer view =
<StevenK> And you're not trying to force a long line to be short enough
<wgrant> wtf
<wgrant> dsd indices are terrible
<wgrant> StevenK: Fixed.
<StevenK> I just saw
<StevenK> You're a terrible person
<StevenK> r=me
<wgrant> Thanks
<wgrant> And the 1s DSD-finding query is now mysteriously 2ms
<wgrant> (* may actually be index, not actual mysteriousness)
<StevenK> Haha
<wgrant> StevenK: https://dogfood.launchpad.net/ubuntu/quantal/+localpackagediffs
<wgrant> And that's without the DSD index
<wgrant> Still slow, but that's hopefully just because DF is DF
<wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/localpackagediffs-rargh/+merge/155657
<StevenK> wgrant: What's the index out of interest?
<wgrant> (derived_series, status, difference_type)
<wgrant> Not going to add it for now
<wgrant> But it's something we might want to look at later
<StevenK> wgrant: Going to link 801097 to the MP?
<wgrant> A good point.
<StevenK> wgrant: r=me
<wgrant> Thanks.
 * StevenK glares at the critical list
<StevenK> wgrant: I do wonder about a BugJob that does notification
<StevenK> But I worry that I'd get murdered
<wgrant> StevenK: Sort of
<StevenK> I'd get sort of murdered?
<wgrant> Deferring recipient calculation by storing the criteria in a table would work
<wgrant> But it's a fair bit of work
<StevenK> wgrant: Suggestions for what I should look at, then?
<wgrant> Um
<wgrant> Stuff.
<wgrant> And things :)
<StevenK> Thanks for being so very helpful. :-P
<StevenK> I do wonder how txlongpoll is run, WRT 960868
<wgrant> It might almost be worth making another pass at the loggerhead criticals
<wgrant> You can probably fix several of those in one day
<StevenK> But it's loggerhead ...
<StevenK> wgrant: How do I land stuff against loggerhead?
<StevenK> Hmmm, we're 7 revs behind
<wgrant> Should just need to commit directly to trunk
<wgrant> Check the recent history
<StevenK> wgrant: We have no codebounce on staging/qas ?
<wgrant> StevenK: Yes, it's there
 * jelmer waves
<czajkowski> jelmer: boo
<jelmer> StevenK: migrating MIRRORED branches to code imports should do the trick
<jelmer> StevenK: I don't think there was a reason for not doing that, apart from the time it would take to manage that transition
<jelmer> czajkowski: hey hey :)
<wgrant> StevenK: Can you QA your loggerhead thing?
<wgrant> StevenK: And shouldn't that LH task be Fix Committed?
#launchpad-dev 2013-03-28
<StevenK> wgrant: http://bazaar.qastaging.launchpad.net/~launchpad-pqm/launchpad/devel/files looks okay to my testing, but I'm not sure how to stress it
<wgrant> StevenK: Have you tried some non-ASCII paths?
<StevenK> That means finding a branch that happens to have some, or making my own...
<wgrant> danilos: Are bug #869264 and bug #730610 duplicates?
<_mup_> Bug #869264: Old upstream translations overwrite newer upstream translations in Ubuntu <lp-translations> <message-sharing> <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/869264 >
<_mup_> Bug #730610: translation uploads from old series replace the current series translations <message-sharing> <Launchpad itself:Triaged> < https://launchpad.net/bugs/730610 >
<StevenK> wgrant: http://bazaar.qastaging.launchpad.net/~stevenk/+junk/foo/files
<wgrant> StevenK: Does that crash on prod?
<StevenK> Nope
<wgrant> Then it's not a dreadfully good testcas
<wgrant> e
<wgrant> It's in a dirname function
<wgrant> So it will probably need to be a directory name
<StevenK> Yes, I'm pushing r2 now
<StevenK> http://bazaar.launchpad.net/~stevenk/+junk/foo/files/head:/foo%C7%80%C6%B6/foo%C6%86/ => OOPS
 * StevenK reaches for qas
<StevenK> wgrant: OOPS-3b6ba4d894000fb4d96e6e6e947ef2cd on prod, a lack of oops on qas
<wgrant> StevenK: Marvellous
<wgrant> Does it look sane on qas?
<StevenK> Yes
<wgrant> So it does
<StevenK> wgrant: http://bazaar.qastaging.launchpad.net/~stevenk/+junk/foo/files
<wgrant> qa-ok!
<StevenK> wgrant: Does sir wish for a NDT?
<wgrant> Indeedily.
<wgrant> Ahhh
<wgrant> Tracked down the translation consistency bug finally
<wgrant> The export bug
<wgrant> czajkowski: ^^
<StevenK> wgrant: Oh?
<wgrant> StevenK: Filtering a left join in the wrong place
<wgrant> Similar to a mailing list bug we had a couple of years ago
<StevenK> Handy
<StevenK> And sort of subtle without tests
<wgrant> Yeah
<wgrant> It excludes messages when the potmsgset is shared, and the only translation is a diverged one for the other side.
<wgrant> jtv: Around?
<jtv> Hi wgrant
<jtv> I mean: yes.
<wgrant> jtv: Hi
<wgrant> jtv: http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/view/head:/lib/lp/translations/doc/vpoexport.txt#L101
<wgrant> The last section
<wgrant> I think the test is insane
<wgrant> Do you remember enough about this to sanity check it?
<wgrant> The test asserts that the untranslated messages aren't returned by getTranslationRows
<wgrant> But normal untranslated messages *are* returned
<jtv> Im looking.
<wgrant> The ones in this test are not returned due to a probably SQL bug which excludes messages if the only translation is a diverged one on the other side
<jtv> Oh, by the way: it's a doctest, so safe assumption that it's insane.
<wgrant> If I were to create a new template item, it would show up there
<wgrant> Hhe
<wgrant> Yes
<jtv> Excludes messages from what?  From the list of suggestions?  Or from the translation?
<jtv> Because if the only translation is a diverged one on the other side, that means there's no translation where you're looking.
<wgrant> getTranslationRows is used for PO exports, so it's meant to return all of the template's messages
<wgrant> Even if they're untranslated
<wgrant> It achieves this, except that the left join condition is in the wrong place, so if the only translation for a potmsgset is a diverged translation for another template, the message doesn't show up at all, even with a null translation.
<jtv> Oh, the potmsgset is being left out?
<jtv> Gotcha.
<wgrant> The test asserts that behaviour; both of those examples at the end should also return a None row AFAICT
<jtv> Let me just look closer into what getTranslationRows was meant to do...
<wgrant> Sure
<wgrant> https://pastebin.canonical.com/87982/
<wgrant> If I add a third message with no translations at all, it shows up, as I assume the correct behaviour is
<jtv> #@$ 2fa
<wgrant> But the messages that have only diverged translations don't show up at all.
<wgrant> http://paste.ubuntu.com/5654235/
<jtv> Trying to load that first pastebin first.
<wgrant> The second one is the same
<wgrant> But no 2FA :)
<jtv> Ah
<jtv> I'll have a look.
<jtv> Why, why, why did nobody accept my proposal of using null instead of 0 as a sequence number for obsolete messages?
<wgrant> I have been wondering that for a good portion of the last week
<wgrant> I hoped you'd have an excuse :P
<jtv> Well okay, I think I remember the reason: it was left for later.
<wgrant> Heh
<jtv> I guess the condition on TranslationMessage.potemplate is in the wrong place, yeah.
<wgrant> Yeah
<wgrant> If I move it into the left join where it should be, the bug goes away and that test fails
<jtv> Watch out for performance changes.
<jtv> (Even if it gets faster, I'd like to hear about it :)
<wgrant> There's also the slightly confusing aspect that in its current position it's actually a three-way disjunction: no TM, undiverged TM, or diverged TM for this template
<wgrant> Yeah
<wgrant> The existing query is absolutely dreadful, so I'm not too concerned
<wgrant> But I will check
<wgrant> Oh
<wgrant> The query itself is only a couple of seconds
<wgrant> I guess the data transfer and object loading may be big
<wgrant> 'cause it appears to take about a minute
<wgrant> When run in a harness
<jtv> FWIW VPOExport used to be a view in the database.  Meant as an optimization, had turned into maintenance *and* performance drag.
<jtv> A minute in harness?  What kind of dataset are you  running against!?
<wgrant> ddtp-ubuntu-universe
<jtv> Ah OK then :)
<wgrant> aka. the biggest template
<wgrant> It's what the bug was reported against
<wgrant> And I had no idea what was wrong
<wgrant> So couldn't really minimise it
<jtv> Any major speed changes for that query though?
<wgrant> Checking
<jtv> Congratulations on spotting this.  Good sleuthing.
<StevenK> And wgrant has spent 1.5 weeks with his head in rosetta :-(
<wgrant> (the export of just the French raring translation takes upwards of 5 minutes, so testing without a harness is a bit painful...)
<jtv> Gah
<wgrant> StevenK: This bug is only today, though
<jtv> So many things I would still have loved to have time to clean up...
<jtv> Import approvals in particular!
<StevenK> jtv: Like lp.translations.model.* ? :-)
<jtv> Pretty much.
<jtv> The trick with these things is that you have to hate complexity, ambiguity, and tech debt.  Seek it out & destroy it.
<jtv> Oh, and you want my guess as to why data transfer & object loading seem to take so long?
<wgrant> Oh
<wgrant> It's grabbing context etc.
<jtv> If the query includes POTemplate or POFile, those have text fields containing the headers.
<wgrant> So lots of reasonably sized strings, I suppose
<jtv> Unreasonably sized header strings.
<wgrant> And pomsgids
<jtv> Unicode conversion, awkward data sizes.
<wgrant> No POTemplates or POFiles here
<jtv> Oh that's a relief.
<wgrant> Just POTMsgSets and POMsgIDs
<wgrant> But still not small, especially for that template
<jtv> pomsgids are short and de-duped.
<wgrant> Descriptions are fat
<jtv> Ah true
<jtv> But they should only come in at the end, basically.
<wgrant> This template is about the worst possible case for everything, except that it has no plurals.
<jtv> Quite.
<wgrant> Anyway, thanks for your help
<wgrant> :)
<jtv> No worries.
<wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/bug-1158854/+merge/155892
<StevenK> wgrant: r=me
<wgrant> Thanks
<StevenK> I'd like Stormification, but not now
<wgrant> Translations is difficult to Stormify due to the dynamic column names
<nigelb> Oh hey, jtv is still around! ;)
<jtv> Yup.  Hi nigelb!
<jtv> How's life?
<nigelb> Pretty good! Surving. How about you?
<nigelb> *Surviving
<nigelb> Are you still out East?
<jtv> Yup.
<jtv> Hot  here.
<nigelb> Here too. 35C is *not* pleasant.
<StevenK> It hit 32 in Sydney today
<nigelb> I'm on the top floor of this building. Sometimes I end up sitting on the floor because it's slightly less warmer.
<jtv> I've got a fan blowing at me right now.
<danilos> wgrant, yeah, they seem to be describing the same thing, though 869264 is a bit more specific
<wgrant> danilos: Yeah, I was going to dupe them in that direction
<wgrant> Since 869264 is clearer
<wgrant> Thanks
<czajkowski> wgrant: StevenK translation bug all fixed ?
<wgrant> czajkowski: Not on production yet, but the missing string thing is fixed in trunk
<wgrant> Will be deployed Tue or Wed
<wgrant> That's the two bugs from the 23rd
<czajkowski> excellent
<czajkowski> thanks folks
#launchpad-dev 2014-03-24
<daker> hi
<daker> i think LP UI is broken http://i.imgur.com/gE38808.png
<daker> all the link in this area can't be right-clicked
<daker> links*
<daker> this is what i get http://i.imgur.com/PCruIL1.png
#launchpad-dev 2015-03-23
<cjwatson> wgrant: builder processor sorting> do you mean on id or (contrary to the current UI) alphabetically by processor name?
<wgrant> cjwatson: Anything that's deterministic, but name might be sensible.
<cjwatson> The overall sort key should probably remain id, so that x86 builders land at the top.
<wgrant> Indeed.
<cjwatson> Although I guess that would still be the case since amd64 is lexicofirst
<cjwatson> But might not always be, who knows.
<wgrant> Lucky we don't have aarch64.
<infinity> Aardvarch.
<infinity> I guess we have no concept of sort-of-related processor families?
<infinity> Cause it would be nice for amd64/i386 to sort together, arm64/armhf/armel, ppc64el/ppc64/powerpc
<wgrant> Not at the moment.
<wgrant> But in practise they will eventually be grouped.
<infinity> Yeah, fair.
<infinity> Once they're all grouped, I guess that'll sort (hah!) itself.
<cjwatson> wgrant: Actually, this *is* the overall sort key.
<cjwatson> So if we're doing this in that method it might as well just be sorted(p.name for p in builder.processors)
<wgrant> cjwatson: Oh, is builder.processors sorted already?
<cjwatson> By id
<cjwatson>         return list(Store.of(self).find(
<cjwatson>             Processor,
<cjwatson>             BuilderProcessor.processor_id == Processor.id,
<cjwatson>             BuilderProcessor.builder == self).order_by(Processor.id))
<cjwatson> wgrant: Does that address your MP comment, then?  I thought you knew about that and wanted it explicit in the browser code too.
<cjwatson> I could add a comment indicating the sorting.
<wgrant> cjwatson: I think it's fine as it is. I hadn't realised the property was explicitly sorted already.
<cjwatson> OK
#launchpad-dev 2015-03-25
<rpadovani> guys, the diff page is broken, I don't know since when, but not more of 2/3 days
<rpadovani> http://awesomescreenshot.com/0944pynyd9
<rpadovani> chromium Version 41.0.2272.76 Ubuntu 15.04 (64-bit)
<rpadovani> (diff side by side)
<rpadovani> on Firefox too
<cjwatson> rpadovani: Please file a bug on loggerhead
<rpadovani> sure
<cjwatson> can't look properly now
<rpadovani> https://bugs.launchpad.net/loggerhead/+bug/1436483
<mup> Bug #1436483: The diff page side by side has broken layout <loggerhead:New> <https://launchpad.net/bugs/1436483>
<rpadovani> oh, new commits to loggerhead since beginiing of 2013, sweet
<rpadovani> I love how you continute to improve lp :D
#launchpad-dev 2015-03-28
<robfrawley> so, what's the consensus on bzr?
<robfrawley> is more often used or not used
<robfrawley> *is it
<wgrant> robfrawley: I'm not quite sure what you're asking.
<wgrant> It is trivially true that there are more situations in which bzrÂ hasn't been used than those in which it has.
<wgrant> But that's not a particularly meaningful question :)
#launchpad-dev 2016-03-30
<wgrant> them
<timrc> cjwatson: re: http://www.chiark.greenend.org.uk/~cjwatson/blog/re-signing-ppas.html -- not groking something.  You state "Then last week I finally got around to timing things on one of our staging systems so that we could estimate how long a full run would take. It was taking a little over two seconds per archive,".  You then did some optimizations and then state "On production, this took about 80
<timrc> minutes for 1761 affected archives. This is over two seconds per modified archive".  Am I missing something?
<timrc> I would be interest in knowing what the *actual* performance improvement per modified archive was.  Since it seems the avg is skewed by skipping over unmodified archives.
<timrc> interested*
<cjwatson> timrc: two seconds per modified archive when running over all archives, but as you say the unmodified archives used some time too.  it actually wound up as around .4 seconds for each modified archive.
<cjwatson> iyswim
<timrc> cjwatson: Ah, nice.
<cjwatson> i.e. the final runtime was about 80 minutes, which involved stepping over about 70000 archives of which 1761 were modified.  do the divisions and it makes sense :)
<cjwatson> the production system is similar hardware to the staging one, but quite a lot busier.
<cjwatson> hence a bit slower.
<timrc> Right but relative performance improvement was nice.
<timrc> Nice work!
<cjwatson> timrc: I've clarified the wording a bit now.
#launchpad-dev 2016-03-31
<wgrant> cjwatson: How extensively did you test your requests timeout port? I've possibly found a regression in the new urlfetch, possibly decoding excessively.
<wgrant> Am digging.
<cjwatson> Not very extensively.
<wgrant> gpg key reactivation crashes, as the urlfetch result is not a str.
<wgrant> I assume it must be a unicode.
<cjwatson> Huh, I seem to not have ever received the review you posted eight days ago.
<cjwatson> Weird.
<cjwatson> I think my testing was limited to the test suite.
<cjwatson> Possibly when I originally put it together I tried it against turnip too.
<wgrant> Which review did I allegedly post eight days ago?
<wgrant> Oh.
<wgrant> timeout-with-requests never landed.
<wgrant> Then what on earth broke this.
<cjwatson> Indeed, that review.
<cjwatson> Must have somehow deleted it by mistake
<wgrant> Works on prod, so I guess a keyserver firewall or something.
<wgrant> How odd.
#launchpad-dev 2017-03-29
<neutrino__> Hi. I'm experimenting with building recipes on launchpad and noticed that the build logs include email addresses in clear. Is there any way to disable that?
<cjwatson> neutrino__: Not at present, sorry, though we do consider that a bug: https://bugs.launchpad.net/launchpad/+bug/620137
<mup> Bug #620137: Cannot choose which email to use for DEBEMAIL in recipe builds <lp-code> <recipe> <Launchpad itself:Triaged> <https://launchpad.net/bugs/620137>
<neutrino__> cjwatson: OK, thanks for letting me know!
<cjwatson> neutrino__: It might not be a terribly difficult fix if you have some time to set things up and know some Python.  We could probably reasonably fall back to config.canonical.noreply_from_address if requester.hide_email_addresses is set.
<cjwatson> (it's in lib/lp/code/model/recipebuilder.py)
<cjwatson> That isn't quite the same as providing a selectable alternate address for recipe building as the literal text of that bug asks for, but perhaps it's enough.  Although it would be arguably somewhat surprising in some cases ...
<neutrino__> cjwatson: I know Python and have a launchpad account, but I never contributed to the launchpad code itself. I can have a look though at that file when I get some time. It's a pity that this hasn't been fixed for so long.
#launchpad-dev 2018-03-26
<antenore> Hi all. I'm looking for documentation that explain how to structure project in order that is PPA and Ubuntu "compliant". In particular I'd like my project translations, currently under src/po, are found and imported automatically. Was looking in the PPA doc but didn't find it
<antenore> typo %src/po%remmina/po%
<antenore> I understand that this is possible from this https://wiki.ubuntu.com/Translations/TranslationLifecycle#Import_to_Launchpad_translations but I cannot find informations on ehre the build process expect the translations
<antenore> asking at #launchpad as I think this channel is not for support, sorry
<cjwatson> Yep.  Answered in #launchpad
#launchpad-dev 2018-03-30
<xnox> cjwatson, juliank - downloaded SKS webserver source code, to browse what it supports.... And I think everything what we need already exists
<xnox> https://keyserver.ubuntu.com/pks/lookup?op=get&exact=on&search=0x2d9df1e22f3416238d46f49f157951fe4031d287
<xnox> bah wrong url
<xnox> https://keyserver.ubuntu.com/pks/lookup?op=get&options=mr&exact=on&search=0x2d9df1e22f3416238d46f49f157951fe4031d287
<xnox> downlaods ASC armored public key, with exact match on the fingerprint.
<xnox> it's not clean, but it's perfectly usable by the new enough apt.
<xnox> (in the web-browser that gets downloaded as gpgkey.asc)
<cjwatson> xnox: I mean, we could, but I'd rather just use the existing interfaces that LP already uses.
<cjwatson> i.e. retrieve the key in the usual way (which is op=get&exact=on) and then format it ourselves
<cjwatson> that way we can cache it using our existing strategies
<xnox> as in launchpad.net to proxy the results for that.
<xnox> without 'options=mp' one gets and html page with header bar; with 'options=mp' one gets plain text, armored key
<xnox> so if you are adding code in launchpad to fetch keys, 'options=mp' is useful.
<cjwatson> We already have code in Launchpad to fetch keys
<cjwatson> lib/lp/services/gpg/handler.py
 * xnox looks
<cjwatson> So I'd rather fetch the key in the usual way we already do, and then use gpgme to armor it
<cjwatson> Because that way the same keyserver result can be cached locally and has a better chance of being useful for other requests
<xnox> ack
<cjwatson> In fact we already have all the necessary code.
<cjwatson> PymeKey.export will do it
<cjwatson> (Because PymeKey._getContext sets armor = True
<cjwatson> )
<cjwatson> So it should literally be just handler = getUtility(IGPGHandler); pub_key = handler.retrieveKey(fingerprint); key_text = pub_key.export()
<cjwatson> Sorry, I didn't realise you thought we thought we'd need any special keyserver support for this.
<cjwatson> Could have saved you the bother.
<cjwatson> wgrant: Could you re-review https://code.launchpad.net/~cjwatson/launchpad/job-oops-timeline/+merge/341554, please?  I added some gadgetry to avoid BranchScanJob producing giant timelines, since that seems likely to be the worst offender.
<wgrant> cjwatson: Ah, clever.
<cjwatson> Thanks.
<cjwatson> Also, if you reference any kind of model object from your timeline then the resulting reference cycles are *hell* to debug.
<cjwatson> I tried using objgraph which sort of vaguely helped a little but also produced enormous output.
