#launchpad-dev 2010-05-31
<wgrant> lifeless: security-related and private are separate flags.
<thumper> maxb: ping
 * thumper wonders if maxb is up at 2am
<lifeless> wgrant: at one point one implied the other
<wgrant> lifeless: If you check the security vulnerability checkbox when filing a bug, it will start private as well.
<wgrant> But that's about it.
<poolie> at this point 'security' is almost just a special tag
<thumper> poolie: a tag that adds heat :)
<wgrant> I thought the heat stuff was gone now.
<wgrant> But maybe it isn't.
<thumper> ah... I don't think so...
<thumper> I could be wrong
<thumper> it would be a lot of work to toss out the window
<wgrant> thumper: I mean the security/private flags' contribution to it.
<wgrant> The actual flawed concept still exists.
<thumper> hah
<wgrant> Although it's much better now.
<thumper> I'm not clear on the heat calculation
<lifeless> it just became a stored procedure
<lifeless> which, I think, means noone will ever know again.
<stub> Its in Python in database/schema/trusted.sql
<stub> (need to break that file apart)
<lifeless> stub: yeah, I know; I was mostly joshing
<lifeless> I think its a bit sad a stored proc was needed
<stub> It wasn't strictly needed. Things could have been refactored to calculate heat for batches of bugs at a time and would be more efficient than the current stored procedure.
<stub> Though that would have probably meant encoding the algorithm in SQL...
<lifeless> I thought storm can get pretty close to arbitrary sql?
<poolie> back onto dkim stuff
<poolie> spm, are you home?
<lifeless> poolie: see facebook
<lifeless> air canada airplane fail
<stub> Maybe, but more complex stuff is so under documented SQL gets used instead. I don't think anyone on the team can drive Storm to that level.
<poolie> urk
<poolie> can't he fix it?
<poolie> i thought sysadmins were there to abstract over hardware issues
<adeuring> good morning
<noodles775> Hi adeuring .... Can you open: https://launchpad.net/ubuntu/+ppas  for me?
<noodles775> And turn off the edge redirection (bottom-right link on the page)
<noodles775> adeuring: nm... I've got help from StevenK :)
<wgrant> When's the release?
* wgrant changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.05 | PQM is in release-critical mode | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
<noodles775> wgrant: I haven't seen any email to launchpad-announce either... I'll send an email to sinzui (who is checking email afaik).
<wgrant> noodles775: Thanks.
<henninge> anybody here used lp.testing.record_statements before?
<henninge> I want to count SQL statements but I only get zero. :(
<noodles775> henninge: I haven't... and I'm assuming you've seen the example in lp.testing.tests.test_testcase.TestRecordStatements.
<noodles775> henninge: actually... note the test_store_invalidation_counts there (store.flush and reset).
<henninge> noodles775: you are assuming wrong ... ;)
 * henninge goes to look
<noodles775> Great... then the answer is probably there :)
<henninge> noodles775: My error. Lazy execution. I used it on a method that only returned a (storm) iterator, so no statements happened until that was actually iterated over ... ;)
<henninge> Thanks for the pointer, though. ;)
<noodles775> Aha. np.
<thumper> henninge: I've used record_statements :)
<thumper> given that I wrote it
<thumper> although I see you've solved your problem
<henninge> I saw that but I expected you to be in bed .... ;)
<henninge> Yes, I did. Thanks a lot!
<thumper> just heading that way now :)
<henninge> good night ;)
<sinzui> henninge, ping
<henninge> sinzui: Hi! ;)
<sinzui> henninge, can you verify this issue is fixed: https://bugs.edge.launchpad.net/rosetta/+bugs?field.tag=qa-needstesting
<sinzui> your team is the last to land...it was QA-OK
<sinzui> henninge, I think you may have already verified it as OK if you have use the language js or the js on the import queue page lastg week
<henninge> sinzui: let me try right now ... ;)
<henninge> Argh, login is broken on staging. I cannot log back in ...
<henninge> I get a nice django error on loging.staging.lp.net
<henninge> sinzui: but it worked on the languages page. Cannot try out imports atm.
<henninge> and actually, I have not looked at that page lately. Shame on me.
<henninge> oh, edge should have it, too, right?
<henninge> sinzui: QA done
<sinzui> thanks
<sinzui> henninge, I believe you team is now down with QA. Are you ready for release?
<henninge> sinzui: hey, danilos's still around, my stint only starts tomorrow ... :-P
<henninge> sinzui: but I not aware of ony issues, so I consider us ready.
<sinzui> fab. Since you start on the day of release, I think it is best you are happy with this
<henninge> good point
 * henninge goes to look at the milestone more closely
<noodles775> sinzui: I'm about to EOD, so just FYI, we got most of our soyuz qa done this morning. You'll see two remaining qa items... jelmer is currently working on 503149, and StevenK on 557714.
<noodles775> Night all.
<lifeless> bug tag bugs -> malone still, right ?
<lifeless> I want a metronome in launchpad
<lifeless> something like:
<lifeless> when a series branch has the first commit done to it after a release
<lifeless> start counting
<lifeless> 1 month later nag all the series reviewers / committers to do a release.
<sidnei_> +1
 * maxb wonders what thumper wamted to talk about
<thumper> maxb: hi
<wgrant> sinzui: Is the namespace pollution useful?
<wgrant> He has dozens of teams masquerading as being for projects with which he has no relationship.
<sinzui> wgrant, no it is not.
<sinzui> Do you want me to delete his useless teams?
<wgrant> I think they should probably go. They serve no purpose, and are confusing.
<sinzui> I do not think I am a good person to judge this. I world personally (and do) delete projects I am certain are useless. I will be happy to give someone else those teams if they can help other communities
<sinzui> wgrant, His misuse of teams is not very different than how *most* users register projects that help no one
<wgrant> sinzui: But placeholder project registrations serve a purpose.
<wgrant> And they say "Does not use Launchpad for development"
<sinzui> Most people who register a project claim a name, There will never be code
<wgrant> But they claim the name for a purpose: so they can link bugs upstream.
<sinzui> I will ask the user what he intends do do with all this teams, maybe he wants to invade some OSS hosting site
<wgrant> These teams have no similarly good reason for existence.
<thumper> sinzui: I can't find out how to say "Doesn't use launchpad for code hosting'
<thumper> sinzui: have we got rid of that?
<sinzui> YES
<sinzui> thumper, it is in the model, but I do not know of a reason why a project needs to official use anything.
<thumper> sinzui: fair enough
<thumper> but it is saying 'uses launchpad for branches'
<thumper> when I registered the upstream project
<sinzui> thumper, I would we switch to enums that explain how project can use or integrate with Lp services. hostests/not-hosted has been a disservice
<thumper> it seems a bit weird
 * thumper nods
<sinzui> thumper, the official uses in 3.0 control the portlets on project pages, code does not have any portlets.
<thumper> yet
<sinzui> thumper, Since we decided to extend the Involvement portlet is *not* about what the project owner's ego, anyone can use code if it is configured
 * thumper nods again
<sinzui> What should be different if you check official_code?
<wgrant> sinzui: When that flag is off, the Launchpad project should indicate in a very obvious manner that it isn't actually where the code is.
<poolie> hi there
<poolie> wgrant, could you sometime do a bit of a security review of the dkim lep and patch?
<sinzui> If someone were to provide some design help and use cases, that would help.
<sinzui> wgrant, I really do not know if we care if the project use Lp officially. What is important is that Lp can provide code regardless of where it is initially hosted. Why do I care where gedit's code is not officially hosted in Lp
<wgrant> sinzui: Because I will Google gedit and find the Launchpad project.
<wgrant> Oh look, there's lp:gedit.
<wgrant> I will not notice that it's an obsolete import from two years ago.
<sinzui> I am still not conviced
<sinzui> We only had one item on the page in the past, with bugs regarding the fact that is was useless
<wgrant> Projects already often complain that Launchpad is misleading like this.
<wgrant> Let's not make it even worse.
<sinzui> we have not made it worse because what her made did never told the truth, and since Lp is not a simple hosting service, I do not have an answer for this issue
<wgrant> But it previously said "Does not use Launchpad for development".
<wgrant> Now all projects have their branches stored on LP.
<sinzui> Most project is disable (about 200 this month) used or did not use Lp as they claimed. I can see by activity how the project is really used. I can see by the nature of a branch of bug if the project is a mirror of a project hosted elsewhere.
<wgrant> sinzui: How?
<wgrant> I am a user who wants to submit a patch.
<wgrant> What on Launchpad tells me that it is not the right place?
<sinzui> I think the code team has spent years in this domain and have not solved it
<sinzui> I do not think I can in the next 3 months
<wgrant> Well, until not long ago you could make the project page say "Does not use Launchpad for development."
<wgrant> Now you cannot.
<wgrant> That was at least something.
<wgrant> Not nearly enough, but something.
<sinzui> Where would this text be?
<sinzui> On the code pages perhaps?
#launchpad-dev 2010-06-01
<wgrant> It should be on all pages relating to the project, really.
<wgrant> But yes, particularly a big warning on the code pages.
<sinzui> I could do as bugs has done (though correctly) to ensure apps do not work until someone enables them and state clearly who they represent
<wgrant> But branches are useful regardless.
<wgrant> They just need to be clearly unofficial.
<sinzui> Since We have competing communities, I feel pretty powerless to make everyone happy.
<sinzui> So the code team could make the code pages clear
<wgrant> LP is the only project hosting site that I know of that requires placeholder registration. It needs to go out of its way to do it right.
<sinzui> I know. I feel it is wrong that I am working on this
<sinzui> I an need to go thought 100s (and I really wish that number was an exaggeration) of bugs in all the apps to see what services users are trying to enable or configure, how every step fails. And these bugs are ancient
<wgrant> Launchpad already gives lots of projects enough reasons not to use it.
<wgrant> Let's not get people angry with it before they even start using it.
<sinzui> please help me then. where will people *see* this message?
<wgrant> code.launchpad.net/gedit, at least.
<sinzui> What is the message? what does it mean by not official?
<wgrant> launchpad.net/gedit needs it to be obvious too.
<wgrant> But ideally it should be in the header.
<sinzui> Those are code teams pages
<wgrant> Yes, and this is a Launchpad-wide problem.
<poolie> it seems reasonable to me to show a header across all those pages saying "gedit's real home page is at http://blah (tell me more)"
<wgrant> Right, something like that.
<wgrant> And the name in the header could be something like "gedit in Launchpad"
<wgrant> With poolie's suggestion.
<wgrant> To indicate that this isn't really it.
<wgrant> But, well, this is probably why we have UI designers.
<sinzui> AI approved the reuse of the Involvement portlet so that code was always used. I could put text back on the overview page, but where? what message? The user is trying to take an action and I do not know where he is looking on to inform him this project officially uses lp hosting
<poolie> sinzui, i think probably every page under that project
<wgrant> On every page, yes, but probably also an obvious warning in an extra portlet at the top of the project index.
<poolie> also the "tell me more" gives us a chance to explain
<poolie> why we have this page here at all
<poolie> and what you can do with it
<sinzui> Actually, I think the issue is really about the reverse situation as you suggested. The page should state *where the code is officially hosted*. This is the same stupid problem with bugs, where I can tell Lp where bugs are tracked, but LP will never tell anyone else what I said
<wgrant> This sound related to the "let me point the app tabs to other URLs" suggestion that pops up every so often.
<sinzui> The home page is a poor argument. Lp does not want to host any project's home page. There are all hosted some where else.
<poolie> s/home page/official site
<sinzui> wgrant, I think it is. I think for example I can have two community doing support. With using Lp Answers, and the upstream community using a mailing list. For the good off all other users/communities, Lp should state the situation and the someone make a choice
<poolie> ah
<wgrant> sinzui: Who is someone?
<poolie> it's a bit like the way facebook has pages about people who don't use facebook
<poolie> famous people at least
<poolie> some find it a bit creepy
<sinzui> I should be able to choose how I want support and the community I contact
<poolie> they may be doing better than launchpad at at least making it clear the actual person isn't there
<wgrant> sinzui: The user shouldn't able to; the project owner should.
<sinzui> I am not sure any owner has more authority than any other community.
<wgrant> The owner must be able to tell Launchpad what to do.
<sinzui> poolie, I do believe no service should be enabled until someone states they are using it.
<poolie> it depends what you mean
<sinzui> So I cannot use answers to support a project? Why can I not have Lp branches because a project uses gnome git?
<poolie> ah
<poolie> i think we should just make the context clear
<wgrant> sinzui: You can't use Answers to support a project if you are some random, no.
<sinzui> I do not think there is a on/off. There is several states (hence my suggestion of an enum) that allows Lp to help multiple communities share projects
<sinzui> Well I can be an answer contact for any project
<wgrant> You can't be.
<wgrant> LP exists to serve the project.
<sinzui> No it does not
<wgrant> If the project's owners do not want LP to do something, it should not do it.
<sinzui> that is a common mist conception. Lp could never host all the project that produce code that make up Ubuntu
<sinzui> Lp exists to help communities share information, and to lower the barrier of controbution
<jelmer> wgrant: Why is it an issue for Launchpad but not for e.g. sites like ohloh that also just track projects?
<wgrant> jelmer: Does Ohloh provide services like this?
<sinzui> GNOME should not need to require zeitgeist to abandon Lp merge proposals. GNOME should be happy to know that Lp can help them with their own project dependencies. Lp should provide GNOME with excellent data about bugs and fixes.
<jelmer> wgrant: yes, it allows users to post reviews of projects as well as mirroring the downloads for projects, project description, vcs locations
<wgrant> jelmer: It allows one to link to external download pages, and just displays the real VCS location. That sounds reasonable enough.
<wgrant> And presumably the owner can tell it not to link to an external download page.
<sinzui> I would be happier if it were easier to see real VCS information. I have more than a 1000 projects that need love
<wgrant> Whereas they can't tell Launchpad to not accept questions from people.
<jelmer> wgrant: I think the main problem is the perception that Launchpad is a project-endorsed resource
<wgrant> jelmer: Right.
<wgrant> Ohloh exists solely as a secondary resource.
<jelmer> right, and everybody regards it as that, wheras most people seem to think of Launchpad as a hosting site, not as a free software project tracker
<wgrant> Well, most people think of Launchpad as the site that hosts Ubuntu.
<jelmer> if we can make it clearer when a project is just tracked by lp and not hosted on lp I think this would be less of a problem.
<wgrant> Exactly!
<sinzui> It is bot binary though
<sinzui> not
<sinzui> a project can use git hub for hosting, but still use Lp MPs
<wgrant> That's a very strange use case.
<lifeless> or be on gnome.org and use mps :P
<lifeless> wgrant: not at all
<wgrant> lifeless' case sounds more reasonable.
<lifeless> folk are doing it in fact
<wgrant> But I guess they are similar.
<maxb> thumper: hi
<wgrant> If a little concerning.
<thumper> maxb: hi
<thumper> maxb: I've updated the vcs-imports celeb permissions
<thumper> maxb: are you interested in helping qa?
<thumper> maxb: I could add you to the team, and you could make sure you can't do non-import related bits
<thumper> maxb: as long as you don't use production :)
<maxb> heh
<maxb> did you want to do it now? i can grab a computer
<maxb> (i'm on an android phone at prrsent)
<jelmer> sinzui: btw, the "Configure project branch" link is really nice
<thumper> jelmer, sinzui: my frustration with that is that it isn't used consistently
<thumper> maxb: I could add you to the team and you could look tomorrow if you like
<jelmer> sinzui: it saves quite a few roundtrips when registering a new upstream project
<thumper> although I tried to import a subversion branch as a bazaar branch by mistake
<jelmer> thumper: there  being multiple forms for setting up a project branch you mean?
 * thumper nods
<thumper> I'd like to take the guts of that same page and have a single page for registering a new branch
<maxb> thumper: i can have a quick look now and explore more exhaustively tomorrow
<thumper> it also doesn't use the same permissions / widget checks as the official import page
<maxb> laptop is booting
<jelmer> thumper: we also have all the infrastructure to just detect what type of vcs is present at a remote URL, it'd be nice to simplify the UI
<thumper> jelmer: that would be nice
<thumper> maxb: you should be in the team now
<maxb> IIRC the key things to look at were the new-project workflow, and the code import machines.
<thumper> something like that
<thumper> maxb: actually I should have just added you on staging :)
<thumper> maxb: as you could at least try to tweak the machine settings there
<thumper> maxb: you are now in the vcs-imports team on staging
 * thumper checks that the right revno is on staging
<thumper> maxb: staging should be good to play with
<thumper> maxb: you can try anything there :)
<maxb> or rather, on staging I can't try to tweak the machine settings, so I infer your branches have landed there already
<maxb> It all looks as expected - the one thing I haven't done yet is verify that I don't have any boxes I shouldn't in the new project workflow
<maxb> and I've now done that on staging
<maxb> Modulo the essential trickyness of QA-ing a negative, I'd say it's all fine
<maxb> Erm, pear's /home/importd/.bazaar/subversion.conf is apparently broken: http://launchpadlibrarian.net/49455906/maxb-guice-trunk.log
<maxb> the same failure is apparent in other branches when imported on pear
<wgrant> Yeah, it does that sometimes.
<wgrant> Happens sometimes with concurrent bzr-svn imports.
<maxb> Do we need act-of-losa to get it fixed?
<mwhudson> yes
<maxb> thumper: OK, I'd say QA complete - did you want to deactivate me from ~vcs-imports pending an official ratification of it being ready for community members, or shall I start reviewing imports from time to time?
<lifeless> maxb: if you're on staging, then you're not activated on prod anyway
<thumper> lifeless: I added him on prod too :)
<poolie> hey
<ScottK> Hello
<poolie> first off, thanks for commenting
<ScottK> BTW, it would be nice if non-developers could subscribe to LP wiki pages.
<poolie> there had been a bit of silence aside from vague 'that sounds nice'
<poolie> ah is this the 'action not allowed'?
<poolie> that's a bug
<thumper> lifeless: did you end up testing those merge proposal changes you did early in the cycle?
<ScottK> poolie: Yes.  action not allowed.
<thumper> lifeless: I'd just like to sign off the qa
 * poolie looks
<poolie> scottk, https://bugs.launchpad.net/bugs/586601 has a workaround too
<ScottK> poolie: In any case, what I read in the bug and the wiki page is very concerning.
<mup> Bug #586601: dev wiki toolbar has 'subscribe user' but not plain 'subscribe' <Launchpad Development Wiki Moin theme:New> <https://launchpad.net/bugs/586601>
<ScottK> OK.  Thanks.
<poolie> so
<ScottK> Subscribed now.  Thanks.
<thumper> poolie: https://lp-oops.canonical.com/oops.py/?oopsid=1612BM1
<thumper> poolie: that is (one of) the branch mail job that had memory issues
<ScottK> Also, I'm somewhat laggy at the moment, so if I don't reply, it's not because I'm ignoring you.
<thumper> poolie: it is the kernel
<thumper> poolie: lp:~vcs-imports/linux/trunk
<poolie> scottk, let's try to unpack "to mean anything positive"
<poolie> i don't assume it means the user specifically authenticated that message
<poolie> in the sense that typing a gpg passphrase could mean "yes really"
<ScottK> If you allow the message to do anything that requires authentication, you have.
<poolie> i do think it means we can be incrementally more sure that the message comes from who it claims to come from
<poolie> would you agree?
<ScottK> No.
<ScottK> It means you can be sure it came from the domain it purports to come from, but it tells you nothing about the user.
<lifeless> thumper: sign off on it
<thumper> lifeless: ok
<lifeless> thumper: it doesn't matter if its right or wrong, until we do the other things you wanted, we're not using the queue stuff anyhow
<lifeless> thumper: at your request :)
 * thumper nods
<thumper> :)
<ScottK> The only way to believe it says anything about the user is to believe in implementation details of proprietary webmail services that you really have no insight into and even if they actually do what you think, could change without warning.
<poolie> no, nothing about this is specific to proprietary webmail
<lifeless> thumper: that said, I'm entirely confident of my changes anyway, as they a) have tests and b) weren't at the UI layer.
<thumper> lifeless: ok
<poolie> ok, so are you talking about a case like this:
<ScottK> poolie: Implementation details of the sender.  The ones mentioned are proprietary (mostly) webmail providers.
<lifeless> thumper: its not in the risk sector for me
<poolie> they're just examples
<ScottK> OK.
<poolie> anyhow, so a case like this: smtp.canonical.com lets employees connect over port 529 authenticate and send mail; but it doesn't check that the mail they send is from the user they authenticated as
<ScottK> Right.  That's pretty typical.
<poolie> therefore i can get a mail signed that claims to be from joe@canonical.com
<ScottK> All you really know is that it passed through smtp.canonical.com and they signed it.
<poolie> ok, interesting
<ScottK> DKIM very explicitly makes no assurance that the From address is in any way valid or correct.
<ScottK> Just that since the header is signed, it didn't get modified in transit.
<poolie> hm
<poolie> i understand that this is up to their local policy
<poolie> it would be a bit weird for them to just sign all outgoing mail
<poolie> but perhaps that really is a typical deployment
<poolie> especially if it's just stuck in front of an existing server
<ScottK> Now you may do a security analysis and determine that for some actions (maybe bug status), that is sufficient status.  The action is reversable and doesn't really hurt, but you have to recognize that you don't really know who sent the mail.
<ScottK> Yes.
<thumper> ScottK: I don't think we want to do that type of check
<thumper> ScottK: in fact we do allow that for unsigned email
<poolie> so there are hints in the rfc
<thumper> ScottK: things like general comments
<poolie> and i'll refrain from quoting it, but it does say that they would authenticate the submitter before signing it
<ScottK> In fact, if you get a dkim signed mail d=kitterman.com and From scott@kitterman.com, I really did send it or I have a bad bug, but I've gone beyond what is typical and there's no way to know I've done it.
<ScottK> Yes, but what does authenticate the sender mean?
<ScottK> Typically it means that the sender is an authorized user of the MTA.
<poolie> i assume that means username/password or similar authentication of the smtp submission
<lifeless> thumper: we do ?!
<ScottK> poolie: Yes.
<thumper> lifeless: yes
<lifeless> thumper: I get mail rejected when I try to do status changes without signing it..
<thumper> commenting doesn't require signed email
<thumper> lifeless: status changes do require signing
<thumper> a plain comment doesn't
<poolie> yes, as does filing a new bug
<lifeless> thumper: so the thing you replied two was about bug status
<lifeless> thumper: you can see my confusion
<thumper> what I was referring to was: we already have a distinction
<poolie> thumper, that oops would be good to file a bug about
<thumper> between trusted and weakly authenticated
<thumper> I don't think we want another in the middle
<poolie> me either
<poolie> the question here is whether dkim-signed mail can be treated as 'adequately authenticated'
<poolie> for things like changing bug statuses etc
<thumper> well...
<poolie> if there are a non-negligible number of domains that will sign any mail sent through them
<poolie> then it may not wokr
<thumper> I can set up any number of identites with fastmail
<thumper> will it sign all of them?
<ScottK> So you can see the concern I expressed in the bug?
<poolie> the question is will fastmail let me send mail that pretends to be tim?
<poolie> that would be pretty damn weird if it did
<poolie> ScottK, is that the only concern?
<ScottK> That's the most important one.
<thumper> that is the sort of answer we should get before enabling it
<poolie> so we could have a whitelist
<ScottK> I think an implementation that depends on polling commercial email services and asking the internals of their implementation details is not a really great idea.
<poolie> or we could get this from adsp, though i haven't looked into that much
<poolie> why do you say 'internals'?
<poolie> it doesn't really matter how it's implemented
<ScottK> You need to know if they allow cross user forgery.
<poolie> right
<ScottK> That restriction is an internal implementation issue for them.
<ScottK> There's no protocol basis for discovering it.
<ScottK> I can tell you from experience with SPF and DKIM deployments that senders routinely deploy this stuff and barely understand it.
<poolie> if somebody decided to turn on dkim signing in ubuntu, would it be likely to allow cross-user forgery?
<ScottK> Currently Ubuntu developers have the ability to send mail through fiordland at least to Launchpad.  I don't know what checks they have in place.
<ScottK> At least you've got the data in Launchpad accounts to know what users should use what email addresses, but how you link that up to an SMTP time user authentication, I'm not sure.
<poolie> i think you have a really good point that this will probably be deployed badly
<poolie> but ... if cross-
<poolie> cross-user forgery is common, it seems to substantially defeat the point of dkim
<ScottK> It depends.
<poolie> for example they talk a lot about showing a signed From address to the user as authentic
<poolie> or trustwotrhy
<spiv> In the case of webmail providers, you arguably need to know if they have if they have CSRF flaws etc.  Or more simply you need to know the user hasn't leaked their password accidentally... you can't require absolute trust, because nothing is absolutely trustworthy.
<ScottK> The more common use is to use the d= domain as an input token to a reputation system.
<ScottK> So over time you can measure which domains tend to send good mail and which one tend to send bad mail so you can treat them differently.
<wgrant> spiv: But if you can't trust the webmail providers, they can already reset your Launchpad password.
<ScottK> In that case, you don't really care about cross-user forgery.
<ScottK> You only really care if you try to believe the from address is somehow validated.
<ScottK> It may be, it may not, but it's no part of DKIM to say.
<poolie> ah
<ScottK> All DKIM can tell you about the from address is it wasn't altered in transit.
<ScottK> It was, in fact, contentious in the working group whether or not to require from be signed.
<ScottK> For this exact reason, people would read too much into it.
<poolie> i see it is required
<ScottK> In the end, it's required to be signed only because it's a required part of the message body.
<ScottK> The fact that it's signed, is helpful for the policy component, ADSP, that was developed after the base DKIM signing spec.
<poolie> proposition: a signature by kitterman.com on "From: scott@kitterman.com" indicates kittermain.com asserts this message is from scott@kitterman.com
<poolie> you don't think that's true?
<ScottK> No.
<ScottK> It asserts it's signed by kitterman.com and kitterman.com takes responsibility for it.
<ScottK> If it verifies, you can also trust the signed parts of the message were not modified in transit.
<poolie> but "takes responsibility for it" only in the sense of "takes responsibility for it not being spam" not "takes responsibility for it not being forged"?
<ScottK> Takes responsibility so that you can blame it (in a reputation sense) if it's "bad".
<poolie> ok, so i see your point
<poolie> but the rfc really does seem to allude to a larger scope than that
<ScottK> It is of two minds about it.
<poolie> mm, i understand it may be contentious
<ScottK> In the end, that's all it really does, but there are lots of entities that want to use the DKIM domain as an input to secret sauce reputation systems.
<poolie> there are specific examples saying that a from field signed by the relevant domain may be trusted
<poolie> it may be the deployment is so bad this doesn't work
<poolie> which would be kind of sad considering how long this has been in coming and how new it now is
<ScottK> I think for applications where domain level information is sufficient, it has a lot of potential.
<poolie> ok
<ScottK> Unfortunately, in the LP case, you need more granularity than it can provide.
<lifeless> can't they sign the From: field?
<wgrant> The difficulty is that most implementations do not verify that the authenticated sender is authorized to send using a particular address.
<poolie> well, citation needed for 'most'
<poolie> but it's certainly possible some don't
<wgrant> The common documented Ubuntu setups don't.
<lifeless> wgrant: do they claim that they do though?
<ScottK> lifeless: Signing From just means it wasn't modified in transit.
<wgrant> lifeless: How do we tell?
<lifeless> wgrant: is From signed, no ?
<wgrant> lifeless: Heh, no.
<ScottK> Actually you can't actually tell if they claim it's valid.
<lifeless> lets assume we could tell, what header would we use in launchpad to associate the mail with the account
<lifeless> ?
<ScottK> If you're talking DKIM, you're talking body From.
<ScottK> DKIM is silent on envelope identities.
<lifeless> and do servers routinely sign body From without sender verification ?
<ScottK> There's no requirement in the RFC for it.
<ScottK> Or to put it slightly differently ....
<ScottK> The senders are verified, but generally they are verified to be authorized users of the MTA, not generally that they are specificaly authorized to use the From they are using.
<lifeless> are they able to sign the mail as being from their domain *without* signing body From ?
<ScottK> No.  Signing from is required because it's a mandatory part of the message.
<lifeless> ok
<lifeless> so we're screwed
<poolie> well
<ScottK> I think DKIM verification is not suitable for applications where you need to have some assurance that you have mail from a specific user.
<lifeless> anyone doing DKIM as a 'this isn't spam' effort is indistinguishable from someone doing DKIM with user granularity
<wgrant> Well, someone could define another standard for a signed header that verifies From. But surely such a thing already exists.
<ScottK> The D in DKIM stands for Domain for a reason.
<lifeless> ScottK: some domains may be good enough to use
<ScottK> wgrant: Yes, for this application you use GPG or S/MIME.
<poolie> i thought this was part of the point of DKIM beyond SPF
<poolie> well
<ScottK> lifeless: How would you know?
<poolie> given there is an exposure
<lifeless> ScottK: we could ask them
<poolie> does this actually matter
<lifeless> ScottK: they could tell us
<poolie> how many important users send mail from servers that allow spoofing
<ScottK> lifeless: My experience is a lot of providers wouldn't give you an answer you can rely on.
<poolie> and how bad is it if their mail is spoofed
<poolie> one could already cause a lot of trouble by forging unsigned mail
<wgrant> Status changes are reasonably trusted at the moment.
<wgrant> And used for workflow things in Ubuntu.
<ScottK> poolie: I don't think DKIM 'goes beyond' SPF, it tells you a similar thing about a different identity.
<ScottK> If you have a message where Mail From and From are the same (this is the 80% case) then an SPF pass and a valid DKIM signature tell you about the same thing.
<ScottK> The difference between what they tell you is for most purposes not important.
<lifeless> wgrant: so, if we turned on DKIM for providers that say 'we do per-user authentication on our submission port'
<lifeless> wgrant: would that really decrease the workflow trust?
<ScottK> lifeless: You need to be very careful and specific about how you ask that question.
<lifeless> ScottK: agreed
<ScottK> Also I'm reasonably certain a significant fraction of the Yes answers you get would be wrong somehow.
<lifeless> ScottK: 'the body from header in mail from our servers is restricted to addresses the sender can receive at by submission-time authentication'
<ScottK> I'm continually stunned at how shallow people's understanding of these technologies they are deploying is.
<lifeless> ScottK: where they are wrong, we disable DKIM for that domain again.
<ScottK> lifeless: To the extent you know.
<lifeless> ScottK: that is also a limitation on gpg
<ScottK> Less so.
<lifeless> ScottK: I don't see that it is knowable in either case.
<lifeless> and in both cases, when we have reason to doubt, we can disable it.
<ScottK> So let's say you go down this path ....
<ScottK> I'm kitterman.com and I want you to trust my DKIM signature.  How do I sign up?
<ScottK> Are you going to allow anyone to play that makes the correct assertions or is this just for the big boys?
<lifeless> first cut, lets say there is a DKIM page on dev.launchpad.net, it could say 'file a ticket in answers'
<lifeless> ScottK: Myself, I'd let anyone that comes along, makes [minor] personal contact and asserts that they do the right thing.
<ScottK> So there is an admin cost here.
<lifeless> and have a CHR accessible list to enable.disable folk
<lifeless> ScottK: as a start
<lifeless> ScottK: if it works well, make signup straight forward, with admins to disable and re-enable disabled domains.
<ScottK> Then there's the case of a provider (like say yahoo.com) that doesn't really care about LP and DKIM, but may have a small fraction of their users that do.
<lifeless> but thats more up front dev when we don't know if it will be a) popular b) work well c) not be a screaming mess
<poolie> mm
<lifeless> ScottK: so, if we believe they dtrt (e.g. by testing ourselves), we could turn it on.
<ScottK> You won't get a Yahoo dev filing tickets on launchpad.
<lifeless> ScottK: or we could say 'really please tell us' - and I'm positive we can track the right person at yahoo down.
<poolie> so this doesn't seem that different in principle from a mail domain that allows any user to read any other user's mailbox
<ScottK> poolie: Except it's a lot more common.
<poolie> exactly
<ScottK> Historically "is an authorized user" was enough of a check.
<poolie> well
<poolie> probably signing messages at all is still uncommon
<lifeless> commercial domain providers have a vested interest in avoiding forgery sent through their servers
<poolie> i don't know what fraction of deployed instances are borken
<poolie> but i should probably believe you if you say it's high
<ScottK> Now you also want people to check is an authorized user of the MTA and that they are using an identity they are authorized to use.
<lifeless> home and small business less so, because its not representing multiple entities
<lifeless> ISP's may be a very grey area.
<poolie> one would think that isps probably block outgoing forgeries
<poolie> but probably not all do
<ScottK> Part of the problem was that before email authentication, there was very little value in doing cross-user forger, so it doesn't happen much.
<ScottK> As soon as DKIM or SPF pass starts to mean something, then it's an attack that has value.
<poolie> so how about deploying this and not trusting the results, just logging them
<poolie> right
<poolie> but it's an attack that can potentially be fixed reasonably easily
<mwhudson> i thought the main thrust of this work was aimed at a particular domain that starts with 'g'
<ScottK> If you're in a position to know about it.
<poolie> yeah, and that's the other thing
<poolie> whitelisting about 5 domains will help a lot of people
<lifeless> mwhudson: its a particular provider that we know does DKIM, and supplies some vast fraction of LP user accoun email addresses
<mwhudson> lifeless: right
<poolie> then we can consider kitterman.com etc case by case
<lifeless> mwhudson: but its not the only very popular one ;)
<ScottK> poolie: I'd also consider SPF pass for a domain the same as a DKIM signature.
<poolie> and the largest senders are probably reasonably likely to get it right
<ScottK> Eventually.
<mwhudson> lifeless: it's by far the most popular though
<poolie> SPF is an interesting thought experiment
<mwhudson> i guess there's the apps-for-domains issue too, that kinds of messes things
<poolie> mwhudson, do you mean mwhudson.com being hosted by google?
<ScottK> poolie: Fundamentally a DKIM signature just tells you that the message passed through an MTA authorized by the domain owner and that it hasn't been modified in transit.
<mwhudson> poolie: right
<poolie> this can be accommodated by dkim, but it's not done at the moment
<ScottK> SPF tells you the first part, but not the second.
<ScottK> In transit modification is not a major risk on direct point to point transmissions.
<poolie> there's another difference which is that dkim seems just easier to implement later in the pipeline
<poolie> when we're examining a queued mail
<lifeless> mwhudson: yes, I want apps-for-domains too
<poolie> perhaps not substantially, but parsing the headers to work out when it went into our trusted network seems a bit messy
<ScottK> You'd want to implement SPF checking in the border MTA and then consume with the SPF recieved or Authentication Results header later.
<poolie> right
<ScottK> Spamassassin does this quite well.
<poolie> so in principle, if i sent mail direct from my ip to launchpad i think it would be ok to trust it for, say, voting on merge proposals
<ScottK> It's far more reliable than trying to grovel the connecting IP address out of the recieved headers later.
<poolie> by analogy i would be pretty happy to do that over http and it's just a bug that's not supported at the moment
<poolie> now eventually you could say there are some operations which should require really strong authentication
<poolie> but i think that's a different issue
<wgrant> Merging code is not something that requires strong authentication?
<lifeless> wgrant: voting != setting merge proposal status
<poolie> well, to be pedantic, i said voting
<wgrant> True.
<wgrant> But that's meant to change.
<lifeless> wgrant: voting is an input into someone setting the proposal status, and I'd want strong auth for making something mergable, but not for rejecting/needs-fixing etc
<poolie> but, what level of trust is needed to merge code?
<wgrant> Although it was apparently vetoed for reasons that are not completely obvious to me.
<poolie> not to put words in his mouth, but elmo commented on this that the current authentication is not unimpeachably high
<wgrant> No, the current authentication is crap.
<wgrant> Doesn't mean we need to pull it down further.
<poolie> in that people have long-lived sessions or stored passwords on their laptop, etc
<lifeless> so, how much? enough that *something* the user knows must be used.
<lifeless> or *has*
<poolie> wgrant, if i have a strongly authenticated connection to gmail
<poolie> i think i'd be happy to proxy that trust through to launchpad
<poolie> there is a chain there
<ScottK> poolie: You would also do well to support opportunistic TLS on your MX as well.  That can be useful too.
<poolie> on launchpad's incoming mx?
<ScottK> That helps reduce in transit visibility.
<ScottK> Yes.
<poolie> that would be nice
<ScottK> i.e. mx.canonical.com.
<ScottK> I just checked and you don't.
<lifeless> is it trivial ?
<ScottK> Yes.
<ScottK> At least in postfix.
<ScottK> If you start trying to do certificate verification, it gets hard.
<ScottK> But that's overkill.
<ScottK> (most MTA TLS certs are self-signed anyway)
<ScottK> That would be a relatively easy win for increasing the reliablity of the trust path.
<ScottK> https://docs.google.com/viewer?url=http://www.bits.org/downloads/Publications%2520Page/BITSSecureEmailFINALAPRIL1507.pdf is germane.
<ScottK> (that's the US financial industry best practices document for this area)
<poolie> ok, so, thanks very much for the background on this
<poolie> i was reading wg mail threads
<poolie> and people do seem to be of at least two minds
<poolie> so
<poolie> i would like to continue to push this patch for inclusion
<poolie> with the addition of a whitelist of domains where it's acceptable
<poolie> so we fix the big N
<poolie> and we can at least log and see how many pass or fail and why
<poolie> i should do some real work now :)
<ScottK> poolie: OK.  I'd also encourage you to consider using SPF similarly.  It's more widely deployed and should be pretty reliable for your use case.
<poolie> ok
<poolie> thanks very much for the feedback
<poolie> although it violates my cherished preconceptions i appreciate it :)
<ScottK> It's also well supported in Ubuntu.  I made sure.
<ScottK> Certainly.
<poolie> i might file a bug about opportunistic incoming tls
<ScottK> SPF is clearly a gross hack, but it's a useful one.
<ScottK> poolie: One other thing, the bug you filed on python-dkim, would you please include message samples that demonstrate the problem.
<poolie> ok
<poolie> my launchpad mp demonstrates the problem in its tests
<poolie> there are no real tests in dkim.py
<poolie> i will separate an example out
<ScottK> Thanks.
<poolie> ok, https://bugs.edge.launchpad.net/launchpad/+bug/588105 additional comments welcome
<mup> Bug #588105: launchpad incoming mx.canonical.com should support opportunistic TLS <Launchpad itself:New> <https://launchpad.net/bugs/588105>
<poolie> in a way it's good it's non-canonical staff doing the security review of it
<poolie> s/it/this
<adeuring> good morning
<deryck> Morning, all.
<deryck> gmb, concerning bug 570222.... if you cannot reproduce and gary_poster says a 2.6 builder is coming, perhaps we should wait and see when the builder gets going?
<mup> Bug #570222: checkwatches blows up when using XML-RPC on Python 2.6 <story-reliable-bug-syncing> <Launchpad Bugs:In Progress by gmb> <https://launchpad.net/bugs/570222>
<gmb> deryck, Yeah, I think that's a good plan. Also, this is another one in the 'record-which-bugtrackers-have-plugins/api' column; known which bug trackers to test against (other than gnome-bugs) would have made this a lot easier to work on.
<deryck> gmb, yeah, good point.  We don't have an open bug about that yet do we?
<gary_poster> deryck, gmb, it is RT 39005 FWIW
<gmb> deryck, Don't know; I'll check
<gmb> gary_poster, Thanks
<gary_poster> np
<gmb> deryck, Filed as 588287
<gmb> bug 588287, that is...
<mup> Bug #588287: Launchpad should record which bugtrackers have plugins / api <bugwatch> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/588287>
<deryck> gmb, great.  Thanks, man!
<maxb> thumper: (repeat question from yesterday) Hi, now that QA's done, do you want to remove my ~vcs-imports membership pending a ratification of it being ready for community members, or am I clear to actually review imports?
<gmb> deryck, What's dhrb?
<deryck> gmb, deryck-hodge-real-bug :-)
<gmb> deryck, Nice.
<deryck> gmb, just a shorthand for a quick-hit bugs list.  Something better than the gobby doc.
<mars> jml, ping, have some time for a question or two about subunit in the test infrastructure, and bug 587886?
<mup> Bug #587886: ec2 test mail reports SUCCESS when the suite fails <build-infrastructure> <Launchpad Foundations:Triaged by mars> <https://launchpad.net/bugs/587886>
<jml> mars, sure, I have just a little time
<mars> jml, mumble?
<jml> mars, I'm not set up for that right now, sorry
<jml> mars, IRC though.
<mars> jml, ok
<mars> rockstar or abentley, ping
<rockstar> mars, hi
<mars> hi rockstar
<mars> rockstar, I have a hung ec2 windmill test here, and there is a coincidental intermittent failure in test_branchcollection, and also a hang in the branch-related windmill tests
<mars> rockstar, here, I'll post the log
<mars> rockstar, check the end of this log: http://pastebin.ubuntu.com/442864/
<rockstar> mars, I think I've seen the branchcollection failure before, but I thought that got fixed.
<mars> rockstar, ok, may be a stale branch doing it?
<rockstar> mars, no idea.  I'm not sure what I'm supposed to discern from this though.
<mars> rockstar, actually, hold on a minute or two, the other two instances have hung as well - need to see if it is the same test that died
<rockstar> mars, truthfully, if I were looking into the ec2 hangs, I'd look at the differences between the buildbot instances (where we don't have the hang) and our ec2 test instances.
<mars> rockstar, true, haven't thought of that.  I'm tackling it from this perspective since I have also been messing around with the test_on_merge.py code
<rockstar> mars, okay.
<maxb> https://bugs.edge.launchpad.net/launchpad-code/+bug/327126, linked from https://dev.launchpad.net/ReviewingCodeImports, is private. I wonder if someone could assess whether it needs to be private?
<mars> rockstar, ok, looks like that intermittent test failure may not be the source.  The other branch hung for some other reason.
<rockstar> mars, cool.
<mars> rockstar, and the third branch passed everything just fine :/
<rockstar> mars, yeah, ec2 is becoming less and less reliable.
<mars> HA!
<mars> File "/var/launchpad/tmp/eggs/zope.testing-3.9.4-py2.5.egg/zope/testing/testrunner/runner.py", line 587, in resume_tests
<mars>     time.sleep(0.01) # Keep the loop from being too tight.
<mars> TypeError: unbound method exit_with_atexit_handlers() must be called with TwistedLayer instance as first argument (got int instance instead)
<mars> I have no idea what that error means yet
<mars> but I managed to capture it when killing off the hung ec2 testrunner.
<mars> rockstar, this is frustrating because I can see what is failing: I am getting a partial traceback on the console.  But the rest of the traceback is being held by the subprocess and Python buffers, so killing the process wipes those buffers out.
<rockstar> mars, so there's a twisted issue?  I'm not very good at debugging Twisted stuff, but I bet jml is.  :)
<rockstar> (He knows a little bit about Twisted)
<mars> rockstar, might be twisted.  Looking at the log, that may just be an error with the process shutdown, and unrelated to the original suite hang.
<rockstar> mars, maybe, but maybe if you fix that, you'll get a better traceback.
<mthaddon> maxb: howdy - have a few moments to talk about lucid launchpad-dependencies?
<maxb> sure
<mthaddon> maxb: so it seems the only package we're missing is spidermonkey-bin, which I've been told is in the xulrunner source package in the PPA?
<maxb> erm... define missing?
<mthaddon> (missing from stock lucid, that is, if you ignore python2.5)
<maxb> oh, right
<maxb> yes
<maxb> erm, and postgres-8.3
<mthaddon> maxb: but there's a xulrunner package in stock lucid - so it doesn't include the spidermonkey-bin we need?
<maxb> That is correct
<mthaddon> yeah, and if you change postgres 8.3 -> 8.4
<mthaddon> maxb: ok, cool thx
<maxb> The ubuntu mozilla team chose to not package spidermonkey because of upstream's refusal to maintain a stable ABI
<mthaddon> maxb: I think that's all I need for now - I'll let you know if I need more info
<maxb> www.launchpad.net..... ewww :-)
<mars> EdwinGrubbs or bac, quick question: how do I run the all-in-one JavaScript unit test suite?  Is there a wiki page with instructions?
<EdwinGrubbs> mars:  ./bin/test -vv -t test_yuitests
<mars> EdwinGrubbs, thanks
<EdwinGrubbs> mars: look at lib/lp/registry/windmill/tests/test_yuitests.py    Each lib/lp/*/windmill directory needs one of these files to run tests found in lib/lp/*/javascript/tests
<thumper> maxb: still around?
<maxb> hi
<maxb> (thumper)
<thumper> maxb: I think you are fine to garden imports
<thumper> maxb: the permissions that are on edge will be on production within a day
<thumper> maxb: I see that you've already been doing some, so that's cool
<maxb> I hit the retry button on all the ones that pear's glitch broke
<maxb> A question that I had - when an import goes to Failed because the upstream server has gone away, should it then be set Invalid, to get it out of the Failed list?
<lifeless> maxb: disabled I should think
<maxb> Do vcs imports have a disabled status?
<lifeless> they certainly used to :P
<lifeless> flag, not status
<maxb> oh... 'suspend' ?
<maxb> Would someone be able to review bug 327126 and see if it actually needs to be private?
<maxb> It is referenced on dev.lp.net/ReviewingCodeImports, but I can't access it
<mwhudson> maxb: i'll subscribe you
<mwhudson> maxb: done
<maxb> thanks
<thumper> maxb: suspended seems reasonable I guess
<mwhudson> yes, i think suspended makes sense for the place to put imports we'll never want to look at again
#launchpad-dev 2010-06-02
<mtaylor> thumper: if I'm going to propose some work on launchpad and I want to get direction approved, should I file a blueprint or a bug?
<mtaylor> thumper: or should I write a patch containing a user story and submit that :)
<thumper> mtaylor: you should have a pre-implementation call :)
<lifeless> mtaylor: chat with a dev
<lifeless> mtaylor: everything else is makework.
<lifeless> mtaylor: you could chat on the lp-dev list, here, or on voice
<mtaylor> lifeless: ok.... just wanted to be sure I was prepping for the right thing
<lifeless> if you need a persistent record, I'd start with the list, or a bug [perhaps]
<mtaylor> _I_ don't need a persistent record - I just wanted to make sure I was being friendly and all
<lifeless> whats your ideal, when someone does stuff for dribble?
<mtaylor> lifeless: depends on scope of work - if it's a new feature or something decent sized, we ask them to file a blueprint
<lifeless> what do you use the blueprint for?
<mtaylor> lifeless: tracking work against the task
<mtaylor> lifeless: they'll then tie their bzr branch to that blueprint, we'll use the blueprint during planning meetings to check progress on it
<mtaylor> etc.
<lifeless> mtaylor: but not for deciding on direction :)
<mtaylor> lifeless: well, not directly, no... although we'd certainly like to :)
<mtaylor> lifeless: but fair enough
<mwhudson> lifeless: say, do you have any particular opinions about pyunit-compatible test runners?
<lifeless> why yes, I do
<mwhudson> lifeless: which would you recommend? (he asks, half expecting "bzrlib's")
<lifeless> testrepository if you can
<lifeless> failing that testtools.run
<mwhudson> oh, testtools has a runner?
<mwhudson> didn't know that
<lifeless> and failing that your own glue to bzrlib.tests.$stuff
<lifeless> my current absolute favourite is testrepository
<lifeless> which is generally used via subunit.run
<lifeless> which backs onto testtools stuff
<mwhudson> lifeless: i'm currently writing up a python programming practices document for the $censored work
<mwhudson> so i get to inflict my opinions on how some things should be done
<lifeless> have I shown you testrepository and all the bits
<mwhudson> i'm aware of it
<lifeless> I think you should experience before recommended
<lifeless> s/ed/ing/
<mwhudson> yeah
<lifeless> so, apt-get install testrepository
<lifeless> this doesn't do test discovery yet, it will trivially when someone gets around to testtools + discovery [foords thing] glue
<lifeless> I should ask, do you have say, 10 minutes for a quick tour ?
<mwhudson> yes
<mwhudson> ah, i was going to ask about discovery
<lifeless> it would be good to add that
<lifeless> perhaps you could find a day and JFDI ?
<mwhudson> perhaps
<lifeless> let me see, how hard would it be
<lifeless> http://pypi.python.org/pypi/discover/0.3.2
 * mwhudson looks at unittest2 and finds nothing that looks particularly good, apart from the discovery stuff
<lifeless> yeah
<lifeless> I asked if he'd contribute to testtools
<lifeless> but apparently it was too hard. Or something.
<lifeless> <- a little bitter
<mwhudson> :(
<mwhudson> i think i told him to look at testtools at pycon 2009
<lifeless> anyhow
<lifeless> shiny shiny stuff
<lifeless> so, do you have something small with a test_suite method ?
<mwhudson> um
<mwhudson> no
<lifeless> if not, make something small with a test_suite method (because thats something that stock pyunit, and thus testtools.run, can support)
<lifeless> just
<lifeless> from testtools import TestCase
<lifeless> class foo(TestCase):
<lifeless>  def test_pass(self):pass
<lifeless>  def test_fail(self):self.fail('sad')
<bryceh> bzr question...  I have a branch with an html file in it.  I want whenever I commit to the branch that it updates a "last-updated-date" line in the footer.  Any way to set up bzr to do this easily?
<lifeless> def test_suite():
<jelmer> bryceh: see the bzr-keywords plugin
<lifeless>     return unittest.TestLoader().loadTestsFromNames('foo')
<lifeless> in foo.py
<bryceh> jelmer, thanks
<lifeless> mwhudson: we can test this works with 'python -m testtools.run foo.test_suite'
<lifeless> make another file, .testr.conf
<lifeless> [DEFAULT]
<lifeless> test_command=python -m subunit.run $IDLIST
<lifeless> test_id_list_default=foo.test_suite
<lifeless> EOF
<lifeless> then, testr init; testr run
<lifeless> should show you one failure
<lifeless> testr run --failing
<lifeless> should run the one failing test
<lifeless> tour over
<lifeless> for more detail, keep chatting, and/or read /usr/share/doc/testrepository/MANUAL.txt
<mwhudson> ok
<lifeless> I totally totally depend on the 'run only failing' stuff now that I have it
 * jelmer admits to also being a total testr fanboy
<mwhudson> hm yeah that looks pretty neat
<mwhudson> py.test had this --looponfailing thing that i remember using a bit, ages ago
<lifeless> sounds similar
<lifeless> this is not coupled to test runner or python
<lifeless> as long as there is an interface, and it can output subunit, it works ;)
<mwhudson> so where should discovery fit?
<mwhudson> as an alternative to test_id_list_default ?
<lifeless> so
<lifeless> small bit of glue in subunit.run
<lifeless> then you'd do
<lifeless> python -m subunit.run --discover [discovery arg]
<lifeless> when running it by hand
<lifeless> for .testr.conf you'd do
<lifeless> test_id_list_default=--discover <whatever you decided on>
<lifeless> thats the entire set of changes needed AFAICT
<mwhudson> ah, so subunit needs to change a bit?
<lifeless> tiny tiny
<lifeless> not the protocol, just the glue to setup a test list
<lifeless> rather than calling the unittest loader with the parameters its given
<lifeless> it needs to call the discover loader
<lifeless> thats it
<mwhudson> right
<mwhudson> ok well it doesn't sound too hard...
<lifeless> note that this workflow works with trial --reporter=subunit too
<lifeless> for huge value of entertaining
<mwhudson> right, and i guess trial already has its own approach to discovery
<lifeless> yes
<lifeless> I'd like bzr switch to stash the .testrepository with the branch
<lifeless> or perhaps I want a push/pop thing in testr
<mwhudson> lifeless: i guess i could find this out myself, but does trial have anything like the load_tests() protocol?
<lifeless> it honours test_suite
<lifeless> I would like to teach it load_tests
<lifeless> because load_tests is much better
<lifeless> mtaylor: ^ also another reason to use cppunit -or- make gtest talk subunit
<mwhudson> lifeless: why is load_tests 'much' better?
<mwhudson> i can see how it's a bit more convenient
<lifeless> test multiplication is a lot easier with it
<lifeless> less duplication
<lifeless> and no hard-coding of TestSuite class
<lifeless> and no hard coding of loader
<lifeless> the whole process becomes more easily mutated, 2-line patches to change things, rather than change-per-source-file
<mwhudson> ah yeah, that makes sense
<mars> lifeless, you wouldn't happen to have any idea why the zope testrunner + subunit would produce a partial traceback like this?: http://pastebin.ubuntu.com/443072/
<lifeless> mars: that looks truncated
<mars> lifeless, yes.  I thought a buffer in my own code was holding the output, but it turns out to be somewhere in the testrunner itself.
<mars> lifeless, is the subunit protocol written down anywhere?  I'm wondering if that "error: ..." line is correct?
<mars> lines that say "failure: " end with "[ multipart "
<mars> lifeless, found the protocol, EBNF in the README.  Clever.
<lifeless> :)
<lifeless> so you have the simple output there
<lifeless> [\n
<lifeless> there should be a trailing ]\n
<lifeless> and the output of a simple attachment like that is done at once, so it has nearly no chance of not completin
<lifeless> g
<lifeless> if its blowing up in stopTest
<lifeless> one possibility is stdout being closed or something
<lifeless> but that would fly in the face of actually *getting* that traceback
<mars> yeah :/
<mars> lifeless, could it be that the parent process (that watching the subunit stream) choked on some child process output?
<mars> silently ate it maybe?
<mars> thread dump in the parent says nothing out of the ordinary:
<mars> Thread 1
<mars> #0 0x00007f535ef0cdc2 in select () from None
<mars> #1 0x00007f535ec39784 in time_sleep (self=<value optimized out>, args=<value optimized out>) from /build/buildd/python2.5-2.5.2/Modules/timemodule.c
<mars> /var/launchpad/tmp/eggs/zope.testing-3.9.4-py2.5.egg/zope/testing/testrunner/runner.py (521): resume_tests
<mars> according to the code and thread dump it looks like the parent is running on the while-input loop
<lifeless> how long does this take to trigger ?
<lifeless> and is it reliable ?
<lifeless> if its not too long, you might try running the child under strace
<lifeless> as an ugly debugging hack
<mars> not reliable, maybe 40% failure rate, takes 2 hours to trigger with a full suite run.
<lifeless> also is it only on that test, or a general sort of thing?
<mars> it happens anywhere in the windmill portion of the suite
<lifeless> does it happen if you just run windmill?
<mars> that is why I really really wanted that full traceback :)
<mars> lifeless, not in my testing, no
<mars> and it doesn't happen on buildbot either
<mars> just on ec2
<lifeless> and you can trigger it locally running everything ?
<lifeless> or on ec2 w/windmill only ?
<mars> nope
<mars> only when running the full suite, including the windmill suite, on ec2
<lifeless> bugger
<lifeless> uhm
<lifeless> buffer limit ?
<mars> hmm
<lifeless> oh
<lifeless> you use wait() right
<lifeless> IIRC that can trigger on anything in your process group
<lifeless> want to bet that there is a child of a child which is being allowed to zombie until the child goes, and you're getting a signal on that child-of-child, which you then close the pipe on the immediate child and things go boom
<lifeless> or something
<mars> well, this is without using my code, but I can check the zope testrunner - it may .wait() as well
<lifeless> I'm not terribly convinced of my explanation there
<mars> lol
<mars> my first thought was "yay IPC is fun"
<mars> hmm
<mars> lifeless, looking at the zope testrunner code, what you say may be possible.  I would have to think through the process though
<mars> there is a note in there how reading the child process (the child layer) can be interrupted by EINTR if something sends SIGCHLD
<mars> but it would be really really odd that I have seen the traceback truncated on that stopTest line so many times
<lifeless> thats true
<lifeless> OTOH subunit writes in a very predictable pattern
<lifeless> how do I tell what version of bzr-loom launchpad has ?
<lifeless> mwhudson: thumper: ^
<mwhudson> lifeless: i guess look at utilities/sourcedeps.conf
<thumper> probably quite old
<ScottK> poolie: I was thinking about your comments about high dkim verification failure rates.  You might be better off doing dkim verification at the border too and then consuming an authresults header inside LP.
<poolie> hi scottk
<poolie> do you mean my speculation that many of them may have inadvertently broken headers?
<poolie> two more thoughts there:
<poolie> 1- it's much easier for me to land code into the bit of launchpad that processes incoming mail than it is to do that _and also_ get sysadmins to set up something in the border MTA
<poolie> 2- i'd like to hope that we don't actually damage the mail on the way from our DC border in to the incoming queue
<poolie> imbw
<poolie> but i thought i'd try verifying them there first and if it turns out 100% are broken, i guess we'll know
<mwhudson> given that we successfully verify gpg-signed mail, we can't damage it that badly (although i guess that doesn't sign headers?)
<henninge> jtv-eat: the query looks acutally quite simple, given that I already have the product id and the sourcepackagename(s).
<henninge> http://paste.ubuntu.com/443212/
* mthaddon changed the topic of #launchpad-dev to: Launchpad down/read-only from 11:00 - 14:00 UTC for a code update | Launchpad Development Channel | Week 4 of 10.05 | PQM is in release-critical mode | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
<deryck> Morning, all.
<ScottK> mwhudson: A dkim signature covers a lot more of the message than a gpg signature.  The one case I've managed to see real success data of a large enterprise trying to do dkim verification on a second tier MTA and not on the border it had a significantly lower verification reliability than cases where I've seen it done on the border MTA.
<mwhudson> ScottK: that makes sense
 * mwhudson goes to bed
<deryck> BjornT, ping
<BjornT> hi deryck
* mthaddon changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.05 | PQM is in release-critical mode | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
<wgrant>  /builders is timing out on both production and edge.
<noodles775> Yep... saw that too :/
<wgrant> How's it dying?
<noodles775> view/other_builders
<noodles775> wgrant: I'll paste the oops..
<wgrant> Hm. That's a bit odd.
<noodles775> wgrant: http://pastebin.ubuntu.com/443300/
<wgrant> Ah.
<leonardr> jamesh, i have a storm question
<jamesh> leonardr: shoot
<leonardr> is it possible to define a global hook equivalent to __storm_flushed__, or do i need to create a superclass/mixin and make everything inherit the behavior?
<jamesh> leonardr: there is an internal event for that, but it is not part of the public API
<jamesh> what would you want to use it for?
<leonardr> jamesh: see https://code.launchpad.net/~leonardr/launchpad/test-representation-cache/+merge/26513
<leonardr> i'm invalidating a memcached cache
<jamesh> leonardr: ah.  The event I was thinking of is just fired once in response to store.flush() -- not once per object
<leonardr> ah, ok
<leonardr> it sounds like a superclass/mixin is the best bet
<jamesh> leonardr: there are also internal "flushed" events for each object, but no way to globally register for all objects
<jamesh> leonardr: note that not all data modifying Storm APIs will result in Python objects being flushed.
<leonardr> jamesh: I'm talking to stub about that now. are you thinking of his "4) Using Storm for bulk updates, inserts or deletions."?
<jamesh> leonardr: right.  The ResultSet set() and remove() methods
<noodles775> wgrant: temporary patch - can you check pls: http://pastebin.ubuntu.com/443324/
<jamesh> there isn't a bulk insert method in Storm, but I doubt you care too much about inserts if you're working with a cache
<wgrant> noodles775: Looks fine.
<wgrant> Are we blaming missing indices for the slow query?
<noodles775> I'm assuming so, but haven't checked yet... want to get the page back up so we can fix it properly in time.
<wgrant> Yeah.
<stub> jamesh: store.execute(Update({Person.displayname: 'foo'}, Person.name=='stub')) would be how to get Storm to do updates without clearing the cache (not that we do that, but it would be nice to be able to do that).
<jamesh> stub: you don't even really need to go behind Storm's back like that
<stub> I thought that would be the blessed approach for bulk updates; didn't think that *was* going behind Storm's back ;)
<jamesh> stub: if I use ResultSet.set() to do a bulk update, you'll only see __storm_flushed__() calls for the objects that are actually live in memory
<stub> Ahh... that would be a better approach for bulk updates ;)
<jamesh> it is really only intended to help you keep local caches up to date (e.g. storing computed values on the Python object) -- not external ones like I assume you're talking about
<stub> So there is a check to see if the object is live. We could hook in there to clear the cache at that point... although there are still problems we can only solve at the database level. Not sure if they are problems we really need to worry about at this stage.
<stub> Oh... no we couldn't since we won't know the keys :-(
<deryck> Does sinzui have a mark-released button feature again?  A script?  Or too much time on his hands? :-)
<sinzui> deryck, I used my script from *before* the feature that I had to retracted. I discovered we failed to update a lot of bugs on 10.04, 02
<bac> reviewers meeting starting now
<gmb> deryck, bug 386757 has been fixed for malone, hasn't it?
<mup> Bug #386757: JavaScript for subscribing should be refactored for reuse <Launchpad Bazaar Integration:Fix Released by rockstar> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/386757>
<gmb> Ah, or am I getting confused about branches that have landed but don't actually fix the bug?
<deryck> gmb, not really fixed completely.
<gmb> deryck, Ah, okay.
<deryck> gmb, I did some work, rockstar did some work, and once we have state on the bug subscriptions, then we can do more work to unify our approaches.
<deryck> though I think rockstar went really different from my work
<gmb> deryck, Okay, I'll add it to the subs story.
<gmb> And we can take a look at it later.
<deryck> gmb, yeah, we should look again late in the UI refactor for this story.  It will most certainly have to touch the js subscribing code.
<gmb> Right.
<rockstar> deryck, I think my work was the culmination of discussions you and I had at the lazr-js sprint/UDS
<rockstar> deryck, also, I have a branch that I've been trying to get back to that does the wizard widget, so we should have that at some point as well.
<deryck> rockstar, right, I knew we talked about it.  I just meant it's more different than alike to what we have now in bugs.  but better, definitely.
<rockstar> deryck, ah yes, because branch subscriptions have state, and you had planned to add the same to bugs.
<deryck> rockstar, exactly.
<deryck> So I think it will line up nicely with our subscriptions refactor the next couple months.
<rockstar> deryck, sweet.
<henninge> sinzui: I have a question about packaging if you have 2 minutes ;)
<sinzui> I can help
<henninge> sinzui: Is it correct that we are now copying packaging links whenever a new distroseries is created?
<sinzui> Yes. they were copied.
<henninge> sinzui: for maverick or for all series?
<sinzui> for maverick using the script that bootstraps soyuz
<sinzui> When we copy the packages from lucid to maverick and started the builds, we also copied the packaging links
<henninge> sinzui: and that will happen on each new series?
<henninge> in the future?
<sinzui> yes.
<henninge> sinzui: thanks, great help. ;)
<sinzui> It is only for Ubuntu because only Ubuntu uses soyuz
<henninge> sinzui: well, it looks like only Ubuntu supports packaging anyway, at least UI-wise.
<henninge> "+ubuntupkg" ...
<sinzui> correct
<rockstar> henninge, yes, if you try and build for anything but Ubuntu, you can really bugger the build farm. abentley knows from experience.
<henninge> rockstar: cool, that makes things easier for me, too.
<abentley> noodles, why did you turn buildbase.queuebuild into a staticmethod?
<thumper> jelmer: http://launchpadlibrarian.net/49567000/chicken-chicken-git-mirror.log
 * maxb ponders creating a dev.launchpad.net/FailingBzrSvnImports to gather common failure cases, and wonders if some such thing exists already
<maxb> Also, what is the current situation with svn imports requiring a username password? Does that still require losa intervention on the import slaves?
<mars> gary_poster, ping, buildout question, when you have some spare time
<gary_poster> mars, what's up?
<mars> gary_poster, being annoyed by buildout picking the first zope.interface it sees, then barfing when it reads versions.cfg and realizes that it needs an older zope.interface
<mars> gary_poster, just a sec, ran it through -vvvv, and it says there is a version conflict when loading zc.recipe.testrunner
<mars> so it could be that zc.recipe.testrunner needs a newer zope.interface
<gary_poster> ok
<gary_poster> are you specifying the version of zc.recipe.testrunner?
<mars> gary_poster, yep.  According to zc.recipe.testrunner, it does not need an explicit zope.interface
<gary_poster> mars, sounds odd.  You have a branch for me to look at?
<mars> gary_poster, yes, I'll push the changes
<gary_poster> ok
<mars> gary_poster, bzr branch lp:~mars/lazr-js/1.0
<gary_poster> k
<mars> gary_poster, also worth noting: the Makefile uses a LAZR_SOURCEDEPS_DIR variable, and mine points to ~/.buildout/
<gary_poster> k
<mars> gary_poster, zope.interface-3.5.1 is in the global download-cache, but it doesn't want to use it.  The global eggs/ directory only has zope.interface-3.5.3 (maybe because buildout refuses to build the 3.5.1 egg)
<gary_poster> mars, the buildout.cfg file is trying to do something that appears to be against your goals.  ...wanna talk on mumble?
<mars> gary_poster, sure
<gary_poster> k, I'm there :-)
<lifeless> mars: there was a import failures wiki page some time back
<mars> maxb, ^
<lifeless> bah
<lifeless> sorry mars
<mars> np :)
<mars> lifeless, gary_poster, by the way, by hacking our testrunner I managed to get it to spit out the full windmill suite hang traceback: http://pastebin.ubuntu.com/443544/
<lifeless> grah
<mars> ?
<mars> oh
<gary_poster> "Look," Gary said intelligently, "a ValueError!"  Though it does look like something in subunit on the face of it.
<mars> lifeless, ?
<lifeless> checking the code
<lifeless> so, error is not None - this is a plain pyunit api call
<lifeless> rather than a lovely shiny testtools one
<lifeless> stopTest was called
<lifeless> and then the zope testrunner formatter is calling addError
<lifeless> from within stopTest
<lifeless> thats odd
<lifeless> the error object is failing in _exc_info_to_string
<lifeless> so err is not a exc_info tuple
<lifeless> its something else
<lifeless> look at zope/testing/testrunner/formatter.py
<lifeless> whats the easiest way to get that for me? I have a 2 month old checkout of lp
<lifeless> mars: ^
<mars> lifeless, looking
<mars> lifeless, check in eggs/zope.testing-3.9.4
<mars> do you have that version?
<lifeless> sec, starting the vm
<lifeless> yes, I do
<lifeless> well, a py2.5.egg
<lifeless> should be good enough
<mars> that's the one
<lifeless> ok so this is subunut glue
<lifeless> and its calling _get_text_details
<lifeless> yes, its buggy
<lifeless> I'll put a patch up
<mars> lifeless, wow, thanks!
<lifeless> ok, grab lp:~lifeless/zope.testing/subunit
<lifeless> I don't know if passes tests, because it blows up when I follow the getting started instructions
<mars> ok
<mars> lifeless, it will take a while to run a new round of tests.  A few hours :(
<mars> it will probably have to wait for tomorrow.
<lifeless> https://code.edge.launchpad.net/~lifeless/zope.testing/subunit/+merge/26638
<lifeless> mars: so anyhow, its something blowing up in another thread, but the error reporting codepath was broken
<mars> ok
 * mars waits for the diff to update
<lifeless> so we'll still expect an error
<lifeless> but it should be relevant to the windmill tests this time
<mars> \o/
<mars> that would be a big step forward
<mars> Hurray for cascading failures!
<lifeless> :)
<lifeless> actualy, hurray for untested code :P
<mars> lifeless, many thanks.  I'll let you know how it goes.
<lifeless> please do
<lifeless> sidnei: when you get back
<lifeless> https://code.edge.launchpad.net/~lifeless/zope.testing/subunit/+merge/26638
<mars> by coincidence, this may have been the source?  Exception in thread Thread-3 (most likely raised during interpreter shutdown):
<mars> Traceback (most recent call last):
<mars>   File "/usr/lib/python2.5/threading.py", line 486, in __bootstrap_inner
<mars>   File "/usr/lib/python2.5/threading.py", line 446, in run
<mars>   File "/var/launchpad/tmp/eggs/windmill-1.3beta3_lp_r1440-py2.5.egg/windmill/server/https.py", line 398, in start
<mars>   File "/usr/lib/python2.5/SocketServer.py", line 218, in handle_request
<mars> <type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'error'
<lifeless> very likely
<lifeless> the check was a threading thingy
<mars> so my 'reprint the exception' code in bin/test actually spat out the entire error
<mars> who knows why it was getting truncated
<mars> wait, spoke too soon.  There is nothing in the on-disk capture file, just the console.  May have just been a luck break.
<maxb> Is there a good way to hold a conversation with the requester of a vcs import?
<lifeless> 'contact this person' in launchpad
<maxb> ... and manually keep the whiteboard up to date with the state :-/
<lifeless> yes :(
<maxb> In a most perplexing move, someone's trying to register a ~lrcshow-x/lrcshow-x/trunk vcs-import when ~vcs-imports/lrcshow-x/trunk already exists
<mwhudson> maxb: hey, at least you can see who registered the import now
<mwhudson> that was one of the more frustrating problems with the ye olde system
<maxb> But given the history of their svn repository, they clearly don't understand version control very well
<lifeless> mwhudson: so what were your testing framework thoughts after looking at stuff yesterday
<mwhudson> lifeless: i admit to getting a little sidetracked
<mwhudson> lifeless: i think discover + subunit + testr run looks pretty nice though
<maxb> Could someone look up OOPS-1614M3672 for me? (occurred attempting to rename a vcs-import branch)
<lifeless> sec
<lifeless>   Module lp.code.model.branchnamespace, line 135, in validateRegistrant
<lifeless>     % (registrant.displayname, owner.displayname))
<lifeless> BranchCreatorNotMemberOfOwnerTeam: Max Bowsher is not a member of lrcShow-X team
<lifeless> zomg 500ms of sql time there
<thumper> maxb: arse
<thumper> maxb: was that over the api?
<maxb> No, web ui
<thumper> maxb: what did the form look like for the owner?
<thumper> maxb: you should have had a dropdown
<thumper> maxb: if you didn't, it is a bug
<maxb> oh, I see
<thumper> maxb: BTW I think you are doing an awesome job with the imports
<maxb> I have edit perms by virtue of being in ~vcs-imports, but that means I can edit a branch of which I'm not in the owner team
<maxb> So the vcs-imports special edit hack has been incompletely applied
<thumper> maxb: you should be able to edit the branch yes
<thumper> maxb: however you can only assign to a team you are a member of
<thumper> maxb: the owner widget should be a drop down for you
<maxb> I am not trying to change the branch owner. It is a dropdown. It's objecting about the fact that the existing owner of the branch is not one that I would be permitted to assign the branch to, despite the fact that I can edit the branch
<maxb> i.e. it's over-aggressive validation. I suspect it's never been an issue before because ~vcs-imports have often been ~bazaar-experts previously
<thumper> oh
<thumper> arse
<thumper> this is exactly the same problem as with the source package branches
<thumper> maxb: can you file a bug please
<maxb> Sure
<thumper> thanks
<maxb> lifeless: may I have some more of the traceback?
<mwhudson> thumper: an oops is a better result than maxb accidentally assigning the branch to himself i guess :-)
<sidnei> lifeless, i'm done for the day, but will look into it tomorrow
<lifeless> sidnei: tresquis is on it
<lifeless> sidnei: but thanks
<maxb> 588943 ftr
<sidnei> lifeless, even better. thanks!
<wgrant> Hmm.
<wgrant> It looks like if you have a 1.0 API override (because the devel API has changed, and we need to retain compat), beta (which came before 1.0) doesn't inherit it.
#launchpad-dev 2010-06-03
<poolie> spm, when you have a spare few minutes could you look over https://dev.launchpad.net/LEP/DKIMAuthenticatedMail from a LOSA's point of view?
<spm> poolie: tbh, I don't know a lot about dkim. but the reasonings for, sound fine to me. it's not perfect (what is...), but afaict from the cases there, will assist a large chunk of users. +Â½, given my ignorance on the full topic.
<poolie> thanks
<Ursinha-afk> is it just me or it isn't possible to login to staging right now?
<lifeless> wgrant: do you mean 'beta is breaking'?
<spm> Ursinha-afk: it's not just you. I have the same problem. I believe it's known.
<Ursinha-afk> oh...
<Ursinha-afk> spm, you're not sure it's known? I'm considering filing a bug for that
<Ursinha-afk> what do you think?
<spm> Ursinha-afk: :-) file it anyway. worst case LP QA will mark it as a dupe. :-P
<spm> tbh, I thought it was just me. but if you're having issues too, perhaps not.
<Ursinha-afk> spm, just found out trying to validate a token to use lpapi
<spm> nod
<Ursinha-afk> and it explodes as a django error, not an oops
<ScottK> poolie: In answer to yesterday's question: A dkim signature covers a lot more of the message than a gpg signature.  The one case I've managed to see real success data of a large enterprise trying to do dkim verification on a second tier MTA and not on the border it had a significantly lower verification reliability than cases where I've seen it done on the border MTA.
<ScottK> So yes, I think there's an unintended transformation in there, but experience tells me they are hard to avoid entirely.
<poolie> ok
<poolie> that's useful data
<poolie> i saw a post the other day about exchange breaking signatures
<poolie> and my first attempt at a test broke because when the message is parsed and unparsed the headers come back re-flowed
<poolie> ScottK, i'm kind of hoping we have less of the 'enterprise' software that mangles messages
<poolie> imbw
<ScottK> poolie: I think it's more to do with verify at the border or internally.  The other issue with internal verification is that not only do you not have to break the signatures now, you have to make sure future changes don't either.  Not an issue if you verify at the border.
<ScottK> poolie: Did you add the test messages to the python-dkim bug?  I don't recall seeing the bugmail?
<poolie> ScottK, sorry not yet
<poolie> haven't worked on it again yet
<poolie> it's just itch-scratching/curiousity, not actually my job
<ScottK> OK.  As long as it's not forgotten, not a big deal, you just won't get progress on the bug until I get it.
<ScottK> OK.
 * ScottK looks around to see what's his job.
<poolie> k
<lifeless> \o/ timeouts
<lifeless> spm: can you sync edge oops please?
<lifeless> OOPS-1615EC395
<spm> done
<spm> and not found. wonderful.
<lifeless> \o/
<lifeless> is it me?
 * spm is tempted to say yes, but that'd be the evil talking
<lifeless> https://bugs.edge.launchpad.net/oops-tools/+bug/589007
<mup> Bug #589007: oops OOPS-1615EC395 not found by lp-oops web ui <OOPS Tools:New> <https://launchpad.net/bugs/589007>
<mwhudson> it's there now
<lifeless> grr
<lifeless> spm rsunc it
<lifeless> and it didna work
<lifeless> anyhow
<lifeless> 10410360ms
<lifeless> thats 104  10360ms
<lifeless> SELECT COUNT (CASE WHEN BugTask.status = 10 THEN BugTask.id ELSE NULL END), COUNT (CASE WHEN BugTask.status = 15 THEN BugTask.id ELSE NULL END), COUNT (CASE WHEN BugTask.status = 20 THEN BugTask.id ELSE NULL END), COUNT (CASE WHEN BugTask.status = 21 THEN BugTask.id ELSE NULL END), COUNT (CASE WHEN BugTask.status = 22 THEN BugTask.id ELSE NULL END), COUNT (CASE WHEN BugTask.status = 25 THEN BugTask.id ELSE NULL END) FROM BugTask
<mwhudson> lotsa ubuntu bugs to count up
<thumper> stub: hi
<stub> yo
<thumper> stub: review poke for abentley's branch
<lifeless> another timeout
<lifeless> stub: hows the db looking; getting timeouts on edge on fairly routine things
<lifeless> like filing a bug on the launchpad project, in the 'related bugs' search
<mwhudson> oh
<stub> Nothing traumatic. Load of under 10, log lag. Fair bit or replication going on.
<mwhudson> well
<stub> c/log lag/low lag
<stub> Now, on the other hand, we are having load issues...
<stub> fiera, karma calculater, garbo-daily, lucille all running at the same time.
<lifeless> oops
<stub> So OOPS-1615EA901 (trying to view a merge proposal) caused by load I think
<lifeless> SELECT %s FROM (SELECT BranchRevision.branch, BranchRevision.id, BranchRevision.revision, BranchRevision.sequence FROM BranchRevision WHERE BranchRevision.branch = 352323 AND BranchRevision.sequence IS NOT NULL AND BranchRevision.revision NOT IN ( SELECT revision FROM BranchRevision WHERE branch = 34755) ORDER BY BranchRevision.sequence DESC LIMIT 1) AS "_tmp" LIMIT 1
<lifeless> thats what took 17 seconds
<lifeless> I'd guess lock contention rather than sheer load
 * lifeless handwaves
<adeuring> good morning
<krkhan> can anyone point me to the method for manually calculating the ssha hashes used for accounts?
<jelmer_> when is PQM going to be opened again?
<krkhan> openldap's slappasswd generates ssha hash but it still doesn't login
<thumper> jelmer_: did you see the chicken git import failure log?
<jelmer_> thumper: hi
<lifeless> krkhan: sha hashes? where
<jelmer_> thumper: looking
<thumper> jelmer_: thanks
<krkhan> lifeless: i'm running a local launchpad. i can't login for the default admin@canonical.com account for some reason (invalid email/password). so i thought maybe i changed the passwd and forgot
<krkhan> i guess manually editing the accountpassword table would make things work. but i don't know how to correctly calculate the ssha hash used
<lifeless> AFAIK launchpad no longer stores any passwords
<thumper> krkhan: the local password should be 'test'
<thumper> jelmer_: pqm opens when the release manager is happy :)
<krkhan> thumper: i used to login with 'test' before. it's not logging in with that any longer
<thumper> krkhan: what did you change?
<jelmer_> thumper: I can reproduce it locally, but no idea what it's caused by
<krkhan> thumper: not the password at least. let me recheck using a previous snapshot of the vm
<thumper> jelmer_: some fun debugging then :)
<jelmer_> thumper: can you file a bug against bzr-git ?
<thumper> sure, but probably tomorrow :)
<krkhan> thumper: it refuses to login with admin@canonical.com and test. and i haven't changed anything
<thumper> krkhan: I'd run 'make clean schema' and try again
<noodles775> krkhan: ^^^ assuming you don't care about any data in the launchpad database.
<krkhan> thumper, noodles775: i don't. anything that works will be fine. trying make clean schema
<lifeless> thumper: how does local auth work; didn't SSO split it all out ?
 * thumper shrugs
<thumper> I don't know how it wors
<thumper> works
<bigjools> o/
<thumper> yes bigjools?
<bigjools> just saying hi
<thumper> oh hi
<krkhan> thumper: worked perfectly. thanks :-)
<thumper> krkhan: I have no idea why though
<thumper> krkhan: make clean schema is my reset button
<maxb> ooi, when does the db-devel>devel mergeback happen?
<bigjools> maxb: when the release manager decides it can happen, usually when we open PQM after a re-roll, if that happens at all.
<wgrant> lifeless: The data is still in the DB -- and LP still has a test OpenID provider which uses that data.
<wgrant> Eventually Account will probably die, and testopenid will just have to ask for an email address and no password.
<deryck> gmb or adeuring , can one of you help me interpret bad test documentation please?
<adeuring> deryck: I can try ;)
<deryck> adeuring, see http://pastebin.ubuntu.com/443974/
<deryck> adeuring, what "marker" does this test refer to?
<adeuring> deryck: I think this means the text "This bug is a duplicate of..." (just concludning from the test output)
<deryck> adeuring, ah, ok.
<deryck> adeuring, I was thinking so special email foo something or other.  That makes sense.
<deryck> printing a ga-jillion emails out in doctest, however, does not. :-)
<adeuring> ;)
<deryck> adeuring, thanks
<svaksha> nigelb: hi
<svaksha> bryceh: i was told thatat you have developed a crawler which allows data from LP to be pushed upstream in to a bug tracker. I was looking for a webcrawler which allows data to be entered into forms (ala the kind of crawlers blog spammers use to leave comments)
<svaksha> did i miss a reply to my query, /me was disconnected
<jml> maxb, thanks for the great bug report
<deryck> svaksha, bryceh doesn't really have a crawler.  He has a cgi script that gets bug data from LP and creates an upstream bug report from the data.
<svaksha> deryck: thanks. that is probably not what i am looking for
<deryck> svaksha, np.  cheers.
<svaksha> deryck: i'm trying to test MM lists for errors thatcrop up after some new featureswere implemented. see, http://abiwt.org/mailman/listinfo/dlist-583379
<deryck> svaksha, ok, interesting.  How does this relate to Launchpad data?
<svaksha> sorry, that is why i didnt mention it earlier
<svaksha> umm...i didnt know what bryceh's crawler does. i was told about it so asked to see if it would be useful
<deryck> Does any still find make lint output useful?
<bigjools> deryck: only if I intend to fix it :)
<bigjools> I use a pyflakes vim plugin now, so I tend to pick most lint up immediately.
<deryck> bigjools, I used to find make lint useful when I started on lp, but it has become increasingly noisy and always pointing at stuff unrelated to what I'm working on...
<deryck> which makes me wonder if anyone actually uses it.
<bigjools> deryck: it used to be rigorously enforced
<bigjools> doesn't seem like it is any more
<deryck> bigjools, by buildbot?  Or just in review?
<bigjools> deryck: reviews
<bigjools> I'd mention it in the next reviewers' meeting
<deryck> yeah, I think I will.
<deryck> bac, I can update the wiki, but I'd like to bring up the above ^^ about make lint in the next reviewers meeting.
<bac> deryck: i find make lint useful, if noisy
<deryck> bac, are you ok with me bringing this up in the reviewers meeting, i.e. to ask if we're enforcing lint checks at review and asking if the output of make lint could be made better?
<bac> deryck: for the people that use our supported tools to create MPs, the lint report is always included and is handy for the reviewer.  increasingly i see developers not using 'bzr send' and writing subpar MPs that don't include lint output nor many of the needed sections.
<bac> deryck: definitely
<deryck> bac, yeah, I'm using bzr lp-propose-merge now, which doesn't include lint output like the bzr send plugin.
<deryck> make lint gone bad:  http://pastebin.ubuntu.com/444066/  :-)
<mars> deryck, IIRC there was a discussion on the dev list recently where sinzui talked about his improved lint scripts.  They would be nice to land, but his release duties are blocking that.
<deryck> mars, yeah, that would be cool to land.
<sinzui> My scripts are now gedit plugins. Some work is needed to make them scripts again
<deryck> maybe I should just get bigjools pyflakes plugin for vim.
<deryck> I tend to watch carefully myself and just like the post-work lint check.
<bigjools> the plugin is fabulous
<bigjools> it's on vim.org
<mars> deryck, yes, you should use that plugin.  I don't use vim anymore, but I still give it a huge +1 :)
<mars> I would love to recreate it for Komodo
<deryck> ok, I'll grab it here shortly then.
<deryck> I still think it's worth having a sane make lint, though.  Just for easy run against a branch we're reviewing.
<jelmer> emphasis on "sane" :-)
<gary_poster> deryck[lunch]: Are hwdb questions like this appropriate for the bugs team:  https://answers.edge.launchpad.net/launchpad/+question/113350 ?  If not, any ideas on where I should send it?
<deryck> gary_poster, yup, completely fine for malone.
<gary_poster> thanks deryck
<bryceh> whenever I try to load https://dev.launchpad.net in my browser, it redirects me to the last page on that site I viewed, instead of taking me to the homepage.  Anyone else seeing this behavior?
<jml> bryceh, no
<lifeless> sinzui: ping
<sinzui> hi lifeless
<lifeless> sinzui: bugs are too low bandwidth, I could feel myself getting into itty-bitty mode
<lifeless> the oops thing
<lifeless> I'd like to know why you feel/think that its not a coding issue
<lifeless> I mean, I can see that we have data to fix *too*, but surely the code should have behaved better given the data we gave it.
<sinzui> about Lp identifying reserved names from the db and reserved names in the code and verifying that the object can have the name and that the user can access the object with the name
<sinzui> That is a hard problem
<lifeless> yeah, and filtering from searches appropriately; blacklisting isn't any harder than privacy, is it ?
<lifeless> (he says knowing that privacy isn't all that easy :P)
<sinzui> from the db perspective, no, because it is acl/privacy and we do handle it.
<sinzui> from the code, we need some way to register the names in traversal and their type to verify they can be used.
<sinzui> I think the policy was to ensure the names in the code are added to the db
<sinzui> I am pretty sure that did not happen with 'oops' statik had no problems using his project last year. I do not know when we changed the code the break his project
<lifeless> so there is a bit of code that lists <names>
<lifeless> and a db table that also lists <names> ?
<sinzui> yes.
<lifeless> I guess what I'm saying is this: how do we fix this so that it doesn't happen again; deleting the project only fixes this instance, not the cause of the confusion behaviour and spurious oops
<sinzui> Yes
<lifeless> I was assuming we'd need to enforce a blacklist somewhere we weren't, but if I understand you correctly, there's actually 2 lists that are meant to be identical, and aren't.
<sinzui> I do not know when or what in the app claims 'oops'
<lifeless> there is a ++oops++ handler
<lifeless> which triggers an explicit soft oops
<lifeless> could that be it ?
<lifeless> If so, perhaps the oops project *isn't meant* to be blacklisted
<lifeless> its just a victim of thumper's work to let devs trigger oops on demand, to see where db time is going.
<sinzui> possibly
<lifeless> how do we check the db list of blacklisted names ?
<lifeless> if oops isn't in there, and isn't in the app list of blacklisted names, then its not really a blacklist issue
<lifeless> its a webapp-overusing-a-name issue
<sinzui> that is an open bug. we do a sql query. We want a page to view and edit them.
<lifeless> ok
<lifeless> losa ping
<mbarnett> hello lifeless
<lifeless> we're like to get the current list of blacklisted projects from the db please
<lifeless> s/we're/we'd/
<sinzui> Our fields block admins from renaming a project to a blacklisted name when we know we must do it. They do a sql update to make it happen
<lifeless> mbarnett: I don't know the right query to run, if its not in the LOSA pages, sinzui can supply it
<sinzui> lifeless, I see the data on staging
<lifeless> ok cool
<lifeless> mbarnett: sorry, you're not needed :)
<lifeless> sinzui: so, is oops in the list of blacklisted projects?
 * sinzui still looking for the table
<lifeless> :) ok
<rockstar> sinzui, do you know how I create a browser for a person that isn't logged in.
<sinzui> anon_browser?
<lifeless> is that different to logged in anonymously?
<sinzui> you can just import it from testing.pages I think
 * sinzui reads code to find the table
 * sinzui is an idiot
<sinzui> lifeless, oops is not in the table
<lifeless> ok
<lifeless> and its not defined as a blacklist in the app either, its just the ++oops++ trigger?
<rockstar> sinzui, I'm not using a pagetest, but a unit test, so it's not available in some evil globals thing.
<thumper> rockstar: just call setupBrowser()
<thumper> rockstar: with no auth, it is anon
 * mbarnett is both glad and sad he is not needed.
<sinzui> lifeless,  I lost power just as I was typing that oops is not in the table.
<sinzui> rockstar, import setupBrowser() from canonical.launchpad.testing.pages. calling it without arguments return anon_browser
<lifeless> 08:53 < lifeless> ok
<lifeless> 08:54 < lifeless> and its not defined as a blacklist in the app either, its just the ++oops++ trigger?
<rockstar> sinzui, got it, thanks.
<sinzui> lifeless, I think so
<lifeless> in which case
<lifeless> I think a better phrasing of the bug is
<lifeless> 'non blacklisted webapp hooks subtly break search'
<lifeless> and then we have two related questions
<lifeless>  - can we enumerate these hooks (to make an automated cross-check that the db blacklists things that users cant use anyway)
<sinzui> But I cannot access "oops" from statik's pages, or from the list commercial project either
<lifeless>  - could we change the oops one so that it only triggers as ++oops++ and doesn't cause this broken traversal
<lifeless> sinzui: right, somethings clearly going wrong here, but I think is unintentional fallout from the ++oops++ support
<sinzui> yes, and we can certainly make the same mistake again
<lifeless> right
<lifeless> so I'd like to keep one of the bugs open saying 'we should have some code that helps prevent this sort of mistake'
<sinzui> yes, I agree that is the bug
<lifeless> and if possible, another saying '++oops++ does not need to break the /oops/ project - but perhaps that is hard :- I don't know as I haven't looked into it.
<sinzui> It may be knowable if Navigation always uses @stepto and @step, but I am certain there are exceptions
<thumper> sinzui: is it really conflicting?
<thumper> I'd be surprised
<lifeless> thumper: visit launchpad.net
<lifeless> thumper: search for oops
<lifeless> click on statik's project name there
<lifeless> you get a 404 with an oops generated
<lifeless> thumper: oops isn't blacklisted in the db, nor in the code from what sinzui could see
<thumper> that's the whole reason the namespaces have ++ on each end
<thumper> I'm not sure why this is happening
<lifeless> indeed
<lifeless> I'm going to redraft the bugs to be clearer
<mwhudson> lifeless: the oops generation is not mysterious
<lifeless> about the generic and specific issues, assuming they are separate
<thumper> I'd ask gary_poster, him being a zope master and all :)
<mwhudson> lifeless: it's because the referrer is a launchpad url
<lifeless> mwhudson: oh right
<mwhudson> so we can stop talking about that
<lifeless> mwhudson: thanks
<lifeless> mwhudson: FWIW I only mentioned it here to address thumpers 'is it really conflicting' question
<sinzui> but we are getting about zope person right? there are always two, but which is the master
<mwhudson> i thought i said this last night, but maybe my wifi falling over
<lifeless> mwhudson: which a simple 404 isn't very convincing about
<mwhudson> ok
<lifeless> mwhudson: yes, you did, and I know.
<mwhudson> sorry for the noise then :)
 * gary_poster sees name but nothing to which to respond
<lifeless> mwhudson: but I was tired last night :)
<lifeless> gary_poster: there is a oops hook in the webapp
<gary_poster> for generating oopses you mean?
<lifeless> which handles ++oops++ - e.g. lp.net/launchpad/++oops++
<lifeless> generates a soft oops
<lifeless> yes
<lifeless> however
<lifeless> it seems that lp.net/oops which is a real project, not blacklisted, is denied access and we think its due to the oops handler existing.
<gary_poster> oh!  I suppose "it shouldn't be" is a bit of an obvious reply :-)
<gary_poster> perhaps, "why do you think so?" would be better
<lifeless> well
<gary_poster> that would imply to me that we should be able to dupe on a dev system
<gary_poster> lemme try that
<lifeless> we know of the following mechanisms to make 404s
<lifeless> blacklists
<lifeless> deleting things
<lifeless> and bugs
<lifeless> its not blacklisted
<lifeless> its not deleted
<lifeless> so must be a bug :)
<gary_poster> :-)
<lifeless> I haven't setup my dev VM to let me connect to it from the outside host yet, I've just been doing in-place unit testing :( I really should get around to that.
<lifeless> gary_poster: does it happen for you locally?
<lifeless> we're getting way to many Searching for your bug in Launchpad took too long. Try reducing the number of words in the summary field and click "Check again" to retry your search. Alternatively, you can enter the details of your bug below.
<gary_poster> lifeless: I decided to update post-PQM so...things are just ready to start up now post make schema.  Will report back.
<gary_poster> post PQM opening I mean
<lifeless> I gotcha
<gary_poster> lifeless: it is reproducible locally
<lifeless> I've updated the bug report
<lifeless> sinzui: thanks for teasing this apart with me
<sinzui> your welcome
<lifeless> gary_poster: so, do you think its the ++oops++ thing, or something different?
<gary_poster> lifeless, not sure.  Should be a different code path, of course, but obviously something is wrong, and it's the same string, so....  since we've talked about it, I'm doing a pdb to look around.
<lifeless> cool! thanks
<gary_poster> yes, most definitely this has something to do with the namespace (Zope considers ++*++ to be a URI namespace mechanism).  The failure is specifically that it can't fine an "index.html" for the OopsNamespace object.
<gary_poster> find
<lifeless> gary_poster: but lp.dev/oops shouldn't be going into the ++oops++ code, should it ?
<gary_poster> nope
<gary_poster> possibly a registration error.
<gary_poster> oddly, editing one branch and running a branch doesn't accomplish much.
<gary_poster> running a different
<gary_poster> lifeless: so, the ++oops++ url is registered in an odd way.  It's the problem.  I'll fix it up tomorrow.
<gary_poster> night all
<bdmurray> Is there some way to run 2 specific tests with bin/test?  something like bin/test -cvvt bugtask-search.txt patches-view.txt ?
<lifeless> yes
<lifeless> if you've got a failure back from ec2, you can just unzip the subunit log to .testrepository/failing and do testr run --failing
<bdmurray> I'm working on this branch and I've modified two tests and just want to run those 2 tests ...
<bdmurray> I haven't done anything with ec2 yet
<lifeless> try bugtask-search without the .txt
<lifeless> if you know the testid you can use --load-list and a file listing ids, one-per line
<bdmurray> lifeless: the testid?
<lifeless> yes, all tests have a unique test id
<lifeless> I don't know what it is for doctests
<wgrant> bdmurray: bin/test -cvvt bugtask-search.txt -t patches-view.txt
<wgrant> You need the second -t.
<lifeless> wgrant: thanks, I'll try to remember that :)
<wgrant> (-t's argument is actually a regex, but one normally just uses test names or fragments)
<bdmurray> wgrant: great, I couldn't think of a regex that would hit both of those tests.
#launchpad-dev 2010-06-04
<lifeless> bdmurray: 'bugtask-search
<lifeless> bdmurray: 'bugtask-search|patches-view'
<lifeless> mmm, possible with brackets, depends on the engine
<lifeless> it might be nice to give community devs - those that have reached reviewer level, for instance, access to the OOPS summaries
<lifeless> also
<lifeless> wow, 1135 second page request. ouch.
<mars> lifeless, it looks like your patch worked.  The generic everyday "Threads left garbage" message was raising an error, which locked up the entire process stack in os.wait() calls.
<mars> and that is why it was random - some thread somewhere wasn't shut down before the testrunner proceeded.  Maybe a race condition.
<mwhudson> mars: yay for progress on that one
<mars> mwhudson, yes, I'm happy we finally found the source.  Now we just have to land the fixes for each of the links in the chain of cascading failures.
<mwhudson> the boring bit :-)
<mars> hehe
<mwhudson> mars: do you know what's going on with ec2 thinking failing tests runs are successes?
<mars> that is a mystery to me.
<mars> I couldn't reproduce it.  But thanks to this fix I now know where to look in the zope code for errors.
<cody-somerville> lifeless, That rpm.newrelic.com website crashes my browser :P
<lifeless> oops :P
<lifeless> mars: glad to have helped
<wgrant> lifeless: Why is bug #516709 Soyuz? Isn't it Code? All of the changes seem to be to the branch, not upload rights.
<mup> Bug #516709: revisit official package branch permissions <Soyuz:New> <Ubuntu Distributed Development:New> <https://launchpad.net/bugs/516709>
<bilalakhtar> Hi there lp devs, do I need to run lp on my own computer before submitting a patch? I know it would be better to do that, but isn't lp too bulky to download?
<beuno> bilalakhtar, ideally you would, you cacn try submitting the merge proposal and seeing if you can get a developer to run the tests for you
<bilalakhtar> beuno: oh ok thanks
<beuno> bilalakhtar, what is this patch about?
<bilalakhtar> beuno: I haven't begun work yet, but I want to add the following feature in lp answers: One should be able to assign someone to a question or change its status using an AJAX overlay.
<bilalakhtar> beuno: such a feature already exists in malone
<beuno> ah
<beuno> so, just to give you a tip, you may need to build the API for that, because those are old parts of the code and probably don't have APIs to leverage javascript
<beuno> which means you will need to run LP, because it's a significant chunk of work  :)
<bilalakhtar> ahha
<bilalakhtar> thanksf for the info, beuno
<bilalakhtar> beuno: What do you mean by "API" ?
<bilalakhtar> beuno: launchpadlib?
<beuno> a level down, internal API
<bilalakhtar> beuno: you mean the RESTful API that lp exposes?
<beuno> yes
 * bilalakhtar understands his task, and still he is determined to work on this feature
<beuno> bilalakhtar, that is awesome
<beuno> I look forward to it!
<bilalakhtar> beuno: Thanks, it will take a week, since I don't get to code very often
<bilalakhtar> beuno: Another question: Why are there 4 lp branches? On which one should I work?
<spm> the latter Q is easy. devel. the former... lengthy to explain.
<wgrant> No Answers stuff is exposed over the API yet. This is not going to be a simple task.
<bilalakhtar> spm: I think the answer is this: All branches merge into devel, then go into edge soon, where they will be deployed to edge.lp.net . staging branch is the code behind staging.lp.net . the lp:launchpad branch is for running the production part of lp.
<beuno> I don't say this very often, but what spm said
<spm> basically, the 4 branches allow devs to keep developing without DB changes blocking all updates. such that edge keeps getting updates till we do a release. staging, is where db mods are trialed.
<wgrant> Ignore stable (edge) and db-stable (staging).
<spm> unless your a losa :-)
<wgrant> And probably read http://dev.launchpad.net/Trunk
<wgrant> spm: Shh.
<bilalakhtar> beuno and wgrant: I will try to copy the code from malone :)
<spm> heh
<wgrant> bilalakhtar: It's not that easy.
<bilalakhtar> losa?
<spm> (launchpad, landscape, ubuntu-one and other stuff) operational sys admin
<bilalakhtar> wgrant: ok, then I will try once, if I fail then will searchi for a bug to patch
<poolie> bilalakhtar, that's great
<wgrant> It's probably best to try some smaller things first.
<spm> the l has becomes somewhat overloaded. I prefer the "l == legendary" explanation myself.
<poolie> good idea though
<bilalakhtar> wgrant: good tip, will search for some tiny bugs
<wgrant> API + JS is not the best combination to start with.
<wgrant> But Answers could certainly do with lots of AJAX.
<spm> hey noodles775
<noodles775> Hiya
<bilalakhtar> noodles775: hi there, buildd admin
<noodles775> ehem
<noodles775> erm, what's wrong...
 * noodles775 starts loading pages :)
<bilalakhtar> I can't believe that lp was proprietary once! Actually, I joined lp quite late
<poolie> https://bugs.edge.launchpad.net/launchpad-answers/+bug/58670 would probably be easy and useful
<mup> Bug #58670: Highlight comments from the reporter <feature> <ui> <Launchpad Answers:Triaged> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/58670>
<noodles775> Yeah, lp has certainly benefited lots from being open :) (IMO)
<poolie> or https://bugs.edge.launchpad.net/launchpad-answers/+bug/226690
<mup> Bug #226690: Not obvious that expired questions can be reopened <confusing-ui> <Launchpad Answers:Triaged> <https://launchpad.net/bugs/226690>
<bilalakhtar> poolie: thanks
<beuno> I remember when it was closed, wgrant was cranky all the time instead of hyper-productive   :)
<poolie> and cranky :)
<wgrant> beuno: See, I wasn't just complaining for the sake of complaining.
 * beuno hugs wgrant 
 * poolie hugs you both
<beuno> :)
 * bilalakhtar is amazed to find lp managed by bots :)
<poolie> i certainly feel better about contributing now it's open
<poolie> even though it was possible before, it just feels better now it's properly open not just internally open
<poolie> and there's more infrastructure and documentation towards helping others
<bilalakhtar> God has said: The world will end at a time when non-living things will take control over the jobs of people.
 * bilalakhtar agrees with poolie beuno and wgrant 
<bilalakhtar> A question: How large is the lp devel branch?
<noodles775> wgrant: just in case you're gone when I try later, is there anything non-obvious that I should watch out for when trying to run a SPRecipeBuild locally (that's not on the runningsoyuzlocally wiki)?
<lifeless> wgrant: they seemed to be about upload rights to me
<lifeless> wgrant: besides which, code, soyuz, its all the same
<wgrant> noodles775: Barring the bug that you're trying to fix, it's pretty simple. Just 'make run_codehosting' (also starts the appserver), push a branch up, and create a recipe through the UI, request a build, start buildd-manager.
<wgrant> Also, recipe builds will crash the buildd due to another bug.
<noodles775> Thanks.
<adeuring> good morning
<wgrant> Bug #587109
<mup> Bug #587109: Needs to cope with not receiving package_name from the master <Launchpad Auto Build System:Incomplete> <https://launchpad.net/bugs/587109>
<wgrant> noodles775: Is buildd-manager crashing?
<wgrant> Maybe it's having recipe fun.
<wgrant> The queue is large, and logtails appear to not be updating, at any rate.
<poolie> bilalakhtar, about 213MB
<bilalakhtar> poolie: oops, will take more than 2 hours on my connection
<wgrant> poolie, bilalakhtar: Plus 210MB for one set of deps, and another 100MBish for the other set.
<wgrant> Plus 100-200MB for the apt dependencies.
<bilalakhtar> wgrant: Should I use rocketfuel?
<wgrant> bilalakhtar: You should use rocketfuel-setup, yes.
<wgrant> Doing it manually is possible, but difficult.
<bilalakhtar> wgrant: ok. so which branch should I work on? I am confused between db-devel and devel, even though I read that runk page
<wgrant> bilalakhtar: If you need to make database changes, work on db-devel. Otherwise, use devel.
<noodles775> wgrant: the last synced log looks fine (up to 8:08 UTC).
<wgrant> noodles775: Argh. Maybe it's just being slow at processing uploads.
<noodles775> erm, that's obviously not utc.
<wgrant> Hey, you never know...
<wgrant> Mm, yeah, it's mostly filled up now.
<wgrant> Although something is still wrong.
<spm> noodles775: I've not had a chance to chase; but would the irregularity around retry-depwait be a possible issue here?
<noodles775> spm: possibly - einsteinium's last entry in the log is certainly 2010-06-04 07:42:40+0100 [-] ***** einsteinium is MANUALDEPWAIT *****
<spm> ew. the retry-depwait log is *full* of entries like that. 2010-06-04 07:09:33 INFO    Found 1076 builds in MANUALDEPWAIT state.
<noodles775> but the others (shipova, rosehip) simply have:2010-06-04 07:40:34+0100 [QueryWithTimeoutProtocol,client] <rosehip:http://rosehip.ppa:8221/> marked as done. [0]
<noodles775> yep, last mention some of the other idle builders is also MANUALDEPWAIT.
<noodles775> s/idle/idle 386/
<wgrant> noodles775: Um, is that nearly an hour ago?
<noodles775> Yes.
<wgrant> They're the latest?
<noodles775> Yep... latest mention in the log.
<wgrant> But it's still showing regular scans?
<noodles775> Yep. And starting new builds on other buildds.
<noodles775> wgrant: sorry...
<noodles775> wgrant: It's dispatching new builds, but you're right, last mention of starting scanning cycle is at: 2010-06-04 07:40:39+0100 [-] Starting scanning cycle.
<wgrant> noodles775: It's dispatching?
<noodles775> 2010-06-04 08:31:56+0100 [-] startBuild(http://dubnium.ppa:8221/, shotwell, 0.5.2-1~karmic1, Release)
<wgrant> As in, has been for the last hour?
<wgrant> That's pretty special.
<wgrant> Since the startBuild calls are asynchronous.
<wgrant> Unless the DB calls are slow?
<wgrant> Which they might well be....
<wgrant> But not that slow.
<wgrant> Surely.
<wgrant> And no hints of any recipe builds firing accidentally?
<noodles775> I'll chekc in a tick.... but checking the frequency of "Starting scanning"... aside from one anomaly, they're all around 2hrs apart :/
<wgrant> For whole long?
<wgrant> Er.
<wgrant> *how* long.
<noodles775> It was fine last night (a few times per minute)...
<wgrant> Ah, good. I was hoping the logging wasn't inconsistent.
<noodles775> Seems to have gotten progressively worse since 2010-06-03 19:15:53+0100 [-] Starting scanning cycle.
<wgrant> Since that would be... not unheard off in buildd-manager.
<wgrant> Hmm.
<wgrant> And there are startBuild calls spread throughout the intervals?
<wgrant> This isn't a failure mode I've seen before.
<noodles775> yeah, me either... it's very strange.
<spm> would it help if I mention that I have the utmost confidence in you guys to figure it out? morale booster? ;-)
<wgrant> spm: Ah, you EOD in 5 minutes, that's why you're so happy :P
<spm> my secret is out :-)
<noodles775> wgrant: the i386 buildds have filled up a bit (but all without logs).
<wgrant> noodles775: Are the startBuilds delayed (implicating the synchronous bit), or are they all within a couple of seconds (implicating the async bit, and ewwww Twisted)?
<noodles775> wgrant: there are definite breaks between some startBuilds calls... I'm including that on the bug I'm creating so we can collect info there.
<wgrant> OK, great.
<wgrant> This is a really, really odd one.
<noodles775> wgrant, bigjools : I've created bug 589577 which has a small snippet of the log before I lost my connection to the log server (and can't reestablish)
<mup> Bug #589577: buildd is not scanning regularly <Soyuz:New> <https://launchpad.net/bugs/589577>
<noodles775> bigjools: it's also got a link to the irc conversation so far.
<wgrant> Ugh.
<wgrant> 13 minutes.
<bigjools> my immediate thought is that the network has a problem
 * noodles775 hopes bigjools is right :)
<bigjools> I am checking with IS
<stub> noodles775: So that build page that was timing out. Does this query provide all the information we need for the bits that are timing out? http://paste.ubuntu.com/444495/
<noodles775> stub: checking
<stub> noodles775: Things also run more than twice as fast on a slave node (that particular query takes just over 5 seconds on the master, but 1.4 seconds on a slave). I don't think anyone will care if the stats are maybe a few seconds out of date.
<noodles775> stub: did you try with the SUM too?
<stub> That's what is in the pastebin, isn't it?
<noodles775> stub: ah, i didn't scroll past the first..
<noodles775> stub: er, I was looking at the wrong paste... got it.
<noodles775> stub: great, so I can update the storm code to (1) run on the slave and (2) use the count/sum in the findspec rather than querying once for each. Certainly looks much better.
<stub> Yup.
<noodles775> Or should I just use the query verbatim (so we know exactly what's being executed)?
<noodles775> (and thanks!)
<stub> Using Storm to generate the query should give you pretty much the same thing
<stub> You can check by turning on the storm SQL tracing. Or getting a user requested oops report.
<stub> I've been looking into indexes - BuildFarmJob.status and Archive.require_virtualized help a little with the existing query, but not much and not at all with the count/sum query
<wgrant> stub: Why's the master so slow? Load?
<wgrant> noodles775: buildd-manager still looks a bit unhappy... any progress?
<noodles775> wgrant: bigjools is still investigating it (we tried disabling retryDepwait in case it was table locks), but not yet that I'm aware of (I've switched back to the builders index now that stub's provided one query to rule them all :) ).
<wgrant> Aha.
<bigjools> wgrant: the findBuildCandidate query is taking 10 minutes instead of 10 milliseconds
<bigjools> we have a missing index
<noodles775> Which one? (I thought stub added BFJ.status?)
<wgrant> bigjools: After seeing the location of the break in the logs, I had a suspicion it might be DBish.
<bigjools> noodles775: don't know, stub is looking at the query for me
<noodles775> ah, great.
<noodles775> oh, that's *the* query... ew.
<wgrant> Yes. *That* query.
<wgrant> I wonder if there is actually a bigger one in LP.
<bigjools> yes, the BUDQ
<wgrant> Maybe the one to expire PPA files.
<bigjools> add "F"s to taste in that acronym
<wgrant> Hm.
<wgrant> I wonder if this is related to the getBuildRecords timeouts that started with 10.05.
<stub> bigjools: Looking at that query, I'm tempted to say scrap it and start again.
<bigjools> stub: it's necessarily complicated
<bigjools> because it's built up from different parts of the code
<stub> Maybe scrap all the EXISTS, refactoring it to precalculate them into temporary tables.
<bigjools> stub: see lib/lp/buildmaster/model/builder.py: _findBuildCandidate
<stub> So that sounds like what needs to be refactored. If the code is generating something that complex and unoptimizable because it has to, there is a problem.
<wgrant> stub: Oh, you've recovered from the horror-induced coma already?
<bigjools> stub: I think that's a good approach, but how can we fix this critical problem right now?
<bigjools> can you see a missing index?
<bigjools> it was working fine until noodles' model change
<stub> Strangely enough, I just ran that query = 440ms
<stub> So the horrible bits only got executed 115 times because the raw queue isn't that big.
<bigjools> yeah, some of them run fast, some are slow
<bigjools> I think it depends on the architecture
<stub> So I need a slow one
<bigjools> we could run the b-m with storm tracing on
<stub> The planner will chose different plans depending on table statistics - eg. using a sequential scan instead of an index lookup if it believes a large percentage of the table needs to be retrieved anyway.
<stub> So all those exists get executed for each and every row not filtered by the preceding criteria. That means between 0 and 54k times I think.
<stub> I can try and chose some bad proceeding criteria.
<noodles775> stub, bigjools: if it's any help, you can see how little changed in that query with bzr diff -c10937 lib/lp/buildmaster/model/builder.py (shown here: http://pastebin.ubuntu.com/444570/)
<bigjools> I suspect the massive buildqueue is not helping
<bigjools> I'm gonna blow away any disabled archive buildqueues
<stub> Do you have the algorithm for finding the next build candidate in English?
<bigjools> I can try
<bigjools> stub: http://pastebin.ubuntu.com/444576/
<stub> So if we have 35k items in the queue (such as we have now for processor=1 and virtualized=true), we order them by lastscore and check them one at a time until all our criteria match. That might be a lot of time.
<stub> If an item doesn't match criteria, why do we keep its lastscore high? If we bumped it to the end of the queue (or just increased it by some factor), the queue items with poor scores would bubble to the end.
<bigjools> stub: scores never change unless changed manually
<stub> bigjools: So for the slow cases I've found, it is the 80% utilization check that is the killer
<bigjools> :(
<bigjools> crap
<stub> Not really
<bigjools> are you going to tell me there's a quick fix? :)
<stub> If we have 10k items in the queue, all in the same archive, we currently end up issuing that query 10k times (failing each time) to get past them
<stub> So we move that out of the SQL. Instead, we do that check when filtering the first real item from the potential candidates, and cache it for subsequent checks in the loop
<stub> Or alternatively, we calculate the list of banned archives once first and filter that way.
<stub> Does the theory sound sane?
<bigjools> I'm not sure
<bigjools> the utilisation is very dynamic
<bigjools> the point is that the query is doing what we'd have to do in Python anyway, so caching seems the only option
<bigjools> I'm going to blitz 54k queue items which should speed this up a but
<wgrant> Do we know that the 80% query is actually doing much useful?
<bigjools> yes, it is
<wgrant> and why is destroying those items a good idea? Is it not possible to filter out the suspended ones first?
<bigjools> it stops the daily builds from monopolising the farm
<wgrant> Mmm not really. There are several daily PPAs.
<wgrant> So it stops a single PPA from monopolising it, and just makes several do it :P
<bigjools> well it depends on when they start building
<bigjools> yes it's still possible given enough PPAs
<bigjools> but at least they still get a look in
<bigjools> hmmm actually it won't help by blitzing them
<bigjools> stub: ok for now I suggest we cowboy out the 80% check until we find a better solution
<stub> Its a quick way of confirming the theory. I don't know if the slow query I manufactured is the same as the slow queries we are seeing on production.
<stub> Actually, I can confirm since I can see the important bit. Goes slow when virtualized=true and processor=1
<bigjools> stub: the first part of the query filters out jobs that are not waiting
<bigjools> so it should not be checking that many rows in that 80% check
<stub> If I comment out that chunk, the query stops taking minutes (I give up and cancel), and instead takes 614ms.
<bigjools> ok I will run this stuff that blows away the suspended jobs
<bigjools> and see if it makes any difference
<stub> I've lost the analyze from before that pointed to it... I seem to recall about 56k checks but I'm not sure where they came from since I would have thought only 35k would be checked
<bigjools> 56k is the number of buildqueue rows
<bigjools> like I said, it's not supposed to be checking all those!
<bigjools> 54k of them are suspended
<stub> So PG decided to do the check before filtering because it thought it would be faster :-(
<bigjools> :(
<bigjools> this was fine before the rollout, I can't work out what's broken it
<bigjools> s/rollout/re-roll/
<stub> Still slow if I force the filtering properly - only a max of 1.8k checks
<bigjools> gah
<bigjools> ok it's gotta go
<stub> Why is the archive=2 check inside the exists?
<bigjools> it only applies to PPAs
<bigjools> public PPAs
<bigjools> flacoste: just the man!
<bigjools> flacoste: we've got problems with the buildd-manager being very slow, I need to make a cowboy, can you approve this please:http://pastebin.ubuntu.com/444605/
<bigjools> it removes the slow query part
<stub> Yes, but the Archive table is from the outer scope
<flacoste> bigjools: ???
<flacoste> bigjools: what does the slow part do?
<bigjools> flacoste: something has changed to make the dispatcher query very slow, we don't know what's caused it
<flacoste> bigjools: iow, what functionality/conditions are we disabling?
<bigjools> flacoste: limits builder usage to 80% of an architecture for each archive
<flacoste> bigjools: why do we do that? or what are the consequences of not doing that?
<bigjools> flacoste: the consequences are that a single PPA can hog all the builders of a single architecture
<bigjools> basically the daily build ppas
<bigjools> but that's currently less worse than a 2 hour scan cycle
<flacoste> bigjools: i agree, should i worry about stub comment abouit the Archive table being from the outer scope?
<bigjools> flacoste: that's part of what we're removing from the query
<stub> Don't mind me - I'm just trying to decode this query
<bigjools> I'd like to restore the build farm first, then look at this problem with less pressure
<stub> +1
<bigjools> I can restart retry-depwait as well and see if those indexes worked
<wgrant> While you're considering build-related queries, getBuildRecords timeouts are causing cron to spam me far too frequently since 10.05. It might be related, I guess, so I thought I might point it out.
<bigjools> wgrant: api?
<wgrant> bigjools: That's the one.
<bigjools> ok
<bigjools> did you file a bug?
<wgrant> No. I was going to wait to see if it persisted -- it has. I'll file one tomorrow.
<flacoste> bigjools: did you start an incident report?
<flacoste> bigjools: r=me with an incident report :-)
<bigjools> flacoste: I've not had time to fart, let alone write an incident report
<bigjools> but trust me when I say I've been thinking about it :)
<flacoste> good
<stub> I just can't decode how that EXISTS is supposed to work at all (the one we are removing). It is trying to filter out jobs if the archive is utilizing 80% capacity. It counts the number of jobs currently active for the archive, but does not count the total capacity so how does it make that calculation?
<bigjools> hmmmm
<bigjools> good point
<wgrant> It divides by %s, which is num_arch_builders, doesn't it?
<bigjools> ah yes - can't tell from the raw log :)
<sinzui> maxb ping
<maxb> pong
<sinzui> maxb: I think I recall you used suggested a fix for a gpg error we were/are seeing when we import a key
<sinzui> s/used/once/
<maxb> Um. Can you show the exact error you are talking about, to try to jog my memory?
<sinzui> maxb: I updated bug 568456
<mup> Bug #568456: GpgmeError raised importing public gpg key <oops> <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/568456>
<sinzui> maxb I recall someone suggested using str() in a cowboy one or two released to fix a gpg error.
<maxb> I definitely recall discussing str vs. unicode issues. It's not, however, obvious to me that this is the same or a related issue
<maxb> I believe we got a more informative message than 'General error' at that time
<sinzui> yes. I think the real error is masked. This would be easier to fix if we could reproduce it
* sinzui changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.05 | PQM is open | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
<noodles775> sinzui: you might have been thinking of this: https://code.edge.launchpad.net/~michael.nelson/launchpad/ppa-generate-key-failure/+merge/24871
<sinzui> noodles775, yes!
<sinzui> noodles775, this may help
<bigjools> stub, noodles775: buildd-manager healthy again with that query part ripped out
<stub> bigjools: How many builders for processorfamily 1 are there?
<bigjools> stub: https://edge.launchpad.net/builders
<bigjools> 17
<bigjools> well, 14 for PPAs
<bigjools> I need food, BBIAB
<stub> bigjools: http://paste.ubuntu.com/444626/ is the query modified to use NOT IN to filter out archives that are over capacity, and runs reasonably fast. I'm not sure of the rules though - can only PPA archives go over capacity?
<bigjools> stub: yes, that rule only applies to PPAs
<stub> bigjools: http://paste.ubuntu.com/444637/ then?
<bigjools> stub: one other option is to factor out that query so we have a python list of archives that is evaluated once, and then we plumb that result into the bigger query
<bigjools> the list is never going to be very big
<stub> bigjools: The bit in the 'NOT IN' should only be evaluated once inside that query. If you are calling that function multiple times though in a transaction, then factoring it out would be better
<bigjools> stub: no, it gets called once for each polled builder
<bigjools> stub: anyway, that's awesome, thanks
<krkhan> "bin/test -m bugs" starts running *all* tests. what would be the correct way of running only tests related to bugs
<krkhan> "bin/test -t bugs" does the same
<beuno> -v?
<beuno> I don't remember exactly
<bigjools> -m bugs should only run the tests in the bugs module
<krkhan> -m bugs spends an eternity on setting up layers e.g. "Set up canonical.testing.layers.FunctionalLayer". is that normal?
<bigjools> unfortunately yes
<krkhan> ouch :-) okay
<EdwinGrubbs> Ursinha, do you know if anyone from translations will be available today?
<EdwinGrubbs> adiroiban, ping
<bdmurray> Is it not possible to subscribe to wiki pages on dev.launchpad.net?  I tried subscribe user and got a 'you are not allowed to perform this action' message.
<krkhan> bin/test fails with the error IOError: [Errno 2] No such file or directory: '/var/tmp/mailman/logs/smtpd'. i'm using karmic. any ideas what's causing it?
<bdmurray> krkhan: how are you using bin test? what is the full command?
<maxb> bdmurray: So, the problem is that whoever wrote the moin skin for dev.lp.net did a somewhat poor job, and removed the 'subscribe' action link
<maxb> the 'subscribe user' link you see is, I think, for subscribing other people
<maxb> bdmurray: Workarounds are to either manually append ?action=subscribe to the page url, or to go to https://dev.launchpad.net/UserPreferences and enter page names in the relevant form field
<rockstar> sinzui, if I have required=True in my interface field, and use use_template() to copy the field to a form schema, shouldn't the validation check it if it's blank?
<lifeless> gary_poster: thanks for digging into the oops view thing
<lifeless> gary_poster: I do enjoy finding turtles-all-the-way-down bugs :)
<gary_poster> lifeless: sure.  lol, yeah, that's what this was. :-)
<sinzui> rockstar, I think the answer is true. I have not use use_template, but I have used copy_field, and it does copy the required=* behaviour
 * sinzui had to override the complete copy behaviour in fact.
<rockstar> sinzui, well, it doesn't seem to be grabbing that behavior.
<sinzui> do you know if copy_template is also using copy_field for build a schema?
<rockstar> sinzui, no, I don't.  Lemme try something.
<bdmurray> maxb: thanks for the work around!
<rockstar> sinzui, so, it looks like the required=True part is checked AFTER validate, so you have to check the data to see if the key exists in the validate method...
<rockstar> sinzui, that seems...backward.
<sinzui> rockstar, that is not right, field validation does happen before form validation in LaunchpadFormView.
<rockstar> sinzui, This oops says otherwise: https://lp-oops.canonical.com/oops.py/?oopsid=1615EA2203
<sinzui> rockstar, LaunchpadFormVIew_validate() validates the widget input before the view's validate() method. That is why the view's method can check for field errors at the start
<rockstar> sinzui, hm, so I'm not sure why this bug is occurring then.  If I add the "if data.get('name', None):" it works fine, gives me the validation error, etc.
<rockstar> sinzui, does it continue on, gathering all field errors from _validate and validate before it gives you all errors?
<sinzui> rockstar all field errors are created as the widgets are iterated. The the view's method validation() gets the data to do additional rule checking, invariant kinds of checking
<sinzui> rockstar, does this field also have a default? because that can create a value
<rockstar> sinzui, no, no default.
<sinzui> rockstar, is there another oops id, that url will not load
<rockstar> sinzui, I don't think so:  OOPS-1615EA2203
<rockstar> That's the one from barry's bug.
<sinzui> rockstar, looking at that oops, is the error that name has a space in it?
<rockstar> sinzui, the way I reproduced it, it was entirely empty.
<sinzui> ISourcePackageRecipe?
<rockstar> sinzui, yes.
<sinzui> that is a bad name
<krkhan> bdmurray: i was using bin/test -t lp.bugs. but i'm getting a lot of other failures as well. i guess the devel branches aren't supposed to pass each and every test, are they?
<sinzui> TextLine(title=_("Name"), required=True, constraint=name_validator,description=_("The name of this recipe."))
<sinzui> rockstar, what view is this? I want to read the @action
<sinzui> rockstar, is this it: SourcePackageRecipeAddView(RecipeTextValidatorMixin...)
<sinzui> rockstar, I have seen this
<sinzui> rockstar, When the vocabulary field is invalid, it is not in the data. It may also be true for NameFields I think you want to check for field errors at the start of validate()
<sinzui> rockstar, several views have a guard at the start of validate()
<sinzui> if len(self.errors) > 0:
<sinzui>     return
<rockstar> sinzui, ah, okay, that makes more sense.
<sinzui> I don't see any looking at the errors, they just get the len() and leave it is not zero
<rockstar> sinzui, okay.  I would think that LaunchpadFormView would be a better place for that check.  What do you think?
<sinzui> No, because validate() is allowed to overwrite those errors. The ones we get from zope field are so arcane that gary has to look them up
<gary_poster> heh
<rockstar> hahahahahaha
<sinzui> The bug supervisor message is an example of one that is easy to put into a sentence, but the field validation about vocabularies make no sense to a user
<zyga> gmb, ping
<zyga> gmb, I'd love to use your django-xmlrpc library
<adiroiban> EdwinGrubbs: hi
<EdwinGrubbs> adiroiban, I was told you could answer some questions I have about LP Translations. I'm working on caching the sum of the POTemplate.messagecount in the DistributionSourcePackage table in a new column called po_message_count. I'm wondering if the best place in the code to update the cache would be POTemplate.importFromQueue() and POFile.updateStatistics() since they both update POTemplate.messagecount.
<adiroiban> hm... well, in LP translations can be updated by both an import, or by direct submission via web interface
<adiroiban> ah
<adiroiban> sorry
<adiroiban> for the POTemplate
<adiroiban> so. for messagecount of a POTemplate, importFromQueue() should do it...
<adiroiban> I am not sure why POFile.updateStatitics() is updating the POTemplate.messagecount ...
<adiroiban> looking
<adiroiban> EdwinGrubbs: I don't know why the code from POFile.updateStatistics() is touching the potemplate.messagecount. It was added to fix bug 371453, but I can not find any clue
<mup> Bug #371453: Broken statistics <message-sharing> <ui> <Launchpad Translations:Fix Released by danilo> <https://launchpad.net/bugs/371453>
<EdwinGrubbs> adiroiban, do you think it's safe to add extra logic to those two methods?
<adiroiban> The code from POFile.updateStatistics() that is updating the POTemplate looks strange, in before adding anything I would talk with Danilo ...
<adiroiban> but Danilo is on leave
<adiroiban> POTemplate.importFromQueue should be right place for updating POTemplate.messagecount cached value
<adiroiban> EdwinGrubbs: also, POFile.updateStatistics() is called in POTemplate.importFromQueue() ... after
<adiroiban>             # Update cached number of msgsets.
<adiroiban>             self.messagecount = self.getPOTMsgSetsCount()
<EdwinGrubbs> hmmm, that doesn't seem good
<adiroiban> EdwinGrubbs: also
<adiroiban> the code from POFile
<adiroiban> is using potemplate.getPOTMsgSets().count()
<adiroiban> instead of potemplate.getPOTMsgSetsCount()
<adiroiban> but this is a minor fact
<adiroiban> so, to askwer your initial question
<adiroiban> I would say that POTemplate messagecount cache data should be updated only in importFromQueue
<EdwinGrubbs> thanks
<adiroiban> EdwinGrubbs: I could not find the MP for the branch that added those lines in POFile.updateStatistics()
<adiroiban> if you can find it, maybe you could find some tips regarding those lines
<EdwinGrubbs> ok
#launchpad-dev 2010-06-06
<bilalakhtar> Hi there, people. I have installed the 4 launchpad-*-dependencies packages. I am not using rocketfuel. Do I have all the deps to build and run lp?
<bilalakhtar> Hi there, people. I have installed the 4 launchpad-*-dependencies packages. I am not using rocketfuel. Do I have all the deps to build and run lp?
<bilalakhtar> Hi there, people. I have installed the 4 launchpad-*-dependencies packages. Do I have all the packages required to build and run launchpad? Can I proceed to run rocketfuel? Will rocketfuel install any more packages? or it will just configure the sys?
<bilalakhtar> beuno: Hi there! ^^
<bilalakhtar> Hi there, people. I have installed the packages launchpad-*-dependencies in the lp ppa. Do I have all the deps and can proceed to download branch now?
<bilalakhtar> Sorry, people, my connection was lost, so had to join and ask the question again
<bilalakhtar> Hi there, people. I am having a dependency problem with launchpad. I want to run lp on my computer having a lamp stack already installed. Installing lp will remove PHP, since lp requires apache2-mpm-worker while PHP requires apache2-mpm-prefork. Both of these packages are conflicting. What should I do? Can lp run on prefork? I know PHP cannot run on worker. Please help
<bilalakhtar> beuno: Please help ^^
<bilalakhtar> wgrant: please ^^
<adiroiban> bilalakhtar: try to install LP into a virtual machine
<bilalakhtar> adiroiban: just a minute, seeing how to do that
<bilalakhtar> adiroiban: You mean a chroot?
<adiroiban> bilalakhtar: chroot or a kvm
<bilalakhtar> adiroiban: ok, thanks for the info, will do it in a chroot
<bilalakhtar> Hi there, people. I am trying to run lp in a chroot. I have apache already running in host along with php and mysql. Do I need to stop apache in host before starting the one in chroot?
<adiroiban> bilalakhtar: if they are not running on the same IP:PORT you can have them both
<bilalakhtar> adiroiban: Bot of them will run on the same port by default. I will modify the host one to run on post 8080. Thanks for into
<bilalakhtar> *info
<james_w> lifeless, jml: plane hacking: https://edge.launchpad.net/soupmatchers
<jml> james_w, ooh. /me looks
 * jml is releasing Twisted today
<james_w> yay!
<james_w> thanks
<james_w> there's a README in the branch that will explain
 * james_w -> dinner
<krkhan> is there a place with information about lp layers which are run during testing? i can't figure out which layer does what or which layers do i need for my tests
<jml> krkhan, not really. your best bet is to copy another test that does something similar, or look at lib/canonical/testing/layers.py
<krkhan> jml: thanks
<lifeless> james_w: nice, I shall look more when I get back from the real estate agents
<wgrant> spm: When you appear, can you please kick the i386 buildds back into life?
<wgrant> Most of them died yesterday for no particularly good reason.
#launchpad-dev 2011-05-30
<poolie> thanks for the reviews lifeless
<poolie> will someone land my lazr.restful patch, or should i try?
<lifeless> I think if you set a commit message tarmac will pick it up
<poolie> wow
<poolie> i will then
<poolie> can i just say how satisfied i am that is fixed
<poolie> i should have done it ages ago
<poolie> i have seen that cryptic message so many times
<lifeless> StevenK: you may have closed bug 117163 ?
<_mup_> Bug #117163: Soyuz email generation should be part of the LP mailnotification module <email> <feature> <lp-soyuz> <soyuz-build> <soyuz-upload> <Launchpad itself:Triaged> < https://launchpad.net/bugs/117163 >
<StevenK> lifeless: I have not, but I have made it easier to do so.
<lifeless> hah!
<lifeless> I now have a solid use case for team-subscription-muting: private bugs
<StevenK> lifeless: I'm completly happy to self-review this, but I wanted your opinion if it's pointless or not (hopefully I have explained myself well enough in the MP description): https://code.launchpad.net/~stevenk/launchpad/silence-rf-get/+merge/62788
<lifeless> StevenK: I saw it
<lifeless> your logic is flawed
<lifeless> but I'm happy with the change
<lifeless> the warning was never about private branches
<lifeless> its for community contributors that haven't run 'bzr launchpad-login' - they will get bzr whinging
<lifeless> OTOH removing the crufty explanation is fine IMO
<wgrant> lifeless: That's not a solid use case.
<wgrant> lifeless: Conflation of notification and visibility is a bug.
<lifeless> wgrant: not with a proper overhaul in progress :)
<lifeless> wgrant: but till then ...
<lifeless> wgrant: there is a bug launchpad-bugs is subscribed too that is private and on c-i-p
<wgrant> Right, but it's a hacky use case.
<wgrant> Not a solid one :)
<lifeless> wgrant: even after privacy is overhauled
<lifeless> wgrant: cross-organisational stuff may well want it
<lifeless> wgrant: Having subscription *not* imply access would be weird.
<lifeless> bug 187837 - still relevant?
<_mup_> Bug #187837: ILinkData.sort_key should be removed once we stop using action menus <infrastructure> <lp-foundations> <tech-debt> <Launchpad itself:Triaged> < https://launchpad.net/bugs/187837 >
<lifeless> wgrant: bug 185620 - perhaps fixed in your bugs-oops-crusade ?
<_mup_> Bug #185620: DebBugs comment import doesn't handle invalid senders correctly <bugwatch> <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/185620 >
<lifeless> ditto 185613
<wgrant> We are not planning to remove action menus, AFAICT.
<wgrant> They have been minimised, but still exist...
<lifeless> surely 182076 is fixed already,. SO MUCH CRUFT
<wgrant> Bug #182076
<_mup_> Bug #182076: launchpad-form.pt doesn't present widget errors clearly <css> <lp-web> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/182076 >
<wgrant> Bug #185613
<_mup_> Bug #185613: DebBugs comment import doesn't handle encoding errors correctly <bugwatch> <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/185613 >
<wgrant> Probably not.
<lifeless> I'm trying to see if I can delete launchpad-bugs
<wgrant> By removing non-security private bugs?
<wgrant> And then seeing what's left over?
<lifeless> repeat for branch subscriptions etc. Yes.
<lifeless> also killing bugs like 173972 on the way
<lifeless> wgrant: what about bug 225228 ?
<_mup_> Bug #225228: checkwatches error handling needs refactoring <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/225228 >
<lifeless> we also shouldn't *have* security bugs on ~launchpad-bugs. *sigh*
<wgrant> lifeless: Indeed, closed.
<wgrant> lifeless: Isn't using direct subscriptions for visibility awesome? :)
<lifeless> wgrant: what about bug 210901 ?
<_mup_> Bug #210901: Passing strings through LaunchpadValidationError results in double-escaping. <lp-foundations> <tech-debt> <Launchpad itself:Triaged> < https://launchpad.net/bugs/210901 >
<wgrant> lifeless: No idea.
<wgrant> Well, the only thing that comes to mind is "fuck IE".
<wgrant> But that's not much of an idea.
<lifeless> hhhha
<lifeless> how is ie related to that ?
<lifeless> I wonder if folk would be unhappy if I just closed all these bugs.
<wgrant> JavaScript can't be escaped sensibly, so our template system cannot be safe.
<wgrant> In HTML, that is. And IE<9 don't support XHTML.
<lifeless> mwhudson: bug 196345 - whats the /why/ ?
 * mwhudson waits for mup
<mwhudson> or is it private?
<lifeless> private
<lifeless> with a dead web link to a list archive whose web password I have no idea about
<wgrant> I think we should close bug #220625.
<lifeless> can you expand on why?
<mwhudson> lifeless: trying :)
<lifeless> mwhudson: that was to wgrant :P
<mwhudson> lifeless: it's all about impersonation
<lifeless> I probably want that
<lifeless> is it private these days? if not, lets make it public and copy in the discussion ?
<mwhudson> methods like createBranch clearly act on behalf of a user, and currently rather hack to achieve that
<mwhudson> lifeless: can't see why it would be private at all now
 * mwhudson publicizes
<lifeless> mwhudson: done
<wgrant> lifeless: There are much worse things that a buildd-manager bug can do.
<mwhudson> heh
<mwhudson> racing changes!
<lifeless> mwhudson: can you add sensible content to it ?
<lifeless> wgrant: right, but what about other folk on the machine, for instance.
<lifeless> wgrant: or other services on the same machine going rogue
<mwhudson> lifeless: let me consult my mail archives
<lifeless> mwhudson: thanks!
<wgrant> lifeless: There are far worse things that they can do too.
<wgrant> lifeless: If we don't trust the machine, then we have major overarching design issues.
<wgrant> Far beyond this bug.
<lifeless> wgrant: I'm fine with closing it, or not. But I think you need to explain a little about why in this case.
<lifeless> as its not a cosmetic test-refactoring bug :)
 * mwhudson reads the thread again
<mwhudson> ugh, this is all very confusing
<mwhudson> and the use case from the time has disappeared
<mwhudson> (it wasn't impersonation)
<lifeless> feel free to extirpate it
<lifeless> jtv - bug 237868
<_mup_> Bug #237868: Contain non-ascii str objects in parsers and browser code <lp-translations> <tech-debt> <Launchpad itself:Triaged> < https://launchpad.net/bugs/237868 >
<lifeless> jtv: is that done, or obsolete now?
<lifeless> gmb: bug 237774
<_mup_> Bug #237774: checkwatches doesn't update many SourceForge bug watches <bugwatch> <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/237774 >
<lifeless> gmb: please tell me that isn't still the case?
<jtv> lifeless: it was never consciously done.  It was more a "we need to take a critical look at our code someday" thing than a "here's a change we need to make" bug.
<lifeless> so, we're on storm today
<lifeless> and I'm pretty sure it + our pg driver will barf at non-unicode strings
<jtv> Plenty of test code will happily store non-unicode strings in object properties AFAIK
<LPCIBot> Project db-devel build #594: FAILURE in 5 hr 37 min: https://lpci.wedontsleep.org/job/db-devel/594/
<lifeless> sure, but if it does that to a db column we'll get a barf
<mwhudson> lifeless: https://bugs.launchpad.net/launchpad/+bug/196345/comments/1
<_mup_> Bug #196345: implement endpoint specific authentication for the private xml-rpc server <infrastructure> <lp-foundations> <tech-debt> <Launchpad itself:Triaged> < https://launchpad.net/bugs/196345 >
<lifeless> mwhudson: thanks
<lifeless> bug 196345
<_mup_> Bug #196345: internal xmlrpc service requires impersonation to be handled per-method rather than systematically <infrastructure> <lp-foundations> <tech-debt> <Launchpad itself:Triaged> < https://launchpad.net/bugs/196345 >
<mwhudson> haha
<mwhudson> oh right, you just changed the title
<lifeless> and copied your prosed into the description
<lifeless> lock
<lifeless> stock
<lifeless> and barrel
<mwhudson> i thought that was going to turn out to be another bug that i had reported
<lifeless> wow
<lifeless> https://launchpad.canonical.com/BugPageTwoZero
<mwhudson> are you saying wow because the current bug page actually looks not entirely unlike that?
<lifeless> mwhudson: there is a bug saying we should do that
<lifeless> *still* open
<lifeless> bug 227310
<_mup_> Bug #227310: duplicate bug handling UI is awkward and confusing <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/227310 >
<lifeless> why is bug 300044 on sso ?
<wgrant> Bug #300044
<wgrant> lifeless: It's in the old SSO codebase.
<wgrant> The rewrite uses the same project.
<wgrant> It's Invalid.
<lifeless> you sure ?
<wgrant> Hmm.
<wgrant> Retarget to Launchpad, publicise, retitle to suggest that it should be deleted.
<wgrant> Since we still have those properties.
<wgrant> Despite not being an OP any more.
<wgrant> The last record in the table it checks is dated 2010-04-01, so it isn't being replicated back post-rewrite.
<wgrant> So it can be removed.
<wallyworld_> lifeless: i think storm has a bug in Union. it rejects perfectly valid sql if the 2 queries differ in their exact select expressions, even if the resulting columns are compatible. i can run the sql manually just fine. do you know if there's a reason for this limitation?
<lifeless> wallyworld_: I don't, sorry.
<lifeless> storm is very geared towards its ORM role
<lifeless> which could explain it
<wallyworld_> np. i'll see if i can come up with a reasonable work around
<wgrant> wallyworld_: I suspect it's because it's then unclear how you would reference those columns.
<wallyworld_> wgrant: one would use "as foo"
<wallyworld_> storm should look at the alias and column type to figure out if everything is kosher and not just compare the exact select terms
<wgrant> Right, but that would have to be explicitly done. It won't work by default.
<wgrant> And given that Storm is Storm...
<lifeless> bug 245543
<_mup_> Bug #245543: Call sites that use SPR.createBuild() should really be using SPPH.createMissingBuilds() <lp-soyuz> <Launchpad itself:Triaged> < https://launchpad.net/bugs/245543 >
<wallyworld_> \o/
 * wallyworld_ has a new coffee grinder just arrived from italy. must immediately cease Storm work to try it out :-D
<wgrant> lifeless: Lies.
<wallyworld_> thumper: ^^^^^^^^ :-D
<lifeless> wallyworld_: nice
<lifeless> bug 172869
<_mup_> Bug #172869: SoyuzScript needs more test coverage <lp-soyuz> <tech-debt> <trivial> <Launchpad itself:Triaged> < https://launchpad.net/bugs/172869 >
<thumper> wallyworld_: nice
<lifeless> faceplant. bug 158386
<_mup_> Bug #158386: Close all low-level cursors <infrastructure> <lp-foundations> <tech-debt> <Launchpad itself:Triaged> < https://launchpad.net/bugs/158386 >
<wgrant> zomg
<wgrant> Linux 3.0-rc1
<wgrant> The day has come.
<lifeless> and unlike py3 its not a disaster
<wgrant> Indeed.
<wgrant> :(
<wgrant> Well.
<wgrant> No *more* of a disaster.
<StevenK> If gina create builds, I will have to murder people.
<wgrant> StevenK: It has to when importing binaries.
<wgrant> (which it hasn't done since 2007)
<wgrant> But the feature is still there, somewhere.
<StevenK> Heh, and only once, I daresay.
<wgrant> Twice!
<wgrant> Once to import Ubuntu primary, and again to import -commercial (brokenly)
<jtv> wgrant: mind if I restart the df app server?
<wgrant> jtv: Not at all.
<jtv> Letter-for-letter identical to what StevenK said.  :)  Thanks.
<StevenK> wgrant: Have we fixed that broken -commercial import?
<wgrant> StevenK: No.
<StevenK> Pity.
<StevenK> Hm, not many callsites of SPR.createBuild()
<wgrant> Does archiveuploader still do it?
<wgrant> I forget if we removed mixed uploads.
<wgrant> IIRC they were too entangled with tests to kill.
<StevenK> Yes, nascentupload uses it
<StevenK> Well, nascentuploadfile
<StevenK> Amusingly, createMissingBuilds calls it under the covers.
<wgrant> Is that amusing?
<StevenK> Well, I found it amusing
<wgrant> It is intended that createMissingBuilds should be its only callsite.
<StevenK> Ah
<StevenK> A whole bunch of tests call it too
<wgrant> Yes
<wgrant> lifeless: launchpad-bugs has no contact address, but launchpad has launchpad-bugs@lists.c.c
<wgrant> That's the problem.
<lifeless> uhhhhh fail
<lifeless> ok
<lifeless> so lets remove that
<lifeless> thanks for digging that up
<wgrant> Now that PQM is out it should be fairly safe.
<wgrant> PQM was the main problem last time.
<wgrant> We'll all immediately get loads of vcs-imports spam.
<wgrant> But at least we won't have robots spamming.
<lifeless> fixed
<wgrant> Hm, why is canonical-bazaar in launchpad?
<lifeless> they were a subteam organisationally
<wgrant> Were.
<lifeless> yes. were.
<wgrant> But they were added just before me.
<lifeless> poolie: ^
<wgrant> Like a month before the refactoring.
<lifeless> poolie: what does that team membership do for you ?
<jtv> StevenK, wgrant: does either of you have time to guide me through some Q/A on dogfood?
<wgrant> jtv: Sure.
<jtv> Thanks.  I need to see (1) a pending DSDJob, and (2) a failure on a package copy job.
<wgrant> Ah. StevenK is possibly going to be more effective at that sort of thing.
<jtv> Right ho.
<LPCIBot> Project windmill-devel build #152: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill-devel/152/
<jtv> StevenK: what can I safely to do produce a pending DSDJob to show up on +localpackagediffs?
<StevenK> An upload, copy or removal will create a DSDJ
<jtv> I guess a removal will make the package drop off +localpackagediffs.
<StevenK> Maybe not
<StevenK> A removal will create a DSDJ. Until that DSDJ is run, the existing DSD will not be touched.
<StevenK> Once the DSDJ has run, the DSD will be updated to state unique to derived series or missing from derived series.
<jtv> I see.
<jtv> Would the easiest way though be to sync e.g. a52dec?
<jtv> I guess that'd give me a copy job first, and then when that's done, a DSDJ
<StevenK> Right.
<jtv> Do we run any of the cron jobs on df?
<jtv> ISTR no.
<StevenK> I'm not sure.
<jtv> Well, here goes.  IIRC a52dec is a pretty normal package, safe to copy?
<wgrant> Starting a script takes like 30s of CPU time, so I hope not :/
 * jtv curses zcml once more
<lifeless> micahg: bug 59846 may entertain you with a flashback
<jtv> Argh "same version is already building in oneiric"
 * jtv looks for different arbitrary guineapig
<micahg> lifeless: I don't see anything there
<jtv> StevenK: acct should be safe as well, right?
<lifeless> micahg: retry
<lifeless> micahg: I was just unembargoing it
<micahg> lifeless: heh, yeah
<StevenK> jtv: Yes.
<jtv> Same error.  :(
<jtv> rmadison has nothing special to remark about alarm-clock eitherâ¦
<wgrant> lifeless: You asked of bug #121538 "Can this bug be opened ?"
<lifeless> wgrant: made public
<wgrant> I think you may have been through too many this afternoon :P
<wgrant> Oh.
<wgrant> I see.
<StevenK> jtv: alarm-clock is already identical
<jtv> Is _now_ :)
<jtv> It was -1.1 vs -1ubuntu1
<jtv> It correctly shows up as "updating..."
<jtv> And now it's off the list.
<jtv> So we are running DSDJobs on dogfood.
<micahg> lifeless: should I dupe mine to it
<lifeless> no
<lifeless> its fix released
<lifeless> and sufficiently long it would be a pita to reuse it
<jtv> StevenK: now I'd like to try an asynchronous copy...  I'll disable synchronous copying for a moment.  Any objections to syncing amule, ajaxterm, or amsn?  If I read rmadison output right, they have some version differences across architectures but they're not in nonfree.
<wgrant> Do they?
<jtv> Well not ajaxterm since it's for "all"
<wgrant> Nor amsn.
<wgrant> Nor amule.
<lifeless> hmmm
<lifeless> I wonder if merging -bugs and -security would be better
<jtv> wgrant: I ran e.g. $ rmadison -u debian -s sid amsn
<wgrant> jtv: Why Debian?
<wgrant> Debian binaries are not interesting.
<wgrant> lifeless: Possibly, if there are no non-bugs -bugs references.
<jtv> Because I need to know whether I can copy it into oneiric without creating an invalid publication.
<lifeless> wgrant: if only we had a UI to show subscriptions
<wgrant> lifeless: For everything else, there's psql.
<wgrant> jtv: Ah, I see.
<wgrant> jtv: Those should indeed be safe.
<jtv> And while I love the idea of being the author of an illegal publicationâ¦
<lifeless> ok, how to request a team merge ?
<wgrant> lifeless: /people/+adminteammerge, but let's check this out first.
<jtv> StevenK: next I need to trigger a CannotCopy on an asynchronous copy requestâ¦ is there some off-the-shelf way of getting that?
<jtv> I'll try a regular run first though, to verify the common case.
<wgrant> lifeless: It's still bug supervisor for a couple of places.
<StevenK> jtv: I can't think of an easy way off the top of my head
<jtv> StevenK: on the bright side, the status quo for handling that is an oops.  So from a branch Q/A standpoint, just verifying that it works when there are no errors means the change is good to deploy.
<jtv> What script was the processing of PPCJs hidden in again?
<jtv> process-pending-packagediffs?
<jtv> No
<wgrant> I thought one just used run_jobs.
<lifeless> wgrant: according to you thats not conflated with subscriptions now
<wgrant> lifeless: It has some structuralsubscriptions, some assigned questions and bugtasks, and is the bug supervisor and security contact on some projects. That appears to be it.
<wgrant> lifeless: It's not.
<lifeless> we should nuke the strucsubs
<wgrant> Right.
<lifeless> the rest we can tolerate
<wgrant> But launchpad-security may not be the correct bug supervisor for all our projects.
<lifeless> it shouldn't be a security contact anyway
<wgrant> KIndeed.
<lifeless> wgrant: -security and -bugs are functionally equivalent now
<wgrant> lifeless: OK.
<wgrant> https://launchpad.net/~launchpad-bugs/+structural-subscriptions
<wgrant> It has as few.
<wgrant> I suspect all should be deleted.
<wgrant> Before we merge.
<lifeless> we can't open -bugs to non-staff because of the tonne of old bugs that would need auditing
<lifeless> 400 odd
<wgrant> Yeah.
<lifeless> so if we want different supervisors
<lifeless> we can't use -bugs anyhoo, so merging won't make this worse
<wgrant> We should delete the structsubs first.
<lifeless> yes
<lifeless> am on it
<lifeless> its not ajaxy
<wgrant> Thanks.
<wgrant> malone-developers has bugs assigned, and that's all.
<wgrant> Might merge it into ~launchpad.
<lifeless> qprocd wtf
<wgrant> Heh
<lifeless> dilys?!
<wgrant> .. yes?
<wgrant> You don't remember dilys?
<lifeless> cleared out
<wgrant> lifeless: ~malone-developers isn't a celebrity, only member is ~launchpad, and only artifacts are assigned bugtasks. Any objections to merging it into ~launchpad?
<wgrant> Or even deleting it.
<lifeless> hahha
<lifeless> want to file this bug ?
<lifeless> https://bugs.launchpad.net/~malone-developers/+assignedbugs?advanced=1
<lifeless> note the team portlet
<lifeless> wgrant: have you cleared the bug assignments ?
<wgrant> lifeless: No. Deletion will do that.
<lifeless> I can't see them
<lifeless> (even with closed bugs checked)
<wgrant> Neither. Let me find what they are.
<wgrant> Bug #3460
<_mup_> Bug #3460: Bug reported as no secret is not accessible anymore. <lp-foundations> <Launchpad itself:Incomplete by malone-developers> < https://launchpad.net/bugs/3460 >
<wgrant> It's a dupe.
<lifeless> doh
<lifeless> https://launchpad.net/~malone-developers/+adminteammerge 404s
<lifeless> oh
<lifeless> literal people
<wgrant> /people/+adminteammerge
<wgrant> Yeah.
<wgrant> But that team can just be deleted.
<wgrant> I guess it doesn't make much difference.
<lifeless> I think we're gtg on bugs now
<lifeless> yes?
<wgrant> As long as we don't care about bug supervisor, security contact or question/bugtask assignee, yup.
<lifeless> ah questions
<lifeless> lets see about those
<wgrant> 1 question
<wgrant> 109645
<lifeless> 1 bug
<lifeless> unassigned
<wgrant> Bug #174335?
<_mup_> Bug #174335: [remote-bug-watcher] GStreamer's Trac has switched to https <lp-foundations> <Launchpad itself:Fix Released> < https://launchpad.net/bugs/174335 >
<wgrant> Question unassigned
<lifeless> and supervisor I don't care about
<lifeless> I will raise that its odd
<wgrant> That leaves us with just bugsubscriptions and products.
<wgrant> So let's doit.
<lifeless> products?
<wgrant> bug supervisor and security contact.
<wgrant> Which you say we don't care about.
<lifeless> ah - shrug ;)
<wgrant> Which seems reasonable.
<lifeless> security contact should be -security
<wgrant> Yup.
<lifeless> we do care - its wrong today :)
<lifeless> supervisor won't functionally change
<wgrant> So, -bugs -> -security, malone-developers -> oblivion?
<wgrant> ~malone-developers is already gone, I see.
<wgrant> Excellent.
<lifeless> merged FTR
<wgrant> k
<wgrant> -> launchpad
<wgrant> ?
<lifeless> yeah
<jtv> This is worrying: my branch is "Merged," my MP is "Merged," the bzr log shows my revision, but my changes don't seem to be in the code.
<wgrant> jtv: Which rev?
<jtv> What am I missing?  It's this MP: https://code.launchpad.net/~jtv/launchpad/db-bug-786970
<jtv> db-devel 1617
<jtv> 10617
<wgrant> Which changes are missing?
<wgrant> mailing-list-experts is also no longer a celebrity and obsolete and unreferenced, besides owning the already deleted mailing-list-beta-testers.
<jtv> AFAICT at least the first one in the diff: security.cfg
<wgrant> jtv: It's in db-devel for me...
<wgrant> [sync_packages]
<wgrant> groups=script
<wgrant> public.account                                = SELECT
<jtv> Yeah, just showed upâstrange
<wgrant> lifeless: launchpad-bugs is gone! Thanks.
<poolie> lifeless: hi, what do you mean what does it do?
<lifeless> poolie: just that
<wgrant> lifeless: canonical-loggerhead only controls access to ancient private loggerhead branches. -> canonical-launchpad-branches?
<lifeless> yes
<lifeless> want to do the honours?
<wgrant> Yup.
<wgrant> Done.
<wgrant> So much cruft :(
<lifeless> there may be some config manager configs to cleanup with that last one ?
<wgrant> It didn't own anything.
<wgrant> Only subscriptions.
<lifeless> and the branches ?
<poolie> lifeless: well, for instance we assign bugs to ~canonical-bazaar to indicate they're on our short list
<wgrant> lifeless: Hm?
<lifeless> poolie: oh, econtext. I don't mean 'what do you use the team for with launchpad.net'
<poolie> (we could have a tangential conversation there about per-person or per-team hit lists)
<lifeless> poolie: its in ~launchpad. What does that *membership* do for you.
<poolie> oh i see
<poolie> i don't know
<lifeless> wgrant: 17:25 < wgrant> lifeless: canonical-loggerhead only controls access to ancient private loggerhead branches. -> canonical-launchpad-branches?
<lifeless> wgrant: won't that have changed urls ?
<poolie> what is ~launchpad
<lifeless> poolie: folk that report to francis, more or less
<lifeless> poolie: including himself.
<poolie> :) hm odd name for it then
<wgrant> It also controls who can see tracebacks.
<wgrant> That's about it.
<poolie> so only canonical staff?
<lifeless> and transitively closed
<lifeless> yes
<wgrant> lifeless: No, it only subscribed to branches. It owned nothing.
<lifeless> wgrant: ~canonical-loggerhead controls who sees tracebacks ?
<poolie> i don't think the team membership does anything much then
<wgrant> lifeless: I was speaking of ~launchpad.
<lifeless> heh
<poolie> hm
<wgrant> The only specialness it has within LP is granting tracebacks.
<poolie> i mean it is useful for us to see tracebacks
<lifeless> yeah
<poolie> so you could expand out the membership to the individuals, if you think that is cleaner
<wgrant> Or move it to a flag :)
<lifeless> perhaps we should make that a ff
<lifeless> jinx
<wgrant> Means we can delete more celebrities.
<wgrant> Yay.
<poolie> :) oh speaking of which, the bug about staging flags
<wgrant> The logic way to control that is with flags, yes.
<poolie> would it make sense to do that off static configuration that's different between lpnet and other instances?
<wgrant> Flagception, and all that.
<lifeless> poolie: no
<lifeless> poolie: rather, you could. But I wouldn't :)
<poolie> because... that config stuff is a pain?
<poolie> i was just thinking of what is permanently different between them
<lifeless> lazr.config is a pita yes.
<poolie> it seems kludgy to need to reset it after every synchronization
<poolie> it may be the least kludgy thing i suppose
<lifeless> poolie: there is an is_beta setting etc
<lifeless> but I think its better to have the stuff visible
<lifeless> rather than magically differ based on not obviously related settings
<poolie> oh, well
<poolie> i would have (handwavy) added a static configuration that says what person is allowed to edit, and to see, flags
<poolie> *flag rules
 * lifeless shudders
<poolie> ?
<lifeless> poolie: that would either require a team on production that matches the policy we want on staging (so that you can refer to it statically in the staging config)
<poolie> but surely there is one; perhaps ~launchpad ?
<lifeless> poolie: ~launchpad doesn't include ~admin
<poolie> hm
<poolie> ok
<poolie> i crave enlightenment
<poolie> is that the main thing that would be gross about it?
<lifeless> this doesn't seem like config to me
<lifeless> uhm
<poolie> i guess to me bootstrapping-type information like who should get certain privileges seems like the kind of thing that goes in static configuration
<poolie> in general; but that may not be a good fit for the launchpad idioms
<lifeless> so
<lifeless> things that might vary by where the server/script is running and how other services are talking to it
<lifeless> those should be config (today)
<lifeless> nothing else shold be
<lifeless> because the config is spread out over about 100 locations
<wgrant> thumper: Does ~launchpad-recipe-beta still need to exist?
<lifeless> to deal with bootstrap issues the instance-creation script (make schema) seeds sensible defaults
<lifeless> poolie: ^
<lifeless> wgrant: its gone from the feature rules
<wgrant> It is about to be gone in general.
<lifeless> wow
<wgrant> ?
<lifeless> the wiki bug has gone over the 'can-comment' limit
<wgrant> Heh
<thumper> wgrant: no
<wgrant> thumper: Thanks.
 * wgrant destroys.
<poolie> lifeless: so this wouldn't fall under "vary by where the server/script is running"?
<lifeless> poolie: no
<lifeless> poolie: we wouldn't want it different for two prod servers
<lifeless> therefor it shouldn't be a config setting
<poolie> ok
<lifeless> http://webnumbr.com/launchpad-critical-bugs.derivative
<StevenK> Looks like math homework. "Using only this graph, determine exactly when William Grant switched from maintaince to feature work."
<wgrant> :(
<adeuring> good morning
<jtv> hi adeuring
<adeuring> hii jtv!
<wgrant> jam: Which bugs?
<lifeless> I have a feeling bug 117166 may be fixed
<_mup_> Bug #117166: Soyuz tests should use standard LP logger <lp-soyuz> <tech-debt> <test-system> <Launchpad itself:Triaged> < https://launchpad.net/bugs/117166 >
<wgrant> jam: There should be no team bug subscriptions left, except for ~launchpad-security (which has its own contact address)
<jtv> Any reviewers willing to pick up an easy fix?  https://code.launchpad.net/~jtv/launchpad/db-bug-790084/+merge/62845
<jam> wgrant: so unfortunately I cleared them out of my bug folder, but I'll go track them down, just a sec
<jam> wgrant: 47941 for example
<wgrant> Hm, ~launchpad is apparently subscribed to 351 or so bugs.
<jam> well, 74941 if I could type correctly
<wgrant> Hm.
<wgrant> Do you have the rationale from that email?
<wgrant> there's no ~launchpad subscription near it, AFAICT.
<lifeless> hmm
<jam> wgrant: You received this bug notification because you are a member of Launchpad
<jam> Bug Contacts, which is a direct subscriber.
<jam> https://bugs.launchpad.net/bugs/74941
<_mup_> Bug #74941: Database permission exception in ticket expiration <lp-foundations> <Launchpad itself:Fix Released by flacoste> < https://launchpad.net/bugs/74941 >
<lifeless> person merge happily creates dual subscriptions to one bug
<lifeless> jam: you're in canonical-bazaar which is in launchpad which was in launchpad-bugs
<wgrant> jam: That was pre-merge.
<wgrant> jam: That team no longer exists. Are you still getting mail?
<jam> wgrant: ok, it was about 20 bugs from this morning, about 4 hours ago
<lifeless> that was me :)
<wgrant> Probably lifeless unsubscribing ~launchpad-bugs from half the world.
<wgrant> Just after the contact address was removed.
<jam> lifeless: correct, most of them are you
<jam> though not all unsubscriptions
<wgrant> Some deprivatisations?
<jtv> stub: are you reviewing?  https://code.launchpad.net/~jtv/launchpad/db-bug-790084/+merge/62845
<jam> wgrant: at least one of them, yes.
<jam> Anyway, if it is going away now, great. I don't have any *proof* of that yet, but I also wanted to clearly state that launchpad doesn't clearly let me unsub
<jam> this was a ~bzr issue. The community wants yet-another team
<wgrant> jam: Well, the subscription is already removed.
<wgrant> You can't unsubscribe, since it was done between you receiving the email and reading it :)
<lifeless> jam: you can't unsubscribe because you aren't subscribed
<jam> lifeless: sure. Though having gotten emails within a couple hours, not having actually unsubscribed myself, and not having gotten a clear "you are now unsubscribed" made that a bit unclear
<wallyworld_> lifeless: are you able to run some sql against qastaging for me? i don't think there's a dba around. i'd like to see the result set and query plan
<lifeless> wallyworld_: sure
<wallyworld_> https://pastebin.canonical.com/47947/
<wallyworld_> thanks
<lifeless> jam: FWIW see the activity log
<lifeless> wallyworld_: FTR - losas can do this, we have 24x5 coverage, stub can do it - he's here, and all squad leads can as well :)
<wallyworld_> oh ok. thanks for the clarification
<wallyworld_> you want a losa to do it this time?
<lifeless> no, I said sure :)
<lifeless> besides which, its running now.
<wallyworld_> np.
<lifeless> 50 seconds
<wallyworld_> and hot?
<lifeless> 6.6
<wallyworld_> it's for the person picker. it does correct ordering of results etc
<lifeless> so - too slow by at least 2.1 seconds
<wallyworld_> i think that time is ok for qas?
<lifeless> no
<lifeless> remember vocabs count(*) and execute
<lifeless> qas is only slightly slower than prod at query execution
<wallyworld_> sure. i never know how much to discount for the fact that qas is so slow
<wallyworld_> right
<lifeless> don't discount at all ever
<lifeless> do you want all 100 rows of output ?
<wallyworld_> sure. but looks like it's back to the drawing board since the query is slow. perhaps an explain can say why
<lifeless> have we added tje person fti yet ?
<wallyworld_> i think it's already on qas?
<wgrant> lifeless: I don't think so.
<wgrant> lifeless: I asked stub about it.
<wallyworld_> ah. that could explain it. the new query is actually simpler than the current one i am pretty sure. less subselects
<wallyworld_> does staging or dog food have the person fti?
<lifeless> hah
<lifeless> you have't prepped for this at all
<wgrant> DF has it.
<lifeless> I'm going to skip the results
<lifeless> and talk you through working on queries in our environment a little :)
<lifeless> wallyworld_: https://pastebin.canonical.com/47948/
 * wallyworld_ is scared
* henninge changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: henninge | Critical bugs:209 - 0:[######=_]:256
<wallyworld_> ok got the pastebin
<lifeless> note the estimated cost -  261638
<lifeless> thats huge
<wallyworld_> yeah
<lifeless> the Merge Left Join  (cost=0.34..212979.61 rows=7785 width=834) (actual time=1539.047..51003.757 rows=3942 loops=1)
<lifeless> contains most of the time
<lifeless> Merge Left Join  (cost=0.00..207878.63 rows=1295397 width=830) (actual time=23.991..47925.318 rows=1271338 loops=1)
<wallyworld_> that's to the ircid table i think. that table is quite small i thought
<lifeless> and that then shows whre its going - 1.2M rows
<lifeless> person
<jam> lifeless: its only 1.2Million :)
<wallyworld_> would a subselect be quicker i wonder
<lifeless> wallyworld_: nope
<lifeless> this is a selectivity thing
<lifeless> there is no index on displayname
<lifeless> there is no index on fti
<wallyworld_> the index on displayname is on the todo list ie we need to get it added
<wallyworld_> same with fti
<lifeless> wallyworld_: ok so process
<lifeless> you can't sensibly craft *any* queries without getting measurements with indices added
<lifeless> which losas/stub can do
<lifeless> stub: while you're here ;)
<wallyworld_> i was initially just after the result set, and though why not see the explain while we were there
<lifeless> stub: our schema claims person_fti is indexed.
<lifeless> stub: qastaging has no index on person.fti; I'm guessing production is the same.
<wallyworld_> lifeless: i was initially looking to see the correctness of the results. i shouldn't have asked for explain yet. sorry.
<lifeless> wallyworld_: its all good
<lifeless> wallyworld_: I wouldn't do this in separate phases:
<wallyworld_> well, if the results are crap, no good profiling the query
<lifeless> wallyworld_: the ability to get the right results in the right timeframe (subsecond render time) requires working on both things at once
<wallyworld_> fair enough. i was too eager after fighting with storm for a few hours
<lifeless> generally speaking for collections to meet our goals the query has to be ~500ms stable
<wallyworld_> ok
<wallyworld_> so if we need to get person.fti set up, i'd like to get person.displayname done too
<wgrant> jam: ~launchpad had a few hundred ancient public bug subscriptions. They are currently being removed.
<wallyworld_> i'll try and catch stub later to ask him
<lifeless> wallyworld_: I'm confused though
<lifeless> why are you checking displayname
<wallyworld_> about?
<lifeless> when fti includes displayname
<wallyworld_> because we want exact matches to occur first
<wallyworld_> and afaiu, fti doesn't do that
<wgrant> stub had some compelling arguments for doing a string match first.
<wgrant> stopwords and stemming and such.
<lifeless> the second clause does a plain old fti
<wallyworld_> yes, wgrant beat me to it
<lifeless> the first clause uses an ||
<stub> ?
<lifeless> stub: hi!
<lifeless> stub: a few things lined up for you :)
<lifeless> stub: firstly being whats up with person.fti being unindexed
<lifeless> wallyworld_: in the second part of the query - 'AND NOT (Person.teamowner IS NULL)
<lifeless> '
<lifeless> wallyworld_: is a little weird, and that sort of thing can confuse the planner sometimes
<wallyworld_> lifeless: that is as per the current query
<lifeless> k
<wallyworld_> i've kept the sort of stuff you mention above and reworked the select scaffolding and ranking around it all
<LPCIBot> Project windmill-db-devel build #341: STILL FAILING in 1 hr 10 min: https://lpci.wedontsleep.org/job/windmill-db-devel/341/
<wallyworld_> wow, that's poorly worded
<stub> Person.fti not having an index is a bug. The index probably got lost ages ago when we tried to switch from GIST to GIN indexes, but I have no way of being sure without trawling ancient backups...
<lifeless> stub: can you create on prod/qas/staging ?
 * stub checks if he fixed that on prod last week
<wallyworld_> stub: i have to duck off for dinner - after you have looked at the person.fti i'd love it if you could create a person.displayname index too :-)
<wgrant> stub: Oh, we're using GIN now?
<wgrant> stub: The dev index is GIST.
<lifeless> wgrant: no
<lifeless> wgrant: we tried too
<wgrant> Or is that "tried and failed"?
<wgrant> Heh.
<stub> I'll create the fti indexes now. Do you have the query using displayname?
<wgrant> Right.
<lifeless> stub: https://pastebin.canonical.com/47947/ is the draft
<stub> wgrant: 9.0 or 9.1 adds the feature we need... forget which
<lifeless> stub: I'm not clear why we want to check displayname ?
<wallyworld_> yeah, what he said ^^^^
<wallyworld_> the pastebin, not the question
<stub> wallyworld_: You sure you want exact match, and not say case insensitive prefix match like email address is?
<lifeless> stub: I think we just want fti
<stub> Both can use an index, but they are different indexes.
<lifeless> stub: I'd really like to understand why we don't.
<lifeless> stub: because the current draft seems a bit crazy - its oring the exact and the fti checks together, and the fti match is a superset of the exact
<stub> lifeless: Because we dropped it ages ago and didn't recreate it, and the fti.py script isn't smart enough to catch this error.
<wallyworld_> i was thinking exact match. wouldn't the fti pick up the other sort?
<lifeless> stub: crossed wires. I'm cool on the fti thing - I know how that came to be.
<wallyworld_> the exact matches are ranked higher than the fti ones
<stub> wallyworld_: fti is case insensitive word matches. quite a different thing to an exact or prefix match.
<lifeless> wallyworld_: fti will pick up everything. I don't think there is a single result the displayname check will match that fti won't.
<wallyworld_> lifeless: yes, but the ranking is what we care about
<stub> lifeless: There are cases where stopwords and stemming gives broken results
<wallyworld_> and fti has limiations there
<lifeless> stub: but checking both is only useful if one won't match.
<lifeless> stub: we should turn stopwords off for person.fti
<wallyworld_> but how do we know one won't match ahead of time?
<stub> lifeless: we should tune fti everywhere, or switch to a different search engine.
<lifeless> wallyworld_: I don't know what you mean by that
<wallyworld_> maybe i misunderstood your previous staement
<wallyworld_> "checking both is only useful if one won't match"
<lifeless> you have this
<stub> wallyworld_: It is still one query
<lifeless> OR Person.displayname = 'jon'
<lifeless>  ( Person.fti @@ ftq('jon') )
<lifeless> this is pointless unless Person.displayname = 'jon' can match something that Person.fti @@ ftq('jon') cannot
<wallyworld_> that's what i'm using yes. but the ranking is different for each case above
<lifeless> the ranking is a separate discussion
<wallyworld_> not if we want the results odered correctly
<lifeless> no
<lifeless> you are doing two things
<wallyworld_> it all has to be in the query if we want sql to do the ordering
<lifeless> a) filtering on display name redundantly with fti filtering
<lifeless> b) ranking on a separate test on display name
<lifeless> a and b are entirely separate
<lifeless> I am only talking about a
<wallyworld_> ah, i think i see the problem
<wallyworld_> yes, you are correct
<wallyworld_> but the same token, the comparison to person.name is also unncessesary
<lifeless> we probably want to put the irc nickname into the fti
<lifeless> then we can can drop that from the filter
<wallyworld_> that's a separate table. does that matter?
<lifeless> yes, I agree that the name= check can be dropped too if its in the fti
<wallyworld_> what blinded me was the original query had the check for name as well as fti
<lifeless> needs a little trigger love to update the fti on changes in the ircid table
<stub> yes, the current infrastructure doesn't let us use values from separate tables. Need to replace that old stuff.
<lifeless> stub: it doesn't? I thought it was trigger based...
<stub> it is, but automatically generated and maintained by fti.py
<lifeless> ah right
<lifeless> so the glue we use is limited
<lifeless> the underlying facility is agnostic
<lifeless> ?
<stub> Yes. If we stick with tsearch2, we should hand roll the indexes and triggers in some or all cases
<lifeless> wallyworld_: I thought all persons had to have an email address ?
<stub> fti index exists on production now... doing staging and qastaging
<stub> except teams
<lifeless> ah right
<wallyworld_> yes, what he said
<wallyworld_> stub: the wife is getting anxious i'm not at dinner? you around for another hour or 2? i'll give you a new version of the query
<lifeless> teams can't have irc etc
<lifeless> I suspect a union for the team/nonteam case might be better.
<wallyworld_> lifeless: yes, hence the left join
<stub> wallyworld_: Sure
<wallyworld_> thanks
<lifeless> then you can use an inner join rather than a left join
<wallyworld_> lifeless: fair point
<lifeless> and skip the ircd check on teams
<wallyworld_> i'll look into it after food
<wallyworld_> thanks for the input. i've been looking at it too long :-(
<lifeless> no worries
<lifeless> will kibbitz more tomorrow
<lifeless> fwiw there are 10K emails matching that query
<lifeless> wallyworld_: https://pastebin.canonical.com/47949/
<lifeless> wallyworld_: I think you want a union - one for fti, one for email
<lifeless> at minimum
<lifeless> I seem to recall that the old query had that
<wgrant> lifeless: It's not clear how we want to rank.
<lifeless> sure
<lifeless> we can probably get 500 rows back (5 * union elements) and then rank & filter
<lifeless> in 300-400ms
<wgrant> And what if we get 10000 email address matches back?
<lifeless> wgrant: reread what I said
<wgrant> lifeless: How do you get 500 rows out of 10000 rows/
<wgrant> ?
<lifeless> wgrant: limit 100 in each union component
<wgrant> So rank, filter, collate, rank and filter?
<lifeless> yes
<lifeless> its not ideal
<lifeless> but triggering stupid plans isn't ideal either
<wgrant> This is true, but hopefully missing a less bad solution.
<bigjools> morning
<wgrant> bigjools: Isn't the UK not here?
<lifeless> wgrant: well, I'll take that under advisement, but its clearly possible to deliver a 500ms or so query using this approach.
<lifeless> wgrant: so thats the bar to reach for any alternate solution
<wgrant> lifeless: Sure.
<bigjools> wgrant: looking around me, it's very much business as usual (pissing it down)
<wallyworld_> lifeless: yes the original query had 4 or 5 unions but then proceded to mess up the ordering. i thought i'd see how it ran with fewer queries since it's all about the person table except for the join to email and irc. but if it's suboptimal i'll look to change the approach. i thought it was worth revisiting to see how it ran
<lifeless> wallyworld_: I trimmed the query back to just emailaddress + fti and it triggered the left merge join
<lifeless> wallyworld_: which is size(table) overhead
<wallyworld_> ouch
<lifeless> wallyworld_: individually the email address prefix search and the fti search are both 150ms
<lifeless> combined 5000
<wallyworld_> wow.
<jtv> henninge: hadn't noticed your review â a belated vielen Dank!
<bigjools> wgrant: any thoughts on my email about PCJs in the queue?
<wgrant> bigjools: "aaaa"
<bigjools> yes
<wgrant> bigjools: I think we fairly clearly have to have single-source PCJs for now.
<bigjools> yes
<wgrant> And by the time we have multi-source ones we will hopefully have dropped PackageUpload.
<bigjools> haha
<bigjools> PCJ will end up a copy of PCJ
<bigjools> err
<bigjools> PCJ will end up a copy of PU
<wgrant> Mm.
<bigjools> wgrant: also lifeless doesn't like having a suspended job state
<bigjools> which means PU will have to stay
<wgrant> Neither do I.
<wgrant> Well, something like PU will have to stay, yes.
<wgrant> I have vague plans about unifying them.
<wgrant> Into something sort of like PU, but not really.
<wgrant> But that is too invasive for now.
<wgrant> The PCJ solution will do.
<bigjools> StevenK, wgrant: anyway, what did you think about the override problem I described ?
<bigjools> we need PCJ.copypolicy and PCJ.overridepolicy I tihnk
<wgrant> bigjools: I guess the initial pre-suspension run gets and stores the overrides?
<bigjools> and the overriding done in copyTo() needs to be refactored - again
<wgrant> That is trivial.
<wgrant> It's like three lines.
<wgrant> It always needed to be changed anyway.
<wgrant> To detect newness.
<bigjools> no, the overrides are added after it suspends
<wgrant> How?
<bigjools> I've already done newness detection
<wgrant> It needs to be shown in the UI, you say.
<wgrant> So it has to be done before suspension.
<bigjools> it just uses the ancestry check again
<bigjools> manual overrides are added when it's suspended
<bigjools> auto overrides are applied as they are now, which is finew
<wgrant> 4. Default overrides. - These are currently done at the copying stage in Steve's recent changes.
<wgrant> We actually need them to apply before then, in the PCJ metadata when the job
<wgrant> suspends itself so the defaults appear in the UI.
<bigjools> correct
<bigjools> that was my point in the email :)
<wgrant> Sounds like you need to add the overrides before it suspends.
<wgrant> Then the AA adds the missing ones.
<bigjools> AAAAAAAAAAAAA
<wgrant> I'm not sure whether that's a pun or a scream of realisation.
<wgrant> Or both.
<bigjools> I realised it over the weekend when I was writing lots of code to do all this
<bigjools> wip branch: https://code.launchpad.net/~julian-edwards/launchpad/copies-must-use-queue-bug-789502/+merge/62745
<LPCIBot> Project windmill-db-devel build #342: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill-db-devel/342/
<bigjools> I've also done a change to fix overrideSource to add stuff to the metadata for PCJ
<bigjools> but that failed in ec2 with 1400 errors.  WTF
<wgrant> Hahah
<lifeless> win
<bigjools> wgrant: so you agree with me that I need to add those 2 policies as columns on the PCJ?
<wgrant> bigjools: Why?
<lifeless> bigjools: don't they apply to normal uploads too ?
<wgrant> I don't understand why the PCJ can't find them itself. At least the override policy is trivial to find from Archive.
<wgrant> Hm.
<wgrant> 19:58:28 < wgrant> I don't understand why the PCJ can't find them itself. At least the override policy is trivial to find from Archive.
<bigjools> no
<bigjools> think about sync copies
<bigjools> well, the override policy is probably ok
<bigjools> but the copy policy needs to live on the PCJ
<bigjools> since it depends on the type of copy operation
<wgrant> Argh
<wgrant> For now, yeah, probably.
<bigjools> the other option is to have a SyncPackageCopyJob
<bigjools> but.... <shudder>
<wgrant> Don't ever say that again.
<bigjools> and, I really hope my ISP is not playing up today
<wgrant> 19:57:02 < bigjools> wgrant: so you agree with me that I need to add those 2 policies as columns on the PCJ?
<wgrant> 19:58:25 -!- bigjools [~quassel@canonical/launchpad/bigjools] has quit [Quit: No Ping reply in 180 seconds.]
<henninge> If a team is an administrator for another team then all members of that first team have administrator rights on the second team, correct?
<wgrant> henninge: Yes.
<bigjools> wgrant: that's twice now
<henninge> thanks.
<wgrant> bigjools: Yes. But you spoke just over a minute before you were timed out after three minutes.
<wgrant> A little odd.
<bigjools> now, how to implement policy lookups
<bigjools> yeah odd
<wgrant> Probably named adapters for now.
<bigjools> could be a utility
<wgrant> Er, that, yes.
<wgrant> Wait.
<bigjools> but adapter fits better
<wgrant> A utility that takes names, or a set of named utilities?
<stub> wallyworld_: http://paste.ubuntu.com/614872/ is where I seem to be ending up. Inner query selects all the rows that can be retrieved quickly using indexes, outer query filters unwanted rows. ~400ms on staging with 'jon'
<stub> And the index on lower(displayname)
<wallyworld_> stub: awesome thanks. i'll look. i was just in the middle of doing my own query refactoring. it would be easier if it were pure sql but we also have to weave it into storm
<stub> wallyworld_: Aren't you supposed to be eating dinner?
<wallyworld_> stub: finished :-) doing sql now
<stub> wallyworld_: The with statement can be inlined I think without causing slowness
<stub> wallyworld_: (And does storm support WITH now? I know there were bugs open on this, maybe patches too)
<wallyworld_> stub: ah, i haven't used with before
<wallyworld_> is that standard sql?
<stub> wallyworld_: like a view or temporary table for just that query
<wallyworld_> excellent. that's nice
<stub> wallyworld_: I believe it is standard. Its the same syntax for recursive queries.
<stub> SQL-99 or later I think
<wallyworld_> right. shows my sql is a little dated
<lifeless> stub: store.with_().find()
<stub> wallyworld_: ^^
<wallyworld_> yeah. just found some examples in branchcollection
<wallyworld_> thanks
<lifeless> stub: I was saying to wallyworld that filtering on displayname is likely unneeded
<lifeless> stub: FWIW
<stub> Yer, but it is cheap with an index so what the hey :)
<stub> Just not substring - prefix or exact is fine.
<wallyworld_> stub: thanks for the help. muchly appreciated.
<stub> wallyworld_: Just remember if displayname matching stays in, we need an index: create index person__displayname__idx ON Person(lower(displayname));
<lifeless> wallyworld_: that means you'll need to land this on db-devel
<wgrant> Why?
<wgrant> Can't we create it live?
<wallyworld_> stub: ack that. we don't need the index though if i use a case statement for the rank. ie we don't need any where terms matching on name==? or displayname==?
<lifeless> mmmm
<wallyworld_> just a fti match and a case statement for the rank
<lifeless> wgrant: we probably can, with a -1 patch
<wgrant> lifeless: Exactly.
<lifeless> wgrant: can you guide wallyworld_ through that if ineeded?
<wgrant> No point db-develing it.
<lifeless> <- ETIRED
<wgrant> Sure.
<wallyworld_> btw, i was also thinking we could do the index live *if* required
<wgrant> We discussed this on the call a couple of days back.
<wgrant> Seemed like it should work OK.
<stub> wallyworld_: Correct. You only need the index if we are selecting on displayname, not using it in already selected rows.
<stub> Yes, we can create the index live.
<wallyworld_> indexes cost cpu cycles, disk space etc so if we can get away without....
<wgrant> wallyworld_: That devel landing of yours should unbreak windmill?
<wallyworld_> wgrant: some tests not all :-(
<wgrant> wallyworld_: Well, most of the picker ones should work, at least.
<wgrant> The others we can see in an hour.
<wallyworld_> yep
<wallyworld_> wgrant: afaiu, there were some onkeypress event handling issues with ff4
<wallyworld_> and windmill
<wallyworld_> hence some tests failed
<wallyworld_> but the functiionality works on in real life
<wallyworld_> /son/ok
<wgrant> bigjools:  4. Write a new SyncCopyPolicy that auto-accepts everything not NEW
<wgrant> Huh?
<wgrant> Autosyncs don't have different acceptance rolls.
<wgrant> Er.
<wgrant> rules.
<bigjools> yes they do
<wgrant> Oh?
<bigjools> see the existing Sync policy in the upload processor
<bigjools> for a mass sync you don't want it all hitting the queue
<lifeless> could someone give https://dev.launchpad.net/ArchitectureGuide/ServicesRoadmap a once-over for sanity ?
<lifeless> I haven't brought in the full list of service opportunities from the analysis yet
<wgrant> bigjools: Why not?
<wgrant> bigjools: You're not going to be mass-syncing a frozen series.
<bigjools> hmm good point
<wgrant> And you probably don't want to accidentally do it.
<wgrant> => no point having it bypass the freeze.
<bigjools> although this is what sync-source -a does right now, we just rely on them only running it at the right time
<wgrant> Yes.
<wgrant> Mass syncs are first done immediately after opening, and last done at DIF.
<wgrant> The archive does not normally freeze in the meantime.
<bigjools> we probably still need a separate policy because we don't want to send emails
<bigjools> need to add a send_email bool to them
<bigjools> StevenK: note that --^
<wgrant> bigjools: Right, a separate policy is reasonable for that, but your original reasoning had me a little surprised.
<bigjools> yeah, I was mistaken
<lifeless> stub: ^ can has your eyeballs ? :)
<stub> I'm using them...
<stub> The services road map
<stub> ?
<wallyworld_> stub: using lower(displayname) like ? picks up matches that fti misses so looks like we'll need that index
<stub> k. Do you have an example query btw?
<stub> query string I mean
<wallyworld_> ie if displayname is "Jonathan Smith"
<wallyworld_> then searching usinfg the term 'jon'
<wallyworld_> will only match using the displayname term
<wallyworld_> not the fti
<stub> Right. Because 'jon' isn't a word in that displayname.
<stub> Feel free to add that example into the doctests in case someone decides to pull it in the future :)
<wallyworld_> it's a partial word. i was hoping it would match
<stub> if you get lucky with stemming, it might.
<stub> But not in this case :)
<wgrant> I wonder if there's a name stemmer around.
<wallyworld_> since fti on person is only name,displayname - can we match it match more loosely without costing too much?
<wgrant> "more loosely" is very difficult to define :/
<wallyworld_> sure. substring match then :-)
<stub> fti searches are word searches. If you want substrings, we need a different sort of index
<lifeless> stub: yes
<wallyworld_> ah ok.
<stub> (which is possible, but not yet in LP since most thinking is we want to get this sort of search out of the db)
<wallyworld_> how does google do it?
<stub> google doesn't do substring searches
<wallyworld_> i meant searching in general
<stub> pretty much the same techniques as tsearch2, but more intelligence.
<stub> such as detecting typos etc. and much better ranking.
<wallyworld_> ok
<stub> tsearch2 doesn't do phrases though.
<lifeless> not to mention a massively parallel index implementation
<lifeless> right now we can't sanely do substring matches
<lifeless> (by yes I meant the services roadmap)
<wallyworld_> but at least we can do sensible thinks like startswith ie like which cover what most people expect when using the picker
<wallyworld_> stub: do you feel like creating the displayname index given we will be needing it?
<stub> lifeless: the in datacentre authentication stuff is assuming HTTP communication between a client and server, and that authentication will be handled by a proxy rather than the app. Might want to be explicit about this, and list this proxy feature as a infrastructure dependency.
<stub> wallyworld_: Sure. I've added it to my list.
<wallyworld_> stub: thank you.
<lifeless> stub: we already have that in place with apache though, no ?
<stub> lifeless: Which means in-datacentre http stuff will be going through both haproxy and apache, increasing latency and complexity
<lifeless> stub: somewhat yes, but lowers the requirements for each service
<stub> lifeless: sure.
<stub> lifeless: Document seems fine anyway. I'm fuzzy today though so take my proofing with a grain of salt :)
<lifeless> stub: I have revised that section
<lifeless> stub: thanks for the loan of your eyeballs
<jtv> bigjools: what should existing PackageCopyJobs have for copy_policy?
<bigjools> jtv: InsecureCopyPolicy is the default
<jtv> OK
<bigjools> I am naming it 'insecure' in my branch
<jtv> Going to be a bit nasty extracting all the source package names from the metadata.
<bigjools> jtv: but I think a default of NULL is fine
<jtv> So the string text should be 'insecure'?
<bigjools> if it's NULL we can default it in code, which makes it more flexible
<jtv> In this case I'd read NULL as "not applicable," reallyâ¦
<bigjools> jtv: also we don't have any existing copy jobs to worry about outside of dogfood, I'd be happy to blitz those, so you don't need to worry  about migrating the name
<jtv> Excellent.
<bigjools> what runes do I need to get push --stacked-on to work?  I tried a local file, lp:, bzr+ssh: all of which fail for different reasons
<wgrant> --stacked-on=/~user/project/branch might work... I forget if it was fixed.
<wgrant> A year ago it was impossible to do it cleanly.
<wgrant> You had to --stacked-on=bzr+ssh://..., but then LP would fail to parse that, so you had to change the path manually to /~user/project/branch afterwards.
<lifeless> bigjools: stacking normally occurs automatically
<bigjools> lifeless: which is great until I want to override the default
<bigjools> because pushing db-devel branches is far too painful now
<bigjools> wgrant: yes, it all seems fecked
<bigjools> my very ADSL has poor upstream :(
<LPCIBot> Project windmill-devel build #153: STILL FAILING in 1 hr 8 min: https://lpci.wedontsleep.org/job/windmill-devel/153/
<bigjools> 42MB and still pushing :/
<lifeless> I suspect its not stacking
<lifeless> thats -huge-
<lifeless> pushing a new db-devel branch with the default behaviour should be much smaller than that
<bigjools> I killed it
<bigjools> it's usually around a 14MB push
<bigjools> lifeless: so I am guessing that possibly my previous attempts to push on a stacked branch have tainted something, this bugger will no now longer push up and stack even on db-devel it seems
<bigjools>  30908kB    37kB/s \ Fetching revisions:Inserting stream:Estimate 139517/1074537
<lifeless> bigjools: yes, nuke it and start over.
<bigjools> lifeless: gah
<lifeless> from orbit if you can
<bigjools> lifeless: you mean a local nuke?  I already nuked on LP
<lifeless> I've documented what we need to do to make db-devel pushes fast ongoing
<lifeless> bigjools: no, just on lp
<lifeless> bigjools: 'delete branch' that is
<bigjools> lifeless: well it dint wurk :)
<lifeless> hmm
<lifeless> 0030 here
<lifeless> I has to go
<bigjools> I will make an entirely fresh branch
<bigjools> ah, go then :)
<lifeless> perhaps spiv or vila or someone can give you a hand ?
<lifeless> nn
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #595: FIXED in 5 hr 1 min: https://lpci.wedontsleep.org/job/db-devel/595/
<henninge> adeuring: Hi
<henninge> adeuring: Can we have the call at half past, please?
<henninge> adeuring: I'll am having lunch now.
<adeuring> henninge-lunch: ok; let's do the standup at 1330 UTC
<henninge> abentley: Hi!
<abentley> hi!
<henninge> abentley: stand up?
<abentley> henninge: sure.
<wgrant> bigjools: Before I disappear...
<wgrant> bigjools: You're not supporting separate binary overrides?
<bigjools> yarp?
<bigjools> not for syncs
<wgrant> Hmm. Interesting. OK.
<bigjools> binary syncs?  no :)
<wgrant> This isn't just syncs.
<wgrant> It's going to immediately be used for security updates too.
<wgrant> Still, should be better than it is now, as long as you make overrideBinary throw an error.
<bigjools> security updates are not going through this job code yet, and I don't intend to complicate this project by extending it ad infinitum
<wgrant> True.
<wgrant> (three copy mechanisms, yay)
<bigjools> so, I always intended it to happen later
<bigjools> yes, but we're heading in the right direction
<wgrant> Yeah.
<bigjools> and we'll have the One True Way Of Copying
<wgrant> This is something that will be able to subsume the other two, right.
<bigjools> which will also make a shedload of oopses disappear
<wgrant> And it's more easily mergable with uploads.
<wgrant> At least we are really close to being able to delete delayed copies.
<wgrant> That will all be worth it.
<bigjools> also, the override policies are a bit of a pain with regards to detecting NEW
<wgrant> Oh?
<wgrant> I think they probably need extension.
<wgrant> To return NEWness explicitly.
<bigjools> the way they're written, you can only do it with a combo of two policies
<bigjools> which is what one of them is doing by using .missing()
<wgrant> So we seed it with the origin overrides and everything NEW, override them with target overrides (unsetting NEWness where appropriate), etc.
<wgrant> Anyway, I should depart.
<bigjools> I am doing it differently now
<bigjools> good night anyway
* henninge changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs:209 - 0:[######=_]:256
<henninge> adeuring: did you see Francis' comment on your  MP?
<adeuring> henninge: not, not yet
<adeuring> henninge: I'll add the things he suggested
<henninge> adeuring: ok. Actually, I don't think I'll be able to finish your review today. Do you want me to give it back or shoudl I continue in the morning?
<adeuring> henninge: I don't mind that much. If you fancy to look at it tomoorw, that would be fine, but i can also ask tomorrow's OCR
<henninge> adeuring: ok, I'll give it back and see what the morning brings.
<LPCIBot> Project windmill-db-devel build #343: STILL FAILING in 1 hr 8 min: https://lpci.wedontsleep.org/job/windmill-db-devel/343/
<LPCIBot> Project windmill-db-devel build #344: STILL FAILING in 43 min: https://lpci.wedontsleep.org/job/windmill-db-devel/344/
<abentley> bigjools: I am exposing Archive.external_dependencies over the web service.  Should validate_external_dependencies move to the model, so that invalid dependencies cannot be set?
<bigjools> abentley: yes, there's no much choice since we don't have a browser layer for the api
<abentley> bigjools: I guess the other option would be to export it read-only, and supply a mutator that accepted structured data.
<bigjools> abentley: no, that would be a bad idea, we need to validate it or it can cause all builds to fail
<abentley> bigjools: I think a structured mutator would make it impossible to supply invalid data.
<bigjools> famous last words ;)
<bigjools> the field is a *horrible* hack anyway, don't spend too much time on it because I want to remove it in the future
<abentley> bigjools: Sure, happy to do it the easy way.
<bigjools> once the people who use it are using derived distros in LP then it will get removed
<LPCIBot> Project windmill-db-devel build #345: STILL FAILING in 41 min: https://lpci.wedontsleep.org/job/windmill-db-devel/345/
<LPCIBot> Project windmill-db-devel build #346: STILL FAILING in 41 min: https://lpci.wedontsleep.org/job/windmill-db-devel/346/
<lifeless> morning
<lifeless> flacoste: hiya
<lifeless> flacoste: can talk whenever you want.
<flacoste> lifeless: give me 10 minutes
<lifeless> cool
<lifeless> I'll pat le chÃ¢t some more
<flacoste> lifeless: ok, skype me whenever you want
<LPCIBot> Project windmill-devel build #154: STILL FAILING in 1 hr 12 min: https://lpci.wedontsleep.org/job/windmill-devel/154/
<lifeless> flacoste: lib/lp/registry/model/person.py line 1309
* lifeless changed the topic of #launchpad-dev to: Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: - | Critical bugs:209 - 0:[######=_]:256
* lifeless changed the topic of #launchpad-dev to: Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: - | Critical bugs:208 - 0:[######=_]:256
<lifeless> flacoste: launchpad-owner -
<lifeless>  48 | -owner$              | Reserved for mailinglists                                           |
<lifeless> flacoste: thats why
<flacoste> lifeless: makes sense, then launchpad-manager?
<lifeless> and done
<lifeless> launchpad-leader
<flacoste> thanks
<lifeless> you can change if you like
<flacoste> even better
<flacoste> i prefer to be a leader than a manager :-)
<lifeless> launchpad-leader launchpad-architect launchpad-strategist
<lifeless> all setup
<flacoste> awesome
<lifeless> mailing -dev now
<lifeless> the owner of launchpad-leader needs tweaking I guess
<lifeless> and done
<lifeless> flacoste: I've made you owner of -leader
<lifeless> flacoste: you can hand it off to ~canonical-cdo-manager if you want :)
<flacoste> :-)
<lifeless> however https://launchpad.net/~launchpad-security/+members can be sorted out now \o/
<flacoste> lifeless: done
<lifeless> flacoste: not much point having jml & I as regular members ;)
<lifeless> flacoste: we get that via ~launchpad
<lifeless> Launchpad strategist2 minutes ago âApproved
<lifeless> ahha! now done :)
<lifeless> and now I can fix up the description :)
<flacoste> :-)
<flacoste> yep, all done
<flacoste> and i did ~launchpad also
<lifeless> cool
<lifeless> I've changed the description to 'The contact point for security issues in the Launchpad software (https://launchpad.net/launchpad-project).'
<lifeless> a little more open and a little less confusing IMNSHO
<lifeless> flacoste: wow, we have 56 engineers !
<flacoste> i wish!
<lifeless> beuno and rockstar still there
<lifeless> and tim
<flacoste> and salgado and mwh
<lifeless> I don't know if it matters
<lifeless> the big thing for me was -security fixup
<flacoste> does it matter, the trouble is, i don't know either
<lifeless> now thats done, I'm going to go workup some fake numbers for test performance :)
<flacoste> mwh and salgado do review branches from time to time
<flacoste> but it might not matter
<lifeless> I was wondering
<lifeless> why does -security go to a mailing list ?
<flacoste> archiving
<flacoste> i think
<flacoste> the same reason launchpad-bugs was in a way
<lifeless> so we need those lists to accept any From: ?
<lifeless> I wonder if removing the list would have any negative effect
<lifeless> folk would get the mails by default (because -security gets a subscription to security bugs)
<Ursinha> anyone here knows what dates launchpad use to search for modified_since tasks (lp.searchTasks)?
<lifeless> the one on bug I think
<cody-somerville> lifeless, -security is also the bug supervisor
<cody-somerville> lifeless, so we'd start getting all bug e-mails
<lifeless>  date_last_updated      | timestamp without time zone | not null default timezone('UTC'::text, now())
<lifeless> cody-somerville: no, supervisor does not imply subscription
<cody-somerville> when did that change?
<lifeless> a while back - years.
<lifeless> it surprised me too :)
<Ursinha> thanks lifeless
<flacoste> really?
<lifeless> flacoste: cody-somerville: just checked
<lifeless> you will laugh
<lifeless> if there is *no* supervisor, the owner gets a pseudo structural subscription.
<lifeless> If there is a supervisor, they do not get mailed.
<cody-somerville> that can't be right. I just got an e-mail for a bug because of being indirectly a member of the team thats the bug supervisor
<lifeless> lib/lp/bugs/model/bug.py line 2067
<lifeless> cody-somerville: that team may have a structural subscription
<lifeless> whats the team ?
<cody-somerville> launchpadlib-developers
<cody-somerville> either way, launchpadlib-developers don't show up on the bug as subscribed but I got notified which is confusing in its self
<lifeless> cody-somerville: https://launchpad.net/~launchpadlib-developers/+structural-subscriptions
<lifeless> so supervisor *stops* mail to owner and delegates setting some fields
<lifeless> separately the supervisor may have a structural subscription
<cody-somerville> something must have created subscriptions for all the bug supervisors
<cody-somerville> btw, whats the difference between staging and qastaging?
<lifeless> staging runs the proposed next schema
<lifeless> qastaging runs the live schema
<lifeless> we qa *all* our deploys on qastaging
<lifeless> we check db patch times on staging
<lifeless> there are some other minor differences but they are historical not intended: as and when we can we will eliminate them
<cody-somerville> lifeless, I just tested. I created a project on qastaging and set the bug supervisor and it created a (structural?) subscription.
<lifeless> flacoste: https://launchpad.net/~launchpadlib-developers is owned by leonardr
<lifeless> cody-somerville: yes, thats to keep the old behaviour
<lifeless> cody-somerville: you then delete the structsub
<lifeless> and you're set
<cody-somerville> lifeless, if private bugs are enabled by default, I assume the bug supervisor still gets directly subscribed?
<cody-somerville> *private bugs by default is enabled
<flacoste> lifeless: let's ask a losa to change ownership to launchpad-leader
<lifeless> flacoste: I can't delete the launchpadlib-developers subscriptions atm. Should we re-own the team ? https://launchpad.net/~launchpadlib-developers/+members
<lifeless> cody-somerville: I suspect so.
<lifeless> cody-somerville: at least until the disclosure project gets structural-private-stuff-done
<flacoste> lifeless: only a losa can change ownership, maybe ~launchpad should own the team
<lifeless> flacoste: in the meantime, you're an admin - can you unsubscribe on https://launchpad.net/wadllib/+subscribe and https://launchpad.net/launchpadlib/+subscribe
<lifeless> flacoste: ~launchpad works for me
<flacoste> lifeless: done
<flacoste> lifeless: if you can follow-up with the losa for the reown, i'd appreciate
<lifeless> ok
<lifeless> cody-somerville: I just realised my reply, meant to be a little cheeky could be read as snarky... i didn't mean it that way
<lifeless> flacoste: elmo helped out. Tis done.
<bigjools> "flacoste deactivated by lifeless" - best email subject evar
<elmo> hahaha
<flacoste> indeed, i lol when i saw it
 * lifeless aims to please
<flacoste> bigjools: do we have any soyuz components facility or it's SQL-maintained?
<flacoste> for example, can we select which components are allowed within a distroseries
<bigjools> flacoste: SQL, and yes in ComponentSelection
<bigjools> although trying to change it might be futile since there's a lot of hard-coded crap :/
<lifeless> bigjools: we want to add a component main to principia
<bigjools> should be fine
<bigjools> there's bits and bobs that expect universe to be there as well though
<lifeless> in soyuz ?
<bigjools> yes
<lifeless> or more registryish code?
<flacoste> what happens if there are no components selection
<flacoste> ?
<bigjools> I think it only affects uploading
<lifeless> the former won't matter as we're not doing builds - nothing beyond pushing to branches
<flacoste> well
<lifeless> s/won't/shouldn't/
<lifeless> :)
<flacoste> i was going to reuse Archive.checkUpload for the official package branch permission
<flacoste> and that's where i encountered components :-)
<flacoste> seems like the parameter is required
<lifeless> yes
<bigjools> yes, because there's lots of ways of getting permission to upload
<bigjools> component being one of them
<flacoste> actually
<flacoste> seems like component can be None
<flacoste>         if (component is not None
<flacoste>             and strict_component
<flacoste>             and not self.checkArchivePermission(person, component)):
<flacoste>             return NoRightsForComponent(component)
<flacoste>         return None
<flacoste> so not caring about component here might just work
<flacoste> for principia at least
<flacoste> for Ubuntu... what would component=None means
<bigjools> yeah if you don't care about component don't specify ut
<bigjools> it will check the other permissions; per-package and packageset
<flacoste> bigjools: would using component=None work in the context of bug 365098
<_mup_> Bug #365098: Anyone who can write to a sourcepackage should be able to set the official package branch <lp-code> <package-branches> <Launchpad itself:Triaged> < https://launchpad.net/bugs/365098 >
<bigjools> flacoste: if it's per-package, should work yeah
<flacoste> or per-package set
<bigjools> yep
<flacoste> from the code, i understand that the person must have permission to one component at least to get to the end of that method
<bigjools> if it got that far and if you don't have any component permissions it'll fail you there
#launchpad-dev 2011-05-31
<wgrant> abentley: https://code.launchpad.net/~abentley/launchpad/duplicate-msgid/+merge/62697 seems a bit heavy-handed.
<wgrant> abentley: I can understand skipping duplicate messages to the same place... but not to all of Launchpad.
<wgrant> abentley: (that includes mailing lists, so you can no longer send an email both to a bug and a mailing list)
<lifeless> huh
<lifeless> the preview diff is empty :(
<wgrant> r13139
<lifeless> hmm
<lifeless> I agree
<lifeless> duplicate message ids are normal across the system
<lifeless> only abnormal in one context - same list, or same bug etc
<wgrant> Exactly.
<wgrant> Shall I roll it back?
<lifeless> its 7:30 pm for Aaron
<lifeless> yes, I think it needs rollback
<lifeless> we can't deploy with it
<lifeless> but
<lifeless> lets check just a little more
<wgrant> Sure.
<lifeless> so the error encountered is a duplicate msgid on one bug
<lifeless> when the notification for it is being raised
<wgrant> Well, adding the duplicate message results in a DB exception.
<lifeless> bug 595166
<_mup_> Bug #595166: IntegrityError raised filing a bug using the email interface <easy> <email> <lp-bugs> <oops> <Launchpad itself:In Progress by abentley> < https://launchpad.net/bugs/595166 >
<wgrant> That's the one.
<lifeless> its raised when *notifying* tha the second message was added
<wgrant> Only because that's when the flush occurs.
<wgrant> Oh.
<lifeless> no, *really* when notifying
<wgrant> That's not the OOPS I"ve seen before.
<wgrant> The one I saw violated bugmessage__bug__message__key
<lifeless> thats on a link to message
<lifeless> not to two messages with the same msgid
<wgrant> Hmm?
<lifeless> OTOH so is the notification one
<wgrant> Two emails with same msgid and same content => one Message row
<lifeless> sure
<lifeless> this check isn't complete though
<lifeless> which is the other issue
<lifeless> aieee
<lifeless> other broken code
<lifeless> for email_addr in addresses:
<wgrant> Oh?
<lifeless> discards all but the last handler found
<wgrant> Heh
<lifeless> see the end of handle_one_mail
<lifeless> looks like using cc: can make our mail handling do some interesting and odd stuff
<lifeless> anyhoo
<lifeless> *blinkers*
<wgrant> Why would it be using To or Cc?
<wgrant> It should be using X-Launchpad-To
<lifeless> shoo. Go read.
<wgrant> Which is the envelope sender.
<lifeless> thou shalt be blind. And don't say I didn't warn thee.
<wgrant> So, yes, it uses X-Launchpad-To unless it's not present.
<wgrant> Not that bad.
<lifeless> well
<lifeless> we've some funky in our mail pipeline and we haven't identified it
<wgrant> Well, we fixed some of it two weeks ago.
<lifeless> ok please rollback
<lifeless> I have written up a hopefully helpful explanation for abentley
<lifeless> if you've other or better suggestions for how to tackle it, please say so
<wgrant> lifeless: Done, thanks/
<lifeless> thank you
<LPCIBot> Project windmill-db-devel build #347: STILL FAILING in 1 hr 5 min: https://lpci.wedontsleep.org/job/windmill-db-devel/347/
<wgrant> lifeless: The security hole is this: using a crafted or predicted msgid an attacker can cause Launchpad to generate a link to a mail that was sent to a private bug or mailing list when they have no such permission.
<wgrant> lifeless: I thought so too.
<wgrant> But it's not the case.
<wgrant> I investigated when I looked at trying to fix this bug a few weeks back.
<wgrant> It grabs all the Messages with the right Message-ID, and compares the content.
<wgrant> You can possibly obtain attachments if you know all the textual content, I guess.
<lifeless> wgrant: fromMessage is fine
<lifeless> wgrant: aarons patch wasn't
<wgrant> lifeless: I don't see a new security bug. The returned message isn't used for anything. It just returns None to exit the handler, right?
<lifeless> wgrant: if the patch were changed to call the handlers using the result of messageset.get(rfc822id), it would open a security hole.
<wgrant> lifeless: Yes.
<lifeless> (not to mention still crashing in the same way)
<lifeless> wgrant: thats what I meant.
<lifeless> the usage of messageset.get(rfc822id) is not sufficient to determine 'same email' nor 'can see the email'
<wgrant> Ah, it helps if I read your entire reply.
<lifeless> *snort*
<wgrant> It was long, and my initial skim revealed a statement that I knew to be false!
<lifeless> feel free to reply and editorialise
<LPCIBot> Project windmill-devel build #155: STILL FAILING in 1 hr 10 min: https://lpci.wedontsleep.org/job/windmill-devel/155/
<poolie> hi all
<lifeless> hi poolie
<lifeless> poolie: random question for you
<lifeless> if the lp bzr+ssh/sftp codehosting project were a separate tree, do you think your team would do anything interesting with it ?
<spiv> lifeless: hmm.  Occasionally users ask about how to run a bzr server with some access control and/or authentication that isn't easily satisfied by using authorized_keys directives and filesystem permissions
<spiv> lifeless: so I guess having a less intimidating tree to extract that from, or at least learn from, might be helpful to someone.
<poolie> hi lifeless
<lifeless> spiv: tim has extracted the code but adapted it to a uni hosting solution
<poolie> well
<lifeless> so one of the candidate services - and I think a low hanging fruit - is one that still talks xmlrpc
<poolie> you know there was john's forking lp server project
<poolie> that would probably have needed to touch only that code
<poolie> and landing it has been pretty hard
<poolie> partly just because of bugs that appeared under heavy load; and those would have needed to be solved anyhow
<lifeless> right
<poolie> but, i think it would certainly have made it easier
<lifeless> I was going to say
<poolie> for instance easier to run up an ec2 instance and hammer it
<lifeless> poolie: really?
<lifeless> 'ec2 demo'; wait; hammer it.
<poolie> hm
<poolie> i wonder if jam tried that
<poolie> perhaps not actually any easier
<lifeless> spivs comment is more interesting :)
<poolie> anyhow i guess there are two questions here
<lifeless> the friction-to-change-in-lp aspect is well covered I think
<poolie> :-P
<lifeless> I'm curious about other interesting things
<poolie> i don't think this would really directly satisfy those users
<lifeless> like - adopt it for bzr's on server (make some of the policies pluggable, for instance)
<poolie> it might be a step towards having a common codebase
<lifeless> *own*
<poolie> hm, there is an _apparent_ contradiction here with wanting to make codebrowse more folded into launcphad
<lifeless> how so ?
<lifeless> actually
<poolie> at any rate there is a kind of precedent there for having one program that is used both by lp and externally
<lifeless> contractors here
<lifeless> I will be back in a bit
<lifeless> ah no
<lifeless> false alarum
<poolie> a few dimensions:
<poolie> - would this make it easier to make changes on codehosting in lp.net
<poolie> - are we actually likely to want to make any changes to it
<poolie> - would it make it easier to reuse externally
<lifeless> so cool interesting things is actually what I wanted to ask about :)
<lifeless> you could say no to all your three things there and still do something cool or interesting with it
<lifeless> I merely mean to encourage out-of-box stuff
<poolie> it could well do
<poolie> hm
<poolie> could well do
<poolie> i may be inside the box but it seems like they would still be gated by needing to actually get run somewhere
<poolie> either deployed on lp or run by other people
<lifeless> sure
<lifeless> perhaps saying to to /all 3/ isn't plausible :)
<poolie> at the moment i feel those 3 dimensions all get a weak yes at the moment
<poolie> i'm trying to think of things that would make sense to do in there but not in standalone bzr serve
<poolie> perhaps structured audit logs or notifications?
<lifeless> it implements ssh
<lifeless> so user management
<lifeless> multiplexing onto one account
<poolie> mm
<poolie> oh, easy deployment on windows?
<poolie> fsvo easy :) assuming you have to install twisted and everything
<lifeless> brb
<poolie> it would be an interesting case where you'd like to provide a fake/simple implementation of the other side of the services it uses
<poolie> the registry type things read by xmlrpc
<lifeless> yes
<lifeless> wow
<lifeless> only 2 failures
<wgrant> lifeless: On?
<lifeless> mailed you for your edification
<wgrant> Ah, that bug.
<lifeless> hmm
<lifeless> person.join(team) and team.addMember are asymmetric
<lifeless> I suspect this will cause issues
<lifeless> however
<lifeless> I shall close my eyes and go bravely forward.
<lifeless> (specifically team.join doesn't honour admin or owner rights)
<lifeless> it prefilters, wrongly
<lifeless> hah
<lifeless> it goes deeper though
<lifeless> getDirectAdministrators includes the owner
<lifeless> I'll leave that for now
<wgrant> The bug says that.
<lifeless> bah so it does
<lifeless> actually
<lifeless> I'm going to exclude that
<lifeless> there is a (reasonable) argument that owner implies admin
<lifeless> (or that our admin stuff is not granular enough)
<lifeless> mmm, but then it might be confusing.
<lifeless> it would be odd for an owner to have to join the team to rename it or change its metadata
<lifeless> so - excluding admin for now
<lifeless> why does the sample data have someone called 'dumper' ? ><
<wgrant> Same reason as mark is there, I guess.
<wgrant> Renamed to avoid pings.
<lifeless> ah
<lifeless> wgrant: do you have any ideas about the first failure there ?
<lifeless> its got me a little flummoxed
<lifeless> mark has a row for membership in ~admins
<lifeless> hah
<lifeless> I wonder if we have naffed sample data
<lifeless>     mark.inTeam(getUtility(ILaunchpadCelebrities).admin)
<lifeless> False
<LPCIBot> Project windmill-db-devel build #348: STILL FAILING in 1 hr 5 min: https://lpci.wedontsleep.org/job/windmill-db-devel/348/
<wgrant> lifeless: Hah. Probably.
<wgrant> lifeless: There's a script for that.
<lifeless> its more subtle than that
<lifeless> can_edit_team is returning False
<lifeless> also its checking the way-bong side of the equation but thats a different discussion
<wgrant> Hm, TP seems consistent.
<wgrant> :(
<lifeless> eah
<lifeless> s/^/y
<lifeless> ok its true at the start
<lifeless> something in this test script corrupts it
<lifeless> bsearch time
<lifeless> and looks like we manually expire mark from admins
<wgrant> Yay
<lifeless> line 612
<lifeless> this test was only working because mark owns ~admins in the sample data
<wgrant> Awesome.
<lifeless> which is what I'm changing
<lifeless> ok now its just weird
<lifeless> admin_person *is* an admin
<lifeless> but its still throwing a security proxy error
<wgrant> inTeam cache busted?
<lifeless> I tried disabling that
<lifeless> its not reach inTeam
<lifeless> so a cached security assertion or something
<wallyworld_> lifeless: is there a pattern we use to bulk load (eager fetch) fields of type CollectionField? rather than just single object references
<lifeless> yes, see lp.services.database.bulk.load_referenced
<wallyworld_> thanks
<wallyworld_> lifeless: you mean load_referencing? i'm using that and it's not preventing db hits
<wgrant> You need to make it a cachedproperty.
<lifeless> wallyworld_: that gets the data, it won't seed it for traversals.
<wallyworld_> bugger
<wallyworld_> it's not worht making it a cachedproperty since the field is only referenced once for each object
<wallyworld_> i guess there's no way to bulk load *and* seed the traversals?
<lifeless> why do you say its not worth it ?
<lifeless> also which types are we talking about ?
<lifeless> wgrant: >< its still using 'mark' for some -bizarre- reason
 * lifeless tries to figure it out
<wallyworld_> lifeless: there's a list of 6 people. we display irc nick. we only have to access the irc nic collection field once per rendering per person
<wallyworld_> but that does one query per person
<lifeless> right, you want a cached property.
<lifeless> totally appropriate
<lifeless> see the other use of load_referencing
<wallyworld_> in this case, the irc nic is only defined as a class field of type SQLMultipleJoin. so i will need to introduce a new property?
<lifeless> yes
<lifeless> you need to move the join attribute to a private attribute
<wallyworld_> ok
<lifeless> add a cached property with the same name as the current join attribute
<lifeless> seed it appropriately and deal with fallout
<wgrant> Do you need the join attribute?
<lifeless> well, perhaps not.
<wgrant> Just Stormify it.
<wgrant> In the cachedproperty.
<wgrant> Return a list of a Stormified query.
<lifeless> wgrant: Reference is storm :P
<wgrant> Less SQLObject, better for the world :)
<wgrant> SQLMultipleJoin is not.
<lifeless> true
<lifeless> but stormify != remove join attribute
<lifeless> anyhow
<lifeless> wtf
<lifeless> pasting you a snippet
<wgrant> k, I am intrigued.
<wallyworld_> also - Person.preferredemail is a cachedproperty. but it hits the database once per person cause it does a query inside the property getter. and no luck so far seeding the data
<wgrant> lifeless: Hm? I meant create a Stormified query.
<wgrant> Not Stormify the class.
<wgrant> wallyworld_: Which name are you seeding?
<lifeless> wgrant: yes, you could also just use a storm Reference directly.
<wgrant> ReferenceSet, possibly, yes.
<wgrant> Hmm.
<wgrant> How is admin_person set there?
<wallyworld_> perhaps i'm not doing the seeding correctly
<wallyworld_> bulk.load_referencing(EmailAddress, persons, ['personID'])
<lifeless> wgrant: >>> admin_person = personset.getByEmail(ADMIN_EMAIL)
<lifeless> wallyworld_: use PersonSet.getPrecachedPersonsByIds
<wallyworld_> it does the correct sql from what i can see, but....
<lifeless> wallyworld_: and listify its output
<wallyworld_> right. will take a look. thanks
<lifeless> wallyworld_: *always* use that when dealing with persons
<lifeless> wallyworld_: also change the vocab to just return the Person ids
<wgrant> lifeless: Shouldn't that always be name16?
<lifeless> wgrant: yes.
<lifeless> wgrant: and it is!
<wgrant> So why is the test saying it should be something else?
<lifeless> exactly!
<wallyworld_> lifeless: so this will need to be done across the board for all person related vocabs
<lifeless> wallyworld_: one query at a time
<wgrant> Welcome to massive layering violations :)
<wallyworld_> np, just wondering
<wallyworld_> i guess the vocab stuff predated the getPrecachedPersons... stuff
<lifeless> yes
<lifeless> until 11 months ago we liked to requery everything
<wallyworld_> right.
<wallyworld_> context = makes more sense
<lifeless> can has review? https://code.launchpad.net/~lifeless/launchpad/bug-227494/+merge/62943
<wgrant> lifeless: Have you checked the teams that this will affect?
<wgrant> They were mostly Ubuntu teams when I last looked.
<lifeless> ~admins :)
<lifeless> no I haven't
<poolie> lifeless: putting a commit message on my restfulclient mp didn't cause it to land
<lifeless> 1300 rows
<lifeless> poolie: hmm, need to ask someone that knows. benji or gary come to mind.
<lifeless> poolie: unless README says something useful
<wgrant> "cause it to land"?
<wgrant> Landings don't happen automatically.
<poolie> i guess what i really want to ask is: how is the trunk of restfulclient maintained
<poolie> by pqm, tarmac, manually, etc
<poolie> i could mail them
<lifeless> wgrant: some subprojects use tarmac
<wgrant> lazr.restful is manual.
<wgrant> Not sure about restfulclient.
<wgrant> lifeless: Do they?
<poolie> i'll subscribe benji
<poolie> thanks
<wgrant> lifeless: person.merged IS NOT NULL
<wgrant> Er, IS NULL
<wgrant> Cuts it down to 500ish.
<lifeless> wgrant: 471
<lifeless> wgrant: already done that
<lifeless> wgrant: some are truely noddy - https://launchpad.net/~ubuntu-jewish
<wgrant> lifeless: You're calling *that* noddy?
<wgrant> There are far worse :)
<wgrant> Far, far worse.
<lifeless> wgrant: I am - it has 1 proposed member, and the owner has left the team.
<lifeless> wgrant: I wasn't saying anything about the name or intent of the team.
<wgrant> I know.
<wgrant> But there are the teams which invite every team.
<wgrant> And have no members.
<lifeless> yes
<lifeless> affected by this ?
<wgrant> Like the one that showed up last week.
<wgrant> Invited dozens of teams.
<wgrant> Owner had left it.
<wgrant> Awesome.
<lifeless> win
<lifeless> anyhow
<lifeless> so, I think we want to do this
<wgrant> And then some of the invited teams had bad admin settings, so randoms approved them, and the real admins couldn't leave :(
<wgrant> Yes.
<wgrant> I think so.
<lifeless> I will happily blog before it goes live
<lifeless> its arguably exploitable today
<lifeless> (find something that checks one way but not the other and leverage it)
<wgrant> Yeah
<lifeless> poolie: it should still be approved ;)
<lifeless> poolie: I just meant ask benji about landing mechanisms
<poolie> ffs "ConfigParser.NoOptionError: No option 'consumer_key' in section: '1'" _again_
<poolie> do you ever see this?
<poolie> should we subscribe some lp developer team to loggerhead reviews?
<poolie> at the moment they seem to go by default to.. beuno and ~bzr or something
<lifeless> loggerhead-reviewers is the reviewer team
<lifeless> https://launchpad.net/~loggerhead-reviewers/+members
<lifeless> so they already are
<poolie> oh, ok
<poolie> i must have implicitly claimed it
<poolie> nm then
<LPCIBot> Project windmill-db-devel build #349: STILL FAILING in 1 hr 6 min: https://lpci.wedontsleep.org/job/windmill-db-devel/349/
<lifeless> wgrant: follow up on https://code.launchpad.net/~lifeless/launchpad/bug-227494/+merge/62943 ?
<wgrant> lifeless: You are arbitrarily changing the user to an admin half-way through because the owner has no privs.
<wgrant> That may be invalidating subsequent tests.
<wgrant> Even if it isn't, it's rather confusing to change it partially globally when only that one block needs it.
<lifeless> wgrant: have you looked through that doctest ?
<lifeless> hint: it does this a lot already
<wgrant> It's a doctest. No.
<lifeless> and the previous user was one it thought was an admin.
<wgrant> Ah, I see.
<wgrant> OK.
<wgrant> An admin, but not an Admin.
<wgrant> Not an ~admin, sorry.
<lifeless> yes, a ~admin
<wgrant> How did it stop being an admin?
<lifeless> because it was mark
<wgrant> Because it was the owner of ~admins?
<lifeless> yes
<wgrant> But the test removed that membership?
<lifeless> yes
<wgrant> ahhhhhhhh
<wgrant> Carry on, then.
<lifeless> it was BROKEN
<wgrant> Preferably beating that test to death on the way.
<lifeless> now I just need a reviewer.
<lifeless> Perhaps we should graduate you now
<wgrant> Hi StevenK.
<wgrant> Heh
<wgrant> Possibly.
<lifeless> I have been waiting for you to consistently look below the surface on non-soyuz code
<lifeless> which you have been improving on
<lifeless> I don't recall finding things you missed recently other than the extraordinarily messy gpghandler stuff, which confused everyone.
<lifeless> so, I think graduation is in order
<wgrant> Yeah, the gpghandler example is bad, but I don't think anyone non-Twisted and non-you would have caught that either.'
<wgrant> Since most people don't have much idea of Twisted :(
<lifeless> facepalm
 * lifeless sobs
<wgrant> ?
<wgrant> The other test?
<spm> wgrant: are you being mean to lifeless again???
<lifeless> https://code.launchpad.net/~xaav/loggerhead/export-tarball/+merge/62941
<lifeless> copy and paste from bzrlib
<lifeless> with mild tweaks
<wgrant> Heh
<wgrant> lifeless: So, am I graduating?
<mrevell> Good morning
<lifeless> wgrant: yes, I am mailing -dev
<wgrant> lifeless: Thanks!
<StevenK> wgrant: Grats!
<wgrant>         return dumps({
<wgrant>             'row_id': 'tasksummary%s' % self.context.id,
<wgrant>             'bugtask_path': '/'.join(
<wgrant>                 [''] + canonical_url(self.context).split('/')[3:]),
<stub> lifeless: BugSummary.viewed_by is painful because of indirect subscriptions.
<lifeless> stub: they don't grant visibility
<lifeless> stub: so shouldn't be counted and don't matter
<stub> So only direct subscribers can view private bugs? That should be fine then.
<lifeless> yes
<lifeless> where direct means 'in the team that is in the bugsubscription'
<stub> Do bug supervisors get bugsubscription records created when a bug becomes private?
<lifeless> yes
<lifeless> all indirect subs do (thats a bug)
<stub> Maybe it is now a feature ;)
<lifeless> well
<lifeless> the creation is fine
<lifeless> the fact that all random 'happen to have a structural subscription' get subscribed is the bug
<lifeless> wgrant: so, is that +1 on the branch then ?
<wgrant> lifeless: I rereviewed a couple of minutes ago.
<wgrant> So, yes.
<lifeless> the mail just came in
<adeuring> good morning
<lifeless> stub: are our fti indices bloated again ?
<lifeless> 94 /   42  Person:EntryResource:searchTasks
<lifeless>       70 /   24  Distribution:EntryResource:searchTasks
<lifeless>       45 /   12  Distribution:+bugs
<lifeless> top timeouts
<stub> lifeless: back at 70% bloat, yeah.
<stub> bug_fti I mean
<lifeless> ><
 * lifeless stabs bug heat
<nigelb> Wait, so canonical employees don't automatically become reviewers? Wow, that's interesting
<StevenK> nigelb: It's a little more complicated than that, but not much.
<StevenK> But no, they don't.
<stub> lifeless: it is bloating between 1% and 2.5% per day looks like. It should be trivial to confirm this is bug heat calculation - just turn that script off for a day or two.
<nigelb> StevenK: But how open is the process to outsiders, like me, for instance
<lifeless> nigelb: quite open
* gmb changed the topic of #launchpad-dev to: Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: gmb | Critical bugs:208 - 0:[######=_]:256
<lifeless> nigelb: the one thing you can't do is get to the point of approving other non-staff changes or actually landing changes
<gmb> allenap: Any objections to me marking https://code.launchpad.net/~allenap/launchpad/rabbit-fixture-cookie-file/+merge/62709 as Work In Progress to make it go away from my todo list?
<lifeless> nigelb: we require that all changes have *a* staff member involved.
<lifeless> nigelb: because of the way things deploy and the risks around data exposure
<nigelb> Ah, that explains that yes.
<allenap> gmb: None, done.
<gmb> allenap: Ta.
<nigelb> lifeless: I expect there's a lot of private data with the hidden emails which would be illegal for a non-staff member to access.
<gmb> poolie: Similarly, do you want https://code.launchpad.net/~mbp/launchpad/mail-scope/+merge/60281 reviewed now or should it wait until you've fixed the test failures?
<lifeless> bigjools: :( - bug 790535
<bigjools> is that the bug where mup won't tell me what it is? :)
<lifeless> Distribution:+ppas timeouts
<lifeless> https://bugs.launchpad.net/launchpad/+bug/790535
<poolie> gmb more review now would be good
<poolie> i doubt if the failure is deep
<gmb> poolie: Righto, I'll take a sken presently.
<StevenK> gmb: 'sken' ?
<gmb> StevenK: "Look".
<poolie> reading inference 101
<poolie> ls
<StevenK> gmb: I figured as such, I was just wondering about its entomology.
<gmb> StevenK: It has nothing to do with insects.
<gmb> :)
<StevenK> Oh, arse.
<gmb> But it's Lancs / Yorks (possibly other northern places too) dialectic slang, AFAICT.
<gmb> I grew up with it, so I've no idea where it came from.
<gmb> StevenK: But if it helps, The Beatles probably said it. (Boom boom).
<gmb> It's only 9:41am and I'm being punny. What else will the day hold?
<nigelb> This channel wins for the most interesting conversations EVAR.
<gmb> I suspect that the barrier to entry for that competition is very, very low.
<nigelb> No, I'm in #ubuntu-offtopic, so its very high
<nigelb> I still need to fix a bug I introduced to LP, sigh.
<dpm> hi danilos, morning. Do you happen to remember where rosetta/deleted-templates (the location where we used to put templates we wanted out of the way) went after the unification of all lp applications into a single launchpad project? I already asked you a while ago and you found out, but I can't remember where it was and I can't find it in my logs
<dpm> or if anyone else can help ^
<danilos> dpm, probably the 'null' project
<danilos> dpm, also, there's old-lp-translations that's just a deactivated and renamed 'rosetta' project, so that's what has deleted-templates stuff as well
<dpm> thanks danilos, I'll try both
<dpm> danilos, old-lp-translations/deleted-templates worked, thanks
<danilos> yw
<LPCIBot> Project windmill-devel build #156: STILL FAILING in 1 hr 8 min: https://lpci.wedontsleep.org/job/windmill-devel/156/
<gmb> poolie: r=me
<poolie> thanks gmb
<poolie> gmb: do you think you could pilot-in james's patch  https://code.launchpad.net/~james-w/launchpad/more-matchers/+merge/32057
<gmb> poolie: Sure thing.
<poolie> and perhaps some of the loggerhead patches
<poolie> james's should be trivial
<poolie> hooray :)
<gmb> poolie: I've never landed a loggerhead patch before; anything special I need to know?
<gmb> Or is it PQM all the way down?
<poolie> i don't think there is a pqm
<poolie> imbw
<poolie> i think it's just direct commits
<poolie> the hard thing will be getting to know the code enough to review it :/
<poolie> oh, but jam is online, and he knows about it
<gmb> Ah, good. THat might be safer.
<poolie> but if you drive them that would be good
<poolie> some review or landing is better than none
<gmb> poolie: Sure. I'll take a look at them today.
<poolie> legend
<lifeless> night all
<LPCIBot> Project windmill-devel build #157: STILL FAILING in 1 hr 8 min: https://lpci.wedontsleep.org/job/windmill-devel/157/
<henninge> Hi gmb! Could you have a look at this mp of mine, please? ;-)
<henninge> https://code.launchpad.net/~henninge/launchpad/bug-778129-person-picker/+merge/62894
<gmb> henninge: Sure thing.
<gmb> henninge: r=me with comments
<henninge> gmb: thanks
<henninge> gmb: I copied the indention from another test case in the same file. I can fix both.
<gmb> henninge: Brilliant, thanks.
<henninge> gmb: is !=== also to be used when comparing to literal numbers?
<gmb> Oo.
<gmb> henninge: Good question. In that case, leave it.
<gmb> I'm not sure ( and if it works, don't fix it)
<gmb> But if you want to find out, that's fine by me :)
<gmb> Like I said, it's not worth blocking on.
<henninge> although Crockford acts like there is only === and !===, so it will probably just work.
 * henninge tries out.
<gmb> jam: Do you know what the situation is with https://code.launchpad.net/~malept/loggerhead/standalone-auth/+merge/18828?
<jam> not offhand
<gmb> Okay.
<gmb> I'll add a comment and put it to work in progress then to make it leave the review queue.
<deryck> henninge, ping for standup
<henninge> deryck: sorry
<deryck> henninge, http://pastebin.ubuntu.com/615334/
<LPCIBot> Project windmill-db-devel build #350: STILL FAILING in 1 hr 6 min: https://lpci.wedontsleep.org/job/windmill-db-devel/350/
 * gmb -> short break
<flacoste> wgrant: congratulations for your reviewerhood!
<wgrant> flacoste: Thanks!
<nigelb> oooh, yeah. I saw that mail when I woke up.
<nigelb> Congrats wgrant!
<bigjools> damn, now we have to start being nice to him
<jcsackett> clear
<jtv> gmb: got a free slot on the review queue?  https://code.launchpad.net/~jtv/launchpad/db-bug-790161/+merge/63002
<gmb> jtv: Sure thing.
<jtv> Splenders.
<gmb> Tip-top.
<gmb> jtv: r=me
<jtv> thanks!
<gmb> np
<sinzui> gmb: is it too late for you to  review https://code.launchpad.net/~sinzui/launchpad/display-name-with-id-0/+merge/63006
<gmb> sinzui: Nope. I'll look presently.
 * deryck goes offline for lunch, back soon
<gmb> sinzui: r=me
* gmb changed the topic of #launchpad-dev to: Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: - | Critical bugs:208 - 0:[######=_]:256
<sinzui> thanks gmb
<jcsackett> exit
<bigjools> You have running jobs
<sinzui> jcsackett: do you have time to talk?
<LPCIBot> Project windmill-devel build #158: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill-devel/158/
<LPCIBot> Project db-devel build #601: FAILURE in 4 hr 40 min: https://lpci.wedontsleep.org/job/db-devel/601/
<jcsackett> sinzui: i am available when you are.
<sinzui> jcsackett: I am available to talk now
<jcsackett> sinzui: cool. mumble or sip?
<sinzui> I am on mumble now
 * jcsackett goes to fire up mumble.
<LPCIBot> Project windmill-db-devel build #351: STILL FAILING in 1 hr 4 min: https://lpci.wedontsleep.org/job/windmill-db-devel/351/
<flacoste> wtf
<flacoste> what is this not_exported = TextLine(title="Not exported")
<flacoste> ?
<flacoste> in interfaces declaration in sourcepackage.py
<flacoste> ok
<flacoste> disregard that wtf
<flacoste> i was in a test file, not in sourcepacakge.py
<lifeless> moin
<LPCIBot> Project windmill-db-devel build #352: STILL FAILING in 41 min: https://lpci.wedontsleep.org/job/windmill-db-devel/352/
<abentley> lifeless: can we chat about bug #595166?
<_mup_> Bug #595166: IntegrityError raised filing a bug using the email interface <bad-commit-13139> <easy> <email> <lp-bugs> <oops> <qa-bad> <Launchpad itself:Fix Committed by abentley> < https://launchpad.net/bugs/595166 >
<lifeless> abentley: sure thing
<lifeless> abentley: just give me 5 minutes to sort some cats out
<abentley> lifeless: sure.
<lifeless> ok they seem happy for now
<lifeless> would you like voice ?
<abentley> lifeless: yes, please.  Skype, I imagine?
<lifeless> yeah. .nz.win ;)
<lifeless> interestingly the canonical voip service works well
<lifeless> just mumble hates freedom :)
<abentley> lifeless: btw, I would be curious to try canonical voip.  I recently set it up on my phone.
<lifeless> abentley: nice
<lifeless> abentley: lets do that next time
<flacoste> gary_poster: i'm testing object's permission, I would use check_permission, but I think you had another suggestion
<flacoste> check_permission coming from canonical.launchpad.webapp.authorization
<gary_poster> flacoste, I generally try to use canWrite and canRead (might have forgotten names) from zope.security
<gary_poster> on call
<flacoste> ack
<lifeless> walk the dinosaur?
<flacoste> lifeless: ?
<lifeless> flacoste: ;) referencing a song
<abentley> lifeless: care to do the review? https://code.launchpad.net/~abentley/launchpad/no-dedup/+merge/63044
<lifeless> abentley: sure, done.
<abentley> lifeless: thanks.
<lifeless> I editorialised a little for future reference
<lifeless> I wonder if our mail handling doctests have something about duplication in them?
<abentley> lifeless: could be.  test_messages didnt.
<abentley> lifeless: my new toy "fault line" didn't find strong correlations between changes to messages.py and changes to particular test files.
<lifeless> abentley: does it consider mainline commits only ?
<lifeless> abentley: I can imagine feature branches not having a high association but the mainline diffs having one
<abentley> lifeless: no, it only looks at revisions that modified the file.  I"d like to change that, but I don't know how to do it efficiently.
<lifeless> you need the 'first mainline rev after the rev that modified the file'
<abentley> lifeless: Yes.
<lifeless> this is retrievable from merge_sorted revisions
<abentley> lifeless: isn't that a whole-graph operation?
<lifeless> abentley: yes, 4 seconds for me on lp
<lifeless> abentley: you'd do it once per run though, giving you an efficient data structure to answer the question for all the modified revs
<abentley> lifeless: Acceptable, given the whole operation takes ~20 seconds for me.
<abentley> lifeless: message.txt does fail.  Thanks for the tip.
<lifeless> no worries
<lifeless> sinzui: hi
<lifeless> sinzui: got a few minutes?
<sinzui> I do
<lifeless> I had two questions for you
<lifeless> one simple, one a little complex (perhaps needs voice)
<lifeless> the simple one is with person/fmt:link-name-id
<lifeless> do we still want fmt:link at all ?
<lifeless> I was thinking we were going to change it everywhere
<lifeless> which would be a simpler patch - just change the existing formatter.
<sinzui> yes. It is correct for most cases such as tables and summaries
<lifeless> That confuses me a little but ok.
<sinzui> We have a specialised formatter to communicate the Lp id, but we never fully implemented
<sinzui> I intend to complete the implementation
<lifeless> the other question is about private public team membership interactions
<lifeless> if you can spare some time to talk about it - doesn't need to be now - I'd like to tease the issues apart. I suspect we agree.
<sinzui> lifeless: my sip is 7642
<LPCIBot> Project windmill-db-devel build #353: STILL FAILING in 1 hr 5 min: https://lpci.wedontsleep.org/job/windmill-db-devel/353/
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #602: FIXED in 4 hr 40 min: https://lpci.wedontsleep.org/job/db-devel/602/
<lifeless> sinzui: I hope I captured things well in bug 405277
<_mup_> Bug #405277: Private teams are not able to join other teams (public or private) <disclosure> <lp-registry> <Launchpad itself:Triaged> < https://launchpad.net/bugs/405277 >
<lifeless> sinzui: it seems like 3 fairly contained changes to me.
<sinzui> lifeless: this looks very good. Thank you/
<lifeless> oh, I forgot the bit about 'and tell them the implications'
<lifeless> adding that in
#launchpad-dev 2011-06-01
<sinzui> huwshimi: mumble?
<huwshimi> sinzui: Sure
<lifeless> sinzui: added; please give another once-over. Thanks!
<lifeless> hmm
<lifeless> how does the lp wiki get wikinames these days?
<lifeless> SSO or LP ?
 * sinzui wants to know that answer
<sinzui> we can close about 8 bugs id LP is not involved
<lifeless> hloeung: hi
<hloeung> lifeless: Hi
<lifeless> we're wondering how dev.launchpad.net gets its wikinames for users logging in via openid
<lifeless> is it via the SSO service
<lifeless> or via LP
<wgrant> lifeless: Wikinames? Doesn't it just use usernames?
<wgrant> It's not like the old days of the Ubuntu wiki.
<lifeless> well, consult - https://dev.launchpad.net/UserPreferences
<wgrant> Hm?
<lifeless> if someone does a timestamp
<lifeless> it uses their wikiname
<lifeless> but there is no ui to set it
<wgrant> Hmm, true.
<lifeless> so presumbably its coming from openid or *something*
<NCommander> wgrant: so I'm trying to poke dominator to handle my arch_any_all skew test case, and I'm really concerned about performance. To determine if a skew has happened, we need to query each SPH for a given binary package, and check to see what has published where. That's not going to be DB friendly ...
<wgrant> But it gets it through SSO, which doesn't expose wikinames.
<wgrant> Maybe it just uses displayname.
<lifeless> we're about to delete wikinames from lp
<lifeless> if that table is replicated to sso
<wgrant> It's not.
<lifeless> I can imagine Bad Things Happening
<wgrant> \d lp_<TAB>
<lifeless> NCommander: indeed. That would be fail.
<NCommander> Right
<wgrant> NCommander: What is the algorithm you are using?
<wgrant> We do far worse queries in the dominator now.
<NCommander> wgrant: I haven't written one yet
<NCommander> wgrant: well, for each BPPH, I need the SPH to grab the binary packages it generates, and then checks the dependencies to see if they would still be satisifiable if a package was marked SUPERSEDED
<wgrant> NCommander: I don't think you can sensibly parse dependencies.
<wgrant> NCommander: that is going to be terribly slow.
<lifeless> if they are modelled in the db we can do it
<lifeless> recursive query
<cody-somerville> wgrant, why does it have to be slow?
<lifeless> cody-somerville: death by a thousand cuts
<NCommander> Ideally, I want LP to make sure the archive is in an installable state
<lifeless> NCommander: you might like to look at the conflictchecker schema
<cody-somerville> NCommander, There are tools that exist that do just that.
<NCommander> cody-somerville: your not catching the point
<lifeless> NCommander: it handles in-db dependency queries to answer some very similar questions
<NCommander> The archive should NEVER be uninstallable as it disrupts image building on ports architectures
<lifeless> FWIW I agree
<NCommander> lifeless: right, but conflictchecker isn't run once an hour like the publisher :-)
<wgrant> NCommander: The publisher is run every 5 minutes.
 * NCommander coughs
<lifeless> NCommander: no, but it can handle a single package change in typically 50-100ms
<wgrant> The primary archive publisher is a special case.
<NCommander> lifeless: the dominator does the entire archive at once. Not on a per package basis. Given we have 17-20k binary packages, that's going to add up
<NCommander> fast
<wgrant> NCommander: Not quite.
<lifeless> NCommander: you don't need to recheck things that aren't changing.
<wgrant> NCommander: I think they will still transition from Published->Superseded as normal.
<lifeless> NCommander: you need to check the implication of things that *are* changing.
<wgrant> NCommander: But Superseded will be included in indices, until removal is scheduled.
<wgrant> NCommander: judgeSuperseded will implement this logic.
<NCommander> wgrant: as it stands, shouldn't it be PUBLISHED if it should be in the index?
<wgrant> NCommander: It examines Superseded but uncondemned publications. It needs to implement this extension.
<wgrant> NCommander: Yes, but we are not talking about as it stands :)
<NCommander> At the risk of making a lot of pain for myself, but it might be saner to have a new PUBLISHED_SUPERSEDED status
<wgrant> Why?
<NCommander> instead of making SUPERSEDED now be inconsistant w.r.t. to something publishing status
<wgrant> All tools can now handle multiple versions in the indices.
<wgrant> Since Debian does it.
<NCommander> yes, but if I look at all SUPERSEDED packages, I no longer have an idea what may or may not be held in archive.
<NCommander> er, in the indices
<NCommander> Ideally, PUBLISHED_SUPERSEDED should also include a reason on why it hasn't left the incidies
<wgrant> The idea is that everything on disk will be in the indices.
<lifeless> stuff in superseded is on disk, right ?
<NCommander> So then SUPERSEDED should simply always be in the indicies.
<NCommander> until it reaches its execution time.
<wgrant> lifeless: Partly.
<wgrant> lifeless: status = SUPERSEDED AND dateremoved IS NOT NULL is on disk.
<wgrant> GRAR
<wgrant> s/NOT //
<NCommander> wgrant: won't that become DELETED on the next publisher run?
<wgrant> NCommander: No.
<wgrant> Superseded/Deleted/Obsolete are all final states.
<wgrant> Superseded == package was superseded
<wgrant> Deleted == package was explicitly deleted
<wgrant> Obsolete == series went obsolete.
<NCommander> Ah
<NCommander> So as I read domination, the only thing that tells LP is something is SUPERSEDED is if it pops multiple versions of a given package
<NCommander> That check needs to be made smarter
<lifeless> huh, no
<wgrant> Why?
<NCommander> bah
<NCommander> sorry
<NCommander> my brain seized
<NCommander> so if I'm following you, the 'correct' solution is to have everything that is SUPERSEDED end up in the indicies until its execution time, and then modify deathrow to be smarter about what packages it brings death to.
<NCommander> am I in the right ballpark?
 * wallyworld grumpy. no coffee yet so far today. must rectify
<wgrant> NCommander: Until its *condemnation* time.
<wgrant> Removing it at the time of execution would leave stuff 404ing a lot.
<wgrant> We have no maintenance people around right now, do we?
<wgrant> lifeless: ^^?
<NCommander> Well, then the most straightforard thing to start off with is implementing that SUPERSEDED packages stay in indicies and fixing whatever breaks from that, then make deathrow smarter
<lifeless> wgrant: in this tz atm ? I think not.
<wgrant> lifeless: :(
<lifeless> maybe brad or someone.
<wgrant> lifeless: There are unlanded cowboys :(
<lifeless> they can be reapplied in a pinch
<lifeless> which sucks butwhatare yougoingtodo
<wgrant> Or we just exclude germanium this time.
<lifeless> right
 * NCommander makes a new LP branch for changing how SUPERSEDED is handled
<wgrant> NCommander: We probably want to talk to Julian before you start on anything like this.
<NCommander> wgrant: fair enough. What time zone is he in?
<wgrant> NCommander: He's UK
<wgrant> So BST.
<wgrant> Yay, I can self-review without lifeless killing me now.
<NCommander> right, so I will see him in ~8 hours
<wgrant> NCommander: I hope not.
<NCommander> wgrant: ?, that would be at the start of his day
<lifeless> wgrant: I would never kill you :)
<wgrant> NCommander: Yes, but the middle of your night.
<wgrant> Hmm.
<wgrant> Is my connection dodgy, or is LP sending empty responses?
<NCommander> wgrant: well, that's hat caffiene is for, but hopefully he will be up when I get up (and that your timezone overlaps somewhere in there)
<wgrant> Overlap between the three of us is just about impossible.
<wgrant> Maybe late my evening.
<wgrant> Hm.
<wgrant> ShipIt is inactive, but the bugs are still open?
<wgrant> sinzui: Can we delete mailing-list-experts?
<wgrant> sinzui: It's not a celebrity any more, and its only related artifact is the deleted mailing-list-beta-testers
<wallyworld> wgrant: you have some changes to lazr-js for the picker entry rendering? i need to hack in the same area - i want to extract out the picker entry rendering to a method that can be overridden. if you are done i can merge in your branch before i start hacking
<wallyworld> or i can take just your lazr-js changes for the affiliation rendering work and go with that
<wgrant> wallyworld: Heh, no, the lazr-js changes are terrible hacks that need to be rewritten. And they're tiny. So you do your stuff, and I'll rework on top of that when you're done.
<wallyworld> ok. thanks
<LPCIBot> Project windmill-devel build #159: STILL FAILING in 1 hr 6 min: https://lpci.wedontsleep.org/job/windmill-devel/159/
<timrc> StevenK, hey... where are you guys pulling python-testtools from?  I'm not seeing 0.9.11-r189 on pypi (at least sudo easy_install "testtools>0.9.10" is not finding it) and I don't see it getting built in Ubuntu, either
<lifeless> its not released yet I don't think
<lifeless> __version__ = (0, 9, 11, 'dev', 0)
<lifeless> timrc: thats a snapshot
<lifeless> timrc: its in the lp download cache
<timrc> lifeless, ah... rocketfuel-get is my friend
<jtv> hi folks
<wgrant> Morning jtv.
<lifeless> u
<lifeless> hi jtv
<jtv> hi
<jtv> what does the "u" stand for?
<lifeless> jtv: bigjools suggested we have a call w.r.t. the persona discussion - I'd be delighted to do that sometime today if you want.
<lifeless> jtv: 'u' - up in gmail.
<jtv> ewin?
<lifeless> yup
<jtv> :)
<jtv> Julian also suggested that we chat about another problem, but since I haven't had my coffee yet I can't even speak coherently about it
<lifeless> well there is no rush
<lifeless> I'm still in my workday for at least another 1.5 hours, and probably significantly more
<jtv> Knowing you, you'll probably throw in another 17 of overtime.
<lifeless> yeah
<lifeless> I just won't feel guilty about wandering off during those hours :)
<jtv> Good!
<jtv> Well, I've made a note of the time and will definitely get that coffee first.
<wgrant> lifeless: Good plan to try GPGHandler first.
<jtv> lifeless: the coffee is kicking in.  Yes, I'd be delighted to have a chat about the persona thing if you still have time!
<lifeless> sure thing
<jtv> lifeless: mumble?
<wgrant> lifeless hates freedom.
<lifeless> jtv: do you have canonical voip?
<lifeless> jtv: failing that skype ?
<lifeless> mumble to here is frankly terrible
<jtv> I don't have the voip thing, noâ¦ let me start skype.
<StevenK> wgrant: O hai. Where does one get testr?
<spiv> StevenK: from lp:testrepository, or packaged in https://launchpad.net/~testing-cabal/+archive/archive
<wgrant> StevenK: apt:testrepository
<wgrant> Fail, gnome-terminal doesn't link that.
<wgrant> apt://testrepository
<wgrant> Still fail.
<wgrant> Anyway, it's in the primary archive.
<StevenK> spiv, wgrant: Ta
<StevenK> Is anyone available to review?
<wgrant> StevenK: What's up?
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/notify-announce-list/+merge/63067
<wgrant> You've probably had enough of my reviews lately, though.
<StevenK> Haha
<wgrant> Aha.
<StevenK> This is a hopefully 1: short, 2: safe
<wgrant> Yeah.
<wgrant> It's not the branch I thought it was.
<StevenK> No, Julian subverted me.
<wgrant> StevenK: You've not cleaned up archiveuploader. Your point is invalid.
<wgrant> You can wipe announcelist out of archiveuploader entirely, AFAICT.
<StevenK> wgrant: I had not done so because I wasn't sure how much test fallout there would be.
<wgrant> Test fallout is better than tech debt in the most tech debty part of the tech debt that is our codebase.
<StevenK> wgrant: Let me wave the axe around more.
<wgrant> Thankyou sir.
 * StevenK tries to remember where else he saw it
<wgrant> lib/lp/archiveuploader/uploadpolicy.py should be all that's left.
<wgrant> Except maybe tests.
<wgrant> But there probably aren't tests.
<wgrant> Because this is archiveuploader.
<wgrant> Ah, no, uploadpolicy.txt tests it vaguely.
<StevenK> I've just cleaned that up by removing the *ugly* property
<wgrant> Yup.
<StevenK> However, I can recall something that took announce_list as an argument
<wgrant> acceptFromQueue?
<wgrant> notify?
<StevenK> lib/lp/soyuz/browser/queue.py perhaps
 * StevenK looks
<wgrant> A whole lot of other stuff which was sick and wrong?
<StevenK> Let me see if the test you mentioned is broken first
<StevenK> I changed the file already, so finding it is a simple matter of grep
<LPCIBot> Project windmill-devel build #160: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill-devel/160/
<StevenK> wgrant: It was soyuz/scripts/queue.py
<wgrant> :(
<StevenK> I've cleaned it up too
<wgrant> StevenK: Done.
<StevenK> wgrant: Is http://pastebin.ubuntu.com/615643/ better?
<wgrant> Worse.
<wgrant> That linewrap makes me sad.
<wgrant> There's a reason I did it the other way :)
<StevenK> Sure, but distroseries.changeslist is longer :-)
<wgrant> If you must have the BACKPORTS line wrap, put it on a separate line and indent it.
<wgrant> Complex conditions want to be readable :(
<StevenK> wgrant: http://pastebin.ubuntu.com/615644/
<wgrant> StevenK: At that point you might as well just move the whole thing on to a new line.
<wgrant> No parentheses, and it's clearer.
<wgrant> But that is better :)
<StevenK> wgrant: http://pastebin.ubuntu.com/615645/
<wgrant> StevenK: Looks great, thanks.
<StevenK> Excellent. I've also eradicated announce list from queue.py and its tests, so I'll toss it at ec2.
<wgrant> Perfect.
<StevenK> wgrant: Also, not arguing about my changes? What, are you sick? :-P
<wgrant> This is a largely unoffensive cleanup :)
<StevenK> Tis not large enough for my liking
<StevenK> wgrant: Oh, was that your first code review?
<StevenK> (As opposed to code*)
<wgrant> Well, apart from the one yesterday where lifeless graduated me so he didn't have to find a mentor, yes.
<StevenK> Haha
 * StevenK ponders a bug vs --no-qa
<wgrant> no-qa
<wgrant> lifeless: I see the notification on the blog... no issues with deploying the inTeam fix?
<lifeless> wgrant: no issues
<LPCIBot> Project windmill-db-devel build #354: STILL FAILING in 1 hr 5 min: https://lpci.wedontsleep.org/job/windmill-db-devel/354/
<wgrant> lifeless: Thanks.
<wgrant> Yay
<wgrant> I think person search works OK when karma is added.
<wgrant> wallyworld: Do you have a newer query than the one you gave to lifeless yesterday>
<wgrant> Hmm.
<wgrant> Except team search.
<wgrant> This sort of completely breaks that.
<wgrant> Fail.
<StevenK> wgrant: Is that wallyworld's branch, or something else?
<wallyworld> wgrant: i've included one in the mp: https://code.launchpad.net/~wallyworld/launchpad/person-picker-results-order/+merge/63057
<wgrant> StevenK: I have added karma to his query.
<wgrant> And it works pretty well.
<wgrant> IRC nicks and email addresses are awkward, though :(
<wgrant> Since all we have is non-discriminatory prefix match.
<wgrant> They tend to dominate.
<wgrant> Karma mostly sorts that out.
<wgrant> But something longer-term would be nice.
<jtv> I've had quite enough of this bouncing.  Let's see if it gets better if I take wifi out of the loop.
<jtv> stub: I'm hoping to get a db review & patch number out of you for https://code.launchpad.net/~jtv/launchpad/db-bug-790161/+merge/63002
<lifeless> wgrant: aggregate karma across the team ?
<wgrant> lifeless: That would work for ranking people and teams separately.
<wgrant> lifeless: But ranking them sensibly in a single listing borders on impossible.
<lifeless> average the karma across the team
<wgrant> That might actually work.
<adeuring> good morning
<mrevell> Hello
<adeuring> hi mrevell
<stub> jtv: k
<jtv> à¸à¸­à¸
<lifeless> stub: evening
<stub> jtv: Why is copy_policy free form test rather than something like an enum?
<stub> lifeless: yo
<jtv> stub: otp
<stub> (yeah you know me)
<danilos> anyone knows if there's an easy way to fetch multiple persons (given by their self_links) using API without doing multiple round-trips to the server?
<wgrant> You can't.
<danilos> or, how would one best export a tuple over the API for use in JS (I want to get a tuple (BugSubscription, Person, Person [subscribed_by]) from an API call so I don't have to do multiple round-trips, but I'd also feel that such API wouldn't really be the best publicly facing thing one might have; or maybe it would be)
<danilos> wgrant: "yay"
<danilos> I am guessing I'll have to define a separate interface holding this information, because I want a collection of such tuples; or, is there a way to force "shallow" (1-level) snapshotting of particular attributes on an already exported interface?
<lifeless> danilos: you're probably best off defining a custom view which can return the json youneed
<lifeless> our API is hugely focused on types and URL mapping, sadly
<danilos> lifeless, is that the practice we encourage? I kind of dislike that approach
<lifeless> danilos: to date folk generally throw their hands up and say 'man this is too hard'
<danilos> lifeless, right, it's probably the simplest thing but the bug page is doing a bunch of that stuff and I'd rather do away with it (well, I'd still be able to get rid of 3-4 portlets, and introduce 1 new view, so it'd be an improvement, but I was hoping for 0)
<lifeless> danilos: I recall a page somewhere that exists just to give json to browser js scripts
<danilos> lifeless, yeah, there's a few of those on the bug page, and some of them render the actual HTML that's loaded through AJAX (yuck)
<lifeless> danilos: the main API can do that too
<lifeless> danilos: I agree that its yuck. It makes the service proposal I have have some fugly bits
<lifeless> danilos: but all that said, I think if you have something pretty use case specific, putting it in the public API would be a little ugly
<danilos> lifeless, right; still, useful to know that this is the approach I should take
<danilos> lifeless, agreed, but if I need this (i.e. "get all subscriptions with subscriber Person fetched at the same time"), I am pretty sure public users might need it too
<danilos> lifeless, but, then again it'd be opening a whole new can of worms, and I'd rather not go there
<lifeless> right
<lifeless> there is a broad issue with the API and performance
<danilos> yup
<lifeless> launchpadlib cannot use point solutions
<lifeless> it needs some (perhaps easy) infrastructure and a defined pattern for the solutions to fit into
<danilos> ok, thanks; I'll go with a view-provided JSON and load that on-demand, and I'll let you worry about coming up with nice infrastructure for this :P
<lifeless> There is a document about doing a V2 API by leonardr/benji/gary on the wiki
<lifeless> cool
<wallyworld> stub: hi. i was hoping you could give me a db patch number for the person.displayname index creation script. it's included in mp https://code.launchpad.net/~wallyworld/launchpad/person-picker-results-order/+merge/63057
<stub> wallyworld: I landed a patch to add the person.displayname index already
<stub> patch-2208-60-1.sql
<jtv> stub: back.  And that was opp, not otp.
<wallyworld> stub: ok thanks. i thought you'd create the index but i needed to do the ptach still as part of my mp. i'll remove it.
<jtv> stub: the boss specified a plain text field for the policy name because it's a bit easier to follow than an enum field.
<stub> wallyworld: no probs. I tend to create the patches that I apply live, since the live version is usually very similar to what needs to land and I want to minimize chances of administrative screwups
<bigjools> the policies are named rather than enummed :)
<stub> bigjools: enums are names stored as integers
<wgrant> But the names are crap :(
<bigjools> SIGH
<bigjools> nice constructive criticism there wgrant
<wgrant> 'insecure' doesn't make sense outside the uploader.
<wgrant> That is my only objection :)
<stub> bigjools: Is there a fixed set of names, or are they freeform input of some sort?
<wgrant> Well, it doesn't really make sense *in* the uploader either, but that's what we have.
 * bigjools lets jtv continue
<jtv> stub: fixed set, for matching policy classes.
<stub> So what is the advantage of a text field over an enum for that? We know the advantages of enum over text field which is why we normally use them.
<jtv> Mostly that it takes us from a mapping between policies and names to a mapping between policies, names, and numbers.
<jtv> Not lethal, but inconvenient.
<bigjools> I am not wedded to names, switch it if you want
<stub> The enum infrastructure means it is a mapping between policies and enums, not between policies, names and numbers.
 * stub would love to use pg builtin enums, but not suitable for agile development until PG 9.1
<jtv> Well the enum provides a mapping between names and numbers; we'd add the mapping between names and policies ourselves.
<stub> I don't follow. You reference enums as names, not as numbers using some external mapping repeating what the enum infrastructure gives you for free.
<jtv> Apart from getting ints in the database, of course.
<stub> job.copy_policy = CopyPolicy.ZAPIT
<stub> Right. And you don't see then in the Python code.
<jtv> Well, apart from the one place of course.
<jtv> Not trying to argue, really; just describing the inconvenience.
<stub> There is small inconvenience creating the enum definition, but advantages to using it (smaller tables, no toast, faster lookups, smaller indexes, typo catching, faster python code). So unless there is a good reason, we should switch that column to an integer and use the standard pattern.
<jtv> Toast may be a bit of an exaggeration for this case.  :)
<jtv> But sure, I can switch it.
<stub> I think the toast table gets created when the main table gets created
<jtv> Oh well
<stub> I can list more rationalizations for enums too if you like :)
<jtv> No that's alright, thank you :)
<jtv> stub: updated the patch.
<stub> jtv: Ta. Does allowing NULL still make sense ('not set' vs. explicit enum value)?
<stub> I expect so as not all jobs will use the field (?)
<jtv> stub: that may depend a bit on where the class hierarchy goes, but I guess it would make sense for now to forbid it.
<stub> if it makes sense, make it not null. Or leave it as nullable if we are not sure yet.
<jtv> I'll make it "not null."  If we ever decide that's wrong, we can relax it.  Easier than going the other way.
<stub> jtv: Unless we are mid cycle :)
 * stub isn't helping
<jtv> We'll just wake up an admin in the middle of the night to drop the constraint.
<stub> lifeless: did you see https://code.launchpad.net/~stub/launchpad/bug-758587-tests/+merge/63012 ?
<bigjools> make it default to the current "insecure" policy please jtv
<stub> jtv: Did you find that cpu indicator yet?
<jtv> bigjools: it seems to make more sense to do that in the code, not in the db.
<bigjools> jtv: either is fine
<jtv> At the DB level I think neglecting to provide a value should produce either a null or an error, even if we do require a value for PlainPackageCopyJob.
<jtv> stub: no, I don't have it yet
<LPCIBot> Project devel build #771: FAILURE in 4 hr 52 min: https://lpci.wedontsleep.org/job/devel/771/
<jtv> rvba: reviewed your branch
<rvba> jtv: thanks a lot!
<jtv> np
<rvba> I'll reply to your comments once I've finished my chicken :).
<bac> hello mrevell
<wgrant> Hm.
<wgrant> That is a /lot/ of failurse.
<jelmer> wgrant: it's so bad, it's even affecting your spelling :P
<wgrant> That's more the freezingness :(
<jelmer> wgrant: of your machine or of your fingers ?
<jelmer> hi matsubara
<wgrant> The latter.
<matsubara> hi jelmer
<jelmer> matsubara: I made some more progress on updating bzr - current work is at lp:~jelmer/launchpad/newer-bzr
<matsubara> jelmer, cool. should I merge your branch into mine and run tests to see if the errors are fixed?
<matsubara> oh, you merged my branch :-)
<jelmer> matsubara: it's too early for that, I'm still chasing some errors in bzr-git
<matsubara> jelmer, ok, thanks for letting me know.
<mrevell> Hey bac#
<deryck> Hi, adeuring, abentley, and henninge.  You guys have the standup yet?
<abentley> deryck: no, waiting for you.
<henninge> deryck: we are chatting but waiting for youi
<henninge> yui?
<deryck> ok, cool.  thanks.  coming....
<henninge> yiu?
<henninge> :-D
<deryck> if mumble will ever connect
<cjohnston> henninge: mornin
<abentley> deryck: do you want to Skype instead?
<deryck> maybe so.  let me try one more restart.
<henninge> cjohnston: Hi!
<henninge> cjohnston: otp now
<cjohnston> ok
<cjohnston> mine still need to be worked on?
<henninge> cjohnston: the test run failed because of other errors. I just sent a mail to launchpad-dev.
<henninge> cjohnston: the branch should be fine now, I expect
<henninge> but don't know for sure
<gmb> Is anyone else seeing "AssertionError: Name "id-only" is not registered as a view or navigation step for "Person" on "mainsite"." OOPSes on launchpad.dev instances? It happens on pretty much every page.
<cjohnston> henninge: ok.. If you get an answer, let me know please :-)
<gary_poster> matsubara or Ursinha, will either of you have some time for some exporatory testing tomorrow?  And then we'll actually need some more probably Thursday of next week too
<matsubara> gary_poster, yep
<gary_poster> *exploratory :-)
<matsubara> gary_poster, can you give me more details?
<gary_poster> cool, thanks matsubara, I'll send you details the start of my day tomorrow.
<gary_poster> Overview: the better bug notification feature had two bugs that meant rewriting large chunks of UI.  We have one in db-devel now, and one will be ready for a no-downtime deploy after next Wed.'s downtime deploy.
<gary_poster> Tomorrow, we need to make sure that the intermediate changes we have are OK for a few days in production; then next Thursday we need to make sure that the final changes are OK.  Since they change a lot of code, and touch on a lot of the issues that were raised in the feature review, I thought exploratory testing would be a good idea.
<gary_poster> Moreover, particularly for the intermediate changes, I know that we had hoped to get the final changes in before the db-deploy, so we were not as careful as we usually are, so another pair of eyes would be appreciated.
<gary_poster> (done)
<matsubara> gary_poster, sure thing. I'll wait for the details tomorrow and let you know once I have done the testing
<gary_poster> thanks matsubara!
<benji> I don't know one way or the other.  Let me take a look real quick.
<gary_poster> benji, you and your channels :-P
<benji> so many channels, so few windows
<gary_poster> :-)
<jcsackett> StevenK: you still around?
<StevenK> jcsackett: Barely
<jcsackett> StevenK: fair. i posted a screenshot on the MP you requested one on. but if it's too late for you to take a look at it i can throw it to someone else.
<StevenK> jcsackett: Link me the MP again?
<jcsackett> https://code.launchpad.net/~jcsackett/launchpad/oh-oh-pick-me-pick-me/+merge/63047
<jcsackett> screenshot is in my reply to your comment, StevenK.
<StevenK> jcsackett: Okay, I shall Approve. Do you feel it needs a UI review now or when it starts being used?
<jcsackett> StevenK: i intended to get UI when it gets bolted in. until then it's too much of a pain for a UI reviewer to play with.
<StevenK> jcsackett: Sounds fine. r=me
<jcsackett> thanks, StevenK.
<deryck> henninge, shall we chat now?
<henninge> ser
<henninge> deryck: yes, just getting ready
<deryck> mumble is acting up again
* abentley changed the topic of #launchpad-dev to: Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: abentley | Critical bugs:208 - 0:[######=_]:256
<timrc> If a ppa is made public after its been made private do discard the buildd_secret as well?
<timrc> do we ^
<wgrant> timrc: You can't change the privacy of a PPA after it has been used.
<wgrant> Oh, I guess you're working on the API export?
<timrc> wgrant, right, but before it's been used you can... and so in that case, do we discard the  buildd_secret
<timrc> wgrant, right...
<wgrant> timrc: Yes, you need to discard it.
<timrc> wgrant, cool, thx
<wgrant> The DB doesn't enforce it, but the secret would become public.
<LPCIBot> Project windmill-devel build #161: STILL FAILING in 1 hr 8 min: https://lpci.wedontsleep.org/job/windmill-devel/161/
<adeuring> abentley: fancy a small review (100 lines)? https://code.launchpad.net/~adeuring/launchpad/bug-750274/+merge/63129
<abentley> adeuring: sure.
<abentley> adeuring: r=me
<adeuring> abentley: thanks!
<gmb> sinzui: I'm looking at https://bugs.launchpad.net/launchpad/+bug/61531 at the moment and I could do with knowing whether it's something that the disclosure work is going to deal with.
<_mup_> Bug #61531: Forbidden error when trying to mark a bug as private <403> <lp-bugs> <oops> <privacy> <ui> <Launchpad itself:In Progress by gmb> < https://launchpad.net/bugs/61531 >
<sinzui> gmb: it is not a priority since we are focused on non-subscription mechanisms to provide access
<gmb> sinzui: How far away is that work from being viable? I.e. is it worth me fixing the bug anyway just to make an OOPS go away or should I mark it low (or Invalid or whatever).
<sinzui> gmb: Is this a case where ajax works and html does not?
<gmb> sinzui: No, the problem is that either of them breaks. It's just that the HTML version breaks more obviously.
<gmb> Because you can mark the bug private but then you can't see it. With AJAX it's not immediately apparent there's a problem until you refresh the page.
<sinzui> Since the work is not scheduled, never. We will introduce pillar permissions in 2 months in conjunction with allowing owners to see who is subscribed to what...
<gmb> Okay.
<sinzui> but there are no requests from stakeholders to change how subscription to individual bugs and branches works.
<gmb> Ah, right.
<gmb> In that case, I'll go fix it.
<gmb> sinzui: Thanks for clearing that up for me.
<LPCIBot> Project windmill-devel build #162: STILL FAILING in 44 min: https://lpci.wedontsleep.org/job/windmill-devel/162/
<sinzui> gmb: I think this bug is really about Lp's privacy entitlement rules. We simply do not permit bugs to be private without also being security issues unless an admin has toggled the projects private bug rules.
<gmb> Hmm. I'm not sure that I understand what you mean. Are you saying that we shouldn't allow private bugs unless $SOME_RULE_FLAG is set to True?
<sinzui> gmb: lp bugs conflates privacy and security. They are not equal since the issues are seen by different users. At this time, I think no user can toggle a bug private unless the project is configured to have private bugs. The any user can see and change security though
<cody-somerville> sinzui, I don't think thats accurate.
<sinzui> It will be
<gmb> sinzui: This is the work that you're going to be landing in 2 months, right?
<sinzui> We are separating security from privacy. We have always known Lp did not do a proper implementation
<gmb> s/landing/releasing
<sinzui> gmb yes
<gmb> Right.
<gmb> sinzui: In that case, I'll still fix the bug since the fix is cheap and it kills an OOPS.
<sinzui> gmb When I change the bug supervisor, we will not suffer the subscription insanity. Though I can assign a team I am not a member of to the security contact, thus not have access to security bugs, I can change the role, and get access back
<sinzui> gmb yep
<gmb> Here's to not suffering insanity.
<cody-somerville> lol
<cody-somerville> I read that as sufficient instead of suffering
<LPCIBot> Project windmill-devel build #163: STILL FAILING in 44 min: https://lpci.wedontsleep.org/job/windmill-devel/163/
<jkakar> lifeless: FYI, I remember talking to smagoun at SomeHands last year and he mentioned he'd worked on/built a big system made out of many services connected by message queues.  You might want to ping him to get his input about your ideas for Launchpad.
<lifeless> jkakar: thanks
<jkakar> np
<sinzui> lifeless: This is an example oops I have seen when we attempt to store that exceeds MAX_EMAIL_SIZE:
<sinzui> InvalidEmailMessage: Msg <...> size 13947097 exceeds limit 10485760
<sinzui> see lib/services/messages/model/message.py
<lifeless> yes
<lifeless> thats a large incoming mail ?
<lifeless> we shouldn't record that as an OOPS, its out of our control. Hmm, but the mail path should match that.
<lifeless> I see your point.
<lifeless> even with that set we'd still have trouble.
<sinzui> In a round about way mailman > holds > lp store it for review
<lifeless> sinzui: do mailman list messages go through this code path ?
<lifeless> s/that set/incoming mail size limited in postfix/
<sinzui> lifeless: only the hold path. and mailman now knows not to send the entire message
<sinzui> it also knows to only forward the text part as well
<lifeless> ok
<sinzui> So Lp does not oops no
<sinzui> now
<lifeless> so for lists.launchpad.net we allow bigger
<sinzui> we *could*
<lifeless> but the postfix setup anything for @launchpad.net should limit to 10485760
<lifeless> because our code path enforces that anyway
<sinzui> I believe it is currently configured to be lower
<lifeless> 50MB
<lifeless> which is, I believe, somewhat larger :)
<lifeless> sinzui: I had a question for you
<lifeless> sinzui: why do we blacklist -owner as a team/person name.
<lifeless> sinzui: is it because we wouldn't be able to give the object a mailing list ?
<lifeless> sinzui: and if so, I'm confused because having the owner of list foo as a list itself sounds darn cool
<sinzui> lifeless: I honestly do not know.
 * sinzui looks at nameblacklist
<sinzui> lifeless: mailman does routing based on naming convention. http://www.gnu.org/software/mailman/mailman-install.txt
<lifeless> sinzui: yeah, I realise we'd need to smack mailman upside the head a little
<sinzui> lifeless: mailman3 aka a restful lib might be easy to smack
<lifeless> yeah
<lifeless> so lets leave it for now, I wanted to confirm I understood it properly - thanks
<LPCIBot> Project windmill-db-devel build #355: STILL FAILING in 1 hr 5 min: https://lpci.wedontsleep.org/job/windmill-db-devel/355/
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #772: FIXED in 4 hr 56 min: https://lpci.wedontsleep.org/job/devel/772/
<benji> abentley: do you have a minute for a simple review? https://code.launchpad.net/~benji/launchpad/bug-719637/+merge/63159
<abentley> benji: sure.
<benji> cool
<abentley> benji, do you know about the lp.testing.temp_dir context manager?
<benji> abentley: I do now. ;)
<abentley> benji: why are you deleting the directory in setUp?
<benji> abentley: that was some copy-pasta; I just realized that the entire setup and teardown methods are unneeded; I've pushed a change to remove them
<abentley> benji: excellent.
<abentley> benji: in 9-14, are you actually considering throwing an exception or reporting an OOPS?
<abentley> benji: nm, I must be seeing things.
<abentley> benji: r=me.
<benji> cool, thanks!
<jcsackett> sinzui: i just threw up the MP for the branch that links the new YUI personpicker into the web ui. i requested you as reviewer, if you don't mind, since it needs UI as well as code.
<benji> gary_poster: I used (a slightly modified version of) the lp2kanban script to update a card for me and it wanted to change the card with title "This will be closed automatically by doing this feature [I should not get bugmail when I am subscribed via an invalid bugtask]" to the real title for bug 380205...
<_mup_> Bug #380205: Invalid bugtasks should not cause emails for structural subscriptions <email> <lp-bugs> <story-better-bug-notification> <Launchpad itself:Fix Committed by yellow> < https://launchpad.net/bugs/380205 >
<benji> so if you want to run the script against the board, know that that'll happen
<gary_poster> benji, good to know
<gary_poster> benji, I wonder if script should only be willing to change title if the current card's title is a certain string, or maybe less than N chars or something
<gary_poster> (unless you explicitly request it somehow?)
<gary_poster> If we tweak a title we miht have a reason for it
<gary_poster> might
<benji> well, if we went that route it would be better to some how mark the card as robot-managed
<gary_poster> maybe title would have short prefix?
<benji> for example, if I enter a card, putting in the bug ID, the script fills in the details, then I edit the bug title, I'll want the new title updated on the card
<gary_poster> understood
<gary_poster> or suffic
<gary_poster> suffix
<benji> that's possible, I have a slight preference for the rule being "if it has a bug number (external ID), then it is externally managed"
<gary_poster> that's pure but not as flexible as I think I want.  I s'pose we could try it out and see how much it chafes me
<benji> we could then add an opt-out rule instead of an opt-in rule
<benji> tag the card with "keep your grubby robot hands off me"
<gary_poster> that would be fine-ish, but if instead it is an opt-in, and a card that is synced initially will get the opt-in mark by default, then we might get the best of both worlds: by default, things are synced, but it's clear what to delete if you want to stop the syncing
<benji> I like that better.
<benji> gary_poster: I'll add a card for that functionality.
<gary_poster> cool benji
<LPCIBot> Project windmill-devel build #164: STILL FAILING in 1 hr 8 min: https://lpci.wedontsleep.org/job/windmill-devel/164/
<jcsackett> sinzui: can we remove the card for bug 669933 given its resolution?
<_mup_> Bug #669933: person picker could accept extra criteria to find a user <disclosure> <lp-registry> <person-picker> <Launchpad itself:Won't Fix> < https://launchpad.net/bugs/669933 >
<sinzui> jcsackett: yes, we can delete it
<jcsackett> cool, done.
<LPCIBot> Project windmill-db-devel build #356: STILL FAILING in 41 min: https://lpci.wedontsleep.org/job/windmill-db-devel/356/
* abentley changed the topic of #launchpad-dev to: Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: - | Critical bugs:208 - 0:[######=_]:256
<LPCIBot> Project db-devel build #605: FAILURE in 2 hr 55 min: https://lpci.wedontsleep.org/job/db-devel/605/
<LPCIBot> Project windmill-devel build #165: STILL FAILING in 45 min: https://lpci.wedontsleep.org/job/windmill-devel/165/
<lifeless> poolie: when you're around, I would like to talk about bzr formats and lp
<sinzui> wgrant: wallyworld: jcsackett. I will be 15 minutes late to the stand up.  Sorry.
<wallyworld> np
#launchpad-dev 2011-06-02
<jtv> morning peops
<jtv> Oh how I hate thisâ¦ when we're late paying the bill, my ISP regularly stops routing my packets until it's been able to redirect an http request to its "pay up or da bitch gets it" notice.  But how often do we use http these days?
<StevenK> Ah yes, my ISP does that too.
<StevenK> The way I notice is when my DSL connection gets assigned an internal IP
<jtv> I wonder if this could be generalized into the same problem as those annoying redirect-all-your-browser-tabs-with-no-way-back wifi login pages, so that someday some saviour may come up with a helpful protocol.
<jtv> "Some annoying twit insists on showing you their web page.  They may just be really proud of it, or they may want money.  I've stopped them from messing with your browser tabs, but click here to see what they have to say."
<lifeless> jtv: you mean like all the launchpad operational pages (devpad, lpstats etc)
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #606: FIXED in 5 hr 30 min: https://lpci.wedontsleep.org/job/db-devel/606/
<jtv> lifeless: yes, come to think of it it would be nice there as well.
<jtv> Log in once, then reload all those tabs.
<jtv> I've lost God knows how many pages to broken login redirects.
<jtv> And then there's broken censorship proxies: "your license to use StalinWareâ¢ SafeBrowseÂ® has expired.  Please contact your nearest military dictator for new terms."
<lifeless> rofl
<jtv> (Although that _is_ one of the funniest things I've ever seen â proprietary commercial software and authoritarian state censorship biting each other)
<jtv> I may have misremembered some details of the exact wording, of course.
<sinzui> wallyworld_: I think this fixes the badge issue in the picker: http://pastebin.ubuntu.com/616306/
<wallyworld_> sinzui: thanks. i had the style elements similar to yours but couldn't get the override aspect to work
<sinzui> This is easier to read http://pastebin.ubuntu.com/616309/
<sinzui> wallyworld_: I think this markup means we can show only one badge
<wallyworld_> sinzui: that css should go in lazr-js though. do we still need it in the lp tree?
<sinzui> No keep it in our tree. I am tired of that lib inject rules into Lp
<wallyworld_> sinzui: my css was positon:relative; padding-left: 5px and that works for multiple
<sinzui> I am sure U1 and landscape do not want us injecting rules into their sites
<lifeless> man, all software sucks
<wallyworld_> sinzui: ok. i built a lazr-js tar ball with "setup sdist" but that results in lp jsbuild issues which i'm still trying to figure out
<sinzui> wallyworld_: I think relative will have surprising results across browsers
<wallyworld_> sinzui: so relative isn't interpreted in a starndard way?
<lifeless> ekiga, on startup, reads something like all of /dev
<sinzui> wallyworld_: I have not seen it be reliable
<sinzui> wallyworld_: I just switched my example to relative and it is not working in webkit
<wallyworld_> sinzui: i think we need a way of displaying multipe badges though
<sinzui> There can be only one primary context
<wgrant> sinzui: No.
<wgrant> sinzui: Subscribing someone to a multi-task bug.
<wgrant> Multiple contexts.
<wgrant> (yes, there is a single root context for the page, but that is a bug)
<wallyworld_> so we could use a div and put all the badges in there with no "posiiton" style attribute?
<sinzui> Well relative does not work for me and I do not think we want to tell abandon branches for one case
<sinzui> s/branches/badges/
<sinzui> wallyworld_: I can make multiple appear, but they break the bound box and can only be aligned by percentage. It is very unreliable
<wallyworld_> sinzui: ok. so just stick to one for now?
<wgrant> Can't you just put them in a div, and stick that to the right of the li?
<wallyworld_> that's what i was asking :-)
<wallyworld_> sinzui: so given that img.badge is a new style and we won't be messing with u1 et al since they won't be using it, could we not include the css in lazr-js?
<lifeless> its not a new style is it ?
<lifeless> its a picker injected style which we're overriding
<wallyworld_> lifeless: it is a new style
<wallyworld_> because it pertains to badges being displayed next to the picker item title
<wallyworld_> so css "img.badge" is new
<sinzui> wallyworld_: this will support a few badges, but I think we need to view it in a few rendering engines:
<sinzui> http://pastebin.ubuntu.com/616323/
<sinzui> wallyworld_: Yes, these are new classes
<wgrant> What's position: initial?
<wgrant> I've not seen that before :/
<wallyworld_> sinzui: thanks for the new css. so since they are new, i could reasonably put them in lazr-js?
<sinzui> Please do not
<sinzui> I hate every person who put their sites' rules into Lp that I had ti undo. That is a contributing reason to Lp's CSS woes
<sinzui> These rules belong in Lp's CSS
<wallyworld_> ok :-)
<wallyworld_> i guess i was just thinking that lazr-js should work sensibly "out of the box"
<wallyworld_> and downstream could always override stuff as required
<sinzui> wallyworld_: This is what the sample looked like in my test: http://people.canonical.com/~curtis/badges.png
<wallyworld_> sinzui: nice :-)
<sinzui> wallyworld_: The out of the box experience, much like YUI's out of the box experience imposes rules at too many levels many upgrades difficult.
<sinzui> As you can see by my diff, our rules were broken last august when we updated yui
<wallyworld_> sinzui: sure. so your philospohy is each user of lazr-js needs a stylesheet with all the necessary css rules for the lazr-js widgets defined for their app?
<wallyworld_> wgrant: you may be able to help me. i build a lazr-js tarball with setup sdist (after changing __version__ to 1.6DEV-xxx). i change versions.cfg in lp, do a make clean build, but new new js is not being used :-(
<wallyworld_> i put the tarball in download cache
<wgrant> wallyworld_: rm -r lazr-js/build
<wgrant> make jsbuild
<wgrant> Ah.
<wgrant> make clean_js
<wgrant> make jsbuild
<wallyworld_> wgrant:  tried that. make clean does it also
<wallyworld_> ah didn't try make clean_js
<sinzui> wallyworld_: The CSS chain imposes a style, one that happen to not work well with what Lp is trying to do. We should have about 100 CSS rules, but we have about 800. Most rules are struggling to undo a rule defined before it :(
 * wallyworld_ tries it
<wallyworld_> sinzui: sounds like a job for huwshimi :-)
<wallyworld_> sinzui: but forgetting about lp, as a user of a 3rd party library like lazr-js, i expect sensible defaults that i can customise if i wish. but that's only my view. we can discuss over a beer at the epic :-)
<wallyworld_> wgrant: something is wrong with the tarball perhaps. a compare with the working one shows only the expected source differences, but when i try and make jsbuild, i get:
<wallyworld_>   File "/home/ian/projects/lp-sourcedeps/eggs/lazr_js-1.6DEV_r209-py2.6.egg/lazr/js/build.py", line 72, in update
<wallyworld_>     target_fh = open(self.target_file, 'w')
<wallyworld_> IOError: [Errno 2] No such file or directory: 'lazr-js/build/lazr.js'
<sinzui> wallyworld_: So who uses badge?, what are their image sizes? How many do they require? What is their picker width? How large is their text? These are the kind of questions that were not asked about previous additions, so we need to redefine them
<wgrant> wallyworld_: Have you compared the tarballs?
<wallyworld_> wgrant: yes
<wallyworld_> only the expected changes from an itinial look. i'll fire up meld again
<wallyworld_> sinzui:  all good points
<wgrant> wallyworld_: Oh, heh.
<wgrant> Just reread the error message.
<wgrant> lazr-js/build shouldn't have been deleted.
<wgrant> It just needed to be emptied.
<wgrant> bzr st should tell you.
<wallyworld_> wgrant: make clean_js does a rm -f -r lazr-js/build
<wgrant> That's unhelpful.
<wallyworld_> wgrant: bzr status doesn't complain
<sinzui> wallyworld_: the img.badge class may not be needed. I like this more http://pastebin.ubuntu.com/616332/
 * wallyworld_ looks
<wallyworld_> sinzui: that looks nice and clean. i'll use it. i wasn't passing in any alt attributes. are they strictly required? i would have to modify the IHasAffiliation implementation etc to provide the extra info
<wgrant> wallyworld_: I'd have an alt and title.
<wgrant> I think.
<wgrant> Like team badges do.
<wgrant> No alt == accessibility fail
<wallyworld_> wgrant: ok. btw, the build.py in the latest lazr-js source tree is brooken :-(
<wallyworld_> that's why it won't build :-(
<wgrant> Awesome!
<wallyworld_> i haven't yet looked to see how it happened
<wallyworld_> mwhudson: ping!
<mwhudson> wallyworld_: hello
<wallyworld_> mwhudson: hi. i'm trying to understand a change made to build.py in lazr-js. was that you per chance?
<wallyworld_> its a change to SRC_DIR
<mwhudson> wallyworld_: i think that may have been me
<wallyworld_> it seems to have broken make jsbuild for lp
<mwhudson> wallyworld_: iirc it was so it would work uninstalled
<mwhudson> wallyworld_: i'm sorry to hear that
<wallyworld_> mwhudson: np. i'm trying to see the best way to fix it
<wallyworld_> so that what the change fixed still works and so can lp
<wallyworld_> what's thebest way to test your change so that i can mess with it can make both scenarios work?
<mwhudson> wallyworld_: put "WSGIScriptAlias /combo /path/to/lazr-js/combo.wsgi" in apache config, restart apache, does combo loader work?
<mwhudson> wallyworld_: i'm sure we can change our combo loader deployment to run installed in a virtualenv or something,  which might make the previous version work
<wallyworld_> mwhudson: given the change that was made, i'm wondering if it is possible to support both scenarios?
<wallyworld_> mwhudson:  one would be to try the new SRC_DIR and revert to the old definition if it isn't there
<wallyworld_> a bit hacky perhaps
<mwhudson> i guess that works
<wallyworld_> mwhudson: actually, i've read the source code a bit - it supports a cmdline option to override the src_dir. i'll see if i can modify the make scripts for lp to pass in the relevant directory
<lifeless> huwshimi: hi
<lifeless> 16:01 < micahg> lifeless: BTW, I failed to note it was private until after I posted the first reply to you, I guess I'm not used to the bar on top yet :-/
<lifeless> 16:02 < lifeless> hmm
<lifeless> 16:02 < lifeless> :(
<lifeless> 16:02 < micahg> the benefit on the bar on the side was that you could see it all the way down the page, my eyes went straight to the description
<lifeless> 16:03 < lifeless> thats worth noting on the bug I think
<micahg> lifeless: which bug did you mean for me to note it on?
<lifeless> there is one about the privacy banner
<micahg> k
<StevenK> micahg: Personally, I find the bright red bar enough of a visual warning -- it's bright enough and large enough that my eyes are drawn to it.
<huwshimi> lifeless: Thanks
<micahg> not if you're ignoring the top 100px of the window (I'm on 1920x1080 at the moment)
<micahg> bug 29152?
<micahg> oops
<_mup_> Bug #29152: Suggestions for FAQ guide <Ubuntu Documentation:Fix Released by robertstoffers> < https://launchpad.net/bugs/29152 >
<micahg> bug 298152
<_mup_> Bug #298152: privacy portlet is not visible enough for indicating private objects <disclosure> <lp-web> <oem-services> <privacy> <ui> <Launchpad itself:In Progress by huwshimi> < https://launchpad.net/bugs/298152 >
<micahg> lifeless: is that the one? ^^
<lifeless> yes
<StevenK> micahg: I am also on 1920x1080, on a 22 inch LCD screen, and I don't ignore the top 100px of the window
<micahg> my screen is 15.6" and I still managed to miss it :)
<StevenK> micahg: See a doctor? :-)
<micahg> lifeless: commented in the above bug
<lifeless> micahg: thanks, appreciate the feedback
<cody-somerville> lets get rid of private stuff all together, then no more problem :P
<lifeless> cody-somerville: ^^
<lifeless> cody-somerville: you'd be out of a job then ;)
<cody-somerville> we'd use bugzilla :P
<cody-somerville> (sadly of course)
<lifeless> I don't see it meeting your partitioned-view needs
<StevenK> cody-somerville: Isn't that a downgrade?
<cody-somerville> Yes but I doubt we could get approval to run our own launchpad instance :P
<lifeless> cody-somerville: still wouldn't do what you say you need ;)
<lifeless> cody-somerville: you'd need N separate instances
<lifeless> cody-somerville: and I don't see how that is a usability improvement :>
 * cody-somerville is being entirely facetious... we'd of course use Microsoft Dynamics CRM :P
<cody-somerville> or maybe Microsoft's Team Foundation Server
 * StevenK goes to review his lunch
<wgrant> That sounds less than ideal.
<StevenK> cody-somerville: I'm blaming you for that.
<StevenK> cody-somerville: Besides, isn't using a CRM as a bug tracker a bit like using a wiki as an issue tracker?
<wgrant> *cough* SharePoint
<lifeless> question for the peanut gallery
<lifeless> does the codehosting bzr+ssh server reject attempts to creation branches with experimental formats
<wgrant> bzr: ERROR: Server sent an unexpected error: ('error', "'Bazaar development format 8\\n'")
<lifeless> is that one *in* the bzrlib we run
<wgrant> A good question.
<wgrant> Seems to be.
<lifeless> cool
<wgrant> bin/bzr init --development-subtree works.
 * lifeless closes wontfix
<jtv> stub: I got a review request message for your tests branch.  Did you request a review from me?
<stub> I requested a review from 'launchpad'
<lifeless> hi stub
<stub> yo
<stub> I should have gone out drinking last night so I have an excuse for this hangover.
<lifeless> :)
<StevenK> Haha
<lifeless> getting old ?
<stub> Falling apart
<StevenK> I'm reading The Last Continent, and Rincewind has staggered into the pub, hungover, and the bartender says "You look like you need the hangover cure!" "Oh, what's that?" "More beer!"
<jtv> Thanks for the spoiler.
<StevenK> It's one joke
<jtv> Now I know how my next hangover is going to end.
<StevenK> :-)
<lifeless> stub: index repack time ?
<stub> bug_fti? I did that just two days ago
<lifeless> cool,t hanks
<lifeless> so something else iz making the slow
<stub> Could be another index... bug search again?
<lifeless>     Hard / Soft  Page ID
<lifeless>       98 /   38  Person:EntryResource:searchTasks
<lifeless>       78 /   20  Distribution:EntryResource:searchTasks
<lifeless>       59 /    8  POFile:+filter
<lifeless>       56 /   15  Distribution:+bugs
<lifeless>       52 /   16  MaloneApplication:+bugs
<StevenK> How do you repack it? Hopefully it isn't DROP INDEX, CREATE INDEX?
<lifeless> no
<lifeless> its create drop
<stub> create drop rename
<lifeless> stub: shall we chat ?
<stub> lifeless: sure.
<jtv> Do we have a way to specify an adapter from an enumeration to an interface?
<jtv> Adapting lazr.enum.DBItem works, butâ¦ yuck.
<lifeless> eeeeek
<lifeless> wouldn't a factory function be better ?
<jtv> That's basically what I have, except it's got the name of the interface.
<jtv> "I have this enum for one of a number of policies; give me the matching policy implementation."
<jtv> (It's "got the name of the interface" only when invoking, obviously, not in implementation)
<wgrant> wallyworld_: Your query has a bug.
<wgrant> wallyworld_: The outer EmailAddress join query needs to do the PREFERRED check too.
<wgrant> wallyworld_: Otherwise you'll return multiple rows for teams with multiple addresses.
<wgrant> s/join query/join condition/
<mrevell> Good morning
<jtv> allenap: and when you're done with stubâ¦ https://code.launchpad.net/~jtv/launchpad/db-bug-790761/+merge/63203
<allenap> jtv: Brillig :)
<StevenK> jtv: I was in line first :-P
* allenap changed the topic of #launchpad-dev to: Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: allenap | Critical bugs:208 - 0:[######=_]:256
<allenap> StevenK: I haven't forgotten :)
<jtv> StevenK: and you aren't any more?  Good.
<StevenK> allenap: :-) Do you need a link?
<jtv> I thought "brillig" was a weather condition.
<allenap> StevenK: You seem to have something on +activereviews.
<StevenK> allenap: If it's 'copies-use-notify', that's it
<allenap> Yes.
<jtv> bigjools: afaict we create PPCJs in 2 places: "Sync Sources" and "Upgrade Packages."  Which should have which policy?
<bigjools> jtv: sync is the "insecure", upgrade is the "sync"
<jtv> how perfectly straightforward (!)
<bigjools> this naming in the UI needs to be fixed
<bigjools> can you do a drive-by?
<bigjools> Upgrade should be "mass-sync"
<bigjools> the SyncCopyPolicy should be MassSyncCopyPolicy
<jtv> Probably best done after Gavin finishes my review; got a sequence of branches in flight right now.
<lifeless> evening podeans
<bigjools> hello lifeless
<jtv> lifeless: *You're* an evening podean.
<jtv> That shut him up.  Now, what's an evening podean?
<lifeless> pfttt
<bigjools> did lifeless just fart?
<lifeless> blew a raspberry
<jtv> I guess people on his side of the globe fart out the other side.
<lifeless> Given you're on the same side of the globe...
<bigjools> well he's in between.  the mind boggles
<jtv> Nonsense.  I'm on the Northern Hemisphere, so you're upside down.
<lifeless> dude, you're so far down you'll slide off sideways
<lifeless> and after that, you're clearly southern :>
<jtv> Well people did warn me about the slippery slope when I moved here...
<lifeless> one night in bangkok
<bigjools> confucius say, man who go through check-in desks sideways going to Bangkok
<lifeless> whimper
<bigjools> right, I have my first legitimate change to use matchers.MatchesStructure in a test \o/
<bigjools> chance, even
<jtv> bigjools: which policy did you say we could default to?
<bigjools> jtv: the old insecure
<jtv> Old?
<jtv> What's old about it?
<bigjools> well you're changing the name to an enum
<bigjools> but that one
<jtv> Oh, yes I am.  So now it's INSECURE.  :)
<jtv> bigjools: I've thrown in a PackageCopyJob.getPolicyImplementation() that gives you the applicable ICopyPolicy.
<jtv> (Since you're planning to glue it all together)
<bigjools> coo9l
<bigjools> why not just "getPolicy" ?
<jtv> Because that might cause confusion with PackageCopyPolicy.
<jtv> Horrible, I know.
<bigjools> what confusion?
<jtv> You'd have copy_policy and getPolicy()
<jtv> âand they'd give you slightly different things.
<bigjools> oh, what does copy_policy return?
<jtv> Enum value.
<jtv> The enum is called PackageCopyPolicy.
<bigjools>  /o\
<jtv> ?
<jtv> Or is that just the penny of the above horror dropping?
<bigjools> yes
<bigjools> perhaps s/PackageCopyPolicy/PackageCopyPolicyType/ ?
<jtv> Talk to stub â he seems to have a headache he doesn't need
<jtv> Aaaaaarrrrgh!
<bigjools> PackageCopyPolicy is massively confusing
<jtv> PackageCopyPolicyDBEnumItemsListType?
<bigjools> there's already a bunch of XXXCopyPolicy objects
<jtv> Is there!?
<bigjools> :p
<jtv> Damn RIAA.
<jtv> Sorry, for XXXCopyPolicy it'd be the MPAA.
<jtv> bigjools: ah, but those are the actual ICopyPolicy implementations.
<bigjools> yes
<jtv> So that's not very confusing at all
<bigjools> it is to me :/
<bigjools> gah
<bigjools> whatever, I'm too knackered to argue the toss
<jtv> What I have now is: an ICopyPolicy interface, implemented by both SyncCopyPolicy and InsecureCopyPolicy.  That part does not seem difficult.
<bigjools> ok
<jtv> What does confuse things a bit is that there's a PackageCopyPolicy enum whose items correspond to those two implementing classes.
<jtv> But that's about it.
<jtv> (And there is no longer a direct inheritance relationship between SyncCopyPolicy and InsecureCopyPolicy.  They just both use a helper class for the common parts).
<jtv> Also, and this may help, the docstrings should be fairly clear about the relationship between the policy classes and the policy enum values.
<jelmer> allenap: hi
<allenap> jelmer: Hi there.
<jelmer> allenap: are you still reviewing today?
<allenap> jelmer: Yes. I have one review to finish then two more in the queue. I'll be able to do yours but not for a few hours.
<jelmer> allenap: (it seems like you're always on call?)
<allenap> jelmer: Just Thursdays :)
<jelmer> hmm, I just must've submitted my last couple of MPs on Thursdays then.. :)
<lifeless> night everyone
<allenap> Cheerio.
<bigjools> allenap: helleau, can I have a review for the override stuff plz
<bigjools> https://code.launchpad.net/~julian-edwards/launchpad/override-class/+merge/63221
<jtv> We may have kept him slightly busy.  :)
<wallyworld> wgrant: just got back from soccer and saw your message. the outer email join does do the preferred check, unless i am mistaken. did you see the version of the query in the mp?
<wgrant> wallyworld: I was looking at the latest rev of your branch at the time.
<wgrant> wallyworld: It does that in the WHERE clause.
<wgrant> But not in the join condition.
<wgrant> So it will join all of them.
<wallyworld> let me take a look
<wgrant> And teams are exempt from the preferred check in the WHERE, so all of their addresses will show up.
<wallyworld> you taking about the query in the mp?
<wallyworld> it appears so
<wgrant> I'm looking at r13133 of your branch.
<wgrant> I don't have the MP nearby. Let's see.
<wallyworld> i will have to compare it with the original version of the sql before the changes, but i don't *think* that bit of the logic has changed
<wgrant> Hmmm.
<wallyworld> and all the tests pass so there's inadequate coverage if there is a problem
<wallyworld> i thought there was a comment about ignoring email for teams somewhere
<wallyworld> in the code
<wallyworld> i will try and find that
<wallyworld> actually, it is possible to run both versions of the query (old and new) by turning the featureflag on/off
<wallyworld> wgrant: do you have a search term that shows the issue? is it only apparent on dogfood rather than the sample data?
<jtv> jcsackett: good morning!  Reviewing today?  If so, https://code.launchpad.net/~jtv/launchpad/db-bug-791737/+merge/63220 :-)
<wgrant> wallyworld: Oh, the original query has a DISTINCT.
<wgrant> wallyworld: That would probably solve it.
<wallyworld> ah
<wgrant> Oops.
<wgrant> And my extra stuff to return the rank disturbed that.
<wgrant> Never mind me.
<wallyworld> that's ok. if there is a problem with the query let me know so i can fix it
<wallyworld> wgrant: btw, the latest badges css stuff works on chrome but fails on ff4 :-( i'll look into it
<wgrant> wallyworld: Fails how?
<wallyworld> all the badges are aligned hard right
<wallyworld> over the top of each other
<wallyworld> nicely spaced on chrome through
<wallyworld> i really thought this css shit would be sorted out in 2011
<wallyworld> sigh
<wallyworld> wgrant: fixed the lazr-js build too. it was indeed a regression in the build.py introduced after rev 206
<wgrant> wallyworld: Does it work both ways?
<wallyworld> the default is the new way and of the dir it expects isn't there (which it won't be for lp where it runs from the egg) it uses the old way
<wallyworld> s/of/if
<NCommander> morning wgrant
<wgrant> Hi NCommander.
<NCommander> wgrant: do you plan to be up for a bit longer? I'd like to bridge the discussion on putting SUPERSEDED packages in the indices with bigjools while you are around
<wgrant> NCommander: Not tonight, I'm afraid.
<bigjools> I am too busy to have a conversation, but why do you want to do that?
<NCommander> bigjools: to work towards making archive skew a thing of the past. Having SUPERSEDED but still alive packages in the indices would make it extremely straightforward with the added advantage that apt-cache will also be able to show the status of a package that is still in a state of transition.
<bigjools> you're talking about packages that still have dependencies on them, right?
<bigjools> I'd rather fix that properly than hacking the publisher to keep superseded packages in the repo
<wgrant> bigjools: I think this may be the proper fix.
<bigjools> URGH
<wgrant> Hm?
<bigjools> please no
<wgrant> These packages are superseded.
<wgrant> But they're not yet to be removed.
<wgrant> Which means they are dominated, but not judged.
<wgrant> We already do this.
<wgrant> To keep sources around when their binaries are.
<bigjools> yes but the implication is that they are disappearing RSN
<bigjools> it's a different situation
<wgrant> It's the same situation.
<wgrant> There is a newer version, but archive consistency constraints prevent the removal.
<bigjools> it's not an archive consistency issue
<bigjools> it's a dependency issue
<wgrant> NCommander: Oh, you are still wanting full dependency checking?
<wgrant> Rather than just what Debian has?
<NCommander> wgrant: Debian does full dependency checking
<wgrant> NCommander: Do you have a code pointer?
<wgrant> That's not what I understood.
<bigjools> there's a bug about this somewhere as well
<NCommander> wgrant: its handled by britney, which is where their images are built from
<wgrant> NCommander: That's separate.
 * NCommander finds the spec that explains what dak does
<wgrant> NCommander: britney handles testing migrations.
<wgrant> It's not what keeps unstable installable.
<elmo> nothing keeps unstable installable ;-)
<wgrant> True.
<NCommander> wgrant: yes, but it doesn't matter if unstable is uninstallable if their images are built from testing
<wgrant> NCommander: That's completely different from the spec you pointed at last time.
<NCommander> wgrant: I'm trying to find it
<wgrant> http://ftp-master.debian.org/wiki/projects/arch-all-tracking/
<NCommander> http://ftp-master.debian.org/wiki/projects/arch-all-tracking/
<NCommander> bah
<NCommander> you beat me to it
<NCommander> So to be specific, dak itself doesn't do dependency checking, but britney does.
<elmo> oh, wow, this spec
<elmo> I suggested this *years* ago
<elmo> I didn't actually do it though, so...
<wgrant> NCommander: It's entirely separate.
<wgrant> NCommander: Implementing britney-style consistency in a single suite is a thoroughly different problem.
<jtv> bigjools: eod and everything's up for review.  Anything in particular you'd like me to pick up tomorrow morning?
<wgrant> And some orders of magnitude less feasiblwe.
<NCommander> wgrant: fair enough
<NCommander> wgrant: and for what I need, overkill
<wgrant> elmo: There are lots of XXXs from 2005 about it.
<elmo> it's actually a really good idea
<elmo> ;-)
<NCommander> wgrant: I suppose its a matter if you see britney as part of Debian publishing infrastructure. I do. I realize opinions on this differ.
<wgrant> NCommander: It's more part of their release management infrastructure.
<wgrant> But possibly.
<NCommander> That being said, from the perspective of someone building images, having all the versions appear is desirable. Then deathrow just needs to be adjusted to make sure death only comes to packages that have been completely superseded
<jkakar> Is there an alias for lp:~jkakar/+junk/...?  Having +junk in a branch URL always feels bad.
<jkakar> Couldn't it just be +branches or something?
<wgrant> Part of the point of +junk is discouragement, I believe.
<jkakar> Or maybe +sandbox.
<wgrant> Both of those have been suggested. There's a bug about that...
<jkakar> wgrant: Well, it works I guess.. but I have a number of things I will never create a project for.
<bigjools> it's discouragement
<bigjools> we want people to have projects
<jkakar> wgrant: For example, I keep my emacs configuration in a branch and push it to lp:~jkakar/+junk/emacs.  I don't need (or want) a project for that.
<bigjools> I keep a junk branch for some of my stuff too :)
<wgrant> Bug #147407
<_mup_> Bug #147407: Junk sounds too harsh <lp-code> <Launchpad itself:Triaged> < https://launchpad.net/bugs/147407 >
<jkakar> wgrant: Thanks.
<jkakar> Why hasn't this been marked 'Opinion'?
 * jkakar hides ;b
<wgrant> I contest some of the interpretation on that bug, though.
<wgrant> Perhaps it is a locale issue.
<wgrant> junk does not seem offensive or childish to me.
<allenap> bigjools: Sure, I'll review that after I've done a few others.
<wgrant> And indeed, that was the term used on a few uni systems for a similar purpose.
<wgrant> So who knows.
<bigjools> allenap: priority interrupt? :)
<StevenK> bigjools: No queue jumping!
 * StevenK waves a big stick at bigjools.
<bigjools> There has to be some sort of international directive I can invoke
<StevenK> The international directive of "Wait your turn, bitch"
<wgrant> That's probably just an EU directive.
<bigjools> EU directives amount to "do what we say. oh, and give me all your money"
<wgrant> I hoped for more of a response than that.
<bigjools> EUSSR ;)
<StevenK> It can't the USSR, it isn't a Commuist regime.
<StevenK> Communist, sigh
<bigjools> it has an unelected, authoritarian core, it couldn't be more communist if it tried.
<abentley> bac: the buildbot failure doesn't look like something I did.  Was it you, or something intermittent? https://lpbuildbot.canonical.com/builders/lucid_lp/builds/1004
<bac> abentley: will look
<wgrant> bac, abentley: Spurious. The librarian or other random port service decided to listen on buildd-manager's port.
<NCommander> bigjools: re: implementing indexing SUEPRSEDED, it has the additional side effect of allowing apt-get to grab all files on disk, which is useful when you want to grab an older source package that won't leave the archive. I'm curious on why you feel though having SUPERSEDED in th eindex is a bad thing. When they get deathrow'ed, they'll be removed from the index
<NCommander> so at most its still a handful of packages
<bigjools> NCommander: I will not support indexing superseded.  If they are indexed, they are published.  We need to fix which packages remain published.
<wgrant> Sounds like I have some convincing to do.
* jcsackett changed the topic of #launchpad-dev to: Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: allenap, jcsackett | Critical bugs:208 - 0:[######=_]:256
<bigjools> wgrant: it's a *very* strong meme in Soyuz that published = in the index
<wgrant> bigjools: Possibly.
<allenap> bigjools: I missed your priority interrupt earlier. It came just before X decided that I hadn't seen gdm's loveliness in a while, and that, in the absence of something that is almost, but not quite, entirely unlike tea, to abruptly and irrevocably log me out.
<wgrant> More that it's visible to apt, I think.
<bigjools> allenap: lovely!
<bigjools> wgrant: superseded is a publishing state, not whether the package is actually superseded
<wgrant> bigjools: I always interpreted it as an indication that the publication was superseded.
<wgrant> bigjools: I think we can sensibly rework this.
<wgrant> But it needs much discussion when you are not hideously busy.
<wgrant> Which could be a while.
<bigjools> yes
<bigjools> also, I'm not prepared to spend that much time on it anyway unless it's been officially escalated
<bigjools> bit harsh, but there's a million other things that need fixing too
<wgrant> Indeed.
<allenap> bigjools: I wanted to get the tea thing in even though it doesn't make any sense, and I forgot to ask if you want me to review your branch next, or if I should continue on with jtv's?
<bigjools> allenap: nah, I was half joking.  I am sorta blocked on it but I can assume you'll review it with flying colours and just merge it into my dependent branch :)
<jcsackett> testSMTPServerIsAvailable is one of the intermittently failing tests we have right now, right?
<StevenK> allenap: Many thanks for the review. I thought there was a function to do Person -> formattted address, but neither wgrant or myself could think of one.
<jcsackett> sinzui: time to chat this morning?
<sinzui> jcsackett: yes
<sinzui> let me get some coffee first
<jcsackett> ooo. good idea.
<bigjools> allenap: I am just adding another method to the branch up for review, did you start it yet?
<allenap> bigjools: Nope, not yet.
<bigjools> coolio
<sinzui> jcsackett: I am ready. sip?
<jcsackett> sinzui: sure. ringing you in a moment.
<jcsackett> sinzui: having a bit of an issue with empathy. one sec.
<abentley> allenap or jcsackett: could you please review https://code.launchpad.net/~abentley/launchpad/ppa-api/+merge/63170 ?
<allenap> abentley: Sure, I'll take it.
<abentley> allenap: Thanks.
<bigjools> thanks for the review allenap
<bigjools> MatchesStructure is great eh
<allenap> bigjools: That it is.
* allenap changed the topic of #launchpad-dev to: Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: jcsackett | Critical bugs:208 - 0:[######=_]:256
<allenap> Sayonara jcsackett, happy reviewing :)
<jcsackett> ciao, allenap. :-)
<dpm> thanks flacoste for testing the lp-get-ul10nstats
<dpm> script
<sinzui> jcsackett: can you review https://code.launchpad.net/~sinzui/launchpad/picker-suggestion-menu-0/+merge/63260 today?
<jcsackett> sinzui: i can do it in just a few moments, in fact. :-)
<jcsackett> sinzui: i see text conflict on your MP.
<jtv> jcsackett: thanks for the review â I've responded.
<LPCIBot> Project windmill-devel build #166: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill-devel/166/
<jcsackett> jtv: thanks for the explanation. r=me. :-)
<jtv> thanks!
<bigjools> good night all
<jcsackett> sinzui: comment on your MP.
<sinzui> jcsackett: I just resolved conflicts.
<jcsackett> \o/
<sinzui> jcsackett: I see someone listened to lint, but did not run tests. devel is broken, but my branch had the fix
<sinzui> yeah for running the tests manually
<jcsackett> so the only other thing is if "field.initval-suggestions" was meant to be "field.initial-suggestions" or if initval is correct, sinzui.
<sinzui> jcsackett: initval happens to be the name of a field in the existing test. my spell checker hates it
<jcsackett> sinzui: dig.
<sinzui> I can give it a realistic name or a total nonsense one so we do not think wtf next time
<jcsackett> sinzui: if you like. i don't know that it is necessary.
<jcsackett> you get an r=me without it.
<jcsackett> i am just waiting ofr the diff to update to make it official.
<sinzui> Well, since you were confused, and I was too at first, I will improve the name
<sinzui> jcsackett: I had a few questions for you in your branch as well
<bac> hi matsubara, a lot of those OOPS in your report cause a server error when i try to look them up.
<matsubara> bac, can you give me one as an example?
<bac> matsubara: https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1887O957
<matsubara> bac, hmm I'll have a look at the logs to confirm, but I think those oopses are old ones that were removed by the oops-pruner but are still in the database
<matsubara> so, the pruner removes it from the file system and when oops-tools tries render it, it can't find the file
<bac> matsubara: that one i gave you dates back to march of this year
<bac> matsubara: how old before they are pruned?  i assume referenced ones are kept?
<matsubara> bac, IIRC pruner is set to delete oopses older than 30 days
<jcsackett> sinzui: i saw. i'm preparing a few fixes and will post some answers shortly.
<matsubara> bac, yep, the ones referenced in the LP database (bug messages, answers, merge proposals, etc) are kept regardless of its age
<bac> matsubara: good to know, thanks
<LPCIBot> Project windmill-devel build #167: STILL FAILING in 45 min: https://lpci.wedontsleep.org/job/windmill-devel/167/
<sinzui> I think I have the fix for the windmill fail ^. The picker.js was updated to quiet link, but it was not tested. "temp_spinner !== null" will fail because the object is undefined. The code did not work as we thought it did
<LPCIBot> Project windmill-devel build #168: STILL FAILING in 45 min: https://lpci.wedontsleep.org/job/windmill-devel/168/
<jcsackett> sinzui: replied to your comment and pushed up some changes.
<matsubara> bac, I filed bug 791975 for that 500 error. Fix is here: https://code.launchpad.net/~matsubara/oops-tools/791975-deleted-oops/+merge/63277. Would you like to review it? :-)
<_mup_> Bug #791975: Oops deleted from the filesystem raises an OopsReadError rendering the page <OOPS Tools:Triaged> < https://launchpad.net/bugs/791975 >
<jcsackett> matsubara: if bac can't, i can look at it.
<matsubara> jcsackett, that'd be great. thanks!
<bac> jcsackett: thanks, please do
 * jcsackett looks at it.
<bac> thanks for the quick fix matsubara
<matsubara> you're welcome
<jcsackett> matubara: r=me.
<jcsackett> matsubara, rather. :-P
<matsubara> jcsackett, thank you! :-)
<sinzui> jcsackett: I approved your branch
<jelmer> mwhudson: hi
<jelmer> mwhudson: I'm trying to figure out
<jelmer> mwhudson: I'm trying to update bzr on Launchpad, but hitting a strange test failures; I was wondering if you might have an idea
<jcsackett> sinzui: cool! thanks. :-)
<jelmer> mwhudson: http://pastebin.ubuntu.com/616992/
<lifeless> moin
<lifeless> statik: we're on today right ?
<statik> lifeless, yessir
<mwhudson> jelmer: oh god, those tests are difficult
<lifeless> delete them all
<mwhudson> jelmer: has anything changed in bzr around breaking locks?
<mwhudson> jelmer: is that the only test that fails?
<LPCIBot> Project windmill-db-devel build #357: STILL FAILING in 1 hr 11 min: https://lpci.wedontsleep.org/job/windmill-db-devel/357/
<james_w> wasn't there something about bzr not complaining about its own stale locks?
<mwhudson> right, that's in the sort of area that could affect these tests
<mwhudson> in fact, it might make some of the games the puller plays unnecessary, and then lifeless' solution (delete the tests, but also the code) comes in to play
<lifeless> mwhudson: I was being HHOS
<lifeless> mwhudson: but - moving this out to a microservice would be awesome
<mwhudson> lifeless: HHOS?
<elmo> haha only serious
<flacoste> abentley: how do i get the default cover template when using lp-propose?
<abentley> flacoste: install lp-review-body (should be in the ppa)
<flacoste> abentley: ok, thanks
<lifeless> matsubara: whats south ?
<matsubara> lifeless, it's the schema migration tool for django
<lifeless> thanks
<matsubara> lifeless, http://south.aeracode.org/
<flacoste> abentley: how does I use it afterwards? i thought lp-propose would use it automatically
<abentley> flacoste: It does.  It uses the target branch of the merge proposal to decide whether to supply the launchpad-specific template.
<flacoste> abentley: any idea why it didn't pick it over here?
<abentley> flacoste: what target branch is it showing?
<flacoste> the merge proposal is created with lp:launchpad
<flacoste> https://code.launchpad.net/~flacoste/launchpad/bug-365098/+merge/63301
<flacoste> was created using lp-propose
<flacoste> after having installed the package
<flacoste> oh
<abentley> flacoste: Do you still have the editor window up?  That gives the exact URL that was used.
<flacoste> i see that i have old lpreview and lpreview_body in my .bazaar/Plugins
<flacoste> i'll delete those
<abentley> flacoste: it should match 'bzr\+ssh://bazaar.launchpad.net/~launchpad-pqm/launchpad/(db-)?devel'
<flacoste> abentley: deleting the plugins in my home directory did the trick :-)
<flacoste> sorry for the noise
<abentley> flacoste: Oh, cool.
* jcsackett changed the topic of #launchpad-dev to: Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: - | Critical bugs:208 - 0:[######=_]:256
<LPCIBot> Project windmill-db-devel build #358: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill-db-devel/358/
<maxb> Is there an email interface for opening new questions in the answers app?
<sinzui> no
<sinzui> maxb: We assume the user is not an engineer and has no idea how to do it even if we built it
<maxb> Right. Makes sense. Was just trying to be really sure about a user who was claiming to be unable to access the account that they had asked a question as
<wallyworld> sinzui: the positional style "initial" for the badges doesn't work on ff4. however, "relative" works on both ff4 and chrome. I've not come across "initial" before. I only know about "absolute", "fixed" "relative", "static"
<lifeless> cody-somerville: hi https://launchpad.net/bugsy/development/
<lifeless> cody-somerville: could you check its performance for you please?
<cody-somerville> yay! it loads :)
<lifeless> what was the render time ?
<lifeless> cody-somerville: ^
<wallyworld> lifeless: 1.87 seconds for me :-)
<lifeless> wallyworld: needs to be cody to test
<lifeless> wallyworld: it was scaling due to team membership
<wallyworld> ah sorry
<lifeless> nothing to be sorry about
<lifeless> just explaining why I asked cody rather than just doing it myself :)
* wallyworld changed the topic of #launchpad-dev to: Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: wallyworld* (jtv) | Critical bugs:208 - 0:[######=_]:256
#launchpad-dev 2011-06-03
<cody-somerville> lifeless, 108 queries/external actions issued in 1.91 seconds
<cody-somerville> AJAX Log â
<lifeless> cool thanks
<nigelb> woah, https://code.launchpad.net/~launchpad-pqm/launchpad/devel just timeout'd for me
<wgrant> nigelb: Probably got caught on a branch scanner lock.
<wgrant> :(
<nigelb> wgrant: want the OOPS-ID?
<wgrant> There's already a bug, and it will show up on the reports in an hour.
<nigelb> okay
<nigelb> "oh-oh-pick-me-pick-me" heh, interesting branch name :)
<nigelb> I call today a success. I wrote a test case entirely on my own without help :D
<nigelb> I was about to ping wallworld :/
<nigelb> Could someone review https://code.launchpad.net/~nigelbabu/launchpad/spec-sub-sort/+merge/63315 for me please? :)
<wgrant> He's still around.
<wgrant> We're on a call at the moment.
<nigelb> Okay, I'll wait for later :)
<jelmer> mwhudson: yep, that's the only failing test left
<jelmer> mwhudson: I don't think anything significant has changed
<mwhudson> jelmer: do you understand what the test is trying to test?
<jelmer> mwhudson: I think I understand what it's trying to do, but the actual result surprises me
<mwhudson> jelmer: me too i guess!
<mwhudson> jelmer: as james_w said, has there been any change like breaking locks silently when there is no process with that pid any more?
<jelmer> mwhudson: I missed James' comments, but that sounds plausible
<mwhudson> although
<mwhudson> i don't think that actually applies here
<nigelb> wallyworld: hey
<wallyworld> nigelb: hi. otp. will be finished soon
<nigelb> cool
<mwhudson> jelmer: ui factory changes might also affect this
<mwhudson> PullerWorkerUIFactory.get_boolean is a thing of ... something
<mwhudson> debugging integration test failures is always such a pain
<jelmer> mwhudson: that's a good point
<mwhudson> although i'd naively think that changes would result in the other test failing, i.e. not breaking the lock when it should be broken
<lifeless> mmm
<lifeless> so we're moving soon to multiple bzr+ssh servers
<lifeless> we can't assume a pid lookup on the machine is authoritative
<jelmer> mwhudson: I bet it's confirm_action, which now defaults to true
<mwhudson> jelmer: yes
<mwhudson> jelmer: i've just found the same code :)
<wallyworld> nigelb: available now :-)
<mwhudson> jelmer: it looks like there might be a saner mechanism to achieve the desired results too, perhaps
<nigelb> wallyworld: could you review https://code.launchpad.net/~nigelbabu/launchpad/spec-sub-sort/+merge/63315 ? :)
<mwhudson> jelmer: this confirmation_id thing
<mwhudson> jelmer: does this mean that break_lock() will unconditionally break now in a script?
<jelmer> mwhudson: seems likely, SilentUIFactory (which the UI factory the puller code installs derives from) has a confirm_action implementation that returns True
<mwhudson> i wonder if that's an intended consequence
 * wallyworld shakes fist at irc - keeps disconnecting
<jelmer> mwhudson: hmm, seems like this roughly the right place.. now I have a hanging twisted "thing", which is presumably caused by waiting for the lock to go away
<mwhudson> jelmer: the lock holding process should be killed
<mwhudson> jelmer: the callback called 'cleanup' in _run_with_destination_locked
<jelmer> mwhudson: my neck is twisted from trying to follow everything that's going on here; thanks for the help so far, I'll have a closer look tomorrow
<mwhudson> jelmer: yeah, it's very confusing
<mwhudson> partly that's fairly inherent in what's being tested, but i'm sure it goes above and beyond what's required
<mwhudson> this was the first difficult code i wrote for canonical btw :)
<nigelb> wallyworld_: run lint, meaning run the tests?
<nigelb> wallyworld_: I ran the test I added
<jelmer> mwhudson: :) well, it is neat this sort of thing is actually tested
<mwhudson> jelmer: yeah, and it seems like it caught a genuine issue too!
<wallyworld_> nigelb: sorry, was otp again. lint checks that the code syntax etc is as expected. you run bin/lint
<wallyworld_> nigelb: the output will highlight issues with adherence to the coding standards
<nigelb> wallyworld_: ahh, doing that now
<wallyworld_> nigelb: it will detect which files have changed and only run against those
<wallyworld_> nigelb: also, there's a jslint for any javascript stuff you do
<nigelb> aha
<nigelb> wallyworld_: okay, I updated the branch with lint changes and the other changes you mentioned
<wallyworld_> nigelb: thanks. looking now.
<wallyworld_> nigelb: thanks for doing those changes. for next time you would then include the lint output in the merge proposal description, along with the other sections from the template as discussed. i just need jtv to sign off on it now since i'm only an apprentice reviewer
<nigelb> wallyworld_: awesome, thanks.  Could you also land it once its signed off? I'm not Canonical......yet.
<wallyworld_> nigelb: no problem at all. will do :-)
<nigelb> lifeless: Updated the code for readability as you suggested.
<lifeless> thanks
<wallyworld_> sinzui: i'm confused. i'd love a chat if you had 5 minutes. otherwise i'll ping you later
<LPCIBot> Project windmill-db-devel build #359: STILL FAILING in 44 min: https://lpci.wedontsleep.org/job/windmill-db-devel/359/
<sinzui> wallyworld_: I am back now
<wallyworld_> sinzui: mumble?
<lifeless> http://research.microsoft.com/en-us/projects/trinity/ is interesting
<sinzui> wallyworld_: I think the windmill test failures are still caused the broken spinner I reported. Either my fix failed, or it is not being tested on jenkins yet
<wgrant> sinzui: It's in jenkins now.
<wgrant> AssertionError: Failure in lp/registry/javascript/tests/test_structural_subscription.html.test_title_ellipsisification: test_title_ellipsisification: failed.
<LPCIBot> Project windmill-devel build #169: STILL FAILING in 45 min: https://lpci.wedontsleep.org/job/windmill-devel/169/
<wgrant> Aha.
<sinzui> wgrant: test_title_ellipsisification failed because the header and title were reversed. I think I fixed that
<wgrant> Apparently not :(
<sinzui> wgrant: oh good, it does not have my branch yet
<wgrant> sinzui: Hm? Build #169 had your "Use standard events [...]" rev.
<sinzui> wgrant: I do not see how test_hide_comment and test_personpicker can reliably work since they are missing the complete signal in the test
<wgrant> Heh
<sinzui> Looks like the complete signal is broken in test_picker after a merge
<nigelb> heh, I did think that lifeless's first mail looked sparse.
<jtv> hi wallyworld_!
<jtv> Any reviews today?
<wallyworld_> jtv: hi. there's one so far and one oops tools branch i'm not too familiar with so it's hard to offer anything constructive
<jtv> In that case, ask lots of questions!
<jtv> You'd be amazed how smart you can look by asking the right stupid question.  :)
<jtv> Ever read Surely You're Joking Mr Feynman?
<wallyworld_> jtv: ok. the review requested it be done by a member of the oops tools dev team. is it ok that i do it?
<wallyworld_> no, haven't read that :-)
<jtv> I don't think we're part of the oops tools dev team, no.
<jtv> You must read Feynman's autobiographies.  He worked on the Manhattan Project.
<wallyworld_> sounds interesting
<jtv> He was a bit of a junior type then, but when it was discovered that due to army secrecy rules, the uranium factory hadn't been told about things like "for heaven's sake don't stack it all up, and not on the other side of the wall either!" they just sent him.
<wallyworld_> hah
<jtv> So there he went, young scientist, knowing nothing of industrial processes and construction.  "What, me, little Richard!?" â "Yes you, little Richard."
<jtv> And at the factory, he's the first Manhattan Project scientist they ever see.
<jtv> So they think he's some kind of inhuman genius who knows everything.
<jtv> And they show him the blueprints to the factory, in rapid-fire succession.
<jtv> He thinks he recognizes a recurring mark on the blueprints.  May be a valve.
<jtv> But by that time he's several blueprints in, so he can't very well ask "so what's this then, a valve?"
<jtv> Instead, he bluffs.
<jtv> Points at one, goes "what happens when this valve here gets stuck?"
<jtv> âexpecting them to say "erâ¦ Mr Feynman, you know that's a door, right?"
<jtv> Instead, they go "well, in that caseâ¦"
<jtv> (trace conduit onto other blueprint)
<jtv> (scratch heads)
<jtv> (argue)
<jtv> (sweat)
<jtv> "Gee Mr Feynman, you're right!  When that valve blocks, everything goes!"
<jtv> Anyway, better let the oops-tools review lie then.
<jtv> At least we got a nice story out of it.  :)
<wallyworld_> jtv: as long as he didn't say "wonder what this big red button does, let's find out..."
<jtv> Well he did some of those in his life as well, in his way.
<jtv> He also came up with this neat trick:
<jtv> Science conference.  Late to book.  All hotels are full.
<jtv> So he booked himself and another professor into a room at a hotel that, er, specialized in short stays.
<jtv> As in, "you want to stay a _whole_ _night_!?"
<jtv> wallyworld_: what's the last you saw?
<wallyworld_> jtv: last review request?
<jtv> No I mean, what was the last thing you saw before your IRC connection blinked?
<jtv> StevenK, wgrant: mind if I delete all DSDJobs on dogfood and restart the app server?
<wallyworld_> jtv: it's been blinking all day for some reason :-(
<wallyworld_> the last thing was At least we got a nice story out of it.  :)
<jtv> Probably because I gave up on wifi and stopped blinking.  Someone has to do it.
<jtv> Ah OK, I'll leave the other story for another time then.  Or you can read the book.  :)
<wallyworld_> s/book/ebook
<wgrant> jtv: I just killed the appserver and deleted all the PackageCopyJobs to let the DB patches apply. I'm dealing with the DB directly now, so you can start it up.
<jtv> wgrant: can you delete all dsdjobs while you're at it?  Or I can do it if that's more convenient.
<wgrant> jtv: I'm currently in the middle of fti.py, so not really convenient for me to do that right now.
<jtv> OK I'll do it
<jtv> And done.
<jtv> And I'll start the app server.
<wgrant> Hm, doesn't seem to be running?
<wgrant> Oh, you're making?
<jtv> Trying.
<jtv> Failing.
<jtv> KeyError: u'../images/mute.png'
<jtv> A file that does seem to exist â but not in branches on my own machine.
<wgrant> That was added recently... what was it that filed? Spriteutil?
<wgrant> failed.
<wgrant> HM, indeed.
<wgrant> jtv: Works now.
<wgrant> jtv: I moved icon-sprites.positioning out of the way.
<wgrant> It rebuild it and completed OK.
 * wgrant makes start.
<wgrant> It's up.
<jtv> Thanks.
<jtv> How did you know how to fix that?
<wgrant> I saw the new sprites added this morning, guessed that one of the caches was out of date.
<jtv> Then I wonder why "make" didn't see that.
<wgrant> No idea.
<wgrant> Our Makefile is a bit crap.
 * wallyworld__ just had a kernel panic - lost a big loong email :-(
<wgrant> jtv: I need to bounce the appserver soonish.
<jtv> wgrant: fine with me
<wgrant> jtv: It's back.
<jtv> thz
<jtv> wgrant: mind if I sync some packages?
<wgrant> jtv: Not at all.
<jtv> Then let's find out if I broke that.
<wgrant> wallyworld__: I've fixed the project picker to mostly work, I think.
<wallyworld__> wgrant: excellent
<wgrant> wallyworld__: Have a look at https://bugs.dogfood.launchpad.net/launchpad/+bug/1234. It now uses icons, ranks properly, and doesn't search description, and puts exact name matches first.
<_mup_> Bug #1234: Gina is an unmaintainable mess of command line options, environment variables and shell scripts <lp-foundations> <Launchpad itself:Fix Released by debonzi> < https://launchpad.net/bugs/1234 >
<wgrant> So it's a bit less unusalbe.
<wgrant> Although still ugly because of lazr-js's icon positioning.
 * wallyworld__ looks
<wallyworld> wgrant: i'm searching for "firefox" and it's been going over a couple of minutes now with now result
<wgrant> wallyworld: Took 3.2s for me.
<wallyworld> hmmm. i'll try again
<wgrant> DF is not the most stunningly reliable entity in the universe.
<wgrant> We possibly still want affiliation.
<wallyworld> yeah the gods are against me today. still no luck
<wgrant> But like with team search, they are far more likely to be unique and searchable.
<wgrant> Huh.
<wgrant> Does 'launchpad' work?
<wallyworld> i'll look
<wallyworld> works in a fashion - "Too many matches, please narrow your search"
<wgrant> Which picker is this?
<wallyworld> project picker on https://bugs.launchpad.net/launchpad/+bug/1234/+choose-affected-product
<_mup_> Bug #1234: Gina is an unmaintainable mess of command line options, environment variables and shell scripts <lp-foundations> <Launchpad itself:Fix Released by debonzi> < https://launchpad.net/bugs/1234 >
<wgrant> WTF, which one is that.
<wgrant> It's a different one.
<wgrant> Try the one in the task row on the bugtask index.
<wgrant> While I track down why we have two product pickers.
<wallyworld> one uses "project" the other "product"
<wallyworld> in the title
<wgrant> Oh, no, they are the same vocab.
<wgrant> But you were trying on production...
<wgrant> And yes, they have different titles.
<wgrant> That's fixed locally.
<wallyworld> no luck with the dogfood one one the bagtask row
<wgrant> Sure it's dogfood this time? :)
<wallyworld> might just be me
<wallyworld> yep :-)
 * wallyworld has to run away for a bit to pick up the kid and buy dinner. wife is out so i can get whatever junk food i want :-D
<wgrant> Heh
<wallyworld> mmmm. lasagne sounds good
<wallyworld> wgrant: i've also found a couple of person picker implementations :-/
<wgrant> wallyworld: There are a few pillar pickers.
<wgrant> Hm.
<wgrant> Now the product picker is hanging for me too.
<wgrant> Except it also hangs Chromium's developer console.
<StevenK> wallyworld: How is lasagne junk food? :-)
<wgrant> stub: We can't do FTI column rebuilds live, can we?
<wgrant> wallyworld (when you're back): I tried a couple of quick changes to make the picker less ugly: http://people.canonical.com/~wgrant/launchpad/project-picker/before.png vs http://people.canonical.com/~wgrant/launchpad/project-picker/after.png
<wgrant> StevenK: What do you think?
<wgrant> It's possibly less easily scannable due to the lack of indentation on the summary.
<wgrant> But I think it's better.
<StevenK> I think slangasek has been asking me hard questions.
<wgrant> Oh?
<wgrant> Ah, there.
<StevenK> wgrant: I prefer after
<StevenK> It looks cleaner, but I share your concern.
<wgrant> But the old one is fugly, so meh.
<jtv> wallyworld: with a heavy heart I voted Disapprove on that MP.  By the way, when you do a "code*" review, please always point out what the asterisk means!
<nigelb> jtv: yeah, I noticed.
<nigelb> jtv: I agree with your reason to disapprove, but this is part of 2 bugs I fixed sorting.  This means both of them needs to be re-reviewed.  The other one landed in production already.
<wallyworld> jtv: i had no idea sorting was so involved
<jtv> nigelb: I'm writing that email at the moment.  It'd be nice to have this solved.
<wallyworld> wgrant: i like the after shot - the way the description text is left aligned reall yhelps
<wgrant> wallyworld: I'm also getting rid of the margin at the bottom.
<wgrant> Needs some lazr-js CSS changes :(
<wallyworld> wgrant: i think the preferred approach is to override the styles in lp
<jtv> wallyworld: there's some weird stuff out there.  Letters that you convert to upper-case and back to lower and they're not what you started with; scanning the string backwards for tie-breakers; letters that change position in the alphabet based on use-case; letters that need to be read in one order, written in another order, and sorted in yet another one.
<wgrant> wallyworld: Probably.
<wallyworld> jtv: so what's best preactice for sorting?
<nigelb> I'm curious though, how'd we solve this for bug subscriptions. Aren't they also sorted alphabetically?
<jtv> wallyworld: and my all-time favourite: i/I are not a lower-case/upper-case pair in Turkish.
<wallyworld> wgrant: doing it that way was what curtis wanted for the badge styling
<wgrant> wallyworld: Yeah, but this is probably slightly different.
<wallyworld> jtv: wtf? how so?
<wgrant> wallyworld: anyway, I can probably override this in LP using :empty.
<wallyworld> wgrant: you then don't need to do a lazr-js mp :-)
<wgrant> style-3.0.css makes me sad.
<wallyworld> me too
<jtv> wallyworld: there's no single solution, and the partial solutions all basically sort of hope that you won't be mixing languages.  Good technical practice includes "don't try to deal with letters as such because you can't imagine the horrors in the dungeon dimensions beyond ASCII," "don't think you understand what .upper() and .lower() do," "don't talk to foreigners," and "let somebody else take care of it."
<nigelb> Note to self: Don't touch sorting bugs.
<wallyworld> i like the last one best :-)
<wallyworld> nigelb: it all seemed so easy before jtv pointed out a few realities :-)
<jtv> wallyworld: oh, about Turkish?  They have a sensible but incompatible system where i/Ä° form a pair, and there's another pair without the dot that I can't figure out how to type right now.  Note how in both pairs, one letter is ASCII and the other isn't.
<nigelb> wallyworld: True that.  I only just fixed one bug this way because it was tagged easy and then I went ahead and fixed specifcations too
<nigelb> Now I have to re-fix the sorting bugs I've touched
<nigelb> After we bikeshed on the best way to fix this :D
<wallyworld> nigelb: well whoever tagged it as easy knows as little as you and me about the problem :-)
<jtv> nigelb: /me nods emphatically :)
<nigelb> true, that.
<jtv> Somebody tagged this as easy?  Amusing.
<jtv> It _seems_ easy.
<nigelb> Not this.  Someone tagged sorting the sprints as easy
<nigelb> From there I found the specifications one
<wallyworld> jtv: so does keeping one's wife happy, but their both roads frought with danger
<nigelb> I fixed both but without the lower(), so it wasn't really fixed
<wallyworld> s/their/they're
<nigelb> So, I refixed sprints with .lower() and this was an attempt to fix specifications
<nigelb> Now I have to do this all over again :)
<wgrant> jtv: For a proper solution we surely need to define a collation for each text field, and somehow use them all together nicely.
<wgrant> That sounds like fun.
<jtv> wallyworld: funny you should mention that, given that mine was born with a different script than I, but bilingual where one language is "officially" written in a script I had to teach her.
<jtv> wgrant: not sure that doing it per field is the solution.  I suspect we'll always have a bit of a weird mix, with some ordering coming out of the DB and other ordering happening in Python.
<jtv> postgres now supports per-column collation, but I was one of the ones who did not support that idea.
<jtv> I feel that basically, sorting for display and sorting for computer use are different tasks just like e.g. rendering integers as strings for both are.
<jtv> But does anyone listen?  No, and I can't blame them.
<wallyworld> wgrant: do you hear anything?
 * jtv presses caps lock and prepares to repeat
<wallyworld> :-P
<jtv-aol> I SAIDâ¦
<jtv> nigelb: truly sorry to be such a spoilsport; on the bright side, I hope we can get a better solution.
<nigelb> jtv: that's what I'm thinking too :)
<jtv> wallyworld: fwiw Turkish is apparently sort of the standard smoke-test language for locale handling.  It's got just enough differences from ASCII to trip you up, but not enough to be completely alien to programmers who use mostly ASCII.
<nigelb> jtv: I don't mind refixing things if its "THE FIX" :)
<jtv> The email just went out.
<wallyworld> hah. time to learm turkish
<wgrant> Yay, qbzr works again.
<StevenK> nigelb: THE FIX, NO I REALLY MEAN IT THIS TIME.
<lifeless> jtv: fwiw there is other code with the collation limit defect you point out
<jtv> Limit?
<nigelb> StevenK: Yeah, that one.  It would be the third fix for this bug :)
<wgrant> lifeless: I think it's possibly everything else in Launchpad ever.
<lifeless> jtv: so I don't think blocking this localised improvement for a global one is necessary
<wgrant> wallyworld: extra-form-buttons... do you know why the height: 3em is there?
<wallyworld> is stles-3.0.css?
<wgrant> Yes.
<wgrant> You added it in 12641.6.30
<wallyworld> wgrant: not sure now. to make it work :-)
<wgrant> I guess I'll just remove it and check both users.
<jtv> lifeless: we could go there, but then we'd want to make the sort order determinate again, and before you know it we're still re-inventing the wheel.
<lifeless> jtv: huh?
<wallyworld> wgrant: sorry i can't recall exactly. i guess without it it would have rendered too squashed or something
<lifeless> jtv: I don't follow
<wallyworld> or too tall
<lifeless> jtv: we have *identical* (barring variable names) code to the code nigelb proposed elsewhere.
<lifeless> jtv: how is accepting his patch tied to wheel reinvention ?
<nigelb> lifeless: sprints
<jtv> lifeless: sorting a bunch of strings is hairy, but determinate.  Converting to lower case and then sorting is simple, but indeterminate.  It's not disastrous, but if you test for sorting "a" vs. "A"â¦
<nigelb> at least that I know of.
<lifeless> jtv: I accept that assertion. I don't follow the chain of causation
<lifeless> nigelb: loggerhead, mailman, sprints, persons, branches
<lifeless> at least
<nigelb> woah.
<nigelb> that's a lot.
<lifeless> maybe
<lifeless> 7 call sites a *truely trivial* grep found
<lifeless> I didn't look for multiline sorts
<lifeless> so I'd expect more - maybe 15 or 20
<lifeless> not huge in the scheme of things, but there may be more where its not obvious that we've done a lower()
<nigelb> hrm, just a doubt though.  How often are the strings of the kind that we have a problem with?
<nigelb> Like, are all the fields UTF-8 in the db?
<lifeless> most are yes
<lifeless> some have narrower constraints
<jtv> There's plenty of ascii-only identifiers, even if the columns may be Unicode.
<lifeless> dispaly names are generally free form, the url components are all tightly limited subsets of ascii
<lifeless> anyhow
<lifeless> its 6:30 pm
<lifeless> I'm not really here
<lifeless> I just don't think we should block a temporary improvement for a larger change we haven't yet made that may require reversing this patch at that later date.
<lifeless> unless it would be *worse* today, (in which case its not a temporary improvement)
<wgrant> wallyworld: Ah, turns out you just converted it from inline styles.
<wallyworld> wgrant: right. makes sense
<jtv> lifeless: it's better in some ways, worse in others â and to some extent we just don't know.  General good practice says "don't do this."
<jtv> But see the mailing list.  Martin's got some good comments.
<nigelb> so we write a wrapper and then fix this problem systematically?
<nigelb> fix the wrapper step by step I guess
<stub> wgrant: We can. What fti change where you thinking of?
<wgrant> stub: Dropping product.description from product.fti.
<wgrant> AFAICT it's as deprioritised as it can get, and it's still bad.
<wgrant> Dropping it yields better results.
<wgrant> name/displayname/title/summary should cover all the important data.
<wgrant> description is more of a homepage.
<stub> Why is converting a string to lowercase and sorting indeterminate?
<stub> I don't think any lower() functions invoke random()
<wgrant> stub: lower('foo') == lower('Foo')
<stub> ok
<stub> So the correct thing to do in most cases would be to sort by (lower(foo), foo)?
<jtv> Well it wouldn't work for Thai, for instance.
<lifeless> or not worry
<stub> or in en_GB locale we trust?
<jtv> Exactly.
<jtv> If we use en_GB, or en_US, it's fine.
<stub> jtv: Why would that give indeterminate results with Thai?
<jtv> But currently we seem to use C, which isn't.
<jtv> stub: not indeterminate in that case, just wrong.
<stub> Sure, but any collation will be wrong when sorting a set of strings in mixed languages, which is what Launchpad needs to do most of the time. So we have to make a best attempt.
<jtv> Yes, and I think the best attempt is what you say: use en_GB or some other human-readable locale.
<jtv> Instead of trying to simulate its properties by hand.
<stub> en_GB is just insensitive sorting or maybe sorting on (lower(foo), foo) under the hood, or at least it seems that way under Linux.
<jtv> It also falls back on unicode collation.
<stub> but using locales for sorting was bad in the past, because they only cared about their native alphabets and you ended up with stupid things like 'all japanese strings compare equal'
<jtv> en_US.UTF-8 gives me correct sorting for a bunch of non-English strings:
<jtv> à¸à¸²  à¹à¸  à¸à¸²  à¹à¸  a  A  b  B  Î  Î¾  Î©  Ï  Ð±  Ð  Ð³  Ð
<wgrant> wallyworld__: Does U1 still use lazr-js, or are we down to just L[SP]?
<stub> Cool. If they have fixed that now, we should probably switch to en_?? locale when we migrate to PG 9.x
<jtv> What's the connection to the PG upgrade?
<jtv> (I forgot most of this discussion)
<stub> jtv: We have to rebuild all the databases etc.
<jtv> Why?
<stub> jtv: I'd rather do that once rather than twice :)
<jtv> Indexes?
<stub> Actually, we might not have to rebuild if pgmigrator works for us. But we still need to wait if we go that route as we would need locale specific indexes rather than rebuilding the database in a particular locale.
<stub> So we either dump and restore with a new locale, or rebuild all our indexes with our preferred locale and keep the DB itself in the C locale. I think :)
<jtv> Could we get away with changing the python collation separately?
<stub> I don't think we want to do all our sorting client side.
<jtv> No, but I wonder how much user-visible sorting of freeform strings we do in the DB.
<stub> Most places it won't matter as the field we are sorting on is restricted to lowercase ascii.
<jtv> Exactly.  We care here about user-visible sorting of freeform strings.
<stub> So maybe we should stick to C locale in the DB for speed :-) (although we would need to measure that - what was slower 5 years ago may no longer be the case)
<jtv> Maybe this is actually one of those applications for per-column collations.  :)
<jtv> (But that's 9.0 I think)
<jtv> I suspect fully-collated sorts will still be slower, but maybe we should treat DB sorting more as being for machine consumption.
<jtv> With a few exceptions such as batching queries.
<stub> wgrant: That one is a 'possibly, but very problematic'. We need to drop and recreate a trigger, which is fast enough if we manage to grab a lock but grabbing the lock is problematic. And then we need to apply a db patch *to the slaves only* during the next rollout that updates the trigger.
<lifeless> wgrant: relatedly isd have deropped lazr.restful
<wgrant> lifeless: Oh really?
<lifeless> jtv: stub: I think we could tolerate different collations in db and python - for a given dataset we're only going to sort in one of (db, python) I expect.
<stub> jtv: I think we will shoot ourselves in the foot if people need to remember to set the default collation order before issuing a query, and I'm not sure if it is possible to do something like ORDER BY name IN LOCALE C, displayname IN LOCALE en_US (or if we would want to do that)
<jtv> I agree â I think any problems from that are going to be more along the lines of the symptom that led to nigelb's patch.
<stub> lifeless: Yes. Apart from case insensitive, the vast majority of the time nobody will even notice.
<stub> jtv: Which I haven't followed :)
<jtv> stub: see launchpad-dev list
<jtv> I suspect we're going to run into this very rarely with database queries: we tend to rely on different orderings there anyway.
<stub> Ordering by distribution may be an issue when we have lots from derived distros (AAALinux). Person.displayname is the only other issue, and we should really be ordering using the functional index for that purpose to strip out punctuation etc. (>> l33t d00d <<)
<lifeless> yes
<lifeless> however
<stub> If it is really only those cases, yer. I guess stick with C locale and override for the rare cases.
<lifeless> subscribers-to-an-object - small #
<lifeless> person vocabs - huge #'s
<lifeless> huge #'s we need db sort for
<stub> Functional indexes win there if they have fixed the bug I reported in... erm... 8.1? 7.4?
<wgrant> Grar embedded copies of YUI.
<wgrant> STAB
<stub> (ORDERING BY displayname_sort(displayname) would hit the functional index, but there is a deeply embedded but that caused displayname_sort(displayname) to be reevaluated for every retrieved row.
<stub> Need to retest with modern PG...
<lifeless> stub: it will still do that
<lifeless> stub: remember the performance hit on package versions functional index
<jtv> That overhead of functional indexes still hasn't gone away?
<jtv> Or is there a gotcha like "you must declare your function as pure or else"?
<lifeless> I think its tied to the same node-may-differ thing that makes from-index queries impossible
<stub> jtv: I need to check. It was confirmed as a bug, but deeply entrenched and Tom wasn't sure how to fix it. It was still there at least one release later.
<stub> (check by creating a stable function with a RAISE NOTICE in it, and seeing if the messages are spit out to the logs when queries are hitting the functional index)
<stub> lifeless: Thinking of PG 9.1 rather than 8.4.
<stub> Lots of juicy bugfixes and features coming up
<jtv> I wonder if maybe the re-evaluation is tied to the liveness check (where you need to access a tuple in the table just to see whether it's visible, even if the index has everything else you need)
<lifeless> stub: ah yes
<lifeless> jtv: thats what I was referring to ;)
<jtv> Oh
<lifeless> jtv: I forgot the terminology
<stub> jtv: I don't know. My eyes just glaze over at a certain point :)
<jtv> lifeless: that's alright, I don't know the terminology either
<jtv> Oh blast.  PQM merge failure: test_bug_views.py
<wallyworld__> wgrant: i thin kU1 may use lazr-js but not sure, sorry
<wgrant> wallyworld__: You broke the lazr-js build :(
<wgrant> Traceback (most recent call last):
<wgrant>   File "bin/yuimeta", line 17, in <module>
<wgrant>     import lazr.js.meta
<wgrant>   File "/home/wgrant/Development/lazr-js/trunk/src-py/lazr/js/meta.py", line 9, in <module>
<wgrant>     from lazr.js.build import SRC_DIR
<wgrant> ImportError: cannot import name SRC_DIR
<wgrant> From a fresh make.
<wallyworld__> huh? works locally i swar
<wallyworld__> i'm using it in lp now
 * wallyworld__ checks
<wgrant> wallyworld__: This is outside LP.
<wallyworld__> hmm. ok. still waiting for editor to start. having computer issues today
<wgrant> Any hints on running its tests?
<wgrant> I s/SRC_DIR/DEFAULT_SRC_DIR/ and it built, but doesn't seem to work very well.
<wallyworld__> wgrant: that import is indeed wrong
<wgrant> wallyworld__: There are also some bugs in your new linkfication code.
<wgrant> And I think we might need to entirely customise the rendering anyway.
<wallyworld__> what bugs?
<jtv> gmb: you seem to have a devel/db-devel merge conflict with Gary.
<wgrant> wallyworld__: "<a class = '" + css + "' href=javascript:void(0)></a>");
<wgrant> wallyworld__: Odd spacing around the class, you're using string concatenation rather than addClass, no quotes around the href, and the href should probably be the real value.
<wgrant> li_title.appendChild('&nbsp(')
<wgrant> Entities need to end with a semicolon.
<wallyworld__> wgrant: i don't want href as the real value.
<wallyworld__> i use an onclick handler to open the link
<wgrant> Oh?
<wgrant> Sure.
<wgrant> But it's going to look bad in the status bar.
<wallyworld__> it all seems to work in practice
<wgrant> Having the right href there won't hurt, and it will make it clearer what it's doing. This point is arguable.
<wgrant> The quotes are not :)
<wgrant> I'm trying to think how to implement my prettification properly.
<wgrant> It needs the image to be part of the title a/span.
<wgrant> And needs a custom class on that a/span.
<wgrant> And needs the margin reduced.
<wgrant> Which seems incompatible with not breaking landscape.
<wallyworld__> wgrant: i've merged in the compile fix. i'll do the other things a bit later - i have to take the kid to his violin concert. lord have mercy on my soul
<wgrant> Haha.
<wgrant> Thanks.
<wallyworld__> my ears hurt already
<wgrant> (brave, letting a child learn the violin)
<wallyworld__> thanks for the input on the js
<wallyworld__> yep
<StevenK> I love the sound of the violin.
<jtv> Peacefully crackling in an open fire.
<StevenK> No!
<gmb> Anyone dealing with that PQM failure?
<wallyworld__> StevenK: i like the sounds of a violin played *well*. not by a 10 year old. have you ever heard a cat being strangled? it's worse than that. now off to my ears' implending doom
<gmb> Ah, looks like it's mine and Gary's code fighting.
<gmb> And jtv just saw the same thing.
 * gmb goes to fix it.
<LPCIBot> Project windmill-devel build #170: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill-devel/170/
<poolie> lifeless: so the reason i didn't truncate it is it wasn't obvious
<poolie> firstly if you can just truncate it and still attach it as a message
<poolie> secondly how to do that in python
<lifeless> well
<lifeless> perhaps I'm being naive
<lifeless> I would take the incoming mail as a string
<lifeless> take the first 64K which is pretty sure to get all the headers
<lifeless> and attach that as a original.txt file
<poolie> that could work
<poolie> avoids the issue of what would be valid as a mime message
<poolie> fyi it has already been stored to the librian by the time this crash occurs
<poolie> there might be some other higher limit there
<lifeless> it has ?
<lifeless> I looked at fromEmail which things it may not have been
<lifeless> librarian can store ISOs
<lifeless> brb
<gmb> Apologies for the continuing breakage on db-devel. I've fixed the conflict but am now getting test failures.
<gmb> Oh, obvious failure is obvious.
 * gmb facepalms, fixes.
<stub> I've been in Thailand too long. "I've not really succeed in describing it in understandable English."
<nigelb> So, what's the consensus on the sort thing that jtv raised on the list?
<nigelb> On reading the mail threads, I couldn't really find a consensus.
<stub> nigelb: I think consensus was that we should be running our Python stuff in a known locale and using locale aware sort routines.
<nigelb> stub: hrm, so I can act on that information and update my MP?
<stub> nigelb: Not sure how that applies to your immediate problem ;)
<nigelb> hah
<stub> Might want to confirm that my consensus is not just an over inflated opinion :)
<stub> jtv has dropped off though. lifeless might still be alive.
<nigelb> Nah, I think I'll pick this up again tomorrow morning
<nigelb> Or, reply to the thread :)
<wallyworld_> wgrant: the cleaned up js has now merged. it includes the change to show the href in the status bar
<wgrant> wallyworld_: Ah, great, thanks!
<wallyworld_> wgrant: well, i should never have did it like that in the first place
<wgrant> wallyworld_: http://people.canonical.com/~wgrant/launchpad/project-picker/after.png is what it actually looks like now (previous one was hacked in Chromium), but I need to work out how best to do it without breaking Landscape.
<wgrant> That will be a job for next week, I suppose.
<wallyworld_> wgrant: also, i am redoing the link rendering, but in a lp override of the renderUI method. the lazr-js version is a "well it works generically" thing.
<wgrant> wallyworld_: Ah, so we can easily override that?
<wgrant> wallyworld_: That will make things easier.
<wallyworld_> wgrant: yes, i wrote it so that it worked "out of the box" in lazr-js but easily overridden in lp
<wallyworld_> wgrant: the new screenshot loks good
<wgrant> Excellent.
<wgrant> Yeah, I removed the excess padding and restricted the description to one line and used the custom icon if present and aligned the icons like normal links
<wgrant> Looks a bit less scrappy.
<wallyworld_> wgrant: i'm doing a base lp picker implementation which overrides lazr-js picker and provides the extra rendering. person picker and friends will use that instead of the lazrjs one
<wgrant> wallyworld_: Great.
<wallyworld_> ui work takes to much time / tweaking :-)
<wgrant> It does, but it makes us look less pathetic.
<wallyworld_> can't argue there
<jelmer> mwhudson: thanks for the hints yesterday, managed to nail down the issue - properly implementing confirm_action seems to've done it.
<LPCIBot> Project windmill-db-devel build #360: STILL FAILING in 50 min: https://lpci.wedontsleep.org/job/windmill-db-devel/360/
* bac changed the topic of #launchpad-dev to: Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: wallyworld* (jtv), bac | Critical bugs:208 - 0:[######=_]:256
* bac changed the topic of #launchpad-dev to: Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: bac | Critical bugs:208 - 0:[######=_]:256
<deryck> Morning, everyone.
<flacoste> morning deryck!
<danilos> matsubara, hi, I wonder if you've managed to uncover any terrible issues with staging code regarding bug subscriptions? (as asked about by Gary)
<matsubara> danilos, I didn't start the ET yet.
 * danilos ponders a few seconds before he figures out that ET is extra-terrestrial... nor, ETA with a missing A... but exploratory-testing :)
<danilos> is not*
<danilos> matsubara, ok, thanks
<LPCIBot> Project db-devel build #609: FAILURE in 34 min: https://lpci.wedontsleep.org/job/db-devel/609/
<sinzui> wallyworld: I approved picker-click-item-detail
<jcsackett> sinzui: i saw the bug on yui tests; it's assigned to me now, and today is largely clear to take care of it.
<sinzui> jcsackett: I just forwared you an email. wallyworld is concerned that he isworking in lazr picker and you create a new one.
<wgrant> I am leaving my further picker improvements until one of your implementations wins :/
<sinzui> jcsackett: I think we want to investigate these failures: https://lpci.wedontsleep.org/job/windmill-db-devel/359/#showFailuresLink
<sinzui> I think we should invite deryckto adjudicate
<jcsackett> sinzui: agreed. also, i now need to check my source tree, because my windmill tests are not running.
<jcsackett> i ran layer=windmill and got no failures; this should have been picked up. :-/
<sinzui> jcsackett: did you see the tests actually run?
<jcsackett> sinzui: i saw many tests run.
<jcsackett> it looked like all yui/windmill tests had run and passed.
 * jcsackett considers whether the personpicker link in branch should just be rolled back.
<sinzui> Is saw lp/app/javascript/tests/test_picker.html pass, but it does not now. the test is different after the merge beause other people also worked on that file
<wgrant> sinzui: When I checked a month ago that passed everywhere except Windmill. Even in the same browser, it passed fine if not run under Windmill.
<wgrant> That may not still be the case.
<sinzui> bugger. I think I need to investigate pywebkit again as an alternative browser to run yuitest
<jcsackett> sinzui: which do you think works better for us? prepare a branch to fix stuff that may take a bit, or roll back the link-in?
<jcsackett> (assuming doing so fixes some of these tests).
<sinzui> jcsackett: does rolling back mean you concede your implementaion? Isn't your design the one recommended by deryck?
<jcsackett> sinzui: deryck thought that creating a proper widget would be easier than shimming picker.create. i think that's still true.
<jcsackett> i'm not rolling back the creation of the personpicker; just rolling back the changes that plug it into the UI, pending the problems brought up.
<jcsackett> alternatively, we just need more branches to patch these holes.
<jcsackett> but it would seem that there's a level of brokeness currently introduced we should rollback before plugging things back in.
<sinzui> jcsackett: lets rollback then
<jcsackett> sinzui: dig.
<jcsackett> sinzui: bit of time to chat? might be quicker to make sure i'm on the same page. :-)
<gary_poster> matsubara, hi.  did my email make sense?  any confusions?
<matsubara> gary_poster, yes, it made sense. I'm testing right now. found some small issues, I'll put everything on the wiki once I'm done
<gary_poster> awesome, thanks matsubara
<jcsackett> hm. sinzui, nevermind the rollback. i misread some email. the second person-picker branch didn't land, so anything wrong with the picker is not yet in the tree.
<jcsackett> or rather, not yet in the YUI in the tree.
<jcsackett> so i'll see about fixing some of this today in a new branch.
<jcsackett> lord, i fail at typing. "not yet in the web UI in the tree." there. that's right.
<sinzui> jcsackett: I cannot see anyone's typing errors :)
 * sinzui is not sure he see conjugation
 * jcsackett laughs.
<jcsackett> so, sinzui, have a bit of time to chat?
<sinzui> in a moment
<jcsackett> sure.
<sinzui> jcsackett: I can talk now
<bac> hi matsubara
<matsubara> hi bac
<flacoste> bac: do you feel like reviewing a largish branch (lots of delete on it)?
<flacoste> it's over 1500 lines
<flacoste> the core changes is less than 500 lines
<flacoste> a permission change that had a lot of tests fallout
<timrc> abentley, thanks for the quick turn around time on those api changes!
<abentley> timrc: no problem.  Enjoy.'
<matsubara> gary_poster, danilos: I broke the +subscriptions page heh
<danilos> matsubara, it probably wasn't you! ;)
<danilos> matsubara, (iow, we broke it :))
<danilos> anyway, I am out for the week, file bugs and I'll be looking at them on Monday
<matsubara> danilos, https://bugs.launchpad.net/launchpad/+bug/792445
<_mup_> Bug #792445: It's possible to create the same subscription over and over again <exploratory-testing> <story-better-bug-notification> <Launchpad itself:Triaged> < https://launchpad.net/bugs/792445 >
<gmb> Any bzr-savvy people know what this means? "bzrlib.errors.BzrCheckError: Internal check failed: Cannot add revision(s) to repository: missing referenced chk root keys: [StaticTuple('sha1:d34f39faee56e120a83e7cec4fe01c4fd5fd32f6',)]
<gmb> "
<gmb> It's what caused the latest build failure on db-devel.
<matsubara> gary_poster, I wrote down some of my findings in https://dev.launchpad.net/QA/ExploratoryTesting/BugSubscriptionFeature at the bottom (Second round of testing). there were 3 other issues when I did the initial testing but they were fixed after the staging update. I'm going for lunch now and will continue testing later on and file bugs, etc
<gary_poster> matsubara, cool, thanks
<gary_poster> subscribe/unsubscribe for anonymous is good catch
<gary_poster> Creating multiple identical subscriptions is not a problem in my book
<gary_poster> If you don't allow it, then you get into weird situations
<gary_poster> like disallowing modification of a subscription because it would make it just like an existing one
<gary_poster> and that might even just be a transitional step while you adjust things
<gary_poster> batching is a separate aspect of this
<gary_poster> Having many, many pertinent subscriptions is a broken use case IMO; supporting it gracefully is a low priority IMO
<bac> flacoste: sure, i'll be glad to look at it
<flacoste> bac: thanks a million
<flacoste> bac: https://code.launchpad.net/~flacoste/launchpad/bug-365098/+merge/63305
<flacoste> bac: important changes are in canonical.launchpad.security,  lib/lp/registry/model/sourcepackage.py, lib/lp/registry/interfaces/sourcepackage.py andlib/lp/registry/tests/test_sourcepackage.py
<bac> thanks flacoste.  just reading your MP
<flacoste> rest are tests fallout
<deryck> abentley, did you spend any interrupt time on the burn down lists today?
<abentley> deryck: no, not yet.
<deryck> abentley, I just caught them all up.  if you could take a quick peak before you EOD, they should be good for yellow squad next week.
<abentley> deryck: cool.
<deryck> thanks, man!
<flacoste> deryck: didn't somebody fix something similar to bug 410864 recently?
<_mup_> Bug #410864: 'choose a source package' suggests undefined when search has too many results <LAZR Javascript Library:Triaged> < https://launchpad.net/bugs/410864 >
<flacoste> or sinzui?
<deryck> flacoste, yes, henning did.
<flacoste> deryck: could he or you validate that this one is also fixed and mark it as so?
<flacoste> as such
<deryck> flacoste, sure.
<deryck> flacoste, I'll try to look today, and if not, ask him Monday.
<deryck> flacoste, looking at his diff, I would imagine it covers all pickers.  he changed picker.js in our tree.
<deryck> flacoste, yeah, I did some qa.  it's definitely fixed.
<SpamapS> flacoste: So, I can't file bugs against the packages in principia
<SpamapS> "haproxy" does not exist in Principia Ensemble. Please choose a different package. If you're unsure, please select "I don't know"
<SpamapS> flacoste: is this because its checking to see if there is at least one build?
<flacoste> SpamapS: yes, that's bug 781993
<_mup_> Bug #781993: cannot file bugs on source packages that have not been published <bugs> <escalated> <not-pie-critical> <principia> <stakeholders> <Launchpad itself:In Progress by flacoste> < https://launchpad.net/bugs/781993 >
<SpamapS> flacoste: "me too'd" and subscribed, thanks.
<timrc> lifeless, I submitted a merge proposal for #724740, I currently have cody-somerville taking a look, but is there someone within the LP org that will also need to review?
<_mup_> Bug #724740: setting a ppa private cannot be done over the API <api> <oem-services> <ppa> <Launchpad itself:In Progress by timrchavez> < https://launchpad.net/bugs/724740 >
<cody-somerville> timrc, I imagine lifeless might be sleeping right now.
<cody-somerville> timrc, However, the answer is probably yes.
<timrc> cody-somerville, he never sleeps
<abentley> timrc: if he's asleep at 7:37 his time, he's sleeping in.
<cody-somerville> I'm always asleep at 7:37.
<timrc> cody-somerville, that's because you go to bed at 5
<lifeless> it is saturday ;)
<lifeless> cats have been running rampant for an hour now
<lifeless> timrc: yes, one of the LP reviewers will need to review
<statik> lifeless: since you are here, a very quick question: the GPG validation microservice that you asked for prototype implementations of, I would like clarification of the exposed API
<lifeless> shoot
<statik> "takes crypt text in and returns the signer, cleartext". Does that mean that the service is expected to have the private keys available to it?
<lifeless> oh
<statik> i'm mixed up about whether this is a decryption service or a gpg validation service
<lifeless> so thats a lingo thing
<lifeless> the crypt text is the signed document
<statik> perfect
<statik> and the cleartext is the content that was wrapped in the signature?
<lifeless> or for detached sigs (but I don't think we do them today) it would be (the signature, text) tuple.
<lifeless> statik: yes, thats right
<statik> lifeless, thanks! I'll update the wiki
<lifeless> thank you
<abentley> lifeless: AIUI, documents can have multiple signed parts.  Maybe you want a list of signer/cleartexts?
<lifeless> abentley: for our use cases today I believe that that would be an error
<lifeless> they are - coc, source package uploads, control-of-signature proof.
<lifeless> s/signature/key/
<maxb> This is very weird. bac just closed bug 296365. The bug is linked to question 75760. LP sent me an email about linked bug status changing, but the textual message included in the email is not the one bac entered as comment #8, it's the one sinzui entered as comment #5 two years ago!
<_mup_> Bug #296365: oops when responding to merge proposal review via email <email> <lp-foundations> <oops> <Launchpad itself:Invalid> < https://launchpad.net/bugs/296365 >
<abentley> deryck: I'm doing the interrupt stuff now, and it's mostly very clean, but I think you forgot about RT.
<deryck> abentley, I looked.  it looked like all spam to me, and I didn't see instructions on how to deal with it....
<bac> maxb: that is odd
<deryck> abentley, so I made a note to ask mrevell next week.
<abentley> deryck: Oh.  I've been hitting "resolve" and setting the status to "deleted".
<deryck> abentley, ah, ok.  that makes sense.
<deryck> abentley, I'll do that as well from now on.
<lifeless> deryck: what was the relevation ?
<deryck> lifeless, that critical is not drop-everything-and-do-it-now but do-next.
<lifeless> right
<deryck> or rather do-fist when pulling new work
<lifeless> uhm, I guess we failed to really communicate that change
<deryck> yeah, I think so.
<lifeless> critical as drop-everying-work-24-hours doesn't really make sense
<deryck> I sort of knew it, but it didn't make sense.
<deryck> we've ingrained critical as don't-stop-til-it's-fixed work for so long, it's hard to get free of that.
<lifeless> fwiw as I understand Ubuntu's use of critical it is identical to ours
<lifeless> *incidents* are don't stop :)
<lifeless> but they are often not code bugs
<deryck> it makes perfect sense when you realize the high bugs list is so long it's useless.  you need a real high category.  that's how I think of it now.
<lifeless> yeah
<deryck> useless is a bit strongly worded, but I hope you get my meaning
<lifeless> does this interact with that hi pressure discussion we had last week?
<deryck> yes, I feel better about marking ui regression bugs critical now, given that understanding of critical.
<lifeless> cool
<deryck> I would bet I am not allow in lp devs in not understanding this, too.
<deryck> I think about bugs and statuses a lot and missed the direct meaning.
<lifeless> we updated our docs when we shuffled this around back in oh feb
<lifeless> perhaps you could check that its clear enough to carry the message, and perhaps even mail the list with pointers ?
<lifeless> I'd hate for other folk to be unnecessarily stressed
<deryck> sure.
<deryck> I think its the way "burn the list to 0" meshes with the status that is confusing.
<deryck> we beat the drum of clearing the list, as if this is don't-stop work, but in reality we're queuing up the next queue.
<deryck> now I understand the next queue needs to be small enough to manage.  I'm not arguing over that.
<deryck> and FWIW, people don't get meaning from wiki pages much.  it's more the day to day conversations.
<lifeless> So the idea is that normally there will be approximately nothing in critical
<lifeless> but if we have to queue jump some task, it will go into critical
<lifeless> and get picked up by the next free engineer in a maintenance squad
<deryck> sure, I get that.  but that's more in line with a stop-the-line and do this work flow, then what we have now.
<deryck> it's the world 3-5 months from now, not now. ;)
<lifeless> right
<lifeless> we gotta clear the decks
<deryck> sure, agreed.
<lifeless> lots of bird poop piled up :)
<deryck> indeed :0
<deryck> :)
<flacoste> SpamapS: nice blog post on ensemble!
<flacoste> we now really need to fix these LP bugs as people will report bugs on principia!
<SpamapS> Heh, yeah.
<SpamapS> Maybe I should change the pre-bug-report text to make it clear you can't report against a formula yet?
<LPCIBot> Project windmill-db-devel build #361: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill-db-devel/361/
* bac changed the topic of #launchpad-dev to: Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: - | Critical bugs:208 - 0:[######=_]:256
#launchpad-dev 2011-06-04
<nigelb> lifeless: erm, how do I deal with that branch now?
<lifeless> nigelb: I like the idea of having a helper which we can make better
<lifeless> nigelb: there are some various utility modules around, have a look and see if you can find one to move the sort logic too
<lifeless> then use a helper
<nigelb> okay
<nigelb> this means we've have to move all of the sorts into using the helpers, right?
<lifeless> theres what, 3 so far ?
<lifeless> alternatively
<lifeless> file a bug saying 'we collate in C order'
<lifeless> and reference that bug as an XXX: bug 12345 nigelb this collates using ASCII which is not as nice as unicode collation.
<_mup_> Bug #12345: isdn does not work, fritz avm (pnp?) <isdnutils (Ubuntu):Fix Released by doko> < https://launchpad.net/bugs/12345 >
<nigelb> okay, yeah makes sense
<lifeless> either way is fine with me
<LPCIBot> Project db-devel build #610: STILL FAILING in 5 hr 37 min: https://lpci.wedontsleep.org/job/db-devel/610/
<wgrant> lifeless: Bah, pyunit-friends don't support python3 yet.
<lifeless> wgrant: FSVO
<lifeless> wgrant: testtools does, subunit does, fixtures does.
<wgrant> lifeless: Ah, testtools' LP page doesn't mention 3.x support, but I see on PyPy it does.
<wgrant> PyPI
<wgrant> (although it isn't tagged as such)
<lifeless> patches gratefully
<wgrant> Shh
<LPCIBot> Project windmill-db-devel build #362: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill-db-devel/362/
<lifeless> rsyncing a no-op change to an archive: 16MB
<wgrant> lifeless: The whole primary archive?
<lifeless> the one carried by nz.a.u.c
<wgrant> Ah
<lifeless> rsync://nz.rsync.archive.ubuntu.com/ubuntu
<lifeless> so i386, amd64, oni etc
<lifeless> sent 16256452 bytes  received 31851 bytes
 * lifeless sighs @ btrfs
<NCommander> evening wgrant
<NCommander> Was SUPERSEDED supposed to actually be in the indices are some point? I found a comment in the donminator.py that it was planned to hold the binaries in place, but it suggests they would remain SUPERSEDED
<NCommander> (dominator.py:196)
<NCommander> nm, disregard that
<NCommander> So it appears Soyuz is actually keeping the binaries around in a SUPERSEDED state until the arch-all/arch-any skew is resolved, or a ne wSPR is uploaded
<NCommander> https://launchpad.net/ubuntu/+source/emacs23/23.3+1-1ubuntu1
<NCommander> but since SUPERSEDED isn't in the indicies, its effectively doing nothing beside taking up disk space
<NCommander> ^- wgrant StevenK
<wgrant> NCommander: What do you mean? Superseded is a final state.
<wgrant> They will stay there forever.
<NCommander> wgrant: right, but the dominator won't mark those packages for death
<NCommander> its already checking the condition I need to check for, its simply doing the wrong thing
<NCommander> I can refactor the code to keep these packages from being SUPERSEDED in the first place, but I can't understand why it exists unless the behavior of SUPERSEDED changed sometime in LPs development history
<wgrant> NCommander: There's a comment there describing the behaviour that you desire, but it's not implemented.
<NCommander> wgrant: no, it is
<wgrant> Where?
<NCommander> Its preventing packages in groups from being deleted (deathrow is not set)
<wgrant> Where?
<NCommander> I posted the link to a source that FTBFSed on ARM 7 days ago, but the i386 and other arches are being retained
<NCommander> bah
<NCommander> hold on, I need an unedited copy
<wgrant> It does that for sources.
<wgrant> Not binaries.
<NCommander> Binaries get superseded only when multiple versions of them exist
<wgrant> Yes...
<wgrant> That is the definition of superseded.
<NCommander> bah
<NCommander> ok
<NCommander> now I see it. So its close to what I need, I just need it to retain the arch_all binaries
<NCommander> (or more specifically, the nominated_arch binaries)
<wgrant> I think the correct thing to do is to do something just like the source check.
<wgrant> But that only works if we subvert Julian.
<NCommander> wgrant: Julian said it was acceptable to have multiple PUBLISHED records, just publishing SUPERSEDED was a nono
<wgrant> Yes, but that's expensive.
<wgrant> Because it means we have to analyse everything in groups.
<wgrant> Not just stuff in SUPERSEDED.
<wgrant> Although, having multiple PUBLISHED records long-term is just strange.
<NCommander> We're already determining all the BPPHs per SPPH. The cost as it was is already being paid
<wgrant> Only for superseded SPPHs.
<NCommander> If SPPHs are checked first, BPPHs can be removed from the supersedication queue. That being said, I still feel its clearer to publish SUPERSED or implement a new publishing state
<NCommander> YMMV
<NCommander> The only other idea I can have is instead of have the packages go SUPERSEDED back to PUBLISHED, but I suspect thats just crack.
<wgrant> SUPERSEDED stuff is often left published on disk. It's no stretch to include it in the indices.
<NCommander> ^from
<NCommander> Plus it makes the indices match the pool. Consistancy for the win.
<wgrant> Yes.
<wgrant> Which makes elmo happy.
<StevenK> NCommander: "Consistency" has one e, and no a.
 * NCommander throws a brick at StevenK 
<StevenK> Whereas "pedant" has both.
<NCommander> StevenK: do you have any feelings w.r.t. to publishing SUPERSEDED in the indicies? (or another sane solution to the above issue on archive skew?)
<wgrant> It sounds counterintuitive. But they are on disk anyway.
<wgrant> We would keep sources in indices too, except apt didn't support that until recently.
<wgrant> Debian does it now.
<StevenK> NCommander: Personally, it does sound counter-intuitive.
<NCommander> am I the only one who was ever bugged by SUPERSEDED not being in the indices?
<wgrant> I have long found it odd that there is unindexed stuff on disk.
<StevenK> Keeping them PUBLISHED sucks, but so does putting SUPERSEDED in the indicies.
<wgrant> StevenK: Why does putting SUPERSEDED in the indices suck?
<wgrant> Debian does this now. The tools support it.
<NCommander> Well, the other idea I had was PUBLISHED_BUT_SUPERSED status, but I suspect adding a new publishing state to Soyuz would be painful
<wgrant> NCommander: Painful and entirely pointless.
<NCommander> right
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #611: FIXED in 5 hr 32 min: https://lpci.wedontsleep.org/job/db-devel/611/
#launchpad-dev 2011-06-05
<LPCIBot> Project windmill-db-devel build #363: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill-db-devel/363/
#launchpad-dev 2012-05-28
<wallyworld_> StevenK: remind me - are we eventually going to allow info type -> Proprietary via the API or will that always be verboten?
<StevenK> wallyworld_: We will allow it under certain conditions.
<wallyworld_> same conditions as for when using the web ui no doubt
<StevenK> It's just denied for all with a strange error message because we don't want to deal with Proprietary bugs yet.
<wallyworld_> yep, just wanted to double check, thanks
<StevenK> wallyworld_: I refuse to fix 1005325. We decided we didn't want to support Private over the API when I exported transitionToInformationType()
<wallyworld_> StevenK: i was thinking more that the error message is confusing since we don't support user data or proprierary yet
<wallyworld_> but i guess we can ignore that for now
<StevenK> wallyworld_: We do support User Data and Proprietary via the API. User Data will work, and show up as Private in the UI, and Proprietary will throw the terse error.
<wallyworld_> StevenK: i also was thinking that Private was to be supported as a pseudonym for User Data but as you remind me, not over the api
<wgrant> Right, it's just to avoid user confusion in the UI for now
<wallyworld_> ok. i'll mark the bug as invalid
<wgrant> API users already have to use tasks etc.
<wgrant> They can deal with a terminology difference, or continue using flags as they do already.
<wgrant> wallyworld_: I wonder if the trailing underline is related to the odd display issues in Opera
<wgrant> A box appears after the picker activators, like it's trying to render the nbsp as a character
<wgrant> It's very odd.
<wallyworld_> wgrant: not sure. could be by the sounds of it
<wallyworld_> worth fixing i think
<wgrant> The tree used there is a bit interesting.
<wgrant> Due to some old problems with KHTML and other browsers not showing them at all if the <a> didn't have some content.
<wgrant> And stuff like that.
<wgrant> lifeless: Are you sure the avahi instructions work?
<wgrant> lifeless: I'm pretty sure it'll only work if you're lucky and the container's avahi user has a UID not matching anything running any other processes on the system.
<wgrant> lifeless: Because the avahi initscripts set a user-wide limit of ~2 processes
<wgrant> (it certainly didn't work two months ago if you tried to run avahi in two containers where it had the same UID, or if the UID matched the host's)
<lifeless> the old instructions would work either
<lifeless> they assume persistent ip, which is in no way guaranteed
<lifeless> bah, english robert, english
<wgrant> In practice they're reasonably persistent.
<wgrant> And far better than avahi which in a lot of cases won't work at all
<wgrant> And when it does it's inconsistent at best.
<lifeless> did you record what you dug up ?
<wgrant> http://siphon9.net/loune/2011/02/avahi-setrlimit-nproc-and-lxc/
<lifeless> did you file a bug?
<lifeless> specifically, the uid counting across boundaries is unsafe - avahi can break e.g. postgresql in a container, that way.
<wgrant> Right, but user namespaces will make all this go away :)
<wgrant> And they're meant to be happening for quantal, AIUI
<lifeless> indeed
<wgrant> So it's really not an avahi bug
<wgrant> The nproc limit is probably a reasonable measure.
<wgrant> It's just the lack of user namespacing that's breaking everything.
<lifeless> yes
<wgrant> lifeless: Bug #1005330 -- any objections/opinions before I implement it in 4 lines?
<_mup_> Bug #1005330: Applications can't easily map an SSO account to a Launchpad one <api> <easy> <openid> <Launchpad itself:In Progress by wgrant> < https://launchpad.net/bugs/1005330 >
<lifeless> I dispute 'most correct way'
<wgrant> lifeless: 'most reliable and only vaguely official way'
<lifeless> multi-openid support, let alone browserid, is inherently hostile to using the ubuntu sso response at all :)
<wgrant> Sure
<wgrant> But there's currently no other reliable way :)
<wgrant> StevenK: Are you QAing the information_type thingy?
<StevenK> wgrant: I've just finished off the SPPH+PU QA, so that is up next, yes.
<lifeless> indeed
<lifeless> uhm
<wgrant> StevenK: Great.
<wgrant> lifeless: Yeah, anonymous is fine.
<wgrant> It'll crash and burn if it returns a team, since that's meant to be impossible.
<wgrant> No private people
<wgrant> And identifiers aren't secret
<wgrant> So there's no problem
<wgrant> lifeless: Is that your uhm?
<wgrant> Or is there something else?
<lifeless> it would be nice to ensure the team thing specifically
<lifeless> e.g. via a test or something
<wgrant> Maybe
<wgrant> I guess we can add a test now, since the DB doesn't forbid it
<wgrant> Will be a bit awful, but is easy
<bigjools> StevenK: now that you're setting pu on spph, there's probably a load of locations in the code that can be optimised to use it instead of joining/querying
<StevenK> bigjools: I should probably circle back and populate it for older SPPHs too
<wgrant> eg. changesfile
<bigjools> StevenK: yes, hell yes
<wgrant> Which currently is a hideous inaccurate query to guess the PU
<StevenK> bigjools: You say after objecting to it in the first place :-)
<StevenK> Hmmm, where is bazaar.qastaging.launchpad.net
<bigjools> StevenK: I still object
<bigjools> but since you've done it you can make it useful
<StevenK> bigjools: Right, fair enough
<bigjools> StevenK: another lovely garbo job? :)
<StevenK> Yup
<StevenK> Which will end being my fifth for the year or something
<bigjools> we need a bit of code that rejects bugs with the word "should" in the title
<lifeless> :)
 * dobey files a bug that launchpad "shouldn't" do that :)
<StevenK> wallyworld_: Can you delete the card for 1005325 too?
<wallyworld_> yep
<wgrant> lifeless: https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-b903a8b9cb0e54ccb8b4660ed77bb0f0
<wgrant> lifeless: Several implausibly long SQL statements on the same store on xmlrpc-private -- more evidence that we still have GIL issues with them
<lifeless> wgrant: we could partition the farms, or we could kill the xmlrpc internal socket
<lifeless> wgrant: (e.g. moving into the main socket and vhosting it up)
<wgrant> lifeless: Yeah, I think the latter is probably best, but that gets interesting.
<lifeless> or nuke all the internal users
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/set-spph-packageupload/+merge/107583
<wgrant> StevenK: That's incorrect.
<StevenK> wgrant: Which part?
<wgrant> StevenK: That finds any PU that references the source
<wgrant> StevenK: That should, in fact, crash sometimes
<wgrant> Because delayed coppies
<wgrant> For a delayed copy there will be multiple PUSes for the SPR
<wgrant> There's no foolproof way to do this.
<StevenK> Bleh
<wgrant> You sort of have to match the archive, suite and overrides.
<wgrant> So if (SPPH.archive, SPPH.distroseries, SPPH.pocket) match the PU, and (SPPH.component, SPPH.section) match the SPR, it's probably the one you want.
<wgrant> That will get the real uploads.
<wgrant> Won't handle PCJs or delayed copies -- they're probably less feasible and useful to populate, though
<StevenK> wgrant: http://pastebin.ubuntu.com/1010807/ with appropiate tests?
<wgrant> Your indentation is criminal again.
<wgrant> That's roughly correct, but it relies on my reasoning being flawless, which from 30s of thought it probably isn't.
 * StevenK picks on DF
<StevenK> wgrant: SELECT spph.id AS spph, pu.id AS pu FROM PackageUpload AS pu, SourcePackagePublishingHistory AS spph, SourcePackageRelease AS spr, PackageUploadSource AS pus WHERE spph.sourcepackagerelease = pus.sourcepackagerelease AND spph.sourcepackagerelease = spr.id AND pus.packageupload = pu.id AND pu.archive = spph.archive AND pu.distroseries = spph.distroseries AND pu.pocket = spph.pocket AND spph.component = spr.component AND spph.section =
<StevenK> wallyworld_: ^ And you *want* to learn Soyuz? :-)
<wallyworld_> not unless there's lots of beer involved
<bigjools> ah the soyuz query
<StevenK> wallyworld_: There has to be, one way or the other.
<wallyworld_> yep, either at the start or at the end :-)
<StevenK> Both has benefits, depending on which area of Soyuz
<czajkowski> morning
<bigjools> there has been a lot of beer, caffeine and pizza involved in writing soyuz
<StevenK> You forgot swearing
<StevenK> And Portuglish
<bigjools> and combinations thereof
<StevenK> wgrant: http://pastebin.ubuntu.com/1010839/ modulo working and fixing tests
<wgrant> StevenK: That's probably OK.
* bac changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 3.47*10^2
* benji changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: benji | Firefighting: - | Critical bugs: 3.47*10^2
<mgz> is there a bug entry for +bug/NNNN/+thing pages not having cross project redirects like +bug/NNNN pages do?
<mgz> can't find one with search
<mgz> example case:
<mgz> * user reports bug
<mgz> * triager changes bug project
<mgz> * user tries to add comment, gets 404
<cjwatson> I think my brain is atrophying; I just typoed getUniqueString as makeDistroSeries
<mgz> well, they have a similar rythym
<jml> those keys are right next to each other!
<mgz> ...and that was a creative misspelling
<cjwatson> I think what has happened is that I've internalised about half of LaunchpadObjectFactory and my hindbrain just produces a random method from it on request
<james_w> benji, thanks for the review of https://code.launchpad.net/~james-w/launchpad/suppress-notifications-for-all/+merge/107560 it seems that most of the changes you suggest were fixed in trunk in the meantime. I've made the one leftover change now.
<james_w> benji, would you land that branch for me please?
<benji> james_w: sure thing
<james_w> benji, thanks
<james_w> benji, not taking the day off today?
<benji> james_w: I swapped it.
<james_w> ah, ok
<benji> I did all my memorializing last week.
<lifeless> bac: around ?
<bac> hi lifeless
<lifeless> bac: I want to suggest you don't land your branch
<lifeless> the one with test ids being allowed to be more duplicative
<bac> which one?
<lifeless> bug 682771 and bug 682772
<_mup_> Bug #682771: test runner permits duplicate test ids <lp-foundations> <Launchpad itself:Triaged> < https://launchpad.net/bugs/682771 >
<_mup_> Bug #682772: doctest construction generates duplicate test ids <lp-foundations> <Launchpad itself:Triaged> < https://launchpad.net/bugs/682772 >
<lifeless> I had forgotten about these bugs, but they cover the situation
 * bac looks
<bac> hmm
<bac> lifeless: so doctest constructors like create_interface_test_suite would need to change
 * bac intercepts ec2
<lifeless> bac: testtools.clone_test_with_new_id can be used to assign a unique id
<lifeless> bac: create the test (duplicative id); then clone it to a new id, and add the resulting new-id test into the suite, let the original get gc'd.
<bac> lifeless: are you suggesting doing that in filter_tests or in the various test_doc files?
<lifeless> bac: I was assuming local-to-the-duplicativeness
<lifeless> bac: with a sanity-check in the global test_suite() constructor to enforce uniqueness
<bac> lifeless: that's what i thought but i read your statement the other way.
<lifeless> I think bug 682771 will be fixed if the e.g. ./bin/test --list-tests errors when the are duplicates, and bug 682772 will be fixed when the current duplicates are fixed up
<_mup_> Bug #682771: test runner permits duplicate test ids <lp-foundations> <Launchpad itself:Triaged> < https://launchpad.net/bugs/682771 >
<_mup_> Bug #682772: doctest construction generates duplicate test ids <lp-foundations> <Launchpad itself:Triaged> < https://launchpad.net/bugs/682772 >
<lifeless> you could do just 682772, but doing 682771 at the same time would give confidence that nothing has cracked through the slips
<lifeless> jelmer: ping (subunit)
<czajkowski> lifeless: he;s offline today on hols but has been peeping into irc
<lifeless> czajkowski: thanks
<czajkowski> lifeless:do you have a few mins spare so I can check stuff wiht you if you don't mind?
<jelmer> lifeless: pong
<jelmer> lifeless: subunit is building in the daily PPA these days
<lifeless> jelmer: cool - and the thing gary saw, where the built binaries use python3, but the python3 module isn't installed
<lifeless> czajkowski: sure, shoot
<czajkowski> lifeless: doing the oops summaries as digo is gone and just a bit um confused.
<czajkowski> https://devpad.canonical.com/~lpqateam/oops-summaries/production-2012-05-26.html
<czajkowski> looking at PortConnectionError: DTPFactory timeout  as an example
<lifeless> go on
<czajkowski> sp when I search for bugs with that title they don't exist, so I checked the oops and they aren't reported
<czajkowski> so do I created the bug based on that summary
<czajkowski> sorry laggy this evening I suspect housemate is downloading
<lifeless> what do you mean 'checked the oops and they aren't reported'
<czajkowski> well the way it was explained was I search for the bug summary and then compare the oops that have been reported
<jelmer> lifeless: I hadn't seen that one, looking at it now...
<czajkowski> if they're not report and link the oops to the bug
<lifeless> czajkowski: ok, so yes you search, if there is no open bug, then file one; if there is an open bug, don't bother linking the new oopses in unless the bug hasn't been touched in months - that would just be make-work.
<lifeless> czajkowski: the reports can be dug back through by developers, theres no need to keep touching the bugs
<czajkowski> lifeless: ah ok that wasn't explained
<czajkowski> good to know
<czajkowski> and thank you
<lifeless> also I wouldn't worry about being exhaustive
<lifeless> get the top 4-5 in timeouts, 4-5 in exceptions
<lifeless> by frequency
<lifeless> thats enough for folk to be getting on with.
<jelmer> lifeless: hmm, it looks like it defaults to python3 just on ubuntu and to python2 on debian
<jelmer> lifeless: I'll dig deeper tomorrow
<czajkowski> lifeless: cheers much appreciated
<lifeless> jelmer: yah, so this is the thing gary was complaining about :)
<lifeless> jelmer: thanks!
<jelmer> lifeless: IIRC he was complaining about it not building, which is fixed
<jelmer> I'm now looking at bug 1003910
<_mup_> Bug #1003910: subunit2junitxml failed on quantal: ImportError: No module named subunit.filters <amd64> <apport-bug> <qa-daily-testing> <quantal> <rls-mgr-p-tracking> <running-unity> <subunit (Ubuntu):Triaged> < https://launchpad.net/bugs/1003910 >
<lifeless> jelmer: bug 987938
<_mup_> Bug #987938: subunit trunk packaging breaks subunit-* commands <subunit:In Progress by jelmer> < https://launchpad.net/bugs/987938 >
<lifeless> $ subunit-filter
<lifeless> Traceback (most recent call last):
<lifeless>   File "/usr/bin/subunit-filter", line 33, in <module>
<lifeless>     from subunit import (
<lifeless> ImportError: No module named subunit
* benji changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 3.47*10^2
<cjwatson> I'm thoroughly lost.  Is there any documentation anywhere on how to debug the case where you've done export_as_webservice_entry on a class but it's stubbornly refusing to show up in the generated WADL?
<lifeless> you exported it for devel, and are checking devel ?
<cjwatson> Yes
<cjwatson> At least I think so otherwise I'm going to feel silly, but I've ntuple-checked
<cjwatson> lifeless: e.g. http://paste.ubuntu.com/1012244/
<cjwatson> (as a reduced experiment)
<lifeless> cjwatson: hmm, I'm not sure, its something i have diligently stayed on only passing terms with
<cjwatson> Oh, yeesh, I just found lib/lp/soyuz/interfaces/webservice.py
<cjwatson> DRY much
<cjwatson> Adding the new interface to the obvious two places there fixed it
<lifeless> do you mean ~DRY ?
<cjwatson> I meant sarcasm
<lifeless> :P
<cjwatson> (The right answer is probably in fact not to export that interface, but it's nice to know how to do it)
<lifeless> meep
<lifeless> http://pam-face-authentication.org/
<mwhudson_> cjwatson: if you want to get really depressed about the api, there's always https://bugs.launchpad.net/launchpad/+bug/951365
<StevenK> I have fear.
<_mup_> Bug #951365: Accessing milestone on project group via webservice api returns has_bugs object <api> <milestones> <oem-services> <projectgroups> <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/951365 >
<cjwatson> mwhudson_: nah, I'm trying to extend it, not get depressed about it :)
<lifeless> sigh firefox, please become an OS
<lifeless> the you might not use a full core when in the background
<bigjools> it has to read email before it can become an OS
#launchpad-dev 2012-05-29
<lifeless> bigjools: ... thunderbird
<lifeless> bigjools: it did, then it stopped ;)
<bigjools> yeah I just remembered that :)
<lifeless> ;)
<danilos> wgrant, hi, I wonder what's going on with db-stable r11611 rollout: LPS indicates it's out, but it's not merged yet in stable/devel
<StevenK> danilos: It will not be until it's deployed
<StevenK> danilos: Let me check
<danilos> wgrant, https://wiki.canonical.com/InformationInfrastructure/OSA/LaunchpadProductionStatus lists it in 'past deployments'
<StevenK> danilos: Your DB patch is r11610, and the next deployment for tonight is r11615, which includes your patch.
<wgrant> danilos: Sorry, I forgot to merge it
<wgrant> danilos: Doing now
<wgrant> danilos: It was deployed 12 hours ago
<danilos> wgrant, ah, ok, that explains it, thanks
<StevenK> Ah, so up next is my patch.
<danilos> wgrant, no worries, sorry for being pushy :)
<wgrant> Thanks for the reminder.
<wgrant> danilos: That's landed.
<wgrant> danilos: You can now land the code removal
<danilos> wgrant, cool, thanks
<StevenK> wgrant: I might up put up my builder.description destruction branch then
<danilos> wgrant, except that I can't (at least not that I know, if I can, woohoo and "how"), but I can ask someone to do that for me :)
<StevenK> danilos: Bug lifeless about emeritus
<wgrant> StevenK: Hm?
<wgrant> StevenK: Which branch?
<lifeless> StevenK: danilos: I think danilos is already in emeritus, isn't he?
<lifeless> and yes, emeritus can lnad
<lifeless> *land*
<StevenK> wgrant: The one that drops builder.description from the code since the DB patch to DROP NOT NULL is up next.
<lifeless> while they are still @ canonical
<wgrant> Can by policy, but you have to be in the keyring...
<wgrant> StevenK: Right, can't land for another 9 hours, though
<StevenK> wgrant: Sure, what I meant was "Actually push and put up for review"
<wgrant> Right
<wgrant> Makes sense
<wgrant> Thought that was already done, in fact.
<StevenK> steven@undermined:~/launchpad/lp-branches/drop-builder-description% utilities/rocketfuel-mp-status .
<StevenK>  * .
<StevenK>    - Remote branch is up to date
<StevenK> Hm, it seems you're right.
<danilos> lifeless, ah, cool, so I have to worry myself about running the full test suite, and then can pqm-submit? (how is ec2 test used these days?)
<lifeless> danilos: ec2 land is still used, its unchanged
<wgrant> Well
<lifeless> danilos: it lives in lp-dev-tools now
<wgrant> It's in lp:lp-dev-utils
<lifeless> bah utils
<wgrant> Rather than utilities/
<wgrant> But otherwise the same.
<wgrant> In the branch, 'ec2 land'
<danilos> ah, ok
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/drop-builder-description/+merge/107698 would enjoy your review
<wgrant> StevenK: Can you use the factory to replace most of create_builder?
<wgrant> Most or all
<StevenK> It would end up looking pretty much the same, and the doctest already wants IBuilderSet
<wgrant> factory.makeBuilder(name='hamburger', processor=i386, virtualized=True)
<wgrant> No helper required
<StevenK> wgrant: Although that points I missed out fixing factory.makeBuilder() :-)
<StevenK> wgrant: Diff updated, can you have another look?
<lifeless> wgrant: hey, you know what? We don't need that private API for sso anymore
<lifeless> wgrant: not with your new regular API method
<wgrant> lifeless: For performance I think we do.
<lifeless> wgrant: two in-dc calls vs 1 ?
<wgrant> lifeless: How do you plan to find all of the relevant team memberships?
<lifeless> participation
<wgrant> It would require a new call.
<wgrant> But could be done.
<lifeless> wgrant: super_teams ?
<wgrant> lifeless: Is a collection that returns a limited batch of participations.
<lifeless> pass a batch in greater than the current high-end, and iterate if needed
<lifeless> wgrant: most folk will be <= 1 batch
<wgrant> Yeah, multiple 300ms batch loads per authentication attempt isn't high on my list of things that are good ideas :)
<lifeless> where are you getting that time from ?
<lifeless> wgrant: but looking at it, it includes the whole metacrazydata
<lifeless> rather than just providing references.
<lifeless> \o/
<lifeless> whee
<lifeless> person:+contactuser
<lifeless> 99% under 32 seconds.
<wgrant> Yup
<lifeless> +cve is still horrid too
<lifeless> 23
<wgrant> lifeless: OK, so it looks like without auth you can make an API request in about 100-120ms.
<wgrant> auth will increase that a fair bit.
<lifeless> wgrant: because?
<wgrant> lifeless: Because our auth code is awful
<lifeless> fixable or unfixable ?
<wgrant> Fixable.
<lifeless> there we go then
<lifeless> DistroSeries:+builds is heinous
<lifeless> ever seen Branch:+try-again ???
<wgrant> Yeah
<wgrant> It is always either very fast or extremely slow
<wgrant> Probably due to lock contention, I guess.
<wgrant> But maybe not.
<lifeless>  Distribution:+topcontributors is good
<wgrant> I'm not sure why that's so bad
<wgrant> I haven't looked at a query log.
<wgrant> But it doesn't make sense for it to be slow.
<wgrant> Unless it's all unindexed, I guess.
<lifeless> mhm
<lifeless> Unknown
<lifeless> 9.5M hits
<lifeless> (over a month)
<wgrant> 404?
<lifeless> not sure
<lifeless> need to heckle someone to dig and identify causes
<lifeless> ahhaha
<lifeless> on https://devpad.canonical.com/~lpqateam/ppr/lpnet/latest-monthly-pageids.html
<lifeless> 'SimpleViewClass from /srv/launchpad.net/production/launchpad-rev-15099/lib/lp/bugs/browser/../templates/buglisting-embedded-advanced-search.pt:JsonModelNamespaceView'
<lifeless> page it
<lifeless> page id
<lifeless> it really is
<lifeless> I need a guesstimate
<lifeless> how long to instantiate 122 feature rules
<wgrant> They don't have enumcols, so it should be pretty quick.
<lifeless> yah
<lifeless> and a linear lookup of simple values is fast, given we cache results
<lifeless> so it should be safe
<wgrant> Also, where'd you get 122?
<wgrant> Oh
<lifeless> you should have enough data to infer
<wgrant> You want to add 122 exceptions?
<wgrant> And lower the timeout?
<lifeless> yes
<wgrant> I'm not sure that all of them deserve exceptions.
<wgrant> But perhaps.
<lifeless> it makes reasoning about it easy
<wgrant> I prefer reasoning recklessly.
<wgrant> It generally gives much better results.
<wgrant> More rapidly.
<StevenK> wgrant: Can haz review?
<wgrant> StevenK: Oh right, it's still trying to submit.
<wgrant> I had too many MPs open
<wgrant> Done
<StevenK> So tempted to put a Portal 2 reference in the commit message.
<StevenK> "Delete Builder.description from the code for standing around, smelling and being useless."
<StevenK> wgrant: http://pastebin.ubuntu.com/1012579/  is that what you envisioned, or am I on the wrong track?
<wgrant> StevenK: Looks like a reasonable start, indeed.
<StevenK> Right, excellent.
<StevenK> Needs tests for Branch:+edit and then it's done, I think.
<StevenK> wgrant: I'm a little unhappy about the IBranch.implementedBy() bit, but that should hopefully be short lived.
<wgrant> StevenK: You probably want providedBy, too, but indeed
<StevenK> Blink.
<StevenK> On Bug:+index the text turns up on page load and then disappears.
 * StevenK blames wallyworld_.
<wallyworld_> StevenK: what have i done?
<StevenK> wallyworld_: I've refactored out the Information Type portlet into a seperate class that is subclassed by BugView. Loading up Bug:+index shows the right text in the portlet for a few seconds which then disappears. I think the JS is to blame.
<wallyworld_> StevenK: this isn't on prod is it? seems to work there
<StevenK> wallyworld_: It is not on prod, no.
<wallyworld_> ok, so only in your local branch
<wallyworld_> when you say text, you mean the description that sits below the choice source popup widget?
<StevenK> wallyworld_: Nope, everything
<wallyworld_> hmmm
<StevenK> wallyworld_: Do you want me to commit and push this mess, or you want to mull it over?
<wallyworld_> i was going to look at the mp for the branch i recently landed but since it's merged, the diff is no longer available :-(
<wallyworld_> so go ahead and commit it
<StevenK> wallyworld_: I *HATE* that bug
<wallyworld_> i didn't realise it was a bug
<wallyworld_> i though it was intended
<StevenK> It didn't used to happen, I think.
<bigjools> it's when you push new revisions after merge IIRC
<wallyworld_> i can't remember
<bigjools> or something causes a scan on it
<StevenK> bigjools: I thought it was because the destination branch changes
<wallyworld_> so why does it discard the diff?
<bigjools> I'm not aware that matters
<bigjools> it regenerates a diff - which is empty
<wallyworld_> hmm. ok.
<StevenK> bigjools: Because the target branch now incorporates the changes that in the source branch
<wallyworld_> so perhaps it should retain the last diff in that case
<bigjools> StevenK: wouldn't that mean all of our diffs, ever, get regenerated?
<bigjools> for ever
<StevenK> bigjools: That seems to be happening anyway
<StevenK> wallyworld_: https://code.launchpad.net/~stevenk/launchpad/branch-information_type-ui
 * wallyworld_ looks
<wallyworld_> StevenK: the initialise of the InfoTypePort;et is not being called
<wallyworld_> stupid python class inheritance stuff
<StevenK> Oh, it won't call *all* subclasses that have an initialize() method?
<wallyworld_> nope, not unless they call super themselves
<wallyworld_> afaiui
<StevenK> wallyworld_: But InformationTypePortlet doesn't have a superclass
<StevenK> wallyworld_: Do you want to try that? lib/lp/app/browser/information_type.py
<wallyworld_> StevenK: i think it's stopping the chain at LaunchpadView
<StevenK> LaunchpadView is the first superclass
<wallyworld_> since it's initialise() just passes
<wallyworld_> and so the other superclasses are nw called
<StevenK> Perhaps it will stop after the first one
<wallyworld_> StevenK: put LaunchadView last in the inheritance list
<wallyworld_> it works then
<wallyworld_> StevenK: afaiui, it calls super in order from left->right
<wallyworld_> and if one stops, then boom
<wallyworld_> they each have to call super to keep it going
<StevenK> And BugViewMixin does not?
<StevenK> It doesn't have an initialize(), so I guess not
<wallyworld_> BVMixin does not implement that method
<wallyworld_> bit of a trap that one
<wallyworld_> and so of course it was the bugtask_index javascript that didn;t see the expected stuff in the request cache
<wallyworld_> so it duely hid the info type stuff
<bigjools> ok, how can I convince LP that actually, yes, my GPG signature valid on MP emails
<jelmer> bigjools: use MIME?
 * bigjools does a Marcel Marceau impression
<bigjools> What is mime for fucked?
 * stub mimes 'fucked'
<adeuring> good morning
<czajkowski> aloha
<lifeless> bigjools: I think you want the hip thrust for that one
<jml> hello
<jelmer> ohai
<jml> so, to upgrade to pg 9.1 in dev env, I just remove pg 8.4, install 9.1 and then run database-setup?
<StevenK> jml: Pretty much
<StevenK> jml: Also make sure 9.1 is listening on 5432
<mwhudson> i once tried to figure out how the debian packages created clusters listening on different ports
<jml> so the metapackages are all broken atm then?
<jml> ok, I'm confused.
<jml> launchpad-database-setup configures pg to run on 5433
<jml> so how can I make sure it's running on 5432
<jml> ffs
<jml> now I'm getting "invalid data directory"
<jml> graet.
* gmb changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: gmb | Firefighting: - | Critical bugs: 3.47*10^2
<jml> http://paste.ubuntu.com/1012845/ what am I doing wrong?
<mgz> nothing obvious, tries 5432... what's in /var/run/postgresql actually?
<jml> jml@lpdev:/var/run/postgresql$ sudo cat 9.1-main.pid
<jml> 7623
<jml> just that.
<jml> also,
<jml> $ ls -a
<jml> .  ..  9.1-main.pid  .s.PGSQL.5433  .s.PGSQL.5433.lock
<mgz> okay, so it is listening on the wrong port
<mgz> change that in the postgre conf and restart?
<jml> and then manually run the createuser commands, I guess?
<mgz> there's probably a better way, but naturally it just worked for me previously.
<jml> psql:launchpad-2209-00-0.sql:17: ERROR:  language "plpgsql" already exists
<jml> psql:launchpad-2209-00-0.sql:2646: ERROR:  could not access file "$libdir/plpython": No such file or directory
<jml> are they fatal errors?
<stub> jml: The second is - postgresql-plpython-9.1 is not installed.
<jml> stub: thanks.
<jml> stub: Ubuntu says it's already installed though.
<stub> jml: If you run 'psql', does it say the server version is 9.1 in the startup banner message?
<jml> psql (9.1.3)
<jml> Type "help" for help.
<jml> yes.
<stub> jml: oic. No, that isn't a fatal error. It will go away soon now we are 9.1 everywhere and can clean that up.
<jml> stub: cool. thanks.
<jml> while you're around...
<jml> I'm aiming to rename a DB column. The branch I submit against db-devel needs to update the code for the rename as well as adding the patch that does the rename, right?
<StevenK> jml: You can not do that.
<jml> StevenK: Do what? Rename a column?
<StevenK> jml: You can't change code at the same time
<cjwatson> (Because database and appserver rollouts are decoupled.)
<jml> hmm.
<nigelb> cjwatson: Are you rotating with LP this cycle?
<jml> so either I can't do a rename
<cjwatson> nigelb: No, I just have a number of LP-related projects
<jml> or I have to update the code to somehow work with both naes.
<jml> names, rather.
<nigelb> cjwatson: Ah, nice. :)
<cjwatson> jml: If you figure out a neat approach, I have a low-priority bug that could use it.  IIRC wgrant told me that we haven't renamed any columns since the introduction of fastdowntime.
<czajkowski> *grin*
<jml> cjwatson: Ah, I see.
<cjwatson> jml: In my case (bug 1000570) I renamed the API but left the DB column for "later".
<_mup_> Bug #1000570: "Packageset.score" is badly named <qa-untestable> <tech-debt> <trivial> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1000570 >
<jml> cjwatson: right, that's what I've done. I'm just making later, now.
<cjwatson> William suggested a trigger approach that I'm afraid I didn't understand.
<jml> stub: any thoughts on how to do a column rename?
<stub> jml: Yes. Ideally, minimize the code using a property or something
<stub> jml: Actually, hang on
 * jml does so
<stub> jml: So code changes and db changes are not synchronized.
<stub> jml: Which means first a code change that makes things work with the old column name and the new column name.
<stub> jml: Then the db patch doing the rename. Then a cleanup.
<stub> jml: Which all sucks.
<StevenK> Welcome to FDT.
<jml> ok. hmm.
<stub> Who asked for this again?
<StevenK> And oh, your patch will wait for a week because lifeless lives in 1990.
<jml> me.
<jml> I'm renaming Archive.commercial to Archive.suppress_subscription_notifications. Have done so in code, using a property.
<cjwatson> StevenK: Aren't we almost up to date on DB deployments now
<stub> jml: There could be some magic we do with PG, but I think it will end up more complex. Might be necessary if making a Storm class not care which of two column names is in use is a pita.
<cjwatson> ?
<StevenK> cjwatson: I'm just bitter.
<StevenK> cjwatson: It is usually pretty good, stuff has been held up due to 9.1 so we had a two week backlog
<cjwatson> I know
<stub> (magic being an updatable view, making code use the updatable view, do the rename, make the code use the table again)
<StevenK> And that is counting the five or six DB patches Purple wants to land.
<jml> stub: Archive inherits from SQLBase and we don't ever query on this column, so I think I can hack something up in Python.
<jml> testing will be a pain though.
<stub> jml: Or if that code isn't used, extract it, do the rename, put it back + fix.
<jml> stub: it is used, I'm afraid.
<stub> jml: Just saw your email. Did you fix the PG port issue?
<jml> stub: manually. I changed the port again in postgresql.conf; restarted; ran createuser by hand.
<stub> jml: I think you might see that if /etc/postgresql/8.4/main/postgresql.conf still exists and specifies port=5432
<stub> Can you check? Means there is another step after uninstalling the packages.
<jml> stub: make schema seemed to work OK, apart from the errors I mentioned earlier re plpython
<james_w> if a test is failing because it is escaping isolation due to a /+branch-id/ path being interpreted by bzr as being a filesystem path, does that mean that the test has to run with more of codehosting working?
<james_w> or perhaps it needs more plugins loaded beforehand?
<james_w> jml, do you know?
<jml> james_w: hi
<jml> james_w: plugins, I think.
<james_w> thanks
<jml> hmm
<jml> thinking out loud about column renaming
<jml> if we want one code base to work with both column names, then we either need some way of interrogating the schema at class creation time or some way of trapping errors at query time and updating the class then
<jml> I'm probably missing other options, but those seem to be it for me
<jml> the problem with doing something on class creation is that it will add a query for everything that imports archive.py
<jml> which is probably everything
<jml> a problem with trapping at query time is that there's no obvious single choke point
<cjwatson> Turn the attribute into a @property?
<cjwatson> (This may not be the best approach, more an existence proof)
<jml> cjwatson: well, that's already been done
<jml>     suppress_subscription_notifications = BoolCol(
<jml>         dbName='commercial', notNull=True,
<jml>         default=False)
<jml> in my branch
<jml> note that dbName is the old name.
<jml> but how do I make something that works both before and after the 'commercial' column gets renamed to 'suppress_...'
<jml> storm uses the dbName (somehow) to generate SQL
<wgrant> jml: There's probably no feasible way to handle that in the app.
<wgrant> jml: We've not done a column rename with fastdowntime before.
<wgrant> I would personally duplicate the column on the table and use triggers to keep it up to date while the application is flipped to use the new name.
<wgrant> Rather than trying to get Storm to cope, which it won't.
<wgrant> (unless you make suppress_subscription_notifications a property that executes manual SQL, rather than using Storm)
<sinzui> It's take 1 hour to unsubscribe ~registry for linux/unity bugs reported between 2005 and 2006. We may have another 6 hours of bug mail going to ~registry
<wgrant> (which might work in this case, since it's probably not used for filtering or anything)
<jml> wgrant: huh, you mean something like this:
<jml> @property
<jml> def suppress...(self):
<jml>   try:
<jml>     return do_query('SELECT suppress_... FROM Archive WHERE id = %s' % self.id)
<jml>   except ProgrammingError:
<jml>   return do_query('SELECT commercial FROM Arch...' % ...)
<wgrant> jml: Right
<jml> wgrant: interesting.
<wgrant> jml: I changed SourcePackageRelease.copyright to use a similar manual SQL approach for performance reasons.
<wgrant> But it could also work well here.
<jml> wgrant: I have looked pretty thoroughly, and it's not used for WHERE clauses to the best of my knowledge.
<wgrant> It obviously doesn't work if the column is involved deeply in queries, but since you're probably only getting/setting it it should be OK
<wgrant> Right, it wouldn't make much sense for it to be.
<jml> wgrant: I guess that will slow down code that uses the attribute, but maybe we can bear that for a little while.
<wgrant> jml: By 1 query for about 48 hours.
<wgrant> I think we can live.
<jml> wgrant: ok. I'll give that a shot. Thanks.
<wgrant> jml: Great.
 * wgrant disappears
<czajkowski> wgrant: nn
<czajkowski> 15:39 < jcastro> czajkowski: do you remember the URL to add bug trackers to lp?
<czajkowski> anyone help me out here?
<czajkowski> gmb: ^^
<gmb> czajkowski, https://bugs.launchpad.net/bugs/bugtrackers/+newbugtracker
<czajkowski> gmb: thanks
<jml> wgrant: SourcePackageRelease.copyright defaults to Python's copyright if it's not provided as a keyword argument, I think.
<jml> how do I get a cursor from a store object?
<jml> hmm. just cursor()
<jml> ok.
<jml> wgrant, stub: https://code.launchpad.net/~jml/launchpad/archive-commercial-rename-support/+merge/107819
<jml> gmb also, ^^ (but I talked about it with them)
 * jml has to dash.
<gmb> jml, k, thx.
<gmb> bye
<gmb> ... can't believe I just did that.
<jml> tee hee
* gmb changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 3.47*10^2
<lifeless> morning
<Laney> Anyone fancy giving me some light test education? http://paste.ubuntu.com/1013802/
<Laney> (AFAICT sourcepackage is a page such as https://launchpad.net/ubuntu/+source/mono)
<StevenK> Laney: sourcepackage is a DistributionSourcePackage
<Laney> StevenK: yeah, I think that's what I linked to.
<Laney> so, assuming that it's right, can I cure the ForbiddenAttribute?
<Laney> then I'll get on to figuring out how to make it match the href :-)
<Laney> (porting a doctest here)
<wgrant> StevenK: Um
<wgrant> StevenK: SourcePackage should be named Distro*Series*SourcePackage
<wgrant> Ah, but Distribution.getSourcePackage returns a DistributionSourcePackage, indeed.
<cjwatson> In any case the forbidden attribute is probably because you're using LaunchpadFunctionalLayer (so including the security model) but your test doesn't log in.
<Laney> The security model confuses me very much
<Laney> is there any documentation on it?
<cjwatson> Don't look at me, I zen it from the surroundings.
<cjwatson> And occasionally whine in a confused manner.
<wgrant> Well
<cjwatson> LaunchpadZopelessLayer avoids the security model for cases where it doesn't matter, but (a) I'm never quite sure how evil it is and (b) I don't know whether it'll work in this case
<wgrant> ForbiddenAttribute isn't caused by an incorrect layer
<wgrant> Unauthorized can be.
<cjwatson> (Real LP devs will now tell me I'm on crack)
<wgrant> ForbiddenAttribute means you've used the wrong attribute name.
<StevenK> Laney: Distribution.name does not exist.
<cjwatson> Ah, doh
<cjwatson> Laney's test does not actually mention .name anywhere directly; presumably it's in some tested code
<StevenK> Sigh, I should learn to read.
<cjwatson> Wait, Distribution.name *should* exist.  It's on IDistributionPublic.
<wgrant> Probably trying to set it.
<wgrant> WHich isn't allowed.
<wgrant> If the attribute is entirely unsettable, setting it will ForbiddenAttribute.
<Laney> it comes from the last line there
<Laney> view()
<wgrant> That is, if there's no permission that can possibly be held to grant write rights.
<wgrant> Laney: What's the traceback?
<cjwatson> You can figure out the security on any given attribute by looking at lib/lp/*/configure.zcml, sometimes in combination with lib/lp/security.py.
<Laney> sec
<Laney> wgrant: http://paste.ubuntu.com/1013837/
<Laney> 53 is the assertThat line
<wgrant> jml: That column workaround is disappointingly unevil.
<cjwatson> My queue API branch is getting to the point where it might arguably almost work.
<cjwatson> I might split it up, though, since some of the information about binary uploads is done by exporting bits of BPR and I suppose that might be contentious or something.
<wgrant> cjwatson: Well
<wgrant> cjwatson: Queue overrides are terrible.
<cjwatson> The bit about exporting BPR isn't even so much for overrides, just for displaying information on builds.
<cjwatson> The way I have it at the moment you'd do something like:
<cjwatson> for build in upload.builds:  # probably only ever one of these but the PackageUpload design is weird
<cjwatson>     for bpr in build.binary_packages:
<cjwatson>         print(bpr.name, bpr.version)
<cjwatson> or whatever
<wgrant> Ah, yeah.
<cjwatson> I could probably land accept/reject in advance of any of this, though, which would be a start.
<cjwatson> But I don't see a sensible way to export binary information without at least one new exported object, and BPR seems as good as any.  It's just that the asymmetry with SPR not being exported is a bit odd.
<wgrant> I'd personally just ignore the hideous data model and add a method to retrieve a list of all override tuples for the upload, and another which takes a list of tuples to change them.
<cjwatson> I guess that's a possibility, though there's quite a lot in each tuple.
<cjwatson> It feels better as a proper object so you can have named attributes.
<wgrant> We've deliberately avoided exporting BPR/SPR before.
<wgrant> Because security, URLs etc. are a bit of a problem.
<cjwatson> is_new, name, version, arch_tag, component, section, priority = override  # vomit
<cjwatson> URLs are only about as hard as for build, but yeah, not quite trivial.
<wgrant> Builds have a context.
<wgrant> BPRs do not.
<cjwatson> BPRs have a build
<wgrant> Ugh, I guess.
<wgrant> But ew.
<wgrant> Anyway
<cjwatson> Is it possible to create new objects just for export that don't correspond to anything much pre-existing in the internal data model?
<cjwatson> Basically the equivalent of a namedtuple
<wgrant> The fact that PU overrides are done by writing to BPR and SPR is an internal implementation embarassment that it would be nice to avoid exporting.
<wgrant> It's possible to do that, but there's a fair bit of code.
<wgrant> eg. DSP, SP aren't real.
<cjwatson> FWIW my current branch doesn't expose the writing part of that; I just went for overrideBinaries (which isn't lovely, but it fits reasonable CLIs well enough)
<cjwatson> PU.overrideBinaries that is
<cjwatson> So a new exported interface and everything on it is a property?
<wgrant> Basically.
<cjwatson> I could just add a load of properties to PackageUploadBuild and export that.
<cjwatson> Since it's there.
<StevenK> PU is exported over the API?
<wgrant> That's another thing that probably shouldn't be exposed.
<wgrant> PUB
<cjwatson> StevenK: Slightly, right now.  I'm working on extending it enough to be able to kill LP's queue tool.
<cjwatson> Oh, PUB is wrong because it's one for the whole build, not one per binary in it.
<cjwatson> So OK, PackageUploadBinaryAPIExportOnlyScrewYou or something
<wgrant> Another consideration is that API requests are slow, and some packages have a lot of binaries.
<wgrant> eg. overriding linux will take minutes over the API if you have separate objects for each binary.
<cjwatson> Hm, true
<cjwatson> arch_tag is per-build, so it's only six elements per binary override, and I suppose that list hasn't changed in six years.
<cjwatson> But ew, I hate unnamed tuples for this kind of thing.
<wgrant> Well
<wgrant> Could do list-of-dicts instead
<wgrant> JSON doesn't do namedtuples, unfortunately.
<cjwatson> List of dicts might be OK.
<StevenK> wgrant: I thought I could use setUpFields() to fiddle with the schema, but doing so causes KeyError in my tests.
#launchpad-dev 2012-05-30
<wgrant> StevenK: It depends how you use setUpFields() to fiddle with the schema.
<StevenK> wgrant: http://pastebin.ubuntu.com/1013974/
<wgrant> StevenK: Myes?
<StevenK> wgrant: I was having breakfast -- that diff is in relation to the question I asked you earlier.
<wgrant> StevenK: If it works, great. If it doesn't work, that doesn't tell me why it doesn't work :)
<lifeless> cjwatson: have you not been exposed to my 'imperative bug reports considered harmful' meme ?
<cjwatson> I don't think I used imperative "should" anywhere in that
<cjwatson> Oh, the title.  Whatever.  Rephrase as you see fit; I must say I entirely fail to see the point in that meme.
<lifeless> cjwatson: the title is imperative;) - its an imperative I agree with :)
<cjwatson> And I generally have trouble figuring out what you think I'm supposed to write instad.
<cjwatson> *instead
<lifeless> in this case, I'd have said something like 'queue tool requires direct DB access'
<cjwatson> OK.  I don't really see the benefit in spending time on consolidating the grammatical voice of bug titles, though; it seems like a very minor point at best.
<cjwatson> Perhaps this is the natural result of developers needing to file bugs for work they already have in progress in order to provide a place for QA tags to go.
<StevenK> wgrant: http://pastebin.ubuntu.com/1013992/
<cjwatson> That'll tend to result in people describing what they're doing rather than describing the original problem.
<cjwatson> Anyway, rephrased the title, but I can't promise to remember in future.
<lifeless> cjwatson: I think the QA story is indeed part of the situation
<lifeless> cjwatson: I find imperative bugs from devs or non-devs are equally likely to age terribly; yours wouldn't because the description is comprehensive
<wgrant> StevenK: Sounds like you have information_type in field_names.
<lifeless> cjwatson: also you're about to land a change, which makes the aging aspect less of a concern
<StevenK> wgrant: I'm supposed to?
<wgrant> StevenK: The default LaunchpadForm.setUpFields sets up fields that you specify.
<cjwatson> lifeless: Well, hoping that I actually get all the way through the several incremental changes in this series ;-)
<wgrant> StevenK: Unless information_type is in the schema that setUpFields is using, putting it in field_names isn't a very good idea.
<wgrant> StevenK: The fact that you override setUpFields to add in information_type manually suggests that it doesn't belong in field_names.
<StevenK> wgrant: If I drop it from field_names then it doesn't show up at all
<wgrant> StevenK: Ah, perhaps the default form template uses field_names to determine the order.
<wgrant> StevenK: Why did you move information_type_field into setUpFields anyway?
<wgrant> It doesn't depend on anything there,
<wgrant> StevenK: Normally you'd tweak field_names at some point, not just overriding the schema field because you can.
<StevenK> wgrant: I need to override the schema field for the vocab
<wgrant> Also, super()
<wgrant> This isn't Python 2.1.
<StevenK> Same error with super()
<wgrant> super() is unrelated here, it's just that your code was looking like it belonged in 2002.
<wgrant> StevenK: Grep for form_fields.omit
<wgrant> To see the usual pattern.
<StevenK> wgrant: I can put it in BranchEditSchema, but the vocab doesn't respect the flags
<wgrant> StevenK: If you really can't put it in the schema, you'll probably have to use the super/omit/+= approach.
<StevenK> wgrant: Right, so it stays in field_names, but then I omit and re-add it
<wgrant> StevenK: Except that this whole thing wants to be feature-flagged.
<wgrant> Observe BugSecrecyEditView
<cjwatson> lifeless: I don't mean to be awkward, by the way.  Just found it hard to see the importance of it.  If you think it makes a measurable difference to how useful bugs are a while after they've been filed, fair enough.
<lifeless> cjwatson: The prototypical poor bug looks like this
<lifeless> "Do X"
<lifeless> "Something detailed about the change"
<lifeless> usually, when I ask the filer 6 months or a year later, they have no detailed recollection for what prompted them to file the bug
<lifeless> that is, the symptoms are long forgotten
<cjwatson> Yeah, I can see that being troublesome
<lifeless> so my actual thing is that we should always record the symptoms
<lifeless> and that as a mind-hack, avoiding imperatives seems to encourage capturing the symptoms vs the fix
<cjwatson> Ah, see, I would have found "record the symptoms" a much clearer way to put it than "no imperatives".
<cjwatson> Your mind-hack bounced off me :)
<lifeless> indeed
<lifeless> the other, *much weaker* aspect is that in bug listings, having symptoms visible helps figure out if you're looking at the right bug
<cjwatson> (Because, I think, "no imperatives" reminded me too much of pseudogrammatical nonsense like "passive voice is naughty".)
<lifeless> I say weaker because you can click through and if the description is maintained or just good, its only time
<lifeless> cjwatson: yes, martin pool also fell down that path
<lifeless> I need to evolve how I present this, I think.
<mwhudson> hello, i'm michael i work for a software company called 'pedantical'
<lifeless> MPT has filed dozens, if not hundreds of these poor bugs btw
<lifeless> with pages of exposition
<lifeless> but missing the actual symptoms.
<lifeless> He changed some years in and started recording symptoms too
<lifeless> its quite dramatic when you scan through a couple K bugs, what patterns the brain finds :)
<StevenK> wgrant: Right, so I was attempting to do the same, but there's an adapter in BranchEditView
<wgrant> StevenK: An adapter?
<StevenK>     def adapters(self):
<StevenK>         """See `LaunchpadFormView`"""
<StevenK>         return {BranchEditSchema: self.context}
<StevenK> With a @property
<wgrant> StevenK: Um
<wgrant> StevenK: BranchEditView already has a tonne of logic for this sort of thing.
<wgrant> Sometimes showing the private field, sometimes not
<wgrant> Using custom widgets
<wgrant> in setUpFields.
<StevenK> wgrant: Yes, have you seen my diff?
<wgrant> StevenK: Yes, and it doesn't make much sense.
<wgrant> Because it's defining a setUpFields when there already is one.
<wgrant> Unless you're defining it on a superclass -- hard to tell from that diff.
<StevenK> wgrant: Sorry, I meant the total diff. http://pastebin.ubuntu.com/1014035/
<wgrant> Ah, BranchEditFormView
<StevenK> Yes.
 * StevenK stabs it
<wgrant> StevenK: Line 258
<StevenK> Ah, so the setupFields in BranchEditFormView will never get called
<wgrant> Correct.
<wgrant> THis is why we shouldn't write code like it's 2002.
<wgrant> Because it's not 2002.
<StevenK> wgrant: But the adapter I pasted above is why I can't do what BugEditSecrecyView does
<wgrant> In fact, maybe I should grep for anything calling a method on a CamelCase identifier and kill it :)
<wgrant> StevenK: Why can't you put it in BranchEditSchema? Does the vocab evaluate the flag at instantiation time?
<StevenK> wgrant: It was there originally, but it looks like the flag doesn't take effect
<wgrant> I'd perhaps work out why it's not taking effect.
<StevenK> wgrant: The class is at the top level, and the vocab has a () at the end so it happens at file read time?
<wgrant> StevenK: If you instantiate it there, it will be insantiated there, indeed.
<StevenK> wgrant: So I was trying to work around that like BugEditSecrecyView but that adapters thing uses the schema
<wgrant> StevenK: Are you sure you can't override it?
<wgrant> I mean, you may be able to do the usual thing in setUpFields.
<wgrant> Or define the interface in a method somewhere.
<wgrant> Or not instantiate the vocab at that point (I'm not sure if you have to)
<wgrant> Or not use adapters.
<wgrant> etc
<wgrant> one of those ways will work
<StevenK> wgrant: Why is the adapter even there?
<wgrant> StevenK: Probably to adapt the data into the form required by the interface.
<wgrant> Now
<wgrant> I'm not sure it's entirely useful here, given that the adapter does nothing.
<StevenK> It gives a fail to adapt if I remove it
<wgrant> Well, yes, clearly removing the adapter is going to not work very well.
<StevenK> wgrant: But you said it does nothing! :-)
<wgrant> The adapter internally does nothing.
<wgrant> Its presence does something, but it's probably not useful.
<wgrant> That doesn't mean you can just delete its definition without consequence.
<StevenK> If I change it from using BranchEditSchema to self.schema it also gives fail to adapt
<wgrant> That's not really surprising, since self.schema is BranchEditSchema.
<wgrant> So
<wgrant> What I would try is the usual approach.
<StevenK> But they should be equal :-(
<wgrant> Define the class at view instantiation time.
<wgrant> THey aren't just equal -- they are the same object.
<wgrant> So changing from one to the other isn't going to change anything -- object identity holds between them.
<StevenK> wgrant: I was doing that with def schema: class schema...
<wgrant> StevenK: Right, that's probably the least invasive approach here.
<wgrant> Try something that works.
<wgrant> If it works, it's possibly a good idea.
<wgrant> If it doesn't work, it's probably a good idea to try something else :)
<StevenK> TypeError: ('Could not adapt', <Branch u'~person-name-100039/product-name-100042/branch-100046' (77)>, <InterfaceClass lp.code.browser.branch.BranchEditSchema>)
<wgrant> StevenK: You're probably creating multiple instances of the class.
<wgrant> Erm, wait, that's not a class defined inside the class
<wgrant> You're still using the module-level one.
<StevenK> wgrant: http://pastebin.ubuntu.com/1014057/
<wgrant> StevenK: That has the obvious problem that the interface will be recreated each time self.schema is evaluated.
<wgrant> Which is less than ideal, as the that means self.adapters will be returning a new interface each time, so it's entirely useless.
<StevenK> I thought that was BugEditSecrecyView was doing?
<wgrant> It's not adapting.
<StevenK> Ah, true.
<wgrant> It probably only evaluates self.schema once, and even if it does it multiple times it doesn't care about their identity.
<wgrant> Just their content.
<wgrant> Adaption cares about identity.
<StevenK> :-(
<lifeless> hmm
<lifeless> anyhone happen to know, will the PPR consider e.g. memcache time as 'SQL' time ?
<wgrant> lifeless: I believe so.
<lifeless> wgrant: +login
<wgrant> lifeless: AFAIK its logic predates non-SQL timedactions
<lifeless> 1.4s 99th percentil, 0.2s 99th percentil of SQL
<lifeless> so 1.2 seconds doing <what>
<wgrant> Hm, maybe it's smarter than I thought
<lifeless> We track at least one of our calls to SSO
<lifeless> possibly the nested calls thing is it
<lifeless> ah yes, that will be showing 0ms times
<StevenK> Launchpad Production Errors
<StevenK> OOPS Tools Production Errors
<StevenK> Production Errors
<StevenK> FAIL
<wgrant>         sql_statements = da.get_request_statements()
<wgrant>         sql_milliseconds = sum(
<wgrant>             endtime - starttime
<wgrant>                 for starttime, endtime, id, statement, tb in sql_statements)
<lifeless> StevenK: if thats mail, great success
<wgrant> lifeless: Oh, prod OOPSes are going to oops.c.c now?
<lifeless> yes
<lifeless> easing over
<StevenK> lifeless: Oh, excellent. So now developers have to skip six useless mails a day rather than four
<wgrant> :/
<wgrant> Well
<wgrant> More notably, it's now impossible to analyse OOPSes usefully.
<lifeless> StevenK: we'll disable the other one shortly; then update the linkification logic
<wgrant> We can't get the actual files or grep around to get trends etc.
<lifeless> we haven't finish the setup
<lifeless> its on a dedicated machine for scaling, and we should be able to get shell access
<StevenK> So Oops gets a machine and LP Jenkins doesn't?
<lifeless> I don't know where to even start
<lifeless> bigjools: does maas use package django? just python-django ?
<bigjools> lifeless: yeah pulls in the package
<StevenK> wgrant: I wonder if I can have the schema out in the class and then override the field
<wgrant> StevenK: Define it in the top level of the class body? That'll be a single interface for all instances of that class, so mutating it is not a good idea.
<hbt272> hello
<StevenK> wgrant: Mutating the field is a bad idea?
<hbt272> anybody know vala langueges?
<wgrant> StevenK: If the class is shared between all instances, yes.
<StevenK> wgrant: It's used for BranchEditWhiteboardView BranchEditStatusView BranchEditView BranchReviewerEditView
<StevenK> So it's probably safe to rip out and replace information_type in BranchEditView.setUpFields()
<wgrant> StevenK: I mean the single interface class will be shared between BranchEditView etc. instances.
<wgrant> So you'll be mutating semi-global state
<wgrant> Just like if you had class Foo:
<wgrant>    bar = []
<wgrant> And started poking in bar, all instances would see it.
<StevenK> wgrant: Right, I was going to leave information_type in BranchEditSchema withouth the vocab and then replace the field in BranchEditView.setUpFields()
<wgrant> StevenK: But you can't modify BranchEditSchema if you do it like that.
<lifeless> StevenK: wgrant: what are you guys cooking up?
<wgrant> Well, you can, but it's evil and I'll reject it :)
<StevenK> wgrant: I wasn't going to -- I was going to self.form_fields.omit() and so on
<wgrant> StevenK: If you're going to do that, you needn't move BranchEditSchema at all.
<wgrant> I thought you had a reason for not doing this.
<wgrant> the adapter broke it or something.
<StevenK> wgrant: Right, I have moved BranchEditSchema back
<StevenK> TypeError: unsupported operand type(s) for +=: 'FormFields' and 'Choice'
<StevenK> :-(
<wgrant> += Fields(foo_field)
<StevenK> from zope.formlib import form
<StevenK> RARGH
<StevenK> I hate that, and then form. is scattered all over the file
<wgrant> Hope you never have to work on bzr :)
<wgrant> They do it a bit.
<wgrant> I'm not really a huge fan.
<StevenK> I'm tempted to fix it in this file, but this branch is already 460 lines.
<StevenK> % bzr grep 'from zope.formlib import form' | wc -l
<StevenK> 28
<StevenK> Maybe I will fix them
<StevenK> (In a seperate branch)
<wgrant> StevenK: It's a matter of opinion whether it's something that should be "fixed".
<StevenK> wgrant: It annoys me... It may annoy me enough to bend the codebase to my will.
<hbt272> how can to change gtk window background in vala langueges? Please help
<StevenK> wgrant, wallyworld_: https://code.launchpad.net/~stevenk/launchpad/branch-information_type-ui/+merge/107909 would appreciate a review.
<wallyworld_>  /me looks
<wgrant> StevenK: Does that actually work?
<wgrant> You aren't hiding explicitly_private when the flag is set.
<lifeless> hbt272: this channel has nothing to do with gtk
<lifeless> hbt272: I think you are not getting answers because people don't know. You would be better off asking in a desktop development channel.
<wgrant> StevenK: Oh, you do that separately down the bottom. That's a bit odd. I'd just customise field_names based on the flag and have everything else depend on the contents of field_names.
<hbt272> lifeless: thank you!
<StevenK> wgrant: My reasoning behind that was two-fold: 1) The current rules look complex, and 2) We are going to revisit this anyway due to self-service, so thought it could stay put until then.
<wgrant> StevenK: Does omit break if the field isn't there?
<wgrant> StevenK: I'd replace the stuff in "if not show_private_field" with "self.form_fields = self.form_fields.omit('explicitly_private', 'information_type')"
<wgrant> And the thing which replaces the field on line 257 of the diff with "if self.form_fields.get('information_type') is not None"
<wgrant> That encapsulates the field selection logic in field_names.
<wallyworld_> StevenK: sorry, got pinger by thumper, looks like wgrant is doing the review
<wgrant> Yup
<StevenK> wgrant: But information_type will always be not None?
<wgrant> StevenK: Not if it's not there.
<StevenK> wgrant: information_type is always in the schema
<wgrant> StevenK: if 'information_type' in self.field_names, then.
<StevenK> wgrant: Right, that sounds sensible
<StevenK> wgrant: Are those your only two comments?
<wgrant> StevenK: http://pastebin.ubuntu.com/1014176/
<wgrant> Um, can revert the move of show_information_type_in_ui; that isn't necessary any more.
<StevenK> Why moving the schema?
<wgrant> So we don't have to override the field.
<wgrant> We instead create the class when self.schema is evaluated.
<wgrant> At which time the feature flag is already in place.
<StevenK> But doesn't that end up with beng unable to adapt?
<wgrant> Same thing as BugSecrecyEditView, except with @cachedproperty to preserve identity for adaption.
<StevenK> *being
<StevenK> Ah.
<StevenK> wgrant: That looks excellent.
<StevenK> wgrant: So you approve, or now I should get wallyworld_ to review it? :-)
<wgrant> StevenK: Was waiting for the diff to update.
<StevenK> It should be done
<wgrant> Indeed.
 * wallyworld_ laments that getPrecachedPersonsFromIDs doesn't seed the ValidPersonCache :-(
<StevenK> wgrant: Did you see cjwatson's MP about CustomUpload?
<wgrant> wallyworld_: ValidPersonCache is the most horrible thing in the world.
<StevenK> ... aside from my indenting.
<wgrant> wallyworld_: But there should be an option to getPrecachedPersonsFromIDs that cached it, I thought.
<wgrant> StevenK: I veeeeery nearly said that.
<StevenK> Hahahaha
<wgrant> But decided it could be mean.
<wallyworld_> wgrant: the api seems to indicate that but it doesn't work
<wgrant> wallyworld_: How're you accessing it that it doesn't appear to be cached?
 * StevenK wonders why LP pages are taking ~ 3 minutes to refresh
<wgrant> StevenK: Because you have too many MPs open.
<wallyworld_> wgrant: person.is_valid_person does a ValidPersonCahce query even after calling getUtility(IPersonSet).getPrecachedPersonsFromIDs(grantees, need_icon=True, need_validity=True)
<StevenK> SIGH
<StevenK> LONGPOLL!
<wgrant> wallyworld_: Did you evaluate the resultset?
<StevenK> wgrant: Can I have a +1 now?
<wgrant> wallyworld_: getPrecachedPersonsFromIDs doesn't actually get precached persons from IDs.
<wgrant> wallyworld_: It gets a resultset which gets precached persons from IDs.
<wgrant> Try listifying it.
<wgrant> StevenK: Working on it.
<wgrant> A few more comments.
<wallyworld_> wgrant: i didn't evaluate it. that may be what i'm missing. will try that, thanks
<StevenK> We can't just delete ValidPersonCache?
<wgrant> StevenK: I plan to achieve its demise within the year.
<StevenK> For crimes against humanity
<wgrant> For it is an abomination whose reign of terror must be stopped.
<wallyworld_> wgrant: yes, that was the problem, thanks
<wgrant> wallyworld_: Great.
<wgrant> wallyworld_: I've fixed a few views that were doing the same thing.
<wgrant> Over time
<wgrant> Attempting to precache, but not actually doing anything of value.
<wallyworld_> i can imagine
<wgrant> StevenK: You have my +1 but with various comments.
 * wallyworld_ is glad there was a query count test
<wallyworld_> which failed so it did its job
<wgrant> Yup
<StevenK> wallyworld_: You can add lists together, you know, right?
<StevenK> Rather than .insert(0, ...
<wallyworld_> StevenK: the order is preserved?
<wgrant> StevenK: ... says he who just used [2] :P
<wgrant> But yes, I wouldn't use insert() as wallyworld has.
<wgrant> Mostly because it's confusing doing it twice to get opposite order.
<wallyworld_> StevenK: feel free to do a drive by in your branch so long as the ordering remains as expected
<wgrant> StevenK: Are you looking at Colin's branch?
<wgrant> If not, I will.
<StevenK> wgrant: I have done, I'm happy with it, but given it's wide-reaching I wanted a second opinion.
<wgrant> I don't see a review.
<StevenK> https://code.launchpad.net/~cjwatson/launchpad/custom-upload-parsing/+merge/107656
<wgrant> There's an MP, but no comments.
<wgrant> Did it get eaten by longpoll?
<StevenK> I haven't commented yet
<StevenK> I can if you wish
<wgrant> I shall fetch a drink and look over it.
 * StevenK stabs longpoll more
<wgrant> Hm
<wgrant> MP diffs are a bit more readable with the excessive line-height disabled.
<StevenK> wgrant: The portlet is not used for the edit form class, but BugView and BranchView, and they *don't* implement show_information_type_in_ui
<wgrant> StevenK: Ah, indeed, I am stupid.
<wgrant> StevenK: But they should implement that.
<wgrant> It's only temporary, and it's going to be shorter to have it in the view classes.
<wgrant> And less hideous.
<StevenK> Just so it doesn't look utterly bonkers
<StevenK> wgrant: The portlet is tested elsewhere in a doctest
<wgrant> StevenK: Ah. That's what I would have expected if it was eg. Bugs, but I thought Code had a bit more sense than that.
<StevenK> Clearly not
<wgrant>  /win 3
<wgrant> Bah
<StevenK> wgrant: http://pastebin.ubuntu.com/1014208/
<wgrant> StevenK: hat was that nbsp doing there? A leftover from the link that isn't there yet?
<StevenK> wgrant: It was copied from the bugs template, so likely.
<wgrant> Right. LGTM, then.
<wgrant> Thanks.
<wallyworld_> hurry up launchpad, scan my @!@$!% branch already
<StevenK> wallyworld_: You used to be on the code team, fix the scanner.
 * StevenK hides.
 * wallyworld_ gets out flamethrower
 * wallyworld_ blames thumper
<StevenK> wallyworld_: Do you recall what AMI your ec2 runs this morning used?
<wallyworld_> StevenK: 529 i am pretty sure
<StevenK> That is excellent news.
<wallyworld_> hmmm. i think this branch is stuck. 15 minutes and counting
<wallyworld_> StevenK: yes, glad you fixed it
<wallyworld_> ah, just completed
<StevenK> wallyworld_: I'm still a bit unhappy that it took me until 1:10am to do so.
<wallyworld_> StevenK: take the last hour or so of today off. have a beer :-)
<StevenK> ENOBEER
<wallyworld_> wine then :-)
<StevenK> ENOWINE
<StevenK> ENOALCOHOL
 * StevenK gets tempted to fix ec2 images to not use $EMAIL
<StevenK> Actually how does even *work*
<wallyworld_> you have no alcohol in the house? shame
<wallyworld_> what, is that a whip i can hear?
 * StevenK stabs buildbot
<wgrant> Usual bzr thing?
<StevenK> No, my fault
<wgrant> No
<wgrant> This looks legit.
<wgrant> Yeah
<StevenK> I have a small fix
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/testfix-no-builder-description/+merge/107913
<wgrant> StevenK: If it works, ship it.
<StevenK> wgrant: Did you look at cjwatson's MP?
<wgrant> StevenK: Most of the way there. It's the sort of thing I'd tend to self-review just to avoid subjecting a reviewer to it.
<StevenK> Yah, self-reviewed and landed
<wgrant> No, I mean cjwatson's thing.
<wgrant> But yes, yours was obviously self-reviewable :)
<StevenK> Ah
<wgrant> StevenK: It has my +1
<StevenK> wgrant: Excellent
<adeuring> good morning
<adeuring> hi al-maisan
<al-maisan> moin adeuring
<al-maisan> how are things?
<cjwatson> wgrant: Hmm.  I did initially try a classmethod for this custom upload rearrangement and it didn't seem to come out shaped very nicely; in particular I definitely found moving the target directory stuff out of __init__ to be a win (Jeroen suggested consolidating *AlreadyExists on the bug, and I never liked the way that it sometimes failed in the constructor and sometimes in process anyway)
<cjwatson> But I can try again if you like, maybe with just the classmethod change but leaving __init__ as I currently have it?
<wgrant> cjwatson: Mm, that's true, it could fail in setTargetDirectory if it's badly formatted. So maybe.
<wgrant> Anyway, it's just a suggestion.
* frankban changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: frankban* (rvba) | Firefighting: - | Critical bugs: 3.47*10^2
<cjwatson> Could I have "Status: Approve" on https://code.launchpad.net/~cjwatson/launchpad/queue-api-accept-reject/+merge/107894 so I can land it?  It has a +1 review already.
<StevenK> cjwatson: Sorry about that.
<StevenK> cjwatson: Done.
<cjwatson> NP, thanks
<StevenK> Is scripts/ftpmaster-tools/queue dead yet? :-)
<cjwatson> ... a bit of work left for that
<StevenK> I know :-)
<wgrant> cjwatson: You probably want to make QueueInconsistentStateError a 400 rather than an OOPS
<wgrant> cjwatson: The browser code catches it and turns it into a non-OOPS error, but the API will default to OOPSing.
<cjwatson> wgrant: Hm.  I've already started landing this.  Should I abort?
<cjwatson> (I agree)
<wgrant> cjwatson: No, the OOPS reports are shit enough already that nobody will notice.
<wgrant> But a quick followup would be nice.
<cjwatson> That said I didn't start ec2 land that long ago.  Probably more economical to kill it.
<wgrant> Potentially.
 * cjwatson does so.
<cjwatson> wgrant: So http://bazaar.launchpad.net/~cjwatson/launchpad/queue-api-accept-reject/revision/15326 ?
<jml> can someone please land this approved branch for me?
<jml> https://code.launchpad.net/~jml/launchpad/archive-commercial-rename-support/+merge/107819
<jml> hmm, wait, ec2 land works for me, I think.
<jml> never mind.
<wgrant> cjwatson: Looks good.
<cjwatson> wgrant: Thanks.  Landing for real now.
<wgrant> Great.
<wgrant> Oh
<wgrant> ECHAN
<mgz> wgrant: from #launchpad, so what would be the steps if we couldn't get a branch owner to poke things themselves?
<mgz> the same but a l-osa?
<jml> mgz: there used to be a team that had write privs to all branches. I guess any administrator could still do that.
<wgrant> mgz: Right, webops have write access to all branches.
<wgrant> So they can carry out the same actions, and it's sometimes easier to do that than walk the user through it.
<wgrant> It looks like the relevant commit was directly to trunk, so hopefully none of the stacked-on branches have a copy of the corruption.
<StevenK> jml: That team (bazaar-experts) is long dead.
<cjwatson> wgrant: I tweaked https://code.launchpad.net/~cjwatson/launchpad/custom-upload-parsing/+merge/107656 a bit; if that looks OK to you now, could I have "Status: Approved"?
<wgrant> cjwatson: Thanks. Done.
<wgrant> mgz: The example invocation in the script is out of date.
<wgrant> mgz: The statement in the bug is correct.
<wgrant> PPA stats have not been updating since the 21st.
<mgz> wanted the guy the clarify the report
<mgz> but if it's something you can just poke someone to fix, that's fine.
<wgrant> A quick look at the crontabs shows that the script was disabled on the 21st due to germanium load issues.
<wgrant> germanium is still insanely loaded, so it's possibly not a good idea to reenable it.
<mgz> so... convert to question, say we know about the problem but it's not something being fixed right now?
<czajkowski> or triage it high still and adda note about the scripts?
<wgrant> Or bug IS to give germanium some minions.
<mgz> still need to find a diagram of what box does what with launchpad
<mgz> germanium is in charge of what exactly?
<wgrant> carob:~stevenk/name-to-service is probably the best we have right now.
<wgrant> germanium is ppa.launchpad.net
<wgrant> It does everything relating to PPAs, except builds.
<wgrant> (including serving PPA downloads to every PPA-using machine in the world)
<elmo> there's a ticket about germanium frontends FWIW
<wgrant> StevenK: Your list is missing bilimbi, btw.
<wgrant> elmo: Yeah, but...
<wgrant> StevenK: Also mcmurdo
<noodles775> Hi! I've just setup an LP dev environment, but get the error "String or Integer object expected for key, unicode found" when trying to run a test - http://paste.ubuntu.com/1014687/
<noodles775> Any pointers as to what I've missed?
<jml> Hmm.
<jml> That looks very familiar.
<jml> noodles775: that's a testr bug.
<jml> noodles775: the failure is related to the dirdbm (or whatever) db that itt uses.
<noodles775> Thanks jml - I'll just run the tests old-skool then :)
<jml> noodles775: np.
<noodles775> jml, achuni: an alternative for large htpasswd files: http://stackoverflow.com/questions/6590789/linux-htpasswd-file-too-big-more-than-2000-users
<noodles775> (again, may or may not be relevant depending on the measurements, but looks likely that it will be :) ).
<jml> that would be a GET-time issue rather than a .htaccess-generation-time issue. still relevant though.
<mgz> gmb: on subunit and avoiding stream breakage from other things mixing in
<mgz> what bzr does is use a dedicated pipe, avoiding most of that pain
<mgz> might not be simple with indirection through zope.testing but is much safer than messing with the process std streams
<gmb> mgz: Ah, thanks for the heads-up. That might make more sense, assuming we can untangle how that interacts with the parallelisation story.
<mgz> right, in bzr it's simple because we do the fork ourselves,
<mgz> multiple packages make that a bit more of a juggling act though
* abentley changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: frankban* (rvba), abentley | Firefighting: - | Critical bugs: 3.47*10^2
<cjwatson> jml: I think I'm going to plagiarise your foul column renaming hack.
<jml> cjwatson: best of luck.
<jml> that reminds me
<jml> the example I cribbed from has a bug, I think.
<cjwatson> oh?
<jml> nothing relevant for us, I don't think
<jml> cjwatson: SourcePackageRelease sets 'copyright' to the default Python copyright even if it's not provided
<cjwatson> That's only due to copyright being a builtin, yes?
<jml> yeaah
<jml> otherwise it would be a NameError
<cjwatson> Although I thought we used -S
<jml> oh, we do.
<cjwatson> So NameError
<jml> Right.
<cjwatson> Though do we?  If you run bin/py on its own, you get a copyright builtin.
<cjwatson> Because even though its #! line has -S, it re-execs python.
<cjwatson> But I guess bin/run uses -S in #! too.
<jml> it's also difficult to grep for code that creates one
<tedg> Hey, with the new privacy work going on will that allow blueprints to be private?
<jml> what invalidates a token?
<flacoste> mrevell: are we having the work items checkpoint?
<noodles775> jml: a setting the date-deactivated, let me check the details.
<jml> noodles775: I think it's unsubscribing someone who has an active token
<noodles775> abentley: time for a review? https://code.launchpad.net/~michael.nelson/launchpad/982938-dont-load-all-tokens-for-all-ppas-into-memory/+merge/107980
<abentley> noodles775: Be with you in a minute.
<noodles775> jml: setting ArchiveAuthToken.date_deactivated (see soyuz/model/archiveauthtoken.py:58 for the helper method, but it can also be done via the API with the right privs).
<abentley> noodles775: looking...
<noodles775> Thanks abentley :)
<abentley> noodles775: It looks like you are not certain this change will help.
<abentley> noodles775: Why not do it all in a single query?
<noodles775> abentley: some private PPAs have *lots* of tokens...
<noodles775> abentley: normally in any 5minute period, there will be only a number of PPAs affected, so doing one extra query per PPA seems reasonable to loading those tokens into memory (in a dict)
<abentley> Right.  I'm asking about 1 query vs many queries, not about many queries vs dict.
<noodles775> abentley: Ah, I see. So one query to get a resultset of all the tokens, and if it was ordered by PPA id, we could potentially iterate that...
<abentley> noodles775: right.
<noodles775> abentley: let me try.
<abentley> noodles775: One way would be nested generators.
<abentley> noodles775: Though it's probably easier to handle it in the final loop.
<abentley> noodles775: Something like this for the query? http://pastebin.ubuntu.com/1014950/
<noodles775> abentley: ah, getting a tuple of (archive, tokens) will be much simpler than iterating based on an ordered result set... thanks!
<abentley> noodles775: np
<abentley> noodles775: If it works out, you could also test the query count.
<noodles775> abentley: I don't see a way I can do that without needing to create an in-memory list of tokens for a PPA (ie. it's no longer a result-set of tokens).
<abentley> noodles775: Fair enough.
<noodles775> abentley: but I also fonud another spot where the result-set of tokens will be used in memory (it's sorted in another function...)
<noodles775> abentley: I'm just seeing if I can fix that.
<abentley> noodles775: r=me
<noodles775> abentley: Thanks, but can you hold off on approving until I check this other spot where it'll turn the result set into an in-memory list?
<abentley> noodles775: okay.
* frankban changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: abentley | Firefighting: - | Critical bugs: 3.47*10^2
<noodles775> abentley: I've pushed up some extra revs (15331..15333) which remove one place where the tokens were sorted in memory, instead sorting them in storm.
<abentley> noodles775: Looks good.  r=me
<abentley> noodles775: I wonder whether you could get the behaviour you want (loop by ppa, then tokens) using itertools.groupby
<noodles775> Thanks abentley. re. itertools.groupby, I think it would be possible, but would require quite a refactor as htpasswd_credentials_for_archive() currently assumes it'll be passed just the tokens for the one archive.
<noodles775> abentley: and (really sorry) I've just noticed that after the last refactor you just checked, I can remove _getValidTokensForPPAs completely now, afaics... checking.
<noodles775> abentley: so, r15334 removes the now unecessary _getValidTokensForPPAs.
 * noodles775 updates the test instructions.
<noodles775> abentley: sorry for the incremental realisations there, but I believe that's it :) I need to run, but will check back later in case you've any issues with the recent revisions. Thanks for all your help!
<noodles775> abentley: and if you are happy with it as is, can you please set the status to approved? Thanks.
<jml> why does getNewPrivatePPAs work from the date_completed of the last successful script run, rather than date_started?
<jml> getNewTokensSinceLastRun works from date_started
<abentley> noodles775: r=me
<noodles775> jml: I could only assume that one of them was updated due to a bug, and teh other was missed.
<noodles775> Thanks abentley - do you send it off to test and land? I've got tests running on my dev instance anyway to be sure myself.
<jml> correct code is so much easier to understand
<abentley> noodles775: I can do that.
<noodles775> Great, thanks again.
<jml> noodles775: oh hey, it was you that fixed that bug :)
 * jml discovers https://bugs.launchpad.net/launchpad/+bug/628711 
<_mup_> Bug #628711: generate-ppa-htaccess is too slow <lp-soyuz> <p3a> <ppa> <qa-ok> <software-center> <Launchpad itself:Fix Released by michael.nelson> < https://launchpad.net/bugs/628711 >
<czajkowski> sinzui: are you about ?
<sinzui> I am
<czajkowski> sinzui: having some issues when I renamed a project stuff hasn't followed through
<czajkowski> sinzui: https://answers.launchpad.net/launchpad/+question/198320
<sinzui> czajkowski, Looks like you did rename it? What has not happened?
<czajkowski> bugs from old project name have not been migrated over
<czajkowski> it;s old name is still being displayed on parent project page
<sinzui> that is memcached
<czajkowski> sinzui: and on top of bugs/blueprints its still referred to as old name
<sinzui> The project listing is expensive. It takes 1-3 hours to clear I think
<czajkowski> it was done on the 28th
<sinzui> czajkowski, the other displaynames you mention can only be changed by the project maintainer
<sinzui> only jonobacon can change the display name that the pages show. You can change the Launchpad Id.
<sinzui> czajkowski, tell jono to visit https://launchpad.net/ubuntu-accomplishments-web/+edit update his project
<czajkowski> sinzui: thanks
<czajkowski> sinzui: and the bugs issue? will resolve
<czajkowski> ?
<sinzui> that is right. bugs and blueprints are doing exactly what jono said he wanted
<sinzui> czajkowski, but remember that the projects listed on the project group page are cached for a few hours
<czajkowski> sinzui: thanks thought I was going crazy
<sinzui> czajkowski, we have a very old bug asking us to change name => launchpad_id and displayname => name so it is clear about what is changing
* abentley changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 3.47*10^2
<jam> bigjools, deryck, flacoste, gary_poster, jam, lifeless, mrevell, sinzui team lead meeting in 5 minutes, hopefully I set it up correctly: https://plus.google.com/hangouts/_/495930018d0f18610dd093d77c49542be643d0e5#
<jelmer> jam: it's a good thing you reminded jam about it ;)
<jam> jelmer: I didn't want to forget
<jam> its a copy paste, I didn't feel like editing anything
<jelmer> I suspected it was something like that, just teasing :)
 * jelmer gets back to reviewing
<noodles775> Hrm, while running all LP tests on a precise instance, I get lots of errors like "TypeError: getresponse() got an unexpected keyword argument 'buffering'" (http://paste.ubuntu.com/1015322/). Is this something I can fix on my instance, or does it need to be a certain python version/distroseries?
 * noodles775 checks dev.launchpad.net again...
<lifeless> noodles775: LP *should* run in precise, but is only *validated* to run on Lucid.
<lifeless> noodles775: https://dev.launchpad.net/Running/LXC
<noodles775> Thanks lifeless
<lifeless> we're still running python2.6 in production
<sinzui> wallyworld_, ping
<wallyworld_> sinzui: hi
<sinzui> wallyworld_, I thought the widget required an element with editicon as a class. I suppose not since you tested it in two browsers to verify it renderered
<james_w> does someone have a few minutes to review https://code.launchpad.net/~james-w/launchpad/drop-filecmp/+merge/108021 please?
<wallyworld_> sinzui: yes, it appears to work fine. but i do recall the editicon requirement. i will check the choiceesource code to see what it's doing
 * sinzui is looking too
<sinzui> wallyworld_, isn't it needed by the spinner..which might not be an issue with enums
<wallyworld_> could be, still looking
<jml> james_w: lgtm :)
<james_w> :-)
<wallyworld_> sinzui: the editing works because clickable_content is true
<wallyworld_> sinzui: yes, because no patch plugin is used since it's not going to the server etc, the editicon is not required
<wallyworld_> but i could add that class to the anchor is suppose
<sinzui> okay, thank you. r=me
<sinzui> wallyworld_, I don't like the editicon since it is not ever clear what the element has to be and when it is used
<wallyworld_> yeah
<sinzui> wallyworld_, wgrant, StevenK, jcsackett, Should we do something about this bug? https://bugs.launchpad.net/launchpad/+bug/246964
<_mup_> Bug #246964: The new privacy/security portlet should use lock icons <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/246964 >
<lifeless> wallyworld_: whasts the status with rev 15325
<wallyworld_> lifeless: rollback branch landed
<lifeless> wallyworld_: in  buildbot at the moment ?
<wallyworld_> lifeless: no, bb still has 4 hours to run before it picks up my landing
<lifeless> ok
<lifeless> we have a near-critical situation with software centre
<wallyworld_> lifeless: when will the new work make bb run faster? soon i hope :-)
<lifeless> hardware has been requested for that
<lifeless> we're expecting 30m runtimes
<wallyworld_> yay
<lifeless> so we may end up fiddling with the landing pipeline over the next 15-24 hours
<lifeless> just a headsup, for details see -ops
<lifeless> who has access to qa http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/15323 ? I'm guessing web ops and soyuzy folk only?
<lifeless> wallyworld_: could I impose o you to qa 15327 - information_type in the branch UI if a feature flag is set.
<wallyworld_> lifeless: no problem. we're in a standup now, can it wait till after that?
<jml> meetings!
<lifeless> wallyworld_: sure thing
<wallyworld_> ok
<lifeless> just smoothing the path for this mini-crisis
<wallyworld_> lifeless: so, i didn't grok fully from scanning -ops how lp landings are connected to a software centre issue
<lifeless> oh foo
<lifeless> my landing setup is bong
<lifeless> wallyworld_: can you lp-land a branch for me ?
<lifeless> https://code.launchpad.net/~lifeless/launchpad/generate-htaccess-speed/+merge/108074
<wallyworld_> lifeless: sure
<lifeless> my lp-land setup is still broken and this is time sensitive
<lifeless> need it in the buildbot run
<wallyworld_> lifeless: justing finishing the standup  now. will do it in 3 minutes and 23 seconds
<jcsackett>  the 23 seconds tells you he's serious.
<lifeless> wallyworld_: thank
<lifeless> wallyworld_: I have to go for my monthly allergy shot now
<wallyworld_> lifeless: np. being done was we speak
<lifeless> wallyworld_: I probably won't be back until the buildbot run has started, so I'm going to hand off the entire 'get it landed' bit to you :) - if it doesn't work *and* you need my input to correct it please ring my nz mobile (in the directory)
<wallyworld_> will do. enjoy you alergy shot :-)
<lifeless> thanks
<mwhudson> lifeless: i presume you've seen http://www.infoq.com/articles/cap-twelve-years-later-how-the-rules-have-changed already?
#launchpad-dev 2012-05-31
<cjwatson> upgrading mawson for some QA
<cjwatson> actually, maybe I can do this on qas - never mind, won't *hurt* to have dogfood more up to date
<StevenK> cjwatson: Doing QA after midnight?
<cjwatson> if I'm awake anyway
<wgrant> cjwatson: Can you tell me when you're done, please?
<cjwatson> wgrant: dogfood upgrade finished - sorry if I got in your way
<wgrant> cjwatson: Not significantly :)
<wgrant> Thanks.
<StevenK> % grep -m 1 def lib/lp/code/tests/test_branch_webservice.py
<StevenK>     def test_createMergeProposal_fails_if_reviewers_and_review_types_are_different_sizes(self):
 * StevenK eyerolls
<lifeless> mwhudson: I havdn't. Thanks.
<mwhudson> lifeless: i found it very interesting
<wallyworld_> lifeless: your branch in bb now. let's hope nothing goes wrong in the next 5 hours :-)
<cody-somerville> mwhudson, Are you still working on that project group milestone bug fix?
<mwhudson> cody-somerville: not really
<mwhudson> cody-somerville: i'm certainly not doing it on work time
<lifeless> thakns wallyworld_
<wallyworld_> np
 * cjwatson gets out of the way of lifeless' deployment pipeline steamroller.
<wallyworld_> wgrant: if you get a chance later, could you run this on df https://pastebin.canonical.com/6714 ... it's a standard person vocab query but with an extra left join from person to account
<wgrant> wallyworld_: That looks like it's missing a digit.
<wgrant> Hm
<wgrant> Although that is a very old person vocab query.
<wgrant> Which probably won't work on the current schema.
<wgrant> Is that just a coincidence?
<wgrant> Incredible.
<wallyworld_> that's what a standard test runs
<wgrant> wallyworld_: Your paste URL is from 2008
<wallyworld_> i just logged sql when running a test
<wgrant> It's a Launchpad pillar search query
<wallyworld_> ffs, sorry https://pastebin.canonical.com/67141/
<wgrant> It's a remarkable coincidence.
<wallyworld_> huh
<wgrant>  Total runtime: 127.058 ms
<wallyworld_> seems ok i guess for a vocab query? i'll paste the version without the account left join
<wgrant> Nice move with the CTE
<wgrant> Yeah
<wgrant> It's an effective planner barrier.
<wgrant> So it almost forces a good plan.
<wgrant> Anyway, I'm wandering off for a while -- be back in 30 or so.
<wallyworld_> https://pastebin.canonical.com/67142/
<wallyworld_> ok
<wallyworld_> the cte was already there
<wgrant> wallyworld_: Within 2ms of the same time
<wallyworld_> wgrant: excellent, thanks
<wgrant> The account filtering is done afterwards on an average of about 102 people, so it's trivial.
<wgrant> Really gone now.
<wallyworld_> shoo
<StevenK> wallyworld_: https://code.launchpad.net/~stevenk/launchpad/branch-information_type-factory/+merge/108094
<wallyworld_> StevenK: info type defaults to public, right? ie if left unspecified in factory call?
<StevenK> wallyworld_: No, it defaults to None, which means don't override it
<StevenK> In the old code private=False ended up being an no-op
<StevenK> wallyworld_: If it defaulted to PUBLIC, If you said makeBranch(product=private_product) the branch would get created as USERDATA and then overridden to PUBLIC.
<lifeless> mwhudson: thanks for the link
<lifeless> "The subtle beauty of a consistent system is that the invariants tend to hold even when the designer does not know what they are."
<lifeless> awesome
<mwhudson> lifeless: i thought you'd like it
<mwhudson> the commutative datatypes sound interesting too
 * StevenK stabs update-cache
<StevenK> wallyworld_, wgrant: https://code.launchpad.net/~stevenk/launchpad/no-makebugtask-private/+merge/108096
<wgrant> StevenK: There was really only one callsite?
<StevenK> Only one callsite that did makeBugTask(private=True), yeah
<wgrant> Approved
 * wallyworld_ just went out a bought a new Galaxy S3 \\o/
<StevenK> wallyworld_: And sold the new laptop to pay for it?
<wallyworld_> nah, that's what credit cards are for :-)
<StevenK> Oh, so your plan is to pay for it eventually
<wallyworld_> but i have to get a micro sim :-(
<wallyworld_> yeah , sometime by 2025
<wallyworld_> but i am really bad at waiting
<StevenK> wallyworld_: Naughty. I pay my credit card off in full every month.
<wallyworld_> i try to, but i spend too much money i don't have
<StevenK> Well, that's the root cause, isn't it? Don't spend money you don't have. :-)
<wallyworld_> but it's new and shiny and i wanted it and i couldn't wait
<StevenK> Didn't your mother ever teach that "I want never gets" ?
<wallyworld_> nope :-)
<StevenK> Haha
 * wallyworld_ decides whether to get a box cutter to try and make the old sim card fit
<StevenK> Hahaa
<adeuring> good morning
<lifeless> wallyworld_: thanks for your assistance this morning
<lifeless> noodles775: https://dev.launchpad.net/Running/LXC
<lifeless> noodles775: may help with your development environment
<wallyworld_> lifeless: no problem, i did very little. i hope everything works well.only 1 hr till bb finishes :-)
<mwhudson> "New code import: nyancat/trunk"
<StevenK> Twitch
<mwhudson> yes
<bigjools> !
 * jml googles nyancat
<mwhudson> jml: don't do it!
<gmb> mgz, You said something yesterday about bzr using a separate pipe for subunit output. Where can I look to find out how you actually go about doing that in a safe, sane, clean and above all quick-to-implement way (note that none of the above values have actually to be true).
<jml> mwhudson: it's a repetitive, high-pitched cat-cake rainbow thing
<mwhudson> jml: yes
<mwhudson> jml: congratulations on not knowing about this particularly useless internet meme until now
<jam1> sinzui: when you're around, I'm having a funny bug wrt launchpad answers. I set myself as an answers contact, and I see myself on https://answers.launchpad.net/launchpad
<jam1> however, I *don't* see myself on any given question (like: https://answers.launchpad.net/launchpad/+question/198964)
<jml> gmb: why not insist on a just, fair, true and loyal implementation?
<jam1> though francis said he could see me on the list
<jml> mwhudson: thanks :)
<jam1> and... I'm not getting emails for new requests, I do seem to get posts to old ones
<gmb> jml, I will accept fair, true and loyal, as long as I can have peace, respect for all, free love and a hard-boiled egg.
<jml> gmb: I'm afraid I rather insist on having my eggs soft-boiled, old chap.
<gmb> Well, no-one's perfect.
<jml> hey
<jml> what difference is there between LaunchpadZopelessLayer and ZopelessDatabaseLayer that might affect mail?
<wgrant> Not much.
<wgrant> Unless it uses the librarian...
<jml> I guess this must, somehow.
<jml> As when I switched it to use ZDL, the failures I got were about mail not being sent.
<wgrant> Huh
<wgrant> I don't think we change the mail config in LaunchpadLayer...
<wgrant> If anything, I'd expect it to send less mail.
<jml> anyway, I guess I'll push that forward to another patch
<jml> oh
<jml> james's thing
<wgrant> james's thing?
<jml> wgrant: https://code.launchpad.net/~james-w/launchpad/drop-filecmp/+merge/108021
<czajkowski> jam: want to mumble and I can talk you through it
<jam> czajkowski: I'm on now
<jam> czajkowski: I only hear static when you talk
<czajkowski> hmm
<jam> we can just do IRC if you prefer
<jam> also, I'd like to document it somewhere, so someone else in the future has access to it
<jml> anyone already have a simple script to check current loc score for a branch?
<jml> otherwise I'll hack one up with diffstat, cut & dc
<gmb> jml, If you find one, let me know. I've got a slack-time card for it.
<gmb> Haven't got round to it yet.
<gmb> *find or make
<cjwatson> http://paste.ubuntu.com/1016103/ (~/bin/diffstat-loc-delta here) postprocesses diffstat to a single number
<cjwatson> (yeah, it's perl, bite me)
<cjwatson> or you could probably repurpose bits of loc-contributions in lp-dev-utils
<jml> cjwatson: heh
<cjwatson> which is vastly more overengineered
<jml> cjwatson: at least it's strict perl
<cjwatson> loc-contributions gets kind of confused if you run it in a non-devel branch and (I think) attributes everything after the branch point to the last rev it has from devel and shows the delta for that commit, which in the general case has nothing to do with the author it's supposed to be examining.
<cjwatson> Probably not desperately hard to fix ...
<bigjools> bzr diff -r submit:|diffstat ?
<StevenK> bigjools: I think jml was after one number
<cjwatson> right, so with my script, bzr di -rsubmit: | diffstat-loc-delta
<bigjools> ah ok, didn't catch on to that
<cjwatson> (or -rancestor: for a bit less noise from bzr)
<bigjools> ancestor needs more typing :)
<cjwatson> bigjools: thanks for doing *PPH.requestDeletion() API exporting way back when, BTW - I only just cottoned onto it being there and wrote a client for it
<jml> jml@lpdev:~/src/launchpad$ bzr damage
<jml> Using submit branch file:///home/jml/.repos/launchpad/devel/
<jml> Damage: 67 lines of code
<bigjools> cjwatson: \o/
<cjwatson> jml: cute
<jml> http://paste.ubuntu.com/1016108/
<bigjools> jml: haha!
<bigjools> so it's net difference
<gmb> mgz, nm, I've found the relevant code.
 * jml searches for a team with suitably broad write permissions.
<cjwatson> jml: http://paste.ubuntu.com/1016109/ ? :-)
<jml> cjwatson: :D
<jml> that's on lp:bzr-damage if anyone wants it. ~canonical has commit access. based on lp:difftodo, which I use all the time.
<jml> gmb: ^
<gmb> jml, Awesome, as they say, sauce. Ta.
<gmb> "they," of course, being the "everyone" in "everyone knows."
 * bigjools is off, g'night folks
<StevenK> wallyworld_: Yours is red because r15335 needstesting
<StevenK> And the QA machinery knows r15325 is a bad rev
<wallyworld_> StevenK: but those 2 revs aren't related
<StevenK> wallyworld_: Your rollback is r15336, so everything between r15325 and it has to be deployable for r15325 to be.
<wallyworld_> ah, ok then
<wallyworld_> StevenK: btw, my scissors and the micro sim experiment worked :-)
<StevenK> Oh, twitch
<mgz> gmb: sorry for delay, trying to hack through to getting code commited while it's all in my head
<mgz> will reply in a sec
<stub> Should all branches to fix a bug land with [incr], or only the second+, or just the first, or what?
<wgrant> stub: Anything that doesn't fix the bug, normally.
<wgrant> stub: It just prevents qa-tagger and me from closing the bug.
<stub> ta
<mgz> gmb: okay, I'm through the thicket, if you've got any questions still about subunit, bug me
<gmb> mgz, At the moment, my questions are all very much of the nature of "that's really straightforward! Now, how the sweet hell do I get zope.testing to do that without breaking the entire universe?"
<gmb> But no fear, I'll bug you as necessary.
<mgz> right, that'll be the tricky bit :)
<jml> fwiw, I did zope.testing's subunit support initially
<jml> so I might be able to help a little
<mgz> jml: the basic desure is to keep the subunit stream off stdout when running tests in more than one process
<jml> why?
<mgz> it's a bit to easy to break the world if a test has something that ends up printing to a standard stream
<jml> isn't that just subunit's passthrough stuff being borked?
<mgz> it doesn't help that the format is super-fragile, but TestProtocolClient(testresult.TestResult): """A TestResult whic
<mgz> ...bad paste
<mgz> anyway, you can pass a dedicated stream, which is then less liable to random breakage
<jml> https://code.launchpad.net/~jml/launchpad/generate-ppa-logging/+merge/108040 needs review
<jml> unfortunately I've made it huge in an attempt to clean up
<jml> wgrant: if you're around... ^
<rick_h_> jml: jcsackett should be around in the next hour/two and is OCR. I think wgrant is pretty EOD/night at this point.
<wgrant> jml: I can't review that tonight, sorry. I'll be more than happy to do it tomorrow if nobody else does it.
<jml> wgrant: np. thanks.
<cjwatson> Hmm, members of ubuntu-archive who aren't in ubuntu-core-dev can't run auto-sync
 * cjwatson ponders whether that's correct
<cjwatson> I can see how it arises, obviously; rather impedes automation though
* jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: jcsackett | Firefighting: - | Critical bugs: 3.47*10^2
<jml> jcsackett: hi
<jcsackett> jml: heya.
<jml> jcsackett: would you please take a look at https://code.launchpad.net/~jml/launchpad/generate-ppa-logging/+merge/108040 ?
<jcsackett> jml: happily.
<jml> jcsackett: it's kind of long. sorry.
<jml> jcsackett: I went far & wide to try to make it line-count neutral. as it is it's still 30 lines in debt.
<jml> jcsackett: oh, you should also note that an earlier version of said patch is currently cowboyed, running on germanium
<jml> jcsackett: thanks.
<jcsackett> jml: got delayed on looking at your MP; i'm a little ways through, but i've noticed that jelmer reviewed your code about 19 hrs ago and approved. did you explicitly want a second review, or just hadn't noticed it had a vote already?
<jml> jcsackett: second review. jelmer approved an earlier, shorter. version for cowboying but not for landing.
<jml> s/.//
<jcsackett> jml: aaah. okay, i understand now.
<jcsackett> jml: one quick note on the LoC thing; you can actually run a tool from lp-dev-utils that tells the total line debt you have. in this case, you're at -42, so the extra 30 still leaves you ahead of the game.
<jml> jcsackett: oh, interesting. This is from a separate arc of work though, so I would personally like to count it separately.
<jcsackett> jml: dig.
 * jml think of 'bzr damage' hacks.
<jml> jcsackett: fwiw, the assertContentEqual change is busted. I'm reverting that locally.
<cjwatson> jcsackett: When you get a chance, I could use a review of https://code.launchpad.net/~cjwatson/launchpad/packageset-score-rename-support/+merge/107981 - fixing this is going to take three successive deployments, so I'd like to get it moving
<jcsackett> cjwatson: sure thing. i'll look as i finish with jml's.
<cjwatson> ta
<jcsackett> jml: r=me on the MP. if you address the comments issues and set the commit msg, i can send that out to land for you.
<jcsackett> cjwatson: taking a look at yours now.
<jml> jcsackett: thanks. will do.
<jml> jcsackett: one thing though, when I said 'numbers are arbitrary', I meant the 3, 45 & 16 I was using for testing, not the constants for how many seconds are in a day or microseconds in a second :)
<jml> jcsackett: I can change either or both of those if you want, although I don't think either change will do much to enhance readability
<jcsackett> jml: my comments are poorly placed. i am referring only to the parts about dropping this stuff when we drop py 2.6
<jml> jcsackett: ah right. can do. I assume I don't have to file a bug for those XXX comments then?
<jcsackett> jml: i see no need for that, and there are plenty of XXX comments without a bug. just name and date are fine. :-)
<jml> jcsackett: cool. thanks. worth changing https://dev.launchpad.net/PolicyandProcess/XXXPolicy then, I reckon.
<jcsackett> jml: probably so, yes.
<jcsackett> cjwatson: r=me.
<cjwatson> jcsackett: Great, thanks!  Could you "Status: Approve" it and I'll land it
<cjwatson> ?
<jcsackett> cjwatson: oh right; you can land but not mark approved. strange corner case to be in. :-P
<cjwatson> Aye
<jcsackett> cjwatson: done.
 * jml <3 wikis
<cjwatson> Lovely.  Off to EC2.
<jml> jcsackett: I'm in the same boat actually.
<jml> jcsackett: changes pushed.
<jcsackett> jml: ok, then i've flipped the final toggle and you may land your branch. :-)
<cjwatson> jml: Good to know it's not just me.
<frankban> jcsackett: could you please review https://code.launchpad.net/~frankban/launchpad/bug-996720/+merge/108196 ?
<jcsackett> frankban: already looking at it. :-)
<frankban> thanks jcsackett :-)
<jcsackett> frankban: r=me. good fix.
<frankban> great, thank you
<cjwatson> What happens to OOPSes generated during tests?
<cjwatson> Actually I guess I don't know if it's an OOPS.  I'm getting an Unauthorized with a rather inscrutable response body, and would like to try to figure out what's trying to fetch the attribute ("items", so not very specific) that's the proximate cause of the Unauthorized.
<cjwatson> ... I've fixed my immediate problem by staring closely at print debugging.  I'm still interested if there are more powerful tools I'm missing out on.
<jml> cjwatson: that's almost certainly caused by zope's security proxy
<jml> cjwatson: afaict, oopses ought to be captured by default within tests.
<cjwatson> For once it actually wasn't the security proxy
<cjwatson> Well, maybe it was, not sure.  I was returning a list of lists of dicts by accident when the interface said list of dicts.
<jml> cjwatson: it'll be the proxy raising that error.
<cjwatson> Perhaps my question would be better phrased as "how do I actually get to see that OOPS?".
<cjwatson> I was getting stuff of the form http://paste.ubuntu.com/1016677/ .
<sinzui> jcsackett, do you have time to review https://code.launchpad.net/~sinzui/launchpad/queue-project-email/+merge/108235
<jcsackett> sinzui: sure.
<jcsackett> sinzui: r=me.
<sinzui> thank you.
* jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 3.47*10^2
<bdmurray> I thought I'd look at doing some launchpad dev and I'm having trouble running it at the moment
<bdmurray> Couldn't find a distribution for 'lazr.jobrunner=0.6'
<czajkowski> bdmurray: have you seen the latest write up http://blog.launchpad.net/general/contributing-to-launchpad
<bdmurray> czajkowski: yes and one thing that page doesn't address is actually running launchpad so you can see what it looks like and interact with it via a browser
<bdmurray> or how to keep your launchpad development system up to date
<cjwatson> bdmurray: did you do just 'bzr pull' rather than utilities/rocketfuel-get?
<cjwatson> That looks like lp-sourcedeps is out of date
<wgrant> bdmurray: bzr up ~/launchpad/lp-sourcedeps/download-cache
<bdmurray> thanks, so is utilities/rocketfuel-get the best way to stay up to date or something else?
<wgrant> bdmurray: That's the best way, yeah.
<wgrant> It mostly just pulls devel, ups download-cache, and runs utilities/update-sourcecode
<cjwatson> Could I have a DB patch number for "ALTER TABLE Packageset RENAME COLUMN score TO relative_build_score;", please?  Bug 1000570.
<_mup_> Bug #1000570: "Packageset.score" is badly named <qa-untestable> <tech-debt> <trivial> <Launchpad itself:In Progress by cjwatson> < https://launchpad.net/bugs/1000570 >
<cjwatson> I've already tested the packageset-score-rename-support branch linked to that bug both with and without that DB patch applied.
<StevenK> cjwatson: 18-4
<cjwatson> Thanks.  Earlier than the tip patches in db-devel?
<cjwatson> I guess it doesn't matter
<StevenK> It does not
<cjwatson> OK.  I'll push it for review tomorrow; it's missed the upcoming NDT anyway, so no rush.
<StevenK> cjwatson: DB patch requires FDT
<StevenK> Oh, sorry, I missed your other comment
<cjwatson> I was unclear.  I mean the code support branch has missed the upcoming NDT anyway, so no rush.  I understand that the DB patch will require an FDT.
<lifeless> sinzui: https://bugs.launchpad.net/launchpad/+bug/1007208 may or may not interest you
<_mup_> Bug #1007208: Searching for 'ev' in the 'add a member' picker fails <Launchpad itself:Triaged> < https://launchpad.net/bugs/1007208 >
<StevenK> cjwatson: We have FDTs until at least Tuesday or Wednesday
<wgrant> lifeless: I think that's a dupe.
<wgrant> lifeless: A dupe of searching for mgz.
<wgrant> We certainly considered it in our picker work.
<lifeless> anyone remember why we have both ~launchpad-reviewers *and* ~canonical-launchpad-reviewers ?
<bigjools> less email?  more email? :)
<lifeless> whoknowsIcertainlydon't
#launchpad-dev 2012-06-01
<wgrant> lifeless: Why's the bugsupervisor not -committers?
<lifeless> wgrant: launchpad-security has an external address, as its also the private bug contact
<lifeless> wgrant: committer access is sufficient to know about security bugs, even if you don't get notified
<lifeless> wgrant: (been like this for ages, not a recent change)
<wgrant> lifeless: wut
<wgrant> lifeless: Bugsupervisor != security
<wgrant> Bugsupervisor also != notifications unless the project is misconfigured.
<lifeless> yes, I know that too; short story - I'm too lazy to reconfigure 30 projects bugsupervisors for no functional gain.
<lifeless> *we* don't need separate teams for this, so using separate teams would add complexity.
<wgrant> Right
<wgrant> Launchpad Security shouldn't exist
<wgrant> It only exists because notifications suck
<wgrant> Its rationale for existence should disappear shortlyu.
<lifeless> indeed
<lifeless> but today, going and changing a bunch of stuff - pointless pain
<lifeless> we can probably just merge launchpad-security into canonical-launchpad-committers in a months time
<wgrant> Yeah
<lifeless> or reconfigure 30 odd projects.
<lifeless> Give you a hint which route *I* would take
<wgrant> API :)
<lifeless> merge is massively easier ;)
<wgrant> That's a bug.
<lifeless> you'll need to expand on that rather massively
<wgrant> That project setup is so awful has to be considered a bug.
<lifeless> indeed
<lifeless> but changing one thing easier than changing 30, is natural, not a bug
<lifeless> doing a cascading change from -project might make sense
<wgrant> It involves a project group.
<lifeless> if project groups were to get fleshed out in any direction
<wgrant> Therefore it is a fundamentally nonsensical operation.
<lifeless> hah, troll
<lifeless> LUNCHTIME
<wgrant> lifeless: Do we want to close bug #982938, or leave it open for a better fix later?
<lifeless> is that the optimisation we did yestarday
<wgrant> Yes
<lifeless> close it
<lifeless> we're now at what, 36 seconds latency on average
<wgrant> Roughly.
<lifeless> the bug will get noisy if we talk about killing that there.
<wgrant> The HIB has caused load so we're seeing some minor issues occasionally.
<wgrant> Yeah, true.
<lifeless> You may want to file another bug saying that it takes ~36seconds on average
<wgrant> wallyworld_: The new JS on +filebug is marvellous.
<lifeless> so ca don't need to.
<lifeless> It would be a kindness.
<wallyworld_> wgrant: yeah. and it will be better soon when i also enhance the info type buttons
<wgrant> wallyworld_: Did you tweak the markup for the tag edit icon?
<wgrant> wallyworld_: It's now immediately adjacent to the final tag, rather than having a small gap.
<wgrant> I'm pretty sure it wasn't like that this morning.
<wallyworld_> wgrant: which page?
<wgrant> https://bugs.launchpad.net/launchpad/+bug/827973
<_mup_> Bug #827973: Unify custom-upload filename parsing and handling <qa-untestable> <tech-debt> <Launchpad itself:Fix Committed by cjwatson> < https://launchpad.net/bugs/827973 >
<wgrant> Or any bug
<wgrant> Note "tech-debt(!)" rather than "tech-debt (!)"
<wallyworld_> wgrant: ah, possibly a small regression, removing a nbsp. i'll fix
<wgrant> Yeah, that's what I suspected. Thanks!
<wallyworld_> thnks for noticing
<lifeless> nbsp rather than a style?
<wgrant> Found an old tab -- there was indeed a space there before.
<wallyworld_> lifeless: the current meme is to use nbsp
<wallyworld_> but style is better of course
<wallyworld_> legacy markup from years ago i suspect
<lifeless> perhaps you can fix using a style then ? :)
<wallyworld_> there's a lot of places
<wallyworld_> and a lot of disclosure stuff todo, but we can do as a slack time effort perhaps
<StevenK> steven@undermined:~/launchpad/lp-branches/branch-information_type-factory% zcat ~/Desktop/branch-information_type-factory-r15326.subunit.gz | testr load >/dev/null
<StevenK> String or Integer object expected for key, unicode found
 * StevenK stabs testr for being crap.
<lifeless> StevenK: yeah, so thats fixed, add the ppa
<lifeless> StevenK: and blame python for breaking API in 2.7.
<StevenK> lifeless: Which ppa?
<lifeless> testing cabal
<lifeless> or yellow
<StevenK> mwhudson: Is lp/codehosting/inmemory.py only testing fakes?
<mwhudson> StevenK: yes
<StevenK> Excellent
<StevenK> Maybe it should move to lp/codehosting/tests then :-)
<mwhudson> yeah probably
<wgrant> testing
<wgrant> normally
<wgrant> Not tests
<mwhudson> it implements a very restricted kind of privacy for branches iirc...
<bigjools> what was Python thinking when it decided that it was a good idea to emulate the C ?
<bigjools> arse
<bigjools> what was Python thinking when it decided that it was a good idea to emulate the C ?: operator
<mwhudson> i have no idea
<mwhudson> i raised my hang to vote for the "this is a bad idea" option during a guido keynote
<mwhudson> more than once, i think...
<lifeless> bigjools: they were thinking 'how much worse can it possibly be'?
<bigjools> And while I am ranting, Django is making me want to be violent with inanimate objects.
<lifeless> howso?
<bigjools> because it's enchanted
<StevenK> s/enchanted/cursed/
<bigjools> there's loads of magic you need to know about
<bigjools> I prefer explicit
 * wallyworld_ likes the ?: operator 
<wallyworld_> nice and compact
<lifeless> wallyworld_: have you *seen* the python version ?
<wallyworld_> yes, i use it and i don't like it but i like the concept
<wallyworld_> it's arse about
<lifeless> wallyworld_: trueclause if condition else falseclause
<bigjools> eschew obfuscation
<wallyworld_> yes, i know
<lifeless> wallyworld_: its insane
<wallyworld_> i prefer the C/java variant
<lifeless> wallyworld_: a major feature of condition ? trueclause : falseclause is its brevity ;)
<wallyworld_> i though the original comment was directed at not liking the ?: operator at all
<lifeless> python's version totally fails
<wallyworld_> yes
<wallyworld_> seems like they wanted the idea but *had* to do it differentl out of princioal
<bigjools> ?: is fairly evil too, it encourages unreadable code except if you have the most basic needs
<bigjools> it was summarily banned in one of my previous jobs
<wallyworld_> hmm. i like it
<wallyworld_> any construct can be abused
<StevenK> wallyworld_: But you like Java ...
<wgrant> wallyworld_: There is some speculation that it was done awkwardly because Guido didn't really want to do it. So he made it terrible so people would shut up but not use it.
<wallyworld_> wgrant: that i would believe
<wallyworld_> StevenK: sigh
 * wallyworld_ off for a bit - kid needs to go to orthodontist $$$$$$$$ :-(
<StevenK> wallyworld_: Are you back?
<wallyworld_> StevenK: yep
<StevenK> wallyworld_: How was the orthodontist, aside from hellishly expensive?
<wallyworld_> StevenK: no money has exchanged hands yet. we need to make a decision. it we be in the order of $7000 if we go ahead :-(
<wallyworld_> i can't see why it costs so muh
<wallyworld_> much
<StevenK> Does private health care handle any of it?
<wgrant> Because they can :)
<wallyworld_> private health will cover some but not much, maybe 20%. have t check
<StevenK> wallyworld_: Ouch, that's still ~ $5500
<StevenK> wallyworld_: So you might need to sell that shiny new phone and a kidney?
<wallyworld_> yeah. and it may be less than 20%. i just made that up. but it won't be more than that afaik
<StevenK> wallyworld_: Medibank Private will cover up to $3,000 depending on your extras cover
<StevenK> But that's a lifetime limit
<StevenK> Not sure how it works for family cover
<wallyworld_> StevenK: i'm with ghmba. need to look into it
<wallyworld_> extras cover is soooo expensive for the top level
<StevenK> s/ for the top level//
 * StevenK stabs the branch scanner
<wallyworld_> StevenK: looking in a minute, justing finishing a mp
<StevenK> wallyworld_: Thanks, I was waiting for the branch to actually scan.
 * StevenK prepares the lynch mob
<lifeless> EOW, have a good weekend
<wgrant> Night lifeless.
<wallyworld_> StevenK: i think "If this branch is public " can be removed from the comment
<wallyworld_> s/can/should perhaps
<StevenK> wallyworld_: If this branch is stacked on a private branch ...
<wallyworld_> StevenK: yes, i think so
<StevenK> wallyworld_: Thanks for the approve, I've pushed up the comment change.
<wallyworld_> np
<StevenK> wallyworld_: How are you enjoying the new phone?
<wallyworld_> StevenK: shared shitless of scratching it
<StevenK> Don't be
<wallyworld_> need to buy a case etc
<wallyworld_> it's very, very nice though
<wallyworld_> well made, fast, excellent screen
<StevenK> wallyworld_: www.youtube.com/watch?v=kmqe9G5TIOg
<wallyworld_> also need to root it
<bigjools> wallyworld_: pervert
<StevenK> wallyworld_: That's for the Galaxy S, but the S2 and S3 have the same glass
<StevenK> bigjools: Says the guy who wanted to have sex with his phone
<wallyworld_> StevenK: bigjools says his nexus screen is quite badly scratched
<StevenK> Strange, my S2 has no scratches
<wallyworld_> StevenK: S3 gas Gorilla Glass 2 afaik
<bigjools> scratched mine when I dropped it when it when flying at about 30km/h on a bike trail
<wallyworld_> perhaps nexus didn't have gg
<wallyworld_> i seem to recall that from somewhere
<StevenK> wallyworld_: Watch the video and try not to cringe
<wallyworld_> i just did :-)
<bigjools> impressive
<wallyworld_> i'm not going to "try this at home" though
<StevenK> There was a better one
<wallyworld_> although i did vut my sim card with a pair of scissors to make it into a micro sim :-)
<wallyworld_> cut
<StevenK> wallyworld_: http://www.youtube.com/watch?v=xaNWuGtAaxM was what I was thinking
<wallyworld_> StevenK: those crazy koreans
<wallyworld_> the knife did leave some scratches
<bigjools> wallyworld_: now you're in the real world with a smartphone, I can start recommending apps
<StevenK> Yeah, but are you going to try that? :-)
<StevenK> Cut the Rope!
<bigjools> Facebook!  lol
<wallyworld_> i've got the Whip already
<StevenK> Soundhound
<wallyworld_> nooooo. none of that social networking rubbish
<bigjools> Where's My Water is great
<wallyworld_> got soundhound and shazam already, and the night sky one
<bigjools> at least install G+?
<wallyworld_> nope
<wgrant> Hahahah
<bigjools> google sky map?
<wallyworld_> yep
<bigjools> GPS Status
<StevenK> bigjools: Meh. Where's My Water did not do it for me like Cut the Rope did
<bigjools> juice defender
 * wallyworld_ detests social networking
<StevenK> Ohh, ++ for juice defender
<bigjools> StevenK: did it not cut the mustard? :)
<bigjools> wallyworld_: luddite
<bigjools> Any.DO
<StevenK> Wind Up Knight is pretty awesome
<bigjools> Cardio Trainer
<wallyworld_> if i never see another tweet or facebook update along the lines of "doing a big dump this morning" it will be too soon
<bigjools> StevenK: I uninstalled that as it wanted micro payments
<wallyworld_> bigjools: i love technology so i'm no luddite
<StevenK> bigjools: I only paid for Book II, Book III and IV I unlocked by playing
<bigjools> Sprinkle
<wallyworld_> well, i know what i'll be doing this weekend
<bigjools> WhatsApp
<bigjools> Wifi Analyzer
<wallyworld_> StevenK: bigjools: the other thing is - i need to get stuff off my ubuntu laptop and ubuntu doesn't like the galaxy phones
<bigjools> Authenticator for the 2FA stuff
<wallyworld_> i have used bluetooth but that's slow as
<wallyworld_> i really must look into a samba set up or something
<bigjools> wallyworld_: really? why?  My GN is fine
<wallyworld_> bigjools: it rcognises the phone but wont mount it
<stub> We are moving away from big dumps to point in time recovery.
<bigjools> Data Counter
<wallyworld_> lol
<bigjools> haha
<StevenK> wallyworld_: Turn on USB debugging
<bigjools> wallyworld_: it uses MTP, rather than USB Mass Storage :/
<wallyworld_> StevenK: will that make it work?
<bigjools> lol
<wallyworld_> so ubunti doesn't support MTP?
<StevenK> wallyworld_: Then it will prompt you when you plug in the cable and you can say mount
<bigjools> it does
<bigjools> just makes it harder IMO
<bigjools> ironically :/
<wallyworld_> i'll try getting it sorted out later tonight
<bigjools> at least it's plug and go for 3G data
<wallyworld_> i did install an ftp app and ftped some stuff across also
<wallyworld_> i'm scared of rooting it cause some people have said doing that can fuck the security chip used for wireless payments
<bigjools> yeah it'll stop working
<wgrant> If wireless payment security depends on a DRMed handset, their wireless payment system has some pretty terrible problems.
<bigjools> won't fuck it, just depends on boot loader sig or something
<bigjools> since when have people implementing DRM been rational?
<wallyworld_> i'll wait till people figure stuff out properly, bt i do want to root it so i cab block ads properly and install a full backup app etc
<bigjools> I have a backup app that doesn't need root
<bigjools> MyBackup
<bigjools> pay-for
<wallyworld_> sure, but ones which do "everything" do
<wallyworld_> or so i thought
<bigjools> mine does everything
<bigjools> I have even used it in anger :)
<wallyworld_> hmmm. will have to take another look then. but i'm sure i've seen them and they say "well we can get most stuff but if you want app settings and aps or whatever, you need root"
<StevenK> Titanium
<bigjools> well, that's bollocks
<wallyworld_> i tried one on belinda's S2 and it couldn't do everything without root
<wallyworld_> can recall the details
<wallyworld_> can't
<bigjools> get MyBackup
<wallyworld_> wil do, or titanium
<bigjools> I even have it set to auto backup in the middle of every night
<wallyworld_> but aren't you using your porn then?
<bigjools> my phone has a backuip schedule.... ffs
<wgrant> wallyworld_: Won't that a.sprite margin introduce some oddness with eg. person links?
<wallyworld_> wgrant: they use span i'm pretty sure - i checked that i think
<wallyworld_> that's why is used a.sprite and not .sprite
<wgrant> Nope, person and bug links are straight a.sprites.
<wgrant> We want the margin only on a special class which trails the thing we're editing, I guess.
<wgrant> I don't know if that class exists :/
<wallyworld_> hmmm. yes you are right. i can't recall where i found spans now, but i did see tham
<wgrant> You may be thinking of team links, where the custom icon is an img.
<wgrant> They probably use a span.
<wallyworld_> could be
<wallyworld_> i wonder if we can add a class "nopadding" or something
<wallyworld_> wgrant: adding a nopadding class and new style to exclde the padding seems to work fine
<wgrant> wallyworld_: That seems pretty wrong.
<wallyworld_> or i could go the other way
<wgrant> wallyworld_: Seems more like we want an actionicon class or something like that.
<wallyworld_> a padding style for sprite icons classes
<wgrant> Because the thing with the special case is detached action icons.
<wallyworld_> i can just enumerate the srpite classes in a css selector
<wallyworld_> and introduce padding for those
<wgrant> Won't work
<wgrant> We have some edit icons with text in the <a>
<wallyworld_> why does that affect anything?
<adeuring> good morning
<wgrant> wallyworld_: Not all edit sprites are icons that trail the thing they're editing.
<wallyworld_> wgrant: sure, but they still need a little bit of left padding
<wgrant> wallyworld_: The trait that makes them require the margin is that they trail the thing they're editing.
<wgrant> Not that they're an edit sprite.
<wallyworld_> in the cases i've seen, a little bit of left padding is fine
<wgrant> Perhaps, but we should put padding where we want it.
<wgrant> Not where it accidentally appears and doesn't look terrible.
* adeuring changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: adeuring | Firefighting: - | Critical bugs: 3.47*10^2
<jml> oh btw
<jml> I did a bunch of ec2 test runs yesterday and got no email
<jml> for all I know, the instances are still running.
* bac changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: adeuring,bac | Firefighting: - | Critical bugs: 3.47*10^2
<bac> hi adeuring
<adeuring> morning bac
<bac> adeuring: done many reviews today?
<adeuring> bac: just two
<bac> cool.  i'll take colin's in a bit, unless you want to
<sinzui> czajkowski, a "timeout" is not an "oops". Lots of problems show up in oops reports. We use the "oops" tags for exceptions. Other critical shown in the oops reports are "404", "403", and "timeout"
<flacoste> jcsackett, sinzui: nice work on the new privacy banner :-)
<sinzui> flacoste, I will convey that to the team. We expect to show information types on all Lp next week.
<flacoste> sinzui: it works very smoothly on the filebug screen!
<sinzui> flacoste, It will be very nice when we complete the field updates so that there is one consistent way of showing the information
<czajkowski> sinzui: ok thanks.
<sinzui> czajkowski, maybe we should replace the "oops" tags with "exception"
<wallyworld_> sinzui: "What sprite is not an action?" Answer - person links <a class="sprite person">Fred</a>
<czajkowski> sinzui: I don't know tbh, am only doing this while diogo is away so rather new to me to be honest and the tags still baffling me
<czajkowski> *baffle
<sinzui> wallyworld_, that is an action since I can "activate" it. maybe you mean js-action?
<wallyworld_> sinzui: and also product links etc. those foiled my plans to add left margin to all a.sprite elements
<wallyworld_> i mean the little yellow/red/gree/blue icons to the right of some text
<wallyworld_> which initiate a popup or add or help overlay etc
<jml> (repeat) I did a bunch of ec2 test runs yesterday and got no email
<jml> is this a known issue?
<wallyworld_> jml: sometimes that happens if there are too many failures and the attachment becomes too large. well, that's been the case for me on occasion
<jml> wallyworld_: hmm. thanks. if so, I'm in a poor situation.
<jml> because I'll never know what's wrong with my code.
<wallyworld_> jml: you have to run the test run with the console left open
<wallyworld_> so you csan see the errors and/or access the log
<jml> on that machine that I keep up all the time
<wallyworld_> yeah, that's the one :-)
<jml> now where did I put it?
<wallyworld_> jml: you can also log on via a web browser
<wallyworld_> even after the console session is closed, ie later on
<wallyworld_> it logs the web address to use when ec2 starts
<wallyworld_> from there, you can load the log into the browser
<wallyworld_> so long as you do it before everything terminates at the end of the run
<jml> right.
<jml> instances are down atm.
<jml> so that data is irrevocably lost.
<wallyworld_> :-(
<jml> putting me another day behind :\
<wallyworld_> but i was just guess when i suggested the reason for not getting email
<wallyworld_> it's been the case for me but you may well have a different issue
<wgrant> jml: It may also be that your bzr mail config is unsuitable.
<jml> wgrant: that's very likely.
<jml> wgrant: although I would have guessed that ec2 would have warned me.
<jml> it does a *lot* of verification
<wgrant> jml: Heh
<wgrant> jml: email must be @canonical.com, smtp_{server,username,password} must be normal Canonical ones.
<wgrant> (well, it could probably also be a personal one, but the Canonical one is known to work :))
* adeuring changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: bac | Firefighting: - | Critical bugs: 3.47*10^2
<cjwatson> bac: did you get a chance to review my branch?  you said something earlier that you were going to
<bac> cjwatson: sorry i got tied up with something else.  i can take a break and look at it now.
<cjwatson> np, just wondering
<bac> cjwatson: it looks good.  i'll approve it in a bit.  thanks!
<cjwatson> awesome, thanks
<cjwatson> Now if only I can make LFA webservice serialisation bend to my will ...
<cjwatson> I wonder if exported(List(value_type=Bytes())) just doesn't work
* bac changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 3.47*10^2
<Beret> anyone happen to know if searchTasks(search_text=bugidnumber) is reliable?
<abentley> Beret, I suggest doing bugs[bugidnumber] instead.
<Beret> ok, thanks
<sinzui> The rain is falling hard. I expect to fall off the net in a few minutes
<cjwatson> Any lazr.restful experts about?  I'm having a hard time representing something on the webservice
<cjwatson> I have an object one of whose properties is a list of LibraryFileAliases
<cjwatson> I'd like to expose this; the most elegant representation seems to be a list of Bytes, so that they become HostedFile objects on the client end
<cjwatson> However, this doesn't seem to work.  Firstly, the individual Bytes objects don't get sensible URLs, and I'm not sure how to get them into a useful tree (short of going off and defining extra collection objects, which I was hoping to avoid).  Secondly, even if I hardcode a URL suffix with Bytes(__name__="foo") (which obviously won't work properly for >1 element, but as a test), the URL in the JSON representation still ...
<cjwatson> ... doesn't seem to get deserialised properly.
<cjwatson> Am I just fundamentally out of luck with this approach?  Do I need to punt and return a list of URLs instead?
<cjwatson> Or maybe a collection of little PackageUploadFile objects so that I don't have to anger wgrant by exporting the existing PackageUploadSource etc. interfaces (oh look, I've made the problem concrete)
<lifeless> cjwatson: so, LFAs and the API are horrible IMO
<lifeless> cjwatson: I have no idea what will work well
<lifeless> cjwatson: note that the api client has to talk to LP for each one anyway, if they are private, to get a time limited token
<lifeless> cjwatson: and we really don't want to generate TLTs for things that are not being accessed
<cjwatson> would URLs be better?  they seem less convenient from the client's point of view
<cjwatson> I dunno, I guess you can take the URL and stream the contents of the resource it points to more conveniently than you can get HostedFile to stream
<cjwatson> I have trouble caring about private uploads for this beyond making sure I don't inadvertently expose anything that should be private; I'm mostly trying to get the Ubuntu upload queues more usable and none of the uploads there will be private
<cjwatson> I was hoping that if bug 1007195 were fixed then there might be some definite value in using HostedFiles
<_mup_> Bug #1007195: Librarian-backed HostedFile objects do not expose SHA-1 hash <soyuz-build> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1007195 >
<lifeless> if you know that all the things this api deals with will be public, urls would be fine; TBH I'd need to page in some horrible stuff to even advise properly here
<lifeless> and its saturday, and both girls are sick, so I'm not really here enough to do that
<cjwatson> heh, mind you it's a devel API with one expected client of any interest
<cjwatson> well, one set of clients under the same maintenance
<cjwatson> so I could just say screw it, do URLs for now, and if we need something more elaborate later, do that
<cjwatson> less LoC in LP that way too :)
<cjwatson> er, fewer, whatever
<lifeless> sure
<cjwatson> Thanks for the advice
<cjwatson> Hope the girls are better soon
<cjwatson> My little ones are away so I'm hacking late ;-)
<lifeless> hmm, jaunty->lucid on my old firewall, by hand. Should be fun.
<cjwatson> upgrading mawson for some QA (though I might not actually get to said QA before tomorrow morning, but I have to leave on time ...)
#launchpad-dev 2012-06-02
<cjwatson> StevenK: Help?  Upgrading dogfood results in http://paste.ubuntu.com/1018822/
<cjwatson> 'column "private" of relation "bug" does not exist'
<cjwatson> Hmm.  Which is odd, given that there's no such COMMENT in devel
<cjwatson> Urgh, OK, ton of conflicts, never mind
<wgrant> cjwatson: Ah yeah, it was used for some emergency pre-cowboy QA. reverting it all should work
<cjwatson> Does anyone care about keeping a copy, or just 'bzr revert'?
<wgrant> bzr revert
<cjwatson> OK, it's running again now, thanks
<wgrant> cjwatson: Are you sure we have a new enough dpkg?
<wgrant> cjwatson: This code is run on appservers too.
<cjwatson> I was just wondering that.  I think, though, that the only problem with the version on appservers is that it doesn't know that armhf matches any-arm.
<cjwatson> Which is annoying but no worse than today.
<cjwatson> It should know that armel matches any-arm, which is an improvement.
<wgrant> Also, shelling out randomly to bits of dpkg doesn't strike me as a truly marvellous idea, but perhaps my doubts in its security are unfounded today.
<cjwatson> The arguments are strictly controlled here, and there doesn't exist a better interface.
<cjwatson> And dpkg-architecture is not particularly complex.
<cjwatson> Either trying to parse /usr/share/dpkg/*table or extending the previous broken parser is IMO a worse idea.
<cjwatson> If there were a python-dpkg I'd use it :-)
<cjwatson> I've just checked dpkg-architecture's behaviour if given a pathological argument to -i (containing ;, `, newline).  It reads the argument unmolested and doesn't do anything silly with it.
<wgrant> Yeah, it seems OK.
<cjwatson> So I do think this is safe.  But if you think there's a real security concern then of course feel free to revert.  I'm going to be away from ~8 hours from now until late Tuesday.
<wgrant> Shouldn't you be away until Wednesday? :)
<wgrant> No, I think it's OK, but I am wary of dpkg nowadays.
<cjwatson> Well, I mean technically I'm back home on Tuesday.  Probably won't be doing much that evening.
<wgrant> dpkg-architecture seems to have avoided the horribleification that abounds in most of the rest of dpkg nowadays.
<cjwatson> It might be worth patching in a bit more armhf understanding here, but I don't think it's desperately urgent.
<wgrant> Right.
<cjwatson> Only two packages in quantal are using any-arm* at all.
<wgrant> But we now need to be sure to upgrade the appservers with any new dpkg stuff that we previously only had to push out to Soyuz.
<cjwatson> (picolisp, scsh-0.6)
<cjwatson> Hmm.  Can you think of a sane alternative, if that's particularly painful?
<wgrant> No.
<wgrant> It's just a change from what we've done in the past.
<cjwatson> Also, does http://paste.ubuntu.com/1018886/ look like a vaguely reasonable queue binary override API, based on our earlier conversation?
<cjwatson> If that's usable then I think I have an approximately usable replacement for queue.
<wgrant> cjwatson: That looks pretty good. Doesn't handle overrides differing between archs, but hopefully you don't care about that...
<cjwatson> Not really.  OTOH if we actually want that then we can override individual queue items.
<cjwatson> (This is on PackageUpload.)
<wgrant> Yeah, but delayed copies etc.
<wgrant> All the builds in a single upload.
<cjwatson> I'm not sure I've ever tried overriding a delayed copy :-)
<cjwatson> Maybe I should given the chaos with kernel copies.
<cjwatson> Yeah, for those I don't think we need arch-specific handling anyway.
<cjwatson> If we do in future I could add an architecture key.
<wgrant> Yep
<cjwatson> queue-api LoC +486 and scripts/ftpmaster-tools/queue + lib/lp/soyuz/scripts/queue.py = 862, never mind whatever tentacles that has elsewhere ...
<cjwatson> I'll see about getting that in for review on Wednesday, then.
<wgrant> Excellent.
#launchpad-dev 2013-05-27
<wgrant> StevenK: rejectFromQueue(rejection_comment='foo') sounds a bit redundant.
<StevenK> wgrant: comment ?
<wgrant> Ja
<StevenK> wgrant: http://pastebin.ubuntu.com/5705501/
<wgrant> StevenK: What happens if the user is None?
<wgrant> (it crashes)
<StevenK> wgrant: user is set by the API, and the browser callsite sets it already -- it was added for the auditorclient call
<wgrant> StevenK: Yes, but it's optiona;
<wgrant> l
<wgrant> If anything (eg. a test) doesn't pass it, it will crash
<StevenK> wgrant: http://pastebin.ubuntu.com/5705509/
<wgrant> It may be more clear to if not user/elif comment/else
<wgrant> if user is None even
<StevenK> wgrant: http://pastebin.ubuntu.com/5705514/
<wgrant> StevenK: You could also take this opportunity to move the hardcoded summary_text out of notifications.py:209. Otherwise I think that's fine
<StevenK> wgrant: That MP has updated.
<wgrant> StevenK: r=me
<wgrant> StevenK: Oh, I would have done the garbo after the model+respect branch
<wgrant> But it doesn't really matter
<StevenK> The garbo is going to take about 5 minutes
<wgrant> Yes, but it's another deployment
<wgrant> The garbo is unnecessary for feature deployment, so all it does is delay things
<wgrant> (since the model branch can be deployed fine with the column still nullable)
<StevenK> wgrant: I can reflow them if you wish
<StevenK> wgrant: I have a greasy hack that fixes DF's homepage
<wgrant> StevenK: No need to reflow, just for future
<wgrant> The branches that provide value should be as early as possible
<wgrant> Though won't matter so much later this week, I guess
<StevenK> wgrant: Is there a cronjob that sends the upload rejection mails, or is it done in-request?
<wgrant> StevenK: In-request.
<wgrant> Not a big issue atm, because it rarely results in more than about 3 emails.
<StevenK> Sure, but I want to see what DF is going to send
<StevenK> Well, "send"
<wgrant> StevenK: It'll send it to the staging mailbox
<StevenK> Yeah, I figured that out, and I'm waiting for thunderbird
<StevenK> AttributeError: https://api.dogfood.launchpad.net/devel/ubuntu/saucy/+upload/5592805 object has no attribute 'rejectFromQueue'
<StevenK> Handy
<wgrant> StevenK: Perhaps your WADL is particularly archaic
<StevenK> Killing the cache looks to have helped
<StevenK> Rejected:
<StevenK> Rejected by Steve Kowalik: Because I can.
<StevenK> acl (2.2.51-8ubuntu3+df1) saucy; urgency=low
<StevenK>   * Saucy
<wgrant> :)
<StevenK> wgrant: Changed by you, too
<wgrant> :(
<StevenK> wgrant: Do I need to dig it up again and un-reject it?
<wgrant> StevenK: No
<wgrant> Now we have builders we can actually upload things :)
<RoelV> hi #launchpad engineers!
<RoelV> I am after a way to get the number of Bugs Opened, number of Bugs Closed, number of Bugs In progress
<RoelV> per month
<RoelV> how can I get this?
#launchpad-dev 2013-05-28
<wgrant> StevenK: You broke the build, btw
<StevenK> Blah
<StevenK> wgrant: I think I'll set summary_text to 'Rejected by archive administrator.' rather than None, which will solve one of the failures.
<wgrant> StevenK: Or find any tests that don't pass a user and fix them
<wgrant> There's probably not many
<StevenK> You'd prefer summary_text to be None rather than a generic message in the case user and comment are None?
<wgrant> StevenK: I'd prefer the user argument to be mandatory, since there's no reason for it to not be
<wgrant> And given the few failures it seems like there are probably few tests
<StevenK> wgrant: Well, that can be fixed pretty easily
<StevenK> wgrant: http://pastebin.ubuntu.com/5708682/
<wgrant> -            Rejected by archive administrator.
<wgrant> +            None
<wgrant> Otherwise that looks reasonable
<StevenK> wgrant: That test is calling notify() directly
<wgrant> StevenK: Ah, I'd consider passing in a summary_text, then
<wgrant> Because no rejection email will normally have None for summary_text
<StevenK> wgrant: The diff for that test is now:
<StevenK>              person, None, [bpr], [], archive, distroseries, pocket,
<StevenK> +            summary_text="Rejected by archive administrator.",
<StevenK>              action='rejected')
<wgrant> Great
 * StevenK tosses it at buildbot
<StevenK> iharness seems to want to grab __dict__ for any objects you ask it to print, which is very annoying
<StevenK> Blah
<StevenK> DistroSeries:+queue, stop lying!
<StevenK> Ah ha, the archive is wrong
<wgrant> What are you breaking?
<StevenK> wgrant: Adding the textbox to DistroSeries:+queue
<StevenK> wgrant: The template gets compiled by Zope, or I can change it without restarting the appserver?
<wgrant> StevenK: Template changes don't require an appserver restart today
<wgrant> (which is part of the reason that our current TAL renderer is so ridiculously slow)
<StevenK> Hah
<StevenK> Oh, damn it
<StevenK> It's all one row
<wgrant> Hm?
<StevenK> With <br/>'s and vertical-align: bottom
<wgrant> Oh lovely.
<StevenK> Hmmm, this might work.
<StevenK> wgrant: Do you want a diff of the template or a screenshot?
<wgrant> StevenK: Why not both?
<StevenK> wgrant: http://people.canonical.com/~stevenk/reject-reason-queue.jpg and  http://pastebin.ubuntu.com/5708984/
<wgrant> StevenK: I'td say "Rejection comment" (widget captions aren't title case), and have Reject below Accept so the columns look less bad
<wgrant> And maybe make the textbox wider, given there's so much space?
<wgrant> Like I might put "Rejection comment" in the first two columns like "Override for selected uploads", and then have the textbox span the three override cols.
<StevenK> wgrant: http://people.canonical.com/~stevenk/reject-reason-queue-1.jpg and http://pastebin.ubuntu.com/5708991/
<wgrant> StevenK: The CSS on the label is different
<wgrant> Also your GTK theme is fugly
<StevenK> Meh
<StevenK> wgrant: Refresh -1, that has the updated CSS. http://pastebin.ubuntu.com/5708997/
<wgrant> Your GTK theme still sucks, but otherwise that looks fine :)
<StevenK> Blah
<StevenK> Now to wire it up
<wgrant> Should be nice and easy.
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/reject-reason-plus-queue/+merge/165960
<wgrant> StevenK: That's not an error message
<wgrant> "Rejection comment required." maybe
<wgrant> Also, that queue_action_* stuff seems entirely pointless. I'd inline it to be clearer.
<wgrant> it's even shorter...
<StevenK> wgrant: You'd rather an if action ?
<wgrant> Yes
<wgrant> there's only two cases, and I can't see us adding more
<wgrant> The dynamic lookup almost makes sense for, say, build results
<wgrant> But for accept/reject? Unnecessary complication.
<wgrant> Also, I'd think assertStatus would take a PackagePublishingStatus, not a string, but either works.
<StevenK> wgrant: I'm happy to switch to PackagePublishingStatus
<wgrant> That would put it more in line with every other test in the codebase
<wgrant> So I think it's a good idea
<StevenK> ITYM PackageUploadStatus, but yeah
<wgrant> Er, that, yeah
<wgrant> I'm working with publication methods at the moment, can you tell? :)
<StevenK> :-)
<StevenK> wgrant: Will my new test class cause all tests in the super class to be run twice?
<wgrant> StevenK: Yes
<wgrant> I'd merge the two classes, probably
<wgrant> Rather than extracting a base
<StevenK> wgrant: http://pastebin.ubuntu.com/5709168/
<wgrant> Looks good
<StevenK> wgrant: Any other objections, or shall I push?
<wgrant> Push!
<StevenK> wgrant: The MP is updated
<wgrant> StevenK: r=me
<czajkowski> jtv:  Is there any way to set up rosetta on a branch owned by a public team?
<jtv> Hi czajkowski â you mean to export translations to?  I think there have been changes in this area, so for the current state of affairs you'd best talk to the Australians.
<jtv> The situation when I worked with it was that you had to own the branch in order to select it, but then of course you could change its ownership.
<czajkowski> jtv: cool thanks wgrant has asked for clarification from the user in lp land
<czajkowski> thanks
#launchpad-dev 2013-05-29
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/plus-queue-errors/+merge/166164
<wgrant> StevenK: What if you just use addErrorNotification?
<wgrant> Rather than having separate error styling
<StevenK> wgrant: Because the code wants to bail out early from the POSTed method
<wgrant> StevenK: addErrorNotification and return?
<wgrant> Anyway, lunching, bbs
<StevenK> wgrant: addErrorNotification and return results in no effect
<StevenK> wgrant: addErrorNotification, redirecting and then returning does work, but will probably result in you murdering me.
<wgrant> StevenK: Ah, because the request only picks up notifications at the start, I guess.
<wgrant> StevenK: The existing MP is OK then, I guess.
<StevenK> wgrant: +1 it, I shall land and then we can finally deploy
<StevenK> Which will probably have to wait
<wgrant> StevenK: scan
<StevenK> Oh, damn. span?
<StevenK> -    <scan tal:content="error"></scan>
<StevenK> +    <span tal:content="error"></span>
<wgrant> That's more plausible, unless HTML5 is innovating
<StevenK> It's like the <video> tag
<StevenK> Except it involves instruments of evil.
<StevenK> wgrant: The MP has updated.
<StevenK> Ah, indeed
 * StevenK lands
<wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/ddeb-copy-overrides/+merge/166188
<wgrant> I've done reasonably extensive testing locally, and I think this is the last LP-side bit
<StevenK> wgrant: Sorry, looking.
<StevenK> wgrant: r=me. I'm landing the garbo deletion and the use of permit_obsolete_series now.
<wgrant> StevenK: Thanks
#launchpad-dev 2013-05-30
<wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/bpph-is-debug/+merge/166416
<StevenK> wgrant: You're going to switch other code to making use of IBPPH.is_debug in a later branch?
<wgrant> It's not applicable to most internal callsites
<wgrant> But you're right, there might be one or two
<StevenK> wgrant: In this branch, or later?
<wgrant> this
<wgrant> StevenK: Updated
<StevenK> Much better.
<StevenK> wgrant: r=me
<wgrant> Thanks
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/shift-obsolete-check/+merge/166425
<wgrant> StevenK: k
<StevenK> wgrant: I can write PPA tests for that method for penance if you wish
<wgrant> StevenK: Not particularly
#launchpad-dev 2013-05-31
<StevenK> wgrant: Did you spy bug 1186050?
<_mup_> Bug #1186050: Remove Packages-arch-specific use from Soyuz <lp-soyuz> <soyuz-build> <soyuz-core> <Launchpad itself:New> <https://launchpad.net/bugs/1186050>
<wgrant> Yeah
<StevenK> I think I have a branch that destroys it
<StevenK> Just deciding if I want to completly destroy lp.soyuz.pas, and if so where determineArchitecturesToBuild will live.
<wgrant> StevenK: It even fixes a critical
<wgrant> Ah, no
<wgrant> We still use dpkg-architecture for parsing the hint list, of course
<StevenK> Yeah
<StevenK> wgrant: So I want to destroy lp.soyuz.pas (and I think I do, since the name gives me nightmares), I'm still unsure where to stuff it
<wgrant> StevenK: The new code is only 80 lines. I'd do one of two things: a) move it to model.publishing, or b) switch the pubrec arg to an archive and an SPR, and move the whole thing to model.bpb or similar.
<wgrant> Or c) switch the pubrec argh and move it to a new small module in adapters
<wgrant> Probably c.
<StevenK> Yeah, adapters.arch or dpkgarch was what I thinking
<wgrant> adapters.buildarch, perhaps
<StevenK> Right
<StevenK> wgrant: 'Switch the pubrec argument' ?
<StevenK> To spph?
<wgrant> StevenK: Replace it with an archive and an spr
<wgrant> There's no point having it depend on a publication
<wgrant> When all it needs is an archive and an SPR
<StevenK> Right, I see that.
<wgrant> You could even give it a hintlist rather than an SPR
<wgrant> To simplify tests further
<wgrant> Now that we no longer need to do source/binary name matching
<StevenK> steven@undermined:~/launchpad/lp-branches/destroy-pas-yay% bzr di | diffstat -s
<StevenK>  6 files changed, 41 insertions(+), 371 deletions(-)
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/destroy-pas-yay/+merge/166639
<wgrant> StevenK: Could I convince you to name determineArchitecturesToBuild legally?
<wgrant> Also, order should be archive, distroseries, legal_archseries, I think
<StevenK> wgrant: Sure, rename to what, since there's like two callsites
<wgrant> StevenK: determine_architectures_to_build
<wgrant> It's currently disguised as a method
<wgrant> Also, I thought there was only one non-test callsite
<StevenK> Right, the second callsite is the test
<StevenK> wgrant: http://pastebin.ubuntu.com/5718806/
<wgrant> This will mean that things that go through unapproved or new might actually get the same sort of builds as autoapproved ones :)
<wgrant> And we can remove the stupid builddmaster dirs from haetae and pepo
<StevenK> Yeah
<wgrant> :)
<StevenK> wgrant: Any other issues?
<wgrant> I think that's about it
<wgrant> It was just a matter of deleting code until I convinced you to fix issues that had been bothering me for years :)
<StevenK> Well.
<StevenK> I made the mistake of telling infinity I was looking for stuff to do, and he suggested that since I like deleting stuff ...
<StevenK> wgrant: Just making sure test_buildarch and test_publishing are happy before I push.
<StevenK> wgrant: Can haz +1?
<wgrant> StevenK: Sec, cats needed attention
<StevenK> wgrant: Duh, that's their default state.
<wgrant> StevenK: r=me
<StevenK> wgrant: Is that your QA I see?
<StevenK> Since Thunderbird is convinced it must keep reading the staging mailbox
<wgrant> StevenK: I just rejected lots of stuff
<wgrant> To clear out raring's queue
#launchpad-dev 2014-05-26
<RoelV> join #DS
#launchpad-dev 2014-05-27
<cjwatson> wgrant: not related to http://www.postgresql.org/message-id/E1Vq1BO-00064N-Qn@wrigleys.postgresql.org and thread maybe?
#launchpad-dev 2014-05-29
<StevenK> wallyworld: How much did you enjoy Origin yesteday? :-P
<wallyworld> fuck off :-(
<wallyworld> i was there at the ground too :-(
<wallyworld> first time we have lost the opening game at lang park for 11 years
 * wallyworld is sad
<wallyworld> how's hp? miss canonical yet?
<StevenK> wallyworld: It's great fun, and only the people.
<wallyworld> present company excluded :-)
<StevenK> Haha
<wallyworld> i'm sure wgrant misses you
<lifeless> wallyworld: oh hai
<wallyworld> hi there
<wallyworld> how's things
<lifeless> wallyworld: fantastic :) - you? been in malta?
<wallyworld> nah, stuck here in brisbane
<wallyworld> we had our cloud sprint back in may
<lifeless> wallyworld: it is may :)
<wallyworld> oh, april, early may
<wallyworld> i thought it was june already
<wallyworld> getting ahead of myself
<lifeless> 2 more days I think
<wallyworld> how's all the openstack stuff going?
<lifeless> very well
<wallyworld> good to hear, i woud expect nothing less from you :-)
<lifeless> we're just starting on scale and perf stuff
<wallyworld> always important
<lifeless> got a big project to take heat to the next level
<lifeless> spamaps is in india right now kicking that off :)
<wallyworld> so things hotting up
<wallyworld> ha ha ha
<lifeless> trolololol
<lifeless> :)
<lifeless> yeah
<wallyworld> sad that this channel gets little traffic nowadays :-(
<lifeless> yeah, less devs less chatter :(
<wallyworld> lots of traffic on #juju-dev. we'll all way too busy
<lifeless> ah, I'm only in #juju
<wallyworld> yeah #juju is more end user / customer focused
<dpm> morning wgrant, I'm looking at the translations templates in LP before opening translations, and I see translations are all empty. Has the copy of translations from previous releases happened at all? E.g. https://translations.launchpad.net/ubuntu/utopic/+lang/ca
<wgrant> dpm: Stats are regenerated only every couple of days. If you click on eg the top untranslated link, you'll see that there's actually nothing untranslated.
<wgrant> But the counts shown on the page you linked have to be cached.
<wgrant> Actually, as there are technically no changes these could take a week to update.
<dpm> wgrant, is there a way to trigger the stats update? I wouldn't want to open translations with these stats, and I'd rather not have to wait for another week
<wgrant> I'll see what I can do, but my team's knowledge of Translations is still very limited.
<dpm> thanks wgrant, I appreciate it
<jtv> dpm, wgrant: aren't the stats supposed to update anyway as the imports happen?
<wgrant> jtv: Right, but there were only about 900 approved imports
<dpm> jtv, I've no idea, but I didn't recall having to wait for a week for stats to get updated in previous openings
<wgrant> I'll try to investigate later, but busy sprint is busy.
 * wgrant adds a task.
<dpm> thanks wgrant!
<jtv> IIRC the problem used to be that there were several _layers_ of cached stats.
<wgrant> Yeah, the POT's language totals are a separate nightly job
<wgrant> But even the POFile stats are out of date here.
<jtv> First you had stats per POFile, and then there were aggregated per-language and per-template stats.
<wgrant> dpm: Have you seen anything to suggest that it's more than jsust the stats that are borked?
<wgrant> AFAICT the data itself is fine.
<wgrant> But I'm not 100% sure.
<wgrant> https://translations.launchpad.net/ubuntu/utopic/+source/binutils/+pots/bfd looks fine
<dpm> wgrant, I went through the imports queue and approved a few templates that needed to get in. So the imports queue looked ok to me. I've now checked a couple of translations for 'ca' and they look ok to me too
<dpm> wgrant, yeah, individual templates look good too, it seems to be only the big list of templates for a language
<wgrant> i'd only expect DistroSeriesLanguage to be totally broken at this point, but it's clearly more than that.
<wgrant> Where does that page get those stats :S
<wgrant> Not from POFile.
<wgrant> Hm
<wgrant> I think it's getting dummy POFiles.
<wgrant> But why?
<wgrant> It's not even trying to look for the real ones.
<jtv> There's another separate table somewhere in a forgotten corner for aggregated statistics, isn't there?
<jtv> Dummy POFiles... might those happen because translations aren't open?
<jtv> Shouldn't, but there's probably quite a stack of software to decide whether a POFile is ready for use.
<dpm> wgrant, I've filed an RT to set up the utopic exports to start generating language packs. Who should I ping about it - webops in #launchpad-ops?
<wgrant> dpm: They'll ask for a specific crontab change that I'll have to give them.
<dpm> wgrant, so how do I do this best? Shall I forward you the RT?
<wgrant> dpm: Right, Cc me and I'll reply.
<dpm> wgrant, ok, done
<wgrant> Thanks
<wgrant> cprov: Could you please review the two small branches branchs I have on activereviews?
<cjohnston> what, my reviews aren't good enough
<cprov> wgrant: sure
<cprov> wgrant: am I too late ?
<wgrant> cprov: Too late?
<cprov> wgrant: could not find your branches in my queue.
<wgrant> cprov: Oh, I guess Chris already poked them. Can you check anyway, just in csae?
<wgrant> I hope to deploy thiese and some of Chris' further fixes soon.
<cprov> wgrant: good to go, thanks
<wgrant> cprov: Thanks.
<rpadovani> Hey guys, great work with inline comments, thanks :-) Only one thing: in the announcement bar there is a dead link to a blog post, could you create the blog post or remove the link?
<rpadovani> The link is http://blog.launchpad.net/general/diff-comments
<rpadovani> cjwatson, ^^ IIRC, you're a lp dev, maybe you can fix :-)
<cjohnston> rpadovani: its a known issue with a fix to be released no
<rpadovani> ok, thanks :-)
#launchpad-dev 2014-05-30
<bigjools> blog.launchpad.net/general/diff-comments is a 404.  LOL?
<bigjools> ah already noted above
<jtv> The UI is very colourful now...  Still getting used to it, but it looks quite nice.
<wgrant> jtv: The purple links? That's a bug that'll be fixed in a couple of hours...
<jtv> Ah.  Shame.
<jtv> And then there's the new "beta bar" at the top.
<wgrant> Indeed.
<wgrant> That's some UI that was actually designed by a web designer.
<wgrant> So mysteriously it doesn't look like it was designed by some random person in 2005.
<jtv> Looks nice, though doesn't actually seem to fit the style very closely.
<wgrant> Indeed.
#launchpad-dev 2015-05-25
<blr> I liked anatoly's suggestions for the ppa view.
<blr> wgrant: would be very useful to have pack-objects --revs --shallow sha1 which afaict was added in 2.2.1
<blr> cjwatson: any thoughts on packaging a newer release?
<wgrant> blr: What would that buy us?
<blr> wgrant: entirely selfish, in this case it makes it easier to generate packs for tests
<wgrant> Ah, heh.
<wgrant> blr: If you add a commit, pack, then add another commit, and pack again, it should give you two packs normally.
<blr> by 'pack' do you mean gc? because that seems to merge packs
<blr> I do wonder if we should contribute back pack/re-pack to pygit2 (presumably in libgit2, I haven't looked yet)
<cjwatson> blr: Ideally Debian will get round to it soon and then we'll just need to run backportpackage :)
<blr> cjwatson: ok! yes, let's let them worry about it :) (I'm not desperate)
<cjwatson> blr: PPA view> the yellow top bar looked awful to me, but the rest of it seemed mostly reasonable
<blr> oh agree about the yellow bit
<blr> but generally not repeating ourselves and relying on the context of a heading is good
<blr> adding the add-apt-repository command seemed sensible
<wgrant> Yes, the yellow bit is awful, but the rest is fairly obviously good.
#launchpad-dev 2015-05-26
<blr> you would think repo.lookup_reference('non-existent-ref') could return None...
<wgrant> There are valid arguments to both approaches.
<wgrant> But I tend to prefer returning None, indeed.
<blr> wgrant: would you mind having a look at the awful test test_repo_repack_verify_commits_to_pack in https://code.launchpad.net/~blr/turnip/repack-api/+merge/257312
<blr> it passes, but with 'fatal: fgets: Input/output error' which is probably a little less than ideal.
<blr> wgrant: oops nvm, that.
<blr> wgrant: but wouldn't mind a quick check over the branch again before I merge it (including that test).
<wgrant> blr: Hrm, there wasn't a simpler way?
<wgrant> blr: I'd like to see actual assertions on the numbers of packs, since currently you have a loop over what is presumably one pack.
<blr> wgrant: there are two packs (after pack-objects is called), confirmed if you break and have a look in the repo
<blr> I could certainly add an assertion though
<wgrant> blr: Yep, just some assertions to that effect would be nice, in case pygit2 decides to become better.
<wgrant> Don't want it making the test unbreakable.
<blr> it would be nice to have some of this in pygit2
<blr> wgrant: added a test helper prop @packs and the 2 pack assertion which has cleaned up the test a bit if you want to have a quick look again.
<wgrant> blr: The "for filename in factory.packs" bit is still weird, because it only uses the last pack that it loops over.
<wgrant> It overwrites the local on every iteration.
<blr> ah yes, true
<blr> wgrant: added another assertion that there is 1 pack and just used the first element
<blr> that makes more sense :)
<blr> thanks
<wgrant> Great.
<wgrant> The test makes sense now!
<blr> yay for sense
<cjwatson> wgrant: Mind having a quick look over my updates to https://code.launchpad.net/~cjwatson/launchpad/git-mp-move-outside-ref/+merge/260105 ?
<cjwatson> And I'm testfixing.
<wgrant> cjwatson: Amazing.
<wgrant> Oh, of course, it's keyed on source rather than target branch, so it would never have become a totally huge list.
<cjwatson> Yeah.
<cjwatson> Still.
<mgz> hey there, bzr private branches seem to be broken?
<mgz> smart server says TypeError _makeBranchTransport() got an unexpected keyword argument 'private'
<mgz> er... all bzr branches in fact it seems
<mgz> cjwatson: ^if you're still working
<mgz> I filed bug 1458948
<mup> Bug #1458948: bzr smart server operations fail with TypeError <Launchpad itself:New> <https://launchpad.net/bugs/1458948>
<xnox> the world broke.....
<xnox> bzr: ERROR: Server sent an unexpected error: ('error', 'TypeError', "_makeBranchTransport() got an unexpected keyword argument 'private'")
<xnox> and i'm not pushing to private branch....
<Laney> bug 1458948
<mup> Bug #1458948: bzr smart server operations fail with TypeError <Launchpad itself:New> <https://launchpad.net/bugs/1458948>
<xnox> Laney: i bet code needs to be pushed to deploy the fix for broken pushes right.....
<Laney> could be https://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/revision/17526
<Laney> ...which is now 503ing
#launchpad-dev 2015-05-27
<blr> wgrant: do you think it is okay to refer to the project vcs as 'default vcs' in the code? (Understandably we should avoid that in the UI)
<blr> I think it does add some clarity, given we have 'rcs' floating around in a similar context.
<wgrant> blr: Oh yeah, in the code that definitely makes sense.
<wgrant> Non-topological revision logs are the worst.
<blr> wgrant: there's some very willful ignorance of YUI's selectors in this js.
<blr> amusing
<wgrant> blr: Oh?
<blr> wgrant: heh in a few places where Y.one would have been decidedly easier, but perhaps they were forward thinking!
<wgrant> blr: If it makes more sense to fix it, by all means.
#launchpad-dev 2015-05-28
<blr> wgrant: have the setproject form toggling nicely now (and it actually makes sense!)
<blr> err setbranch
<wgrant> Setting a project's project is a nice concept.
<blr> you can never have too much recursion.
<blr> wgrant: oh btw, there was a sensible reason for the avoidance of selectors, they would require the selector-css3 module which we're not using.
<blr> although I can't see any reason not to
<blr> wgrant: have the comment nav working, it slotted in nicely.
<blr> will ping you in the morning when the diff has rendered :P
<wgrant> blr: Yay :)
<blr> wgrant: chance of rendering an lp diff preview today?
<wgrant> blr: New diffs will work fine once the branch scans.
<wgrant> And we'll hopefully have the missing 25TB in place in a few minutes.
<blr> great
#launchpad-dev 2015-05-29
<wgrant> blr: Does it have a diff yet?
<blr> wgrant: ooh, yes.
<blr> I had given up
<wgrant> blr: Why isn't a nav cursor added on comment navigation?
<blr> wgrant: if the comment is in the edit state, it looks rather ugly.
<blr> decided it was better left out, it really makes more sense when you have a specific lineno at any rate I think
<blr> hmm actually the same would apply for a saved comment - given it intrudes into the left hand margin
<wgrant> Ah, hm.
<blr> factory.makePreviewDiff(size='large') and add a few comments perhaps
<blr> see what you think. I think it is probably okay just there for the code navigation.
<blr> wgrant: not related to this branch, but do you think we need a different branch location widget for git repos?
<blr> or should the existing branch location widget support both
<blr> context, linking to existing git repo
<wgrant> blr: I know the widget you're talking about, but I don't quite understand the question.
<blr> wgrant: hmm, I'll try to clarify, in the search modal, should we list both results for bzr and git, or should we handle the bzr/git cases separately with different widgets
<wgrant> blr: They need to be different widgets.
<wgrant> For one thing it'd be an awkward widget to achieve.
<wgrant> For another it'd be quite confusing, and the UI should be separated anyway.
<blr> wgrant: yep, agreed - was just wondering if there were cases where it would be convenient to see both (fairly certain that widget is used elsehwere)
<wgrant> It's unlikely you'd ever want to search both.
<wgrant> It should only be used in one or two places atm.
<wgrant> And they all need to be fixed for the git/bzr split anyway
<blr> yep
<blr> wgrant: hmm weird, working in chrome.
<wgrant> blr: Firefox here, maybe it's special.
<blr> I'll have a look
<blr> wgrant: working for me in FF O.o
<blr> anything in the console?
<wgrant> blr: Ah, it was because the comments were unsaved.
<wgrant> Hmm.
<blr> if an input has focus none of the keybindings work
<wgrant> blr: The comment navigation doesn't consider drafts to be comments, but the top hunk header is considered to be one.
<blr> wgrant: is navigation between drafts useful?
<wgrant> blr: I think it's weird that they appear the same but don't navigate the same.
<blr> wgrant: the problem is the dom node collection isn't refreshed, otherwise the nav code would work for them I think.
<blr> I'll find the callback, should be easy to fix
<wgrant> Ahh
<blr> wgrant: fixed
<wgrant> blr: Yay
<blr> well, still including the first hunk, not sure wtf is going on there.
<wgrant> Yeah, that's an odd one.
<blr> but the drafts are included now
<wgrant> I'd expect the first *file* header.
<blr> will debug
<wgrant> Neither should happen, but I don't see how first hunk could :)
<blr> yeah, very weird.
#launchpad-dev 2016-05-31
<miken> [, 1
#launchpad-dev 2016-06-02
<cjwatson> wgrant: Any suggestions on speeding up the query shown in http://paste.ubuntu.com/16923480/ ?  It's rather essential to the xPN vocabulary -> package cache work.  I've also tried the "... AND binarypackagename.id IN (SELECT distroseriespackagecache.binarypackagename FROM ...)" strategy, which is slower.
<cjwatson> Or perhaps we can tolerate this given that vocabulary lookups aren't generally fast-path?
<cjwatson> It's a *lot* slower than the couple-of-milliseconds lookups in xPN though.
<cjwatson> Hm.  It would be a lot better if the planner did the name scan first.
<cjwatson> Do I just have to do two separate queries to stop it being stupid?
<cjwatson> Hm, even if I split it into separate queries the DSPC part isn't great.
<cjwatson> Aha!  Needs distroseriespackagecache (binarypackagename, archive)
<cjwatson> OK, with that it's much better in cases where there aren't too many BPNs.  If there are lots then there seems to be a tipping point where it goes back to terrible (e.g. "name LIKE '%man%'").
<cjwatson> Looks like it only uses distroseriespackagecache__binarypackagename__archive__idx if there aren't too many possible BPNs.
<cjwatson> I think it will be OK if I can limit the result set.
<wgrant> cjwatson: Two separate queries might be necessary. What's your latest progress on that, so I can optimise it further?
<cjwatson> wgrant: Further optimisation may not be necessary.  http://paste.ubuntu.com/16934843/ with https://code.launchpad.net/~cjwatson/launchpad/package-cache-indexes/+merge/296379
<cjwatson> The trick seems to be pushing the LIMITs down far enough
<cjwatson> As well as the indexes of course
<wgrant> Try with linux and it will probably take eleven years.
<cjwatson> 1440ms hot
<cjwatson> http://paste.ubuntu.com/16934912/
<wgrant> Thirteen years, then.
<cjwatson> Incidentally I wonder why I made GitRepositoryTargetWidget use BinaryAndSourcePackageNameVocabulary rather than SourcePackageNameVocabulary.  Seems silly.
<cjwatson> Not that it really helps since LaunchpadTargetWidget needs the combined one anyway, but I noticed it in passing.
#launchpad-dev 2016-06-03
<wgrant> Hmmm, getting there.
<wgrant> I'm not quite sure why it's choosing the bad plan for the source query.
<wgrant> But I have it reliably down around 10ms hot, and not toooo much worse cold.
<wgrant> Which will probably do.
<wgrant> Weird.
<wgrant> But that's 4-50ms without any particularly weird indices. Possibly nicer than my other attempt.
<cjwatson> Oh, what did I miss?
<wgrant> http://paste.ubuntu.com/16935503/
<wgrant> Main win was deconfusing the planner by pre-materialising main_archive_ids.
<wgrant> But that only got it down to 100ms.
<wgrant> Which isn't good enough for me.
<wgrant> The last query probably wins here. I don't understand exactly why the planner wanted the index that way, since it's only 3x faster, but it works.
<wgrant> cjwatson: Are you copying the definition of what it means to search from the existing vocab?
<wgrant> Because this definition of search is pretty weird.
<cjwatson> Sort of; this was from my rewritten vocabs.
<cjwatson> Can we just use DSPC.name instead of going through xPN at all?
<wgrant> See the third query.
<wgrant> That was my attempt at that.
<wgrant> It works well, but needs an extra extension (installed by default, just not in the DB)
<wgrant> The indexes are all in place on DF if you want to experiment.
<cjwatson> Ah right.  You could do it for DistroSeriesPackageCache.name as well I think.
<wgrant> (btree_gin lets us include an integer in a GIN index, which isn't normally possible)
<wgrant> Yeah, the same sort of thing applies there.
<wgrant> But that seems to be planned sensibly anyway.
<wgrant> Aaand you could not use DSPC at all.
<wgrant> Er, not use DistroSeriesPackageCache at all, that is.
<wgrant> Since we have the binary names in a column on the other table.
<cjwatson> Bit harder to pick out, but true.
<cjwatson> Not going to experiment now, since bedtime.
<wgrant> This query is pretty weird since it prefers all substring-matching source names, then all substring-matching matching binary names.
<wgrant> A good plan.
<wgrant> Night.
<wgrant> (otoh the weirdness makes the query quite fast in the primary archive case, since it can just run straight down SPN(name) and return the first 10 it finds that exist. probably not so fast in a PPA.)
<wgrant> Er, no, that was my hacked query, I guess. the ORDER BY name at the end prevents that.
#launchpad-dev 2016-06-04
<cjwatson> wgrant: Thanks for that analysis.  I think then that the indexes in https://code.launchpad.net/~cjwatson/launchpad/package-cache-indexes/+merge/296379 should be enough, since those are basically the inverted ones you got to in the end.  The bit I missed was probably pulling out the archive ids.  I don't agree that the query is artificial and can be restricted by distro, because users of these vocab
<cjwatson> ularies are basically always ...
<cjwatson> ... selecting both distribution and package at the same time, unfortunately; but we could get hold of all main archives in all distributions, which is still pretty fast.
<cjwatson> wgrant: In fact inverting the binary index too ("CREATE INDEX temp_dspc_really_2 ON distroseriespackagecache (binarypackagename, archive);") makes it comfortably sub-20ms even without pulling out the archive ids.
<cjwatson> Not sure why I wasn't seeing that before but it seems quite consistent now.
<cjwatson> wgrant: So the thing I still need to work out is how to push the LIMITs down through the UNION in the combined vocabulary, which I bet will be tedious Storm fiddling but should be doable.
<cjwatson> wgrant: Could you elaborate on what you meant by the definition of search being weird?
<wgrant> cjwatson: I'd be more comfortable with the archive IDs pulled out anyway, since the stats are totally different for archive 1 and archive every else, so they should affect the planning.
<wgrant> cjwatson: Substring matching then sorting by name isn't the most useful kind of search.
<cjwatson> wgrant: Do we have better patterns?  I'm not very familiar with our search code really.
<wgrant> cjwatson: Most other searches are FTI, which is a bit less silly. But it would, for example, be better here to list an exact match first. And perhaps also prefer $term% and %-$term.
<wgrant> Not preferring exact matches is infuriating for eg. retargeting bugs.
<wgrant> cjwatson: Pushing LIMITs down through the union is somewhat complicated, because the result of the union is ordered. But with appropriate indexes postgres should terminate the search when it has enough.
<wgrant> I hope.
<wgrant> Hm, maybe not.
<wgrant> Difficult for it to realise that the underlying results are ordered the same as the top level.
<wgrant> otoh it's not much of a problem if we don't do substring matching until the usual 3-character threshold.
<wgrant> (but avoiding even exatch matching until that threshold? again, infuriating)
#launchpad-dev 2017-06-02
<mz_> run "cronscripts/process-job-source.py  IBranchScanJobSource", output  "NotBranchError: Not a branch: "lp-internal:///~maozhou-3/ubuntu/trusty/tomboy/test1".  How to fix it?
<cjwatson> Perhaps codehosting.codehosting_endpoint and/or codehosting.internal_branch_by_id_root are misconfigured in launchpad-lazr.conf.
<mz_> codehosting.codehosting_endpoint and internal_branch_by_id_root are configured,but "http://bazaar-internal.launchpad.dev/" can't visit
<cjwatson> There's something for you to investigate then
<mz_> ok,thanks.
#launchpad-dev 2018-05-28
<antenore> Hi all. I think I've misunderstood an old launchpad blog. https://blog.launchpad.net/general/beta-test-asynchronous-ppa-package-copies
<antenore> Is it possible scheduling the copy of binaries between PPAs?
<antenore> in the specific, I need to programmatically copy binaries from ~remmina-ppa-team/+archive/ubuntu/freerdp-daily to https://launchpad.net/~remmina-ppa-team/+archive/ubuntu/remmina-next-daily
<antenore> is there a way to do it automatically at each succesfull builds of freerdp-daily?
<cjwatson> You can do the copy programmatically using launchpadlib, but there's currently no way to be signalled asynchronously when a successful build completes, so you'd have to write something that polls every so often.
<antenore> ok, thanks cjwatson
<cjwatson> (If we ever add such a signalling mechanism, it would probably be a webhook along the lines of https://help.launchpad.net/API/Webhooks - not a bad thing to have but it would take some work)
<antenore> yep, I'm reading https://launchpad.net/+apidoc/beta.html
<antenore> found a good starting point https://github.com/mschlaeffer/tuxonice-ppa-scripts/blob/a8487ba9093fac279892a0c518f24eeb78a09d12/ppa-copy-package :-)
<antenore> cjwatson: cool! Thanks a lot again. Time to sleep in my time zone. have a good day/evening/night
#launchpad-dev 2020-05-26
<cjwatson> Anyone fancy Sphinx documentation for Storm?  https://code.launchpad.net/~cjwatson/storm/doctest-cleanup/+merge/384530 https://code.launchpad.net/~cjwatson/storm/sphinx-doc/+merge/384532 https://code.launchpad.net/~cjwatson/storm/docstring-syntax/+merge/384534
<cjwatson> I have some more stuff pending after that to add more docstrings for various things, but that'll do for now
<tomwardill> pappacena: talking of QA, what were you trying to do that lead to this patch: https://git.launchpad.net/launchpad/commit?id=90eaad5839f6f684cedbab35f3f8030cedc294c1
<tomwardill> something to do with editing OCI projects?
<pappacena> It looks like. Something to do with the new oci_project_admin attribute on distribution?
<pappacena> I don't quite rememeber... I think I couldn't edit OCIProject on dogfood, even though I was in the team that managed recipes for that distribution.
<tomwardill> yeah, makes sense
 * tomwardill tries
<tomwardill> sigh, dogfood, not gofdood
 * tomwardill can't even
 * pappacena hahaha
<tomwardill> okay,I can edit and rename an OCI PRoject
<tomwardill> I think I'll call that good
<tomwardill> and mark recipes as official
<pappacena> ð
<pappacena> cjwatson, could you help me QAing dockerhub push by forcing a build for https://dogfood.paddev.net/~pappacena/ubuntu/+oci/pappacena-oci-project-1/+recipe/test-recipe ? I have just created a pushrule for that.
<cjwatson> pappacena: Can do, can you hit "request builds"?
<pappacena> (actually, created 2 push rules, but the first one was wrong and I do not have permission to delete it...)
<pappacena> sure. I'll do it on the API. The interface shows a strange error saying that I should select a processor, but no processor is shown at the page.
<pappacena> Requested the build: https://api.dogfood.paddev.net/devel/~pappacena/ubuntu/+oci/pappacena-oci-project-1/+recipe/test-recipe/+build-request/47675195
<cjwatson> The recipe might be wrongly configured without any architectures.  Should be fixable on +edit
<pappacena> Right! I forgot about that on edit page
<cjwatson> 2020-05-26 13:43:09 INFO    Running <OCIRecipeRequestBuildsJob for <lp.oci.model.ocirecipe.OCIRecipe object at 0x7f47f0d87a50>> (ID 47675195) in status Waiting
<cjwatson> Ah, and because it had no architectures, it didn't create any builds
<cjwatson> So you'll need to fix that and try again
 * pappacena doing it
 * pappacena done!
<pappacena> https://dogfood.paddev.net/~pappacena/ubuntu/+oci/pappacena-oci-project-1/+recipe/test-recipe/+build/20
<cjwatson> Yep
<cjwatson> Let me know when I need to run the push job
<pappacena> Ok! I'll ping you as soon as the build finishes
<pappacena> cjwatson, build finished
<cjwatson> https://pastebin.canonical.com/p/yxkc4dBpMN/ hmm
<cjwatson> MissingSchema: Invalid URL 'registry.hub.docker.com/v2/': No schema supplied. Perhaps you meant http://registry.hub.docker.com/v2/?
<cjwatson> pappacena: https://pastebin.canonical.com/p/xB8jR8TgMm/ is the traceback
<pappacena> Yes, it was the frist push rule that I've created. But I have created a second one fixing it
<pappacena> It doesn't try the second push rule if the first one fails?
<tomwardill> not atm
<tomwardill> I think
<pappacena> uhm... I don't have a way to fix nor delete the first rule :-(
<pappacena> (using the API)
<pappacena> Could you delete it for me on iharness, cjwatson?
<cjwatson> Oh right
<pappacena> https://api.dogfood.paddev.net/devel/~pappacena/ubuntu/+oci/pappacena-oci-project-1/+recipe/test-recipe/+push-rule/3
<cjwatson> sec
<pappacena> thanks!
<cjwatson> We should also have some URL validation in there I suspect
<cjwatson> There's a gadget somewhere for validating schemes
<cjwatson> I SQLed it out, anyway
<pappacena> I agree. I'll create a MP today to validate that.
<cjwatson> Just fixing the job
<pappacena> Should I ask a build again? Or just re-run the upload script will do?
<cjwatson> I'm doing it
<pappacena> Thanks!
<cjwatson> I just set the job status back to Waiting
<cjwatson> https://pastebin.canonical.com/p/zSPRpW35ZM/   hmm, hanging there
<cjwatson> timed out eventually
<cjwatson> This will probably need use_proxy=True to urlfetch, plus possibly some IS proxy configuration
<pappacena> uhm...  it's possible. We are not using HTTPS for ROCKS?
<cjwatson> Generally speaking these systems don't have direct access to external sites, only via squid
<cjwatson> ROCKs is internal to our network
<cjwatson> (We are using HTTPS; but that's not the relevant distinction, anyway)
<pappacena> Right. Makes sense
<pappacena> I'll open a MP to fix that...
<cjwatson> The proxy configs are in lp:canonical-is-internal-proxy-configs
<pappacena> Thanks
<cjwatson> launchpad_script_servers, launchpad_stg_script_servers, launchpad_dogfood_servers will need access to dockerhub
<cjwatson> And in fact also to whatever the ROCKs endpoints are
<cjwatson> (While that works at the moment, it'd probably otherwise stop working once we switch to using the proxy)
<cjwatson> So probably best to get the proxy config changes landed first
<pappacena> :+1
<pappacena> cjwatson, does that make sense? https://code.launchpad.net/~pappacena/canonical-is-internal-proxy-configs/lp-docker-images-pushing/+merge/384549
<cjwatson> pappacena: Is that the API endpoint for ROCKs?  And I think we'll need upload.image-registry.staging.canonical.com too, for staging
<tomwardill> both urls I have for ROCKS have 'staging' in them
<tomwardill> I'm not actually sure what the endpoint for production would be
<pappacena> I'll add a rule for upload.image-registry.staging.canonical.com. The current acl is for rocks.canonical.com. Maybe that's the production one?
<cjwatson> Could be
<pappacena> Ok, updated the MP. I'll open that for review. Should I ping someone from IS to review/deploy it? What is the process here?
<cjwatson> Ask the help contact in the #is topic as a first step
<pappacena> Great! Thansk!
<cjwatson> They might redirect you to RT depending on how swamped they are
<pappacena> On another topic: does anyone has any idea on how can I simulate a 500 error or similar, to see the OOPS page? In my machine it was quite easy to force a `raise Exception` somewhere, but it's not that easy on dogfood... :)
<cjwatson> A git ref page should do I think due to the fairly broken way those are set up on dogfood
<cjwatson> https://code.dogfood.paddev.net/~cjwatson/turnip/+git/turnip/+ref/master
<pappacena> Cool! Thanks!
<pappacena> cjwatson or tomwardill , I'll have lunch now, but if you find some minutes to review it: https://code.launchpad.net/~pappacena/launchpad/+git/launchpad/+merge/384557
<pappacena> It's the MP to use proxy when uploading images to ROCKS and DockerHub.
<cjwatson> r=me
<cjwatson> pappacena: I misclicked and marked your OOPS links commit green on deployable, but I wasn't sure what its previous state was.  Feel free to change that if my change was wrong
<cjwatson> pappacena: For your sync-signingkeys change, I'm just going to run it again on dogfood and mark that green if it doesn't crash, since the new test should deal with the specific case at hand.  Sound OK?
<cjwatson> But I think I'm done for the day; will pick it up tomorrow
<pappacena> Ok for both messages, cjwatson.
#launchpad-dev 2020-05-27
<tomwardill> sigh, whoopsied and `git clean`ed my wheels directory
 * tomwardill waits for everything to rebuild
<cjwatson> pappacena: I updated dogfood and reran your push job.  https://pastebin.canonical.com/p/8tW3jn8fBz/ - looks good now I think?
<tomwardill> "Found node-sass binary"
<tomwardill> proceeds to rebuild it
<tomwardill> gaaaah
<cjwatson> I remember that taking some work to get right on SCA but don't quite remember the details
<tomwardill> I think I've just run into the same problem as I did there
<tomwardill> it needs to be an absolute path and only works in the config file
<tomwardill> might have to copy it to somewhere like /tmp/ and then reference it
<pappacena> cjwatson: it looks good now! :D
<pappacena> https://hub.docker.com/repository/docker/pappacena/foo-image
<pappacena> `docker run -it --rm pappacena/foo-image:edge bash` seems to be fine too :)
<cjwatson> Excellent.  Shall we mark green and deploy then?
<cjwatson> pappacena: ^- above question was to you BTW :)
<pappacena> Ah, sorry. I thought I have marked. Just marked it now
<cjwatson> pappacena: OK - can you work with Tom to sort out a deployment then?  I updated the docs this morning so you should only have a single place to look
<pappacena> Yes, I'm taking a look at the document right now. Maybe we can do it together after the standup, tomwardill ?
<tomwardill> sure!
<tomwardill> oookay, closer
<tomwardill> now just to work out why my Make isn't working
<tomwardill> cjwatson: any idea why `NODE_ABI = $(call lazy_eval,NODE_ABI,nodejs -p process.versions.modules)` is returning empty if I put it in a LP Makefile?
<cjwatson> tomwardill: What's lazy_eval defined as?
<tomwardill> aha!
<tomwardill> yeah....
<cjwatson> I thought that might be it :-)
<tomwardill> defining it
<tomwardill> that's a thing
 * tomwardill might try that
<tomwardill> oh, look, a value!
<tomwardill> <_<
<tomwardill> >_>
<SpecialK|Canon> :D
<cjwatson> Can anyone review https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/384556 for me?
<tomwardill> node-sass installation: https://code.launchpad.net/~twom/launchpad/+git/launchpad/+merge/384644 and dependencies: https://code.launchpad.net/~twom/launchpad/+git/launchpad/+merge/384614
<cjwatson> Looks very familiar, I suspect the lazy_eval stuff was originally me :)
<tomwardill> it was :)
<tomwardill> all of that preamble was yours
<tomwardill> I forgot that you made it actually none-hardcoded after I just plonked a bunch of version numbers in there
<cjwatson> Dependencies URL should be https://code.launchpad.net/~twom/lp-source-dependencies/+git/lp-source-dependencies/+merge/384643
<cjwatson> right?
<tomwardill> main difference is that yarn uses the environment variable, where npm uses a cli switch
<tomwardill> cjwatson: oh yes, copy paste fail
<cjwatson> What a lot of dependencies, but that's node for you I guess
<cjwatson> r=me
<tomwardill> I haven't attempted to prune them or antyhing, it's just the result of `yarn add node-sass`
<tomwardill> from the SCSS docs: "Prefer @use over @import as it is deprecated"
<tomwardill> what they don't say: "only the dart-sass implementation actually has a working @use"
<SpecialK|Canon> rargh
<tomwardill> https://github.com/facebook/create-react-app/issues/8316
 * tomwardill just uses @import
<tomwardill> oh hey, it works now!
<tomwardill> 17k of CSS, lovely
<tomwardill> yet the old one is 86k
<tomwardill> hmm, what have I missed
<tomwardill> 130k now
<tomwardill> that's... more believable
<tomwardill> 92k in compressed mode, that's probably close enough
 * tomwardill -> EOD
<cjwatson> tomwardill: diff -w I guess
<cjwatson> Well, -bu maybe
#launchpad-dev 2020-05-28
<tomwardill> use node-sass for (nearly) no-op CSS compilation: https://code.launchpad.net/~twom/launchpad/+git/launchpad/+merge/384706
<tomwardill> the output file isn't exactly the same as the current one, but it contains all the same rules afaics (and the site seems to work...)
<tomwardill> the watch command has some slightly strange output due to mishandling of subdirectories and imports, but it works and I think just needs some layout tweaks that I'll hopefully get to next
<cjwatson> tomwardill: Can we remove lib/lp/scripts/utilities/js/combo.py and lib/lp/scripts/utilities/js/tests/test_combo.py as well?
<cjwatson> I guess all of lib/lp/scripts/utilities/js/tests/ in fact
<tomwardill> cjwatson: I think that's still used for the JS files
<cjwatson> Are you sure?  It was at one point, but grep doesn't find any other references
<cjwatson> Unless I'm misreading
<cjwatson> Worth a bit of hunting around to see what else we can remove, anyway
<tomwardill> I'll have another poke at it
 * tomwardill randomly deletes things to see what breaks
<lifeless> somewhat randomly, is LP using pybars? There are forks of it adding python3 and newer handlebars.js features
<lifeless> I don't know if anyone reached out to the LP team about it, but it might be an idea to hand over the pypi project to one of the forks if not
 * lifeless hands over whatever blessing might possibly be needed for that
<cjwatson> lifeless: no
<cjwatson> at least I don't think so
<cjwatson> we use pystache in the buglisting tests
<cjwatson> and a copy of mustache.js I think
<tomwardill> cjwatson: removed those files: https://code.launchpad.net/~twom/launchpad/+git/launchpad/+merge/384706 dont' seem to be able to remove any further without a re-engineering of the jsbuild stuff
<tomwardill> I'd like CSSComboFile to go away, but stuff breaks if I try
<cjwatson> Should hopefully be possible to redo Builder.build_skin etc. using scss, but doesn't have to be in this MP
<tomwardill> yeah, that's where I'd like to get it eventually
<teward> cjwatson: just as an FYI for the bug i filed about code review DMARC compliance, I'm still working on it, but some more major IT Security related things came up @ employers so i had to address that stuff heh
<teward> in case anyone was asking :)
<teward> (i'm still working on that branch heh)
<cjwatson> Nobody has been asking, but thanks :)
<cjwatson> No immediate rush, whenever you have time
<teward> yeh the status change on the bug is what caught my attention :)
<teward> (since i filed it xD)
<teward> cjwatson: i'm not subscribed to the bugs and questions list, has there been any flak from the Community At Large with the email change yet?  Or edge cases and bugs and stuff?
<cjwatson> teward: not a word
<teward> lucky us.
<teward> guess the vast majority of people don't care enough to keep an eye on it as long as it works heh
<cjwatson> Oh, one blog comment
<cjwatson> Basically asking for name mangling
<teward> name mangling how?  (Comments are invisible heh)
<cjwatson> Hm?  No they're not, https://blog.launchpad.net/notifications/bug-emails-now-use-the-bugs-address-in-the-from-header
<teward> they are if you ain't logged in
<cjwatson> I'm not logged in
 * teward just logged in with SSO and can see it now
<cjwatson> I checked in w3m
<teward> hmm
<teward> wonder if my system is being weird.
<teward> *goes to check it*
<cjwatson> shift-reload
<cjwatson> Same sort of thing we discussed as maybe a possibility, anyway.  I don't think it's immediately urgent but we should probably have a bug report for it
<teward> i mean
<teward> that's pretty fixable
<cjwatson> And TBH I'd rather avoid name mangling if we can
<teward> but that would require the level of complexity we had before
<teward> with special case exclusions, etc.
<cjwatson> The fact that GitHub doesn't do it means we're not alone
<teward> yeah most places dno't
<teward> don't*
<teward> those that DO have mangling in line do that as a design decision in their systems / lists
<cjwatson> Anyway, we don't need to discuss it in depth here IMO
<teward> yep
<teward> *goes to consume breakfast*
<tomwardill> huh, css to scss converters exist
<tomwardill> how badly can that go?
<SpecialK|Canon> <cthulhu/>
<tomwardill> weirdly, it's... not that bad?
<pappacena> tomwardill: I don't know... maybe it would change randomly the color of dropdowns? ð
<tomwardill> ah, from 'grey' to 'slightly not that grey'?
 * pappacena hahaha!
<SpecialK|Canon> ;P
<tomwardill> I dropped 'base.css' into https://css2sass.herokuapp.com/
<tomwardill> and the result is actually readable
<tomwardill> (at least to me, as someone who wouldn't know good scss if it wrote me a letter)
<tomwardill> hmm, actually going to go with that
<SpecialK|Canon> One day I'll ship a tool like that and have it embed zalgo in 0.01% of responses or sth, see who notices ;)
 * tomwardill pays careful attention to any SpecialK|Canon MP
<SpecialK|Canon> lol
<tomwardill> wow, we have some fun about overriding link colours
<tomwardill> "ImportError: No module named PIL"
<tomwardill> ...not an error I was expecting in this work, I must admit
<cjwatson> That's linked using link-system-packages.py
<cjwatson> Have you done 'make compile' in that tree recently?
<tomwardill> yeah, think I messed up my env
<tomwardill> make clean fixed it
<tomwardill> found a bug, the combo code rewrites relative urls, so removing it... doesn't
<tomwardill> so css declared icons now 404
#launchpad-dev 2020-05-29
<tomwardill> wgrant: do you know much about our CSS build pipeline and why it's structured the way it is?
<tomwardill> something something 'components are odd'
<wgrant> tomwardill: I don't. The most recent serious contact I had with our CSS was before I was with the company, laughing at the silly idea of redesigning the whole thing in one month.
<tomwardill> ah, right
<wgrant> ... they did it and you see the result.
<wgrant> Colin might have touched it a bit more recently.
<wgrant> But I haven't made non-trivial changes in a decade.
<tomwardill> we appear to rewrite all the URLs in the CSS, which means the plan to just do a straight node-sass concatentation doesn't work.
<tomwardill> I was wondering if something along the lines of a collectstatic approach might be better
<wgrant> Is the rewriting thing just for absolute icing paths, or something else?
<tomwardill> the former I think
<tomwardill> at the point of concatenation, we grab all the CSS files from components and yui, then rewrite any urls in the CSS (for icons, etc) to point at the right place
<tomwardill> then do the concat
<tomwardill> but there's also something to do with revision numbers that I don't quite understand yet
<SpecialK|Canon> pure guess but prepending revision number to output paths so users get stable UI experience?
<wgrant> Right, outside dev we always prefix media with a path containing revinfo and Expires: 5eva
<wgrant> (there should be an RFC for that)
<tomwardill> yeah, that makes sense
<tomwardill> doesn't always seem to apply though
<tomwardill> I may well be reading the access logs wrong however
<wgrant> Some HTML or JS likely uses non-revved paths because it's lazy
<tomwardill> before swearing a lot and trying to work out why your fixed code is still generation 404s, ensure that the file actually exists on disk and that you haven't rm'd it and not regerated it
<tomwardill> I'M LOOKING AT YOU, bg_steps-estatus.gif'
<ilasc> :)
<ilasc> nice one
<tomwardill> can someone re-review https://code.launchpad.net/~twom/launchpad/+git/launchpad/+merge/384706 given the last commit is a change in behaviour and is new since ilasc looked at it last.
 * pappacena checking
<tomwardill> pappacena: it might be worth starting the branch (via checkout, make clean, make run) and clicking some pages
<tomwardill> and checking the logs for 404s
 * pappacena ð
<tomwardill> entirely possible it's still broken and I've just missed some
<pappacena> it depends on this one, right? https://code.launchpad.net/~twom/lp-source-dependencies/+git/lp-source-dependencies/+merge/384643
<tomwardill> oh, ydx
<tomwardill> *yes
 * pappacena pulling it!
<tomwardill> pappacena: you may need to apt upgrade nodejs too
<pappacena> Should it work with `$ node --version` == v8.10.0 ?
<pappacena> because apparently it worked oO
<tomwardill> ah yes, that is the updated version
<tomwardill> (origianl is v4)
<tomwardill> SpecialK|Canon: I have no idea what longopts is
<tomwardill> otherwise, maybe?
<SpecialK|Canon> tomwardill: ah sorry! --help vs -h
<tomwardill> ah, fair
<SpecialK|Canon> (I had to look up `ln -n`)
<tomwardill> oh, I wasn't going to change that one (it's copy+paste fromt he block above it)
<SpecialK|Canon> fair enough!
<tomwardill> there's perhaps a job in going through the Makefile for that though
<SpecialK|Canon> Sure, no biggie, I just like/encourage the general convention
<tomwardill> I've changed the one on the node-sass line to `--recursive` over `-r` though
<tomwardill> well, this is a fun diff, but transcription to SCSS: https://code.launchpad.net/~twom/launchpad/+git/launchpad/+merge/384834
