#launchpad-dev 2010-05-03
 * thumper tries to remember how to programmatically subscribe to an event
<thumper> mwhudson: I don't suppose you remember where jml had the scanner subscriber stuff before it was moved to zcml?
<thumper> wgrant: ping
<mwhudson> thumper: lp.codehosting.scanner.fixture maybe?
<thumper> mwhudson: yeah, looking at that now
<wgrant> thumper: Hi.
<thumper> wgrant: when do you finish uni?
<wgrant> thumper: Novemberish.
<lifeless> wgrant: oh, I thought it was earlier ?
<lifeless> wgrant: are you full time at uni ?
<wgrant> lifeless: I am :/
<thumper> and some would think fill time on soyuz :)
<wgrant> Heh. Not quite.
<thumper> WWWOOOOOOOOOOOOOOOOOOOOOOOOOO!!!!!!!!!!!!!!!1
<thumper> OMG
<thumper> something worked first time
<lifeless> bbq puppies?
<ajmitch> thumper: you sound surprised about that
<thumper> I am
<thumper> especially since it is hooking into the zope event infrastructure
 * thumper goes for a walk to get Maia
<thumper> :(
<thumper> ââ¹
<thumper> damn tests
<lifeless> its a little on that clicking on a bug in the candidate-dups list *isn't a link to the bug'
<wgrant> But it's green!!!!!
<lifeless> s/on/odd/
<thumper> if up in pdb takes you up the stackframe
<thumper> what takes you down?
 * thumper hangs head
<lifeless> thumper: down down down down
 * mwhudson off
<jtv> jamesh: would you happen to know how I run lazr.batchnavigator's doctest?
<jamesh> jtv: I'm not sure how the tests for that module would be hooked up, but assuming the test runner lets you select a subset of tests, can you use the test file name?
<jtv> jamesh: oh, using the lp test runner?
<jamesh> jtv: yeah.  The DocFileSuite() helper adds tests that use the filename as the test name
 * jtv tries
<jamesh> so assuming the doctest has been hooked into the test suite you're running, that should work
<jtv> setting up lp branch...
<jamesh> presumably you can run just the lazr.batchnavigator tests somehow though, if the module has been split out?
<jtv> jamesh: yes, presumably... I just don't know how.  Don't even know how to use setup.py.  No dice on the lp test runner.  :(
<jtv> Even tried ./bin/test -vv -t README but that only runs the Launchpad ones.
<jamesh> jtv: it is quite possible that the LP test suite doesn't run the test suites of all its dependencies
<jamesh> after all, changes to LP shouldn't break its dependencies: it is the reverse you need to worry about
<lifeless> .win 14
<mtaylor> heyho
<lifeless> fun in the sun - http://launchpadlibrarian.net/47476014/fail.png (https://bugs.edge.launchpad.net/malone/+bug/574233)
<lifeless> mtaylor: this is where lp and lplib developers hang these days
<mtaylor> neat
 * mtaylor adds to list of channels
<mup> Bug #574233: clicking next with empty summary in bug filing wizard shows summary field twice <Launchpad Bugs:New> <https://launchpad.net/bugs/574233>
<mtaylor> anybody around got a good idea on how to use launchpadlib from jython in a maven project? (or have a jar already done up?) OR, have any ideas on using launchpad API from java
<mtaylor> or have already done my job for me?
<mtaylor> thumper: where you been hiding?
<thumper> mtaylor: hi
<thumper> mtaylor: release tomorrow and a fix of mine isn't working
<thumper> mtaylor: I'm debugging
<mtaylor> thumper: ah. fair enough
<thumper> mtaylor: apart from that, working on wikkid
 * mtaylor was just talking about that to someone today
<mtaylor> the masses are all looking forward to it
<lifeless> thumper: I will join the list eventually, I swear
<thumper> mtaylor: really?
<thumper> :-( the unregistering of fixtures isn't working
 * ajmitch sort of dislikes having to join the team to subscribe to the mailing list :)
<thumper> or at least the remove handler isn't
<mtaylor> thumper: yup. we were talking about what a great idea it was, and how much we would enjoy such a thing
<thumper> mtaylor: :)
<thumper> BjornT: did you see my change to the branch that records the oopses?
 * thumper primarl screams
<thumper> primal
<thumper> whatever
<thumper> fuck it
 * mtaylor imagines it's more like a primarl scream
<thumper> I found my bug
<thumper> sometimes I hate the python object model for inheritance and shitty destructors (or lack of them)
 * thumper goes to brush the kids teeth while ec2 does its thing
<thumper> mtaylor: I'm hoping to have a wikkid release within the next month
<thumper> fingers crossed
<BjornT> thumper: no, i didn't. i'll look at them now
<thumper> BjornT: I got mwhudson to take a look at the changes
<thumper> BjornT: and he is pretty happy with them
<thumper> BjornT: it adds the ability of the base LP TestCase to record the oopses that occur during a test run
<thumper> BjornT: by adding an ObjectEvent for new oopses, and listenting to it in setUp and removing it in tearDown
<thumper> BjornT: as a drive by I fixed a spurious test failure that was failing for the exact reason of testing that an oops was generated
<thumper> BjornT: spurious in that it failed if that particular test ran over midnight :)
<thumper> BjornT: I also added the tests you were after about testing that the errors were recorded properly in the different situations
<BjornT> thumper: those changes look good. firing off an event is a good idea, and the tests look good.
<thumper> BjornT: just running everything through ec2 again as I came across a test that had multiple inheritance, set up both base classes setUp methods (they were both test case bases) but failed to declare a tearDown
<thumper> BjornT: yes it was a code tests
<thumper> BjornT: I've added the required tearDown and re-running the tests
<thumper> BjornT: if all is good, I'll land it in about 3 horus
<thumper> hours
<BjornT> thumper: ok
<thumper> lifeless: ping
<thumper> jml: ping
<thumper> someone who understands testtools: ping
 * thumper goes to clean-up, checking back in shortly
 * thumper just remembered it is a public holiday in the UK - jml please check IRC anyway!
<BjornT> danilos: what's the status of the remaining Translations QA items?
<danilos> BjornT, (let me check, I am off today so can't spend much time on it)
<thumper> super(TestCase, self).setUp()
<thumper> fucking super killing my tests!!!
<BjornT> danilos: thanks. maybe you can delegate to someone?
<danilos> BjornT, it seems we got a few landings by Adi
<danilos> BjornT, henninge is the only translations dev around and he is busy with half of the QA :) some items are simply QAed already (on dogfood) even before they landed
<henninge> danilos: I was going to go through the items anyway
<danilos> henninge, thanks
<henninge> danilos: there is only one item that is not related to build-from-branch.
<danilos> BjornT, henninge: so, most of them seem QAed anyway (buildfarm related work), and there is one that isn't (bug 568891) and one that is not 'fix committed' (bug 531768)
<mup> Bug #568891: Support "weird" language codes for KDE translations <import-queue> <qa-needstesting> <Launchpad Translations:Fix Committed by danilo> <https://launchpad.net/bugs/568891>
<mup> Bug #531768: Import queue status popup displaces page content to the left on Firefox <css> <import-queue> <qa-needstesting> <ui> <LAZR Javascript Library:Fix Committed by intellectronica> <Launchpad Translations:In Progress by intellectronica> <https://launchpad.net/bugs/531768>
<danilos> henninge, right
<henninge> oh, that one ;)
<danilos> henninge, if you've got time to QA 568891 you'd have to do it on dogfood as well :) if not, I'll QA it post-release tomorrow; basically, uploading a kde-l10n-ca-valencia package and one regular kde-l10n-de package should do and distribute their translations across different KDE source packages :)
<danilos> henninge, trivial, isn't it :)
<henninge> I'll give it a try
<danilos> henninge, I'm letting you mark all the template generation bugs as qa-ok
<danilos> :)
<thumper> jml, lifeless: unping, got it sorted
<henninge> thanks ;P
<danilos> cheers, I am really gone now (will only be back later to write up a blog post about generation of templates :)
<henninge> good stuff
<henninge> BjornT: bug 531768 is fixed in lazr-js and would need an lazr-js upgrade in LP to be fixed for us.
<mup> Bug #531768: Import queue status popup displaces page content to the left on Firefox <css> <import-queue> <qa-needstesting> <ui> <LAZR Javascript Library:Fix Committed by intellectronica> <Launchpad Translations:In Progress by intellectronica> <https://launchpad.net/bugs/531768>
<henninge> BjornT: but I am not sure how if that will happen anyway or if we can do anything. It's not target to this release, anyway .... ;)
<BjornT> henninge: right. upgrading lazr-js is a bit too risky to do now.
<deryck> Morning, all.
<poolie> deryck: hi
<lifeless> thumper: hi
<thumper> lifeless: all sorted now
<lifeless> thumper: kk
<BjornT> noodles775: how's the soyuz QA going? you still have a few needstesting items
<noodles775> BjornT: I've updated the two assigned to me. I'm pretty sure StevenK's two items are now QA'd, but I'll let him updated them.
<BjornT> noodles775: well, isn't StevenK in australia?
<noodles775> BjornT: correct. I helped him QA the fix for the first one this morning.
 * noodles775 checks the kanban
<BjornT> noodles775: so it would be good if you could get them updated, and qa them if ncessary. doing it tomorrow is too late.
<noodles775> The second bug hasn't been QA'd it seems. One of us will follow it up.
<sinzui> BjornT, The registry has two fixes that we can have in review in the next hour.
<sinzui> We want RCs for them
<BjornT> sinzui: what kind of fixes?
<sinzui> BjornT, bug 574142 <-- cause by Edwin's RC landing on Friday
<mup> Bug #574142: Could not adapt ProductSeries to IBranchTarget <oops> <Launchpad Registry:Triaged by sinzui> <https://launchpad.net/bugs/574142>
<sinzui> BjornT, bug 574431 <- The feature is ready for release
<mup> Bug #574431: Remove lpnet guard from project package suggestions <Launchpad Registry:In Progress by sinzui> <https://launchpad.net/bugs/574431>
<BjornT> sinzui: ok, that sounds like reasonably RC candidates. you might have to ask flacoste for RCs, though, since i might be unavailable.
<sinzui> BjornT, okay. thanks for your advice.
<sinzui> Is the branch scanner knackered? My branch has been waiting for 45 minutes
<sinzui> oh, I see launchpad-reviews explains the issue
<shilbert> Hi everyone, are there any problems known involving the automatic import of translations from bzw into launchpad ?
<adiroiban> shilbert: nope
<shilbert>  it seems the translations are not updated in launchpad although the imported files have updates
<adiroiban> I have imported the translations of an bzr yesterdayt
<adiroiban> and everthing was smooth
<adiroiban> shilbert: can you give me the link to the project?
<shilbert> yes, just a second
<shilbert> https://translations.launchpad.net/gnumed/trunk/+pots/gnumed
<deryck> Is inlinehelp.js the only thing using MochiKit?  anyone know for sure?
<deryck> sinzui, perhaps you know? ^^
<sinzui> deryck, I believe there are several show/hide/expand/collapse methods that use MochiKit in comments. There is a translations dependency too
<deryck> sinzui, ah, ok.  thanks.  debating whether or not I want to fix inlinehelp to use yui or just do my 3 line fix.
<sinzui> deryck, bug 294656 sums of the deps
<mup> Bug #294656: Every page requests two JavaScript libraries (remove MochiKit) <tech-debt> <Launchpad Foundations:Triaged> <Launchpad Bugs:Triaged> <Launchpad Translations:Triaged> <https://launchpad.net/bugs/294656>
<deryck> nice.  thanks!
<sinzui> deryck, I suspect you want to do the 3 line fix unless you can dedicate 2 days for YUI
<deryck> sinzui, well, I was thinking one day flying.  But you're probably right that's it's two days.
<deryck> have to sleep some time :-)
#launchpad-dev 2010-05-04
<bryceh> I'm getting an error about loom.branch when trying to run 'make schema': http://pastebin.com/d8i40sE1
<bryceh> any ideas?  I tried installing bzr-loom but that made no difference.  Am I missing something else?
<mwhudson> bryceh: what's in sourcecode/ ?
<mwhudson> launchpad doesn't use the system plugins for bzr
<bryceh> from bzrlib.plugins.loom.branch import (
<bryceh>     BzrBranchLoomFormat1, BzrBranchLoomFormat6)
<bryceh> from lp-branches/devel/lib/lp/code/bzr.py
<mwhudson> sigh
<mwhudson> bryceh: there's a directory called sourcecode in the launchpad tree
<mwhudson> bryceh: what's in there?
<bryceh> bryce@cardport:~/launchpad/lp-branches/devel$ ls sourcecode/
<bryceh> bzr-git  dulwich               mailman   pygpgme  subvertpy
<bryceh> cscvs    launchpad-loggerhead  Makefile  shipit   testresources
<mwhudson> that's an odd mix
<mwhudson> bryceh: run ./utilities/update-sourcecode
<bryceh> mwhudson, ok will give that a shot
<bryceh> hmm, that gives this error:  http://pastebin.com/nraERpsX
 * bryceh tries deleting the .bzr dir
<wgrant> You probably have to delete the entire shipit directory.
<bryceh> wgrant, thanks, seems to be proceeding now
 * thumper heads to lunch
<thumper> bryceh: welcome
<lifeless> can I get someone to land a patch for me ?
<lifeless> https://code.edge.launchpad.net/~lifeless/launchpad/merge/+merge/23523
<StevenK> lifeless: PQM is in RC
<lifeless> oh
<lifeless> what does that mean
<StevenK> lifeless: release-critical fixes only
<spm> lifeless: fwiw, unless it's like ZOMG STOP THE PRESS urgent; hasn't got a hope of getting in todays release.
<lifeless> spm: oh clearly
<lifeless> just keeping the inventory low
<spm> heh
<wgrant> lifeless: Option 1: violate separation of concerns in an already hackish script. Option 2: respect separation of concerns slightly more, and make your service look even less reliable and attractive than its already bad reputation.
<lifeless> wgrant: doing the revisions at the same time would make the script take much much longer to run
<lifeless> doing the scan staggered, so that new work happens fast but the new branches do get scanned, would allow the branches to be available more quickly.
<adeuring> good morning
<mrevell> Goodly morning
<allenap> Good morning mrevell.
<mrevell> yo allenap
<allenap> mrevell: Today I am lanscallenape
<mrevell> allenap, Oh! Man. Congrats :)
* henninge changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.04 | PQM is closed | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes | staging is down!
* henninge changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.04 | PQM is closed | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
<leonardr> i need to have a conversation with someone who knows a lot about how launchpad uses apache. i don't want to bother a losa about this since it's development work, but if a losa is free i'd love to talk
<leonardr> jml, can you recommend someone?
<jml> leonardr: flacoste or BjornT would be my two guesses
<mwhudson> so...
<mwhudson> code hosting hasn't imploded?
<wgrant> Does this mean we're running hostedless?
<jml> umm
<jml> it probably hasn't imploded, but there have been some weird email behaviours: https://answers.edge.launchpad.net/launchpad-code/+question/109575
<jelmer> jml: fwiw I received that email as well
<james_w> jam apparently just proposed a moloolaba branch when I assume he is in bed, so presumably not an isolated incident
<wgrant> I haven't got any.
<james_w> https://code.edge.launchpad.net/~jameinel/bzr-fastimport/mooloolaba/+merge/14894
<jelmer> jml: I've seen jam's fastimport proposal as well, but IIRC those are the only two.
<james_w> Date: Tue, 04 May 2010 10:32:28 -0000
<mwhudson> there's a new job script to send merge proposal email
<mwhudson> maybe there were some odd old jobs in the db for some reason
<mwhudson> jml: i notice https://code.edge.launchpad.net/~jml/zope.testing/subunit-error has a strange error message
 * mwhudson goes to bed
<jml> mwhudson: yeah, that is weird. sleep well.
<james_w> jml: it seems that special permissions for source package branches have broken, but the code looks fine to be in lp:launchpad/devel at least
<james_w> is there somewhere else I should be checking?
<james_w> at least BranchMergeProposalEdit is no longer granted to uploaders, but that could be because they have lost EditBranch
<jml> james_w: hmm.
<jml> james_w: I'm dead certain that I added a unit test to check that uploaders have edit on official package branches.
<jml> james_w: so, I guess I'd look for that test and make sure it was still valid
<james_w> I'm dead certain I added a similar unit test to check that uploaders have edit on merge proposals too
<jml> james_w: I don't know which branch most accurately represents the code that's on Launchpad.
<jml> james_w: I'd recommend finding someone who is supposed to have upload permissions and getting them to try changing a branch's status and try pushing up to a branch.
<gary_poster> mrevell: hi. :-)  did you notice during rollout if we can mark https://bugs.edge.launchpad.net/launchpad-foundations/+bug/553171 qa-ok?
<mup> Bug #553171: "LP is read-only" and similar messages still point to old maintenance page <qa-needstesting> <trivial> <ui> <Launchpad Foundations:Fix Released by matthew.revell> <https://launchpad.net/bugs/553171>
<mrevell> gary_poster, Looked good to me :)
<james_w> jml: they have edit permissions, and the code looks good in all the lp:launchpad branches
<mrevell> gary_poster, updated
<gary_poster> thanks mrevell :-)
<jml> james_w: so it's just merge proposals?
<james_w> I think so
<james_w> I'm going to check further
<jtv> gary_poster: hi!  How do I run the doctest for lazr.batchnavigator?  I'd like to verify my fix for that oops I reported.
<gary_poster> jtv: great, thank you! I hope ``bin/test`` will do the trick.
<gary_poster> It is supposed to
<jtv> gary_poster: I didn't see a bin in that branch, and the LP test runner doesn't seem to exercise it either.
<gary_poster> jtv, acknowledging that this sucks and that we have a slow-burning process to make it better, did you see https://dev.launchpad.net/HackingLazrLibraries ?
<jtv> gary_poster: just found it
<gary_poster> great
<jtv> running bootstrap... looks like that'll take a while
<gary_poster> jtv, next time se the "Global cache" instructions
<gary_poster> that makes subsequent efforts a lot faster
<jtv> gary_poster: ok, thanks!
<james_w> jml: not a regression, it's just that the check says that they can't upload to lucid now, but we have no way to propose the creation of a lucid-proposed branch with the changes. This will require some discussion.
<leonardr> flacoste, bjornt, when you get a minute i'd appreciate your thoughts on bug 574697
<jml> james_w, gnarr
<mup> Bug #574697: Launchpad strips incoming TE header <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/574697>
<jml> james_w, you right to arrange that discussion?
<james_w> yeah
<jml> ta
<jtv> gary_poster: all done & ready for review...  note: the hacking page says to merge your branches directly into trunk, but doesn't mention pushing.
<jtv> Oh, that's further down.
<jtv> No, it's not.
<flacoste> leonardr: didn't we established that it was apache that removed those headers
<flacoste> ?
<flacoste> leonardr: if i recall, we had put a work-around in place for that behaviour?
<leonardr> flacoste: i don't remember encountering this before
<leonardr> we have a workaround for apache's treatment of Content-Encoding, but apache's behavior stops the workaround from ever happening
<gary_poster> jtv: hacking page: it means bzr co the trunk and then merge and commit
<gary_poster> jtv: forward me the review email when you get a chance
<flacoste> leonardr: i'd suggest sending an email cc launchpad-dev to our local resident proxy expert: lifeless
<leonardr> ok
<mars> henninge, ping, looking for a CHR swap?
<henninge> mars: Hi. Yes, I am.
<henninge> mars: What can you offer? ;)
<mars> henninge, how about Monday May 24th?  That's a National Holiday here in Canada.
<mars> henninge, and I will be looking to swap it
<henninge> mars: although it's a Monday, it should be fine ... ;)
<mars> yeah, sorry about that :)
<mars> henninge, great, I'll tell the list
<henninge> last month was  a Monday too, but it wasn't too bad.
<henninge> mars: yes, please do.
<rockstar> abentley, should we allow recipes to be based of junk branches?
<abentley> rockstar, I have no strong opinion.
<abentley> rockstar, I guess so?
<rockstar> abentley, I was thinking we shouldn't.  If they want to do that, they should create a project.
<jml> rockstar, personally, I'd allow it.
<rockstar> jml, rationale?
<abentley> rockstar, or create the sourcepackage?
<jml> rockstar, there's no compelling reason to forbid them that I know of. it requires work on our part to add the restriction. it requires work on their part to work around the restriction
<maxb> It may be useful for people to experiment with the functionality in an obviously personally-sandboxed way
<rockstar> jml, why not allow merge proposals on junk branches then?
<jml> rockstar, partly because it makes the UI harder for us and harder for the user
<jml> which branch do you merge into by default? how do you see the list of active reviews?
<maxb> I think MPs ought to be allowed on +junk branches, if only to keep consistency, and allow people to play around with the feature when learning
<maxb> default branch: none, they have to specify. list of reviews: only for a specific branch
<jml> maxb, regarding junk branches, what I want before absolutely anything else is a dead simple way to turn them into the trunk branch of a new project
<maxb> That would be nice. Bring back the ability to rename branches across projects
<jml> in general, the reason why we make junk branches less awesome is to encourage people to use projects as the basis for collaboration
<jml> (so bugs can be filed, translations done, related branches found etc)
<rockstar> jml, so recipes would be an exception to that rule?
<jml> rockstar, they'd buck the trend, certainly. I mean, you could also argue that "capable of being owned by teams" breaks the rule.
<rockstar> jml, yes, I think it does to some extent.
<mrevell> Night
<leonardr> james_w, your wadllib code is present in the new version of wadllib
<james_w> leonardr: great, thanks
<kfogel> sinzui: if possible bug in Answers, should question assignee be ~launchpad-registry?  This is re https://answers.edge.launchpad.net/launchpad/+question/109538/  (that question is not an example of the bug, but it describes the bug's manifestation in 109518, which is a confusingly similar number)
<sinzui> kfogel, I do not have an immediate answer. retarget it to launchpad-answers (not registry). One of us will get to it
<kfogel> sinzui: no such team 'launchpad-answers' (that was my first try)
<sinzui> kfogel, you do not need to assign teams or person in answers. We have an answer contact system
<sinzui> kfogel, All the contacts get the email. I may not be the first person to answer the question
<kfogel> sinzui: I am very confused by what just happened.
<sinzui> kfogel, I get about 10 questions a week because I am a contact
<kfogel> sinzui:  I first tried to retarget to launchpad-answers, and got an "invalid" sign underneath.  I wish I'd saved that screenshot.  So then I tried to re*assign* to a team of that name, but no such team.
<kfogel> sinzui: now I re-blanked assignee and retargeted again, and this time it worked
<sinzui> target not assign
<kfogel> sinzui: right, that was the order I tried in.  And now it's wroking.
<sinzui> assign is for actions that require a *specific* person or role. That is to say, assignee to not providing an answer but completing a task
<kfogel> sinzui: so, nm, but that was very confusing.  No doubt I filled in a wrong field somewhere, but my memory has "corrected" that.
<sinzui> I get 15 questions a weeks for the projects I am contact for and I answer within the hour
<kfogel> sinzui: thanks for the quick response to that question (109538)
<thumper> what do we tag things that can't be QAed?
<thumper> like test fixes?
<thumper> do we just remove the qa-needstesting tag?
<salgado> qa-untestable
<gary_poster> bac, ping
<bac> hi gary_poster
<gary_poster> hey bac.  OK, I have unit tests and changes for bug574493.  I only have theoretical proof that it should actually do what we want.  What else needs to be dine:
<gary_poster> done
<bac> gary_poster: skip lunch?  Freudian slip?
<gary_poster> :-)
<gary_poster> - actually verify that it does what we want.  Maybe you can do this on a dev machine, but I'd be more tempted to do it on staging, if possible
<gary_poster> - get reviewed
<gary_poster> - get landed
<gary_poster> - get CP'd
<gary_poster> bac, so, if you are willing, I need you to take over most of that.  What I can do is pass off the branch to you with reviewer notes.  We could do that verbally looking ata diff or something else like that
#launchpad-dev 2010-05-05
<wgrant> Ah, crap.
<lifeless> where?
<wgrant> Bug 575426 is bad.
<mup> Bug #575426: copy-package broken <Soyuz:New> <https://launchpad.net/bugs/575426>
<wgrant> I wish our data model didn't make reasonable things impossible :(
<wgrant> Hm, and I imagine that check is going to be very slow.
<maxb> copying packages has not fared well this rollout
<maxb> see also bug 575450 bug 574552 :-(
<mup> Bug #575450: +copy-packages nearly unusable due to timeouts <Soyuz:New> <https://launchpad.net/bugs/575450>
<mup> Bug #574552: "different contents" error when unembargoing security update or copying packages <Soyuz:In Progress by stevenk> <https://launchpad.net/bugs/574552>
<wgrant> Yeah, the fix for 574552 caused 575426
<wgrant> It's a bit unfortunate that all the hashes are expired along with the actual file content.
<wgrant> Can't something be done about the scanning delays? Like, say, restaggering all maverick branches?
<wgrant> This is not actually an awesome way to get people to hate you less.
<adeuring> good morning
<mrevell> Helllo
<jml> mrevell, hello :)
<mrevell> Howdy jml. Don't forget to take blankets, a torch and some water, in case you get stuck in the tunnel.
<jml> mrevell, good point!
<henninge> What does it mean when I get this error message while trying Branch.getBzrBranch() locally?
<henninge> UnsupportedProtocol: Unsupported protocol for url "lp-internal:///~stevea/thunderbird/main"
<mwhudson> henninge: you haven't started a server
<mwhudson> server = get_ro_server()
<mwhudson> server.start_server()
<mwhudson> try:
<mwhudson>    ...
<mwhudson> finally:
<mwhudson>     server.stop_server()
<henninge> mwhudson: thanks
<henninge> mwhudson: get_ro_server is imported from where? (Saves me the grep)
<henninge> I am running make harness, btw.
<mwhudson> henninge: lp.codehosting.vfs
<henninge> cheers
<mwhudson> henninge: there's get_rw_server too
<henninge> mwhudson: NotBranchError: Not a branch: "lp-internal:///~stevea/thunderbird/main".
<henninge> missing something else?
<mwhudson> henninge: is that a sample data branch?
<henninge> yes ;)
<mwhudson> there's probably not actually any underlying branch data
<henninge> ah, ok. I'll push one, then.
<mwhudson> maybe ./utilities/make-dummy-hosted-branches is out of date now
<mwhudson> yes, looks like it
<henninge> mwhudson: yes, that is ... ;I)
<henninge> mwhudson: but I was able to push to ~mark/+junk/testdoc
<henninge> mwhudson: when I tell an admin to try this on  a production server, will they need to start the server as well?
<henninge> I'd think not, as it would access the production server.
<mwhudson> henninge: no, you will
<mwhudson> 'server' is perhaps a strange term
<mwhudson> it's the thing that knows how to handle the lp-internal:/// URL scheme
<henninge> ok, so it's a local thing
<henninge> mwhudson: so, this should work on production? http://paste.ubuntu.com/428201/
<henninge> it worked locally ... ;)
<henninge> _hasPotteryCompatibleSetup calls getBzrBranch
<mwhudson> oh ok :)
<mwhudson> then yeah, that should be ok
<maxb> Is there a date set for re-rollout yet? Not being able to copy packages between PPAs is rather awkward
<noodles775> maxb: 14UTC is the info I've got, but afaik it won't include the fix for 575426 - StevenK has a patch which he'll get CP'd asap.
<maxb> bug 575426
<mup> Bug #575426: SHA1-based copy checking breaks when there are expired sources in the target <Soyuz:Triaged by stevenk> <https://launchpad.net/bugs/575426>
<wgrant> And that still won't fix the inefficiency, AIUI.
<maxb> ugh
<wgrant> So timeouts will still occur for dailies.
<wgrant> (the deciding factor is the number of source publications in the target archive with the same name)
<maxb> Well, if it worked at all, I could while 1; do ./myscript; done until it works :-)
<wgrant> But I suspect the extra load caused by that would finally let Launchpad fulfill its life-long dream, which it has been steadily recently: to service requests in greater than finite time.
<noodles775> wgrant: afaiui, it'll not do the check if the source and dest. archives are the same... not sure if that's what you were refering to.
<wgrant> Er, steadily approaching.
<wgrant> noodles775: Right, but maxb needs inter-archive copies.
 * StevenK waves his arms.
<leonardr> bac, maybe you can address https://answers.edge.launchpad.net/launchpad-registry/+question/109701 or redirect it to someone more knowledgable?
<leonardr> bac: i'm also wondering who has responsibility for scripts (bug 575615 and bug 575611). is it foundations?
<mup> Bug #575615: scripts/gina.py is not logging to scriptactivity <Launchpad itself:New> <https://launchpad.net/bugs/575615>
<mup> Bug #575611: cache-country-mirrors.py is not logging to scriptactivity <Launchpad itself:New> <https://launchpad.net/bugs/575611>
<StevenK> The first is soyuz
<StevenK> The second I'm not sure about
<bac> leonardr: right, first is soyuz second is registry
<leonardr> cool, thanks
<wgrant> Is changing the project of a bug via AJAX meant to still hang forever?
<beuno> wgrant, yes, still unfixed
<beuno> deryck, knows about it
<wgrant> But it's been like a year.
<beuno> something about the URL changing
<mars> BjornT, ping
<poolie> hi wgrant, beuno, mars
<wgrant> Hi poolie.
<mars> hi poolie
<beuno> heya poolie
<mars> wgrant, :(
<deryck> wgrant, a year is nothing.  There are 4 digit open bugs on Launchpad. :-)
<wgrant> deryck: Yeah, but they're not generally obviously broken buttons that never worked :)
<deryck> wgrant, depends on who you ask, doesn't it? :-)
<beuno> deryck, while we're all complaining
<beuno> millions of bugs are not filed per day
<beuno> due to the person picker re-directing after you select
<beuno> so you loose everything you wrote
<beuno> is that on your radar at all?
<wgrant> There's a person picker in the bug-filing workflow?
<wgrant> (apart from the broken assignee input for bug supevisors)
<deryck> beuno, that is.  I have about 3-4 filebug bugs that I have on my list of 10-15 bugs that need looking at in the short term.
<deryck> wgrant, the broken assignee picker is what beuno means, I believe
<beuno> yes
<beuno> and hi deryck  :)
<deryck> beuno, hello :-)
<mars> sinzui, ping
<sinzui> hi mars
<BjornT> hi mars
<bac> reviewers meeting starting soon
<sinzui> wgrant, ping
<sinzui> jelmer, ping
<jelmer> sinzui: pong
<rockstar> jelmer, ping
<jelmer> rockstar: hey
<rockstar> jelmer, where are the unittests for IArchiveSet?
<jelmer> rockstar: are you sure there are unit tests?
<rockstar> jelmer, no.  I guess I should have said "Where are the tests for IArchiveSet?"
 * rockstar whimpers
<rockstar> Please don't be doctests, please don't be doctests...
<mrevell> night all
<intellectronica> rockstar: it's not a doctest, it's a codestory :)
<rockstar> intellectronica, those kinds of stories give me nightmares...  :)
<intellectronica> lol
<rockstar> jelmer, I don't see a test at all for it.
<rockstar> jelmer, is it alright if I create a unittest for it?
<bryceh> I'm seeing an error "ImportError: No module named tickcount" when trying to do rocketfuel-branch
<bryceh> http://pastebin.ubuntu.com/428444/
<bryceh> I did a complete re-checkout from scratch on a fresh machine, re-ran rocketfuel-branch and got the same error
<maxb> bryceh: This is interesting. I wish I could figure out why
<bryceh> I checked and I do have python-tickcount installed
<bryceh> I cal also 'import tickcount' just fine from the python prompt
<bryceh> s/cal/can/
<salgado> bryceh, can you do that on python2.5 as well?
<bryceh> bdmurray, did you run into this too?  ^^  Any tips?
<maxb> bryceh: The difference will be if you use python2.5
<bryceh> Python 2.6.5 (r265:79063, Apr 16 2010, 13:09:56)
<maxb> Please could you check what version of python-support you have installed?
<bryceh> yeah forgot to mention, both systems were lucid
<bryceh> python-support:
<bryceh>   Installed: 1.0.4ubuntu1
<bryceh>   Candidate: 1.0.4ubuntu1launchpad2
<bdmurray> bryceh: it's been quite some time since I set it up and I'm using a karmic virtual machine
<maxb> bryceh: that'll be the problem
<maxb> So the question is why don't you have the version from the launchpad ppa
<bryceh> I'll do a dist-upgrade
<bryceh> rocketfuel-setup looked like it completed successfully, not sure why it would have left that un-upgraded
<maxb> bryceh: I think this is an initial-setup corner case. Established developers are so used to being up-to-date with the PPA, so the relevant dependency to enforce this never got added to meta-lp-deps
<bryceh> ok, looks like that worked.  Thanks!
<bryceh> hmm, alright I've done 'make run' and tried loading http://launchpad.dev.  It prompts me to confirm the certificate but then I get a 503 error
<bryceh> [Wed May 05 11:11:49 2010] [error] (111)Connection refused: proxy: HTTP: attempt to connect to 127.0.0.1:8086 (localhost) failed
<bryceh> [Wed May 05 11:11:49 2010] [error] ap_proxy_connect_backend disabling worker for (localhost)
<bryceh> whoohoo got it
 * mtaylor got some feedback from someone today that they were trying to find code on launchpad for something, and when they finally found a loggerhead view, they freaked out that the launchpad skin/branding/nav went away
<jelmer> rockstar: sorry, was out for dinner
<jelmer> rockstar: I think we would have serious concerns about having our code properly unit tested.
<rockstar> jelmer, I found some integration tests, but you seem to lack unit tests in general.
<jelmer> rockstar: We're usually pretty good about adding them for new code, but there's a lot of existing code that lacks tests.
<thumper> mtaylor: that's a bit sad
<thumper> mtaylor: I'd love to integrate more
<thumper> mtaylor: but we need to make loggerhead scale better first
<mtaylor> thumper: put it in the cloud
<mtaylor> thumper: doesn't that make everything scale better magically?
<thumper> mtaylor: geez, why didn't I think of that?
<thumper> mtaylor: we are getting our own private cloud :)
<thumper> mtaylor: sugar lined and everything
<mtaylor> speaking of clouds... when are you guys going to get a cloud in the us so I don't have to deal with transatlantic lag all the time?
<thumper> it seems like we could maybe have a read only slave over there...
<thumper> with some app servers
<thumper> multi-write masters seem to be complex :)
<mtaylor> well sure - if you're using postgres :)
<mtaylor> we'll hook you up with some spiffy new drizzle servers with jay's new replication stuff
<thumper> and easy for dizzle yes?
<mtaylor> of course!
<mtaylor> we're a database for the cloud after all :)
<wgrant> sinzui: Hi.
<wgrant> LOSAs: Is buildd-manager running?
<wgrant> It looks like it has been dead for perhaps seven hours.
#launchpad-dev 2010-05-06
<spm> wgrant: it is running; but appears it is rather beautifully broken. possibly from the reroll
<spm> wgrant: ok. that looks to be working again. reverted back to prior the rollout and seeing stuff happen again. I call shenanigans
<wgrant> spm: As you might have noticed, I've been dropping in and out a bit. What's happened since I LOSA-pinged?
<spm> lala. I hadn't.
<spm> wgrant: it is running; but appears it is rather beautifully broken. possibly from the reroll
<spm> wgrant: ok. that looks to be working again. reverted back to prior the rollout and seeing stuff happen again. I call shenanigans
<spm> is all you may have missed
<AdamDV-iPod> Would it be revealing too much if someone was able to tell me what is the lp servers run?
<thumper> hardy right now
<spm> my 2c observation is a misconfig/code mess - looks like it's trying to talk to a local postgres instance, vs the db servers
<thumper> or do you mean hardware?
<AdamDV-iPod> Nope, hardy is a good answer
<AdamDV-iPod> Are there plans to upgrade to lucid?
<thumper> yes
<AdamDV-iPod> I see
<wgrant> OK.
<wgrant> What's it dying with?
<wgrant_> Argh.
 * wgrant crosses fingers, and asks again: How's it failing?
<spm> wgrant of the dropouts: psycopg2.OperationalError: could not connect to server: No such file or directory Is the server running locally and accepting connections on Unix domain socket....
<wgrant> So, it seems that my entire Optus cable node drops out whenever I ask how it's broken :P
<wgrant> But that looks like a broken config.
 * ajmitch would blame the censorship
<wgrant> spm: Ahem. So, nothing else is showing that sort of failure?
<spm> wgrant: not yet........
<wgrant> spm: Because that doesn't look even slightly Soyuzy.
<wgrant> I refactored buildd-manager lots during 10.04, but nothing that low-level.
<spm> this seems to be a change between the roll; and the reroll; i've reverted to the version from the roll; and that's fine. so... the opportunitys for brokenness are reduced.
<AdamDV-iPod> Why python over php or another language?
<wgrant> Oh, so it's actually OK now?
<wgrant> AdamDV-iPod: Um, well, PHP is... PHP.
<spm> and the configs are the same revno; so not config based. my 2c.
<wgrant> spm: Bah. I was going to accuse the prod configs.
<spm> :-)
<AdamDV-iPod> Haha
<wgrant> spm: Has it died again, or is it just busy processing uploads?
<wgrant> it's not done much for a few minutes.
<spm> manually stopped; chasing stuff with jelmer
<wgrant> Whereas it should be doing lots every 5 seconds.
<wgrant> Ahh.
<spm> who isn't here btw. he is 'away'. says so in his irc client.
<AdamDV-iPod> Launchpad depends on got-core. How ironic?
<maxb> huh?
<jelmer> AdamDV-iPod: ah, you mean git-core?
<mwhudson> it's for testing git imports
<AdamDV-iPod> Yea. Stupid touch screen.
<AdamDV-iPod> Oh
<AdamDV-iPod> Lp supports git imports?
<jelmer> AdamDV-iPod: That's probably to test the git import functionality
<AdamDV-iPod> The functionality never ends!
<spm> the kitchen-sink module is my personal favourite
<maxb> Is there really one?
<maxb> Subversion has a kitchen_sink.c :-)
<spm> maxb: I have no idea; but I wouldn't be terribly surprised :-)
<thumper> AdamDV-iPod: we do everything except what people really want, like wikis and dashboards
 * thumper tries not to be too bitter
<thumper> must be coffee time
<AdamDV-iPod> Haha
<AdamDV-iPod> People want wikisband dashboards eg?
<thumper> AdamDV-iPod: I don't grok what you just said
<AdamDV-iPod> Err, eh. I hate the iKeyboard
<Guest44380> Translation: People want wikis and dashboards now, ey?
<thumper> Guest44380: how many ids do you have?
<AdamDV2> Nah, Nickserv hates me.
<AdamDV2> I have AdamDV and TuxIce
<AdamDV2> Pidgin randomly cuts out, and doesn't identify to nickserv.
<thumper> luckily someone is working on wikis :)
<thumper> well that sucks
<AdamDV2> Ooh nice :)
<AdamDV2> And yes, it does. However, its my only option. I loathe Empathy.
<jelmer> thumper: Ooh, I wasn't aware of that - who?
<thumper> although since they aren't a priority, it is an idle / evening task
<thumper> jelmer: me
<thumper> :)
<AdamDV2> :D
<thumper> jelmer: I should have something that will serve a bzr branch as a wiki soon
<AdamDV2> Rocketfuel is installing obsolete versions of postgresql, apparently.
<thumper> jelmer: it is my evening fun
<thumper> AdamDV2: yeah, not yet updated to 8.4
<jelmer> thumper: that sounds really cool
<thumper> AdamDV2: it does work with 8.4
<AdamDV2> I see.
 * thumper uses 8.4
 * AdamDV2 uses MySQL
 * thumper needs food and coffee
<AdamDV2> Heh
<jelmer> thumper: It looks like Dulwich trunk itself works but the combination dulwuch+bzr-git is broken against http repos atm :-/
<thumper> :(
<maxb> Presumably once we go lucid and 2.6 for real, there'll be somewhat more motivation to get on with the move to 8.4
<thumper> jelmer: which versions should I take then?
<thumper> maxb: there is no reason not to use 8.4 now
<jelmer> thumper: the last version I know at least works is from 13 april
<thumper> maxb: it was just that stub hadn't gotten around to it
<thumper> jelmer: which versions are they?
<thumper> jelmer: cause I really wanted 909 of bzr-git
<thumper> jelmer: oh, and you commit *a lot*
<AdamDV2> I have *never* had apt-get take 1 hour, 15 minutes and going.
<AdamDV2> You guys aren't exaggerating when rocketfuel says "This will take awhile"
<jelmer> thumper: bzr-git 889, dulwich 562
<thumper> AdamDV2: you obviously don't upgrade to the beta OS early
<AdamDV2> I don't usually upgrade.
<thumper> jelmer: [rs=jelmer] upgrade to lp:bzr-git 899
<jelmer> thumper: alternatively I can look at fixing the issue with http, it shouldn't be too hard
<thumper> jelmer: from lp's bzr-git branch
<thumper> jelmer: if you can fix http, I'll upgrade bzr-git on LP asap
<jelmer> thumper: I don't really have time to do that now though, probably friday evening
<thumper> jelmer: ok, np, lets look to upgrade early next week
<thumper> jelmer: I can add it to my Monday tasks
<jelmer> thumper: are you going to be at UDS?
<thumper> jelmer: nope
 * thumper afk for food
<wgrant> thumper: This is where I am confused. Why are the things that would make LP less completely unattractive not priorities?
<thumper> wgrant: best answer for that is "ask jml" :)
 * thumper takes laptop to coffee ship
<thumper> shop
<wgrant> Also, I should look at wikkid at some point.
 * wgrant looks at wikkid.
<thumper> wgrant: I've just pushed my work from the last few days into trunk
<maxb> Something deeply weird is happening to the Debian package syncer: https://edge.launchpad.net/debian/+source/subversion/+publishinghistory
<ajmitch> it's been happening for awhile, I think
<ajmitch> StevenK has been looking at it
<ajmitch> at least with regards to out-of-date stuff :)
<ajmitch> the old version is due to debian being a bit strange, and having 2 different versions of source packages, see this in 'rmadison -u debian subversion'
<maxb> oh, it's picked the version in hurd-i386
<maxb> ugh
<ajmitch> ugly, isn't it?
<maxb> Do you know where the bug might be found (I assume there is one) - registry? soyuz?
<ajmitch> most likely related to bug 568745 which I filed
<mup> Bug #568745: Debian unstable record of gpt missing <gina> <Soyuz:Triaged by stevenk> <https://launchpad.net/bugs/568745>
 * ajmitch should put in the info about duplicate info in the Sources.gz from debian
 * AdamDV2 curses
<wgrant> ajmitch, maxb: But gina has been able to handle multiple versions for either a few weeks or a couple of days.
<wgrant> StevenK: That was one of your first branches, wasn't it?
<StevenK> wgrant: I think so
<ajmitch> wgrant: that subversion +publishinghistory is showing the first version listed in Sources as being the latest
<wgrant> ajmitch: That suggests that it just hadn't picked it up before.
<wgrant> Probably because StevenK's change hadn't landed yet.
<wgrant> So when his change landed, it picked up both versions. But the real latest version was already there, so it wasn't recreated.
<wgrant> So only the old one was new, so it appears to be the latest.
<ajmitch> that sounds reasonable
<ajmitch> it just looks a little odd when https://edge.launchpad.net/debian/+source/subversion has that old version as the latest upload
<wgrant> Yes, that is slightly COMPLETELY INSANE.
<StevenK> Right. The bug is that gina is of the opinion that gpt 1.0.4-3 already exists in the database, so it doesn't import it
<wgrant> But it's right.
<ajmitch> that's special
<wgrant> It's not a bug.
<wgrant> Oh, wait, different one.
<StevenK> wgrant: It does not exist for Debian sid.
<ajmitch> at the time I filed that bug the sync hadn't been requested for lucid either
<wgrant> Hmm.
<wgrant> StevenK: What suggests that it thinks it's already there?
<StevenK> wgrant: Because of the log messages
<wgrant> StevenK: Pfft.
<StevenK> wgrant: You don't have much faith in logs now, do you? :-)
<wgrant> StevenK: I might point out that this is gina we are talking about.
<wgrant> I don't have much faith in it, let alone its logs.
<StevenK> wgrant: Point
<StevenK> wgrant: So I think the bug is in _getSource in lib/lp/soyuz/scripts/gina/handlers.py
<StevenK> wgrant: Since that function is effectively the one that decides if the package has been imported and can be skipped, or hasn't, and needs to be imported.
<wgrant> StevenK: See, I looked at that, but it looks fine.
<wgrant> What is the log message that tells you that it's skipping it?
<wgrant> I only see a log call when it isn't being skipped.
<wgrant> And, well, https://edge.launchpad.net/debian/+archive/primary/+copy-packages?field.name_filter=gpt&field.status_filter=&field.series_filter= makes it pretty clear that 1.0.4-3 was never in that archive.
<wgrant> And _getSource's query looks correct to me.
<wgrant> StevenK: So, I doubt it's skipping it because it's already there -- unless you have a log message to prove it.
<StevenK> Just getting logs synced
<StevenK> 2010-05-06 02:06:20 INFO    SourcePackageRelease already published with no chang
<StevenK> es as Pending
<StevenK> wgrant: ^
<StevenK> 2010-05-06 02:06:20 INFO    gpt already exists in the archive
<wgrant> Oh, interesting.
<wgrant> It really could do with logging the version string, though.
<wgrant> StevenK: So, there's only one source version in unstable, but are you sure that's not from a stable/testing run?
<StevenK> wgrant: Exactly the same message for sid, lenny and squeeze
<wgrant> Hrmph.
<StevenK> wgrant: But yes, the SQL reads cleanly and looks right
<wgrant> StevenK: Have you tried convincing it to spew SPR/SPPH versions and IDs? I am really really confused at how this can go wrong.
<wgrant> My first thought was that it didn't know about archives.
<wgrant> But it does.
<wgrant> It restricts the location correctly.
<StevenK> wgrant: I can't recall how to pull the SPPH out, since the SQL returns the SPR
<wgrant> StevenK: The thing prints out 'Pending', so it has the SPPH somewhere there too.
<wgrant> But that's probably in publish_sourcepackage.
<StevenK> I was thinking in _getSource
<StevenK> A debug print of what it thinks is actually in the archive
<wgrant> Right, but if you're going to get a patch applied, it's probably best to get as much debugging in as possible.
<wgrant> I'd be printing out the version we're searching for, and the found SPR and SPPH IDs.
<StevenK> wgrant: http://paste.ubuntu.com/428746/
<StevenK> wgrant: I don't think that's enough, but I welcome suggestions
<wgrant> StevenK: SPR IDs too, please.
<StevenK> wgrant: It's .id?
<wgrant> StevenK: Yes. http://paste.ubuntu.com/428748/ would also be handy.
<StevenK> wgrant: http://paste.ubuntu.com/428750/
<wgrant> StevenK: LGTM
* spm changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.04 | PQM is open | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
<spm> wgrant: LGTM ? Lets Go To Mothers????
<StevenK> Looks Good To Me
 * spm holds up the humour meter near StevenK ... hrrm. reading near 0....
<wgrant> So, yes, hopefully that patch will tell us something less useless.
<adeuring> good morning
<mrevell> Morning Launchpadderisers.
<StevenK> wgrant: Still here?
<wgrant> StevenK: I am.
<StevenK> wgrant: Right, so we get this:
<StevenK> 2010-05-06 07:15:51 INFO    SourcePackageRelease already published with no chang
<StevenK> es as Pending (SPPH 478847)
<StevenK> 2010-05-06 07:15:51 INFO    gpt already exists in the archive
 * wgrant hunts out SPPH 478847
<StevenK> I have it, going to paste
<wgrant> gpt 1.0.4-1 in sid
<StevenK> gpt          | 1.0.4-1        | debian | sid
<wgrant> Intriguing.
<wgrant> So... you're sure it's looking for -3? Does the log say so?
<StevenK> My debug message doesn't fire at all :-(
<StevenK> Oh, duh
 * StevenK asked spm for the wrong thing
<wgrant> archive.txt is too long.
<noodles775> yep (well, *exceptionally* long... all the doc tests are long :/)
<wgrant> Heh.
<maxb> Hrm.
<maxb> When the datacentre upgrades to lucid, as a side effect of the updated dpkg, the form of the quilt metadata in all source format 3.0 package import branches is going to change
<maxb> From the .dpkg-source-applied file to the .pc/ directory
<wgrant> maxb: Hm, it's not already running a very recent dpkg?
<maxb> Not *that* recent
<maxb> The change is in 1.15.5.4
<wgrant> Oh, really really recent.
<wgrant> This would be a james_w thing, right?
<maxb> yes
<james_w> eh?
<leonardr> BjornT, can you help me track down the apache configuration for launchpad in production?
<leonardr> i thought it would be in lp-production-configs but that only has launchpad configs
<BjornT> leonardr: i think you need to ask a losa for that
<leonardr> losas, will you induct me into this secret?
<maxb> james_w: Once the udd importer starts running on lucid, lucid's dpkg will be producing a different form of quilt metadata in unpacked sources, so there will be spurious churn in the first versions of every 3.0 package imported after the change
<maxb> I don't think there's anything to be done about that, though
<james_w> no, I don't think so
<maxb> In a perfect world we'd move the format change into its own revision, but ETOOMUCHFAFF
<manish> leonardr, you free?
<leonardr> manish: i'm actually about to go afk for a bit. are you available at around 12:30?
<manish> leonardr, 12:30 UTC?
<leonardr> manish: sorry, that was stupid
<leonardr> in about an hour and a half
<manish> leonardr, sure
<leonardr> great
<jml> wgrant, I'm happy to answer an email, or talk at UDS or what have you
<manish> leonardr, am back. Whenever you are free, just tell me
<leonardr> manish, i'm back
<manish> leonardr, I was just checking your mail again. It is highly insightful. I never thought of that situation
<manish> the major question is have is what I mentioned at the end of the second mail I sent. How will the apps know about the new API? this makes Apps API specific?
<leonardr> manish: let me pull up your second email to respond in detail
<leonardr> but, it's true that there's a limit to the server-side changes that a client can hide from an app
<manish> leonardr, i see, this was the mail I was talking about https://lists.launchpad.net/launchpad-dev/msg03314.html
<leonardr> thanks for link
<leonardr> the advantage of the launchpadlib approach is that even in the worst case, where a client based on launchpadlib totally breaks, launchpadlib itself does not break
<manish> leonardr, Yeah. i can see that. Probably that caching is taken care by the library (launchpadlib) rather than the app
<manish> another questions is that how will I come to know what all versions of API are present?
<manish> means via code? not manually
<leonardr> manish: that's a very good question. right now there is no machine-readable advertisement of the available versions
<leonardr> i would really like to add one, and if you need it, that would give me a reason to fight for time to add it
<manish> or maybe we can parse the +apidoc page to check the existence, but it would be a brutal thing
<leonardr> exactly
<leonardr> i want to publish a json equivalent of the +apidoc page at http://api.launchpad.net/
<manish> another problem with LP is that the WADL file is present behind OAuth
<manish> it should be publicly available
<leonardr> manish: i agree. the service root and the version list should not be protected
<manish> leonardr, what I intend to make is a component which takes in WADL file and then converts it to a library which can be loaded at runtime
<manish> and another component which checks the API version and calls the above component
<leonardr> that sounds pretty good. let me respond to your message in order
<manish> leonardr, sure
<leonardr> so we covered #1 a little bit
<leonardr> a client written against version 1.0 that uses checkBugIsValid will not break when we release version 2.0 with checkBugValidity
<leonardr> it will break, but that's not the time when it will break
<leonardr> it will break when 1) someone tries to upgrade it to use 2.0 instead of 1.0
<leonardr> or when 2) we end-of-life version 1.0
<leonardr> our goal is to work with developers so that they upgrade their applications (and fix problems like this) well before an old version reaches end-of-life
<manish> ^^ point. The most apt one
<leonardr> so, now on to #2 with that for context
<leonardr> wait, does that answer your question about having multiple backends?
<leonardr> the idea is that you will have one backend and you will gradually upgrade it to keep it up to date with launchpad
<leonardr> since launchpad is always available there's no need to have a 1.0 version and a 2.0 version. once you port your app to 2.0 you can get rid of the 1.0 backend
<manish> leonardr, I meant that there is a backend for example named foo
<manish> and the app just tells it what version of API it is built for
<manish> and the backend foo passed it the library which is linked
<leonardr> ah, i see
<manish> I havnt done it, just planning to do it this way
<leonardr> yes, the app will say it's based on version 2.0, and you will give it a library based on the current wadl file for version 2.0
<manish> ^ exactly
<leonardr> this should work fine
<manish> I think this way there wont be hundreds of copy of the library
<leonardr> the one caution i would give you is to make sure you take advantage of whatever caching features are present in your http client
<manish> and every app doesnt need to be bundled with a copy of library
<leonardr> so that you don't retrieve the wadl file every time
<manish> yeah. it would be surely there. It
<manish> when the app requests for example v2.0 of API
<manish> then foo would hunt for library for it
<manish> if not present, it would download the WADL and generate the library
<manish> and then passed
<manish> the generated library is stored in the library cache
<manish> means relevent location as per the OS
<leonardr> ok, i just want to be clear that the wadl file for 2.0 may change once a month when a new launchpad is rolled out, or even more often than that on edge
<leonardr> you will need to occasionally check for changes and possibly re-download the wadl and regenerate the library
<manish> is it so?
<manish> once a stable API is released, it should be stable
<manish> any new changes should be in a new version.
<leonardr> for most new features, it's easier for us to just add them to all versions than to specifically exclude them from old versions
<manish> another problem with WADL is the enormous size
<manish> it takes minutes to download it
<leonardr> you will need to build this flexibility in anyway, to allow clients to access the 'devel' service
<manish> something has to be done here
<leonardr> manish: this is why i talked about the caching
<leonardr> we have made a number of improvements
<leonardr> i am working on one final improvement, which is compressing the wadl in transit
<manish> leonardr, that's cool
<leonardr> (i implemented this a long time ago and only recently discovered that it didn't work)
<leonardr> let me quickly outline the levels of caching
<leonardr> the first time a user starts an application the full wadl will be downloaded
<leonardr> once i make the compression fix, this will be about 100k instead of over 1 megabyte
<manish> Yes. based on the app API version
<leonardr> yes, per version
<leonardr> this wadl file will be used for 1 week (unless it is the devel wadl, but you won't use the devel wadl in a real application)
<leonardr> no requests for the wadl at all
<leonardr> during that week
<leonardr> at the end of that week, the client will make a _conditional_ request for the wadl
<leonardr> if the wadl changed in that week (because we rolled a new version of launchpad), the client will download the new wadl--another 100k
<leonardr> if the wadl has not changed (because launchpad has not been updated), the client will download nothing
<leonardr> and will use the existing wadl for 1 more week
<manish> well, how to achieve this?
<manish> some headers in the request?
<leonardr> yes. it is already implemented in launchpadlib + httplib2
<leonardr> i want to make sure you implement it too, or use an existing http client library that understands caching and conditional requests
<leonardr> so you can receive the same benefit
<manish> I mean does the request to fetch WADL returns a specific header which contains a specific code?
<leonardr> yes
<manish> like the revision no of the API
<leonardr> sort of. there are two headers
<manish> then I could do a HEAD request and read this revision no
<leonardr> Cache-Control: max-age=604800
<leonardr> ETag: "004e1871cab3546c277f7ebae72513058bc6b50f"
<leonardr> the ETag is analogous to the revision no
<leonardr> the Cache-Control tells the client that it can use this document for 1 week
<manish> okay. For each WADL revision,  ETag is different?
<leonardr> yes
<leonardr> the ETag must change when the document changes
<manish> okay. Sort of Hash value?
<leonardr> exactly
<leonardr> so there's a week when you can just use this document from the cache without requesting it again
<leonardr> at the end of the week, you can make a conditional request
<leonardr> GET /1.0
<leonardr> If-None-Match: "004e1871cab3546c277f7ebae72513058bc6b50f"
<leonardr> this tells the server to compare the ETag you give it in If-none-Match against the current ETag
<leonardr> if the ETags are the same, that means the wadl has not changed
<leonardr> the server will send:
<leonardr> 304 Not Modified
<leonardr> Cache-Control: max-age=604800
<leonardr> that means you can use the existing wadl for another week
<leonardr> if the ETags differ, the server will send
<leonardr> 200 Ok
<leonardr> Cache-Control: max-age=604800
<leonardr> ETag: "some-new-etag"
<leonardr> and then the new wadl, which you can cache for a week
<leonardr> you can make a HEAD request, but then if it turns out the wadl has changed, you will have to make a GET request anyway
<leonardr> so you might as well make a conditional GET request
<leonardr> does this make sense?
<manish> Absolutely. :)
<leonardr> great
<manish> It clears so many of my doubts
<leonardr> like i said, hopefully you can use an existing C# http client that understands these rules
<leonardr> they are standard parts of http
<leonardr> but if not, they are fairly simple to implement
<manish> It's pretty simple to implement this
<leonardr> yeah
<manish> If-None-Match headers solves so many of my doubts
<manish> otherwise I would have gone crazy
<leonardr> yes, it's very useful
<manish> my biggest fear was what about the people who are under slow internet connection
<manish> or who pay per kb
<manish> so compression of WADL is very useful
<leonardr> yes, it's very bad that the compression system has been broken for so long
<leonardr> unless you have any questions, i'm going to go back to work on that
<manish> Nope. Nearly all doubts solved which I had presently
<leonardr> great
<leonardr> ping me again if you have more questions
<manish> leonardr, sure.
<mrevell> nytol
<sinzui> mars, ping
<sinzui> salgado, ping
<salgado> hi sinzui
<sinzui> salgado I am hacking on bug 575317. I cannot delete the account because it is
<sinzui> used required by the OpenIDSummary table. I think I need to update that table
<sinzui> to point the entries to the remaining account. This may be pointless because
<sinzui> other schema parts I do not know about also link to Account. I could change
<sinzui> the openid of the merged account to ensure authentication falls back to email
<mup> Bug #575317: Person merge must delete merged account <Launchpad Registry:Triaged> <https://launchpad.net/bugs/575317>
<sinzui> address
<sinzui> salgado, maybe our OpenIdRPSummary table is obsolete. Lp is not the real OpenId provider
<salgado> it is obsolete, yes
<sinzui> fab
<salgado> but you're right that there may be other tables referencing the row you want to delete
<salgado> I guess changing the OpenID identifier of the merged account is the easier solution, while we don't get rid of the Account table completely
<mars> sinzui, pong
<sinzui> There is a function that allows us to change the openid. I could use it to make the merge account unmatchable
<sinzui> mars, salgado has confirmed my discoveries about account. ^ I will instead change the account's openid identifier so that there is no match--fallback to the email address to select the Lp person
<mars> sinzui, cool
<sinzui> mars, I will prepare a script to that we can run in production that will update the merged accounts.
<salgado> Chex, the script sinzui described above should make it unnecessary to delete that account you were trying to delete
<sinzui> Chex, yes it will. I have will not know the rules of what must change until I complete the test I am writing
 * maxb is amused by the Code/Branches backtracking
<thumper> maxb: I wasn't particularly happy with the change when it happened
<maxb> I like Code better too
<wgrant> Can someone please EC2 https://code.edge.launchpad.net/~wgrant/launchpad/bug-576168-no-disabled-latest-ppas/+merge/24812? Vanished instance count so far: 1
<wgrant> sinzui: About bug #576478: it doesn't make sense to have Packagings at all for virtual packages like that.
<mup> Bug #576478: Common link for virtual package providers <Launchpad Registry:Triaged> <https://launchpad.net/bugs/576478>
<sinzui> wgrant, no?
<sinzui> wgrant I think we need to know which ones are virtual then...Edwin may have a step to a solution. He is adding real database data for DSPs. We can ignore the packaging questions if the DSP is virtual
<wgrant> sinzui: *Binaries* are virtual, not sources.
<wgrant> And they won't exist in the DB, since they are virtual.
<wgrant> They only exist by being Provided by another binary.
<wgrant> They are used for things like mail-transport-agent, where any one of many packages can fulfil the dependencies.
<wgrant> They do not represent a particular piece of software.
<sinzui> wgrant, I see. Thanks for explaining that
<wgrant> Can someone please suspend https://login.launchpad.net/+id/mHhGP3h? He spammed the dev wiki overnight.
<maxb> Identity theft too :-(
<wgrant> Hm, yes, the name was different a few hours ago.
<wgrant> "Eugene Griffin"
<wgrant> Oh.
<wgrant> Wrong URL.
<wgrant> https://login.launchpad.net/+id/7k7hHkp is the real one.
<wgrant> (had to steal it from the wiki source, since the links point to the LP person, which doesn't exist)
#launchpad-dev 2010-05-07
<lifeless> mwhudson: if I ask nicely, could you land a branch for me ?
<lifeless> https://code.edge.launchpad.net/~lifeless/launchpad/merge
<lifeless> thumper: got a minute to talk package reviews? I want to flow a few high level things past you
<thumper> lifeless: not right this instance, I've asked aaron to talk to james at uds
<thumper> lifeless: perhaps in an hour or so
<mwhudson> lifeless: ok
<lifeless> thumper: the bzr team is working on this problem/area too - we should talk together
<lifeless> james_w: ^
<lifeless> mwthanks!
<lifeless> mwhudson: thanks!
<lifeless> thumper: ok, will poll you ~2:30
<mwhudson> lifeless: want to set a commit message?
<lifeless> doing so
<lifeless> done
<mwhudson> lifeless: kicked off
<lifeless> thanks!
<thumper> beuno: I bet your not really around are you?
<thumper> I wish leonardr was around
<beuno> thumper, I am. My brain, OTOH....
<nigelbabu> bryceh: you're right, it doesn't turn up on UI
<nigelbabu> I assumed it came up because the srting was right for the UI
<thumper> hmm...
<thumper> is mailing list stuff foundations?
<thumper> I guess it is
<wgrant> It's Registry.
<thumper> wgrant: is it?
<thumper> hmm... ok
<wgrant> thumper: Yes. Think where barry was, and think where teams are.
<wgrant> (and also look where all the bugs are)
 * thumper moves the question again...
<lifeless> thumper: poll()
<thumper> lifeless: here, but collecting girls from school in about 3 minutes
<lifeless> heh
<lifeless> after that ?
<thumper> sure
<thumper> lifeless: here now
<thumper> lifeless: if you don't mind listening to me munching
<lifeless> sure
<jonathan__> Hi all
<jonathan__> I'm installing launchpad on Ubuntu 10.04 and I'm facing a problem during the "make schema"
<wgrant> jonathan__: What's the issue?
<mwhudson> jonathan__: pastebin the log you gt?
<jonathan__> yes I am doing this
<jonathan__> you can have the result of the "make schema" command here :http://paste.ubuntu.com/429292/
<wgrant> jonathan__: sudo apt-get upgrade
<wgrant> Then make schema again.
<jonathan__> ok let me try
<wgrant> You need a new version of python-tickcount from the PPA that rocketfuel-setup activated.
<lifeless> thumper: https://bugs.edge.launchpad.net/bzr/+bug/576778
<mup> Bug #576778: easy multi-user server setup <Bazaar:Confirmed> <https://launchpad.net/bugs/576778>
<wgrant> +lots
<jonathan__> wgrant: ok thanks it seems to work
<wgrant> jonathan__: Great,
<wgrant> spm: Is the current mailman problem known? It seems to not be (de)activitating lists or subscriptions.
<thumper> wgrant: yes, and bug is filed
<wgrant> Ah, good.
<StevenK> wgrant: So, it looks like the Sources.gz on iron is out of date
<StevenK> wgrant: The pool has 1.0.4-3 in it, but the Sources.gz for sid only mentions 1.0.4-1
<StevenK> (And hence, gina only tries to import 1.0.4-1)
<spm> and we're hoping to get some info out of debmirror in ~ 1.5 hours as to whats up with that
<wgrant> StevenK: Heh, that's why I wanted the extra debugging info.
<StevenK> wgrant: Yeah, I got it before lunch
<wgrant> spm: ... 1.5 hours?
<StevenK> Now less than 1
<wgrant> What's taking so long?
<StevenK> We're waiting for an automatic run
<spm> scheduled outa cron. this is not so urgent a task it needs a manual force re-run on what may be a rather... aggressive task. unfreidnly to the remote system.
<wgrant> Ah, right.
<wgrant> If it's debmirror, it's just downloading a few megabytes each run, but OK.
<spm> oh? is that all? is that a fairly fast process?
<wgrant> Yes. It's not like rsync.
<lifeless> spm: debmirror reads the package metadata
<spm> bugger it. runing now.
<wgrant> It just grabs the indices, works out what it doesn't have, and grabs it.
<lifeless> spm: contrast with rsync :P
<StevenK> Haha
<spm> damn. very fast.
<spm> wooo. we have logs.
<spm> Hmmm. I can't see what's wrong tho. does this mean anything to you guys?
<spm> [0%] Getting: dists/lenny/Release... dists/lenny/Release failed 500 Can't connect to ftp.uk.debian.org:80 (connect: Connection refused)
<wgrant> Who broke the firewall *again*?
<wgrant> It's broken a few times in the last couple of years.
<spm> ahh. point. that may be pebkac....
<spm> trying agin with proxy settings...
<StevenK> Hehe
<spm> nope. same.
<wgrant> Using the special debmirror proxy options?
<wgrant> It has its own, which suggests that it might not respect $http_proxy.
 * StevenK checks that
<spm> bleh, lets give that a whirl
<StevenK> debmirror, how I hate thee
<spm> although; wget seems to be having issues as well; so the problem is looking firewallish...
<StevenK> And debmirror does respect http_proxy
<wgrant> StevenK: Did you try it, or venture deep into the lovely, lovely Perl?
<StevenK> The latter
<spm> I call access shenanigans. will create an RT to look into; no GSA's around atm.
<StevenK> And debmirror's Perl makes me vomit
<wgrant> StevenK: As I said: lovely, lovely Perl.
<spm> oh ffs.
<StevenK> (Coming from someone who used to code Perl for a job)
<StevenK> spm?
<wgrant> Heh, yes.
<spm> I exported the proxy settings, and it's working.
<StevenK> Excuse me while I cackle
<wgrant> Hahah.
<wgrant> Now, it'll probably get a hash or key mismatch or something.
<spm> I was quite sure you don't need to do that - but the way this script is being called... i suspect breaks that nicely...
<wgrant> Although the latter is less likely if the stuff's already in the pool.
<spm> http://paste.ubuntu.com/429339/
<wgrant> Er.
<wgrant> Er what.
<wgrant> What.
<wgrant> What.
<wgrant> let's see what we have here.
<wgrant> Huh.
<wgrant> Sources is broken.
<wgrant> debmirror is perfectly correct.
<wgrant> sid's main Sources.gz refers to this file:
<wgrant>  ebe76d2199e758df9aa7e3eaaf499190 9250730 kde-l1
<StevenK> In which stanza?
<wgrant> kde-l10n 4:4.4.3-1
<wgrant> In fact, the whole Files section is massively truncated compared to the Checksums-*.
<wgrant> I call broken a-f cache.
<StevenK> Yay, we proved Debian is broken
<spm> I'm not sure that's something to celebrate over :)
<StevenK> Are you sure? It's a problem with debmirror, and not gina
<wgrant> Not with debmirror.
<wgrant> Somewhere in dak! Even better.
<StevenK> Heh
<StevenK> Somewhere where *I* don't have to fix it
<wgrant> I see no ftp.d.o bugs, so I wonder if they know about it.
<spm> on the grounds I like my job; I refuse to believe dak could ever have problems
<StevenK> Haha
<wgrant> Heh.
<StevenK> spm: So, dak is just like TeX? It doesn't have bugs, it has "porting issues" ?
<spm> it has no issues. no further discussions will be entered into
<spm> right. added those proxy settings to the wrapper script and exported same. if THAT doesn't keep it working; I'll break down and cry, or maybe sniffle.
<wgrant> FSVO "working".
<StevenK> spm: And kept the logging?
<spm> yarp
<StevenK> Okay, sweet
<wgrant> So... someone should go and attack an ftpmaster, I guess.
<spm> StevenK: that we weren't seeing ANY emails is bad. At least with logging we get something.
<spm> I'll add some date stamps in there as well; make it useful.
<jonathan__> Hi all (again)
<wgrant> Hi jonathan__. Did you get it working?
<jonathan__> to run launchpad the correct command is make run, Am i right ?
<wgrant> That's right.
<jonathan__> yes it worked
<jonathan__> I restarted my virtual machine
<jonathan__> and now I have a problem when I run 'make run'
<wgrant> You may need to start Apache manually, depending on your setup.
<wgrant> What's the problem?
<jonathan__> wgrant: no it's not a problem of apache
<jonathan__> the script tries to kill the process librarian and it can't find it
<jonathan__> http://paste.ubuntu.com/429355/
<wgrant> Ah. Perhaps remove the pid file manually.
<wgrant> You didn't stop Launchpad cleanly before you restarted it?
<jonathan__> Unhappily you should have right ...
<jonathan__> so how to restore/repair Launchpad if it was not stopped cleanly ?
<wgrant> Just remove the PID file.
<wgrant> It should do it itself, but apparently doesn't at the moment.
<wgrant> There's nothing actually broken.
<jonathan__> the pid of launchpad ?
<wgrant> Yes.
<wgrant> noodles775: Morning. Can you please reEC2 that branch? It appears to have vanished.
<noodles775> Hi wgrant , sure.
<jonathan__> wgrant: Yes, right, I deleted the development-launchpad and development-librarian pids. It's ok now
<wgrant> jonathan__: Great.
<wgrant> noodles775: Thanks.
<mrevell> Hello!
<noodles775> wgrant: I'm having issues with ec2 land, so haven't sent it off again yet. I'll let you know if I do.
<wgrant> noodles775: Eep. OK. Thanks for trying.
<cody-somerville> Shouldn't the topic say week # of 10.05?
<ricotz> StevenK, hello, is this the bug i am experiencing here? OOPS-1588ED588 - https://bugs.launchpad.net/soyuz/+bug/575426
<mup> Bug #575426: SHA1-based copy checking breaks when there are expired sources in the target <oops> <ppa> <Soyuz:In Progress by stevenk> <https://launchpad.net/bugs/575426>
<mars> cody-somerville, that's a grey area
* mars changed the topic of #launchpad-dev to: Launchpad Development Channel | Week Î¼ of 10.05 | PQM is open | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
<mars> bac or matsubara, ping, what command did you use when you experienced the ec2 suite hangs?
<matsubara> mars, ec2 land
<mars> matsubara, same here (I think).  Thanks.
<matsubara> mars, np, when I used ec2 test it didn't fail for me but salgado reported that he had experienced the hang with ec2 test too
<jpds> https://edge.launchpad.net/+search?field.text=error+setting+MTRR â does that page reference itself in the first hit?
<mars> jpds, it most certainly does :)
<jpds> mars: Impressive.
<mars> jpds, we should not be indexing the search results page.  That is a bug.
<bac> mars: ec2 test is what hung on me yesterday.  i've seen it with land and test
<mars> bac, ok, thanks
<salgado> abentley, how do source package branches get updated?  does people commit to them or are they imported branches?  if the former, how do we control who can commit to a given source package's branches?
<abentley> salgado, it doesn't matter.  All kinds of branches can be sourcepackage branches.
<abentley> The owner can commit to a sourcepackage branch, (if it's not an import/mirrored branch)
<abentley> salgado, for example, you could commit to ~salgado/ubuntu/lucid/bzr/my-change
<salgado> abentley, my question was not clear, sorry.  I was actually thinking about the "official" source package branch (e.g. lp:ubuntu/bzr?) but I guess that's just an alias to another branch, similar to the development focus of a project branch?
<abentley> salgado, that's right.
<salgado> I see.  it makes sense now
<salgado> thanks abentley
<abentley> salgado, np
<leonardr> edwingrubbs, is there some way to use the storm equivalent of prejoins to reduce the number of database queries necessary to get all of an object's fields?
<EdwinGrubbs> leonardr, yes, let me get you an example
<leonardr> EdwinGrubbs: i'm in a tricky situation because i want to do this from lazr.restful, with no special knowledge of the object's schema
<leonardr> just as a random example, it would be nice if bug.can_expire didn't make its own sql query
<EdwinGrubbs> leonardr, hmmm, that will takes some introspection on the storm attribute definitions
<leonardr> edwingrubbs: let me state the actual problem rather than possible solutions
<leonardr> i benchmarked the creation of a lazr.restful json representation
<leonardr> it took 6 database queries to build that representation
<leonardr> two bug.bugtasks, one bug.can_expire, one bug.permits_expiration, two more i can't track
<EdwinGrubbs> leonardr, it looks like the easiest thing would be to copy some of the logic that the fake sqlobject wrapper around storm uses to handle prejoins. It looks like prejoins handle complicated situations like loading bug.assignee.team_owner, so you could probably simplify it quite a bit.
<leonardr> edwingrubbs, could it be simplified to the point where lazr.restful could simply adapt an object to an IPreloadedObject (if such an adapter was available) and proceed in the knowledge that the right thing had been done?
<EdwinGrubbs> leonardr, actually, it looks like you could take a storm object and just call SQLObjectResultSet(storm_object, prejoins=attr_list)._prepare_result_set() where you just populate the attr_list with the names of attributes that are instances of SimpleProperty. So, using an adapter should work. _prepare_result_set() will call find() and return a normal storm resultset to you.
<leonardr> edwin: spell it out for me. i have a bug, i make a SQLObjectResultSet with prejoins= all the fields that are published in the web service
<leonardr> and then i get what--the same bug? a new bug? a list of objects of some kind?
<EdwinGrubbs> leonardr, if you are expecting a single item, you would call resultset.one(), and that will raise an exception if you get more than one item. If you are expecting a list, you can just iterate over resultset.
<leonardr> the problem is i don't know what to expect
<leonardr> i have an instance of Bug
<leonardr> i know which fields i'm going to access
<leonardr> and i want some other code that knows something about how Bug works to do all the queries at once, behind the scenes
<leonardr> looking at how properties like can_expire are implemented, this seems like a pipe dream
<EdwinGrubbs> leonardr, let me try to mock something up for you.
<leonardr> ok
<leonardr> for the sake of argument, i have a bug and i want to access can_expire and permits_expiration
<EdwinGrubbs> leonardr, I hadn't looked at permits_expiration and can_expire. Those are much more complicated than what prejoins let you do. can_expire actually calls permits_expiration first to try to avoid the really expensive query.
<leonardr> ok, how about is_complete?
<EdwinGrubbs> leonardr, all three of those query all the bugtasks for a bug. storm caching uses the primary key to look up objects, so accessing BugTask.bug would use the cache, but Bug.bugtasks depends on the foreign key on the BugTask table. The only way to load all the necessary data in one query is to make an adhoc query with lots of subqueries.
<leonardr> edwin: i think this is getting complex enough that there's clearly not an easy win here (though there does seem to be a general performance problem if we're getting the list of bug tasks for a bug twice in a single user request)
<EdwinGrubbs> leonardr, well, I think storm with cache the Bug.bugtasks attribute that is used both in permits_expiration and is_complete, but can_expire does a much more complicated custom query without actually using the Bug.bugtasks attribute.
<leonardr> edwingrubbs: i'm eod. gary, take a look at my conversation with edwin. let's talk briefly on monday about how far this is worth pursuing
<gary_poster> leonardr: ack will do thanks.  have a good weekend
<EdwinGrubbs> maxb, ping
#launchpad-dev 2010-05-08
<mthaddon> staging DB server being rebooted
#launchpad-dev 2010-05-09
<deryck> hi.... anyone around?  Is there a "null" person the way there is a null project?  Anyone know?
<AdamDV> Hey all, how can I un-do what launchpad did to my system?
<AdamDV> Particularly apache.
<jelmer_> AdamDV: launchpad is just a website, it shouldn't affect your system. What are you trying to undo exactly?
<AdamDV> No, I mean I branched and compiled the code on my system.
<AdamDV> Upon attempting to access phpmyadmin today, I noticed apache is not parsing php, and is my browser is attempting to download the index.php file.
<jelmer_> AdamDV: Ah, rocketfuel-setup. I'm not aware of anything that reverses the changes it makes.
<AdamDV> Lovely.
 * AdamDV sighs
<AdamDV> Question 2, does rocketfuel completly fuck php? Or does it just un-install mod-php?
<thumper> AdamDV: I don't think it does
<thumper> AdamDV: the only apache setup it does is to add locallaunchpad to /etc/apache2/sites-available
<thumper> and enables it
<thumper> check to see if you still have mod-php installed
<AdamDV> Wouldn't it also have to install and enable mod_python and fastcgi?
<AdamDV> DOing now.
<thumper> although I have php and LP on my server
<thumper> running fnie
<thumper> fine
<thumper> jelmer_: are you at UDS?
<jelmer_> thumper: yep
<thumper> jelmer_: can you poke poolie about the bzr config bug?
<AdamDV> Heh, my eventual goal is to examine the LP source and write a somewhat dumbed down PHP+MySQL equiv.
<thumper> jelmer_: it is almost becoming critical for us with imports
<thumper> AdamDV: really?
<AdamDV> Ah-ha. libapache2-mod-php5 is not installed, according to apt.
<AdamDV> thumper: SOmething like that.
<AdamDV> MOre like just code hosting.
<jelmer_> thumper: will do
<AdamDV> Err, sticky caps lock.
<thumper> jelmer_: ta
<AdamDV> Problem solved. Anyone have any ideas why rocketfuel would remove mod-php5?
<ajmitch> because it uses a different apache mpm
<AdamDV> That makes sense.
#launchpad-dev 2011-05-02
<lifeless> thanks python 2.7
<lifeless> String or Integer object expected for key, unicode found
<wgrant> What for?
<wgrant> Some fake dict?
<lifeless> DBM
<lifeless> https://bugs.launchpad.net/testrepository/+bug/775214
<_mup_> Bug #775214: On python 2.7: String or Integer object expected for key, unicode found <Testrepository:New> < https://launchpad.net/bugs/775214 >
<wgrant> Ah :(
<lifeless> wgrant: the fun of it is it that one can expect python 3.x to be diametrically opposed
<wgrant> lifeless: Surely 3.x expects bytes/int?
<wgrant> (if not I may cry)
<lifeless> wgrant: for a dict interface?
<lifeless> wgrant: I haven't started a 3.x port of testrepository yet.
<wgrant> lifeless: Yes.
<lifeless> I may tonight or something; right now am just yak shaving my way back up to a working desktop
<wgrant> Since Unicode makes very little sense in DBM, surely...
<lifeless> wgrant: no less than it does in the sql interface postgresql uses
<wgrant> lifeless: Hm? postgresql does Unicode...
<lifeless> wgrant: yes
<lifeless> dbm doesn't do collation
<lifeless> so implementing unicode for a user is simply encode on the way in, decode on the way out.
<lifeless> no good reason not to support that
<wgrant> Mmm
<lifeless> the only reason not to that I can think of is that its slow
<lifeless> but thats python's choice of string model more than anything else
<wgrant> lifeless: https://lp-oops.canonical.com/oops.py/?oopsid=1944DQ278
<wgrant> Heat updates are expensive.
<wgrant> (although this seems like a particularly inefficient way to do it)
<StevenK> There was a change for that in the deployment report I noticed
<wgrant> Yeah, but different.
<StevenK> Of course it is. :-)
<lifeless> wgrant: https://lp-oops.canonical.com/oops.py/?oopsid=1944DQ278#repeatedstatements should be helped by abels change?
<wgrant> lifeless: I thought that looked specific to Bug.markAsDuplicate.
<lifeless> wgrant: I only read the cover letter
<lifeless> so it may need an equivalent change
<wgrant> lifeless: Except this is somewhat deeper :(
* lifeless changed the topic of #launchpad-dev to: PQM in RC mode | https:/â/âdev.launchpad.net/â | On call reviewer: - | https:/â/âcode.launchpad.net/âlaunchpad-project/â+activereviews
<wgrant> lifeless: How well do you know Storm's codebase?
<wgrant> I am considering digging around to see why inherited References don't have the correct class, while normal properties do.
<wgrant> But the descriptor/metaclass stuff is not entirely clear.
<lifeless> I can generally find what I am looking for
<lifeless> not far enough along to have a sense of how to tackle things tastefully
<wgrant> Ah.
<wgrant> k
<lifeless> so you have a reference on class A
<lifeless> and a concrete class B
<wgrant> Yes.
<lifeless> and the reference cls handle when you access B.reference is A
<wgrant> The Property descriptor handles that nicely.  But Reference does not.
<lifeless> or
<lifeless> is it that you have a reference into the class hierarchy ?
<wgrant> PackageBuild.archive is inherited by BinaryPackageBuild.archive.
<wgrant> 'BinaryPackageBuild.archive == foo' creates a query for PackageBuild instead.
<wgrant> BinaryPackageBuild.archive_id == foo works fine.
<wgrant> Because the descriptor recreates the column.
<wgrant> With the correct class.
<lifeless> now
<lifeless> this is with sqlobject sugar right
<lifeless> ?
<wgrant> No.
<wgrant> Raw Storm.
<lifeless> PropertyPublisherMeta looks interesting
<wgrant> Well. BinaryPackageBuild is SQLBase, but PackageBuild and SourcePackageRecipeBuild are native.
<wgrant> And I tested it with native non-LP Storm.
<lifeless> PropertyRegistry
<wgrant> Mm, it's not that interesting.
<wgrant> I think that's mostly for string property lookups.
<wgrant> Yes.
<wgrant> class PropertyRegistry(object): """ An object which remembers the Storm properties specified on classes, and is able to translate names to these properties. """
<wgrant> So not relevant here.
<wgrant> It's just the descriptors.
<wgrant> lifeless: Compare Property.__get__ and Reference.__get__
<wgrant> Property uses _get_column, which creates a new PropertyColumn based on the right class. Reference just returns self.
<wgrant> (where obj is None, obviously)
<lifeless> yes
<LPCIBot> Project windmill build #227: STILL FAILING in 1 hr 3 min: https://lpci.wedontsleep.org/job/windmill/227/
<wgrant> lifeless: So, BuildFarmJobSet.getBuildsForBuilder
<wgrant> This method is The One.
<lifeless> wgrant: you have a fix for storm ?
<wgrant> Did you envision any particular solution to it?
<wgrant> No, I just worked around it by using _id.
<lifeless> wgrant: what file is that in ?
<lifeless> wgrant: did you file a bug ?
<wgrant> lib/lp/buildmaster/model/buildfarmjob.py
<wgrant> I didn't.
<lifeless> wgrant: please file one
<wgrant> I will. I spent most of Friday afternoon dealing with this so was pretty displeased by the end, so didn't end up filing a bug.
<wgrant> Bug #682989 is probably the same thing.
<wgrant> Or rather similar.
<lifeless> wgrant: so that query
<lifeless> wgrant: one option is to union
<wgrant> That's what I was thinking.
<wgrant> But the sort may be slow.
<wgrant> I was going to half-perform the migration in a transaction on DF and see what happens.
<wgrant> But populating BPB takes forever.
<wgrant> lifeless: Although I guess key-based offsets with a union could be done rather cheaply.
<wgrant> s/rather/very/
<lifeless> yes
<lifeless> the key question is whether sql will sensibly do partial iteration on each input
<lifeless> UNION ALL should let it do this
<lifeless> I'm looking at uses
<lifeless> builder.getBuildRecords is one
<lifeless> its binary_only flag will permit a single-table lookup
<wgrant> I believe that should be the only user.
<lifeless> looks like it
<lifeless> sadly getBuildRecords is used as a name for other things
<wgrant> Yes.
<wgrant> It's on like 5 or 6 other classes.
<lifeless> I don't think its sensible to report back to the start of time on builders
<wgrant> It's not essential, no.
<wgrant> I'd considered adding a threshold if the sort was too slow.
<wgrant> But it may not be.
<lifeless> If we consider dynamic provisioning of builders
<lifeless> it becomes actively undesirable
<lifeless> hmm, no wallyworld today
<wgrant> This week.
<wgrant> Or next.
<wgrant> He's in Budapest.
<lifeless> oh right
<lifeless> anyone know how to land stuff on oops-tools ?
<wgrant> pqm-submit?
<lifeless> grrr
<lifeless> PATCH -> no idea what was changed
<wgrant> Oh?
<lifeless> in oops-tools
 * wgrant vomits a bit.
<wgrant> The old bug subscription is awful :/
<wgrant> Not obvious how to fix without reverting and deconflicting.
<wgrant> The old bug subscription *JS* is awful, that is.
<wgrant>     // Determine if this is a dupe.
<wgrant>     var is_dupe;
<wgrant>     var icon_parent_div = icon_parent.get('parentNode');
<wgrant>     var dupe_id = 'dupe-' + person.get('css_name');
<wgrant>     if (icon_parent_div.get('id') === dupe_id) {
<wgrant>         is_dupe = true;
<StevenK> LPCIBot: I'm also not sure about the recent Jenkins picture change
<wgrant>     } else {
<StevenK> Doh
<wgrant>         is_dupe = false;
<StevenK> lifeless: ^
<wgrant>     }
<lifeless> StevenK: the new logo?
<wgrant> Ignoring the whole if (foo) bar = true; else bar = false; thing, that's still a bit bad.
<StevenK> lifeless: Yah
<lifeless> I like it
<StevenK> The old one was mostly one colour, this is ... too red
<wgrant> StevenK: Agreed.
<StevenK> And French, rather than British. :-P
<wgrant> lifeless: I don't think this needs reversion.
<wgrant> Its only effect is minor.
<wgrant> Everything else is still fine.
<wgrant> And it doesn't revert cleanly.
<wgrant> Although that's fixable...
<lifeless> are you familiar with curtis question work?
<lifeless> the last remaining qa item
<wgrant> untestable, I believe, but let me check.
<StevenK> Oh dear. The last 55 windmill builds on Jenkins have failed.
<wgrant> StevenK: Yes, there is that one test which fails only in Windmill :(
<wgrant> lifeless: It is unused, but there are QA instructions in the bug. I will grab spm.
<jtv> wgrant: thanks for checking out the latest generate-contents-files... did it all work then?
<wgrant> jtv: Yup!
<wgrant> All looks good.
<jtv> What a relief.
<jtv> wgrant: maybe we should just schedule that bugger then
<jtv> ah no, downtime
<jtv> :(
<wgrant> Ja.
<jtv> Hmm that means about the same thing in all 3 Indo-European languages I can think of that have that word, so I get your drift.
<jtv> In Spanish, however, it could be interpreted as you laughing at me which would be less than kind.
<wgrant> :(
<jtv> By the way, wgrant, isn't this a holiday for you?
<jtv> Grrr this edge connection isn't very suitable for actual work.  Lightning seems to have fried my dsl modem because I was too lazy or worried about signal quality to loop the phone line through the UPS.  I'll try once more and if not successful, get a new one.
<wgrant> jtv: No, not a holiday here.
<wgrant> Should it be?
<jtv> Labour Day.
<wgrant> In the UK, maybe.
<wgrant> Ours was in March.
<wgrant> NSW's is in October.
<wgrant> QLD is today, though.
<jtv> Here, certainly.  The reasoning seems to be that there must be a day without labour specifically for the occasion of labour day, and just having the day off because it's Sunday isn't enough.
<StevenK> It's "May Day" in the UK, isn't it?
<jtv> Quite possibly.
<StevenK> Northern Terrority also have a 'May Day' public holiday.
<jtv> May Day is when pilots stop working and just let their planes drop to the ground, right?
<wgrant> NT doesn't exist.
<StevenK> wgrant: Wha?
<jtv> This could be bad news for any inhabitants the NT may have.
<jtv> Meanwhile, how can it be a national holiday for Graham and Julian but not for Gavin?
<wgrant> It has a population of like 200000. It doesn't exist.
<wgrant> jtv: Swap day?
<wgrant> Maybe.
<LPCIBot> Project windmill-devel build #1: FAILURE in 1 hr 4 min: https://lpci.wedontsleep.org/job/windmill-devel/1/
<jtv> wgrant: no, the list I'm looking at would still show a national holiday.
<StevenK> wgrant: By your reckoning, the small towns dotted around Australia don't exist either.
<wgrant> Perhaps he just forgot.
<wgrant> StevenK: They're not a whole semi-state.
<StevenK> It's a terriority!
<wgrant> Right, it's like a state except more pathetic.
<StevenK> We should be trying to emulate them! They have less politicans.
<wgrant> True, true.
<StevenK> And windmill-devel failed, surprise.
<jtv> StevenK: I'm looking into the "synchronize updates" button for the packages diff page.  Do we have code landed to synchronize a package?
<StevenK> The code has existed for ... eons
<spm> Ãons?
<lifeless> night all;
<jtv> StevenK: damn, connection bounced again... where does this code live?
<LPCIBot> Project windmill-db-devel build #228: STILL FAILING in 1 hr 5 min: https://lpci.wedontsleep.org/job/windmill-db-devel/228/
 * jtv traces through the call chain
<StevenK> jtv: I'm not what the button calls, but likely IArchive.syncSources()
<jtv> Ah, I thought that was a recent development.  Thanks.
<stub> What review type is for a release critical? rc or release-critical or what?
<StevenK> release-critical
<StevenK> I think
<stub> https://code.launchpad.net/~stub/launchpad/pending-db-changes/+merge/59640 if the RM is in the house. I'm out of here to enjoy this holiday :)
<StevenK> Hm. We don't have RMs anymore?
<wgrant> lifeless is the perma-RM.
<StevenK> Heh
<adeuring> good morning
<rvba> Hi wgrant. If you get a chance, would you take a look at https://code.launchpad.net/~rvb/launchpad/change-perm-sync/+merge/59341 ? I've modified the packages copier method carefully but since the modification is rather sensitive, I'd be happy if you could take a look at this. You will see that I've solved the problem about passing a None person to the copier method by avoiding this case with code in the view code. Please also take a l
<rvba> ook at adeuring's remark in the MP's comments.
<wgrant> rvba: Looking.
<rvba> wgrant: great, thank you.
<wgrant> 405	+        :param strict_component: True if access to the specific component for
<wgrant> 406	+            the package is needed to upload to it. If False, then access to
<wgrant> 407	+            any package will do.
<wgrant> Access to any *component* will do.
<rvba> wgrant: right ... so I'll have to change the doc in the other method as well (where it came from)
<wgrant> Ah. Indeed.
<wgrant> rvba: Hmm, so anonymous users are prevented from doing everything by line 19 of the diff?
<rvba> wgrant: yes, this protects the call to sync_source.
<wgrant> rvba: That's a bit riskier than I'd like :/
<rvba> wgrant: yeah, I confess I'm not really happy with that either.
<wgrant> rvba: Could you make checkCopy take a check_permissions arg in addition to the person?
<rvba> and set this to False by default?
<wgrant> No, True. But having check_permissions=True and person=None will forbid the copy.
<wgrant> You will have to change the existing callsites to use check_permissions=False, but then at least the dangerous action is explicit.
<wgrant> And this is the very definition of dangerous action.
<rvba> ok, I've done it this way to avoid having to change the call sites ... but your solution is bar better ...
<wgrant> If there is one security hole we don't want in Launchpad, it's probably in this method.
<rvba> all right. I'll do this.
<wgrant> Thanks.
<rvba> wgrant: Thank _you_ ;)
* henninge changed the topic of #launchpad-dev to: PQM in RC mode | https:/â/âdev.launchpad.net/â | On call reviewer: henninge | https:/â/âcode.launchpad.net/âlaunchpad-project/â+activereviews
<MonkZ> hi together, I want to setup a intranet launchpad is there a howto? with public/private projects, account management etc
<henninge> MonkZ1: I am not aware of such a howto. It certainly is not supported "out of the box" but will require some work.
<henninge> MonkZ1: I know that others have done it but I don't know how sucessful they were.
<MonkZ1> henninge: there is no need for "out of the box" (mach ich gern). just need a light guideline
<henninge> MonkZ1: I am not aware of anthing like that, at least not on offcial LP pages.
<MonkZ1> henninge: ok thanks
<MonkZ1> henninge: the "private project"-feature is free available?
<henninge> MonkZ1: AFAIK we have private bugs, private branches and private teams.
<MonkZ1> henninge: ok nice
<henninge> MonkZ1: these are available to commercial subscriptions.
<MonkZ1> :(
<henninge> MonkZ1: https://launchpad.net/+tour/join-launchpad#commercial
<MonkZ1> henninge: so i need to subscribe to get the possibility of a private * in a intranet environment on a selfhosted launchpad?
<henninge> MonkZ1: no
<henninge> MonkZ1: sorry, misunderstood you
<henninge> MonkZ1: you can use your own instance as you like it.
<henninge> MonkZ1: you get the exact same source tree that Launchpad.net is using.
<henninge> MonkZ1: so all these features are available.
<MonkZ1> henninge: aaah k this was the bit i need
<MonkZ1> henninge: thx
<henninge> bitte schÃ¶n ;)
<MonkZ1> henninge: ach dann hÃ¤tten wir auch auch deutsch quatschen kÃ¶nnen ;)
<MonkZ1> bye
<henninge> MonkZ1: no, this is an English language channel ... :-P
<henninge> TschÃ¼Ã
<wgrant> jtv: So ArchiveDependency would have a series at both ends?
<jtv> wgrant: hadn't thought of that--it'd be a bit of a misnomer then wouldn't it?
<jtv> Then again, with component & pocket already in there...
<wgrant> jtv: You propose it has at least one series, though?
<wgrant> And if there is one there must surely be two.
<jtv> Then yes.
<wgrant> Otherwise how would it work?
<wgrant> ArchiveDependency on Ubuntu Natty... for which series?
<jtv> Yes, I see that now.
<wgrant> Great.
<wgrant> Then I mostly agree with your idea of merging them.
<jtv> Phew.  I thought that was going to be a stumbling block.  :)
<wgrant> I haven't run through all the cases yet, but it seems like it should work...
<jtv> wgrant: the operative question remains of course: would it make our lives any better?  There's probably enough flexibility in that schema to cause us a few headaches.
<wgrant> jtv: I'm not sure it adds significant complication.
<wgrant> It mostly makes things simpler.
<wgrant> And a little less inflexible.
<jtv> Very, very relieved to hear that.
<jtv> Apart from the fact that it'd take actual work to implement, of course.
<wgrant> I'm trying to think how the implicit primary archive dependency would work.
<wgrant> I guess it would be implicit per-series.
<jtv> What are the problems with making it explicit?
<wgrant> ie. if there's not an explicit one for that series and the primary archive.
<wgrant> Well, maintaining that automatically is difficult.
<wgrant> Added a new series to Ubuntu? Need to add 20000 new ArchiveDependency rows.
<jtv> That is rather a lot, yes.  What are all those archives?
<wgrant> PPAs.
<wgrant> Lots of PPAs.
<jtv> Part of the idea though is that for the general case, an archive dependency doesn't _have_ to be restricted to a series.
<jtv> That's where the headaches come from.
<wgrant> Right.
<jtv> But it might just help avoid that per-series cost.
<wgrant> My original proposal or yours will both work fine. Yours is probably a bit cleaner. But maybe more expensive.
<wgrant> But...; ArchiveDependency.component and ArchiveDependency.pocket really need to be series-specific anyway.
<wgrant> So we might as well merge AD and DSP and solve everything.
<jtv> henninge: saw the request to open Oneiric translations?  I'm on feature rotation and rather busy... are you on maintenance by any chance?
<wgrant> Where does that go? And what's involved?
<StevenK> wgrant: I'd suggest you reply to jtv's mail with how to merge AD and DSP?
<wgrant> StevenK: I need to think about it.
<wgrant> The trivial solution is sort of obvious.
<wgrant> But it's not clear that it's perfect.
<jtv> wgrant: you're asking about the translations opening?  It's a matter of running a script, much like init-from-parent, and it should be automated similarly.
<StevenK> wgrant: My thought was DSP at *least* needed to grow pocket
<wgrant> StevenK: Yes.
<wgrant> StevenK: Julian disagrees.
<StevenK> Or we create a Series table that maps all this crap
<wgrant> But he's not here today, so stuff him :)
<wgrant> jtv: Right.
<wgrant> jtv: It's fairly cheap nowadays with sharing and stuff?
<jtv> y
<wgrant> jtv: Which list/address was notified?
<StevenK> wgrant: Okay, will you at least agree to replying with "Oh, here's this trivial solution I thought of."
<jtv> wgrant: the former translations team still has a mail alias
<wgrant> StevenK: Add two series fkeys to ArchiveDependency. Adjust code to cope. Done!
<wgrant> (delete DistroSeriesParent too)
<wgrant> (bonus points for making DSD point to an ArchiveDependency)
<wgrant> jtv: Ah. That's a bit sad.
<StevenK> wgrant: Bah, we just added it
<wgrant> StevenK: Yes, and that's why I argued about it.
<wgrant> Because it was premature and ill-thought.
<StevenK> And I wrote a 2,000 line branch to *use it*
<wgrant> Sure. But it's mostly reusable.
<wgrant> Just need to change the class it looks at.
<wgrant> It's the ArchiveDependency code that needs to change significantly.
<wgrant> DSP-using code just has to use the other class.
<jtv> wgrant: I'm surprised to hear we were thinking along such similar lines all the time.
<jtv> (including the "premature and ill-thought" bit)
<wgrant> Heh.
<wgrant> Your combined solution is better than the one I had come up with.
<wgrant> But I knew the existing one was entirely inadequate :)
<wgrant> This is why I like non-trivial discussion before making large changes :)
* benji changed the topic of #launchpad-dev to: PQM in RC mode | https://dev.launchpad.net/ | On call reviewer: henninge, benji | https://code.launchpad.net/launchpad-project/+activereviews
<henninge> jtv: I was pinging you to ask you about that ... ;-)
<jtv> Ah!
<henninge> jtv: do we have it documented somewhere?
<jtv> Errr yes somewhere I think...
<wgrant> How hard is it?
<wgrant> Can NRCP just say to poke a LOSA to do it?
<jtv> wgrant: not hard at all.
<jtv> It used to be a big deal where lots of things could go wrong and bugs would surface with merciless savagery.
<wgrant> Yeah.
<jtv> But nowadays, yes, a LOSA could do it or better yet, a job runner.
<wgrant> And you had to take down LP for 12 hours.
<wgrant> Those were the good old days.
<jtv> 12 hours?  you kid had it easy
<henninge> jtv: also, I am wondering if "Mail rosetta@..." is still a good instruction for the Ubuntu guys.
<jtv> henninge: we were just talking about that--you're right, it isn't.
<henninge> right
<wgrant> I have my cursor on the NRCP edit link :)
<wgrant> Is there a LOSA doc I can link to?
<jtv> It should be a simple matter of running scripts/copy-translations-from-parent.py -d ubuntu -s <new series> and, once it's done, enabling imports for the series.
<wgrant> Nothing under LPHowTo like there is for branch-distro?
<henninge> jtv, wgrant: I can see to it that it happens this time around and wirte up a doc for it.
<jtv> I'm searching the wikis atm
<henninge> (which would mainly include those two instructions)
<henninge> jtv: cool. I have to relocate right now. Please let me know what you find ... ;-)
<jtv> OK
<wgrant> jtv: https://answers.launchpad.net/launchpad/+question/154984 <- is that just a matter of changing the series on +admin?
<jtv> wgrant: the great irony of the translations opening is that the code relies on DistroSeries.parent_series.
<jtv> wgrant: looking...
<wgrant> jtv: Of course :)
<huwshimi> Anyone around that might be able to help me figure out why I can't get Launchpad running locally?
<wgrant> huwshimi: What does it do?
<wgrant> rvba: What's this -updates thing?
<huwshimi> wgrant: I can't make a connection between the browser and the local server. At least that's what I suspect.
<huwshimi> wgrant: launchpad.dev just times out
<rvba> wgrant: the card says exaclty this "Synching to a released series should put packages in -updates" ... but I haven't talked to anyone yet (and Julian is only back tomorrow)
<huwshimi> wgrant: and the server does not log any events
<rvba> wgrant: https://code.launchpad.net/~rvb/launchpad/sync-to-updates/+merge/59653
<wgrant> huwshimi: ping launchpad.dev
<wgrant> Which address is it trying?
<wgrant> rvba: I saw the diff, and I think the idea of hardcoding it like that is silly.
<wgrant> It is probably what Julian wants.
<wgrant> But it is still silly.
<rvba> wgrant: :) I'll be glad to hear about any improvement you would care to suggest.
<huwshimi> wgrant: Ah, it's trying to use my ip address at home
<huwshimi> wgrant: I am a long way from that router :)
<huwshimi> wgrant: I should have noticed that before
<rvba> wgrant: not that the destination pocket was previously hardcoded to "release".
<rvba> s/not/note/
<wgrant> rvba: Right. We need to think how pockets are going to be handled.
<wgrant> rvba: At the moment we handle !Release for source or destination. Hardcoding this is probably not the solution.
<wgrant> Er.
<wgrant> We *don't* handle !Release for source or destination.
<lifeless> StevenK: wgrant: I'm not RM; RM role is dissolved.
<wgrant> lifeless: But you are the effective RM, aren't you?
<lifeless> StevenK: wgrant: rc exceptions can be handed out by reviewers as long as it *is* rc (that is we can't deploy without it)
<wgrant> Oh. I didn't hear that.
<lifeless> wgrant: I don't have any permission bits that I'm aware of that haven't been delegated out to be team responsibilities
<rvba> wgrant: what do you mean exactly "we don't *handle* release for source or destination" ...?
<lifeless> wgrant: I'll be closely involved, this is our single highest risk activity
<jcsackett> rc also means less than it used to, since most things can go out shortly after.
<lifeless> wgrant: did you get spm cycles for qastaging ?
<wgrant> rvba: Different distros will want to do different things with pockets. At the moment we are hackishly hardcoded to ignore -backports and -proposed when generating DSDs. Now we're adding more hacks to map release into -updates when something has released. This is just going to become an ever-increasing stack of hacks.
<lifeless> ugh
<wgrant> lifeless: For which? The questions QA?
<lifeless> how do you turn off 'desktop integration' for launchpadlib
<rvba> wgrant: I see.
<lifeless> wgrant: applying the db patch to qastaging
<wgrant> lifeless: No :/
<wgrant> lifeless: I'll grab $US_LOSA.
<wgrant> When they appear.
<lifeless> cool
<deryck> Morning, all.
<lifeless> so, desktop integration. How does one turn that off?
<deryck> henninge, standup ping
<lifeless> [I don't trust every python script with private access to lp]
<rvba> wgrant: so a possibility would be to abstract the behaviour of things related to pockets (as much as possible ... and especially new things) so that whenever this pocket-thing get cleaned up properly we don't have to modify things everywhere ... correct?
<wgrant> lifeless: Well, you have POSIX to argue with about that..
<benji> lifeless: take a look at https://dev.launchpad.net/LandingChanges#Credential%20storage; there I describe how to force the file-based crendential storage
<lifeless> cool
<wgrant> rvba: Right, the ArchiveDependency rework we discussed earlier could solve this.
<lifeless> I need to hunt down whomever wanted this carte blanche approach and determine their real needs
<lifeless> so we can make it sane
<wgrant> lifeless: Or do you just not trust them to not do things out of stupidity, not malice?
<lifeless> stupidity mainly
<lifeless> I want to know when client X is going to futz with my not-journaled webapp thank-you-very-much
<wgrant> Maybe one day our operating systems won't suck and we can run potentially malicious code.
<lifeless> (its why I tend not to use fat clients for most web services I use)
<wgrant> lifeless: All protests on this front were ignored.
<wgrant> lifeless: Well, dismissed with cries of approximately "DX! DX!"
<wgrant> IIRC
<lifeless> benji: thats file based, but does it restore the previous *one file per app* logic
<lifeless> benji: because thats what I want; I want each /app/ to get oked
<wgrant> lifeless: bwahaha.
<wgrant> bwahahahah.
<wgrant> You'll have to hack the code.
<rvba> wgrant: Yeah ... but the ArchiveDependency rework will not happen tomorrow I'm afraid so we will have to figure out a way to deal with this pocket related work in a way that will make the future work easy (well, not to painful let's say ;)).
<wgrant> And undeprecate the APIs.
<lifeless> benji: think android
<wgrant> rvba: Why does -updates have to be dealt with now?
<lifeless> wgrant: we may have time for oni; the big thing is finding out *who* drove it.
<benji> lifeless: nope, it's still one file for all; the app can force it's own store, but off the top of my head I don't know of a way for the user to do it
<rvba> wgrant: no other reason than this being a task on the kb board with high priority
<lifeless> Every time I spoke to a dev about it, the requirements were coming from some unspecified 'users'.
<benji> hmm, there might be a way, let me look
<lifeless> benji: thanks!
<lifeless> I'm crashing, but if you find something, I should see it in backlog - or you could drop me a pointer in mail if you felt so inclined
<wgrant> rvba: Looks like Priority: Normal to me. But hmm.
<lifeless> night all
<wgrant> Night lifeless.
<rvba> wgrant: you're right ;)
<wgrant> Anyway, we have Julian back tomorrow.
<wgrant> It looks like the hack is done.
<wgrant> So if he wants it it's there.
<rvba> wgrant: sure ... but the hack itself is a one liner (sort of) so I'm very much in favour of making this more clean ... especially since, like you said, this whole pocket stuff will have to be reworked.
<wgrant> rvba: It's not critical that this is fixed now, and we may have a proper solution soon. So I'd be a little averse to landing more hacks.
<rvba> wgrant: I completely understand ... and I agree.
<wgrant> If Ubuntu was complaining about this *now*, sure :)
<wgrant> That would be the Soyuz Wayâ¢.
<wgrant> But that is fortunately not the case right now.
<henninge> adeuring: https://bugs.launchpad.net/ubuntu/+source/wxwidgets2.8/+bug/559822
<_mup_> Bug #559822: editra is provided by both the editra package and python-wxtools and conflicts <apport-package> <i386> <lucid> <verification-done> <editra (Ubuntu):Fix Released> <wxwidgets2.8 (Ubuntu):Fix Released by d.filoni> <editra (Ubuntu Lucid):Fix Released> <wxwidgets2.8 (Ubuntu Lucid):Fix Released> < https://launchpad.net/bugs/559822 >
<jcsackett> wgrant: isn't the Soyuz Way mor of a patent-pending thing? :-P
<rvba> although I'm very much in favour of figuring out a way to do things and preparing for the future at the same time. if that's possible that it :).
<wgrant> jcsackett: Indeed.
<rvba> wgrant: thanks for keeping an eye on my doings in the Soyuz code base though ;)
<wgrant> rvba: Sorry about this, but I want to try to minimise the piles of hacks on top of piles of hacks that are already happening :(
<wgrant> (and have been happening forver)
<rvba> wgrant: no irony here ... I was about to ask you about this
<rvba> wgrant: "Started to work on new task: syncing packages in a released series should put packages in the Updates pocket.  Will discuss with someone knowledgeable about Soyuz. "
<wgrant> Heh
<rvba> wgrant: from this morning call notes ... see ;)
<rvba> anyway, I'll do something else now ... and land the other work we've been talking about ...
 * rvba adds wgrant to *the* list
<rvba> (of the people to whom I owe a beer)
<wgrant> Fortunately in lots of places you won't be able to fulfil that :P
<rvba> I reckon in Irland it's ok.
<StevenK> rvba: wgrant doesn't drink. Perhaps you'll need to settle for warm milk or something.
<wgrant> I do... just not where it's US-illegal :)
<rvba> StevenK: wgrant I'm sure we will find a way to settle this :P
<wgrant> Heh.
<StevenK> wgrant: Actually, there is no restriction in Ireland, as long as you have guardian permission, so if you ask your mother, you should be good. :-P
<benji> henninge: I claimed https://code.launchpad.net/~sinzui/launchpad/question-email-3/+merge/59574 but wanted to be sure you weren't working on it.
<henninge> benji: I am not but thanks for keeping me informed. ;)
<benji> thanks
<sinzui> benji: I was going to beg jcsackett to review that since it contains a 900 line fix to doctest that required a lot fixes to preserve the tests since the infrastructure changed.
<benji> sinzui: oh, ok
<jcsackett> benji: i consider myself begged. i'll take a look at it in a moment.
<jcsackett> er, actually, ^ sinui
<jcsackett> ...i fail at typing.
<sinzui> benji: The real change is very small.
<benji> yeah, it looked like mostly test fallout
<LPCIBot> Project windmill-db-devel build #229: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill-db-devel/229/
<jcsackett> sinzui: looking at your branch now.
<jcsackett> sinzui: approved, with a minor quibble on some classnames (as listed in my comment).
<sinzui> okay thanks
<jcsackett> sinzui: i am very excited to have all of this landed. :-)
<sinzui> jcsackett: testing the the previous branch shows two failures. One I already have a fix for, and one very preplexing...
<sinzui> At least I know I wrote a very good test for qastaging
<adeuring> sinzui: do you see bad side effects if the result of Person.getAdministratedTeams()  would be cached? (too many call of this method are the cause of bug 768443)
<_mup_> Bug #768443: ProductSeries:+index timeout with many releases <timeout> <Launchpad itself:In Progress by adeuring> < https://launchpad.net/bugs/768443 >
<sinzui> well
<adeuring> there are other places where we could cache the result
<sinzui> adeuring: We want to remove that method. It forces the team owner to be a member of the team. I believe there are two call sites last month. One that can easily be factored out and one that looked tricky
<sinzui> adeuring: I think callsites should be using team.adminmembers
<adeuring> sinzui: ah, that would be another option. thanks!
<sinzui> adeuring: mailing lists has one call site. It can switch to team.adminmembers. It has a test that says the owner can configure the list...we can remove that
 * sinzui looks for the other callsite
<adeuring> sinzui: that's in lp.bugs.browser.structuralsubscription
<sinzui> adeuring: sorry. I am wrong. I misread the method
 * sinzui reads the code and sips coffee
<sinzui> adeuring: I am +1 to add caching to getAdministratedTeams. I believe the only oddness we need to think about is when the user in the interaction is was added/removed to team.owner either directly or indirectly. This same user cannot add himself, but he can remove himself. Since the operation to change yourself is separate from where we show that list. I think there is 0% chance of caching oddness today
<adeuring> sinzui: yeah, stuff like is what concerns me too. I'll look a bit closer at thie issue
<sinzui> adeuring: Subscriptions are only place that list your teams. I believe there is a bug reported that it is not possible to see and change the teams you admin. I think the person so fixes that bug will need to clear the cache after the change operation.
<adeuring> right
<adeuring> sinzui: there is alreay a method Person.clearInTeamCache() It can and should clear the new cache too, I think
<sinzui> adeuring: may not. I am not a member of may teams I own
<adeuring> ah, right. so I'll ignore possbile problems with the cache for now -- could not find any real issues..
<sinzui> adeuring: InTeam() is broken because it forces the team.owner to be a member. It should not. So our choices are to add temporarily add the cache clear rule, but we would want to remove it. inTeam() should instead be changed to not include the team owner.
<adeuring> sinzui: maybe. But that's more than I want to change for this bug ;)
<jcsackett> is anyone else having issues with "make run" ? http://pastebin.ubuntu.com/602346/
<jcsackett> ah, looks like memcached is gone post-update.
<abentley> benji: Could you please review https://code.launchpad.net/~abentley/launchpad/nonascii-email/+merge/59671 ?
<benji> abentley: gladly
<benji> abentley: done with https://code.launchpad.net/~abentley/launchpad/nonascii-email/+merge/59671
<abentley> benji: thanks.
<benji> np
 * jcsackett is down one machine.
<sinzui> \o/ NotifcationRecipientSet.remove() has always been broken
 * sinzui prepared an apocalyptic fix.
<dobey> sinzui: is that like cooking heroin from brimstone and hellfire?
<sinzui> dobey: eek
<sinzui> dobey: apocalypse  is a tag meaning code code and tests need to moved to where people can find them. The two were separated for years so it is very difficult to discover the code has always been broken
<jcsackett> so named, if i recall, b/c the people doing the fixing were nicknamed the four horsemen.
<jcsackett> which invites the question, how is your horse, sinzui?
<dobey> sinzui: it was just your choice of words :)
<sinzui> hmm, well that has meta insinuations as well
<sinzui> jcsackett: my wife was muling laundry down the steps this morning and fell. A child had left shoes on the steps. We pondered going to the emergency room
<sinzui> jcsackett: so I ran up stairs where I stepped on a shovel my wife left on the steps, puncturing my foot.
<jcsackett> sinzui: good lord! are you too relatively okay?
<jcsackett> s/you too/you two/
<sinzui> jcsackett: My wife tells me she had no idea I would be running up the steps to help her; she claims left the shovel on the steps as a trap for the children.
<sinzui> She is making my open the jars and I am limping a bit
<jcsackett> sinzui, you frequently take the cake for anti-charmed life.
<jcsackett> and i thought nuking my macbook today was a bad day.
<dobey> heh
<sinzui> jcsackett: I think I can top than. On Easter day I drove my children to their grand parents. I noticed my speedometer did not work. We made jokes about how we should get their in no time at zero speed. I also noted that the the miles we were traveling were not ticking away. Near our destination the car started to chug and shake.  I had run out of fuel, though the gage was at 50%. I marched my well-dressed children to a co
<sinzui> rner by the road to wait for a relatives to find us. ;)
 * jcsackett can no longer tell when sinzui is joking
<sinzui> jcsackett: I think mundane misfortunes are best expressed as humour. I make them bearable.
<jcsackett> sinzui: fair.
<lifeless> morning
<jcsackett> what's the magic to run a javascript test? the usual "bin/test -t pattern" isn't doing it.
<abentley> jcsackett: You open the html file in Firefox.
<jcsackett> abentley: huh. okay.
<jcsackett> abentley: so do we just have some plumbing to open all of those via windmill or something on ec2?
<abentley> jcsackett: yes, there's a particular layer you can run.  I forget its name, but it was announced in launchpad-dev in the last month or so.
<jcsackett> abentley: okay, thanks.
<flacoste> lifeless: https://code.launchpad.net/~flacoste/launchpad/ppr-enhancements/+merge/59700
<flacoste> if you care to review my collected enhancements to the ppr
<flacoste> it's a big branch (844 lines), turned out that the trickiest part was changing the histogram resolution :-)
<sinzui> jcsackett: do you have time to mumble?
<jcsackett> sinzui: sure, just one moment.
<jcsackett> sinzui: i think the problem is on my end.
<jcsackett> i have a number of things failing today.
<sinzui> jcsackett: I just restarted since I saw you speaking but heard nothing
<lifeless> flacoste: hi
<lifeless> voip/skype?
<flacoste> lifeless: voip is good
<lifeless> ext #?
<flacoste> 7356
<lifeless> currently unavailable
<flacoste> lifeless: try again
<jcsackett> sinzui:i have gone through the wizard again and appear to have a functioning microphone.
<jcsackett> sinzui: hm. i'm guessing you still do not hear me?
<jcsackett> ...sound settings have never been right on this machine...
<sinzui> I do not, but I hear the pings from this app and music
<jcsackett> sinzui: yeah, problem is on my end, i'm sure.
<jcsackett> sinzui: give me a few more moments, and if we cannot get it working, just call me?
<sinzui> jcsackett: does sound preferences show the mic working
<sinzui> ...the input tab
<jcsackett> sinzui: it does. i ran through the wizard and had functioning sound. :-/
<jcsackett> er, microphone.
<sinzui> I do not use the wizard. It died when I tried to configure it. I guess it did what it needed to. I use the pulse sound preferences panel that is found in the sound indictor.
 * sinzui sees mumble and can adjust the volume/input of speakers and mic
<sinzui> jcsackett: I see your status changing, but I no longer see an indication that you are speaking.
<sinzui> maybe you can use deafen to send a morse coded message
<lifeless> matsubara: hi; I have an oops-tools patch, how do I land it?
* benji changed the topic of #launchpad-dev to: PQM in RC mode | https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<matsubara> lifeless, you need to set a commit message and set the mp as approved
<sinzui> jcsackett: https://launchpad.net/~mactel-support/+archive/ppa
<sinzui> jcsackett: https://help.ubuntu.com/community/MacBook5-1/Lucid
<sinzui> https://help.ubuntu.com/community/MacBook5-1/Natty
<lifeless> flacoste: http://developers.facebook.com/blog/post/358/
<sinzui> jcsackett: the brightness instruction on the natty page does indeed fix bightness
<jcsackett> sinzui: that is awesome.
#launchpad-dev 2011-05-03
<jtv> morning folks
<lifeless> hahhahahahahaahahahahahahahahaha
<lifeless> ha
<lifeless> wgrant: yo, soyuz question
<lifeless> https://lp-oops.canonical.com/oops.py/?oopsid=1904K1614
<lifeless> queries 10 and 12
<wgrant> lifeless: What about them?
<lifeless> why does the file size count not have the status enum check
<wgrant> lifeless: It uses dateremoved instead.
<wgrant> Which is similar, but not quite the same.
<wgrant> dateremoved lags a bit.
<wgrant> But it represents the actual size on disk.
<lifeless> uhm
<lifeless> would it be incorrect to put the status enum in ?
<wgrant> It depends on your definition of correctness.
<wgrant> It would make it no longer reflect the archive disk usage.
<wgrant> lifeless: Erm, 6to4 doesn't do that.
<wgrant> Maybe you mean NAT64.
<wgrant> Which is in no way widely supported.
<wgrant> And then you probably need DNS64 too.
<wgrant> Pure IPv6 is not a very friendly environment at the moment.
<lifeless> ah
<lifeless> there used to be a 6 -> 4 thingy
<wgrant> Still, World IPv6 Day soon.
<lifeless> 6-7 years ago
<wgrant> Yeah.
<wgrant> That was deprecated.
<wgrant> And replaced by NAT64.
<wgrant> But neither has ever been widely supported.
<wgrant> Doable, but not common.
<lifeless> I feel for the guy
<lifeless> but its totally not a bug in lp's code
<StevenK> I just love how The Plan for IPv6 is "Oh, everyone will move. At some point."
<wgrant> StevenK: What more plan can there be?
<wgrant> People have to stop being hideously lazy.
<wgrant> And just do it.
<spiv> I just like to imagine IPv6 causing DNS servers the world over to scream âAAAA!â
<wgrant> It's not that hard.
<StevenK> spiv: Boo. Hiss.
<StevenK> wgrant: No, it just costs time and lots of money.
<wgrant> StevenK: And they've had more than a decade to do it.
<StevenK> And telecommunication companies prefer to *recieve* lots of money and spend very little.
<lifeless> wgrant: thats -barely- 2 equipment refreshes of core switch gear
<wgrant> lifeless: Sure. But most companies are not even thinking about it yet.
<lifeless> wgrant: and its not been /stable/ for all that long
<lifeless> anyhow
<lifeless> archive:+repository-size
<lifeless> not ip6
<lifeless> kgo
<wgrant> At least aarnet's tunnel broker is reasonable.
<wgrant> lifeless: What about it?
<lifeless> wgrant: thats the oops I'm looking at
<lifeless> https://bugs.launchpad.net/launchpad/+bug/739070
<_mup_> Bug #739070: Archive:+repository-size timeout <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/739070 >
<lifeless> 9+ seconds to query file sizes
<wgrant> Yes.
<wgrant> lifeless: Would the status filter improve it significantly?
<lifeless> I'm not 100% sure
<lifeless> exploring the space atm
<lifeless> is it 2 for binaries as well?
<wgrant> Yes.
<wgrant> PackagePublishingStatus is universal.
<wgrant> ~/launchpad/lp-branches/bfj-different-urls$ bzr push
<wgrant> Using saved push location: lp:~wgrant/launchpad/bfj-different-urls
<wgrant> bzr: ERROR: Server sent an unexpected error: ('error', '<ProtocolError for xmlrpc.lp.internal:8097/codehosting: 502 Bad Gateway>')
<wgrant> I don't think that's meant to happen.
<lifeless> wgrant: would 2+3 be equivalent ?
<wgrant> lifeless: No.
<wgrant> lifeless: What's 3? Superseded?
<wgrant> (it's working this time)
<lifeless> wgrant: yes
<lifeless> 4 is deleted
<lifeless> 1 is pending
<wgrant> 5 Obsolete.
<wgrant> Right.
<wgrant> I never remember the 3/4 order.
<wgrant> So, 3/4/5 are all potentially removed.
<wgrant> In the beginning there was Pending/Published/PendingRemoval/Removed.
<lifeless> anyhow
<wgrant> Then ArchiveRemovalRedesign happend, and the removedness was moved into dateremoved instead.
<lifeless> with (2) its a 50ms query hot
<wgrant> With status indicating the reason for removal.
<wgrant> Hmm.
<wgrant> With dateremoved how is it hot?
<lifeless> 73ms
<lifeless> :P
<lifeless> look at the plan
<lifeless>     "securebinarypackagepublishinghistory__archive__status__idx" btree (archive, status)
<lifeless> is the index it is using
<lifeless> so it reads every row ever in that archive
<lifeless> wgrant: ^
<lifeless> wgrant: ping
<lifeless> create index bpph__dateremoved__for__size__idx on binarypackagepublishinghistory using btree (dateremoved) WHERE dateremoved IS NULL;
<lifeless> can you create that on dogfood
<lifeless> and get me an explain analyze for the query in https://bugs.launchpad.net/launchpad/+bug/739070/comments/1 ?
<_mup_> Bug #739070: Archive:+repository-size timeout <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/739070 >
<wgrant> lifeless: Ah, didn't see the plans in the bug.
<wgrant> Let's see.
<wgrant> lifeless: The index is building.
<lifeless> I suspect we'll want (archive) where dateremoved is null
<lifeless> or
<lifeless> we should make dateremoved go away
<lifeless> either folding it into the status enums
<lifeless> or dropping the librarian reference or whatever
<wgrant> lifeless:                      ->  Index Scan using temp_bpph_archive_dateremoved_idx on binarypackagepublishinghistory  (cost=0.00..1112.40 rows=391 width=4) (actual time=56.562..107.930 rows=915 loops=1)
<wgrant>                            Index Cond: ((archive = 14516) AND (dateremoved IS NULL))
<StevenK> So the bpph scan is cheap with an index
<wgrant> 34ms hot.
<jtv> Is that dateremoved check a status check in disguise?
<wgrant> 10s cold, but DF.
<wgrant> jtv: Yes.
<StevenK> lifeless: There is a stay of execution for [bs]pph, we'd need to think very carefully about it.
<lifeless> StevenK: We don't need to remove it
<lifeless> StevenK: but we do need to model it. Modelling it as a separate field may be worse than not.
<lifeless> wgrant: the query was 10s cold ? did you get the full explain analyze?
<StevenK> We could add another DBItem to PackagePublishingStatus -- PURGED or so
<wgrant> lifeless: I had it, but then I made the mistake of pressing PgUp, which DF does not enjoy at all.
<wgrant> StevenK: Doesn't quite work.
<wgrant> StevenK: PPS has three separate end states.
<wgrant> (whether this is sane or not is in question)
<wgrant> lifeless: But cold data from DF is entirely useless.
<wgrant> lifeless: The numbers, at least.
<wgrant> The plan is OK.
<lifeless> wgrant: can I see?
<lifeless> wgrant: cold data still tells me where time is going, and thats useful.
<lifeless> wgrant: so I dispute entirely useless
<lifeless> wgrant: oh, I see, lost.
<lifeless> kk
<wgrant> Yeah.
<wgrant> Get one from qastaging, I guess.
<lifeless> select count(*) from branchrevision; then try again ?
<wgrant> It doesn't tell you where the time goes, really.
<wgrant> It tells you that DF has like 4MB of RAM.
<StevenK> 4MB. 'lol'
<lifeless> 640k surely
<lifeless> ugh, timeline only works if you run upstream builds of chromium? wtf
<wgrant> lifeless: Hm?
<wgrant> Works for me.
<lifeless> "Pfffttt, Speed Tracer is not working.
<lifeless> Please double check a couple of things:
<lifeless> You must start Chrome with the flag: --enable-extension-timeline-api
<lifeless> You must be running the Chrome Dev channel.
<lifeless> For more details, see our getting started docs."
<wgrant> Oh. Not an upstream build.
<lifeless> I have enable-extension-timeline-api in CHROMIUM_FLAGS in /etc/chromium-browser/default
<wgrant> Just not a stable build.
<lifeless> webdevs are stable users too
<StevenK> And error message that includes "Pfffttt," Kwality
<wgrant> eg. ppa:chromium-daily/beta or so.
<lifeless> I question the sanity of this
<wgrant> lifeless: *That* makes you question the sanity of Chromium?"
<StevenK> I have yet to see any *compelling* reason to switch to Chromium, TBH.
<wgrant> StevenK: I use Chromium.
<wgrant> It's not bad.
<StevenK> However, I question Google's motives.
<wgrant> Firefox 4 is OK, except not with fglrx.
<lifeless> wgrant: so can has plan?
<wgrant> lifeless: The hot one?
<lifeless> the cold one after querying branchrevision a lot
<wgrant> It's been running that select for 8 minutes, still hot as ever.
<lifeless> argh hhaa
<lifeless> ok
<wgrant> Yes.
<lifeless> hot will be fine
<wgrant> http://paste.ubuntu.com/602605/
<micahg> funny, 3.6 was broke with nvidia, 4.0 is broke with fglrx
<wgrant> Our test suite is slow.
<lifeless> wgrant: https://bugs.launchpad.net/launchpad/+bug/739070/comments/4
<_mup_> Bug #739070: Archive:+repository-size timeout <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/739070 >
<wgrant> lifeless: 19ms
<lifeless> this is 1/2 the estimated cost
<lifeless> which is daft but there you go
<wgrant> http://paste.ubuntu.com/602606/
<lifeless> why are we bringing back the rows ?
<wgrant> Which rows?
<lifeless> these rows
<lifeless> why aren't we summing
<wgrant> NFI
<wgrant> But it's only a single column.
<wgrant> So it's not that bad.
<wgrant> (yes, it should be fixed)
<wgrant> Oh. This query is not a single column.
<wgrant> Ahh, this is query 11, not query 10.
<lifeless> wgrant: https://bugs.launchpad.net/launchpad/+bug/739070/comments/5
<_mup_> Bug #739070: Archive:+repository-size timeout <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/739070 >
<lifeless> wgrant: actually I'm in https://lp-oops.canonical.com/oops.py/?oopsid=1948BB197
<lifeless> query 21
<lifeless> 8929ms	SQL-launchpad-main-master	
<lifeless> SELECT DISTINCT LibraryFileContent ...
<wgrant> lifeless: Still 19ms.
<lifeless> spm: can you run https://bugs.launchpad.net/launchpad/+bug/739070/comments/5 on the readonlymode replica - *one* run, I need the explain analyze (looking for cold cache effects)
<_mup_> Bug #739070: Archive:+repository-size timeout <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/739070 >
<spm> oki
<lifeless> why does packagefile exist?
<lifeless> https://bugs.launchpad.net/launchpad/+bug/739070/comments/6
<_mup_> Bug #739070: Archive:+repository-size timeout <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/739070 >
<lifeless> wgrant: ^
<lifeless> StevenK: ^
<wgrant> lifeless: packagefile? You mean BinaryPackageFile?
<lifeless> yes
<StevenK> There are two
<lifeless> and I assume sourcepackagefile
<wgrant> SourcePackageReleaseFile is essential.
<StevenK> It's SourcePackageReleaseFile
<StevenK> (Usually called SPRF)
<wgrant> BinaryPackageFile is for potential expansion to multi-file binaries.
<wgrant> (yes, seriously, one has Release and one does not)
<lifeless> ok
<lifeless> I can see the expansion factor
<lifeless> but
<lifeless> if we makde bpph be an owner of LFC (wrapping the LFA fields in it, and glued into the gc process) we'd save 2.2 seconds of overhead.
<wgrant> aaaaaaaaaaaaaaa
<wgrant> aaaaaaaaaaaaaaaaaa
<wgrant> So, an SPR can have any number of SPRFs.
<wgrant> And there are often lots of BPPHs for a BPR.
<lifeless> sure
<lifeless> but the whole point of LFA is to reference-count-own LFC
<lifeless> ignore SPRF for now, I see why the 1:M is needed
<lifeless> we could roll up some stuff there too, but separate problem.
<wgrant> No point solving BPPH without SPPH.
<lifeless> wgrant: 3:1 bpph -> spph rows
<lifeless> or more
<wgrant> A constant factor of three is not very compelling.
<lifeless> wgrant: solving a problem is solving a problem
<lifeless> so the question is, can we solve
<lifeless> or
<lifeless> do we need to precalculate
 * StevenK murders gina.txt
<lifeless> stub: hi
<jtv> Speaking of stub, where did his wonderful "prejoin" decorator go?
<lifeless> jtv: deleted
<jtv> in favour of what?
<lifeless> DecoratedResultSet
<lifeless> stub: is 2.8 seconds 'about right' for reading in 1000 cold rows from libraryfilecontent?
<jtv> With cold rows, how would you know?
<stub> yo
<lifeless> jtv: https://bugs.launchpad.net/launchpad/+bug/739070/comments/11
<_mup_> Bug #739070: Archive:+repository-size timeout <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/739070 >
<lifeless> jtv: science :>
<stub> lifeless: I don't think is very good
<jtv> It's still considerably less than you'd expect from a full seek per row.
<lifeless> stub: https://bugs.launchpad.net/launchpad/+bug/739070/comments/6
<_mup_> Bug #739070: Archive:+repository-size timeout <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/739070 >
<jtv> lifeless: no I don't mean how do you know it's 2.8 seconds for 1000 rows, but how would you know whether it's "about right."
<lifeless> jtv: we're seeing 2.5ms per row, thats plausible with many redundant spindles
<lifeless> jtv: I would ask someone familiar with our db's guts :>
<jtv> Regardless, it's all a question of access patterns isn't it?  It could probably be 5x worse, it could probably be 500x better.
<lifeless> jtv: not really
<lifeless> jtv: if this is slower than we'd anticipate given our hardware and prior experience, then we can go digging for issues like table/index bloat, contention whatever
<lifeless> jtv: if this is approx as fast as we'd expect on an average day, then we're wasting time if we do that and we should instead look at how to either avoid it being cold or avoid accessing the rows
<StevenK> wgrant: Want to hear something awesome?
<StevenK> wgrant: gina.txt has been running for 20 *minutes* locally.
<stub> So that is 695 index lookups and row lookups, so pulling a minimum of 1400 blocks off the disk random access
<jtv> lifeless: what I'm trying to say is that cold random reads have so many unknowns that this sample size seems inadequate.
<wgrant> StevenK: It normally only takes 60 seconds. I think you have a bug.
<StevenK> wgrant: Or I've broken it into pieces.
<StevenK> And since it's a doctest, I have to printf-debug
<lifeless> StevenK: how do you go from 695 -> 1400 blocks ?
<lifeless> bah
<lifeless> stub: ^
<lifeless> stub: heuristic of 2:1 (index + table) ?
<jtv> (It would be pretty nice if postgres could at least profile distribution of I/O latencies)
<lifeless> jtv: that would be awesome
<lifeless> jtv: I agree that there are many unknowns
<stub> Read a bucket in the index == pull one block from disk. Read that row from disk == pull one block from disk.
<jtv> (The stupid thing is, we've been thinking of I/O costs in terms of FS cache misses, which is impossible to predict and may soon be an outdated concept anyway.  If we could just look at it from an empirical standpoint, we might win something.)
<lifeless> its about 150 bytes per row min, so what 25 per page
<lifeless> on a 14M row set we're very likely to get one page per row
<stub> So 700*2 == 1400, == the minimum if we somehow magically only load up wanted information from the index
<lifeless> stub: kk, thats cool, i have the same basic model
<lifeless> stub: was just checking
<stub> Ok. I was assuming the size of the table would make it unlikely to get multiple rows in a page.
<lifeless> yeah, an assumption I agree with
<lifeless> the BPPH table is very spread over time
<lifeless> jtv: thats not my model of IO costs
<jtv> Not "we" personally; I mean the world.
<stub> The index should be much more tightly packed too.
<lifeless> jtv: ah; so I don't think the world thinks of IO that way per se ;) - see for instance the scale free algorithms work
<lifeless> jtv: which models IO as tiers of progressively bigger and slower cache
<jtv> Well okay, the database world.
<jtv> And that model is still wrong, is my point.
<stub> lifeless: Oh... and the toast lookups for the md5 and sha1 hashes which are stored as text, so potentially another 4 lookups
<lifeless> jtv: they seem to have /very/ good results doing this - noting that they start with the CPU L1 cache and work out
<lifeless> stub: even if we're not looking at the columns (as my rephrased cheaper-estimate query doesn't)
<jtv> Yes, but that's cache-oblivious algorithms isn't it?  I'm talking database.
<lifeless> jtv: it is, but database will get those algorithms eventually I suspect
<stub> lifeless: I think that will avoid loading the data from the toast tables, yes.
<lifeless> that or mlock() the entire table
<jtv> (And I'm not talking cache-oblivious database either: we can worry about that once that world finally catches on to space-filling curves)
<lifeless> jtv: I think its percona that have a space-filling curve index
<jtv> lifeless: There have long been cache-oblivious algorithms for databases.  I worked on a cache-oblivious join algorithm 2 jobs ago.
<lifeless> jtv: very similar to bzr's index /packing logic in fact
<lifeless> jtv: \o/
<lifeless> stub: so what I'm trying to assess, is there some way to get this down to < 2 seconds reliably (we have to do a parallel query for the source package data)
<jtv> But when it comes to buffer management and individual query optimization, my point is is we can't afford to keep thinking "what if this misses in FS cache and the seek is long" because it just gets too complex: SSD, on-disk cache, bad blocks, network latency.
<lifeless> stub: or should we be maintaining the archive size as a cache
<lifeless> stub: its currently 9 seconds with the original query, 6.6 seconds in my rephrased one
<jtv> StevenK, wgrant: should the "synchronize the simple updates" button on +localpackagediffs update blacklisted differences as well?
<lifeless> stub: and we're seeing ~ 7 of these a day where its stuttering (presumably on IO)
<wgrant> jtv: No.
<stub> lifeless: So your theory is this query will often be cold due to access patterns so the lfc lookups will sometimes be too slow? I'm trying to think if we have other pages accessing more than a few dozen lfc rows.
<stub> More of the ones I can think of are dealing with bulk packaging data and are slow too
<lifeless> stub: bug attachments show their size in the comment, for some bizarre reason
<lifeless> very busy bug pages could access hundreds; *but* we will paginate those eventually
<lifeless> so not relevant here
<wgrant> lifeless: That's also the only non-packaging one I can think of.
<stub> lifeless: Bug attachments are fine as it would be rare for a page to have more than a dozen attachments.
<wgrant> Some have hundreds.
<jtv> wgrant: I'll take your word for it.
<lifeless> stub: its rare, but I can link you to the bugs :>
<wgrant> jtv: That is sort of the point of blacklisting.
<lifeless> stub: I don't think its *often* cold, but I think its cold *often enough*
<lifeless> specifically I don't think we do this query on this dataset anywhere else except *perhaps* on package upload (for quota checking)
<jtv> wgrant: well, unless maybe we have automated syncing and then users start blacklisting dsds they feel will behave correctly without human attention.
<wgrant> jtv: Then we've redefined the term.
<jtv> wgrant: so it sort of requires a conscious decision that blacklisting means "I don't want this change" and not "I don't want to see this change."
<stub> yeah. so caching the archive size is fine for this query. I'm wondering if there is something more generic we can do to solve slowness in similar pages. The packaging tables always seem to be an issue - large, and the users want complex reports.
<wgrant> jtv: At the moment the Ubuntu sync blacklist blacklists the sync.
<lifeless> stub: so in this particular case; there is a missing index that would drop the rows examined by 6000 in the packaging table
<lifeless> stub: I don't know if its worth adding
<lifeless> I don't think it will solve the problem
<StevenK> jtv: It effectively means "I don't care"
<jtv> wgrant: fine by me; the "I'll take your word for it" still stands but in light of yesterday's conversation I thought I'd at least ask.  :)
<wgrant> Heh.
<wgrant> StevenK: ^^
<jtv> StevenK: that seems to contradict wgrant's take.
<wgrant> Do you agree?
<wgrant> Oh, he's here.
<jtv> Alas, yes.
<jtv> <wink>
<wgrant> StevenK: You mean "I don't care what the parent thinks"?
<lifeless> stub: ok, I'll update this bug to say 'needs cached figures'
<StevenK> More correctly: "I don't care about this difference"
<jtv> wgrant is showing his age
<jtv> "I don't care what my parents think"
<StevenK> "I'll hack on Launchpad even if my father thinks that Web 2.0 is wrong! I'll show them!"
<jtv> StevenK: humans may not think they care, but it'd be nice to have a clear notion of whether they have a right to be disappointed if the blacklisted difference stops the package from updating.
<StevenK> jtv: Yes, a blacklisted difference stops a sync
<jtv> Of course the button I'm building will sort of eliminate an excuse for blacklisting packages that should be updated.
 * jtv updates his tests.
<StevenK> Keep in mind there are two blacklist states
<stub> lifeless: So that query is broken anyway, because the distinct on is lacking an order by so it isn't doing what the author thought it is doing
<lifeless> stub: my one ?
<stub> the one in the bug comment you linked me to
<lifeless> stub: my rephrased one is the one with the subselect
<lifeless> stub: it doesn't need the order by
<StevenK> Ran 1 tests with 1 failures and 0 errors in 35 minutes 35.459 seconds.
<lifeless> stub: look at the plan: it orders to do the distinct and unique automatically
<StevenK> WHEEE!
<stub> lifeless: That is fine if you happen to get the plan you are expecting.
<jtv> StevenK: not too worried about the two blacklist states; I'm whitelisting, not blacklisting statuses.  Are we confused yet?  :)
<stub> lifeless: So we really need the order by, esp if we want this to survive future PG upgrades.
<stub> (and with luck, it will be a noop with the expected plan)
<lifeless> stub: I'd like to learn more about this
<lifeless> stub: which docs should I read?
<StevenK> jtv: Don't make me kick you.
<stub> lifeless: This is documented in the DISTINCT ON section of the SELECT page in the SQL Reference guide for PostgreSQL
<stub> Sorry - 'DISTINCT clause' section
<stub> file:///usr/share/doc/postgresql-doc-8.4/html/sql-select.html
<lifeless> stub: so, I read that
<lifeless> stub: and it doesn't conflict with what I've done
<lifeless> stub: because - 'Note that the "first row" of each set is unpredictable unless ORDER BY is used to ensure that the desired row appears first'
<lifeless> thats fine, we're selecting identical rows, so whichever row comes first is ok
<lifeless> stub: anyhow, I can add a sort in
<lifeless> stub: but we don't use that crafted query /anyway/ because I was seeing if we could make it fast without caching etc
<stub> lifeless: nm. You are correct I think.
<stub> I wonder if it changed, or if I'm thinking of a similar statement? I'm sure there was one that if you neglected the ORDER BY you can get duplicate results
<stub> lifeless: So in this case you can use the SQL standard "SELECT DISTINCT libraryfile, filesize" rather than SELECT DISTINCT ON (libraryfile) filesize, but the cost saving is statistically insignificant.
<lifeless> stub: interesting; we don't need libraryfile at all anyhow, just the summed sizes
<stub> lifeless: Should we be using rabbitmq instead of triggers to maintain BugSummary btw?
<lifeless> stub: it would remove all the races
<lifeless> stub: I have a branch for rabbitmq but ec2 threw it out
<lifeless> stub: I don't know why yet and haven't gotten back to it
<stub> (now that I think the DB patch is done, and just waiting on tests to confirm this hypothesis :) )
<lifeless> :>
<lifeless> stub: thanks!
<lifeless> I've written up my theory in https://bugs.launchpad.net/launchpad/+bug/739070
<_mup_> Bug #739070: Archive:+repository-size timeout retrieving many hundreds of package sizes <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/739070 >
<lifeless> just the top of the bug summary
<stub> lifeless: I needed to change the structure a bit - we might be race free now with the simpler triggers that I thought were needed to catch some potential modifications
<lifeless> bug_fti || 76% || 612 MB of 798 MB
<StevenK> wgrant: I fixed gina!
<lifeless> stub: that would be cool, but the case of 'person subscribes and the bug is made public' - how will that be race free?
<wgrant> StevenK: Oh?
<StevenK> wgrant: Well, the test now takes 65 seconds, rather than 2135.
<wgrant> That's more normal.
<wgrant> What did you break?
<StevenK> wgrant: gina needed access to DSP
<wgrant> Ah.
<StevenK> Since she creates DSDJs
<wgrant> Yes.
<wgrant> Anyone want to review https://code.launchpad.net/~wgrant/launchpad/bfj-different-urls/+merge/59730? Fairly long, but mostly just sed applied to test.
<wgrant> +s
<jtv> wgrant: I'll take it
<lifeless> stub: have we got enough bloat-changing data to point at (or exclude) bug heat yet ?
<lifeless> stub: or equally, backups
<stub> lifeless: yer... I think the race is still there.
<lifeless> stub: I thought of a way to make it detect races
<stub> lifeless: No, I haven't got the data to confirm the cause.
<lifeless> stub: but you'll either cry or aim a nuke at me.
<lifeless> stub: and I don't want to cause either event.
<stub> We can serialise it by using an advisory lock.
<lifeless> stub: the way would be in any transaction affecting a bug, update a row in a common table, to for serialization detection
<lifeless> a lock sounds better :>
<stub> And advisory lock will do the same thing, and be explicit and faster. yes.
<StevenK> bzr di can't handle multiple revspecs? :-)
<StevenK> :-( rather
<lifeless> StevenK: -r x..y
<StevenK> lifeless: I'd like to skip one in the middle
<lifeless> you may want difftastic
<LPCIBot> Project windmill-devel build #2: STILL FAILING in 1 hr 5 min: https://lpci.wedontsleep.org/job/windmill-devel/2/
<StevenK> lifeless: Google doesn't show anything relevant for that?
<lifeless> aarons plugin for lp merge proposals to show incremental diffs
<wgrant> difftacular?
<StevenK> Perhaps I'll just use filterdiff
<lifeless> whata the rev you want to skip? a merge?
<StevenK> Yes
<lifeless> interdiff / difftcular is what yo uneed then
<StevenK> I can generate the two diffs, interdiff shows the diff between them, how does that help?
<StevenK> Oooh, I see. combinediff
<jtv> wgrant: done
<lifeless> bbiab
<jtv> wgrant: in case I didn't get through earlier, your review is done.
<jtv> Anyone care for a simple, short cleanup review?  https://code.launchpad.net/~jtv/launchpad/db-pre-747546/+merge/59737
<StevenK> Bah, I was about to answer him, too
<StevenK> jtv: r=me
<jtv> thanks
<lifeless> jml: hi
<lifeless> jml: sorry I was afk; was fooding
<jtv> StevenK: and fwiw you're absolutely right about moving it.  One battle at time though.
<StevenK> jtv: Sure, which is why I just approved it
<jtv> We're in loud agreement.
<StevenK> jtv: You probably could have targetted devel, too
<jtv> Yes, but I already have these changes in another branch and I want to had them off at the diff.
<StevenK> jtv: Just saying, is all
<jtv> That is understood.
<wgrant> jtv: Thanks. It changes the URLs used by the API, but not the API itself.
<jtv> Very well.
<adeuring> good morning
<rvba> wgrant: Hi! I modified the call sites of Archive.syncSource like you suggested ... you might want to have a look since the modification is kind of sensitive (https://code.launchpad.net/~rvb/launchpad/change-perm-sync/+merge/59341).
<wgrant> rvba: I think CannotCopy exceptions are exposed in the UI, so you probably want to check that their values will be sensible.
<rvba> wgrant: good point.
<wgrant> Otherwise that looks great.
<bigjools> morning all
<wgrant> Morning bigjools.
<rvba> bigjools: Morning!
<bigjools> what's broken? :)
<wgrant> You survived your vacation without PSN?
<bigjools> yeah, I got a  copy of Portal 2 :)
<wgrant> Ah, 'tis an excellent game.
<bigjools> I like Valve stuff anyway
<wgrant> although dissapointingly short.
 * bigjools sees size of inbox
<StevenK> bigjools: And you run screaming?
<StevenK> A GLaDOS reference somewhere in Soyuz would be apt.
<bigjools> I am gently rocking backwards and forwards while hugging a small furry toy
<bigjools> StevenK: ha
<mrevell> Good morning!
<lifeless> jml: ready when you are
<jtv> hi mrevell
<jml> lifeless: hi
<lifeless> hi
<jml> lifeless: sorry, I got distracted by my RSS feeds.
<lifeless> skype/voip either is good
<jml> lifeless: ok. what's your voip number?
<lifeless> jml: thats ok, its only getting vs is late :)
<lifeless> jml: actually, skype is better because I don't need a headset ;)
<lifeless> ekigas echo detection is still poor
<jml> lifeless: ok. skype.
<StevenK> allenap: http://pastebin.ubuntu.com/602641/ ; thanks!
<allenap> StevenK: Cheers.
<henninge> wgrant: Hi!
* gmb changed the topic of #launchpad-dev to: PQM in RC mode | https://dev.launchpad.net/ | On call reviewer: gmb | https://code.launchpad.net/launchpad-project/+activereviews
<wgrant> henninge: Hi.
<henninge> wgrant: Do you happen to know if bug 776160 is  a duplicate?
<_mup_> Bug #776160: mail headers inconsistent between FAILEDTOUPLOAD and FAILEDTOBUILD <Launchpad itself:Triaged> < https://launchpad.net/bugs/776160 >
<wgrant> henninge: I don't think it is.
<henninge> wgrant: thanks
<henninge> I have a problem writing a view test for bug 644872.
<_mup_> Bug #644872: UnicodeEncodeError in search_text field <faqs> <lp-answers> <oops> <Launchpad itself:In Progress by henninge> < https://launchpad.net/bugs/644872 >
<henninge> I can trigger it in the UI by searching "portuguÃªs" here: https://answers.launchpad.net/ubuntu/
<bigjools> wgrant, rvba: regarding the sync package permission change, did you think about how it'll fit in when we start doing it asynchronously?
<wgrant> bigjools: Do we have a plan for that yet?
<rvba> bigjools: I'm afraid not. But how is this particular change (the permission change) related to that exactly?
<bigjools> wgrant: sorta.  The job is done but not used.  allenap is working on changing the job schema so we can query across archive/series and display something to the user to show pending jobs.  Then we'll switch over the syncing first, maybe PPA copies in the future.
<bigjools> rvba: the check will happen in the job, not the UI.
<bigjools> it would be nice if it happened in the UI but I think it'll bust the query time
<wgrant> bigjools: This seems like it's only useful for mass copies.
<rvba> the permission check is now done in the copyChecker along with the various other checks
<bigjools> wgrant: correct.  We might decide an arbitrary number of packages, over which we use the job rather than synchronously.  The next step is to design the UI.
<wgrant> bigjools: Great.
<wgrant> bigjools: As long as you're not trying to do async for normal copies.
<wgrant> When immediate feedback is helpful.
<bigjools> wgrant: define "normal"
<bigjools> not trying to do async for *anything* yet :)
<wgrant> bigjools: PPA copies, mostly.
<wgrant> Not let's-sync-20%-of-the-archive.
<bigjools> not initially but I have PPAs in mind
<bigjools> since some PPA copying times out still
<wgrant> It's very fixable.
<wgrant> Most of the time in most cases is now spent in the UI.
<wgrant> Determining the list of sources.
<bigjools> ok
<wgrant> But there are some very expensive build queries when there is a potential source conflict in the destination (which is rare)
<wgrant> And delayed copy creation is slow :(
<bigjools> wgrant: what do you think is a good number?  50? 100?  (total source+binaries)
<wgrant> bigjools: NFI
<wgrant> We may need science.
 * bigjools licks finger and holds to the wind
<wgrant> henninge: Wishlist doesn't exist. We are using Low instead.
<wgrant> (just hope Rob doesn't notice you doing that ;))
<henninge> wgrant: thanks ;)
<henninge> wgrant: I'll fix the documentation
<wgrant> I thought it already was fixed :(
<henninge> We map these buckets into:
<henninge> critical : generally empty, bugs that need to jump the queue go here.
<henninge> high: bugs that are likely to get attention within 6 months
<henninge> low [or perhaps wishlist]: All other bugs.
<henninge> that sounds old
<henninge> is the critical definition current?
<wgrant> Yes.
<wgrant> But ZOP means it's not empty.
<wgrant> Yet.
<henninge> Ah, I see.
<wgrant> We have 5 years of debt to repay.
<henninge> ok, doc changed.
<henninge> low: All other bugs.
<henninge> We don't use wishlist.
<wgrant> Thanks.
<bigjools> rvba: regarding the permission changes to syncing, I am concerned that users with zero permissions can cause a load on the job runner
<bigjools> and in the webapp for direct syncs, for that matter
<rvba> bigjools: good point, but is there a "small" permission that can be checked before doing the sync?
<bigjools> rvba: I'm trying to work that out now :)  Ideally, we need a "has any permission at all?" check
<wgrant> If it's too expensive to check all permissions at the time of the request, you could just check for any ArchivePermission on the Archive.
<bigjools> right
<bigjools> in fact the upload processor does this somewhere
 * bigjools digs
<wgrant> If you use the upload processor as a model for anything I will cry.
 * bigjools slaps wgrant
<wgrant> But yes.
<wgrant> It's struct_component on checkUpload.
<wgrant> s/struct/strict/
<wgrant> But it's not quite right.
<wgrant> Oh, no, there's that other thing too.
<wgrant> Argh this is such a mess.
<bigjools> yes
<bigjools> blame Ubuntu policies
<wgrant> Yeah, see Archive.verifyUpload
<wgrant>         if not self.getComponentsForUploader(person):
<wgrant> But it's buggy because it only considers components.
<bigjools> le sigh
<wgrant> A bug exists for that, I believe.
<wgrant> But the any-permission-at-all check is pretty simple to query for.
<bigjools> yep, we can just join TP to AP
<wgrant> Exactly.
<wgrant> Really cheap.
<bigjools> rvba, ok?
<rvba> bigjools: sounds ok. Just tell me what TP and AP are
<rvba> ArchivePerm
<wgrant> TeamParticipation
<wgrant> It is a flattened version of TeamMembership.
<rvba> ok
<wgrant> So it contains indirect memberships too.
<bigjools> make life so much easier when querying
<wgrant> And faster.
<wgrant> And less recursive.
<wgrant> And less reliable...
<bigjools> so if someone has any AP at all, show the button
<bigjools> otherwise, nada
<rvba> all right. I'm on it.
<rvba> wgrant: why "less reliable"?
<wgrant> There are occasional bugs in its maintenance.
<wgrant> So it gets inconsistent with TM.
<rvba> I suppose it gets periodically whipped out and recalculated then.
<LPCIBot> Project windmill-db-devel build #230: STILL FAILING in 1 hr 4 min: https://lpci.wedontsleep.org/job/windmill-db-devel/230/
<jtv> allenap: are you all caught up?
<allenap> jtv: In what? :)
<jtv> allenap: ah
<allenap> jtv: I assume you want to talk about the sync button?
<jtv> Yes!
<allenap> jtv: IRC, Mumble, Skype?
<jtv> I've been asked to make a change to the indexes initialization code, which means dropping my ongoing work on the sync button.
<jtv> IRC would be convenient for me right now (headset is charging).
<allenap> jtv: Okay.
<jtv> Now, the code I have should be in a pretty nice shape.
<jtv> I'm trying to make useful notes.
<jtv> And Steve said you'd been working on something called... SyncPackageJob?
<allenap> jtv: SyncPackageJob has been reincarnated as PackageCopyJob, but yes.
<allenap> Instead of calling IArchive.syncSource() it calls do_copy() directly, and so can copy multiple packages in a single job.
<henninge> Can anybody give me a clue why this test http://paste.ubuntu.com/602716/ produces this output http://paste.ubuntu.com/602717/ ?
<jtv> allenap: ahhh so copying and syncing really are more or less the same thing?
<henninge> For some reason, "search_text" is an "object", not a string.
<allenap> jtv: Yeah, so it seems :) syncSource() and syncSources() both end up calling do_copy().
<jtv> henninge: no idea; looks like it might be one of those uses of object() as a way of creating a guaranteed-unique value in cases where None already has a meaning.
<jtv> allenap: Thanks.  I'll update my comments so at least that is correct.
<henninge> jtv: thanks for looking at it ;)
<jtv> allenap: another point is that the button's working title is Upgrade Packages.  There may well be a better name for that.  I'm told cjwatson would probably know.
<henninge> adeuring: Moin! Do you have an idea ^^ (My question with the two pastebins)
<allenap> jtv: Cool.
 * adeuring is looking
<cjwatson> the main thing to be careful of is to draw a distinction between pulling in new versions of unmodified packages from the upstream distribution, which can be done verbatim, and pulling in new versions of packages that have been modified downstream, which requires human attention (and is, I hope, outside the scope of this button)
<jtv> cjwatson: thanks.  That is correct: the harder case is outside the scope of this button.
<jtv> The big question right now is what should be on the button.
<cjwatson> in Ubuntu jargon we call those "sync" and "merge" respectively
<bigjools> its label?
<jtv> bigjools: yes
<bigjools> "sync all unchanged" ?
<jtv> Well they are unchanged, but in the parent.
<bigjools> or sommat
<adeuring> henninge: after looking at an "advanced search" page, I'd suggest: s/search_text/searchtext/
<henninge> hm
<bigjools> cjwatson: do you differentiate in nomemclature between a single sync and the mass-sync?
<bigjools> nomenclature, even
<henninge> adeuring: I'll try but this is the equivalent in the UI:
<henninge> https://answers.launchpad.net/ubuntu/+questions?field.search_text=portugu%C3%AAs&field.actions.search=Search&field.status=OPEN
<jtv> bigjools, cjwatson: I'd also like to be very clear in the distinction between the "interesting" cases and this simple case.
<cjwatson> bigjools: no
<bigjools> ok
<henninge> adeuring: does not help but interestingly gives the same error.
<jtv> Just repeating the word "sync" all over the place and making the button labels very long doesn't quite do it for me.
<adeuring> henninge: weird...
<henninge> adeuring: it's like it does not see the parameter.
<bigjools> cjwatson: when someone wants a brand new package, presumably its uploaded rather than synced?
<lifeless> henninge: thanks
<jtv> henninge: then it may just mean that the data dict is uninitialized and there's a get(foo, object())
<cjwatson> bigjools: that's generally synced too
<bigjools> interesting
<cjwatson> bigjools: well, if it was in Debian and we can just reuse that, anyway
<cjwatson> bigjools: if they do it from scratch for Ubuntu, that's an upload
<bigjools> well, sync-source is rather esoteric
<adeuring> henninge: have a look at https://bugs.launchpad.net/launchpad/+bugs?advanced=1 . There is this tag: <input class="textType" id="field.searchtext" ...
<cjwatson> not especially, it's rather simple-minded
<bigjools> cjwatson: right, ta
<cjwatson> in our terminology, "sync" is any case where we copy a package verbatim from another archive
<jtv> Packages that are in the parent but not the derived series are another thing I'd very much like to stay away from, naming-wise.
<bigjools> yeah, makes sense
<cjwatson> (source-only)
<henninge> adeuring: I am quite sure it's "search_text" in this case.
<cjwatson> I'm not wedded to that terminology, but if you do use the word "sync" then it should match those semantics, IMO
<bigjools> yup, that's been my assumption all along
<bigjools> we're not touching mergers
<bigjools> merges
<adeuring> henninge: argh... your bug is about questions, not bugs, right?
 * adeuring was looking in the wrong place...
<henninge> adeuring: it is. Sorry.
<jtv> bigjools, cjwatson: would "upgrade" be a good word to use then, for the case where it's merely an upgrade that's being pulled in?
<wgrant> The same button will also probably sync new packages. So I doubt it.
<cjwatson> personally, "upgrade" sounds like something users do
<cjwatson> and yeah, I'm not sure I see any value in a UI distinction between install/upgrade in the button name
<bigjools> cjwatson: oh one more thing, we're going to auto-create indexes in new series as part of the first publisher run.  I think you'll still be able to do compare-archive even without disabling the publisher since the release pocket won't change and the new series is frozen
<wgrant> I think we should reuse the terminology from everywhere else in LP: just say copy.
<cjwatson> bigjools: doesn't the first publisher run already create indexes?  unless you're talking about some indexes I'm not aware of
<cjwatson> I think I agree with wgrant
<cjwatson> it's a source-only copy
<bigjools> cjwatson: the "careful" mode I mean
<jtv> The button as specified doesn't copy new packages.
<wgrant> cjwatson: initialisation will automatically create the indices.
<wgrant> cjwatson: No need for a manual run any more.
<cjwatson> oh, right
<cjwatson> sure, whatever :)
<bigjools> heh
<wgrant> Click button in the web UI, it will do the rest.
<bigjools> cjwatson: is compare-archive really necessary anyway?
<wgrant> No manual i-f-p or p-d or blah.
<cjwatson> bigjools: it's just paranoia
<bigjools> figured
<cjwatson> and it doesn't cost you guys anything
<cjwatson> but yeah, less manual work => good
<bigjools> yeah I didn't think there'd be any complaints :)
<bigjools> we need to do it for derivations, or they will need someone to ssh in to somewhere
<cjwatson> jtv: hm, well, sync-source can only go away once we have some other way to copy new packages
<cjwatson> so that sounds like a spec weakness
<bigjools> there is a way, it's in the spec
<wgrant> I wasn't aware it wasn't going to copy everything :(
<wgrant> bigjools: Also, how are we going to push copies through the queue?
<bigjools> we're not
<wgrant> Erm.
<wgrant> Erm.
<bigjools> well, it doesn't do that right now
<wgrant> Clearly.
<wgrant> But it needs to.
<bigjools> out of scope
<cjwatson> so right now, syncs get away with going through a specialised queue (ignoring unapproved etc.) because only archive admins can run it
<bigjools> right
<cjwatson> once it's opened up to everybody, not going through the queue means that any developer can break our release management around milestones and we can't stop them
<cjwatson> so if it's out of scope, it needs to be turned off :-)
<bigjools> heh
<wgrant> That is my point, yeah.
<wgrant> We can't allow syncSource into a primary archive unless it goes through the que.
<wgrant> +ue
<bigjools> the other option is to restrict copying to archvie admins until we add queueing
<wgrant> Since it would allow anyone to copy into -updates, for example.
<wgrant> == bad
 * cjwatson nods
<bigjools> cjwatson: so, you'd be happy with all syncs going through the normal queue process that uploads do?  (including auto-acceptance etc)
<adeuring> henninge: trying to reproduce your problem, I got at first another error: SyntaxError: Non-ASCII character '\xc3' in file ... After s/Ãª/\x2345, and after adding "with_person_logged_in", create_initialized_view worked for me.
<henninge> adeuring: thanks, I'll check that after lunch.
<cjwatson> bigjools: totally
<cjwatson> bigjools: er, modulo mail announcements
<bigjools> cjwatson: great, I always wondered why sync-source never did that anyway
<cjwatson> manually requested syncs need announcements on the -changes list; mass autosyncs need to not flood the -changes list
<bigjools> but as you say, it needs shell anyway
<wgrant> Yeah. I think mass syncs will have to be AA-driven with a no-mail flag.
<bigjools> AA?
<bigjools> oh nm
<cjwatson> bigjools: I think it's because if you put it through the queue then there's no way to suppress announcements
<cjwatson> IIRC
<bigjools> there's always a way :)
<cjwatson> wgrant: doesn't really help derivatives ...
<wgrant> Also because the hack wasn't dirty enough, they needed a bit more dirt.
<wgrant> cjwatson: I mean in the UI.
<cjwatson> bigjools: well, I mean when we set it up
<cjwatson> wgrant: ok
<wgrant> cjwatson: We don't want normal people to copy things in without announcement, do we?
<cjwatson> oh, actually, there was a BIG reason they went through a separate hacked queue
<cjwatson> syncs aren't GPG-signed
<cjwatson> wgrant: no, indeed
<bigjools> we can handle that now
<wgrant> bigjools: Oh?
<StevenK> Perhaps we just implement NSS :-)
<cjwatson> at the time, though, that's the basic reason why Daniel set it up with a different queue
<wgrant> StevenK: That's what this is...
<bigjools> I can't remember *why* but there's code that handles lack of signer
<wgrant> bigjools: Right, and that's what the backdoor policies use. :(
<bigjools> not those
<wgrant> Huh?
<wgrant> What else accepts uploads with no signer? :/
<bigjools> there is actual code that doesn't fall over blindly accessing the signature
<StevenK> buildd?
<wgrant> Oh.
<StevenK> I guess.
<bigjools> like I said, I can't remember why it's there
<wgrant> buildd doesn't go anywhere near sigs.
<StevenK> But it does do uploads.
<wgrant> Oh, you mean -C buildd?
<bigjools> cjwatson: which queue?
<wgrant> -C sync, I presume.
<wgrant> That is SyncUploadPolicy.
<cjwatson> yes.
<cjwatson> ~lp_queue/sync-queue/ for those with cocoplum access.
<cjwatson> the processing command is LPCONFIG=ftpmaster /srv/launchpad.net/codelines/current/scripts/process-upload.py -d ubuntu -C sync $DRYRUN $NOMAILS -v .
<StevenK> cjwatson: Of LP devs, that's ... me
<bigjools> how do the packages get to the local FS?
<wgrant> sync-source with evil...
<StevenK> They are grabbed from ftp.uk.debian.org and/or the librarian
<bigjools> ahhhh
<wgrant> More evil, that is.
<bigjools> right, I forgot that sync-source doesn't actually do any uploading does it?
<cjwatson> no, it creates the files in good order to be copied into a queue
<bigjools> which queue?
<cjwatson> see above!
<bigjools> I mean, LP queue.  I presume they just hit accepted?
<wgrant> They go through whatever SyncUploadPolicy puts them into. Which is normally DONE>
<bigjools> auto-accepted then
<bigjools> ok I'll dig later
<wgrant> Indeed.
<wgrant> AbstractUploadPolicy.autoApprove returns True.
<wgrant> The FROZEN handling is in InsecureUploadPolicy. Odd.
<bigjools> this makes things a little more complicated than I'd anticipated
<cjwatson> right now yes - but as I say they'd need to be treated slightly more like uploads once this is open to all developers.
<wgrant> That seems like the sort of thing that really should be elsewhere.
<cjwatson> the real distinctions we actually need are insecurity and announcements.
<cjwatson> (AFAICR.)
<wgrant> Insecurity?
<wgrant> Oh.
<wgrant> Right.
<wgrant> Yes.
<cjwatson> (no signed .changes)
<wgrant> Announcements are going to be amusing.
<bigjools> for NSS that won't be an issue since there's no changes file
<wgrant> Since we don't have changes files at all for Debian uploads.
<wgrant> We'll have to parse the changelogs.
<bigjools> yeah, *that*'s the issue :)
<cjwatson> sync-source does that already.  just move the code around.
<wgrant> sync-source must burn.
<wgrant> BURN.
<cjwatson> but that code is the very bit you need.  burning it out of pique is a tad silly. :)
<wgrant> python-debian makes it easy these days.
<cjwatson> sure - reuse the business logic though
<wgrant> Don't need sync-source's regexps.
<wgrant> It does things like this:
<wgrant>             if previous_version is None:
<wgrant>                 previous_version = "9999:9999"
<wgrant> I'm reluctant to reuse logic that has been near that.
<bigjools>  /o\
<cjwatson> I thought I proposed a patch at one point to fix that.
<wgrant> There was a fix at one point for < 0 versions.
<wgrant> But I don't recall anything about this bit.
<cjwatson> sigh, trapped in a local branch
<cjwatson> http://paste.ubuntu.com/602728/
<wgrant> Well, that was easy.
<cjwatson> (since < 0 versions don't work right now)
<wgrant> Yeah.
<wgrant> cjwatson: Can we get away with not attaching a fake changes file -- just compiling the usual email contents?
<cjwatson> probably
<wgrant> I don't know of anything that parses the changes files.
<cjwatson> oh, hm
<cjwatson> I do
<wgrant> Can it use the API instead? :)
<cjwatson> it does
<cjwatson> we have an API-based thing that we use for compiling point release notes
<wgrant> Oh, right.
<cjwatson> it grabs .changes files using the API
<wgrant> HMm.
<bigjools> so when we have this in the UI, one button "mass sync" will not send announcements, the other one "sync" will.  I assume that's good enough.
<wgrant> That's slightly upsetting.
<wgrant> cjwatson: Can I see that code somewhere?
<cjwatson> http://paste.ubuntu.com/602729/ is the most recent version of it I have to hand
<wgrant> Hopefully it doesn't want too much.
<wgrant> Ah, so it just wants bugs?
<cjwatson> right now I think so, though I think in general having access to the equivalent of changes['changes'] would be good.
<wgrant> Yeah. Hmm. That's difficult.
<wgrant> Since the definition of that is a little hazy.
<cjwatson> it's tricky in cases where the sync skips over several versions
<wgrant> Right.
<wgrant> We need to determine the old version.
<wgrant> Whatever that means...
<cjwatson> publishing history?
<wgrant> Sure, we have the data, but we have to work out how to interpret it. And then how to store the changelog entries.
<wgrant> Unless we just provide a method on SPPH which finds the last published version and grabs the library files and works it out.
<wgrant> Not quick, but meh.
<wgrant> Since we don't really have anywhere to store it.
<cjwatson> bigjools: announcements> yes, sounds good enough
<cjwatson> (whether it meets your UI guidelines is another matter, but it meets our requirements)
<wgrant> Come now, you've been around Launchpad for long enough to know how strict and excellent our UI guidelines are....
<bigjools> cjwatson: I'll work that out later but it was to clarify the difference for me, which you did
<cjwatson> it's certainly better than archive admins having to remember which ones need a special flag passed
<wgrant> Or forgetting to flush normal sync queues before someone does a no-mail run...
<cjwatson> yeah, though the person who does such a run is supposed to check
<cjwatson> but it's a bit error-prone
<StevenK> allenap: O hai -- has my diff made you blind yet? :-/
<bigjools> wgrant: I think the queue stuff will be easy-ish.  We can refactor the nascentupload checks so they can be re-used.  If the package is not auto-accepted, we make the appropriate PU, otherwise copy as we do now.  Does that sound sane?
<wgrant> bigjools: We also need to log the copies.
<bigjools> log?
<wgrant> Who did them, and where from.
<wgrant> At the moment we have no idea.
<bigjools> joy
<wgrant> I had considered just adding a creator to [SB]PPH. I still think that might be reasonable.
<wgrant> But it sounds relevant to the queue stuff.
<bigjools> well we're not losing anything by not doing it if it's not already done, so it's not critical
<wgrant> cjwatson: Do you want to know who copied things? I presume so.
<wgrant> It would seem mildly insane to not.
<cjwatson> wgrant: at the moment we record who requested the sync by pretending they're the uploader
<wgrant> Right, I know that, but we can't do that any more.
<cjwatson> that kind of loses information a bit and it would be better to have that in a separate slot
<wgrant> Yeah.
<cjwatson> yes, definitely do want to know, for audit trail
<wgrant> But in the current copy model there is no record (besides appserver logs)
<wgrant> Right.
<wgrant> bigjools: ^^
<bigjools> well, they are effectively the uploader
<cjwatson> and it should go out in mail announcements of manual syncs too
<bigjools> ok
<cjwatson> bigjools: in the past, Debian people have justifiably got upset at what looks like an Ubuntu person taking credit for their work
<wgrant> cjwatson: But we also have them getting upset about the opposite :/
<cjwatson> so we should have both names
<wgrant> "I didn't upload that"
<bigjools> cjwatson: don't they see the different users involved?  uploader vs maintainer vs creator?
<wgrant> uploader/creator are one field in LP.
<cjwatson> bigjools: by overwriting uploader, we lose one of those
<wgrant> signer is different, but that's not used for syncs.
<bigjools> not exactly
<cjwatson> and signer may be the sponsor in Debian anyway, not the person who actually did the work
<cjwatson> wgrant: I'm much happier to defend the case where we give too much credit
<bigjools> sourcepackagerelease.creator
<wgrant> bigjools: That's Changed-By, right?
<bigjools> I thought it was the dsc signer?  can't remember!
<cjwatson> as for recording the Ubuntu side of it: we do need to do that because it's used as part of people's applications for Ubuntu upload rights
<wgrant> No.
<wgrant> bigjools: SPR.dscsigningkey
<wgrant> Huh.
<bigjools> I would like to maintain uploaded-by as-is
<wgrant> Why dscsigningkey and not changesigningkey? :/
<wgrant> That's pretty bad.
<wgrant> bigjools: As-is? What do you mean?
<bigjools> if something's getting overwritten that's a separate bug
<wgrant> bigjools: We have no control over that. gina sets it.
<bigjools> I'm talking about syncs
<wgrant> Syncs will stop existing.
<bigjools> gina should not set uploader
<wgrant> Hm?
<wgrant> What do you mean by uploader?
<bigjools> of course they won't, we're syncing here
<wgrant> We normally use SPR.creator as uploader.
<bigjools> instead of sync-source
<wgrant> And that is Changed-By.
<bigjools> that's crack
<wgrant> bigjools: Why?
<wgrant> And even so, we can't use anything on SPR to record the requestor.
<bigjools> the uploader is whoever signed the changes file
<bigjools> or in this case, did the sync
<wgrant> bigjools: Right, that's (mostly) dscsigningkey.
<cjwatson> um, ish
<bigjools> not really
<wgrant> At present it's all we have.
<bigjools> right, and it needs cleaning
<wgrant> Where do you propose to store the sync requester?
<wgrant> It can't be on SPR>
<bigjools> dunno
<bigjools> not got that far yet :)
<wgrant> I am initially inclined to store the uploader/requestor in some generic version of PackageUpload, which works for copies too and goes through the queue.
<bigjools> mebbe
<bigjools> why can't it be on spr?
<bigjools> ah I see
<wgrant> What happens if I sync it to two places?
<bigjools> yeah
<bigjools> so, maybe on spph
<bigjools> not sure about creating PUs all the time
<wgrant> We need to think and see if we can unify this and PackageUpload and all the checks in archiveuploader that duplicate packagecopier.
<bigjools> yes
<wgrant> Since it's currently stupid.
<bigjools> yes
<wgrant> Really duplicated and really stupid.
<bigjools> yes
<bigjools> one step at a time
<allenap> StevenK: I was talking about other things this morning, sync buttons, PackageCopyJob, etc, so I haven't got to the diff yet, but I'm clear now.
<wgrant> Right, but some of these steps are DD-critical, so it is probably a good idea to go some way to at least identifying that they exist :)
<bigjools> wgrant: do you have a list/idea of which checks are duped in packagecopier and archiveuploader?
<wgrant> bigjools: Not really... but everything in packagecopier is either buggily duplicated or buggily omitted from archiveuploader.
<wgrant> It is *entirely* checks that archiveuploader needs too.
<bigjools> sorta, but yeah
<wgrant> I don't really know how we can unify them.
<bigjools> archiveuploader deals with one at a time
<wgrant> Unless we make the internal copy logic take a set of SPRs and BPRs and their overrides.
<bigjools> I don't either, we'll just have to look at it and work it out
<wgrant> And then the copier can extract those from the source archive, and archiveuploader can use the ones it's just created.
<wgrant> The post-refactor underlying logic mostly does that, anyway.
<wgrant> It just needs to be dragged up a bit further.
<bigjools> it needs a coding sprint
<wgrant> Yes
<wgrant> the copying stuff is not going to get fixed without that, I don't think :/
<wgrant> It needs thought and experimentation and throwing at people.
 * bigjools eats
<henninge> adeuring: thanks, that helped! I also had use a different layer.
<adeuring> henninge: out of curiosity: Do you know what type this object was you got for the string "portuguÃªs"?
<henninge> adeuring: "object" ...
<henninge> generic
<henninge> nothing
<adeuring> indeed
<henninge> ...
<henninge> adeuring: I also found it that it was using the widgets from zope.app.dav not zope.add.form.
<henninge> s/add/app/
<adeuring> intereting
<henninge> I don't know if the wrong layer influenced that but I could imagine that.
<jcsackett> clear
<jtv> gmb: free to review a simple peephole cleanup branch?  https://code.launchpad.net/~jtv/launchpad/pre-747546/+merge/59771
<gmb> jtv: Sure.
<jtv> thanks
<jtv> allenap: my branch for the upgrade button is here as a WIP... https://code.launchpad.net/~jtv/launchpad/bug-747546/+merge/59765
<allenap> Cool, thanks jtv.
<jtv> A lot of that diff is actually just cleanup, which is currently being reviewed as a separate branch to lighten the diff.
<gmb> jtv: r=me
<jtv> thanks!
<jtv> allenap, correction: the cleanup branch _has been_ reviewed.  :)
<jtv> It's set as a prerequisite for the main branch.
<jtv> (So actually I'm not even sure why its changes show up in the diff)
<allenap> jtv: Mmm, suspicious :)
<henninge> Hi deryck! ;)
<deryck> Hi hi
<deryck> henninge, stand up?
<henninge> deryck: I need a minute alone with my laptop ... :(
<deryck> henninge, tmi
<deryck> henninge, join us when you finish :)
<henninge> sure
<jtv> allenap: the big remaining issue with my branch is that the few people who actually know soyuz stuff get into long, incomprehensible discussions when you ask "what label do you want on the button?"  :-)
<allenap> jtv: lol!
<jtv> I'm serious!
<deryck> gmb, ping
<gmb> deryck: Hi
<sinzui> deryck: This page has directions to enable the backlight buttons on you computer: https://help.ubuntu.com/community/MacBook5-1/Natty
<deryck> sinzui, oh happy day!
<deryck> sinzui, it worked for you?
<sinzui> yes
<deryck> lovely.  I shall disappear shortly as I relog. ;) :)
<sinzui> I did the same :)
<deryck> oh happy not-overly-bright-or-overly-dim day!
<deryck> sinzui, many thanks for the pointer!
<bigjools> heh
 * deryck turns the brightness up and down repeatedly, just 'cause he can
<sinzui> deryck: I believe every feature of older macbooks are now working working without hacks
<deryck> sinzui, yup, that was my last issue.
<sinzui> Now I need to avoid buying a new computer for a few years
<jcsackett> sinzui: in a few years you won't be able to install any software on a macbook anyway, so you'll have to buy a different computer *anyway*.
<jcsackett> sooner or later usb ports will be deemed as too messy for the unibody. :-P
<sinzui> jcsackett: I doubt that. I do not use the macos except as a way to buy music from time to time. I buy from U1 and amazon UK too because my indie tastes are poorly represented in all music shops
<StevenK> jcsackett: But then how will fanbois hook up their iPhones and iPods!
<StevenK> jcsackett: I mean, oh noes!
<jcsackett> sinzui: ah, so your next computer may be a thinkpad or something?
 * jcsackett is considering his next upgrade
<jcsackett> StevenK: ipods will be updated via a proprietary protocol that is basically tarted up bluetooth. they will call it iSync, and it will return "childlike wonder" to wireless devices.
 * jcsackett stops his cynical futurism.
<StevenK> Where did I put that hangman's noose ...
<jcsackett> ?
<StevenK> jcsackett: iSync as a 'feature' makes me want to inflict harm
 * jcsackett laughs.
<jcsackett> i joke, but apple did just by the iCloud domain.
<jcsackett> s/by/buy/
<sinzui> StevenK: hmm, I think that tarted up protocol will be based on what the hacker community created to enable mounting the pnhone over the net.
<StevenK> allenap: Thank you for the review. No fair not +1'ing my shell output.
<allenap> StevenK: Okay, okay, +1 on shell dump.
<StevenK> That sounds like Apple. What Free Software project can we take, bend to our will and never release as source?
<StevenK> allenap: :-)
<allenap> StevenK: The whole diff was actually repeated twice. It was a nice surprise to find that I was done once I was only half way through.
<StevenK> allenap: Oh, drat. Sorry about that. In which case, there is *one* change between the two diffs, can you find it? :-D
<allenap> StevenK: Yeah, there was a change to a docstring iirc.
<StevenK> Right
<StevenK> allenap: Nicely done. At least you didn't mention the fix in the review mail :-)
<adeuring> gmb: could you please review this mp: https://code.launchpad.net/~adeuring/launchpad/bug-768443/+merge/59782 ?
<deryck> mrevell, ping
<mrevell> Hey there deryck. How are things in Alabama?
<henninge> gmb: Hi! Are you free for a review?
<henninge> https://code.launchpad.net/~henninge/launchpad/devel-644872-unicode-error-in-search-text/+merge/59783
<deryck> mrevell, getting better here.  still lots of damage to clean up, though.
<gmb> adeuring: Sure.
<adeuring> thanks!
<gmb> henninge: I'll take yours afte adeuring's
<henninge> gmb: thank you ;)
<gmb> adeuring: r=me
<adeuring> gmb: coool, thanks!
<gmb> henninge: r=me
<henninge> gmb: thank you ;)
<rvba> gmb: Hi! Could you please review this MP: https://code.launchpad.net/~rvb/launchpad/sync-to-updates/+merge/59653 ?
<gmb> rvba: Sure
<rvba> thanks.
<gmb> rvba: Approved.
<rvba> gmb: thanks a lot.
<gmb> np
<jcsackett> abentley: can i beg your assistance on some yui/test stuff?
<abentley> jcsackett: Sure.
<jcsackett> cool. so, i am trying to just get a stubbed out suite with two tests that fail. right now i have http://bazaar.launchpad.net/~jcsackett/launchpad/spam-button-ui/view/head:/lib/lp/answers/javascript/tests/test_question_spam.html
<jcsackett> and it's js file is http://bazaar.launchpad.net/~jcsackett/launchpad/spam-button-ui/view/head:/lib/lp/answers/javascript/tests/test_question_spam.js
<jcsackett> the js file being tested is stubbed out here: http://bazaar.launchpad.net/~jcsackett/launchpad/spam-button-ui/view/head:/lib/lp/answers/javascript/question_spam.js
<jcsackett> i am pretty sure i have failed in setting something up, as when i open up the html i expect to be told of two failures (for the Assert.isTrue(false)) stuff, but i get nothing.
<abentley> jcsackett: For me, that's usually failure to include a JS file, or perhaps a wrong path.
<LPCIBot> Project db-devel build #512: FAILURE in 5 hr 39 min: https://lpci.wedontsleep.org/job/db-devel/512/
<jcsackett> abentley: i thought that, but i've double checked the paths and they all look correct.
<jcsackett> abentley: in the html harness, i link in test_question_spam.js and ../question_spam.js; as the test_*js file is in the same dir as html and the other file is one above, that should be right, shouldn't it?
<abentley> Yes, that does look right.
<abentley> jcsackett: Your paths to yui etc don't look like mine at all.
<abentley> jcsackett: e.g. I have ../../../../canonical/launchpad/icing/yui/yui/yui.js
<jcsackett> abentley: ah, that could be. i didn't double check those b/c i copied them out of the wiki page.
<abentley> jcsackett: I know the wiki was wrong, but I thought we'd fixed it.
<abentley> jcsackett: Anyhow, try copying them out of lp/translations/javascript/tests/test_sourcepackage_sharing_details.html
<jcsackett> abentley: okay, thanks.
<timrc> Is 404 to an empty PPA really the correct response?  When you create a PPA, it should exist at the URL, even if it's empty IMO.  Maybe I'm about to split blasphemy but wouldn't a 204 No Content be an improvement?
<bigjools> indexes are deliberately not published to save space/inodes
<jcsackett> abentley: i have updated the paths, still nothing. :-/
<jcsackett> abentley: do you have any knowledge regarding hooking in namespace stuff? perhaps i've screwed up in getting the stub of a js module hooked in...
<timrc> bigjools, but changing the HTTP code to a 204 would distinguish is from a "you mistyped the URL" error
<abentley> jcsackett: sorry, OTP
<bigjools> timrc: how do we know it wasn't a mistype?
<bigjools> it's just a plain Apache server
<timrc> bigjools, launchpad should be smart enough to know that the ppa was created, but is empty and then return a code better reflecting that.. based on launchpad's implementation 404 is probably technically correct, but it's not intuitively correct IMO
<bigjools> timrc: fair enough.  I think it's quite a lot of work to do for a small gain though.
<timrc> perhaps :)
<bigjools> patches welcome :D
<abentley> jcsackett: I did have to hook up a namespace for the work I did, but I just followed an example I found.
<abentley> jcsackett: Are you using firebug or similar?  That sort of problem will usually generate error messages.
<jcsackett> abentley: no, firebug isn't catching anything either, which is weird.
<jcsackett> abentley: does our test stuff work with firefox4?
<abentley> jcsackett: there were some firefox4 compatibility issues.  deryck, could you remind us what those were?
 * deryck looks at scrollback
<abentley> deryck: with javascript testing in general?
<deryck> abentley, jcsackett -- ah.  so yui test has no ff 4 issues.  windmill has issues with the magic comment bugs that only become enabled when you type text into them.
<deryck> abentley, jcsackett -- and then getting a profile windmill can use with ff 4 is a bit tricky
<deryck> but running yui test in the browser should work fine
<abentley> deryck: thanks.
<jcsackett> ok, so that eliminates that as a source of the problem. thanks, deryck.
<deryck> np!
<jcsackett> ok. i have managed to get an error. yuitest is not loaded. except that i can see the .use('test') bit. hm.
<jcsackett> success! also, i cannot read.
<abentley> jcsackett: yay!
<jcsackett> abentley: so, i was doing use('yuitest'), not use('test).
<abentley> jcsackett: doh!
<jcsackett> when i went to doublecheck, i was actually looking at your file opened as an example, not mine, and saw 'test'. :-P
<jcsackett> thus, i cannot read. :-)
<jcsackett> thanks for the help, abentley. :-)
<abentley> jcsackett: you're welcome.
<benji> jml: do you have a minute to chime in on bug 772763?  in particular the importance of "low"
<_mup_> Bug #772763: Unmuting a bug's notifications should restore your previous direct subscription <story-better-bug-notification> <Launchpad itself:Triaged> < https://launchpad.net/bugs/772763 >
<jml> benji: not until tomorrow afternoon. sorry.
<benji> jml: no worries; I'll ping you again then
<jml> benji: I'm already late for an appointment, and am flying out to Budapest tomorrow morning. will make sure that bug doesn't slip through the cracks.
<jml> benji: ta
 * jml gone
* gmb changed the topic of #launchpad-dev to: PQM in RC mode | https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<JonOomph> Greetings!  I am having a LaunchPad PPA issue and I'm not sure who to direct it to.  I have many builds that are marked as 'Needs building', and they have been stuck in the queue for about 12 hours now.  Any thoughts?
<bigjools> JonOomph: there is currently a very large backlog of builds, the queue is very long
<bigjools> see https://launchpad.net/builders
<bigjools> we're currently missing a number of builders as they are temporarily used during the Ubuntu release
<JonOomph> Ahhh, that makes perfect sense
<bdmurray> bac: what's the right way to give you feedback regarding better bug notifications?
<JonOomph> Having a few days of delay in the PPA build system makes releasing a new version of OpenShot a bit tricky, but I'm patient and can wait.  =)
<sinzui> jcsackett: do you have time to mumble?
<jcsackett> sinzui: sure, just one moment.
<benji> sinzui: will you have some time this afternoon to do a small UI review for me? (https://code.edge.launchpad.net/~benji/launchpad/click-to-close-boxes/+merge/59818)
<sinzui> benji: I can starting it now
<benji> sinzui: I am thanking you now.
<jelmer> rockstar: hi
<lifeless> moin
<jelmer> g'day lifeless
<rockstar> jelmer, hello sir
<jelmer> rockstar: What's the policy on landing changes in lp:tarmac?
<jelmer> rockstar: related, is it intentional that lp:tarmac is owned by an open team?
<jelmer> argh, it isn't.. nevermind
<abentley> deryck: there's no OCR.  Could you please review https://code.launchpad.net/~abentley/launchpad/allow-noop-claims/+merge/59795 ?
<deryck> abentley: sure.
<abentley> deryck: thanks.
<benji> sinzui: Thanks for the review.  Do you want the "Hide \2715" bit in parenthesis to distringuish it from the text of the message?
<sinzui> No, that would introduce difference. We want to be the same
<benji> (I'm also investigating using the sprite.)
<benji> k
<deryck> abentley: r=me
<abentley> deryck: thanks.
<lifeless> benji: hi
<lifeless> merge proposals don't do attachments
<benji> lifeless: I know that now. :)
<lifeless> benji: where else could one see these screenshots?
<lifeless> ah, I see, nvm
<benji> I edited the MP to point to...
<benji> right
<lifeless> so, UI question (perhaps curtis has asked this)
<lifeless> we have a X icon in the top right of other things that are closable
<abentley> lifeless: they do too!
<lifeless> abentley: oh ?
<abentley> lifeless: attach a patch and we'll even colourize it!
<lifeless> abentley: supporting attachments is a superset of supporting attachments-that-are-patches
<benji> lifeless: right; in the review he suggested emulating the "Hide X" target on the new private bug ribbon
<lifeless> benji: ok cool; I can butt out
<lifeless> benji: separately, I declined the team-join for lp to alpha testers
<abentley> lifeless: you didn't say "launchpad doesn't support attachments of particular kinds"
<lifeless> I think a feature flag is better and easier to clean up
<benji> abentley: my problem with that MP was that I submitted it after forgetting the attachments and it took less time to find the edit link then to figure out how to add attachments after the fact
<lifeless> abentley: you're splitting hairs I think; theres a very large set of more refined meanings I could have meant
<lifeless> abentley: mps don't support attachments *like bugs support attachments*
<benji> lifeless: i.e., just change the flag to predicate on team:launchpad instead of team:malone-alpha?  if so, that sounds good to me.
<lifeless> benji: yeah, or add a row
<lifeless> benji: depending on whether you anticipate having non-lp folk participate in the alpha stage
<benji> sounds good; I'll get that done tomorrow
<lifeless> I would expect something like adding launchpad-beta-users as a rule when it comes out of alpha
<abentley> lifeless: It looked to me like you were saying "merge proposals don't allow attachments" as an absolute.
<benji> lifeless: I wasn't aware of launchpad-beta-users, sounds like the ticket
<lifeless> hmm, spelling error, one sec
<lifeless> benji: https://help.launchpad.net/GetInvolved/BetaTesting
<lifeless> launchpad-beta-testers
<lifeless> is the team
<lifeless> 2K early adopters
<benji> cool
* flacoste changed the topic of #launchpad-dev to: Merge to devel are open again | https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<benji> sinzui: hmm, the privacy ribbon's hide affordance is underlined on hover and changes the mouse cursor.  I'll match that behavior.
<lifeless> benji: oh, btw - you have 'edge' in your browser bookmarks :>
<benji> heh, so I do; fixed now
<lifeless> hmm
<lifeless> we have lots of css warnings in the ff4 console
<sinzui> lifeless: pocket-list also complains about CSS. There is no to that is aware of supported and proprietary rules to provide a competent report of broken css.
<sinzui> s/pocket-list/pocket-lint
<lifeless> sinzui: hi
<sinzui> hi lifeless
<lifeless> sinzui: I'm thinking of closing https://bugs.launchpad.net/launchpad/+bug/741092
<_mup_> Bug #741092: Archive:+subscriptions times out with many subscribers <qa-ok> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/741092 >
<lifeless> because it seems ok now
<lifeless> IMBW, what do you think?
<sinzui> I honestly do not know. When was the last time we saw an issue
<lifeless> I looked back a few days, couldn't see one
<lifeless> it may be very sporadic though
<sinzui> I favour closing it, though I also like being able to say this issue has not been see in 30 days
<lifeless> better matching in oops-tools would be nice
<lifeless> Any follow on issue will not need the sql optimisation
<sinzui> lifeless: actually, I still wonder if qastaging was a bad env to test one.
<lifeless> so I think I'll close it
<sinzui> +1 I believe there are some performance issues that can only be experienced in production
<lifeless> losa ping
<lifeless> can you please change a feature flag on qastaging:
<lifeless> malone.bugmessage_owner default 0 on
<sinzui> wgrant: I cannot attend the meeting in 15 minutes. I can we talk later? I want to understand the effort needed to address bug bug/747558
#launchpad-dev 2011-05-04
<wgrant> sinzui: Sure.
<wgrant> sinzui: We're on maintenance for a little longer, then?
<wgrant> Ah, I see the email.
<lifeless> wgrant: to -dev? or squad only?
<wgrant> lifeless: Squad only.
<wgrant> lifeless: What's your opinion on BPB/SPRB URLs?
<wgrant> Currently I'm reverting BPB to +build, and SPRB to a new +recipebuild.
<lifeless> wgrant: works for me
<wgrant> The BPB URL probably isn't ideal, but it's not introducing a new one so I don't care too much.
<lifeless> yeah
<lifeless> you could do sourcebuild or something
<lifeless> but I think recipe build is fine
<wgrant> Hmm, bigjools didn't start the mawson restore.
<wgrant> sinzui: Did you want to talk about that bug at some point?
<lifeless> I'm going to get tired of clicking 'one hour' soon, I can tell.
<StevenK> lifeless: Hm?
<lifeless> launchpadlib desktop integration
<sinzui> wgrant: hi. I do want to talk about that bug. I can now now or any time in the next two hours
<LPCIBot> Project windmill-devel build #3: STILL FAILING in 1 hr 4 min: https://lpci.wedontsleep.org/job/windmill-devel/3/
<lifeless> anyone know what translationtemplateitem.sequence means?
<wgrant> lifeless: It's just used to preserve the POT's order, AIUI.
 * StevenK grumbles. I don't want to add DistroSeriesParent to the sampledata
<lifeless> so don't
<lifeless> I don't want you to either
<wgrant> getById or getByID?
<wgrant> I think the latter.
<wgrant> But we use both.
<wgrant> But I need to change one of them to the other.
<wgrant> So I want more opinions.
<lifeless> the latter
<wgrant> Thanks.
<lifeless> fooID
<lifeless> is the table convention
<lifeless> so folk will expect that elsewhere
<wgrant> Only for SQLObject.
<wgrant> For Storm we use foo_id.
<lifeless> which is 90% of our ORM objects
<wgrant> And that's not camelCase anyway.
<lifeless> headdesk
<lifeless> select count(*), potemplate is null from translationmessage group by potemplate is null;
<lifeless>   count   | ?column?
<lifeless> ----------+----------
<lifeless>  17773385 | f
<lifeless> where's jeroen when you need him
<lifeless>  51852904 | t
<wgrant> Why?
<lifeless> to find out why tm.potemplate is set in only some rows
<wgrant> Bah. Our Storm sugar class defines getById :/
<lifeless> I'm looking at bug https://bugs.launchpad.net/launchpad/+bug/534203
<_mup_> Bug #534203: Timeouts on POFile:+filter (filter by person) <lp-translations> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/534203 >
<wgrant> lifeless: I think POTemplate is only used for non-shared translations.
<wgrant> lifeless: Shared ones use POTMsgSet.
<wgrant> Yeah.
<wgrant>     def shareIfPossible(self):
<wgrant>         """See `ITranslationMessage`."""
<wgrant>         if self.potemplate is None:
<wgrant>             # Already converged.
<wgrant>             return
<lifeless> so we'll eventually drop the column?
<wgrant> I believe so.
<lifeless> hmm
<lifeless> this is a good reason not to abuse semantic fields as indicators
<lifeless> s/reason/example of/
<LPCIBot> Project windmill-devel build #4: STILL FAILING in 40 min: https://lpci.wedontsleep.org/job/windmill-devel/4/
<LPCIBot> Project windmill-devel build #5: STILL FAILING in 41 min: https://lpci.wedontsleep.org/job/windmill-devel/5/
<LPCIBot> Project devel build #685: FAILURE in 5 hr 32 min: https://lpci.wedontsleep.org/job/devel/685/
 * StevenK frowns at Jenkins
<LPCIBot> Project windmill-db-devel build #231: STILL FAILING in 1 hr 4 min: https://lpci.wedontsleep.org/job/windmill-db-devel/231/
<StevenK> Oh, sigh. Jenkins hasn't got a clue how long windmill-devel takes since one build hasn't been sucessful yet.
<wgrant> Heh.
<StevenK> However, four slaves up and building makes me happy.
<wgrant> But not Canonicloud.
<StevenK> Huh?
<wgrant> It probably doesn't make the cloud very happy.
<StevenK> No idea, TBH, but IS haven't chased me.
<wgrant> StevenK: Linode native IPv6 is handy. My tunneled v6 routing to Fremont is apparently 10ms faster than native v4.
<StevenK> Mmmmm, I was going to jump when they fix the DNS to handle AAAA.
<wgrant> You can add AAAA, but the NSes don't have AAAAs themselves yet.
<StevenK> Which means you can't resolve v6-only anyway :-)
<wgrant> Right :(
* spm changed the topic of #launchpad-dev to: Launchpad Down/ReadOnly 0800-0930 UTC | Merge to devel are open again | https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<LPCIBot> Project windmill-devel build #6: STILL FAILING in 1 hr 10 min: https://lpci.wedontsleep.org/job/windmill-devel/6/
<adeuring> good morning
<bigjools> morning folks
<allenap> Morning bigjools, morning gnuoy, morning all.
<LPCIBot> Project windmill-devel build #7: STILL FAILING in 6 min 20 sec: https://lpci.wedontsleep.org/job/windmill-devel/7/
<StevenK> Uh oh
<wgrant> StevenK: codehosting is down.
<wgrant> I suspect that's it, but haven't checked the logs.
<mrevell> Morning
<StevenK> Yeah, that's it
<gnuoy> Morning, allenap.
<LPCIBot> Project windmill-devel build #8: STILL FAILING in 0.53 sec: https://lpci.wedontsleep.org/job/windmill-devel/8/
<StevenK> allenap: http://pastebin.ubuntu.com/603166/
<LPCIBot> Project windmill-devel build #9: STILL FAILING in 0.5 sec: https://lpci.wedontsleep.org/job/windmill-devel/9/
<StevenK> Bah, stop that, Jenkins!
<allenap> StevenK: That looks beautiful :)
<StevenK> allenap: Thank you :-)
<LPCIBot> Project windmill-devel build #10: STILL FAILING in 0.48 sec: https://lpci.wedontsleep.org/job/windmill-devel/10/
* mthaddon changed the topic of #launchpad-dev to: Merge to devel are open again | https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<LPCIBot> Project windmill-devel build #11: STILL FAILING in 1.6 sec: https://lpci.wedontsleep.org/job/windmill-devel/11/
<lifeless> wheee, killed my lp vm doing a full homedir backup with lmirror (OOM killer)
<lifeless> I think I need to check the memory footprint a little :>
<LPCIBot> Project windmill-devel build #12: STILL FAILING in 0.5 sec: https://lpci.wedontsleep.org/job/windmill-devel/12/
<rvba> wgrant: I'm working on a branch that will use the StormStatementRecorder to monitor the number of statements used by packagecopier.CopyChecker.
<rvba> The goal is the measure how many queries have been added by the recently merged perm check thing ... and also to have a test to keep the number of queries under a tight leash if we add more things in the future.
<rvba> Steven told me you might have done something like that already ... but I can't find a test that does this ... any advice?
<wgrant> rvba: I don't have a test for copychecker itself, but some of the methods it uses are constrained (eg publishBinaries)
<rvba> wgrant: ok ... so I guess the work I'm doing makes sense then.
<wgrant> rvba: Yup.
<wgrant> rvba: We do need tests for that.
<wgrant> Just nobody's written them yet.
<wgrant> And they're pretty easy to write these days.
<rvba> right
<rvba> I'll take a look at was as been done with publishBinaries.
<rvba> wgrant: thanks.
<rvba> s/as/has/
<wgrant> I may have misremembered. But grep around for StormStatementRecorder to see examples.
<rvba> I've done that yeah.
<wgrant> I think I was going to do it, but something was still not constant enough for it to be practical.
<rvba> I see. My plan is to see how many statements it takes on average (after having created a bunch of sources) ... have create a test with that number hardcoded so that if a changes increases the number of queries issued a lot, one will get a warning.
<rvba> I can't think of a better way to do this given that checkCopy does quite a lot of things in one call.
<LPCIBot> Project windmill-devel build #13: STILL FAILING in 0.51 sec: https://lpci.wedontsleep.org/job/windmill-devel/13/
<lifeless> rvba: wgrant: the worker to look for is HasQueryCount
<wgrant> That too.
<rvba> lifeless: yeah, I took a recent test by Gavin that uses that as an inspiration
<lifeless> cool
<lifeless> if you're testing a page, RendersWith something or other is a wrapper
<rvba> I've only tested a few things ... but it does not seem to be constant (or maybe I have not properly cleaned up the cache) ... with 30 sources I've an avg of 12.23 (.23?) queries, with 300 sources the avg becomes 12.023
<rvba> maybe I should stick to testing small individual methods as opposed to the whole checkCopy method.
<LPCIBot> Project windmill-devel build #14: STILL FAILING in 0.49 sec: https://lpci.wedontsleep.org/job/windmill-devel/14/
<LPCIBot> Project windmill-devel build #15: STILL FAILING in 1.3 sec: https://lpci.wedontsleep.org/job/windmill-devel/15/
<henninge> Why is PQM rejecting my landings?
<henninge> Command failed!
<henninge> running 0 tests...
<henninge> ----------------------------------------------------------------------
<henninge> Ran 0 tests in 0.000s
<wgrant> henninge: What's in the attachments it sends back?
<wgrant> We are probably in testfix.
<wgrant> stdout should tell you.
<henninge> it does
<wgrant> Looks like buildbot is a bit unhappy. I've forced it.
<henninge> why is that not in the mail itself ...
<wgrant> No idea :/
<henninge> it used to be
<henninge> wgrant: tanks
<henninge> thanks
<StevenK> henninge: It's because PQM is terrible
<henninge> how have we been planning to switch to tarmac?
<henninge> PQM script success ;-)
<bigjools> wgrant: we can't land StevenK's changes to use DSP on prod as he's got more schema changes.  Can you think of anything that will break in oneiric if we change its parent_series for a month?
<wgrant> bigjools: Translations might.
<wgrant> bigjools: Although hopefully only initialisation uses that.
<bigjools> yeah
<bigjools> it's been initialised, so ... I am looking for anything else
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #513: FIXED in 5 hr 32 min: https://lpci.wedontsleep.org/job/db-devel/513/
<bigjools> LPCIBot gets a woody
<wgrant> bigjools: SourcePackage.packaging uses it...
<wgrant> Which may break bug linking.
<bigjools> easily fixed
<LPCIBot> Project windmill-devel build #16: STILL FAILING in 0.55 sec: https://lpci.wedontsleep.org/job/windmill-devel/16/
<wgrant> I'm thinking about getBuildByArch.
<wgrant> It's *probably* not going to matter, but it's hard to be sure immediately.
<bigjools> bug 643369
<_mup_> Bug #643369: IDistroSeries.deriveDistroSeries() should use the security adapter <derivation> <lp-registry> <Launchpad itself:Triaged> < https://launchpad.net/bugs/643369 >
<bigjools> wgrant: euargh
<bigjools> that code is gross
<wgrant> And slow.
<wgrant> Damn slow.
<bigjools> wgrant: wtf does it need to traverse over parent_series - it should just go backwards over versions
<wgrant> bigjools: :/
<bigjools> although, we didn't make distroseries.version a debversion did we?
<bigjools> yeah, that will break horribly if we change parent_series.  Damn.
<wgrant> I'm not sure the breakage will matter.
<bigjools> wgrant: it might break build uploads
<bigjools> actually
<bigjools> wtf
<bigjools> ah it was only used for security uploads
<bigjools> the old style
<LPCIBot> Project windmill-devel build #17: STILL FAILING in 0.56 sec: https://lpci.wedontsleep.org/job/windmill-devel/17/
<wgrant> bigjools: Yes.
<wgrant> bigjools: Well, not entirely.
<wgrant> It only creates builds for that case.
<wgrant> But it will use parent_series when searching for existing builds too.
<wgrant> eg. in createMissingBuilds.
<wgrant> But that's probably not relevant here.
<bigjools> yeah, but it relies on getBuildByArch working
<wgrant> Since everything should be found in the series itself.
<bigjools> if it finds nothing it'll start creating new builds
<LPCIBot> Project windmill-devel build #18: STILL FAILING in 1.5 sec: https://lpci.wedontsleep.org/job/windmill-devel/18/
 * StevenK cleans up after that
<StevenK> Right. Slave deleted, build scheduled.
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #686: FIXED in 5 hr 8 min: https://lpci.wedontsleep.org/job/devel/686/
<danilos> lifeless, I see you declined a request to join 'launchpad' team into malone-alpha with "a flag would be better": we do have a flag, but we've set it to team malone-alpha, and we'd like to expand that a step at a time, so if we can't have multiple team rules, can we please add it? (unless other stuff has been arranged in the meantime)
<danilos> lifeless, and hi :)
<wgrant> danilos: You should be able to have multiple team rules.
<wgrant> danilos: Does it not work?
<danilos> wgrant, I don't know, I never knew it was supposed to work
<wgrant> danilos: It will scan through until it finds a rule that matches.
<wgrant> From highest priority to lowest.
<wgrant> There was a bug until last week that meant the 'default' scope took precedence, but multiple team rules should have worked anyway.
<danilos> wgrant, cool, I'll try that then
<danilos> losa ping: hi, just to check, if we want feature flags changed/added to, I should use LPS "DB query" section or what'd be the preferred way to communicate them?
<mthaddon> danilos: that's fine, yep
<danilos> mthaddon, cool
<LPCIBot> Project windmill-devel build #19: STILL FAILING in 1 hr 4 min: https://lpci.wedontsleep.org/job/windmill-devel/19/
<deryck> Morning, all.
<deryck> adeuring, abentley, I'll be about 5 minutes late for standup.  Sorry.
<adeuring> deryck: that gives me time to make a coffee :)
<abentley> deryck: okay.
<deryck> adeuring, henninge -- could be the wifi point I'm using
<deryck> adeuring, I'm mooching off the church next door until att arrives :-)
<abentley> deryck: have you started?
<deryck> abentley, trying to start, connection or mumble not working for me
<abentley> deryck: ah.
<deryck> abentley, henninge, adeuring -- yeah, just hold the standup and I'll listen
<deryck> if only you could hear me, henninge
<deryck> abentley, adeuring, henninge -- thanks, guys!
* abentley changed the topic of #launchpad-dev to: Merge to devel are open again | https://dev.launchpad.net/ | On call reviewer: abentley | https://code.launchpad.net/launchpad-project/+activereviews
<jcsackett> if i'm adding a class to something in YUI to change its display (in this instance, color), do i have to do anything beyond addClass?
<jcsackett> ah, no i do not, but it helps to get the classname right.
<sinzui> henninge: ping
<henninge> Hi sinzui!
<LPCIBot> Project windmill-devel build #20: STILL FAILING in 1 hr 6 min: https://lpci.wedontsleep.org/job/windmill-devel/20/
<sinzui> henninge: +1 for your fix, but I am not sure you fixed the actual bug since your encoding line is like the original test.
<henninge> sinzui: it is testing for multiple lines now, is it now?
<henninge> s/now/not/
<sinzui> henninge: I think I am running the checkers in the wrong order. The text checker that sets utf_8 must run first, not last
 * henninge thought it did
<benji> sinzui: I'm really happy with the improvements you suggested to the UI review yesterday, it's ready for another review when you have the time: https://code.launchpad.net/~benji/launchpad/click-to-close-boxes/+merge/59818
 * sinzui looks
<henninge> sinzui: ah, now I see. text_check runs last.
<sinzui> benji: r=me
<benji> great, thanks
<sinzui> henninge I can make the ordering change if you do not have time. I should have noticed my stupidity years ago
<henninge> sinzui: I can add it, np.
<sinzui> henninge: ping me when you want me to merge it into trunk
<henninge> sinzui: I don't think it need a test, does it?
<sinzui> It does not
<henninge> sinzui: pushed, you can merge now.
<sinzui> thanks. If the builder at available, this will arrive in the lp ppa in a few hours.
<bigjools> sinzui: have you seen the queue? :)
<sinzui> yes
<sinzui> :(
<maxb> Whats the package?
<maxb> Oh, and that reminds me
<maxb> Would someone like to review my lpreview-body packaging fix?
<benji> mrevell: I added a comment to https://bugs.launchpad.net/launchpad/+bug/771231 that asks you to comment on/approve the fix.
<maxb> https://code.launchpad.net/~maxb/lpreview-body/fix-package/+merge/59137
<_mup_> Bug #771231: There is no confirmation of what I've done after I create a structural subscription <exploratory-testing> <qa-needstesting> <story-better-bug-notification> <Launchpad itself:Fix Committed by benji> < https://launchpad.net/bugs/771231 >
<mrevell> thanks benji, /me looks
<maxb> You fail, _mup_
<bigjools> sinzui: poke me the build ID and I'll get it building quicker
<maxb> Oh, whoops, I fail
<benji> maxb: heh :)
<sinzui> bigjools: I do not think this is necessary. I think we can wait as long as a week to propagate this fix to all the ppas.
<mrevell> benji, That's superb. Thanks. Tag changed to qa-ok.
<benji> mrevell: great, thanks
<maxb> abentley: Hello. I have a review pending for lpreview-body. It has been missed because lpreview-body is not part of launchpad-project. Could you (as project maintainer) consider amending that?
<abentley> maxb: sure.
<abentley> maxb: however, I'd say the reason that branch has been missed is because it's for packaging, about which I know little.
<jml> sinzui: I see there's been a fair bit of discussion on bug 80902 recently. Anything I should be sticking my nose into?
<_mup_> Bug #80902: Can't target bug report from project to distribution, or vice versa <bugs> <disclosure> <escalated> <javascript> <linaro> <lp-bugs> <questions> <ubuntu-qa> <Launchpad itself:Triaged by launchpad-green-squad> < https://launchpad.net/bugs/80902 >
<sinzui> jml: I do not know. I am certain the work is harder than most people suppose, and I am certain there is a lot over lap with disclosure
<sinzui> jml: At the worst, out work on pickers will make that bug easy to fix. at best, we fix that bug
<jml> sinzui: ok.
<sinzui> At the worst, *our* work on pickers will make that bug easy to fix. at best, we fix that bug
<jml> sinzui: are you planning on starting the picker stuff while on maint?
<LPCIBot> Project windmill-devel build #21: STILL FAILING in 41 min: https://lpci.wedontsleep.org/job/windmill-devel/21/
<sinzui> I have not committed to it, but I was looking at the timouts and ui issues in the vocabs yesterday.
<jcsackett> sinzui: have you ever encountered in YUI work the method hasClass failing to work in if else statements?
<sinzui> jcsackett: no.
<sinzui> jcsackett: can I see a paste of the code
<jcsackett> sinzui: http://paste.ubuntu.com/603311/
<sinzui> jcsackett: I think hasClass() is at fault
<sinzui> jcsackett: isn't it always returning True
<jcsackett> sinzui: no. if the class isn't on the element i see false.
<jcsackett> if i do like {{{ alert(hidden); }}}
<sinzui> jcsackett: then you are saying addClass() does not work
<sinzui> jcsackett: have you tried something simpler, like toggleClass()
<jcsackett> sinzui: i didn't know such a thing existed.
<jcsackett> sinzui: still, here's the thing, i can set up conditions so that hasClass comes back false.
<sinzui> jcsackett: http://developer.yahoo.com/yui/3/api/Node.html
<jcsackett> but the "true" evaulation still fires off.
<abentley> jcsackett: See my Javascript Learnings #3.
<jcsackett> abentley, sinzui: i can see how toggleClass does the same thing i'm doing, but it doesn't explain why when hasClass is return 'false' the if clause is evaluting for 'true'.
<jcsackett> i'll move on, but this bugs me.
<jcsackett> clearly there's a gotcha or subtlety i don't get.
<rvba> jcsackett: is hidden a boolean all right? (this bugs me too ;))
<sinzui> jcsackett: write the if like if (hidden == true) { to determine if you are a victim of coercion.
<abentley> jcsackett: what is hidden_class ?  You're not defining it in the snippit you posted.
<jcsackett> hidden_class = "adminHiddenComment"
<abentley> jcsackett: I wonder if hidden_class is evaluating to undefined?  I don't know how hasClass would behave with that input.
<rvba> jcsackett: I would make sure 'hidden' is of the right type ... a boolean being 'true' or 'false' always evaluated to 'true' in a test is kinda strange.
<adeuring> abentley: could you have alook at this (small) MP: https://code.launchpad.net/~adeuring/launchpad/bug-746866/+merge/59954 ?
<abentley> adeuring: sure.
<adeuring> thanks!
<wallyworld> sinzui: hi. i've recently arrived in Budapest. 38 hours with little sleep. running on caffine. gotta stay awake for the welcome dinner :-) when you have a moment, could you please take another look at this mp? i've reworked the implementation to address the issues raised with the first approach. https://code.launchpad.net/~wallyworld/launchpad/poppy-sftp-gpgconf/+merge/59154
 * sinzui does
<abentley> adeuring: r=me, but the spacing change on line 30 of the patch looks wrong.
<adeuring> abentley: oops... I'll fix it. thanks for the review!
<abentley> adeuring: np
<sinzui> wallyworld: r=me
<wallyworld> sinzui: thanks.
<bigjools> we really need to move the "What next" at the bottom of +filebug to the top-right menu.
<bigjools> talking of which, someone will get bug 777777 today
<bigjools> good night all
<LPCIBot> Project windmill-db-devel build #232: STILL FAILING in 1 hr 15 min: https://lpci.wedontsleep.org/job/windmill-db-devel/232/
<cody-somerville> OOPS-1950CQ433
<sinzui> deryck: do you have a few minutes to mumble?
<deryck> sinzui, I do, sure.  I had mumble issues before though.  so you can help me test too
<deryck> sinzui, I hear you
<deryck> sinzui, let me log off and work on sound and come back
<sinzui> okay
<deryck> I assumed it was bad wifi this morning
<sinzui> deryck: My sound is set to use pulse, I do not see any hacks for the sound driver in /etc/modprobe.d  anymore
<dobey> maybe your wifi got stolen too, from PSN
<deryck> ah, that's it!
<deryck> actually that does help....
<deryck> I was hacking sound to get games running under wine with no psn ;)
<deryck> I had forgot
<dobey> heh
<dobey> glad to be so helpful :)
<LPCIBot> Project windmill-devel build #22: STILL FAILING in 1 hr 4 min: https://lpci.wedontsleep.org/job/windmill-devel/22/
<deryck> sinzui, right
<deryck> sinzui, I'll try again.  maybe it wasn't anything gaming related then ;)
<sinzui> I do not see your icon changing, My computer does not think you are talking
<deryck> sinzui, logging off while I futz with settings again
<lifeless> flacoste: https://bugs.launchpad.net/launchpad/+bug/534203/comments/10 may interest you
<_mup_> Bug #534203: Timeouts on POFile:+filter (filter by person) <lp-translations> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/534203 >
<deryck> sinzui, and I've got solid red lips on my end.  but the sound settings all seem ok on my end
<lifeless> flacoste: cold cache we're 60 seconds for the relevant query
<lifeless> flacoste: tis a miracle it works at all
<lifeless> flacoste: the most expedient solution would be main memory > db size for all db servers.
<lifeless> flacoste: this would scale badly because we have complete replicas, no partitioning
<lifeless> danilos: hi
<lifeless> danilos: are you still around?
<flacoste> lifeless: i'll ask charlie/elmo about what's the RAM situation nowadays with our DB servers
<flacoste> not sure we can get more
<lifeless> flacoste: I want to hear elmos voice when we ask about a 250GB upgrade.
<lifeless> flacoste: so it can wait for our biweekly ISP call :>
<flacoste> lifeless: :-)
<lifeless> flacoste: for clariry - I'm not recommending this option at this point
<lifeless> flacoste: just noting that we're starting to hit things which we can't reasonably expect to be hot given our 3:1 db:memory ratio
<deryck> sinzui, I take it you can't hear me again
<flacoste> lifeless: on an unrelated note, we should address bug 297052 at some point
<_mup_> Bug #297052: Webservice requests never use a slave database because last-write time is unknown <api> <lp-foundations> <Launchpad itself:Triaged> < https://launchpad.net/bugs/297052 >
<flacoste> given that we have a 2:1 API to web ratio
<lifeless> flacoste: indeed; or change our scaling tech and obsolete that :>:>
<flacoste> now that we support read-only API requests, we might trivially send all of them to slaves
<lifeless> anonymous ones - for sure
<flacoste> not sure how much of our traffic is anonymous API though
<sinzui> deryck: https://dev.launchpad.net/Registry/ProjectReview
<lifeless> benji: thanks
<benji> my pleasure
<lifeless> jml: btw - subunit trunk -> py3 ok
<LPCIBot> Project windmill-db-devel build #233: STILL FAILING in 48 min: https://lpci.wedontsleep.org/job/windmill-db-devel/233/
<LPCIBot> Project windmill-devel build #23: STILL FAILING in 41 min: https://lpci.wedontsleep.org/job/windmill-devel/23/
<flacoste> lifeless: i think i'll leave the total time column in, i do use it for sorting to see what page are high-impact (hits * mean)
<lifeless> flacoste: high impact in what way
<flacoste> lifeless: free more resources, drop significantly our oops count
<flacoste> although the later could be get at by sorting on total hits on the timeout candidate report
<flacoste> and the first one may be dubious
<lifeless> flacoste: so, for oops count the profile of oopses in the oops summaries is more reliable
<lifeless> flacoste: e.g. questions rarely oops (though they do) even though they have 50 second long requests
<flacoste> (i do have the changes made on a local branch, just got doubts when about to submit it)
<lifeless> flacoste: as a metric of footprint on the system sum of time spent processing isn't unreasonable
<lifeless> flacoste: but it doesn't tell us much about the fat in the page
<flacoste> and less is more
<lifeless> flacoste: something like fat-in-page*hits would be an interesting order
<flacoste> so let's remove it
<flacoste> how would you assess fat-in-page?
<lifeless> I don't know
<lifeless> :)
<lifeless> uhm
<lifeless> high sql *count* would be a reasonable first metric.
<lifeless> so 99the percentile of sql query count * hits
<lifeless> as a broad proxy for inefficient + high use
<lifeless> (not 99th percentil of sql *time*, because things like bug search are not inefficient, they are slow backends - and as such not fat on page but need redesign/new systems
<lifeless> 99th% by time doesn't need the ht count multiplier to be useful I guess is my point
<flacoste> ok, i'll remove the total_time column
<flacoste> not sure about the real usefulness of the 'Fat-in-page' column, i'll add it and let's remove it later if it's doesn't see usage
<lifeless> ok!
<lifeless> it will be interesting to see if it lines up with the oops report
<flacoste> lifeless: actually, looking at it locally, it seems pretty useless :-/
<flacoste> all the top-pages are high hits low sql statements
<vokoda`> when I try to view a file on launchpad with a file_id param I get a redirect loop error - is this a launchpad or loggerhead bug? is there a report?
<lifeless> flacoste: with real data?
<flacoste> lifeless: well, a subset of real data
<flacoste> lifeless: (the log of one app server)
<lifeless> vokoda`: that would be a loggerhead bug, I think there is a bug filed already
<lifeless> flacoste: so, that suggests that our massively assymettric app will benefit more by reducing the sql count slightly on high frequency pages than by reducing it on low frequency pages
<flacoste> lifeless: it does, yeah
<lifeless> flacoste: its the basic input into a pareto analysis
<flacoste> if we would want to drive DB usage down
<lifeless> flacoste: I would like to see it for a bit, if thats ok
<vokoda`> lifeless: yep just found it, filed just 10 days ago.. I'm sure this bug's been around much longer
<flacoste> on the top50.html report, the lowest hits*sql statements is /+bugtarget-portlet-bugfilters-stats
<lifeless> flacoste: that matches my intuition
<flacoste> and the highest one is api/beta/ubuntu
<lifeless> flacoste: cool
<lifeless> flacoste: so this makes it useful:
<flacoste> and api/beta/ubuntu is 276 more 'important' than the stats portlet
<lifeless>  /+bugtarget-portlet-bugfilters-stats is rare we know, and we know it has issues because the thing is in the oops + timeout candidates
<flacoste> according to that metric
<flacoste> fair enough, i'll merge this change
<jcsackett> sinzui: mumble?
<sinzui> jcsackett: sure
<lifeless> flacoste: so you're looking by url thre
<lifeless> flacoste: if you look by pageid
<lifeless> dum de dum browsers hate these pages
 * flacoste waits for the JS script to finish rendering
 * flacoste makes note to try those in chromium
<lifeless> flacoste: Distribution:EntryResource
<lifeless> 231101 hits
<lifeless> 0.36 99th percentile
<flacoste> Bug:EntryResource ,  BugTask:+index, DistroSeries:EntryResource
<flacoste> are the top ones
<flacoste> (on my local report)
<lifeless> 33 statements 99th percentile
<lifeless> thats quite a few :)
<flacoste> 132 of BugTask+index
<flacoste> for
<flacoste> 48 for Person:EntryResource
<lifeless> Bug:EntryResource 779685
<flacoste> yeah, i think this will be interesting
<lifeless> 43 99th percentile sql statements
<lifeless> breakfast time
<lifeless> abentley: bug 739921 - I think you may have missed a subtlety in the filing, I've made it more clear and reopened it
<_mup_> Bug #739921: The link "see all merge proposals" on person/product/+activereviews 404s <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/739921 >
<abentley> lifeless: good catch.
<LPCIBot> Project windmill-devel build #24: STILL FAILING in 1 hr 12 min: https://lpci.wedontsleep.org/job/windmill-devel/24/
<LPCIBot> Project windmill-devel build #25: STILL FAILING in 46 min: https://lpci.wedontsleep.org/job/windmill-devel/25/
<deryck> Later on everyone
<lifeless> jml: still up?
<lifeless> wgrant: recipe + binary builds
<lifeless> wgrant: is there any reason we shouldn't insert the binary build when a recipe finishes at the same point in the queue that the recipe was (e.g. front)
<lifeless> I'm not sure how to qa http://launchpad.net/bugs/773261
<_mup_> Bug #773261: The permission check for syncing packages on the differences pages (+localpackagediff) should be done on a per-package basis and not by checking lp.Edit on the series. <derivation> <qa-needstesting> <Launchpad itself:Fix Committed by rvb> < https://launchpad.net/bugs/773261 >
#launchpad-dev 2011-05-05
<wgrant> lifeless: No.
<wgrant> lifeless: I'll do it.
<wgrant> It is rather intimidating.
<LPCIBot> Project windmill-db-devel build #234: STILL FAILING in 1 hr 5 min: https://lpci.wedontsleep.org/job/windmill-db-devel/234/
 * wgrant cries at r12970
<lifeless> \o/ 12976
<wgrant> !!!
<LPCIBot> Project windmill-devel build #26: STILL FAILING in 1 hr 10 min: https://lpci.wedontsleep.org/job/windmill-devel/26/
<wgrant> lifeless: Just by looking at the diff I can see it is bad.
<lifeless> wgrant: 12970
<lifeless> ?
<wgrant> Yes.
<wgrant> It affects the API.
<wgrant> When this was to most definitely not affect the API yet.
<wgrant> It will break security unembargoing.
<wgrant> (since the security team doesn't have upload privileges)
<lifeless> they don't ?
<wgrant> Hee hee hee.
<wgrant> lifeless: I invite you to look at canonical.launchpad.security.AppendArchive
<lifeless> wgrant: blink
<lifeless> so how does anyone upload to Ubuntu ?
<wgrant> lifeless: AppendArchive isn't used for uploads.
<lifeless> class AppendArchive(AuthorizationBase):
<lifeless>     """Restrict appending (upload and copy) operations on archives.
<wgrant> It's used for protecting syncSource, basically.
<lifeless> that seems misleading
<wgrant> Yes.
<lifeless> so
<wgrant> It's also used for PPAs, I guess.
<wgrant> But not primaries.
<wgrant> And:
<wgrant> wgrant@lamuella:/tmp/ubuntu-archive-tools$ ./edit_acl.py query --person ubuntu-security
<wgrant> == All rights for ubuntu-security ==
<wgrant> Archive Upload Rights for canonical-partner-dev: archive 'partner', component 'partner'
<wgrant> There is no ArchivePermission granting them access.
<lifeless> is this a defect or oversight
<wgrant> Merely one of the stack of hacks that is security-in-soyuz.
<lifeless> I mean, it seems to me that only folk that can upload should be able to unembargo
<lifeless> otherwise there is a backdoor-to-uploading
<wgrant> Sort of. ubuntu-security is only meant to be able to upload to -security.
<wgrant> But that can't be done yet.
<wgrant> And syncSource has to be extremely restricted.
<wgrant> Because it is a hack.
<wgrant> It is full of hacks.
<wgrant> And it cannot be widely used yet.
<lifeless> so, is this unembargoing untested?
<wgrant> (it doesn't respect the queue, announcements, etc.)
<wgrant> lifeless: Unembargoing is normal package copying by this special set of users that probably aren't represented in sampledata.
<wgrant> But let's see.
<wgrant> Headdesk.
<wgrant> It only checks that launchpad.Append is sufficient to call syncSource (with somebody who already has upload privileges), and then checks that adding someone else to ubuntu-security gives them launchpad.Append.
<lifeless> so the test is invalid
<lifeless> because the control is incomplete
<lifeless> ?
<wgrant> Yes, and because security-in-soyuz is mostly P3As + a stack of several horrible special cases.
<wgrant> (which will hopefully be eliminated as part of the second stage of DDs)
<wgrant> (but not yet)
<wgrant> Anyway, let me make myself a security person on dogfood and actually try it.
<lifeless> http://www.lifehacker.com.au/2011/05/the-cognitive-cost-of-doing-things/
<lifeless> flacoste: http://www.google.com/support/analyticshelp/bin/answer.py?hl=en&answer=1205784&topic=1120718
<wgrant> virtinst 0.500.1-2ubuntu6.1 in lucid (Signer is not permitted to upload to the component 'main'.)
 * wgrant rolls it back.
<magcius> jam, did you take a look at the new CSS stuff I added?
<wgrant> StevenK: Deleted delayed copies yet?
<wgrant> All of the 70ish syncSource timeouts are from the unoptimised delayed copy path :(
<spm> lifeless: wrt the page speed via GA, last I used that in anger it gave accurate percentages; but lousy actual numbers of X's doing Y. ie treat as strong guideline vs any sort of "X many Users are..."
<lifeless> spm: thanks
<StevenK> wgrant: Ah
<wgrant> lifeless: Time to whitelist Distribution:+bugs and Distribution:EntryResource:searchTasks?
<lifeless> wgrant: the poor queries are > 20 seconds
<wgrant> lifeless: We've only had frequent complaints about these over the last month or so.
<lifeless> yes
<wgrant> lifeless: I suspect they used to work eventually.
<wgrant> After the cache got reasonably warm.
<wgrant> As it stands, common single-tag searches fail consistently.
<lifeless> since we added the optimised index which is causing pathological tag lookups
<lifeless> its not coldcache
<wgrant> Ah. Wonderful.
<lifeless> all of bugtags,bugs,bugtasks sit in hot cache
<lifeless> either the vm cache or the pg shared region
<lifeless> fun 1 /  121  RootObject:+login
<lifeless> StevenK:  271 OperationalError: FATAL: Ident authentication failed for user "sync_packages" FATAL: Ident authentication failed for user "sync_packages"
<lifeless> StevenK: do you need bugs filed about this? [there are two issues - the auth, and the OOPS prefix not being configured.
<StevenK> lifeless: Sounds like a plan -- please assign them to Gavin, since he worked on it and it should be fresh in his mind.
<wgrant> The first one needs a LOSA nudge, not a bug.
<lifeless> both are config
<wgrant> The second needs a trivial prod-config branch, which is just as fast as filing a bug..
<lifeless> right
<lifeless> been two days though
<lifeless> allenap: ^
<lifeless> well, > 2, but as top oops
<StevenK> Sigh. TALES, please DIAF.
<spm> it's problematic that ident faileds are getting into prod. these should be being added to the "funky rollout notes" at least. :-(
<wgrant> lifeless: Hm, that RootObject:+login is a bit bd. The OpenID assoc takes 20s :/
<StevenK> It raises LocationError, but it doesn't tell me which template causes it.
 * StevenK works it out using the test
<wgrant> That is definitely the biggest problem I have with TALES most of the time.
<StevenK> lifeless: So, Gavin's work is based on much earlier work by jelmer that was written, but never used. I guess he assumed it was already in the prod configs, but I can't say for sure.
<StevenK> spm: ^ you too
<StevenK> Rargh, so the test and ZCML all point me to a view and template that don't make use of parent_series
<wgrant> StevenK: Which view is erroring?
<StevenK> (distroseries, '+index')
<wgrant> It's not using SourcePackage.packaging at all?
<mwhudson> coming from nowhere in particular, but why does launchpadlib use oauth signing if all requests are over https?
<StevenK> wgrant: I don't think that's relevant, given the LocationError is (None, 'displayname')
<mwhudson> oh hang on, it doesn't
<wgrant> mwhudson: Doesn't it?
<mwhudson>         oauth_request.sign_request(
<mwhudson>             OAuthSignatureMethod_PLAINTEXT(),
<mwhudson> my lplib might be hilariously out of date, mind
<mwhudson> ok, it was, but it still seems OAuthSignatureMethod_PLAINTEXT is used
<wgrant> Yeah.
 * mwhudson tries to remember what oauthnonce is for then
<wgrant> PLAINTEXT doesn't mean unsigned.
<wgrant> I don't understand how it works, though.
<mwhudson> http://tools.ietf.org/html/rfc5849#section-3.4.4 says It does not
<mwhudson>    utilize the signature base string or the "oauth_timestamp" and
<mwhudson>    "oauth_nonce" parameters.
<wgrant> That indeed makes sense.
<mwhudson> i'm not sure it's what lib/contrib/oauth.py does though
<mwhudson> whee
<wgrant> Yeah :/
<lifeless> noone is
<mwhudson> hooray
<StevenK> wgrant: distroseries-index.pt was a red herring, the ZCML points out the page includes distroseries-portlet-derivation
<wgrant> StevenK: Ah.
<LPCIBot> Project windmill-devel build #27: STILL FAILING in 1 hr 5 min: https://lpci.wedontsleep.org/job/windmill-devel/27/
<lifeless> stub: morning
<stub> Indeed! It really is!
<lifeless> stub: I've put https://bugs.launchpad.net/launchpad/+bug/534203 in the dba bugqueue
<_mup_> Bug #534203: Timeouts on POFile:+filter (filter by person) <lp-translations> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/534203 >
<lifeless> stub: yeah, congrats!
<stub> Oh yay. Translations. The *other* place we have large amounts of asymmetric data yielding crappy plans.
<lifeless> if translationmessage.potemplate was usable
<lifeless> I think it would be fine
<stub> So a lot of these queries have already been optimized at some point by jtv & danilo, trying to get certain cases to work. I don't know which queries they are though.
<stub> But we should jtv in on translations queries when he is available.
<lifeless> this bug was filed by jtv in fact
<stub> c/should/should bring
<lifeless> I wanted to ask him about the potemplate oclumn
<lifeless> yes, I agree
<stub> lifeless: About it being inconsistently populated?
 * stub wonders if the data model changes have been completed, and all the unwanted columns now dropped.
<lifeless> stub: yeah, and/or repopulating it and using it to make the query fast
<stub> lifeless: We need a modern OOPS code in that bug btw. I think the linked ones are all from last year still.
<lifeless> stub: hmm, I refreshed it yesterday :)
<stub> ok... not obvious in the web ui that the description is fresh.
<stub> oic
<stub> Just looking at one of the example queries in the desc, but TranslationMessage.variant no longer exists.
<lifeless> stub: yeah, I kept most of jtv's analysis because I wasn't sure how much was still relevant or not
<lifeless> speak of the devil
<jtv> uh-oh
<StevenK> Haha
<jtv> what is this about?
<jtv> am I liable?
<stub> lifeless: how long before we make decisions on moving the translations to Cassandra or not? Is that a 6 month or multiyear change do you think?
<stub> jtv: Analyzing https://bugs.launchpad.net/launchpad/+bug/534203
<_mup_> Bug #534203: Timeouts on POFile:+filter (filter by person) <lp-translations> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/534203 >
<lifeless> stub: making a decision we could do now; scheduling the work ...
<jtv> Ah
<lifeless> stub: I have some thoughts on restructuring the app stack so that we can carve off parts of the system
<lifeless> stub: I want to finish a current analysis I'm on, but then if you're up for the weekly call I can run it past you
<stub> So, if it stays in PG, partitioning the data on language makes sense. But that is laborious and time consuming to set up, especially if we are avoiding large outages.
<stub> And will make all-language queries slower, but I don't think we have any of them.
<stub> Another alternative is language specific partial indexes.
<stub> Probably tm__english__sumitter__idx ON TranslationMessage(submitter) WHERE language=1; for all hundred odd languages. We could combine too for lesser languages, but I suspect better just to create one index per language and put up with the noise.
<jtv> Would it make sense to cluster on language?  That wasn't an option in the past, but is now.
<stub> jtv: The queries I'm looking at, yes. But I don't have a comprehensive overview on the queries.
<lifeless> jtv: what is translationmessage.potemplate
<jtv> lifeless: that means "this message is not shared, but rather diverged for this specific template."
<lifeless> jtv: so if its null many different templates may use the message?
<jtv> Yes.
<lifeless> jtv: and if its not null the list of templates is 1
<jtv> It should be mostly null.
<lifeless> jtv: is it going to be deleted?
<jtv> No.
<lifeless> ok
<stub> The schema migration is all done now btw?
<lifeless> so this page - is it showing /who/ contributed to the translation, or /what translations/ the person did?
<stub> I think it was completed.
<jtv> stub: you mean for translations message sharing?  We still have a few obsolete database columns but I don't think we use them now.
<stub> jtv: Do you know if there are bugs open on dropping them?
<jtv> lifeless: pofile:+filter shows a particular person's contributions.
<jtv> stub: some, I think
<LPCIBot> Project windmill-devel build #28: STILL FAILING in 41 min: https://lpci.wedontsleep.org/job/windmill-devel/28/
<lifeless> jtv: the tti.sequence component in the sort seems unuseful to me
<lifeless> jtv: or do I misunderstand the model
<jtv> lifeless: sequence expresses the order in which the translatable strings occur in the template.  Even to someone who doesn't know or care about the text of the template, it conveys context that may be useful.
<jtv> I'm loading up the bug so I can be more specific.
<stub> bwahaha... there is a blessed SQL 'hack' that makes that final pastbin work in 1% of the time
<lifeless> jtv: so we're looking at a users contributions to the one project, is the sequence an ordinal in the file?
<lifeless> jtv: if it is, the the date sort is useless
<lifeless> jtv: if it isn't, then wouldn't most-recent-first be simpler?
<jtv> lifeless: both sort criteria are needed.
<lifeless> why?
<jtv> A single user may have submitted multiple translations for the same message.
<lifeless> and we show them all? even rejected/old/stale ones?
<jtv> Yes.  With indication of status.
<stub> compare https://pastebin.canonical.com/47090/ with https://pastebin.canonical.com/47136/ . The latter is identical except for the OFFSET 0 in the subquery, which is an unofficial query hint that lots of people use. Now I know why.
<lifeless> grah
<jtv> This is useful in evaluating a translator's work.
<lifeless> jtv: it slowly degrades over time though
<stub> c/query hint/planner hint
<jtv> It's also useful for a translator to get insight into review status.
<lifeless> stub: !!
<lifeless> stub: that may be hot
<stub> lifeless: it is. I get 20 s with the original query.
<lifeless> stub: can you do that explain on the standalone replica ?
<lifeless> which should be cold
<stub> cold, 5ms
<stub> This is silly
 * stub double checks his query
<lifeless> the 84756 lookups in the related table are the issue in the first place
<stub> lifeless: standalone is being build. The queries are strangely fast when the tables exist and have 0 rows.
<lifeless> hhhaaaa
<lifeless> ok
<jtv> I suspect the difference is mainly due to the planner selecting by language first and by user after, which the optimization barrier inverts.
<jtv> I'd expect more selectivity from selecting by translator first and then by language.
<lifeless> doing subselect vs join also did that
<lifeless> plans are in the bug IIRC
<lifeless> stub: on qastaging
<lifeless>  explain select count(*) from bug where latest_patch_uploaded is not null;
<lifeless>                             QUERY PLAN
<lifeless> ------------------------------------------------------------------
<lifeless>  Aggregate  (cost=644851.46..644851.47 rows=1 width=0)
<lifeless>    ->  Seq Scan on bug  (cost=0.00..644777.55 rows=29562 width=0)
<lifeless>          Filter: (latest_patch_uploaded IS NOT NULL)
<lifeless>     "bug__latest_patch_uploaded__idx" btree (latest_patch_uploaded)
<lifeless> stub: its indexed, and a small % of rows have that set:
<lifeless>  select count(*) from bug where latest_patch_uploaded is not null;
<lifeless>  count
<lifeless> -------
<lifeless>  29562
<lifeless> stub: so why would it do a table scan ?
<stub> small = 5-10%
<lifeless> yeah, we have 800K bugs, 30K is < 5%
<lifeless> with seqscan off:
<lifeless>  Aggregate  (cost=10000644851.46..10000644851.47 rows=1 width=0)
<lifeless>    ->  Seq Scan on bug  (cost=10000000000.00..10000644777.55 rows=29562 width=0)
<lifeless>          Filter: (latest_patch_uploaded IS NOT NULL)
<lifeless> it doesn't think the index is valid?
<lifeless> stub: can you reproduce, and/or see if prod has the same issue?
<stub> lifeless: possibly because the bug table is pretty much pinned in RAM
<stub> but yeah, on prod seq scan for pulling 4% of the bugs
<stub> lifeless: probably fixable by making it partial - WHERE latest_patch_uploaded IS NOT NULL
<lifeless> explain select id from bug where latest_patch_uploaded is not null order by latest_patch_uploaded ; grabs the index
<lifeless> stub: could you add a test index like that to qastaging ? I'm working on the +patches perf bug
<stub> lifeless: And does explain analyze show it takes longer real time?
<lifeless> checking
<stub> Building the index atm, so your checks will suck.
<lifeless> http://pastebin.com/FBRt1HPr
<lifeless> its using your new index :>
<lifeless> can you run those two to compare on prod ?
<stub> So that is quite interesting. See that PG correctly picked the plan that starts returning rows soonest? The 'order by' query runs slightly faster to completion, bug doesn't start returning rows for 240ms
<stub> which is rather stupid when this is a COUNT()
<stub> But the estimated time for the second query is higher than the first, so understandable. And the differences probably statistically insignificant.
<lifeless> stub: right, I couldn't compare counts to counts, so changed it to get th eid
<lifeless>                ->  BitmapAnd  (cost=43791.26..43791.26 rows=23966 width=0) (actual time=16801.315..16801.315 rows=0 loops=1)
<lifeless>                      ->  Bitmap Index Scan on bug__latest_patch_uploaded__idx2  (cost=0.00..397.06 rows=29562 width=0) (actual time=15.446..15.446 rows=29562 loops=1)
<lifeless>                      ->  Bitmap Index Scan on bug_duplicateof_idx  (cost=0.00..43382.43 rows=577826 width=0) (actual time=16773.721..16773.721 rows=578095 loops=1)
<stub> partial index built and table analyzed btw. Look for '_idx2' in your plans
<lifeless> yeah
<lifeless> stub: could you add a similar one for duplicateof ?
<lifeless> (where duplicateof is null)
<lifeless> I don't know that that will be sufficient
<lifeless> or...
<lifeless> perhaps index id where duplicateof is null and latest_patch_uploaded is not null
<lifeless> I wonder if we need to denorm the openness or not of a bug into the bug
<lifeless> this is the bug I'm looking at https://bugs.launchpad.net/launchpad/+bug/618392
<_mup_> Bug #618392: Distribution:+patches slow 10% of requests timing out <lp-bugs> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/618392 >
<stub> We have some indexes elsewhere on id WHERE [partial clause] to speed up joins and ordering - may well work here too.
<stub> duplicateof index built and table analyzed
<lifeless> nope, unused
<stub> and the id one
<lifeless> ah, with just duplicates by id
<lifeless> this is what its doing:
<lifeless>                ->  BitmapAnd  (cost=43791.26..43791.26 rows=23966 width=0) (actual time=20175.537..20175.537 rows=0 loops=1)
<lifeless>                      ->  Bitmap Index Scan on bug__latest_patch_uploaded__idx2  (cost=0.00..397.06 rows=29562 width=0) (actual time=16.256..16.256 rows=29562 loops=1)
<lifeless>                      ->  Bitmap Index Scan on bug_duplicateof_idx  (cost=0.00..43382.43 rows=577826 width=0) (actual time=20146.004..20146.004 rows=578095 loops=1)
<lifeless>                            Index Cond: (duplicateof IS NULL)
<lifeless> the expensive  bit is pulling all the bits out from the nondupes
<lifeless> could you drop all these indices
<lifeless> and try
<lifeless> (id) where duplicateof is null and latest_patch_uploaded is not null
<lifeless> OTOH this may be index bloat I guess
<stub> whoops... I'd created duplicateof index IS NOT NULL, not IS NULL
<lifeless> :>
<lifeless> ok, so lets try fixing that first
<lifeless> general purpose things being better, all other stuff equal
<lifeless> booyah
<lifeless> thats fast
<stub> You now have two indexes - on id - WHERE duplicateof IS NULL , and WHERE duplicateof IS NULL AND latest_patch_uploaded IS NOT NULL
<stub> which one gets used?
<lifeless>          ->  Index Scan using bug__new_patches__idx on bug  (cost=0.00..1013500.47 rows=23045 width=4) (actual time=0.086..93.803 rows=28607 loops=1)
<lifeless> 500ms for the batch, 300ms for the count
<lifeless> do you think this is reasonable to add live ?
<stub> sure
<lifeless> \o/
<lifeless> -65-1.sql ?
<stub> first, I'll drop the more complex index. Lets see if it is nearly as fast with just the id WHERE duplicateof IS NULL
<lifeless> ok
<stub> That index will be useful for other stuff too
<lifeless> yeah
<stub> lifeless: ok. try now.
<stub> yeah - 65-1
<lifeless> Time: 3089.849 ms
<lifeless> faster than it was without it I think,
<lifeless>  Aggregate  (cost=7085778.93..7085778.94 rows=1 width=0) (actual time=2905.470..2905.471 rows=1 loops=1)
<stub> ok. crappy time. Wonder if I should create both indexes or just the fast one for this case?
<stub> or is the time close?
<lifeless> 2.5 second difference
<stub> right. crappy time :)
<lifeless> 500ms vs 3000ms
<lifeless> I suggest just the tuned one
<lifeless> less chance of destablising some other query
<stub> Should be impossible to destabalize - if it is possible to use the partial index, it will be faster because it is smaller. Assuming no memory pressure on the indexes because index size is insignificant compared to total RAM.
<stub> But from change management pov, yeah.
<LPCIBot> Project windmill-db-devel build #235: STILL FAILING in 1 hr 5 min: https://lpci.wedontsleep.org/job/windmill-db-devel/235/
<lifeless> stub: https://bugs.launchpad.net/launchpad/+bug/735977 appears to be a destablisation result to me
<_mup_> Bug #735977: MaloneApplication:+bugs timeouts searching for tags across all bugs <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/735977 >
<lifeless> stub: so my paranoia is heightened
<stub> I'll do that now and repack bug_fti while I'm at it.
<lifeless> \o/
<lifeless> paste me the sql
<lifeless> and I'll make it into a branch
<stub> I'll do the -1 patch if I may - I do the patch first then translate it to live statements to run on the slaves.
<stub> less chance of screwups
<lifeless> I'm always happy for someone else to do work
<lifeless> once you've done this lets do the catchup call
<LPCIBot> Project devel build #689: FAILURE in 5 hr 7 min: https://lpci.wedontsleep.org/job/devel/689/
<stub> I should flag this patch as fixing Bug #618392 yes?
<_mup_> Bug #618392: Distribution:+patches slow 10% of requests timing out <lp-bugs> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/618392 >
<lifeless> yes
<stub> prod_1 done
<lifeless> 361 queries/external actions issued in 2.65 seconds
<lifeless> \o/
<stub> prod_2 complete
<lifeless> wiiin
<stub> prod_3 taking its time
 * stub checks for long running transactions - probably the backups at this time
<stub> yer... prod_3 blocked until the backups stop or I kill 'em
<lifeless> thats master right ?
<stub> yes
<lifeless> up to you
<lifeless> are the backups offsite?
<spm> s/u/a/ <== stub
<stub> spm: eh?
<stub> The reapers are killing archivepublisher jobs for about a day now.
<spm> stub "... or I kill 'em" ergo stub ==> stab. s/u/a/ :-)
<stub> lifeless: I think we can wait another 3 hours until they complete.
<lifeless> sure
<lifeless> so, call?
<stub> lifeless: sure
 * stub kills his u1 sync
<stub> So I installed wonderwall as a workaround for some throttling issues, and it seems to be helping my system a lot. Lets see if this call is stable - Skypeout was crappy yesterday before that install.
<lifeless> I'm here
<lifeless> stub: I'm here when your network comes back :>
<lifeless> stub: I can hear you mostly fine
<stub> Sre. I'm watching the network monitor. I'm seeing a sine wave
<stub> Like there is some sort of really crappy traffic shaping going on, or I'm only getting
<lifeless> of traffic or signal ?
<lifeless> stub: brewaking up?
<lifeless> stub: hows the graph?
<stub> still broken
<stub> think it is isp conjestion
<LPCIBot> Project windmill-devel build #29: STILL FAILING in 1 hr 4 min: https://lpci.wedontsleep.org/job/windmill-devel/29/
<lifeless> sine wave?
<lifeless> stub: another 'fun' one for you - https://bugs.launchpad.net/launchpad/+bug/777601
<_mup_> Bug #777601: BugComment:EntryResource timeouts <dba> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/777601 >
<stub> lifeless: so that was qa staging yesterday. Was something happening there at 11:18:48 UTC like a database update?
<lifeless> stub: huh, that was prod
<stub> really?
<lifeless> OOPS-1950DQ231
<lifeless> DQ
<stub> oic. 'last seen' was qastaging
<lifeless> so (IIRC) wampee
<stub> I saw QASTAGING in big type :)
<lifeless> :>
<stub> I'm not aware of any activity that would explain that. I'll need to dive into the logs.
<stub> But first a shower. I'm all sticky.
<StevenK> TMI, stub
<allenap> lifeless: Okay, I'll follow that up (the sync_packages thing). Nothing is creating jobs that use the sync_packages user yet so it's good someone saw this now before we do start creating jobs. Thanks.
<bigjools> morning dudes
<mrevell> hello!
<StevenK> allenap: O hai! Can you look at http://pastebin.ubuntu.com/603553/ ? It's the last bit for the mega-branch you've been reviewing.
<StevenK> allenap: This cleans up the UI tests to work
<allenap> StevenK: Sure.
<bigjools> wgrant: did you see the MP I assigned you?
<allenap> StevenK: _create_child_and_parent() is defined several times. If they're all the same perhaps you could factor them out?
<allenap> StevenK: Otherwise it looks good, r=me.
<StevenK> allenap: Thanks!
<StevenK> allenap: I was not amused at all of the duplication of derived_series = ...
<allenap> No :)
<LPCIBot> Project windmill-devel build #30: STILL FAILING in 1 hr 4 min: https://lpci.wedontsleep.org/job/windmill-devel/30/
<wgrant> bigjools: Ah, sorry missed it in the middle of lots of other review emails that I didn't care about.
<wgrant> Looking now.
<bigjools> wgrant: heh, I unsubscribed from the review list a while ago
<wgrant> bigjools: Hmm, that simplifies the code a little bit, doesn't it...
<bigjools> a tad
<bigjools> dunno what that guadalinux stuff was in there for
<wgrant> For DDs with binary copies, without considering at all how it would work.
<wgrant> StevenK/anyone: https://code.launchpad.net/~julian-edwards/launchpad/slow-build-hunt-bug-777183/+merge/59968 would like to be mentored.
<lifeless> allenap: cool thanks
<adeuring> good morning
 * stub wonders why launchpadlib is pulling credentials from kwallet when he is using Unity
<bigjools> the irony, I am using KDE and it pulls them from gnome-keyring
<LPCIBot> Project devel build #690: STILL FAILING in 5 hr 11 min: https://lpci.wedontsleep.org/job/devel/690/
<bigjools> lifeless: around?
<stub> I wonder if it remembers somewhere where you first used it?
<bigjools> IIRC it depends on the ordering of importing dependent modules, whichever works first it uses
<bigjools> you can make a .keyringrc as well
<stub> I'm sure there must be an environment variable to sniff for a heuristic.
<bigjools> just the config I think
<bigjools> StevenK: still there?
<bigjools> ok, allenap?
<lifeless> bigjools: vaguely
 * stub wonders what a .keyringrc file looks like
<bigjools> lifeless: if you have time, can you mentor wgrant's review of my branch please?
<bigjools> https://code.launchpad.net/~julian-edwards/launchpad/slow-build-hunt-bug-777183/+merge/59968
<bigjools> 81 lines, mostly removals
<bigjools> stub: the file's actually called keyringrc.cfg
<bigjools> you can use it to override the backend
<bigjools> I think it's supposed to live in ~ which is somewhat annoying
* allenap changed the topic of #launchpad-dev to: Merge to devel are open again | https://dev.launchpad.net/ | On call reviewer: allenap | https://code.launchpad.net/launchpad-project/+activereviews
<bigjools> allenap: ooo I had no idea PG had a notification mechanism
<bigjools> stub: apparently you didn't like it though
<stub> bigjools: Unity? I worked out how to turn off the scrollbars that were driving me batty when they popped up and am now starting to appreciate it. KDE was nice, particularly plasmoids built in and working and some decent ones shipped, but still a little busy for me or something.
<bigjools> stub: I was talking about http://www.postgresql.org/docs/8.4/interactive/sql-notify.html
<bigjools> :)
<stub> oh... notification. Its fine. The only reason I don't like it is we really shouldn't be spending time reimplementing rabbitmq badly.
<bigjools> maybe
<bigjools> but considering we already did that anyway ...
<bigjools> Job *cough*
<stub> bigjools: I can't quite recall if it works outside transaction boundaries - if LISTEN is stuck inside a transaction, we could be shooting ourselves in the foot with long running transactions.
<bigjools> I was thinking more of doing it in a twistd process, we have a bit more control over transactions
<stub> Because if we do something wrong once, we should keep making the same mistake until everyone believes it is the right thing to do.
<bigjools> heh
<stub> Or am I thinking of politics?
<lifeless> if you want queues
<lifeless> land my rabbitmq branch
<lifeless> :)
<bigjools> lifeless: well we keep adding polling jobs everywhere, it's about time ...
<lifeless> I have a branch, tests all pass locally, something in ec2 hates it and I haven't had time to debug
<rvba> wgrant: bigjools: Just pushed the modifications to the branch William marked as bad earlier today. Could you have a look? (http://bazaar.launchpad.net/~rvb/launchpad/change-perm-sync/revision/12950)
<rvba> I changed the MP's status back to WIP but the partial diff does not get updated.
<bigjools> rvba: you need a new branch + MP
<rvba> ok
<bigjools> rvba: I would add a comment explaining why lp.append is needed for the security updates
<LPCIBot> Project windmill-db-devel build #236: STILL FAILING in 1 hr 5 min: https://lpci.wedontsleep.org/job/windmill-db-devel/236/
<bigjools> it's a bit of a hack unfortunately
<wgrant> bigjools, rvba: I think we will be able to remove the hack this year, but until then it needs to stay (and probably have a test)
<bigjools> yeah he has a test
<bigjools> rvba: I would move that test somewhere else though, let me think
<wgrant> s-i-s seems to be basically P3As + a stack of hacks that all need to be genericised for DDs to work properly.
<lifeless> 'special cases'
<bigjools> rvba: lib/lp/soyuz/scripts/tests/test_copypackage.py
<wgrant> lifeless: Have you *seen* them?
<lifeless> some
<rvba> bigjools: ok
<lifeless> just noting that they may me lacking the 'elegant solution to a non trivial problem' aspect of 'hack'
<bigjools> rvba: I'd be tempted to also move the syncing tests out of test_distroseries
<bigjools> that's not the first place I'd look for them
<bigjools> but the waters are already muddied by this feature being half in lp.registry and half in lp.soyuz
<wgrant> It should be entirely in lp.soyuz, or the trees should be merged :/
<wgrant> Why was it in lp.registry to start with?
<bigjools> wgrant: noodles put the DSD stuff there
<wgrant> Ah, right.
<wgrant> That seems so long ago now :/
<lifeless> wgrant: are you around tomorrow?
<bigjools> about a year :)
<wgrant> lifeless: Unless they've instantiated a new public holiday while I wasn't looking.
<bigjools> that happened here!
<stub> We only get that on military coups.
<bigjools> The Royal Wedding could be considered a military coup
<lifeless> wgrant: I'd like to run some concepts I'm tossing round - starting to socialise - past you; I'd use bigjools but ETIMEZONE
<rvba> bigjools: right, I'll leave one to check for UI notifications and see if some of the other tests can be moved to test_copypackages.
<lifeless> wgrant: so skype/mumble at some convenient point of th day for you
<bigjools> rvba: thanks
<bigjools> lifeless: I can be around later today my time if you want
<lifeless> bigjools: I'm doing big picture sketching and looking to understand more of the bits of soyuz that glue together
<bigjools> lifeless: you definitely want me then :)
<wgrant> lifeless: Sure, whenever.
<lifeless> bigjools: I'd be delighted, but I plan to sleep till i wake tomorrow, after the 5:45 start this morning
<lifeless> bigjools: I don't want to interrupt your evening unnecessarily
<wgrant> Sleep till you wake.
<wgrant> What a novel concept.
<rvba> bigjools: btw, I'm not sure I know why exactly "lp.append is needed for the security updates" ... it's a hack all right ... but why is it there?
<bigjools> lifeless: nae prob
<lifeless> wgrant: theres a book on it
<bigjools> rvba: it was the only reasonable way for the security team to get permission to copy to -security at the time
<lifeless> bigjools: if you are around in my morning and want to chat, I'd be delighted; from experience (this will be the 4th/5th time I've run it up the flagpole) its about a 45-60 minute topic
<bigjools> heh
<lifeless> bigjools: failing that, we can make a time next week if you like
<bigjools> I'll pop in to irc tonight
<wgrant> rvba: Basically, the security team needs to be able to copy any package into the Security pocket, even if they lack upload privileges. And *only* the security team can be permitted to use syncSource into the primary archive, since it bypasses the queue and announcements.
<lifeless> bigjools: to give you context, with performance a Solved Problem, I'm starting to look longer term at our architecture
<lifeless> bigjools: at how we evolve the system, what platform we want to be on, and how we can break out of the monolithic situation we're in safely
<rvba> wgrant: bigjools ok. Thanks, I'll had a comment then.
<bigjools> lifeless: excellent, I have a lot of ideas and semi-plans about that
<wgrant> lifeless: I am relieved to hear that you are looking at this.
<rvba> s/had/add/
<wgrant> rvba: Thanks!
<bigjools> a lot of our architecture was designed around more machines being in the pipeline
<bigjools> rvba: thank you
<lifeless> bigjools: its nearly 11pm and I've worked straight through except for dinner with Lynne; I guarantee 0% retention of anything you say now :>
<wgrant> rvba: We'll want to redo what you did in a few months, I suspect. But we can't get ahead of ourselves :(
<bigjools> lifeless: I hear you
<bigjools> wgrant: baby steps
<lifeless> anyhow, I'll be active here again in 9ish hours.
<lifeless> night all
<bigjools> nn!
<wgrant> Night lifeless.
<wgrant> lifeless: I'd like to be around for this specialised deploy, so please don't try it before I'm around.
<bigjools> wgrant: just sent you email so you can sanity check the steps I think are necessary for syncs via the queue
<wgrant> bigjools: Looks sane.
<wgrant> bigjools: We'll need to shuffle the upload policy stuff around a bit... but it should be pretty much OK.
<wgrant> I think +queue and the queue tool should mostly work, as long as PackageUpload.displayname works.
<bigjools> yeah, I want to do the policy stuff in such a way that it's easy to convert for generic suites
<stub> Speaking of holidays, apparently it is coronation day and I'm not supposed to be here. I guess I'll be slacker than usual tomorrow.
<wgrant> And acceptFromQueue does the right thing,
<wgrant> But if not, easy enough to fix.
<bigjools> stub: how will we notice?  /me hides
<bigjools> wgrant: right, the change will be in acceptFromQueue
<bigjools> although I expect we'll need to amend both for display purposes so it says it's a sync
<wgrant> Yeah.
<wgrant> but it's not going to be as hard as I have long feared.
<wgrant> Mostly because of that convenient job.
<wgrant> StevenK: Did you get anywhere with the overrides?
<bigjools> 20 bugs away from #777777
<wgrant> bigjools: Have you had any more thoughts about how to arrange the generic copier underpinnings?
<bigjools> no
<bigjools> too busy thinking about this project :)
<wgrant> Yeah.
<bigjools> now, where's the damn table for copy jobs
<wgrant> Is it DistributionJob or ArchiveJob or similar?
<wgrant> lib/lp/soyuz/model/packagecopyjob.py:class PackageCopyJob(DistributionJobDerived):
<bigjools> Bug #776283
<_mup_> Bug #776283: PackageCopyJob needs separate archive and series columns for query purposes <derivation> <Launchpad itself:Triaged> < https://launchpad.net/bugs/776283 >
<bigjools> it uses DistributionJob
<wgrant> gmb: Well done!
<gmb> wgrant: Yeah. Trigger happy, premature filing.
<gmb> The failure state of "clever" is "arsehole"
<wgrant> Yeah.
<wgrant> Still, 4 bugs to go!
<gmb> I'll leave it to mpt.
<gmb> And stick to being doggedly successful instead of smart.
<wgrant> That sounds like a plan.
<wgrant> Any Yellow people want to handle the bug in #launchpad?
<wgrant> YUI doesn't seem to like + in an id selector.
<wgrant> bigjools: Are you going to separate the two different facets of upload checking?
<wgrant> Some of the checks are uploader policy (eg. signature checks), while others are distroseries acceptance policy.
<wgrant> They are currently conflated, but that doesn't really work.
<wgrant> Is unmuting a bug meant to give me the option to unsubscribe the team?
<wgrant> That seems sort of the opposite action...
<bigjools> wgrant: yes
<bigjools> uploader policy is fixed for the most part
<wgrant> bigjools: Right. Or it's controlled by an option on process-upload.
<wgrant> Not series-specific.
<bigjools> exactly
<wgrant> So it must be split.
<bigjools> that's the plan
<wgrant> Excellent.
<bigjools> anyone know if you can change the billing date for AWS?
<wgrant> You mean so it appears more than a day before expenses are due?
<wgrant> That would be lovely.
<wgrant> But I don't think so.
<bigjools> they bill just at the exact wrong time for me
<henninge> danilos: could you comment on bug 196679, though?
<_mup_> Bug #196679: ValueError on +language-packs <lp-translations> <oops> <Launchpad itself:In Progress by henninge> < https://launchpad.net/bugs/196679 >
<henninge> danilos: I mean, do you have a hint for me if this still needs to be done?
<danilos> henninge, I thought jtv and you got the hang of it (I remember seeing his comment about how it's still there somewhere)
<henninge> danilos: in a mail but it sounded much like guess work on his part.
<henninge> Just wondering if you had more insight.
<danilos> henninge, let me take a look
<wgrant> bigjools: Have you seen the repeated archivepublisher transaction killings over the past couple of days?
<LPCIBot> Project windmill-db-devel build #237: STILL FAILING in 1 hr 6 min: https://lpci.wedontsleep.org/job/windmill-db-devel/237/
<bigjools> wgrant: no, I'm not paying any attention to oopses
<bigjools> and, BTW, ****ing keyboard drivers
<wgrant> bigjools: Not OOPSes, but pgkillidle spam. The first was on the 3rd, and there have been ~8 since.
<wgrant> I can't find anything in the logs.
<wgrant> Don't even know which host it's from.
<bigjools> wgrant: I suspect it's to do with the timeout getting lowered
<bigjools> if that happened yet
<wgrant> Extremely unlikely.
<wgrant> These transactions had been idle for half an hour.
<bigjools> that it happened?
<wgrant> It's been 9s for a week now.
<bigjools> not that timeout
<bigjools> there's a script timeout IIRC
<wgrant> Hmm.
<danilos> henninge, it seems it should be able to reproduce the bug with more than 15 language packs (shortlist default), and the base pack not being in the list that shortlist returns, as jtv noted as well
<bigjools> I vaguely remember stub saying something about lowering it and panicking
<bigjools> and then forgot about it
<wgrant> Heh
<henninge> danilos: I see, I'll try that. thanks
<bigjools> wgrant: so I'd check with stub to see if he changed anything
<stub> eh?
<stub> wgrant: I'll grab the hosts for you - they should be in the pg logs
<wgrant> stub: Oh, that would be great, thanks.
<danilos> henninge, I think the right fix would be to use a larger shortlist value there (find what's the most we had for a distro release)
<wgrant> stub: Then we can hopefully identify the process from the ps logs.
* jcsackett changed the topic of #launchpad-dev to: Merge to devel are open again |
<jcsackett>           https://dev.launchpad.net/ | On call reviewer: allenap |
<allenap> Oops.
<allenap> :)
<jcsackett> crap, i didn't even hit enter.
* jcsackett changed the topic of #launchpad-dev to: Merge to devel are open again | https://dev.launchpad.net/ | On call reviewer: allenap, jcsackett | https://code.launchpad.net/launchpad-project/+activereviews
<deryck> Morning, all.
<deryck> henninge, ping for standup
<bigjools> wgrant, I am going to start the mawson DB upgrade later today, you don't need it do you?
<LPCIBot> Project windmill-devel build #31: STILL FAILING in 1 hr 5 min: https://lpci.wedontsleep.org/job/windmill-devel/31/
<wgrant> bigjools: I need to do some minor QA on it tomorrow.
<wgrant> bigjools: So probably best to run it over the weekend...
<wgrant> Or I can start it when I'm done tomorrow.
<wgrant> That's probably best.
<bigjools> wgrant: what QA?
<wgrant> r12979
<bigjools> I need DF *done* before end of Sunday so I can have it ready for monday
<wgrant> buildd-manager path change.
<wgrant> Ah, true.
<bigjools> you can QA that on staging
<wgrant> I guess we can do this on staging.
<wgrant> Yeah.
<wgrant> OK, blow it away!
<bigjools> cheers :)
<wgrant> I guess we also need to clean the librarian out, but can leave most of the archive.
<bigjools> librarian purging is part of my monkey-following script
<wgrant> Great.
<bigjools> wgrant: for this policy checker I am thinking to sort of copy the uploadpolicy one but makeit  a) use DB objects, b) only have distro policy checks.  Then add a PackageUpload.checkPolicy(policy) or something.
<wgrant> bigjools: Can't it basically be encapsulated in DistroSeries or Archive?
<wgrant> There's only one or two methods that need to be exposed.
<bigjools> no
<bigjools> well
<bigjools> I'd rather it was a totally separate module
<bigjools> then it can be re-used in more ways
<wgrant> Do we need to replace anything other than autoApprove and autoApproveNew?
<bigjools> at the moment, no, but I expect it to grow
<wgrant> Right.
<bigjools> rejectPPAUploads, for example
<bigjools> maybe announcelist
<wgrant> Most of that is archiveuploader-specific.
<wgrant> Except for announcelist, which is just wrong.
<bigjools> it should be distro-specific
<bigjools> s/it/they/
<bigjools> now,  the hardest part has just arrived.  What do I call it and where should it live :)
<wgrant> It is similar to archivedependencies, so it might belong alongside it in adapters/
<bigjools> adapters is a horrible name but there's nowhere else better that I can see
<bigjools> I mean, packagecopier is in scripts/  ffs
<wgrant> Hmm, I argued with you about packagecopier a while ago and you said it belonged in scripts/ :)
<bigjools> did I? :)
<deryck> henninge, we don't hear you any more
<henninge> deryck: I still hear you
<bigjools> wgrant: bugger, test failure from the getBuildByArch change
<wgrant> bigjools: Ew, which?
<bigjools> publishing.txt and testDuplicatedBinaryUploadGetsRejected
<wgrant> Hmm. The second one is a braindead test.
<wgrant> Copying to the same distro and archive and expecting a new build to be created.
<wgrant> Nonsensical.
<bigjools> not look at it yet, just going in now
<wgrant> publishing.txt seems to be the same.
<benji> abentley: will you be able to QA bug 747844 today?  I have a revision behind it that I'd like to get released.
<_mup_> Bug #747844: process-mail oopses with TypeError due to non US-ASCII chars in the email address <oops> <process-mail> <qa-needstesting> <Launchpad itself:Fix Committed by abentley> < https://launchpad.net/bugs/747844 >
<wgrant> It copies from one series in a PPA to another.
<wgrant> ANd expects builds to be created.
<bigjools> wgrant: huh, the description is insane
<abentley> benji: yes.  I am QAing it right now.
<bigjools> "source gets copied to another suite in the same archive..... The binary rebuild"
<bigjools> !
<benji> abentley: great! thanks
<bigjools> wgrant: that's completely bong, wtf
<wgrant> bigjools: Yeah, so it asserts that the duplicated build will fail to upload.
<wgrant> bigjools: It's a good check.
<wgrant> But it relies on a bug: that the build is created at all.
<wgrant> Need to create the build manually.
<benji> rvba: will you be able to QA bug 775529 today?  I have a revision behind it that I'd like to get released.
<_mup_> Bug #775529: Synching to a released series should put packages in -updates. <derivation> <qa-needstesting> <Launchpad itself:Fix Committed by rvb> < https://launchpad.net/bugs/775529 >
<wgrant> And publishing.txt needs to be reworded to use a different archive.
<wgrant> bigjools: DF is still usable?
<wgrant> I'll QA that thing now.
<bigjools> wgrant: yes, still waiting for the dump
<rvba> benji: I would need DF to be up for that ... bigjools?
<rvba> bigjools: can you update DF?
<bigjools> it's up, but we can defer QA since it won't block rollout
<bigjools> mark it untestable for now
<rvba> benji: bigjools will do.
<benji> awesome, thanks
<wgrant> benji: You need to deploy at least 12980 due to a bad rev, and 12979 has an unusual rollout requirement on LPS -- could you make sure that's done if you request one before I'm around again?
<benji> wgrant: I'm doing a lazy release.  I'm just lining things up so my revision will be ready when you do your deploy.
<wgrant> benji: Ah, OK. That makes things easier.
<benji> for the both of us ;)
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #691: FIXED in 5 hr 8 min: https://lpci.wedontsleep.org/job/devel/691/
<abentley> losa: please run process-mail for qastaging
<hloeung> abentley: hi, give me a sec...
<abentley> hloeung: thanks.
<wgrant> Ah, I was wondering why DF was so slow.
<wgrant> Driven into iowait by a network file copy. Yay.
<hloeung> abentley: done... see https://pastebin.canonical.com/47159/
<wgrant> bigjools: I'm giving up for now, will do it on staging tomorrow. Do with mawson what you will.
<abentley> hloeung: Hmm.  That doesn't look right, and it hasn't generated the outgoing mail I expected, either.
<abentley> hloeung: that was definitely qastaging, not staging?
<hloeung> abentley: yep, qastaging
<bigjools> wgrant: ok
<abentley> hloeung: could you try running it on staging, please?
<hloeung> abentley: process-mail runs every 10 minutes on both staging and qastaging
<abentley> hloeung: Oh.  I was under the impression that no scripts run on qastaging.
<abentley> hloeung: I sent mail to 5977@bugs.staging.launchpad.net more than 20 minutes ago, and it still hasn't appeared in the staging mail folder.
<bigjools> wgrant: publishing.txt is also on crack
<bigjools> it's copying intra-archive and expecting rebuilds
<jtv> henninge, mrevell: one very small note about the sharing announcement: where it says "That sharing also works between different versions of a project or package:" it may be helpful to say "already" instead of "also" (to set the right expectation, i.e. that this is not the part that has changed most recently).
<mrevell> jtv, Thanks, that's a good suggestion.
<jtv> Also, "series" is one of those things that are obvious to us devs but I'm worried about how clear they are to others.  I'd prefer to say "release series" a lot of the time to avoid leaving those "not in the know" behind.
<jtv> mrevell, in case you were already off typing: ^  :)
<mrevell> ah, thanks jtv, I did wonder about that but then I kinda thought that we don't have much of a better alternative.
<jtv> Release series.
<jtv> When we talk internally, that's needlessly verbose.  But every time we omit it, a potential user may go "what are these people talking about?  All I want is to host my project!"
<jtv> And consequently, strangle a kitten.
<jtv> mrevell: also, the the dreaded repeated word across line line breaks: "opt"
<mrevell> ah
<jtv> (It is a dark day for travelers worldwide.  All the sudokus I picked up on this trip were so easy that they might as well have said "each little square should have a number in it, but some are empty.  Can you see where they are?")
<jcsackett> allenap: can i get you to take a look at https://code.launchpad.net/~jcsackett/launchpad/spam-button-ui/+merge/60072
<allenap> jcsackett: Sure.
<jtv> mrevell: moving on, the paragraph starting with "Previously, Ubuntu took translation templates" introduces two consecutive notions with the word "this" (which makes it a bit unnatural to read) and uses the word "allowed" to describe an undesirable effect (which may confuse readers who lack confidence with the subject matter).  Especially because it also uses the word "diverge" in a way that's hard to avoid, but doesn't sit quite right with the 
<jcsackett> allenap: i believe the diff appears pretty big, but a goodly junk of that is boilerplate for javascript tests and the fact that i renamed a test file.
<allenap> jcsackett: No worries. It's such a cool feature I'd read it twice.
<jcsackett> allenap: :-)
<mrevell> thanks jtv, this is very helpful.
<jtv> mrevell: how about moving the cause ahead of the consequence there?  "New upstream translations made after that initial import had no way [...].  Thus Ubuntu often ended up with different translations as translation work upstream and in Ubuntu continued independently over Ubuntu's 6-month release cycle."
<mrevell> that's great, much clearer jtv
<jtv> mrevell: explaining things is important, and any improvement is multiplied by the number of readers.  That's why I always appreciated your documentation work so much.
<mrevell> Ah, thanks man. Looks like I'll have to buy you a beer in Dublin :) Although, I seem to remember that buying beer in Dublin requires a hefty bank loan first.
<jtv> Well let's pool our mortgages and try the Guinness.  :)
<sinzui> jcsackett: do you have time to mumble?
 * bigjools will be putting some beverages in the car
<jcsackett> sinzui: i do. one moment.
 * jcsackett fires up mumble and prays
<jtv> bigjools: bringing your own lukewarm English beer to Dublin?  How does that make things any better?
<bigjools> jtv: you, as a Dutch man, have no right at all to talk about beer
<jtv> (Apart from the slight reduction in lukewarm English beer the United Kingdom will experience)
<jtv> Ohhh yes.  Bring it on!
<jcsackett> sinzui: i am here, but it seems that you cannot hear me.
<mrevell> haha
<bigjools> if you want to drink beer served at sub-zero temperatures, be my guest!  But I prefer mine to actually taste of something ;)
<jtv> bigjools: I appreciate that English cooking gave you a taste for cat's urine, but proper beer is best served at 4 degrees centigrade.
<bigjools> jtv: you lie.  like a cheap carpet.
<bigjools> the only reason it's served that cold is so you can tell it from urine my friend
<jtv> bigjools: if you wanted to hit me where it hurt, you would have said "is someone trying to teach me about beer from the United States of America?"
<jtv> Wouldn't have hurt _me_, but at least it would have hurt _someone_ who deserved it.
<bigjools> jtv: anyone who drinks beer at 4C deserves it :)
<sinzui> what's to know? American is like making love in canoe.
<sinzui> We do have real breweries now, but that does not mean you will find drinkable that beer in an established
<bigjools> I did find some great beer in the US once
<jtv> bigjools: thank you, I'm glad you don't begrudge me my delicious cool beer.
<bigjools> it had been imported from England :D
<jtv> Except the Americans probably knew to refrigerate it, and it was a whole new experience of pain-free beer-drinking.
<huwshimi> Actually the states is where a lot of the micro brewers say modern beer is at
<bigjools> jtv: get back in your box, troll!
<huwshimi> They influence a lot of beer these days
<huwshimi> particularly a lot of pale ales
<jtv> Face it, mainland Europe figured out how to make lager taste good while the Brits mumble out ale and "proper beer" into their warm, smelly pilsner ripoffs.
<jtv> Ahhh this feels good.
<jtv> Even American coffee can make you irritable if you drink enough of it.
<huwshimi> Something like the Sierra Nevada Pale Ale is amazing and has influenced a bunch of amazing beers in Aus
<huwshimi> I used to hate on American beer, but you can find crap beer anyway
<jtv> I'll see if I can get some.  :)
<huwshimi> jtv: You should
<jtv> mrevell: I can't escape the impression that the blog post is trying to go in multiple directions at once.  Is it highlighting the new sharing, or more the whole new data flow that emerges from it?  The bit about branch imports is at the same time useful and a bit of a side track.
<jtv> mrevell: it may help to separate the two aspects of the story.  "What has just changed?  How does it compare to existing sharing features?  What do these and other recent improvements add up to in terms of data flow?"
<abentley> benji: qa-ok
<benji> abentley: thanks!
<jtv> mrevell: Not very different from what you do now, except I think it will make it clearer what each individual part of the text is driving at.
<mrevell> jtv sorry otp
<jtv> OK
<rvba> allenap: could you take a look at https://code.launchpad.net/~rvb/launchpad/change-perm-sync2/+merge/60077 ? (I think you are a good candidate to review this, because you've heard the whole story about this branch ;))
<rvba> bigjools: ^^
<allenap> rvba: Sure.
<allenap> rvba: I'm doing another review right now, but I can do your's once I've finished.
<rvba> allenap: great, thanks.
<jtv> bigjools: when we discussed creating distroseries indexes from the regular publisher cron, you mentioned that the manual verification steps were a bit of a holdover from when this was all a much bigger deal.  But since this goes into the new python-based publish-ftpmaster (another big change), shall I just leave the notification of release managers in place for the time being?  I can make it say something like "please verify manually."
<bigjools> TDD = Tedious Driven Development with our test layers
 * jtv adds TLA
<bigjools> which is turning into RDD, Rage Driven Development
<jtv> bigjools: hang on, I was already uploading TDD
<bigjools> jtv: you could ask them if they would like to keep it!
<bigjools> I honestly don't know
<jtv> bigjools: what I mean is, it'd be good to keep their existing procedure for at least one more cycle, but more to ensure that my changes didn't screw things up.
<bigjools> jtv: well sending them an email is not existing procedure, so I'd check about that since it might not be necessary.
<jtv> The email is a relatively superficial change, I hope--from "run this and then verify" to "wait until the system tells you this has been run and then verify."
<jtv> But as long as there's a manual step triggered by completion of an automated step, I think the notification makes sense.
<jtv> I'll ask cjwatson though.
<bigjools> ok cool
<jtv> cjwatson, are you here?
<cjwatson> jtv: we'll likely do the manual checks whether you want us to or not :-)
<jtv> Good to hear.  :-)
<jtv> The question is:
<jtv> now that we're automating the manual "publish-distro.py" runs to create indexes for a new series,
<jtv> do you want Launchpad to notify the release managers when that has been done?
<cjwatson> oh, we don't need e-mail about that, no
<cjwatson> we are already thoroughly used to monitoring publisher activity
<jtv> Well, that certainly makes things simpler!
<cjwatson> e-mail> or any other notification
<cjwatson> there are plenty of existing Ubuntu processes that involve waiting for a publisher cycle to complete
<jtv> Okay, then I'll rip out the email and you'll just have to assume that the regular publish-ftpmaster run will, the first time around, just initialize the indexes and quit.
<jtv> (By the way, those publish-ftpmaster runs will be a bit faster now because of changes in process architecture and cache management)
<cjwatson> IOW we no longer need to stop the publisher and run publish-distro by hand?
<jtv> Definitely no longer the latter; let me just verify the former.
<cjwatson> (the first publish-distro run already created indexes, it's just that it had to be given careful flags to do so)
<cjwatson> well, and that we had to run initialise-from-parent beforehand, which I understand is going away
<jtv> The first publish-distro run already created indexes, but that's currently a manual run.  We're talking about moving that into the regular cron job.
<jtv> Yes, initialise-from-parent will be going away AIUI but I'm still fuzzy on the details there so can't make promises on whether you'll still need to stop the cron jobs.
<jtv> But I'm at least taking steps 12-14 out of that window.
<cjwatson> you're not taking out step 13, because that's not in Launchpad's gift to remove. :-)
<cjwatson> but ack on 12 and 14
<cjwatson> that's what I meant by careful flags (-A)
<jtv> I am taking step 13 out of the disabled-cron window, but not out of the process.  Steps 12 & 14 will become automated.
<cjwatson> having the publisher notice that the suite hasn't been published yet and Just Doing It is all that needs to happen to remove 12 and 14.
<jtv> Exactly; that's what I'm working on.
<bigjools> everything's frozen anyway so no harm by JFDI
<cjwatson> i.e. bug 55211
<_mup_> Bug #55211: shouldn't require careful publisher run when cloning new released distrorelease <lp-soyuz> <new-release-cycle-process> <soyuz-publish> <Launchpad itself:Triaged> < https://launchpad.net/bugs/55211 >
<bigjools> 5 digit bug!
<mrevell> Hey jtv, I'll run up another draft of the announcement
 * cjwatson closed two five-digit bugs just in the last week or so
<cjwatson> they're less rare than people seem to think ... :-)
<jtv> ack mrevell
<bigjools> we're probably unwittingly closing a bunch with derived distros
<jtv> I'll mark that one as in progress then.
<jtv> bigjools: if I make the initial-run check check for Frozen status, then the change won't affect previously released distroseries.
<bigjools> sounds very sane
<jtv> bigjools: it probably doesn't make much sense to have a feature flag for this then, either.  The work will simply come with the new publish-ftpmaster.
<bigjools> yup
<jtv> yuuuup
<bigjools> phrasing!
<allenap> rvba: I'm not going to get to your branch until later this evening. Is that okay?
<jtv> It's not a competition, bigjools!
<rvba> allenap: of course, no worries.
<allenap> rvba: Thanks, sorry about that.
<bigjools> I am executing a remarkably satisfying sql statement on mawson
<bigjools> DROP DATABASE launchpad_dogfood
<cjwatson> jtv: as long as it's clear that the Frozen status doesn't apply only to just-opened series
<jtv> cjwatson: otp, brb
<poolie> bigjools, :)
<poolie> should i have a bug for even cleanup type landings, like https://code.launchpad.net/~mbp/launchpad/mail-cleanup/+merge/58867 ?
<bigjools> poolie: it's getting the last laugh though, the production restore is failing!
<LPCIBot> Project windmill-devel build #32: STILL FAILING in 1 hr 6 min: https://lpci.wedontsleep.org/job/windmill-devel/32/
<jcsackett> allenap, rvba: i can take rvba's MP.
<jcsackett> unless there is a particular reason you should do it, allenap.
<LPCIBot> Project windmill-devel build #33: STILL FAILING in 41 min: https://lpci.wedontsleep.org/job/windmill-devel/33/
<jcsackett> allenap: saw your notes on my MP; i concur on the on and failure success handlers; meant to connect them up and clearly forgot.
<jcsackett> regarding the permission checks tho, i was under the impression that check_permission did cache permission stuff on it's own. i take it i'm laboring under false info?
<LPCIBot> Project windmill-db-devel build #238: STILL FAILING in 1 hr 5 min: https://lpci.wedontsleep.org/job/windmill-db-devel/238/
<benji> jcsackett or allenap: hi guys, I have a very small branch for you to review: https://code.edge.launchpad.net/~benji/launchpad/bug-777766/+merge/60100
<jcsackett> benji: i can look at it in a bit, but i'm currently looking over a 1400+ diff. :-P
<benji> cool, thanks
<jcsackett> np. just wanted you to know of the wait. :-)
<sinzui> Why do we keep seeing edge URL in reviews. That env is deprecated, soon to be obsolete?
<jcsackett> benji: i am looking at your branch now.
<benji> sinzui: I used to have bookmarklets with edge in them, and prompted by you I just found "edge" in one of my MP-related scripts which was the source of the "edge" above
<sinzui> ah
<allenap> jcsackett: Ah, I didn't see your comment about taking rvba's branch. I'm back now so I can do it, but thanks for the offer.
<jcsackett> allenap: it's done. :-)
<allenap> jcsackett: Oh, awesome :)
<jcsackett> benji: only one question on your branch. does the crazy character mash that is base64 show up in a url now? or is this just in ids? from the test, i'm assuming the latter.
<benji> jcsackett: yeah, it's just in IDs; user's wont see it
<jcsackett> benji: good stuff. :-) r=me, then.
<benji> I just used the URL base 64 because instead of + and / it uses - and _ (so it's ID-safe (except for the stuping paddin characters (which I remove after the fact))).
 * lifeless stretches
<lifeless> wgrant: if its specialised, is it on unusual deployment notes ?
<LPCIBot> Project windmill-devel build #34: STILL FAILING in 1 hr 4 min: https://lpci.wedontsleep.org/job/windmill-devel/34/
<abentley> lifeless: is it true that testtools doesn't provide a matcher for regular expressions?
<lifeless> I'm not aware of such a matcher; it would be great to have one
<lifeless> matchers were an experiment, one I think that has panned out well, but we haven't gone and migrated all the custom asserts etc into matchers
<lifeless> I was thinking yesterday that its about time we did that
<abentley> lifeless: Ah, I see.  It seemed like an odd oversight.
<lifeless> abentley: we started with just a couple - equals and doctestmatches
<lifeless> and have grown from there as people found them useful and nice to work with
<lifeless> flacoste: what are these wiki edits all about
<lifeless> e.g. ... +  * r9242  [r=jtv][ui=none][bug=297458] For wgrant: export bug nominations.
<flacoste> lifeless: testplans -> Trash
<lifeless> flacoste: but they are being edited not deleted
<flacoste> lifeless: nah, i'm using Delete Page
<flacoste> not sure how the notification looks liek
<lifeless> -weird-
<lifeless> I can see its gone now I try the page url itself
<lifeless> let me forward you the notification so you can go WTF
<flacoste> lifeless: i don't nee dmore WTF moments :-)
<lifeless> sent :>
<flacoste> lifeless: Moin crack :-)
<lifeless> yeah
<lifeless> you can understand my confusion
<flacoste> it sure is confusing
<LPCIBot> Project db-devel build #518: FAILURE in 5 hr 8 min: https://lpci.wedontsleep.org/job/db-devel/518/
<LPCIBot> Project windmill-devel build #35: STILL FAILING in 40 min: https://lpci.wedontsleep.org/job/windmill-devel/35/
<lifeless> sinzui: can you spare 45m to an hour sometime today to talk with me ?
<sinzui> I can
<lifeless> sinzui: I've started looking beyond the immediate exigencies of performance... great
<lifeless> bigjools is apparently popping back into irc at some point his evening; given tz constraints I want to chat with him when he does that; we could either wait for that to have happened, or have a call that might be interrupted
<lifeless> I need a keyboard with catguards
<lifeless> I'm thinking raised 2cm ridges at each end
<deryck> jcsackett, you might like to review my current js fix.  It's stuff you and I chatted about before.
<deryck> jcsackett, if you have the time, of course.
<lifeless> deryck: I'd like to chat with you similarly
<lifeless> deryck: but I imagine you are right on EOD as you run to UK times for some bizarre reason?
<deryck> lifeless, about js?
<lifeless> deryck: about our architecture
<deryck> ah
<deryck> lifeless, I've got about 1/2 hour left.  but I have some CHR-like duties I've yet to do.
<benji> jcsackett: I have another small review if you have another few minutes: https://code.launchpad.net/~benji/launchpad/bug-771239/+merge/60113
<deryck> lifeless, but I don't mind chatting.  Or we can chat first thing for you tomorrow?
<lifeless> I would (even though tomorrow is sat) but we have to go pick up a kitten
<jtv> allenap: you're not really still here, are you?
<lifeless> 1/2 hour can probably touch the high notes
<lifeless> deryck: so, lets do it!
<deryck> lifeless, ok, let's do.  firing up mumble.
<lifeless> deryck: d you have skype or voip?
<lifeless> mumble is crappy from here
<deryck> ah
<deryck> I don't know if I ever installed skype.  let me see.
<lifeless> the canonical voip is pretty nice if you have that
<lifeless> if you haven't got skype we can try mumble, we might get lucky
<deryck> lifeless, I haven't setup voip yet.  doing skype install now.  should be quick.
<lifeless> kk
<deryck> I need to get voip up.
<deryck> I'll warn you its loud here.  Working from my wife's shop and it's after school time. :-)
<lifeless> thats ok
<deryck> you say that now ;)
 * deryck is not bothered by noise, but late day calls are fun
<benji> allenap: I have a small review if you have a few minutes: https://code.launchpad.net/~benji/launchpad/bug-771239/+merge/60113
<maxb> lifeless: If you have a moment between all the calls, could you revisit the question of giving me access to the lpreview-body packaging branch as broached in bug 766149? Thanks.
<_mup_> Bug #766149: bzr-lpreview-body from launchpad ppa blocks python 2.6 install on natty by using site-packages <packaging> <Launchpad review body plugin:Confirmed> <bzr (Ubuntu):Invalid> < https://launchpad.net/bugs/766149 >
<lifeless> maxb: I have no issue with that.
<lifeless> flacoste: ^
<flacoste> maxb: me neither
<jtv> jcsackett: are you available for a review?  It's large, but more than half the overall diff is deleted lines.  https://code.launchpad.net/~jtv/launchpad/bug-777941/+merge/60116
<maxb> Great, in that case, could you migrate ownership of https://code.launchpad.net/~launchpad/+recipe/lpreview-body-daily and https://code.launchpad.net/~launchpad/lpreview-body/packaging to ~launchpad-committers ?
<LPCIBot> Project windmill-devel build #36: STILL FAILING in 44 min: https://lpci.wedontsleep.org/job/windmill-devel/36/
<lifeless> sinzui: so with that in mind, what time suits you for a call
<lifeless> bah
<lifeless> I have to get that battery sorted
* lifeless changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: allenap, jcsackett | https://code.launchpad.net/launchpad-project/+activereviews
<lifeless> maxb: what were those urls
<maxb> <maxb> Great, in that case, could you migrate ownership of https://code.launchpad.net/~launchpad/+recipe/lpreview-body-daily and https://code.launchpad.net/~launchpad/lpreview-body/packaging to ~launchpad-committers ?
<maxb> (thanks)
<lifeless> apparently not
<poolie> hello lifeless
<lifeless> I get 'undefined' on the picker widget
<lifeless> hi poolie
<lifeless> sinzui: if you replied I missed it
<poolie> i'm trying to ec2 test some branches
<poolie> they seem to be sinking without trace
<poolie> i suppose while trying to send mail or something
<lifeless> if the failure is too big it can hit gmails size limits
<sinzui> lifeless: sorry. I was distracted. we can talk now or in 4 hours
<lifeless> sinzui: now sounds good to me
<poolie> is there any way to ask it to just leave the instance running serving the file?
<sinzui> lifeless: skype?
<lifeless> poolie: I don't think so
<poolie> :/
<flacoste> poolie: patch welcome :-)
<flacoste> poolie: there is an option not to detach from the instance
<flacoste> you could tail the output
<poolie> tail the output of what?
<poolie> the hotel wireless drops out occasionally
<poolie> i guess i could start an ec2 machine, ssh there and run screen, then run things from that
<salgado> poolie, you can use ec2 demo to start the instance; that way you have everything ready to run the test suite when you ssh into the instance
<poolie> am i supposed to normally be able to ssh in to the test versions?
<poolie> they seem to have a server but it doesn't accept my key
<salgado> poolie, I think your key is associated to the ubuntu user there
<poolie> oh ok
<poolie> no, apparently not?
<salgado> I'm pretty sure your key is there, just need to find out under which user
<poolie> oh ec2test@
<jcsackett> deryck and benji: sorry for not replying, i seem to be having IRC connection issues today. i have looked at your branches, and they are a-ok. :-)
<jcsackett> jtv: i am looking at yours now.
<poolie> ok, so ssh'd in, the log shows
<poolie> Running command: /usr/bin/xvfb-run --error-file=/var/tmp/xvfb-errors.log --server-args='-screen 0 1024x768x24' /var/launchpad/test/bin/test -vv --subunit -vvv
<poolie> and nothing more
<poolie> it does seem to be using cpu
<LPCIBot> Project windmill-devel build #37: STILL FAILING in 41 min: https://lpci.wedontsleep.org/job/windmill-devel/37/
<mwhudson> hmm, i seem to be getting launchpad bug mail again
<jcsackett> jtv: r=me. i have two suggestions in the MP, but they don't block landing--just things to consider.
<mwhudson> i guess this is to do with the better subscription story stuff being enabled for ~launchpad?
<mwhudson> ah no, it's ~launchpad loosing its contact address i think
* jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<mwhudson> luckily the better subscription story lets me turn off the firehose
<jelmer> ooh, the new bug subscriptions are nice :)
<LPCIBot> Project windmill-db-devel build #239: STILL FAILING in 1 hr 5 min: https://lpci.wedontsleep.org/job/windmill-db-devel/239/
#launchpad-dev 2011-05-06
<wgrant> lifeless: Yes, it's on LPS, but that normally gets ignored :)
<lifeless> gawd I hope not
<lifeless> so whats special
<lifeless> also when is good for you
<wgrant> It needs buildd-manager to be shut down before the update, then process-upload run, then proceed as normal.
<lifeless> why?
<wgrant> The directory name format has changed.
<wgrant> And accepting and testing both seems more trouble than having a slightly special one-off upgrade.
<lifeless> maxb: https://code.launchpad.net/~launchpad-committers/lpreview-body/packaging
<lifeless> is done
<lifeless> the other needs the regression in LP fixed
<maxb> thanks
<lifeless> maxb: can you follow up on the list so folk know
<lifeless> wgrant: so whats the filename format change about
<wgrant> lifeless: Removing BuildFarmJob.id.
<wgrant> It was used to identify temporary upload directories.
<lifeless> what did the filename encode, what does it encode, why does it encode it
<wgrant> So, buildd-manager takes an upload and dumps it into a directory for process-upload.py --builds to handle.
<wgrant> It used BuildFarmJob.id to identify the relevant build.
<wgrant> Now it uses the job type and BPB/SPRB ID.
<lifeless> is that in a metadata file or the dirname?
<wgrant> dirname.
<wgrant> (yes, ew)
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #519: FIXED in 5 hr 12 min: https://lpci.wedontsleep.org/job/db-devel/519/
<lifeless> wgrant: when would you like to talk?
<wgrant> lifeless: Now's good for me.
<wgrant> lifeless: You still prefer Skype?
<lifeless> cooking lunch, will ping in a bit
<wgrant> Sure.
<lifeless> wgrant: ping
<LPCIBot> Project windmill-db-devel build #240: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill-db-devel/240/
<LPCIBot> Project windmill-devel build #38: STILL FAILING in 1 hr 10 min: https://lpci.wedontsleep.org/job/windmill-devel/38/
<wgrant> lifeless: Hi.
<lifeless> hi
<wgrant> I'm ready.
<lifeless> voip or skype is bestest
<wgrant> I'm on Skype, I've not got SIP set up.
<lifeless> poolie: jetlag?
<lifeless> hmm, code free day :(
<lifeless> wgrant: https://bugs.launchpad.net/launchpad/+bug/777794 - please triage as you file :)
<_mup_> Bug #777794: Bug subscription AJAX popups are too bold <Launchpad itself:New> < https://launchpad.net/bugs/777794 >
<lifeless> 777789 too
<lifeless> and 777786
<wgrant> lifeless: I can't triage those.
<lifeless> why not?
<StevenK> Hm, I meant to file a low bug about that
<wgrant> Because I don't know what importance they are.
<lifeless> wgrant: take a good guess
<wgrant> Given that there is meant to be a UI polish mode at the end of each feature, they should probably be High.
<wgrant> But I don't know.
<lifeless> the feature squads are not watching the tracker.
<StevenK> If you set the importance and such like when you file a bug, we shouldn't say "Your bug will be triaged soon"
<StevenK> Perhaps ...
<wgrant> StevenK: That's for external people.
<lifeless> StevenK: that prose is just the post-filing-notice, its not programmatic
<wgrant> Right.
<lifeless> wgrant: please triage them, even if its just to high + an email to gary to say 'these need your hand to really assign importance'.
<lifeless> wgrant: you can certainly assess them in the context of the BugTriage wiki page
<lifeless> wgrant: a nontrivial reason for doing is so that end user filed bugs which are not triaged are obvious on the portlets on https://bugs.launchpad.net/launchpad-project/+bugs
<lifeless> the [non]zero distinction is easier for everyone to observe than a floating figure
<poolie> hi lifeless
<poolie> a bit
<wgrant> DF's up to 62GB... this is going to take a little while.
<StevenK> As in DF's DB?
<wgrant> Yes.
<wgrant> Hmm. Might need to clean some stuff out.
<wgrant> lifeless: \l+ on staging?
<lifeless> its fat too
<lifeless> it has the garbo running nowadays
<lifeless> qastaging is 275GB fwiw
<lifeless> wgrant: ok, I've triaged your  bugs... next time I'll make em all opinion :P
<lifeless> ohh nice
<lifeless> +patches gone from oopses
<wgrant> lifeless: Thanks.
<wgrant> Hmm, the missing structural subscription listing a bit unfortunate.
<LPCIBot> Project windmill-db-devel build #241: STILL FAILING in 1 hr 5 min: https://lpci.wedontsleep.org/job/windmill-db-devel/241/
<LPCIBot> Project windmill-db-devel build #242: STILL FAILING in 41 min: https://lpci.wedontsleep.org/job/windmill-db-devel/242/
<adeuring> good morning
<mrevell> Hello!
<poolie> hi mrevell
<poolie> lifeless, did people ever consider changing the ec2 image to run tests on a tmpfs?
<poolie> it doesn't seem to be using its whole cpu
<bigjools> good morning
<poolie> hi bigjools
<lifeless> poolie: I don't think our ec2 stack has been all that optimised
<lifeless> bigjools: missed you this morning
<bigjools> lifeless: yeah sorry
<lifeless> no worries
<bigjools> was knackered
<lifeless> perhaps we could talk in ~30 ?
<bigjools> lifeless: I have a standup, but in  ~45 would work
<lifeless> ok
<LPCIBot> Project windmill-devel build #39: STILL FAILING in 1 hr 4 min: https://lpci.wedontsleep.org/job/windmill-devel/39/
* adeuring changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: adeuring | https://code.launchpad.net/launchpad-project/+activereviews
<poolie> hm so just moving bits of /var to ram doesn't immediately fail
<poolie> and is doing little io
<poolie> still lots of idle cpu, but i think that's just lack of parallelism
<wgrant> bigjools: Bug #778408 may amuse you.
<_mup_> Bug #778408: Deleting an unpublished publication behaves unexpectedly <soyuz-ftpmaster-tools> <soyuz-publish> <Launchpad itself:Triaged> < https://launchpad.net/bugs/778408 >
<wgrant> (Side-effects of amusement may include uncontrollable sobbing and a sense of hopelessness.)
<al> hehehe
<wgrant> rvba: Have you seen the stable->db-devel merge conflict? It seems to be your stuff.
<wgrant> So you're probably best to resolve it.
<rvba> wgrant: I just saw that.
<rvba> wgrant: can you guide me with how I can fix that?
<rvba> wgrant: I understand the problem of course but I'm unsure about the procedure to solve it.
<bigjools> wgrant: colour me unsurprised
<wgrant> rvba: Grab stable and db-devel locally, merge stable into db-devel, fix the conflict as you would normally. Push the branch up somewhere, then pqm-submit it to db-devel.
<bigjools> wgrant: we need yet more code to stop people from being stupid
<wgrant> rvba: if you are feeling forgiving you could ask for a review, but I never do and haven't broken anything yet.
<rvba> wgrant: thanks. I'll do that.
<wgrant> bigjools: Do you have a list of stuff to kill from mawson? Its free space will probably not last the weekend.
<bigjools> wgrant: yeah I was looking at that
<wgrant> We could just start wiping out the archive, I guess.
<bigjools> yes
<bigjools> that archive is too fat
<wgrant> Easy enough to bring back in a few hours if we need to.
<wgrant> (sorry mizuho)
<wgrant> It is fat, but was useful for testing a few things.
<wgrant> But that's all done now.
<bigjools> can be restored easy
<bigjools> given time :)
<wgrant> Yep.
 * wgrant deletes.
<wgrant> Mass deletion + restore == molten mawson?
<bigjools> I bet it's hot it mawson's cabinet :D
<bigjools> s/it/in/
<bigjools> wgrant: it'll all be invalid once the db is restored anyway
<bigjools> local librarian is 5.2G, purging
<wgrant> Not really. This is all from frozen series.
<wgrant> maverick + natty
<bigjools> not all
<bigjools> not sure they weren't frozen when we last restored
<wgrant> maverick was released, and natty was basically identical to maverick.
<wgrant> So it was mostly just maverick's final packages plus a few more.
<wgrant> Anyway, deleting now.
<wgrant> Very slowly.
<bigjools> the disk stepper motor is going to fall off its track
<LPCIBot> Project windmill-devel build #40: STILL FAILING in 1 hr 4 min: https://lpci.wedontsleep.org/job/windmill-devel/40/
<rvba> wgrant: would you mind taking a look at this for a quick sanity check? https://code.launchpad.net/~rvb/launchpad/db-devel-fix-merge-failure
<wgrant> rvba: The diff is large, but as long as the tests you've changed pass it should be fine.
<rvba> wgrant: yeah, the tests in the modified file pass.
 * rvba bzr pqm-submit -m "Merge stable."
<wgrant> rvba: You'll need a --submit-branch
<bigjools> "fix conflict with stable" would be more accurate
<wgrant> To target db-devel.
<bigjools> and that
<rvba> wgrant: oops ... too late.
<wgrant> It should reject it.
<lifeless> rvba: hi; its my weekend now, but wgrant may be able to answer your perf tuning thing if you're lucky
<lifeless> rvba: otherwise I'll shoot you some thoughts on mondayish
<wgrant> Where's that?
<lifeless> wgrant: I'l forward
<rvba> lifeless: thanks a lot. Have a good weekend!
<wgrant> I have no email client at the moment, but I'll get one in a sec.
<wgrant> rvba: So, what's the issue with it being a list?
<wgrant> rvba: checkCopy is already pretty fast, except for one code path.
<rvba> wgrant: since it's not a ResultSet there is not way to try to eager load related objects ... right?
<rvba> s/not/no/
<wgrant> rvba: You can't add a preiter hook to it, no. But that's just a handy place to do it... you can do the same thing manually afterwards.
<wgrant> What are you looking to eager-load?
<wgrant> (the slow bit is when _checkArchiveConflicts finds some destination_archive_conflicts)
<rvba> I was just *thinking* of eager-loading related things since I saw the the kind of db queries issued looked like related objects being loaded.
<rvba> but I might be wrong.
<rvba> I you say it's pretty quick already, it might not be worth the bother then.
<wgrant> It's already pretty good. Most copies check within half a second, except for _checkArchiveConflicts.
<wgrant> So yeah, not worth spending time on that yet.
<rvba> s/I/If
<rvba> ok, so I guess the tests to make sure the number of queries does not increase is enough for now.
<wgrant> Yup.
<wgrant> That would be useful.
<gmb> adeuring: Can you take a look at https://code.launchpad.net/~gmb/launchpad/bug-772609/+merge/60176 for me please?
<adeuring> gmb: sure
<gmb> Thanks
<LPCIBot> Project windmill-db-devel build #243: STILL FAILING in 1 hr 4 min: https://lpci.wedontsleep.org/job/windmill-db-devel/243/
<deryck> Morning, all.
<jtv> hi deryck
<deryck> hi jtv
<adeuring> gmb: overall, nice work. but could you remove the "#import pdb; pdb.set_trace()" line in test_bugs_views.py?
<SteveA> yay, code reviews :-)
<adeuring> gmb: and you disabled the get_notified_subscribers() call in bugs/subscribers/bug.py
<adeuring> gmb: there is an XXX from bac about the call which has no bug number
<adeuring> gmb: also, I am not sure if we can really simply drop the call.
<adeuring> deryck: could you please run these queries  on staging: https://pastebin.canonical.com/47200/ ?
<deryck> adeuring, indeed I can.
<LPCIBot> Project windmill-db-devel build #244: STILL FAILING in 40 min: https://lpci.wedontsleep.org/job/windmill-db-devel/244/
<deryck> benji, I'm triaging bug 778323 as high as I assume it's related to y'all's current work.  Is story-better-bug-notification  still the best tag?
<_mup_> Bug #778323: On the subscription page, the help popup button opens up a 404 page. <Launchpad itself:Triaged> < https://launchpad.net/bugs/778323 >
<benji> deryck: yep, that's the current tag
<deryck> benji, cool.  done.  Thanks
<deryck> I don't think anyone understood the implications of dropping ~launchpad-bugs mailing list.
<jtv> danilo, henninge: I wonder if those links in the "untranslated" etc. numbers on the overview pages disappeared because of the Prague work.
<danilos> jtv, I don't know, I don't have the time to investigate right now
<jtv> Same here.  :(
<danilos> jtv, I filed a bug so when I switch to maintenance I'll take a look
<jtv> OK
<danilos> jtv, it's tagged "regression", which makes it "Critical"
<jtv> I guess it doesn't absolutely require a Rosetta veteran.
<jtv> Hi Ursinha
<danilos> jtv, indeed
<jtv> I wonder why I'm suddenly being showered in bugmail.
<deryck> Hi, gmb.  Could I get you to review my GCC credentials branch for me?  It's simple, but I want to make sure I haven't missed something I don't know about.
<gmb> deryck: Sure thing. Diff me.
<deryck> gmb, lp branch is:  https://code.launchpad.net/~deryck/launchpad/add-gcc-keys-schema-lazr-745794/+merge/60186
 * gmb looks
<gmb> Oo, complex!
<deryck> heh
<gmb> That looks fine.
<deryck> gmb, and I did a matching lp-production-configs one first:  https://code.launchpad.net/~deryck/lp-production-configs/add-gcc-bugzilla-creds/+merge/59956
<deryck> gmb, so I understand land the lp branch, nodowntime rollout, then the lp-production-configs one, yes?
<gmb> deryck: I believe so. I haven't had to do one of these since before there were nodowntime rollouts.
<gmb> But that sounds sane.
<deryck> gmb, ok, cool.  yeah, this is my first config change believe it or not.
<gmb> Ah, always fun.
<wgrant> deryck: Land the branch, land the configs, nodowntime rollout.
<wgrant> No need for a rollout in the middle.
<deryck> wgrant, ah, ok.  thanks.
<deryck> wgrant, and no need for any special rollout requirements, i.e. the usual nodowntime will cover everything I need?
<wgrant> deryck: That's right.
<deryck> wgrant, awesome sauce.  thanks, again.
<wgrant> We grab the requested rev of stable and the latest lp-p-c rev.
<deryck> gotcha
<LPCIBot> Project windmill-devel build #41: STILL FAILING in 1 hr 5 min: https://lpci.wedontsleep.org/job/windmill-devel/41/
<adeuring> gmb: did you see my questions about your mp?
<gmb> adeuring: Oops, missed them. Let me look...
<adeuring> bac: are you around?
<gmb> Ahaha
<gmb> Thanks for catchign that pdb.
<gmb> Oh. I don't remember removing that...
<adeuring> gmb: np. but can we reall drop the get_also_notified_subscribers() call?
<gmb> Ah, I see. That's from bac's work and I've not spotted it...
<gmb> adeuring: Hmm. Lemme check.
<gmb> adeuring: I honestly don't know.
<bigjools> adeuring: hi, I've got a small branch in the queue, no rush.  TIA :)
<gmb> adeuring So, I'll put it back and run EC2 and see what happens.
<adeuring> bigjools: I'll look
<adeuring> gmb: ok, sounds good
<gmb> Ta
<jcsackett> sinzui: would you like to mumble this morning?
<beuno> hello lp devs
<beuno> I _love_ this new email management story
<beuno> I get this:
<beuno> Launchpad Bug Contacts subscription: (unnamed)You do not receive emails from this subscription
<beuno> any idea what the (unnamed) bit is?  should I file a bug?
<huwshimi> beuno Did you give the subscription a name?
<huwshimi> beuno: Did you just create the subscription or was it an old one?
<beuno> huwshimi, no, not even sure where (or why!) to do that
<beuno> huwshimi, old
<jcsackett> allenap: i've made changes to the branch you looked at yesterday, if you could take another look.
<huwshimi> beuno: I don't know heaps about it, but I'm guessing you might be able to give it a name somwhere
<beuno> huwshimi, why would I want to name something so efimeral?
<huwshimi> beuno: Are you around somewhere and have a minute? I wouldn't mind looking at the UI of what you have.
<wgrant> Anyone in ~launchpad can see the same thing at https://launchpad.net/launchpad/+subscriptions
<beuno> huwshimi, I am! I'm in the canonical sumit room, and would love to meet you. I have another hour of sessions, but then I'm free.
<deryck> beuno, if you name it, bug mail gets the name in headers and in the footer of emails, to help you filter on that name.
<huwshimi> beuno: I'll try to catch you at some stage. Not sure what I'll be doing in an hour though
<beuno> deryck, right, that makes a bit of sense
<huwshimi> deryck: Does that mean someone needs to go back to the subscription and give it a name?
<beuno> huwshimi, I'm here this and next week
<huwshimi> beuno: ok, I'll be around until Wednesday
<deryck> beuno, huwshimi -- you can edit the subscriptions off the target or the +subscriptions or +structural-subscriptions pages, I believe.
<adeuring> bigjools: r=me
<bigjools> adeuring: cheers
<deryck> beuno, huwshimi -- should be able to name it from:  https://launchpad.net/launchpad for example
<beuno> deryck, so maybe it needs a default name?
<deryck> I may have some details wrong, though, since it's Yellow Squad working on this now :-)
<deryck> beuno, yeah, probably some string concact of $target-$user-sub or some such.
<deryck> (Unamed) seems not clear or useful
<allenap> jcsackett: Already done :)
<jcsackett> \o/ thanks, allenap!
<danilos> beuno, there should still be old headers which one used to use for subscriptions filtering
<deryck> beuno, can you file a bug?  I can triage it and assign to story tag.
<danilos> beuno, fwiw, the bug mail you might have started getting could be due to a change in launchpad product configuration, and is unrelated to the beta fwiw
<danilos> deryck, thanks for helping out with this :)
<deryck> danilos, np. :-)  I don't mean to nosy my way into your work ;)
<danilos> deryck, not at all, just go ahead, I am off now as well :)
<deryck> ok cool :-)
<danilos> deryck, and we appreciate all the help we can get :)
<deryck> It's an awesome feature.  Many will love it.
<danilos> as long as launchpadders don't confuse this configuration change with the feature :)
<danilos> (i.e. ~launchpad losing the contact address)
<beuno> deryck, I can
<beuno> danilos, right, no, I get the same amount of email (I was subscribed to everything)
<huwshimi> danilos: But this change should mean you can stop getting all that mail though right?
<beuno> this was me taking advantage of muting some email  :)
<danilos> huwshimi, right
<beuno> deryck, I'll file the bug, I need the karma
<danilos> beuno, I am sure you'll find a bunch of bugs to report :)
<huwshimi> danilos: Do you know how I can mute all of this new mail?
<deryck> I don't even see why we need ~launchpad-bugs.  Just unsubscribe it from the ~launchpad, no?
<deryck> isn't that the account causing issues?  huwshimi ^^
<huwshimi> deryck: I think it might be
<wgrant> ~launchpad-bugs' subscription to /launchpad can be removed by a Launchpad admin. But it's not the main problem.
<deryck> ah
<wgrant> ~launchpad also gets vcs-imports spam, loggerhead question spam, probably launchpad-reviewers MP spam.
<deryck> ah
<wgrant> None of that is fixable.
<deryck> and ~launchpad lost it's email address
<deryck> I see
<wgrant> We may have to readd the address, once everyone is pissed off enough to file critical bugs about all the unstoppable mail.
<huwshimi> deryck: It's not just that. I'm getting heaps of mail from Loggerhead and stuff now too
<wgrant> But until we have lots of bugs filed it would be silly to readd it.
<wgrant> Because we'd never fix it properly.
<deryck> yeah
<wgrant> (see eg. the last 5 years of having a contact address)
<deryck> we just shouldn't expand teams to individual emails.  why would anyone really want that ever?
<deryck> could be easier to fix than file the bug ;)
<wgrant> deryck: Then who gets the mail?
<wgrant> eg. ~launchpad is an admin for some teams. Who gets the mail that somebody is trying to join?
<danilos> huwshimi, you should be able to go to any of the bugs with emails, click on "Edit bug mail" and mute the team subscription
<deryck> wgrant, if you don't set a contact for the team, no one.
<wgrant> I think team subscriptions to any object are stupid and should be banned.
<danilos> huwshimi, with something like "Don't send me email"
<wgrant> But we use subscriptions for security too.
<wgrant> So we can't do that.
<deryck> create a security mailing list for the team that people optionally subscribe to.
<deryck> this is how the rest of the world works, no?
<huwshimi> danilos: Thanks
<wgrant> This sounds a lot like working around an incomplete advanced subscriptions feature to me :)
<deryck> maybe in one sense
<deryck> but I think my point is that you want a security or team mailing list, not a team subscription.
<danilos> wgrant, for security stuff, direct subscriptions are created for the security contact on all security bugs
<deryck> that magically expands
<huwshimi> danilos: What about stuff that comes from questions etc?
<LPCIBot> Project windmill-db-devel build #245: STILL FAILING in 1 hr 5 min: https://lpci.wedontsleep.org/job/windmill-db-devel/245/
<wgrant> deryck: Why?
<danilos> huwshimi, can't help there, other than filtering
<huwshimi> danilos: OK thanks :)
<wgrant> deryck: Security email isn't special enough that it deserves a list. That's just the only way it can sensibly be done now, because LP is broken.
<wgrant> Hmm, poolie's not here.
<wgrant> We should unsubscribe loggerhead-team from loggerhead's questions.
<wgrant> That will stop a lot of the immediate spam.
<deryck> wgrant, maybe you're right.  I'm just being clear that the rest of the world uses a security mailing list, not team membership to manage security bugs.
<deryck> and that we're conflating the two
 * danilos -> really out
<deryck> what happened now is that we have a team subscribed for many reasons, not to manage mail
<deryck> and the conflation is the issue.
<wgrant> Right.
<wgrant> A lot of these reasons are obsolete.
<deryck> right
<wgrant> Ah, flacoste is readding the address now.
<deryck> and that's all I mean.  I don't mean to overanalyze or suggest the one true way to fix it.  just pointing out the root issue.
<wgrant> deryck: The root issue is that we need to separate permission and subscription, and we probably also want to extend the excellent new structural subscription stuff to cover everything.
<flacoste> wgrant: yeah, PQM bot will receive all the email we are receiving
<wgrant> flacoste: We should really fix that PQM bug.
<flacoste> so it will be sending those kind of error message in a lot of case
<deryck> wgrant, yes, agreed.
<flacoste> wgrant: or we should fix launchpad spam :-)
<flacoste> to be honest, i'm not sure what is the PQM bug in this case
<flacoste> that it's a member of ~canonical
<flacoste> actually
<deryck> yeah, some things we really just should never send mail for
<flacoste> i don't know why PQM should be a member ~canonical
<flacoste> it's mainly for branch
<flacoste> and we always subscribe it explicitely
<flacoste> which is better
<wgrant> flacoste: The PQM bug is that it crashes when it can't verify a signature.
<wgrant> Rather than just ignoring the mail.
<wgrant> Also it probably shouldn't be a member of ~launchpad, I guess, but it might need those privileges.
<flacoste> well, even if it didn't oops
<flacoste> it should send an email about the error
<flacoste> it can be a legitimate user error
<wgrant> Possibly.
<wgrant> I guess having ~launchpad-pqm's email address set to PQM's email address is pretty pointless.
<flacoste> also
<wgrant> I can see no possible benefit of that.
<flacoste> me neither
<flacoste> but it's hard to set a LP profile to a sink hole
<flacoste> address
<wgrant> Yeah.
<wgrant> We need robot profiles.
<deryck> sinzui, ping
<deryck> benji, are we making noise about +subscriptions or +structural-subscriptions as part of y'all's work?  or keeping that on the dl?
<abentley> adeuring: Could you please review https://code.launchpad.net/~abentley/launchpad/daily-build-permissions/+merge/60207 ?
<adeuring> abentley: sure
<abentley> Thanks.
<benji> deryck: I don't think they're secrets; both should be linked to from user-visible parts of the UI (those users who can see them because of the feature flag that is)
<benji> adeuring: I have a small review for you when you have a minute: https://code.launchpad.net/~benji/launchpad/bug-777794/+merge/60209
<adeuring> benji: sure, I'll look
<benji> thanks
<deryck> benji, ok, thanks.  Early on in that story there was talk of hiding them until they were ready, and I didn't want to assume they were ready. :-)
 * deryck is doing IRC duties and wants to be prepared as people ask about new work
<adeuring> benji: r=me
<benji> thanks
<adeuring> abentley: r=me, just one note: there is an "import re" in a method in test_sourcepackagerecipe.py. I think this import can happen at the top of the file
<sinzui> deryck: I think all registry admins can hide bug and question comments. I do not think https://answers.launchpad.net/launchpad/+question/156277 should be assigned to an admin
<deryck> sinzui, ah, sorry.  from the web UI?  I didn't realize this.
<deryck> sinzui, or using an api script?
<sinzui> deryck:  https://dev.launchpad.net/MaintenanceRotationSchedule has link to dealing with spam and there is a script that you can run
<deryck> sinzui, ok, I'll take a look.
<deryck> I have such a low tolerance for administrivia.  I really suck at keeping up with all this.
<sinzui> deryck: jcsackett is working on the UI to toggle comment visibility now. You may not ever run that scrip again after today
 * bigjools is with deryck
<deryck> nice
<deryck> sinzui, and if I have a user spamming bugs with linkedin invites, do I deactive or suspend the account?
 * deryck sucks at reading 45 wiki pages too :-P
<sinzui> suspend and include the question URL in the comment so that we know why
<deryck> gotchas
<deryck> sinzui, why isn't hide-comments in the lp tree?
<sinzui> ask gmb he wrote it
<sinzui> deryck: I modified that script a week ago so that we could work with questions too
<sinzui> As I said, jcsackett is adding the UI now
<deryck> sinzui, is that the copy in the wiki?
<deryck> ah true
<sinzui> yes it is
<deryck> so shall I wait until Monday to hide this comment then? ;)
<sinzui> jcsackett: when do you estimate we can QA comment hiding in the UI
* adeuring changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<deryck> sinzui, i was just kidding around.  I've got the script running now.
<deryck> feels so 2004 to have to moderate Launchpad
<deryck> don't people automate this sort of thing these days ;)
 * deryck is just kidding around
<jcsackett> sinzui: Out to ec2 land now.
<jcsackett> so I can qa sometime this weekend, or Monday depending.
<sinzui> jcsackett: thanks, I was just confirming my commitment not to add the api script to the lp tree
 * jcsackett nods 
<jcsackett> There should be no need.
<gmb> deryck, sinzui: No he didn't. That was intellectronica. At least I think it was. I could be lying.
<sinzui> gmb: my apologies
<gmb> sinzui: Don't apologise :) I just don't want people to get the impression that I know anything.
<sinzui> yes. I know exactly how you feel. People keep asking me questions expecting me know the answer
<sinzui> Help! I see that @operation_returns_collection_of errors if the method it decorates returns a Set. Can I make it like the Set, or do I need to change it to a list?
<sinzui> I think there may be good reason the method returns a Set.
<SteveA> hi, quick question. I couldn't find the answer on a cursory look at the launchpad dev wiki
<SteveA> how long is a launchpad dev cycle?
<EvilUrsinha> SteveA, it used to be about a month
<SteveA> specifically, how long does a squad work on one area?
<abentley> SteveA: Mostly, we release features when they are done: https://dev.launchpad.net/LEP/ReleaseFeaturesWhenTheyAreDone
<EvilUrsinha> SteveA, that's a bit different question, what abentley said
<abentley> SteveA: as long as it takes to complete the feature.
<SteveA> cool :-)
<SteveA> for the most part (the modal average, say) how long does a squad work on a feature?
<SteveA> or rather, on an area of the codebase
<abentley> SteveA: We have two feature squads and two maintenance squads at a time.  When a feature squad finishes, they swap with a maintenance squad.
<abentley> SteveA: I don't think we've got enough experience to answer this.  We've only been doing this since the last Epic.
<sinzui> SteveA 4-13 months, but that we want to scope features into small units so that no feature is longer than a few months. We want teams to rotate to do maintenance more often.
<SteveA> ok, I understand, I think. Thanks sinzui
<SteveA> thanks abentley and EvilUrsinha too
<EvilUrsinha> no problem :)
<abentley> SteveA, np
<LPCIBot> Project windmill-devel build #42: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill-devel/42/
<deryck> benji, wouldn't the views I was asking about this morning count as a single view of subscriptions.  i.e. https://launchpad.net/~deryck/+subscriptions and https://launchpad.net/~deryck/+structural-subscriptions ?
<deryck> well where single == two views ;)
<benji> deryck: I think so!  I had forgotten that +structural-subscriptions works for a user; I thought there was only a list of directly subscribed bugs (i.e., ~user/+subscriptions)
<deryck> benji, right.  jkakar might be interested in that  ^^ then
<jkakar> Hiya benji, deryck. :)
 * jkakar looks at +structural-subscriptions
<benji> hmm, I can't find any links to those pages; I wonder why not
<benji> it seems to me that there should be a link from the ~user page to those
<deryck> benji, right.  that's why I asked this morning.
<jkakar> benji, deryck: Uhm... my first impression is that I have no idea why I'm seeing what I'm seeing on this page. :)
<deryck> benji, early on they needed a lot of polish
<deryck> excatly! :-)
<benji> the style looks messed up on +structural-subscriptions, those boxes should be full width
<benji> oh, and the edit links take you to the old-style subscription edit pages
<deryck> yeah, maybe they're not ready to talk about
<benji> deryck and jkakar: ok, those are definatly not meant for public consumption; we need to either kill those or fix them posthaste
<deryck> benji, right
<benji> sorry for the confusion
<deryck> np
<jkakar> benji: Okay, cool.
<deryck> but as a rough, unpolished sketch of what you're subscribed to, they're not bad
<deryck> lp subscriptions easter egg
<jkakar> benji: I think what I want to see is something like (1) a list of bug subscriptions I have (ie, things I explicitly subscribed to), (2) a list of projects that I get bug or merge proposal mail from because I'm in a team, and (3) a way to control each of those things.
<benji> That's one of the downsides of deploying incomplete features, they are sometimes never finished.
<jkakar> benji: Yeah.  Though, seeing something unfinished in production can also be a strong motivator to land further improvements.
<jkakar> Anyway, I'm really glad you guys are pushing this work forward.  It's going to be so great to have control of these issues.
<benji> yeah, that would be quite nice; you'll also probably want a list of all the non-subscription reasons you can get email; there are quite a few of those
<jkakar> benji: Yeah, I probably do want that... particularly because I don't understand what they are off the top of my head... it would be nice to see it all in one place so I could complete my mental model of how LP notifies me of things.
<jkakar> Uh oh, am getting repeated OOPSes trying to load: https://launchpad.net/fluidinfo/+milestone/11.05
<jkakar> Like OOPS-1952AY485 for example.
<sinzui> jkakar: You have discovered a bug! looks like the page dies trying to render the  structural subscription menu items
<LPCIBot> Project windmill-devel build #43: STILL FAILING in 43 min: https://lpci.wedontsleep.org/job/windmill-devel/43/
<jkakar> sinzui: Ah, oops. :)
<sinzui> I am reporting the bug now
<sinzui> jkakar: I can see the page as anonymous since we do not try to render the bad link
<sinzui> I think the issue may just be project group milestons
<jkakar> sinzui: But I guess you can't see any bugs because the project is private, right?
<sinzui> I see 1
<jkakar> sinzui: Interesting, I can load https://launchpad.net/storm/+milestone/0.19 without issue.
<jkakar> sinzui: Ah, which one? :)
<sinzui> bug 664548
<_mup_> Bug #664548: Rearrange chairs on upper decks <story-brave-new-world> <Fluidinfo:In Progress by fluiddb-dev> < https://launchpad.net/bugs/664548 >
<jkakar> sinzui: Ah, okay, that one is public indeed.  Thanks.
<LPCIBot> Project windmill-db-devel build #246: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill-db-devel/246/
<LPCIBot> Project windmill-devel build #44: STILL FAILING in 43 min: https://lpci.wedontsleep.org/job/windmill-devel/44/
 * deryck reboots, brb
<LPCIBot> Project windmill-devel build #45: STILL FAILING in 43 min: https://lpci.wedontsleep.org/job/windmill-devel/45/
<lifeless> jkakar: I didn't realise fluidinfo used lp. Cool.
<LPCIBot> Project windmill-devel build #46: STILL FAILING in 43 min: https://lpci.wedontsleep.org/job/windmill-devel/46/
<lifeless> spammer on bug 411296
<_mup_> Bug #411296: traceback if python has no ssl support: httplib module has no HTTPSConnection <https> <Bazaar:Confirmed for vila> < https://launchpad.net/bugs/411296 >
<LPCIBot> Project db-devel build #523: FAILURE in 5 hr 13 min: https://lpci.wedontsleep.org/job/db-devel/523/
#launchpad-dev 2011-05-07
<jkakar> lifeless: Yeah, I switched them over when I was working as an advisor.
<jkakar> lifeless: I haven't found anything better than Launchpad for supporting a process that involves code reviews (which is also something I introduced while I was an advisor).
<jkakar> lifeless: In general, I think LP is a great tool for teams.
<jkakar> lifeless: I think GitHub has a bit of an advantage for developer-to-developer collaboration, because it's UI is really focused on that problem.
<LPCIBot> Project windmill-db-devel build #247: STILL FAILING in 1 hr 8 min: https://lpci.wedontsleep.org/job/windmill-db-devel/247/
<lifeless> jkakar: I'd like us to improve on d2d
<lifeless> not sure how
<jkakar> lifeless: Yeah, it's a tricky problem.  I think the answer lies in UI.
<wgrant> Argh.
<wgrant> UnparsableDependencies again :(
<jkakar> lifeless: I think the pieces (bugs, reviews, etc.) are all in place... the Launchpad UI needs to make them more powerful.
<jkakar> lifeless: I think Launchpad is a bit hamstrung by Ubuntu.  Ubuntu has all kinds of weird and particular requirements that no one else cares about, and that take a lot of energy to maintain...
<wgrant> s/maintain/ignore/
<wgrant> *cough* Blueprints
<jkakar> lifeless: If we didn't have to deal with those problems we'd be able to focus our energy on problems that are more widely felt by users.
<jkakar> wgrant: Perhaps, yes.
<wgrant> But yes.
<jkakar> And I'm not discounting the power of Launchpad in terms of the Ubuntu project.
<jkakar> I think it's clear that Ubuntu could exist without Launchpad...
<wgrant> Nobody before has tried to combine a distribution and open source project and commercial project everything-tracker, I don't think.
<wgrant> And it sort of shows :/
<jkakar> But it means that the folks that who use GitHub look at Launchpad and see it as not solving their problem.
<jkakar> wgrant: Yeah, it's a tricky problem, for sure.
<jkakar> wgrant: I think a big problem is that (and maybe I'm completely wrong here, I hope I am) there is no UI designer focused on Launchpad, focused on making it coherent, and with enough manpower behind them to make real change.
<jkakar> wgrant: Launchpad's UI has been getting steadily getting better, but it is also very confusing and could do with a lot of polish/love.
<jkakar> I sometimes think we're too driven by technological problems and not enough by user problems.  I guess that's a reality of having a lot of technical debt, which I know Launchpad has and I also know that a tremendous effort has been put in to reducing it.
<wgrant> We are now actually in a position where we can reduce it, and we are doing so.
<jkakar> It does feel like we're getting to a point where we'll be seeing the benefits of that investment, though.
<wgrant> lifeless has saved the world, again.
<jkakar> wgrant: Yeah, that's my sense too.
<jkakar> wgrant: It's a tricky problem and I think the team has done an excellent job in tackling it head on.
<jkakar> That said, systems like GitHub, which I guess don't have that same problem, seem to be moving forward quickly.
<wgrant> GitHub doesn't have massive tech debt and design flaws from 2004 :(
<jkakar> In Fluidinfo, for example, we use Launchpad but we're talking about doing some interesting things and I see people saying, "In order for this to work we'd have to ditch Launchpad... because the crowds are at GitHub."
<jkakar> wgrant: Yep.
<wgrant> They also do a lot less.
<jkakar> Yep.
<wgrant> But they do most of it well, or at least shinily.
<jkakar> But also, Launchpad does much more than most people care about.
<jkakar> And the things that most people care about, it doesn't do amazingly well.
<jkakar> I still think Launchpad is the best collaboration tool for teams... but I don't feel confident saying that for individuals.
<jkakar> It's tricky business having a large and varied audience to cater to.
<jkakar> Okay, bedtime here.  Thanks for being awesome, folks. :)
<LPCIBot> Project windmill-devel build #47: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill-devel/47/
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #524: FIXED in 5 hr 12 min: https://lpci.wedontsleep.org/job/db-devel/524/
<LPCIBot> Project windmill-db-devel build #248: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill-db-devel/248/
<LPCIBot> Project windmill-db-devel build #249: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill-db-devel/249/
<LPCIBot> Project db-devel build #525: FAILURE in 5 hr 13 min: https://lpci.wedontsleep.org/job/db-devel/525/
<poolie> lifeless, hi?
<poolie> is there a global acl on dev.l.n?
<poolie> skaet doesn't seem able to edit anything
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #526: FIXED in 5 hr 11 min: https://lpci.wedontsleep.org/job/db-devel/526/
<jml> poolie: on https://code.launchpad.net/~mbp/launchpad/feature-install-cleanup/+merge/60279, should you be using u'on' rather than 'on'?
 * jml doesn't know
<jml> but the old code did.
<jml> <jml> poolie: on https://code.launchpad.net/~mbp/launchpad/feature-install-cleanup/+merge/60279, should you be using u'on' rather than 'on'?
<poolie> hi jml
<poolie> good question
<jml> the old code did. I don't really know the answer myself.
<poolie> the names and values are unicode
<poolie> the code using it would have to be being unusually pedantic to notice, but it's possible
<poolie> i'll fix it
<jml> poolie: thanks.
<jml> poolie: can you land branches, or should I land it for you?
<poolie> i can!
<poolie> it took a few attempts but i'm pretty sure it will work now
<jml> poolie: sweet.
<poolie> indeed
<poolie> thanks for the speedy review
<lifeless_> poolie: storm and pythons postgresql driver are unusually pedantic
<poolie> hi
<poolie> it probably wouldn't be comparing that string, and if the test passes i guess it's not rejecting it from the update
<poolie> statement
<poolie> but i changed it anyhow
<poolie> my attempt to land was blocked by https://bugs.launchpad.net/launchpad/+bug/750274
<_mup_> Bug #750274: librarian.txt fails sometimes <build-infrastructure> <librarian> <spurious-test-failure> <Launchpad itself:Triaged> < https://launchpad.net/bugs/750274 >
<lifeless> poolie: I haven't looked at the diff; was just letting you know
<poolie> hopefully it will pass on a second attempt
<poolie> thansk
<lifeless> jml: hi
<lifeless> jml: are we going to try to complete that catchup before the ensemble call; I think it has bearing on how to try to do it
<lifeless> jml: e.g. can we do it on monday morning
<lifeless> poolie: I'm having trouble finding skaet online ; if she would like to chat she can ping me anytime
<lifeless> poolie: also yes there is an acl on help.lp.net; spammers love that site
<lifeless> poolie: do you know here lp username ?
<poolie> skaet
<poolie> but perhaps oh i don't know
<poolie> ~canonical could be added or something?
<lifeless> I don't know the history
<lifeless> I'll add her now
<lifeless> and probably talk to flacoste / mrevell when I see them about ~canonical
<lifeless> wgrant: how big is the supportted archive, do you know?
#launchpad-dev 2011-05-08
<wgrant> lifeless: The supported Ubuntu archive?
<lifeless> yeah
<lifeless> I'm doing some lmirror testing
<lifeless> considering buying a couple of TB disks to work with
<wgrant> archive.ubuntu.com should only be a few hundred gigabytes.
<wgrant> debmirror will give you a pretty good estimate.
<lifeless> the Mirrors wiki page says 400GB
<lifeless> bbiab
<lifeless> wgrant: do you have a sense for the churn (GB/month) in a mirror ?
<StevenK> My local mirror averages 4GB a week or so
<StevenK> That's lucid, maverick, natty and oneiric i386 and amd64
<StevenK> But I don't keep sync logs for more than a day
<StevenK> steven@thrashed:~$ tail -n 1 /srv/mirror/var/archive-log.0
<StevenK> Downloaded: 378 files, 1.2G in 30m 31s (672 KB/s)
<lifeless> StevenK: thanks
<lifeless> hmm, 1GB memory to process the metadata for the current archive
<lifeless> tolerable
<lifeless> mmm
<lifeless> except the journal is 74MB
<lifeless> thats huge bloat
<lifeless> on the upside, it seems constant, its not leaking
<jml> lifeless: the call is on Thursday, we can have a call on Monday. At 9am, I'm getting all the Launchpadders together to share each others UDS plans.
 * jml => breakfast.
<flacoste> jml: hi
<lifeless> jml: I got an update that said it would be tuesday, from claire
<lifeless> jml: should I have ignored that?
<lifeless> jml: yes, lets do a call monday
<LPCIBot> Project windmill-devel build #48: STILL FAILING in 1 hr 6 min: https://lpci.wedontsleep.org/job/windmill-devel/48/
<wgrant> StevenK: The DF restore has finished, and I managed to hit it hard enough that it seems to be working OK.
<LPCIBot> Project windmill-db-devel build #250: STILL FAILING in 1 hr 8 min: https://lpci.wedontsleep.org/job/windmill-db-devel/250/
<LPCIBot> Project windmill-devel build #49: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill-devel/49/
<lifeless> morning
<lifeless> interesting http://code.google.com/p/google-protorpc/
<lifeless> similar to lazr.restful in some regards, but more generic
<lifeless> and less deeply typed
<LPCIBot> Project windmill-db-devel build #251: STILL FAILING in 1 hr 9 min: https://lpci.wedontsleep.org/job/windmill-db-devel/251/
<lifeless> hah
<lifeless> http://validator.w3.org/check?uri=http://launchpad.net
<thumper> morning
<lifeless> 8hiya
#launchpad-dev 2012-04-30
<lifeless> who is around
<StevenK> You want a roll call?
<lifeless> hmm, wgrant  - hey, so - do we track enough detail to determine how often contact-this-team is used; by admins, by members, by non-members ?
<lifeless> StevenK: no, I wanted a victim
<wgrant> lifeless: Yes.
<wgrant> lifeless: The abuse:realuse ratio is roughly infinity, I believe.
<lifeless> if we can get a little data on that
<lifeless> I will happily drive a sharp scope reduction on it
<spm> my personal view is that it often shows up high in the UI, so if you're looking for help, then it's THE most obvious place folks see and hence "it's up here, must be important". I'd suggest just dropping it down the UI below lists or something and see what happens.
<wgrant> Hm
<wgrant> AJAX batchnaving probably shouldn't calculate the total every time.
<lifeless> wgrant: why not?
<wgrant> Because it's pointless?
<lifeless> batch lengths are dynamic
<lifeless> some collections more so than others
<wgrant> And their accuracy is probably not particularly critical. I'm not sure the client even respects it if it changes.
<lifeless> mmm, if a collection zeros out, it will get a bug report if we don't update it
<lifeless> I'm fairly sure gmail updates theirs, for instance.
<lifeless> Lets not paint ourselves into a corner either way.
<StevenK> lifeless: Oh sure, because gmail does it is a reason we should?
<lifeless> also, how to stop users reporting bugs
<lifeless> StevenK: gmail is a paginated ajax app with years of experience dealing with users and user expectations.
<lifeless> StevenK: observing what it does is merely one data source we should use. Not using it would be foolish.
<StevenK> lifeless: You hit a nerve.
<lifeless> StevenK: tl;dr - yes, because gmail does it *is* /a/ reason we should do it, transitively, because they also are trying to keep support costs down and user satisfaction up
<lifeless> StevenK: Did I now? I didn't mean to. Whats the nerve?
<StevenK> wallyworld_: O HAI
<wallyworld_> hello
<lifeless> anyhow, we know that few people click next anyhow
<lifeless> so optimising next() to be faster than first-load is less interesting than making first load faster
<StevenK> wallyworld_: I know Friday was a long time ago, but: http://pastebin.ubuntu.com/956524/
<wgrant> lifeless: It's more the sorting that I'm hitting it on.
<lifeless> wgrant: count() doesn't sort though, does it ?
<wgrant> lifeless: No, but changing the sort order of a bug listing does an AJAX request for the new batch, which also returns the count.
<wgrant> So rather than 10ms of DB queries there's lotsams.
<lifeless> buglistings are a bit weird anyhow, not being an API call and all
<wgrant> AJAX batchnavs aren't API calls in general, are they?
<wgrant> Only the pickers are.
<lifeless> FIIK
<lifeless> I would like them to be
 * lifeless dreams
<wgrant> But ++model++ is awesome layering-violation goodness :)
<lifeless> anyhow, the bugs batchnav currently caches too much anyhow, so if the count isn't updating, thats something we can poke deryck to fix at the same time ;)
<wgrant> Dammit mawson, vacuum faster
<wallyworld_> StevenK: looks good. i would use with FeatureFixture to avoid the separate cleaup call
<lifeless> you can also use self.useFixture(..) to avoid cleanup calls.
<StevenK> Not in a doctest I can't.
<wgrant> ALso 'with FeatureFixture'
<lifeless> spelt as f = self.useFixture(MyFixture())
<StevenK> I knew what he meant, so meh.
<lifeless> hmm, i thought I or someone else had added cleanups support to doctests.
<StevenK> wallyworld_: Right, changed to 'with', commiting and pushing
<wallyworld_> StevenK: awesome thanks. happy landing
<StevenK> wgrant: Finally got around to loading up AA last night. Looks like I have to wait, since 1080p causes my 9500GT to melt.
<wallyworld_> lifeless: perhaps. most/all other doctests seem to still use 'with ...'
<wgrant> Kill all doctests :)
<StevenK> Agreed.
<StevenK> I should kill bug-change.txt and merge it into test_bugchanges.py
<wallyworld_> some doc tests are good though
<StevenK> wallyworld_: LIES
<wallyworld_> the ones which are, you know, api documentation are ok
<wgrant> That's a slippery slope.
<StevenK> Some would argue so was adding doctests.
<wgrant> lifeless: You might want to rephrase that self-notification thing to not be stupid
<wgrant> lifeless: Currently it complains that the feature to disable bug self-notifications doesn't disable answer notifications, which is clearly an invalid complaint.
<wgrant> Perhaps it wants the feature to be extended to all notifications.
<lifeless> perhaps; arguably though, a comment on a bug with that option set should not result in a notification
<lifeless> the fact we have naive pub-sub based notifications isn't their problem
<StevenK> wgrant: Is there anything stopping us jumping to rabbit 2.7.1?
<wgrant> No.
<lifeless> hippity hop
<StevenK> My precise laptop has rabbit held at 2.6.1, and apt keeps yelling at me.
<wgrant> Ah, yeah, I should fix that. Might just drop -management for now, since we're not using it yet
<StevenK> wgrant: Did you want my help, or you need to sit down and JFDI?
<wgrant> I'll hopefully get to it today.
 * StevenK tries to figure out how to package auditor up as an egg
<lifeless> woo auditor woo
<StevenK> Yeah, I know it's been a few weeks
<StevenK> Sigh
<StevenK> This web page talks about the 3 python files with django projects and then skips straight into setup.py?
<wgrant> lifeless: Do you happen to know if the row length quoted by VACUUM FULL VERBOSE includes the tuple header?
<lifeless> no
<lifeless> also more freaking init discussions on d-d. Sob.
<wgrant> Oh yeah
<wgrant> Glanced over that this morning and decided I didn't care any more :)
 * StevenK stabs apt on his laptop more.
<StevenK> The Partial Upgrade message from upgrade-manager is getting tiresome.
<wgrant> StevenK: https://code.launchpad.net/~wgrant/meta-lp-deps/no-rabbitmq-management/+merge/104055
<spm> http://somethingofthatilk.com/index.php?id=407 <== wgrant
<spm> 1 guess for where I see you in that panel
<wgrant> Heh
<wgrant> That tends to work well for Launchpad, though :)
<spm> why do you think I posted it in *this* channel :-D
<StevenK> wgrant: r=me
<wgrant> lifeless: Can we upgrade to postgres 9.1 tonight? :)
<StevenK> Haha
<wgrant> StevenK: Thanks.
<StevenK> wgrant: Does that fix the 2.6.1 madness or is there another step?
<wgrant> That should be that.
<lifeless> spm: also hah - http://i.imgur.com/VC9Ye.png
<StevenK> That's terrible
<lifeless> wgrant: up to stub really; oops report looks decent enough
<spm> heh
<lifeless> I'd have no concerns about pulling the trigger if mthaddon is happy
<StevenK> What about DF?
<lifeless> what about tit?
<StevenK> Or do we just blow up the DB, get GSA to fiddle packages and then wait 48 hours for an import?
<lifeless> you can up grade whenever you like
<bigjools> mm tit
<wgrant> StevenK: I intend to refresh DF this weekend anyway
<bigjools> StevenK: combine with restore from prod
<wgrant> So I plan to upgrade to postgres 9.1 at the same time
<wgrant> Hopefully part of prod is done before then.
<bigjools> I usually refresh when ubuntu is released
<wgrant> I considered doing it on Saturday, but we basically need to do it for 9.1 anyway, so might as well wait
<stub> In place upgrade is fine too, which will take maybe 30 mins on that hardware plus fiddling time getting postgresql.conf, pg_hba.conf etc. sorted
<wgrant> StevenK: r-d-b should have sotred out rabbitmq for you
<StevenK> wgrant: I fixed it about 30 seconds after the meta package was published. :-)
<wgrant> Heh
<StevenK> wallyworld: Your headers branch has hit qas.
<StevenK> wallyworld: If you start syncing the staging mailbox right now you should be able to QA the branch on Wednesday morning? :-)
<wallyworld> StevenK: :-P
<alexh> wgrant: the API is supposed to return a 401 when it requires authorization. strikes me as fairly buggy returning an empty set but a non-zero count
<wgrant> alexh: For collections it simply omits the inaccessible items, like a normal search. In this case it's probably a bug that the activity of a public bug requires authentication, but the non-zero count is not a bug.
<wgrant> Counting is very expensive; it is not necessarily correct.
<alexh> wgrant: also not sure why that requires authentication, attachments, etc, do show up
<wgrant> As I said, probably a bug.
<alexh> oh, ok
<wgrant> But in general, inaccessible items in collections are just hidden.
<alexh> wgrant: thanks, I'll give it a try after authenticating
<wgrant> Oterhwise pretty much everything would 401 :)
<czajkowski> aloha
<alexh> hm, also struggling with the auth,
<alexh> "The information provided by the remote application was incorrect or incomplete. Because of that we were unable to identify the application which would access Launchpad on your behalf. "
<alexh> when visiting the authorize-token page
<alexh> I'm sending oauth_consumer_key and oauth_signature{,_method}
<alexh> (and I do get an oauth_token back)
<wgrant> alexh: Which oauth library are you using?
<alexh> none so far, just a regular HTTP post to get the token in the first place
<alexh> following https://help.launchpad.net/API/SigningRequests
<wgrant> alexh: Any reason not to use launchpadlib?
<wgrant> Which does all this for you?
<alexh> I'm using ruby, not python
<wgrant> Ah
<wgrant> What is the URL you're using for authorize-token?
<alexh> https://launchpad.net/+authorize-token?oauth_token=MVlGSWMKbZ6GhsBQwV5H
<alexh> MVl... being the oauth_token I get back from POSTing to https://launchpad.net/+request-token
<alexh> meh, pebkac
<wgrant> heh, what was the issue?
<wgrant> Just tried it manually, works for me.
<alexh> doing GET instead of POST
<alexh> sorry
<alexh> yea, now that works
<wgrant> Great.
<alexh> wgrant: you were right, it was indeed missing authentication
<jml> rvba: just checking, is your testtools "ContainsAll" equivalent to a (probably hypothetical) "IsSubset"?
<rvba> jml: it is indeed.
<jml> rvba: ta, will get to the review shortly.
<rvba> Cool.
<jml> oh wait, it's not
<rvba> jml: why?  We're using it on strings so IsSubSet would sounds strange but conceptually it is rather similar.
<jml> I'm typing it out in the MP, but I would have guessed from the name & the docstring that self.assertThat(X, ContainsAll(Y)) would mean that Y is a subset of X.
<jml> However, it apparently means: for x in X, for all y in Y, y in x
<lifeless> meep
<lifeless> that is indeed surprising
<lifeless> isn't that MatchesAll(Contains(Y)) ?
<lifeless> hmm, MatchesListwise(Contains(Y))
<jml> MatchesAll(*map(Contains, Y))
<lifeless> jml: All? *blink*, well you've read the code.
<jml> matchers could probably use a little more algebra
<jml> https://code.launchpad.net/~rvb/testtools/testtools-contains-all/+merge/104065
<lifeless> I'll read tomorrow, if I should
<lifeless> for now, -> sleep
<jml> :D
<jml> MatchesAll should really be called And or something :\
<jml> rvba: http://pastebin.ubuntu.com/957278/ in case you missed it.
<rvba> jml: thanks.  I was disconnected indeed.
<rvba> jml: now it's my turn to be confused by your review.  "self.assertThat([1, 2, 3], ContainsAll([3, 1]))" is exactly what this matcher does.
<rvba> jml: and the tests prove that.
<jml> rvba: hmm.
<rvba> Maybe the fact that I've written the tests with strings (instead of lists of ints) is confusing.
<jml> rvba: hmm, yeah, that and the way the matcher interface tests work
<jml> rvba: the problem was that I read this:
<jml> +    matches_matches = ['foobar', 'foozbar', 'bar foo']
<jml> as this:
<jml> +    matches_matches = [['foobar', 'foozbar', 'bar foo']]
<rvba> I see.
<jml> rvba: ok. sorry for the confusion.
<rvba> jml: That's funny because I see testtools as a (really nice) was to have nice testsâ¦ and the way testtools tests itself is not really clear ;).  I was myself confused by this matcher interface thingy.
<rvba> Maybe that's a chicken and egg problem :)
<jml> rvba: yeah. sorry about that.
<jml> rvba: in this case, no.
<rvba> jml: that's all right.  As the creator/maintainer of testtools, you've earned my gratitude ;).  I'm a happy user.
<jml> rvba: I think the matcher interface tests are one of the things that make perfect sense once you've figured them out. could probably be made a bit more approachable.
<rvba> Right.
<rvba> There is quite a bit of code which uses it so it's easy enough to understand how it works though.
<jml> crap crap crap
<jml> late for something.
<jml> brb
<jml> rvba: back. and have replied.
<rvba> jml: ta.  I'll fix the code as you suggest.
<jml> rvba: thanks.
<sinzui> jcsackett, Do you have time to review https://code.launchpad.net/~sinzui/launchpad/progressive-enhancement-ftw-2/+merge/103970
<sinzui> jcsackett, I am waiting for allergy medication to work. I do feel very useful at the moment
<czajkowski> jcsackett: ello ello
<czajkowski> sinzui: oh hope you feel better
<sinzui> czajkowski, I am going to write something for the blog today. Danhg and yourself and ensure it makes sense
<czajkowski> danhg is off but I can look over it
<jcsackett> czajkowski: hello. :-)
<jcsackett> sinzui: sure, i can review.
<jcsackett> ...and he signs off just before that makes it through.
<jcsackett> sinzui: r=me on that branch.
<sinzui> thank you
<cjwatson> I'm working on bug 990219 and could use some advice on security.  I think it would be reasonable for build scores to be editable only by launchpad-buildd-admins (an existing celebrity), rather than allowing the owner of a packageset to give themselves arbitrary bonuses (since build scores arbitrate access to a global resource, while packagesets are more local than that).  Is the right way to do this to create a new ...
<_mup_> Bug #990219: Reprioritize package build scores based on packageset <feature> <packagesets> <soyuz-build> <Launchpad itself:Triaged by cjwatson> < https://launchpad.net/bugs/990219 >
<cjwatson> ... interface that just has the score setter in it, and add a new class for that in security.py?
<cjwatson> IPackagesetBuild or something
<cjwatson> Or should I be using some specialised permission like launchpad.Moderate or something for that instead?
<sinzui> cjwatson, We want to normalise the check to a permission like launchpad.Moderate or launchpad.Admin where we check for membership/identity of the celebrity
<sinzui> cjwatson, We have had trouble with this in the past when the object/interface has many users working with it. I do not think that is the case here
<cjwatson> The thing I'm unsure about is that this is a permission applied to only one member of Packageset; most of the other members want to work with the packageset owner
<cjwatson> I'm not clear on how best to express this
<sinzui> yes, this does get tricky
<cjwatson> I suppose it could be launchpad.Admin since that's currently unused for Packageset, but it seems like a hack
<sinzui> cjwatson, maybe not. Does this celebrity have more knowledge and and skill than a member of Launchpad Admin?
<cjwatson> Yes, this celebrity consists of people at the coal-face of running buildds
<sinzui> There are several permission in security.py where membership in one of several teams suffices, and we might think of ~registry as equivalent as ~admin for example?
<cjwatson> Oh, that wasn't what I meant
<cjwatson> AdminByBuilddAdmin already exists, that much is fine
<cjwatson> What I meant was is that it seems like a hack to use launchpad.Admin on IPackageset to control access to setting score, because at some point somebody might want to use launchpad.Admin on IPackageset for something more central to its functionality
<cjwatson> Or at least something with different requirements
<sinzui> I see
<cjwatson> So I was wondering if I ought to break out a separate interface for this, and if there are any similar patterns elsewhere I could look to
<cjwatson> Maybe I'm just borrowing trouble and should apply YAGNI?
<sinzui> cjwatson, I do not see .Moderate used and I think that is appropriate if the celeb has limited control of some properties
<cjwatson> OK, thanks, I'm happy with that
<rick_h_> abentley: ping, can I borrow some eyeballs for a sec?
<abentley> rick_h_: sure.
<rick_h_> abentley: http://paste.mitechie.com/show/653/
<rick_h_> abentley: so trying to move tests from doctest to unit tests and moving this exception to assertRaises, but it's still blwoing up the test and I can't see why here
<abentley> rick_h_: That's surprising.
<rick_h_> ok, just wondering if I've got a case of the Monday's and I'm blind to something
<abentley> rick_h_: An alternative way to express this is testtools.testcase.ExpectedException.  It's a ContextManager, so you can write the code a bit more naturally.
<rick_h_> ah ok, I tried to use the unittest context mgr but it's py2.7 only
<rick_h_> I'll check that out, thanks
<abentley> rick_h_: e.g. http://pastebin.ubuntu.com/958176/
<rick_h_> abentley: awesome, that passed
<abentley> rick_h_: So the thing is that Unauthorized is generated by attempts to *access* transitionToStatus, not by attempts to *execute* it.
<rick_h_> ic, so maybe the unittest assertion is wired a bit wrongly to catch that case
<abentley> rick_h_: Yes.
<rick_h_> abentley: very cool, thanks again. Headache averted wahoo!
<abentley> rick_h_: Or you can pass in getattr as the callee.  E.g. assertRaises(Unauthorized, getattr, upstream_task, "transitionToStatus")
<rick_h_> ah, gotcha. That shows it a bit better. Makes more sense.
<abentley> rick_h_: Although I think it would be far more intuitive, zope.security doesn't provide a way to defend against execution, which is why the restrictions on methods are actually permissions on accessing the methods.
<lifeless> sinzui: hey, got a few minutes to chat about contact this user ?
<StevenK> lifeless: We are on our stand up, feel free to join.
<lifeless> ok, though I don't want to chew up all your times
<lifeless> bah, plurals hard
<sinzui> wallyworld_, I cannot think, but I recall the examples in this page was not good enough: maybe You can make this work for staging inbox for date range: http://stackoverflow.com/questions/3180891/imap-deleting-messages
<wallyworld_> thanks, will look
<sinzui> pause@!
<sinzui> I have no mumble
 * StevenK stabs buildout
<StevenK> It's been running for eight minutes with no output.
<StevenK> wgrant: The privacy portlet contains the code for changing the background colour of ... well, any privacy portlet.
<StevenK> Sigh, privacy javascript
<wgrant> StevenK: That should all die.
<wgrant> It's not used any more.
<StevenK> wgrant: Do you think it's worth writing a test for, or just rip it out and see that nothing breaks?
<StevenK> Oh, look at that. buildout now takes twelve minutes. :-(
 * StevenK attempts to get wgrant's attention again.
<wgrant> StevenK: Writing a test for the absence of some code is silly.
<wgrant> YOu must have a nice CPU, if a fresh buildout only takes 12 minutes
<StevenK> wgrant: I upgraded to Precise yesterday afternoon, still watching for issues. That could be one.
<StevenK> The only on-going issue is that gnome-do has broken into tiny pieces.
<wgrant> StevenK: Oh, that'll probably mean you're building with 2.7 now, if you weren't already.
<StevenK> Ah ha
<lifeless> yay 429 is official
<StevenK> lifeless: Hm?
<lifeless> http://www.rfc-editor.org/rfc/rfc6585.txt
<StevenK> The loss of gnome-do is affecting my muscle memory :-(
<wgrant> lifeless: Ah, finally.
<StevenK> wallyworld_: Due to wgrant being a little preoccupied, do you want to quickly review https://code.launchpad.net/~stevenk/launchpad/privacy-portlet-colour-change/+merge/104188 ?
<wallyworld_> StevenK: sure.
* wgrant changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: wgrant | Firefighting: - | Critical bugs: 3.47*10^2
<wgrant> wallyworld_: Thanks
<wallyworld_> StevenK: do you recall, on qas the bug notification scripts are now run via cron right?
<StevenK> wallyworld_: I don't, I'm afraid.
<wallyworld_> StevenK: do we know why the recent privacy portlet changes triggered this bug? were we not calling hide_privacy_notification() before?
<StevenK> wallyworld_: It does call it.
<wallyworld_> StevenK: so why is the issue showing up now and not before the info type ui changes?
<StevenK> I seem to recall jcsackett being involved in turning that off, but I'm not certain what he did.
<wallyworld_> StevenK: np. i just wanted to be extra cautious deleting code we were not 100% sure of
<StevenK> wallyworld_: wgrant is certain it should die, at least.
<wgrant> wallyworld_, StevenK: That code dates from back when the privacy banner was hideable.
<wgrant> You could click X on it and it would hide, and turn the privacy portlet red instead.
<wgrant> That went away when hiding was disabled, but the code was never removed.
<wgrant> However.
<wgrant> We should track down why this is only showing up now.
<StevenK> wgrant: Ah ha.
<wgrant> The old privacy switching code doesn't have that bug.
<StevenK> wgrant: The hide JS looks for .portlet.private, and the legacy code changes the portlet class to public before calling that hide code.
<wallyworld_> StevenK:  ok. r=me
<wgrant> Ah
<wgrant> Erm.
<wgrant> Wut?
<StevenK> wgrant: var privacy_portlet = Y.one('.portlet.private');
<StevenK> wgrant: That's in the privacy JS
<wgrant> I was wuting at the message from wallyworld followed less than a second later by a 180s timeout
<StevenK> Ah
<wgrant> StevenK: That's what I suspected, except I was expecting body.private instead of .portlet.private
<StevenK> Yes, that is somewhat interesting.
<wallyworld_> wgrant: i docked the laptop and that hickups the network for some reason
<StevenK> wallyworld_: I'm not sure if you saw my explanation as to why this code is being hit now.
<wallyworld_> StevenK: no, missed it
<StevenK> [09:56] < StevenK> wgrant: The hide JS looks for .portlet.private, and the legacy code changes the portlet class to public before calling that hide code.
#launchpad-dev 2012-05-01
<wallyworld_> ah right, glad we found out how it was happening
<StevenK> Let me toss that ec2 and then forage for some breakfast.
<RyuGuns> Hey guys.
<RyuGuns> I've been in the process of registering a new account...
<RyuGuns> But I am not allowed to.,.
<RyuGuns> Wait, actually, nevermind.
<RyuGuns> How do I merge accounts?
<bobweaver> RyuGuns,  https://help.launchpad.net/YourAccount/Merging
<bobweaver> :)
<RyuGuns> :)
<RyuGuns> Funk.
<RyuGuns> How do I change my ID?
<RyuGuns> Nevermind.
<lifeless> RyuGuns: bobweaver: you might prefer #launchpad for customer support
<RyuGuns> https://launchpad.net/ubuntu+mobile+phone
<RyuGuns> Anyone interested?
<StevenK> wgrant: Free to review, or still distracted by *redacted*?
<StevenK> And no wallyworld.
<StevenK> Maybe bigjools' tech has broken TPG.
<wgrant> StevenK: 'sup?
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/hack-itemwidget/+merge/104202
<wgrant> oh god
<wgrant> StevenK: Why are you setting term.name now?
<StevenK> wgrant: Because the test in test_itemswidgets requires it
<StevenK> And I'd rather just use self.assertRenderItem
<wgrant> StevenK: r=me before lp melts again
<StevenK> Haha
<wallyworld> wgrant: StevenK: have you seen bug emails with the same info twice? eg "** Visibility changed to: Private" is listed twice in the one email
<wallyworld> i seem to recall i may have seen this but am not sure
<StevenK> I can't recall that one, personally.
<StevenK> I a note to track down why information type: Public -> Private turns up twice when you change the information type via JS.
<wallyworld> StevenK: that's what i'm seeing also but i'm fairly sure it happened before the info type field was introduced
<wgrant> I don't recall seeing that before.
<wgrant> I would suspect a regression.
<wallyworld> wgrant: i seem to recall it may have happened for eg "** Visibility changed to: Private" which was before disclosure
<wallyworld> but i can't be 100% sure
<wgrant> That stuff was dangerously refactored a few months ago
<wgrant> It may have been a regression then.
* wgrant changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 3.47*10^2
<czajkowski> aloha
<jml> czajkowski: hi
<czajkowski> morning
<jml> ok.
<jml> so I'm going to do some Launchpad hacking.
<jml> stand back.
<wgrant> Uhoh
<jml> does it work in precise yet?
<wgrant> We're doomed
<wgrant> Um
<wgrant> Well
<wgrant> Maybe.
<wgrant> You'll need python2.6
<jml> I have Python 2.6!
<wgrant> It might Just Workâ¢, then.
<wgrant> I fixed the other issue yesterday.
<wgrant> But haven't retried since.
<czajkowski> wgrant: was sure that was one of the resonas why gmb was setting up an Ec2 for uds lp clinic was it not working on precise just yet
<jml> KeyError: 'STORM_CEXTENSIONS'
<wgrant> Doing what?
<wgrant> How old is the tree?
<jml> I just pulled
<jml> but I've forgotten everything I need to do to update
<wgrant> make clean build, to be safe
<jml> ah, ok.
 * wgrant tries.
<jml> Error: pg_config executable not found.
<jml> I get that in the middle of a 'make build'
<wgrant> Do you have launchpad-developer-dependencies installed?
<wgrant> I shall return after dinner. Hopefully germanium will give me everything I need while I eat.
<jml> wgrant: heh
<jml> wgrant: ta
<jml> wgrant: no, I don't. It seems to be broken.
<jml> ah, you need python2.6-dev
<jml> I've just got python2.6 from https://launchpad.net/~fkrull/+archive/deadsnakes
<wgrant> jml: Yeah, I'll probably replace it with python-all-dev and co. once I check it functions without 2.6
<wgrant> We mostly just use the default version, but some things still point at 2.6, I believe.
<wgrant> Everything of note has been updated to just use the default, so it will hopefully work.
<jml> coole.
<jml> I'll keep hacking in my lxc for the moment.
<wgrant> Probably better for your sanity anyway.
<jml> yeah.
<wgrant> Interesting
<wgrant> pg_createcluster now fails if it's run in a nonexistent locale.
<wgrant> dpkg doesn't notice.
<wgrant> lifeless: Any objections to dropping !precise host support from /Running/LXC?
<wgrant> The instructions are like half the size.
<lifeless> +1
 * wgrant mauls.
<wgrant> jml: Just ran various tests successfully without 2.6, so I'll drop it from the deps.
<jml> wgrant: yay
<jml> wgrant: do you know why TestArchivePrivacySwitching is in LaunchpadZopelessLayer?
<wgrant> jml: Probably because whoever wrote it was wrong.
<jml> wgrant: heh.
<wgrant> jml: It should be ZopelessDatabaseLayer or DatabaseFunctionalLayer, since it probably doesn't need librarian etc.
<jml> wgrant: yeah, trying DFL now.
<wgrant> The difference between those two is now pretty much just the permission model.
<wgrant> Zopeless is still PermissiveSecurityPolicy
<jml> ah cool.
<jml> wgrant: how much work does "pretty much" hide?
<wgrant> jml: Working out what FunctionalTestSetup etc. does that is different.
<wgrant> I merged most of it in a large branch series in September, but am yet to obtain the courage to complete the merge.
<wgrant> Becauze Zope testing setup stuff is somewhat intimidating.
<wgrant> It's also not clear quite how it's best to do it.
<jml> yeah.
<wgrant> But I suspect an attribute on the test class to specify the security policy might work.
<wgrant> s/work/not be as terrible/
<jml> that's what I was thinking
<jml> and then one by one change the test classes to do something better.
<jml> I wonder if there's a better mechanism than attribute (subclassing maybe?)
<wgrant> StupidLegacyTestCase? :)
<jml> well, we need to separate concerns
<jml> StupidLegacyPermissiveTestCase
<wgrant> Heh
<jml> actually, you could probably do something with testtools's `run_tests_with` thing.
<jml>         publisher.prepareBreezyAutotest()
<wgrant> That's actually not a terrible idea.
<wgrant> Run away
<jml> oh yeah, that needs the librarian
<wgrant> It does, yeah.
<wgrant> Because inserting fake LFAs is hard, I guess.
<wgrant> It tries to upload chroots.
<wgrant> It could be taught not to participate in such stupidity, though.
<wgrant> I'm not sure if our test suite's antics are back to being hilarious rather than depressing yet.
<jml> I'm guessing getPubSource has side-effects that belie its name
<wgrant> Oh yes
<wgrant> makePubSourcePlusSomeOtherBitsActually
<wgrant> You can usually get away with the factory's makeSourcePackagePublishingHistory these days
<wgrant> SoyuzTestPublisher is deprecated and new usage of it incurs wrath.
<jml> oh
<jml> that's nice
<jml> ISTR the last time I tried this sort of thing the accepted wisdom was "use SoyuzTestPublisher because you'll never be able to get it right any other way"
<jml> is there a safe way to use the Librarian in tests without using a layer that provides it?
<lifeless> no
<lifeless> the layer is a thin shim on the fixture now
<wgrant> jml: The appropriateness of STP depends on the test.
<lifeless> but you still need all the bits joined together
<wgrant> You can sometimes use the FakeLibrarian.
<wgrant> Translations uses it for something in like one test.
<jml> If there were a safe way, then lots of test runs could be made a bit faster easily. I could, for example, switch this test to ZopelessDatabaseLayer and then just use that safe way to run this test with the librarian.
<lifeless> zopeless is no cheaper than zopeful
<jml> lifeless: I wasn't contending that point.
<jml> ZopelessDatabaseLayer is heaps cheaper than LaunchpadZopelessLayer though.
<jml> librarian, memcache and rabbit all take their toll.
<cjwatson> I don't seem to have much luck with FakeLibrarian, usually.  Maybe I'm not smart enough.
<cjwatson> I tend to get incomprehensible storm tracebacks.
<jml> battery threat!
* rick_h_ changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: rick_h | Firefighting: - | Critical bugs: 3.47*10^2
<jml> https://code.launchpad.net/~jml/launchpad/narrow-commercial-celebrity/+merge/104236 up for review.
<jml> rick_h_: poor chap, you always seem to end up with my sad offerings.
<rick_h_> jml: hah, all good
<jml> I have to go find electricity. Will be back sooner than you'd like.
<czajkowski> and yet he comes back for more
<czajkowski> cant be that bad
<rick_h_> jml: ok, going to grab some grub, so will take me longer than you'd like :P
<jml> ok. I'm back.
<jml> rick_h_: are you far into reviewing that branch, because I thought of some cleanups that I can tack on to it.
<czajkowski> jml: we have date/times now for the LP clinic at uds https://wiki.ubuntu.com/UDS-Q/LaunchpadClinic
<rick_h_> jml: I'm peeking at it, but please go ahead and update and I'll get back to it in 10 then
<jml> rick_h_: ok, finally done
<rick_h_> jml, ok, working on another review atm, but will get to it shortly
<jml> rick_h_: 30s per test run has a surprisingly strong negative effect on speed.
<rick_h_> jml: tell me about it :/
<wgrant> jml: I'm pretty sure it hasn't become worse since you defected. I guess you've just repressed the tramautic memories :(
<jml> wgrant: oh, I'm sure that's the case.
<StevenK> The brain does repress bad memories.
<jml> otoh, the nice thing with working on LP is there's so much obvious stuff to do
<jml> it's very hard to get stuck.
<wgrant> Indeed.
<wgrant> Indeeeeeeed.
<wgrant> So much to delete.
<wgrant> Like Translations, Answers, Blueprints.
<wgrant> And Soyuz to rip out.
<jml> or update, or fix, or refactor.
<jml> yeah
<jml> ripping out the buildmaster would be fun
<jml> step 1, rationalize the transactions
<wgrant> It, Soyuz and Translations have no business at all being in LP core.
<wgrant> The others might have some justification.
<jml> step 2, xmlrpc (or whatever) interface
<jml> step 3, yoink!
<wgrant> Yep
<wgrant> It just turns into a process which bridges the LP and buildd XML-RPC interfaces.
<jml> chunks of lp.codehosting could be ripped out pretty easily too.
<wgrant> I'm likely to rip out lp.services.sshserver into a separate project in the next few months.
<wgrant> As part of destroying poppy.
<wgrant> It's mostly self-contained.
<jml> wgrant: oh, I made lp:txsshserver once upon a time with roughly that intent
<jml> wgrant: maybe you should take over that project when you do it?
<wgrant> Maybe indeed.
<jml> it has no users.
<wgrant> All the better!
<jml> hmm. maybe I killed it :\
<wgrant> One thing I do like about Code(hosting) is that the architecture isn't entirely stupid, unlike say most of the rest of LP.
<wgrant> apart from branchrevision.
<jml> wgrant: aww shucks.
<jml> is the a graph of LP LoC over time?
<wgrant> <http://www.ohloh.net/p/launchpad>, but cjwatson had a better one of the delta
<wgrant> I'm not sure how accurate ohloh's is.
<rick_h_> heh, wasn't ready to seem my name up there in that
<rick_h_> but as a re=me so it's ok :)
<rick_h_> jml: so you're not removing the concept of commercial ppas, but this single getCommercialPPA helper then?
<jml> rick_h_: that's right.
<rick_h_> ok, cool
<StevenK> ohloh's one tends to be out-of-date and imprecise
<jml> rick_h_: if we remove the commercial concept entirely, well, this is half (hah!) the work
<rick_h_> jml: just making sure I'm following, saw the sea of red but some bits left and wanted to make sure I was 'getting' it right
<jml> :)
<StevenK> jml: The Soyuz's team blood, sweat and tears (mostly tears) went into making IArchive.commercial!
<jml> StevenK: sunk cost fallacy.
<wgrant> Woah
<jml> StevenK: Sorry, I meant to say that we really appreciate it and hope some day we can return the favour.
<wgrant> Never seen someone brave enough to invoke that on LP before :)
<StevenK> jml: Oh, I know it's a sunk cost, I'm just saying think of our tears. :-)
<rick_h_> tears wash away easily with some soap and water...all good
<StevenK> wgrant: You mean what I said?
<wgrant> No, the sunk cost fallacy.
<StevenK> Ah.
<wgrant> It could be applied to harmlessly remove 90% of our functionality :)
<rick_h_> how dare you use logic and management terminology in here! :)
 * StevenK removes 90% of the JS, thereby removing the need for convoy or jsbuild.
 * rick_h_ watches the site go boom!
<StevenK> It already was earlier today.
<rick_h_> phew, glad I missed it then
<wgrant> Production thrice, qastaging once.
<jml> wgrant: well, there you'd compare cost/benefit of deleting with cost/benefit of user migration
<wgrant> Fun day.
<rick_h_> ugh, what went kaput?
<jml> oh, also, perplexing thing: https://code.launchpad.net/~jml/txsshserver/trunk (200); https://code.launchpad.net/txsshserver (404)
<wgrant> jml: Deactivated project, probably.
<jml> ah ok
<wgrant> "This project is currently inactive"
<jml> rick_h_: anyway, I don't have commit so if you approve of that branch please land it for me.
<rick_h_> jml: ah ok will do
<rick_h_> going to have to footnote you on my ec2 bill :P
<jml> rick_h_: I left two LP instances running accidentally, I think to support some pair programming. I have a $300+ bill from last month.
<rick_h_> ouch!
<jml> wgrant: how come you are so low on ohloh?
<wgrant> jml: Hm?
<rick_h_> I'm constantly afraid of that. I have two of my own I keep running to have to keep thinking "hmmm, 2 ok, >2 doh!"
<sinzui> wallyworld, got a bill for $1,500
<sinzui> It ran for 45 days
<rick_h_> oooh...45 days?!
<wallyworld> sinzui: for ec2?
<rick_h_> that must have been pre the large instances?
<jml> wgrant: you don't appear on http://www.ohloh.net/p/launchpad/contributors
<sinzui> yes
<wallyworld> ah, my bill
<rick_h_> I'd have thought 45 days would be higher
<wgrant> jml: I'm probably on like the third page.
 * wgrant hunts.
<jml> wgrant: 2nd
<rick_h_> when I see these start ups with 100s of ec2 servers I cringe at that monthly bill
<wgrant> Ah, not too bad.
<wgrant> I am the top committer of the last 12 months, though :)
<StevenK> I think I'm on like the 6th page. :-(
<sinzui> We can all run starry-eyed into the arms of canonistack
<wgrant> Although sinzui is close behind.
<jml> hmm.
<rick_h_> sinzui: yea, are there instructions for that?
<jml> I don't know how to get that report from ohloh
<StevenK> sinzui: Sounds good to me. Except no one seems to care.
<wgrant> jml: Code Analysis
<wgrant> jml: You can click on people in the legend to turn them off.
<wgrant> (ie. to dispose of PQM)
<wgrant> I'm not quite sure how it picks who to show.
<wgrant> Looks vaguely like the top 5 of the last year and then the top 5
<wgrant> of all time
<sinzui> me? I have done bugger all for 12 months. I am atrifying.
<rick_h_> jml: so side curiousity here, is this how people are getting listed in the software center? They have a commercial ppa and that's listed out as apps in software center?
<jml> rick_h_: yeah, that's right.
<jml> rick_h_: but only for commercial (i.e. they cost money) apps
<rick_h_> jml: right, the newish paid apps stuff
<jml> yep
<rick_h_> ok, just curious, wasn't sure how that stuff worked and this review has the brain thinking
<wgrant> jml: Aren't the proprietary free apps also there?
<wgrant> eg. Vendetta Online
<wgrant> Or whatever that free one is
<jml> this work we're doing with LP now is aimed at people being able to upload apps to developer.ubuntu.com and have them packaged and published to a PPA automatically
<jml> wgrant: I don't know about proprietary free.
<cjwatson> wgrant,jml: http://people.canonical.com/~cjwatson/tmp/loc.png is a currently-up-to-date graph of delta per commit.  But it's kind of hard to read; it looks mostly nice and negative but the net change over that time period is in fact +2755.
<jml> wgrant: I know that libre+gratis goes to extras (or the archve)
<wgrant> jml: I like to pretend that extras doesn't exist :)
<jml> :)
<wgrant> cjwatson: :(
<jml> cjwatson: do you have a graph of the integral of that?
<cjwatson> jml: that's ohloh's graph, isn't it?
<cjwatson> I guess you might want zoomed in and zero-based
<jml> oh yeah.
<jml> I think I'll just hook up bzr & ggplot2 in my own spare time :)
<StevenK> Some counter on lpqateam.c.c would be nice.
<jml> what I want is a command that tells me my score.
<rick_h_> haven't you heard, you need graphite graphs these days to be cool
<StevenK> I just want to see if I'm negative.
<rick_h_> with hourly commit points and fancy colored moving charts
<rick_h_> and we all put them on a display on the wall over our desks
<StevenK> rick_h_: Speak for yourself, hipster.
<rick_h_> lol
<rick_h_> I'm on LP, I think that disqualifies me for any hipster movements
<StevenK> Haha
<wgrant> Nah
<wgrant> GitHub's too mainstream
<rick_h_> when we get some BDD JS going then we'll chat about rushing house hipster
<jml> StevenK: yeah, if I'm negative that's a good thing to know. But I also want to be able to compete with people
<rick_h_> where's jono when you need some new achievement badges
<rick_h_> "negative LoC leader for month of May"
<StevenK> If we hook the LOC metric into Ubuntu Achievements, someone will get fired.
<rick_h_> lol
<jml> StevenK: how so?
<StevenK> ... or I'll quit in disgust.
<StevenK> jml: There's an implied :-P
<cjwatson> jml: but, http://people.canonical.com/~cjwatson/tmp/loc-cum.png
<jml> StevenK: oh, I just didn't (don't yet) get the joke.
<jml> cjwatson: thanks!
<StevenK> I'm waiting for my UI work to end on prod.
<wgrant> I wonder how it changes if we filter out database/schema
<wgrant> Since that's append-only.
<StevenK> Then I can rip out 2,000 lines of stuff.
<cjwatson> jml: (easy once I remembered 'smooth cumulative'.  gnuplot is ugly as sin but quick to hook trivial stuff up in.)
<jml> cjwatson: ah cool.
<wgrant> bzrlib + matplotlib ftw
<jml> yeah. I've enjoyed the stuff I've done recently with R & ggplot2
<wgrant> Enjoying R? I think you're doing it wrong.
<jml> webulating graphs is too hard though
<jml> i.e. taking a one-off graph made and turning into something that can be hit on the web and is always relatively up-to-date
<jml> wgrant: the thing I really like is that it's really easy to use my data analysis again at a later point. Mostly my Python graphing stuff doesn't seem to work out like that.
<wgrant> Yeah
<jml> Maybe I should try that panda thing GvR posted on G+ about.
<jml> also, R is actually pretty good for interactive mucking around with data
<jml> even if it's rather poor considered as a language.
<jml> distributionmirror_prober: is that a candidate for splitting out?
<wgrant> Yes.
<wgrant> It should in all probability be a separate database.
<wgrant> The interactions with LP internals are minimal.
<wgrant> But it should probably be deleted and replaced with an existing third-party solution
<jml> ooh
<jml> its tests are an affront.
<jml> I'd like to fix them, but removing them & the code they test entirely would be much better.
<wgrant> Exactly.
<wgrant> Lots of Launchpad components like to pretend they're solving a unique problem.
<wgrant> So stuff like that happens :)
<jml> :)
<jml> james_w: hello
<james_w> hi
<jml> I'm wondering whether I should make a celebrity team for which membership means "you can set commercial"
<jml> or whether I should re-purpose the software_center_agent celebrity somehow
<wgrant> Or add a field to distribution which permits it.
<jml> wgrant: e.g. Distribution.commercial_team
<jml> ?
<wgrant> nasty_proprietary_software_people, but yeah :)
<jml> hmm.
<deryck> wgrant, hey.  I saw something in scrollback about lp being down for you guys.  are there incident reports for this?  or anything I can look at.
<jml> I need a better name than either of them.
<wgrant> deryck: https://wiki.canonical.com/IncidentReports/2012-04-26-LP-release-performance-issues covers the first two of the day's issues, the other one is in the hands of IS, I believe.
<jml> 'commercial_admin' would be pretty good if it weren't already a heavily-loaded term
<wgrant> Yeah
<deryck> wgrant, thanks!
<wgrant> deryck: Alternatively we can replace commercial with a shut_up flag that anyone can set, I guess.
<wgrant> s/anyone/the archive owner/
<wgrant> Bah
<wgrant> jml: ^^
<deryck> heh.
 * deryck votes for shut_up flah, too, FWIW. :)
<deryck> flag
<wgrant> jml: I thought my team name was pretty catchy, though.
<jml> bigjools seemed to dislike the idea
<jml> wgrant: sure, it has zing. It could be easily misunderstood though.
<rick_h_> abentley: ping, can you take a peek at this for me when you get settled? https://code.launchpad.net/~wallyworld/launchpad/revoke-access-delete-subscriptions-job/+merge/104198
<abentley> rick_h_: Yes, I'll have a look at that.
<rick_h_> abentley: want to make sure it's all celery approved, not sure about the any special needs/better ways of doing things for that.
<rick_h_> thanks!
<jml> wgrant: as it would read to me as "all of the propriety uploaders to d.u.c" rather than "the d.u.c bots"
<wgrant> True, true.
<james_w> jml, I'd lean towards re-using the celebrity, but wouldn't be against a field on distribution I don't think
<jml> james_w: one thing about the celeb is that (I think) it's a Person not a Team
<wgrant> jml: It can be either.
<jml> wgrant: I mean that currently it is a Person.
<wgrant> People are teams, war is peace, etc.
<wgrant> Ah
<jml> wgrant: I don't know what the fallout of changing it would be.
<wgrant> It would break.
<wgrant> I think.
<jml> well, yeah, that's generally what we mean by "fallout of change"
<wgrant> Since I believe the XML-RPC API may actually set up an interaction as it.
<wgrant> Which won't work if it's a team.
<wgrant> But I may be misremembering, and it just passes it in as arguments to horrid methods, in which case it might work.
<wgrant> There's one easy way to find out...
<jml> I'm more worried about how we use it on our side
<jml> that's harder to co-ordinate.
<wgrant> Howso?
<wgrant> How do you use it on your side?
<wgrant> I thought you mostly poked the xmlrpc-private API
<jml> I don't know.
 * jml looks
<wgrant> Ah
<wgrant> There's a single xmlrpc-private call, getOrCreateSoftwareCenterCustomer
<wgrant> Rest must be through lazr.restful.
<jml> yeah
<wgrant> So, you could create a team, add the agent to it, then make that team the celeb.
<rick_h_> jml: have a bug to tie the MP against please?
<rick_h_> well guess there's not a lot of QA for this removed is there :/
<jml> rick_h_: https://bugs.launchpad.net/launchpad/+bug/992622
<_mup_> Bug #992622: getCommercialPPAs is not needed <tech-debt> <Launchpad itself:New> < https://launchpad.net/bugs/992622 >
<jml> rick_h_: as you wish.
<rick_h_> jml: yea, that ec2 land wants things to all be setup right, picky little thing. :) thanks. Landing now
<jml> rick_h_: thanks! :)
<jml> hmm
<jml> I have to write this code somehow.
<jml> I prefer the 'shut_up' field approach, but suspect that it'll be rejected as Whiggery.
<jml> james_w: am probably going to start implementing it as field on Distro
<wgrant> My vote goes to shut_up.
<wgrant> I don't see a good reason to restrict it.
<james_w> shut_up on archive, or subscription?
<wgrant> Probably archive.
<wgrant> But I'm not quite sure.
<james_w> either way I think I prefer that too
<james_w> assuming it's roughly similar in implementation time
<sinzui> jcsackett, ping
<james_w> no need for this to be special
<jml> I guess one archive owner could use it to secretly upload evil stuff to a PPA without other owners finding out
<jcsackett> sinzui: pong
<jml> james_w: I think they are both similar and both fairly small.
<wgrant> jml: Hm?
<jcsackett> rick_h_: enjoying your first day of full reviewing? :-)
<sinzui> jcsackett,  is there a card on the board that interests you?
<rick_h_> jcsackett: wheeeeee
<wgrant> jml: AIUI the flag would just control subscription notifications.
<james_w> if it has to be available in the UI then it will probably be more work to implement, and be quite LOC-positive
<jcsackett> sinzui: i was looking at banners without javascript.
<wgrant> jml: "subscription" refers to packages from the archive, not upload notifications.
<jml> wgrant: ah, not upload notifications.
<wgrant> Upload notifications go to the uploaders, which are the owner and anyone with an ArchivePermission
<sinzui> jcsackett, do you want to talk about it?
<jcsackett> sinzui: yes. hangout?
<sinzui> yes
 * sinzui looks for headphones
<jml> so, it would just be suppress_subscription_notifications or something to that effect.
<jml> ok.
<jml> I'm going to relocate to somewhere that serves food & has internet.
<sinzui> jcsackett, I am in the google messenger  hangout
<jcsackett> hm. it's telling me you're offline
<sinzui> My head is
 * sinzui tries again
<rick_h_> abentley: thanks for checking that out, appreciate the assist
<abentley> rick_h_: no problem.
<jml> I haven't submitted a db patch since we switched to nodowntime stuff. what do I need to know?
<jml> https://dev.launchpad.net/Database/LivePatching doesn't seem to mention anything about adding, removing or renaming columns
<jml> james_w: so I'm thinking of doing this shut-up patch by re-purposing 'commercial' and then following up with a db patch to rename it.
<jml> james_w:  thoughts?
<jml> anyone?
<rick_h_> jml: looking for the doc
<rick_h_> my understanding is you need to have a MP against the devel-db branch and get it ok'd by db peeps (lifeless/etc) and then it has to land first
<cjwatson> jml: https://dev.launchpad.net/PolicyAndProcess/DatabaseSchemaChangesProcess
<rick_h_> there you go and https://dev.launchpad.net/WorkingWithDbDevel
<jml> that doc doesn't say anything about removing or renaming columns
<cjwatson> adding columns comes under cold patches
<jml> nor is there anything I can see that categorizes new columns as either 'hot' or 'cold'
<jml> cjwatson: ah
<cjwatson> by virtue of not being explicitly under hot patches
<jml> right.
<cjwatson> so it'll be a fastdowntime thing
<jml> thanks.
<cjwatson> if you're removing columns, you'll need to make sure that no appservers are using the old columns first via a nodowntime deployment (I don't know if there are stricter requirements beyond that)
<jml> just got a weird error trying to run 'ec2 test': http://pastebin.ubuntu.com/960374/
<jml> does the erstwhile Launchpad team hang out somewhere else these days?
<rick_h_> sorry, no idea there
<rick_h_> repeatable error? e.g. not temp bzr connection issue?
<jml> I'm trying to repeat it now.
<jml> just repeated it.
<jml> it happens in the initial connection.
<rick_h_> k, sec, pulling that branch ( was going to peek at it for review anyway) and I'll test it
<cjwatson> jml: public IP problem again?  what does checkip.amazon.com say?
<cjwatson> sorry, checkip.amazonaws.com
<jml> heh, it might be because...
<jml> cjwatson: yeah, that seems to be it.
<cjwatson> when I was landing a branch from Millbank last week I hardcoded Millbank's external address in ec2test/account.py
<jml> cjwatson: I'm actually in a pub tethered to my phone
<rick_h_> what's the 'public ip problem'?
<jml> cjwatson: but checkip says 10.92.29.164 and mumak.net (which I'm ssh'd into) says something else
<jml> cjwatson: I'm angry enough to patch lp-dev-utils to have an option for that.
 * jml hulks out
<rick_h_> ruh roh
<rick_h_> ok, well running fine here so glad it's something special
<jml> rick_h_: thanks.
<cjwatson> I have no idea why checkip thinks it's clever to honour X-Forwarded-For.
<jml> Well, what is honour, if not an occasional rejection of the pragmatic?
<cjwatson> We could use http://whatismyip.akamai.com/ instead
<cjwatson> WFM
<jml> me to
<jml> oo
<cjwatson> rick_h_: the problem is that amazon's checkip service is stupid and if you're connecting to it through an HTTP proxy it returns the address that the proxy sees (via the X-Forwarded-For HTTP header)
<cjwatson> this is useless if you're trying to find the address amazon will see for you, since proxies are often on private networks
<rick_h_> cjwatson: ah ok, and this is setup as the address you can ssh from to the new instance?
<cjwatson> right
<rick_h_> gotcha
<rick_h_> bah, can't edit a review comment after the fact? oh well
<rick_h_> jml: small note on my end, but yea the others will be better to go over things.
<james_w> jml, +1 on repurposing commercial
<james_w> jml, if we strip away the other meanings of it, rename it, and allow anyone to set it if they own the PPA then it works for us, is a pretty minimal change, and IMO reduces the maintainence burden by taking something special-purpose and arcane and makes it pretty obvious
<james_w> rick_h_, looks like your comment on https://code.launchpad.net/~jml/launchpad/drop-special-commercial-permissions/+merge/104270 got truncated
<jml> james_w: yeah. and we've actually got line count credit on this already from deleting getCommercialPPAs
<rick_h_> james_w: well, I was going to say I'll invite people from the permissions/etc squad but then realized they were already up there and missed hitting delete before submitting
<rick_h_> and I can't edit it back out, so I fail
<james_w> rick_h_, cool
 * jml has to go soon :(
<jml> ok. this doesn't seem to actually *work*, but it's a start: https://code.launchpad.net/~jml/lp-dev-utils/public-ip/+merge/104273
<jml> I have to go.
* rick_h_ changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Criti
* rick_h_ changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 3.47*10^2
<lifeless> flacoste: hiya
<flacoste> hi lifeless
<lifeless> flacoste: hows the sprint going ?
<lifeless> bwah... deryck: around ? if so, could you put bug 901892 as the external board # on https://canonical.leankitkanban.com/Boards/View/14028616/101173433
<_mup_> Bug #901892: bug search cache makes next/prev return old data <bug-columns> <Launchpad itself:Triaged> < https://launchpad.net/bugs/901892 >
<deryck> hi lifeless
<deryck> lifeless, done
<lifeless> deryck: thanks
<jelmer> hi deryck, lifeless
<deryck> hi jelmer
<jelmer> lifeless: I'm wondering if it would make sense to kill the code review notification emails; they just seem to be noise at this point. Do you have any opinions on that?
<jelmer> IIRC you brought it up a while ago, but I can find the relevant log at the moment to confirm.
<lifeless> code imports ?
<lifeless> yes, I proposed removing at minimum all the non-action ones (successes, initial-failures)
<lifeless> I have no particular opinion about non-initial failures.
<jelmer> It's probably easiest to remove both at the same time
<jelmer> I've stopped paying attention to the non-initial failures, and I haven't seen anybody else use them.
<lifeless> sure
<lifeless> mpt: hoya
<mpt> hi lifeless
<lifeless> so, what questions are you and ev trying to answer with this mtbf thing ?
<mpt> lifeless, for the MTBF specifically, the first question: (1) How reliable is Ubuntu?
<mpt> For example: Is it more reliable than it was last week? Is it more reliable than the previous release? And how much of a difference would it make if all Ubuntu users had installed all updates?
<sinzui> what is MTBF? mounted tabernacle bee floggers
<lifeless> yes
<lifeless> mpt: those are good questions
<lifeless> mpt: I have one; how do we define reliable, and is it a definition users would agree with ?
<mpt> lifeless, that introduces the issue that our definition of reliability will expand over time
<mpt> lifeless, in Ubuntu 12.04 it is "programs other than the kernel don't crash"
<mpt> In Ubuntu 12.10 it may be something like "programs (including the kernel) don't crash, and don't hang for more than 20 seconds"
<mpt> Depending on time and difficulty, it may also include other kinds of errors: package installation failures, debconf prompts, etc
<mpt> So, one thing we've discussed is keeping track of when we started recording error type, to fairly answer questions like: "Is Ubuntu 13.04 more or less reliable than Ubuntu 12.10, using the same standards that we were using for Ubuntu 12.10?"
<mpt> otherwise each successive release would seem less reliable than the last, even if it's more reliable. :-)
<mpt> lifeless, but there will always be swathes of problems that aren't caught in an automated way, e.g. "this program thinks it copied all these files but it didn't".
<mpt> And a user might classify that under "reliability".
<mpt> If a test suite is the fence at the top of the cliff, Whoopsie and Daisy are the ladies with pens and clipboards standing at the bottom.
<lifeless> mpt: I like that image
<lifeless> mpt: so, are you factoring in time in your equations? And how?
<mpt> lifeless, no, we'd completely overlooked it
<mpt> If by "time" you mean "uptime"
<mpt> The current formula implicitly assumes that every machine is running 24/7
<lifeless> mpt: perhaps the client could report TSLF in each report, or some approximation thereof.
<mpt> lifeless, I think it will have to, yes. Which means we run the risk that the system clock changed in the intervening period, but oh well.
<lifeless> mpt: that should be rare though
<lifeless> mpt: and rare events should be discarded as outliers
<mpt> I suppose system clock adjustments will be roughly normally distributed anyway :-)
<david0rk> hi all
<david0rk> anyone on?
<david0rk> nevermind.  I figured out using a bit of common sense and tinkering.   BAZAAR IS AWESOME!
<SpamapS> hrm, I want to make sure I understand. bug #915505 means we can't use bzr builder format 0.4, right?
<_mup_> Bug #915505: bzr: ERROR: exceptions.AttributeError: 'cStringIO.StringI' object has no attribute 'split' <launchpad-buildd:Triaged> <Open Sesame: libaccounts-glib:New> < https://launchpad.net/bugs/915505 >
 * SpamapS is sad, that would let him automate stable releases building into a PPA
<bac> hi lifeless
<jelmer> SpamapS: yes
<SpamapS> jelmer: and will LP have to be migrated to precise before that works?
<jelmer> SpamapS: no, it will just need to be updated to a newer python-debian or we can update bzr-builder to support the older python-debian.
<SpamapS> ah
<jelmer> SpamapS: the fix for bzr-builder would be trivial, it's mostly the management around it to get it deployed to the builders that's more complicated.
<SpamapS> Will probably be easier to just automate my packaging branch to have a new debupstream every time I commit to my stable branch
<SpamapS> but, thats a pity, as 'latest-tag' is really cool
<lifeless> bac: hi there
<aquarius> I have a program, not of my creation, which calls launchpadlib. Launchpadlib, if you've not been authed to use LP, opens up LP in a browser and then blocks, waiting for you to auth the token. Is there some envar or similar I can set so that launchpadlib will just fail that request instead because it's not being run interactively?
<cjwatson> aquarius: It uses the webbrowser module in Python's standard library, which honours BROWSER, so you could set that to ... err, something appropriate
<aquarius> aha! sneaky :)
<cjwatson> Not entirely sure how to make that fail immediately rather than spinning.
<StevenK> export BROWSER=/bin/false ?
<StevenK> That might be a little mean.
<aquarius> I shall see if it works :)
<cjwatson> I don't think that will work; webbrowser will notice the non-zero exit code and fall back to its next option, as I read it.
<cjwatson> But /bin/true might work.
<cjwatson> Hopefully you'll get EndUserDeclinedAuthorization.
<cjwatson> I don't see another way to control it in an entirely different process.  If you can modify the program then you can pass in a different authorization_engine.
<cjwatson> You might have to write a miniature browser that actually interacts with the URL it's given and presses the Decline button or whatever it's called.
<aquarius> doesn't seem to honour BROWSER, afaict :(
<wgrant> You won't get a button -- you'll be redirected to the login page.
<aquarius> ah, I'll try the /bin/true trick :)
<cjwatson> Really?  It's right there in webbrowser.py.
<cjwatson> Mm, maybe launchpadlib will just think you haven't got round to logging in and deciding yet. :-(
<aquarius> that's what happens
<aquarius> no browser opens (no problem there), but then lplib just sits about waiting for me to finish authing.
<aquarius> darn.
<aquarius> and I can't see how to get out of that code, either
<cjwatson> Are you trying to do something headlessly?
<aquarius> yep
<aquarius> specifically, run the test suite for quickly
<cjwatson> That sounds like a bug in the test suite, then.  Surely it shouldn't fail without a Launchpad login.
<cjwatson> (Or hang.)
<aquarius> it *is* a bug in the test suite
<aquarius> but the test suite's a terrifying mass of shell scripts which I do not have time to entirely rewrite from scratch :P
<aquarius> so I was hoping for a semi-quick fix to make the tests fail, rather than just hang indefinitely :)
<cjwatson> Quickest fix would be to give them a login someho. :-)
<cjwatson> *somehow
<aquarius> that's approach number 2 if I can't find a way to do this, which is fractionally vanishingly more proper a solution :)
<aquarius> and I don't think this is doable, annoyingly enough. Ah well.
<cjwatson> I thought bribing didrocks was the traditional answer to quickly problems, anyway
<aquarius> s/didrocks/mterry/ in this particular case, and he said "yeah the test suite is weird, feel free to just put in a separate test which is not tied to it" :-)
<aquarius> so I have plenty of get-out clauses
<aquarius> I just thought I'd try and do things properly (well, sorta) rather than super-hackily for once, and lo! I am thwarted. there is a lesson here :)
* StevenK changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: StevenK | Firefighting: - | Critical bugs: 3.47*10^2
#launchpad-dev 2012-05-02
<wallyworld> wgrant: i don't think there's a bug search api that can be used to find all bugs a person is directly subscribed to is there? you can search and find bugs for structural subscriptions on a pillar but I want to find bugs subscribed to directly
<wgrant> wallyworld: https://launchpad.net/~wgrant/+subscribedbugs
<wgrant> wallyworld: But you don't want to remove all subscriptions.
<wallyworld> wgrant: if a person has had their access to a pillar revoked, then we want to unsubscribe them from all bugs for that pillar, no?
<bac> lifeless, wgrant: do you know if storm makes any guarantees about the default ordering of results if no order_by is specified?
<wgrant> bac: It does not. That would be evil.
<wgrant> wallyworld: Their direct access to the pillar has been revoked.
<wgrant> wallyworld: But they may still have access through a team.
<bac> wgrant: yeah, well we have unit tests that rely on order by id...and they fail when run out of order
<wgrant> bac: You'll need to order explicitly. Do not add implicit order to the model, as it is likely to break queries.
<wgrant> Sorting is expensive.
<bac> you don't say.  :)
<wgrant> bac: Some old SQLObject-based classes have a default order defined, but it causes no end of performance trouble when you write what seems to be an innocent query but it actually sorts behind your back.
<wallyworld> wgrant: the argument about team access could be applied to the case where their access is revoked to a single artifact via the sharingdetails page and we unsubscribe there
<wgrant> wallyworld: For example, once the UI is turned on we'll want to remove about 130 of the Launchpad grants.
<bac> thanks wgrant
<wgrant> wallyworld: But we'll want to keep lots of the subscriptions.
<wgrant> wallyworld: Because engineers have subscribed to things that they're interested in. We're just revoking their explicit access because they should get it through ~launchpad instead.
<wallyworld> sure, so that's what i'm doing in this job - removing direct subscriptions for an individual when access is revoked
<wgrant> You need to remove direct *grants*, and subscriptions to artifacts that are no longer accessible.
<wgrant> But not all subscriptions to private artifacts.
<wallyworld> grants are removed already, but i was told i also need to remove subscriptions also
<wallyworld> it's in a bug on the board
<wgrant> You need to remove some of them, yes.
<wallyworld> so there's 3 cases 1. revoke access to individual artifact, 2. revoke access to entire pillar, 3. revoke access to artifcats of an info type
<wallyworld> so far in the job i've done case #1 - simply unsubscribe user from given artifact
<wgrant> In all cases, if there is a subscription and no remaining access (eg. through a team), the subscription must be removed.
<wgrant> However
<wgrant> This gets complicated, because eg. team membership changes affect which subscriptions are allowed.
<wallyworld> if the grant is revoked, they won't have access even if there is a team subscription
<wgrant> They will so.
<wgrant> Well
<wgrant> Not a team subscription.
<wgrant> But a team grant.
<wallyworld> right. but the sharing details page only shows direct access
<wgrant> Sure
<wgrant> But revoking the direct access does not necessarily revoke all access.
<wallyworld> and when the delete button is hit, the grant is revoked and the invisual sub to the aertifact is removed
<wgrant> That is incorrect behaviour.
<wgrant> There may still be access, through a policy grant or a grant to a team.
<wallyworld> but if there's a policy grant, you don't see that artifact on the sharing details page
<wallyworld> so you can't revoke it individually
<wgrant> Ah, through that UI that is true indeed (although it might be visible on the bug page)
<wgrant> But it's still true for team grants.
<wallyworld> extending that metaphore of case #1 above, if i revoke a user's access to userdata things, that user needs to be unsubscribed from those things
<wgrant> No.
<wgrant> Because you might be doing it to clean up.
<wgrant> Not to revoke access.
<wgrant> https://launchpad.net/launchpad/+sharing <- most of those grants are invalid, but the subscriptions are fine.
<wallyworld> wgrant: see bug 933839, the second bit of the description indicates we do need to remove subscriptions
<_mup_> Bug #933839: Share nothing with a user or team <disclosure> <job> <Launchpad itself:Triaged> < https://launchpad.net/bugs/933839 >
<wgrant> Bug descriptions from 3 months ago don't override logic :)
<wallyworld> but i discussed this work over the past 2 standups and nothing was said to the contrary
<wgrant> It doesn't explicitly state either way.
<wallyworld> yes it does, it says "Unsharing also removes subscriptions to bugs and branches"
<wgrant> One behaviour clearly produces undesirable results, the other potentially contradicts one reading of a vague statement in an old bug report.
<wallyworld> it wasn't just in this bug report, the same issue has come up in conversation over the past little while
<wgrant> I'd always heard it phrased more along the lines of "subscriptions to which there is no longer access must be removed"
<wgrant> Not "subscriptions to artifacts from which a grant is being revoked must be removed"
<wgrant> I don't think the latter version makes much sense.
<wgrant> As it is clearly inconsistent with intended behaviour elsewhere.
<wgrant> I can subscribe to a bug I have access to, even if it's through a team.
 * StevenK stabs BugChange
<wallyworld> sure. the intent of the ui then becomes an issue
<wgrant> The UI will make it clear that people may still have access through teams.
<wallyworld> if a user clicks the delete icon on the sharing page, they think they are revoking access for that user
<StevenK> I can clearly see two BugInformationTypeChange's being created, but printing a traceback when they're created is less than helpful.
<wgrant> wallyworld: Removing subscriptions doesn't make that true or false.
<wgrant> wallyworld: It's already false.
<wgrant> wallyworld: Because access may remain through teams.
<wallyworld> so does access policy grant flat take account of team membership, i can't recall
<wgrant> Nop.
<wgrant> No.
<wgrant> Remember the concerns around private team expansion etc. which led us to defer the audit view.
<wallyworld> but if the requirement is to unsubscribe users if direct access is revoked and they have no further access via teams, i could join apgf to teamparticiaption and check that way
<wgrant> That would be one way to do things, inded.
<wallyworld> so i think the current mp which unsubscribes when revoke is done from the sharing details page is correct
<StevenK> wgrant: Can you see any difference in http://pastebin.ubuntu.com/961462/ ? I'm printing the arguments BugInformationTypeChange is getting created with and then forcing a stack trace
<wgrant> wallyworld: mmm
<wgrant> wallyworld: It's not incorrect, because it doesn't specify how to calculate the bug IDs.
<wgrant> However, I question the design. eg. this probably makes it impossible to revoke apport's access to Ubuntu bugs.
<wgrant> As the request will have to grab like 300000 bugs out of the DBs, calculate their IDs, put them in a list, serialise them to JSON, stuff them into a multi-megabyte text column in the DB.
<wallyworld> wgrant: you misunderstand - the bug ids list is only for individual revokes, not info type or pillar based ones
<wgrant> wallyworld: Ah, that makes me a lot less worried.
<wallyworld> i'm not *that* stupid
<wgrant> Most people don't quite realise the magnitude of some of the horrific exceptional cases we have.
<wgrant> Like, no reasonable person could think that one principal would have 300000 bug grants.
<wgrant> But they do :)
<wallyworld> seriously? 300000
<wgrant> Maybe only 200000
<wgrant> But a few
<wgrant> That order of magnitude, anyway :)
 * wallyworld needs food
<wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/no-job-oops-prefix/+merge/104315
 * StevenK scratches his head over this double BugInformationTypeChange
 * wgrant looks
<StevenK> I have a diff between them
<wgrant> Those are tracebacks for the two calls?
<StevenK> Yeah
<StevenK> wgrant: http://pastebin.ubuntu.com/961476/ is the diff
<StevenK> The first one is -, the second is +
<wgrant> StevenK: lazr.restful calls notify() itself
<wgrant> Presumably it diffs the object too
<mwhudson> lazr.lifecycle woo
<wgrant> Using lazr.lifecycle, yeah
<StevenK> The one from lazr.restful is wrong, though :-(
<mwhudson> for some reason the fact that that isn't part of launchpad any more has never really settled into my brain
<StevenK> wgrant: Your MP looks excellent
<wgrant> StevenK: How's it wrong?
<StevenK> -BugInformationTypeChange(None, <security proxied lp.registry.model.person.Person instance at 0x2b5d6c031c50>, information_type, Unembargoed Security, Public)
<StevenK> +BugInformationTypeChange(None, <security proxied lp.registry.model.person.Person instance at 0x2b5d6c031c50>, information_type, Unembargoed Security, <security proxied lazr.enum._enum.DBItem instance at 0x54ab810>)
<wgrant> StevenK: Thanks
<wgrant> Well
<wgrant> You could make it not wrong.
<mwhudson> zope.security woo
<StevenK> wgrant: This explains why it's going wrong, though. Bug:+secrecy is calling transitionToInformationType directly which is firing off one notify. The JS is calling it via the API and lazr.restful is doing the notification and then transitionToInformationType is doing it.
<wgrant> That is clear, yes.
<StevenK> It's only clear to me as of 60 seconds ago.
<StevenK> So transistionToInformationType shouldn't notify if from_api is set, or is there a better pattern?
<wgrant> It should perhaps not notify at all. Check what corresponding things to, eg. status/assignee changes
<StevenK> transitionTo{Assignee,Target} at least don't notify
 * StevenK comments out the notify() and see what happens with Bug:+secrecy and the JS fire.
<StevenK> wgrant: Looks like Bug:+secrecy requires the notify() at least.
<StevenK> So I can fix Bug:+secrecy to make an API call, which I don't like, or change transitionToInformationType to not notify() when called from the API, which I don't like either.
<wgrant> Or change Bug:+secrecy to notify like other edit forms.
<wgrant> That is the correct solution.
<StevenK> Ah ha
<lifeless> bac: if they are ordered on id, how are they failing - are the objects having the wrong ids, or was the sort actually undefined ?
<StevenK> wallyworld, wgrant: https://code.launchpad.net/~stevenk/launchpad/double-bugdelta-js/+merge/104316
<StevenK> wallyworld: I think that MP might fix your double e-mail issues, too
<wallyworld> cool
<wallyworld> StevenK: should you do a check to ensure info type really has changed?
<wallyworld> like is done for private etc
<StevenK> wallyworld: I just didn't want to unilattery notify on both fields there.
<StevenK> For information_type I don't think it matters
<wgrant> StevenK: Did you try making it a LaunchpadEditForm?
<wgrant> LaunchpadEditFormView, sorry
<wgrant> That should fire the events automatically.
<StevenK> wgrant: Will that deal with private+security_related/information_type?
<wgrant> StevenK: It diffs the objects like lazr.restful does.
<wallyworld> StevenK: if you do use LEFV you will likely need to override updateContextFromData()
<StevenK> I didn't know about LaunchpadEditFormView
<StevenK> My concern about diffing the object is private, security_related and information_type all change.
<wgrant> StevenK: Doesn't the subscriber handle filtering that?
<StevenK> I'm not certain, I'd left most of the notification stuff alone due to HBD
<StevenK> I'd prefer to switch to LEFV when I drop the legacy behaviour.
<wallyworld> the ff complicates it so if it works as is i'm ok with that
<wgrant> get_bug_changes filters based on the FF
<wgrant> So notifying with the entire diff should be fine.
<wallyworld> StevenK: there were no failing tests, right? so do we need a test to check that notifications for info type are as expected?
<StevenK> wallyworld: No, the tests all exist
<wgrant> Clearly not.
<wgrant> buildbot would have failed.
<wallyworld> but there we no failures
<StevenK> I perhaps need a notification test when changing via the API
<wallyworld> and/or a test that changing info type via the ui produces the correct notifications
<StevenK> wallyworld: Given that involves JS, I'm not sure how to do that.
<StevenK> We know it fires the API call, if the API call only generates one notification we're good
<wallyworld> js is irrelevant here - you just need to write a normal view test
<wallyworld> initialise a view with form={} with the right things in the dict
<StevenK> wallyworld: Right, hold on
<StevenK> wallyworld: Bug:+secrecy has always done one notification, this branch doesn't change that.
<wallyworld> i got 2 the other day
<StevenK> wallyworld: The error this branch fixes is when you change the information_type via the JS on Bug:+index, since that uses the API.
<wgrant> The solution with the minimal amount of code, so minimal bugs, is to use LEFV
<wgrant> So use that
<wallyworld> not necessarily given the ff complication
<lifeless> LEFV?
<StevenK> LaunchpadEditFormView
<wallyworld> LaunchpadEditFormView
<lifeless> thanks
<wgrant> LaunchpadEditFormView
<StevenK> Peer pressure
<wallyworld> what he said
<wallyworld> StevenK: afaik, the js used to call the BugSecrecyEditView - there's even code in the submit action to handle it.perhaps it's changed now
<StevenK> wallyworld: I'll switch it to LEFV, there is already notification tests for Bug:+secrecy, I'll write a webservice test.
<StevenK> wallyworld: The legacy JS calls BugSecrecyEditView, yes.
<wallyworld> ah, ok. but the new js for the info type choice popup doesn't
<StevenK> Yes.
<wallyworld> i didn't really that, sorry
<wallyworld> realise
<StevenK> wallyworld: I thought I explained it well enough in the MP's description.
<wallyworld> i makes more sense if i re-read it now. i didn;t connect legacy and js
<StevenK> wgrant: Which is why I'm probably a little retiscent to change it to LEFV. The legacy JS calls the form.
<StevenK> The legacy *untested* JS
<lifeless> WCPGW
<StevenK> It's already taken me four hours nailing down this issue in the first place.
<StevenK> The next time I want to touch this code is when I remove the legacy bits.
<wallyworld> +1
<wallyworld> StevenK: i'm still i little curious why there were no failing tests, that's my remaining concern
<wallyworld> ideally you'd address that before landing
<StevenK> wallyworld: Right, so I'm writing a webservice test now
<wallyworld> ok, r=me otherwise
<StevenK> Which will probably get interrupted by lunch, but meh.
<StevenK> Oh, blah. the bugchanges tests assume that transitionToInformationType is the one generating the notifications.
<StevenK> wallyworld: http://pastebin.ubuntu.com/961639/
<StevenK> wallyworld: That brings the information_type and visibility tests into line with things like IBugTask.status which make the change and then notify
<wallyworld> looking good so far, need to finish reading the diff
<StevenK> wallyworld: I've confirmed that if I revert the changes in the MP you reviewed that the API test fails due to getting two notifications.
<wallyworld> that's helpful
<wallyworld> StevenK: the change_activity 'person': private_bug.owner (was self.user); what's the reason for that?
<StevenK> wallyworld: I'm notify()'ing with user=private_bug.owner, I can change that to self.user and revert those bits if you wish.
<wallyworld> StevenK: i'd just do it the same as the other similar tests for consistency
<wallyworld> whatever way that is
<StevenK> wallyworld: Fixed
<wallyworld> thanks. send it to ec2
<StevenK> Gladly.
<mwhudson> did you know
<wallyworld> good to see this one fixed
<mwhudson> that editing shell scripts while they are running seems to be a bad idea
<wallyworld> who would have thought :-)
<mwhudson> well it works for python scripts :)
<StevenK> wallyworld: Let me XXX BugEditSecrecyView to note about LEFV too
<wallyworld> sounds like a plan
<wallyworld> can't wait to get rid of more legacy
<StevenK> Tell me about it
<StevenK> Launchpad encountered an internal error during the following operation: generating an incremental diff for a merge proposal.  It was logged with id OOPS-b36d837108d341fbaba9b6ef591ac4f0.
 * StevenK stabs the job system over and over and over.
<lifeless> wgrant: did you try using an aggregate + comparise rather than AND's ?
<lifeless> or an aggregate with set ops ?
<wgrant> lifeless: You mean the abomination that's used for structural subscription sorting and is slow as hell?
<wgrant> NULL_COUNT and such?
<lifeless> not sure
<wgrant> well
<wgrant> "comparise" isn't a word
<wgrant> So I was trying to work out what you meant
<wgrant> Adding more aggregates is unlikely to help performance.
<lifeless> bah, typo
<lifeless> aggregate and compare
<wgrant> In this case the estimates are important for correct performance.
<lifeless> and aggregates are throwing it off ?
<wgrant> What sort of structure were you thinking of?
<wgrant> I've found that aggregate conditions tend to be hard to plan through.
<lifeless> mapping the tags into an array
<lifeless> what performance do you get with the AND's ?
<wgrant> Right, that's what's done for structural subscription filtering.
<wgrant> That's always entirely thwarted estimation when I've tried it in the past, but I'll recheck just in case.
<wgrant> lifeless: I'm not sure it's feasible, given the complex constraints that are possible.
<wgrant> We could do it nicely with the complex query stuff in intarray
<lifeless> what performance do you get with the AND's ?
<wgrant> https://pastebin.canonical.com/65322/
<wgrant> The FTI is pretty slow, but hopefully GIN will fix that.
<lifeless> 540 for a moderate search is fine
<wgrant> FSVO
<wgrant> We've got that + a similar count right now
<wgrant> So it's more like 1.3s before we get any other time.
<wgrant> It actually ends up planning really well like this.
<lifeless> was thinking about counts
<lifeless> we could make that async
<lifeless> issue two requests
<wgrant> Because it has accurate stats for fti and tags
<wgrant> Yeah, I've considered that fairly strongly.
<wgrant> Also remembering counts.
<wgrant> And guessing from bugsummary sometimes
<wgrant> Most counts can be satisfied straight from bugsummary
<wgrant> And those that can't are mostly complex things with FTI that nobody will notice if they're cached for a while.
<lifeless> mmmm
<lifeless> caches tend to get you in trouble
<wgrant> Although the estimates are much better with bugtaskflat, so rough_length could possibly be a bit more workable.
<wgrant> But I still don't like it much for this sort of thing.
<lifeless> simple ones people complain about, complex ones are complex
<wgrant> lifeless: Also, I have a branch landing in 30 minutes that rips out the oops_prefix per-section customisability, and was thinking of pushing forward the one-config-per-user thing next week so I can eliminate the rest. Any objections? Should basically just mean adding a few new configs, getting them deployed, throwing a crontab diff at LOSAs and getting a few init scripts fixed up.
<wgrant> Then we can delete like 1000 lines of prod config :)
<wgrant> And a similar amount of config schema.
<wgrant> (we currently have lots of overrides in eg. the production config because different scripts run as different users, so need different error_dirs)
<StevenK> wallyworld: Do you want to review https://code.launchpad.net/~stevenk/launchpad/bugs-factory-information_type/+merge/104326 ?
<lifeless> wgrant: sounds lovely
<wallyworld> ok
<wallyworld> StevenK: r=me
<StevenK> wallyworld: Sorry for the massiveness
<wallyworld> that's what i say to the girls
<StevenK> wallyworld: http://picardfacepalm.com/
<wallyworld> well, you started it :-)
<adeuring> good morning
<wgrant> wallyworld: You didn't run your new table through ec2?
<wgrant> Looks like it's missing person merge permissions.
<lifeless> rollback time
<wgrant> Dinner time, actually.
* StevenK changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 3.47*10^2
<jml> heavens
<jml> I need to make my branch listing page more readable.
<jml> rick_h_: did you ever get any test results from that ec2 test run?
<rick_h_> jml: that landed
<rick_h_> and was merged I thought
<jml> rick_h_: there was another one
<rick_h_> oh, I didn't complete the test run, I thuoght we were just debugging why it wouldn't run
<jml> rick_h_: it's not a huge deal. I'll probably be able to run it myself today.
<jml> rick_h_: ah, right :)
<rick_h_> my apologies, I thuoght once you had it figured out why it was ok and set
<jml> rick_h_: np.
* abentley changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: abentley | Firefighting: - | Critical bugs: 3.47*10^2
<deryck> abentley, you meant that you and adeuring would stay on the call, right?  not me?
<abentley> deryck: yes.
<deryck> abentley, great, thanks
<jml> https://code.launchpad.net/~jml/lp-dev-utils/public-ip/+merge/104273 needs review
<rick_h_> hah, jml finding someone else today :P sorry abentley
<jml> :)
<jml> got this error on 'ec2 demo': http://pastebin.ubuntu.com/962408/
<jml> and then this error after terminating with Ctrl+D: http://pastebin.ubuntu.com/962410/
<abentley> jml: why is it a guess?
<jml> abentley: because it might be wrong.
<abentley> jml: why might it be wrong?
<cjwatson> In general, if your HTTP routing is different from your SSH routing.
<jml> abentley: I'm being pessimistic. checkip.amazonaws.com was wrong sometimes, I don't see what makes akamai guaranteed to not be wrong. All I know for sure is that it's less wrong than checkip.amazonaws.
<abentley> cjwatson: fair point.  I suppose we could determine our IP via SSH?
<cjwatson> For example, if you have a public IP address, but you connect through an HTTP proxy on another public IP address (say, for performance, or because it's mandated in some way by your network layer), then akamai would return the address of the proxy; but you might be connecting directly to the cloud instance over SSH.
<cjwatson> Amazon had the opposite problem; in proxy cases, it would always return the address of the client system as seen by the proxy, private or not.
<jml> hmm.
<cjwatson> I would argue that it's more common for Amazon's case to break, in the modern internet; but the other kind of breakage is *possible*.
<cjwatson> So I support jml having added an option for this.
<jml> then IIUC the correct thing is to always use the akamai address for 80 & 443 ports and only apply the provided IP to the SSH port?
<cjwatson> I don't know of any way to reliably determine our IP address via SSH, in the absence of some new service.
<cjwatson> jml: The behaviour in your MP is the best I can think of.
<jml> cjwatson: thanks.
<abentley> cjwatson: And yet, the who command knows your IP/domain, and when you log in, it will tell you your last ip/domain.
<abentley> jml: Anyhow, fair point that this is only a guess.
<abentley> jml: although if you SSH and HTTP routings are different, then security_group.authorize('tcp', 22, 22, '%s/32' % public_ip); security_group.authorize('tcp', 80, 80, '%s/32' % public_ip) is also wrong.
<jml> abentley: I think that's what cjwatson meant by "some new service". You'd need to have an SSH service up that could be got into by everyone who needs to run 'ec2 test'.
<jml> abentley: well, that's sort of what I thought, but cjwatson responded otherwise when I raised the question with him.
<cjwatson> abentley: But you only get to ask it that after you've told Amazon to authorise you to connect from a given IP address.
<cjwatson> abentley: Chicken and egg.
<abentley> cjwatson: I see.
<cjwatson> jml: Oh, sorry, I misunderstood that part.  Um, maybe.
<abentley> jml: I'm not sure whether "transparent proxy" or "intercepting proxy" is the right terminology in this case.  Do you know?
<jml> abentley: no, I don't.
<abentley> jml: Okay, let's leave it as is.  Rob can tell us different if he wants.
<abentley> jml: r=me
<jml> abentley: thanks.
<cjwatson> abentley: Proper proxies have a problem too, anyway.
<nik90> hi everyone, I would like to know how to read a user's translation activity using launchpadlib
<nik90> this is for ubuntu accomplishments project started by jono
<nik90> any help would be appreciated
<jml> nik90: hmm.
<jml> nik90: you know about the API, right?
<nik90> jml: yes I did go through the API..but couldn't find a suitable one for translation...i am new to this..
<sinzui> nik90, API does not provide that useful information. Some of this information is implied in karma activity which idoes not have the detail and not over the API
<nik90> sinzui: oh so using the API, I cannot access the translation activity of a user but must rely on the user's karma?
<sinzui> No, because karma is bogus.We only export a number which is itself ogus
<sinzui> nik90, https://launchpad.net/+apidoc/devel.html does not provide any means to answer you questions
<nik90> sinzui, that was the API list that I went through
<nik90> sinzui, the closest I came to was the translation_import_queue_entries...but that could not check the translations still
<sinzui> nik90, right, but entries in the queue is just uploading
<nik90> sinzui, would I be able to use translation_template by checking with the user's name
<nik90> would that check the translations?
<sinzui> nik90, https://launchpad.net/~sinzui/+karma shows a bogus score for types of activity, and a list of recent activity
<nik90> sinzui: hmm, yes it does..I guess that should do it temporarily
<nik90> sinzui, the thing was, I wanted to confirm that the translation made by the user were approved and useful before giving the accomplishment
<sinzui> nik90, I argue that we care about seeing the frequency of activity and batched listing that allows me to see contributions over a year, month, and week
<nik90> but through this I wont be able to...but atleast I can check if they made a translation in the first place
<nik90> sinzui: thanks for your help
<sinzui> nik90, ah, that is a challenge. I really don't think that can be answered at all
<nik90> sinzui: going through the api to distunguish between different karma
<sinzui> nik90, we only provide a number over the API. karma is broken so we choose not to encourage people to use it over the API
<nik90> sinzui: :( alrite that's ok
<lifeless> morning
<lifeless> deryck: / abentley: Are you guys planning on blogging (blog.l.n.) about the bundles-to-mp code being removed?
<abentley> lifeless: no.
<lifeless> abentley: is there more to it than that ?
<deryck> lifeless, hey.  I didn't think we needed much public notice for it....
<abentley> lifeless: It's unlikely that anyone will notice, since no one's used the feature at all this year.
<deryck> lifeless, it hasn't been used since June of last year.
<deryck> calling attention to it seems like asking people who don't use it to complain about it being removed. ;)
<lifeless> I can understand that angle
<lifeless> OTOH we're been very forthright about other removals and redefinitions with a generally positive response.
<lifeless> I think it is better to be clear than to have folk surprised later (even though that is unlikely).
<lifeless> We're doing a good thing here
<lifeless> focusing the system on what users need
<lifeless> making it possible for us to develop and iterate faster
<lifeless> *I* think you should be proud of the removal
<lifeless> (btw, I presume you have it on your todo list, but https://help.launchpad.net/Code/Review still references the feature)
<abentley> lifeless: Yes, I didn't want to remove it until the change had landed.
<lifeless> cool cool
<rick_h_> 816831
#launchpad-dev 2012-05-03
<lifeless> -> monthly allergy shot. Urgent things you can SMS/ring.
<mwhudson> wtf
<mwhudson> how can bug 951365 be a regression
<_mup_> Bug #951365: Accessing milestone on project group via webservice api returns has_bugs object <api> <milestones> <oem-services> <projectgroups> <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/951365 >
<mwhudson> wgrant: so i think that happens because a project groups milestones are IProjectGroupMilestones, not IMilestones
<wgrant> mwhudson: The only thing I can think of is milestone tags.
<wgrant> That may have changed things.
<mwhudson> ah
<mwhudson> i suppose that might have resulted in some shuffling
<mwhudson> milestone tags was only a 3000 line change...
<wgrant> Pfft
<mwhudson> wgrant: good guess
<wgrant> What happened?
<mwhudson> wgrant: well, that was the change that made IProjectGroupMilestone not inherit from IMilestone
<wgrant> Ah
<wgrant> That would do it indeed.
<mwhudson> i _guess_ the milestone collections in iprojectgroup need to be overridden to declare returning IProjectGroupMilestone
<wgrant> ARGH
<wgrant> I hate lazr.config
<wgrant> Particularly some of the ways we use it.
<wgrant> config.error_reports.error_dir can be overridden by error_dir in a script-specified config section.
<wgrant> Except that the config schema has some dev paths as the codehosting sections' error_dirs.
<wgrant> Which means that dropping the codehosting error_dirs from the prod configs won't fall back to the configured production default, but the dev default.
<mwhudson> hm, i wonder if i've run rocketfuel-setup on this machine
<StevenK> mwhudson: If /etc/hosts is 600,000 lines long, yep.
<mwhudson> appears i have not
<wgrant> LXC! LXC!
<mwhudson> ah good point
<wgrant> It's nice and easy on precise
<wgrant> I cleaned up the howto on whatever day was earlier this week
<StevenK> Speaking of precise, I'm trying to figure out how to upgrade my fileserver.
<StevenK> It's Ethernet module isn't in Lucid's kernel, so it's dkms'd, but I don't want it to be dkms'd on upgrade, since the module is in Precise's kernel.
<wgrant> '''
<mwhudson> wgrant: this howto? https://dev.launchpad.net/Running/LXC
<wgrant> That one
<mwhudson> cool
<lifeless> wgrant: fix the schema ? I don't see any reason for it to have dev data in it, we have a dev config for that
<wgrant> lifeless: Yeah, fixed in devel a few minutes back.
<wgrant> It's still full of crap, but it's slightly less offensive and stupid crap.
<mwhudson> wgrant:   launchpad-dependencies: Depends: python-apt (>= 0.7.94.2ubuntu6.4) but 0.7.94.2ubuntu6 is to be installed
<mwhudson> (in the lxc)
<wgrant> mwhudson: Sure you ran rocketfuel-setup? That should be in the launchpad PPA
<StevenK> mwhudson: No -updates?
<wgrant> Or -updates
<mwhudson> StevenK wins
<wgrant> Hmm
<wgrant> I'm pretty sure my fresh container had that by default
<wgrant> mwhudson: Has this machine run LXC before?
<lifeless> bad precise lxc cache probably
<mwhudson> wgrant: yes, i lxc-destroy-ed the lxc i had
<mwhudson> sources.list was 1 line
<lifeless> mwhudson: there is a cache as well
<mwhudson> oh excellent
<wgrant> Ah
<wgrant> I'd suggest restarting
<mwhudson> oh ok
<wgrant> sudo rm -r /var/cache/lxc/lucid
<wgrant> Then lxc-create again
<wgrant> Vast improvements were made in the last couple of months
<wgrant> Such that it's no longer torture to get LXC working.
<wgrant> It'll redebootstrap an environment that actually works
<mwhudson> heh
<mwhudson> ok
<mwhudson> maybe a note along these lines could be added to the wiki
 * wgrant adds
<cody-somerville> mwhudson, Are you fixing LP #951365? :)
<_mup_> Bug #951365: Accessing milestone on project group via webservice api returns has_bugs object <api> <milestones> <oem-services> <projectgroups> <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/951365 >
<mwhudson> cody-somerville: its a possibility
<mwhudson> come on intertubes, go faster
<wgrant> mwhudson: Going OK?
<mwhudson> wgrant: yes
<wgrant> Great.
<wgrant> I haven't actually completely tested the latest version of the instructions from scratch :)
<mwhudson> heh
<mwhudson> well rocketfuel-setup --no-workspace is grinding away
<wgrant> Great.
<StevenK> wgrant, wallyworld: Either of you want to look at https://code.launchpad.net/~stevenk/launchpad/bugs-information_type-mail/+merge/104483 ?
<wgrant> StevenK: Underscores?
<wgrant> Really?
<wgrant> I don't think we have underscores anywhere else in the UI
<wgrant> Mail or Web.
<StevenK> wgrant: 'information type' doesn't work for the command, 'information_type' does. :-(
<wgrant> informationtype does as well :)
<wgrant> Does this follow value parsing rules similar to status?
<StevenK> wgrant: informationtype works, I guess
<mwhudson> information-type
 * mwhudson bikesheds
<StevenK> wgrant: Status just calls transitionToStatus(), no checking
<StevenK> I didn't think CreateBugParams supported that method, but ICBW.
<wgrant> StevenK: But how does it parse the value?"
<StevenK> wgrant: It's done via setAttributeValue, so the parsing is in the guts of the mail handler, I guess.
<wgrant> StevenK: The behaviour of status and informationtype parsing has no reason to differ.
<StevenK> It sets a dbschema = BugTaskStatus
<StevenK> wgrant: Except for 'Private' and the forbidding of 'Proprietary'.
<wgrant> True.
<wgrant> The the latter is unrelated.
<wgrant> It's a temporarily post-parse filter operation
<wgrant> Bah
<wgrant> It is too cold to type sensibly
<StevenK> Oh?
<StevenK> It's 24degC in my study
<StevenK> wgrant: I think after we drop Private, we can change informationtype to be like Status
<StevenK> Or we could just declare we won't support Private on the mail interface, and I can change it now.
 * StevenK laughs
 * mwhudson wonders if rocketfuel-setup has asked for his root password for the last time...
<stub> Bug #992184  seems rather special
<_mup_> Bug #992184: lib/lp/services/database/doc/textsearching.txt fails intermittently/rarely on parallel tests <paralleltest> <Launchpad itself:Triaged> < https://launchpad.net/bugs/992184 >
<mwhudson> yes!
 * mwhudson disappears
<wallyworld> StevenK: i was doing school pickup, i assume you don't need me to look at your branch?
<StevenK> wallyworld: I was having a discussion with wgrant until his Internet broke due to no power.
<wallyworld> lol
 * wallyworld remembers a time when there wasn't any internet even with power
<spm> wallyworld: and where 9600 baud seemed like an impossible speed dream?
<wallyworld> my first modem was 300/75
<wallyworld> so yes :-)
<spm>  Igot lucky. started on 1200
<spm> LUXURY
<wallyworld> took ages to download all that ascii 'porn'
<StevenK> wallyworld: Download? Feel sorry for the poor bugger that had to type it in.
<wallyworld> rotflmao
<wallyworld> i still laugh every time an 80's movie shows a modem instantly connecting without the audible handshake
<wallyworld> i used to love the handshake
<StevenK> I had a friend at uni would could whistle 2400 baud
<StevenK> s/would/who/
<wallyworld> so he could connect to the CIA and order a nuclear strike?
<StevenK> Haha
<lifeless> spm: bah, 300/75? thats speedy
<spm> lifeless: itym wallyworld
<lifeless> ohbahdee
<wallyworld> what was slower than 300/74?
<lifeless> yesIdoo
<wallyworld> 75
<lifeless> wallyworld: a) I was trolling, b) 110 baud
<wallyworld> i was genuinely curious :-)
<wallyworld> 110 baud, wow that's slow
<lifeless> http://en.wikipedia.org/wiki/Bell_101
<wallyworld> but since baud refers to state changes per second, if each state change encoded 16 bits it wouldn't be soooo bad
<lifeless> My first modem was a 300/75
 * wallyworld is now reminiscing about the Hayes command set 
<wallyworld> those were the days
<lifeless> http://en.wikipedia.org/wiki/List_of_device_bandwidths is pretty good for reminiscing
<wallyworld> they left off the 2 cans either end of a taught piece of string
<spm> I'm rather pleased to say that I've compeltely forgotten all modem commands beyond AT. and I'm proud that I'm unsure if even that's correct.
<StevenK> I can remember ATDT<number>
<lifeless> +++
 * wgrant stabs u-boot
<wgrant> Or flash-kernel
<wgrant> One of those
<StevenK> Haha
<wgrant> I started with a 56kbps modem, you old people.
<bigjools> luxury
<lifeless> wgrant: modern luxury
<adeuring> good morning
<wgrant> There we go
<wgrant> Running/LXC tested and minimised and prettified
<wgrant> https://dev.launchpad.net/Running/RemoteAccess is a bit sad.
<wgrant> With SNI widely supported it seems like we don't really need the two IP addresses any more.
<maxb> widely supported enough, really?
<lifeless> for dev use
<lifeless> yes
<wgrant> The only thing in the config that needs it is private loggerhead, which will only be accessed by a web browser, and everything except those using Windows XP's crypto stack supports SNI.
<lifeless> long as we don't use python to access anything
<lifeless> wgrant: ^
<wgrant> Yes
<wgrant> As long as we have the main vhost first, python will be happy
<lifeless> oh, is that the degraded behaviour ?
<wgrant> I'm pretty sure it degrades the same as missing Host with HTTP
<wgrant> It picks the first matching vhost
<wgrant> I'm thinking I'll add a make target which tweaks the config to listen on * and allow a user-specified subnet.
<jml> how can I set up an 'ec2 demo' server so that it's possible to log on?
<wgrant> jml: What breaks when you try?
<cjwatson> SNI - only ten years late
<jml> wgrant: I get told that the openid provider is unavailable
<jml> OpenID Provider Is Unavailable at This Time
<jml> The openid provider was unavailable. Please try again in a moment.
<jml> You can also join the #launchpad IRC support channel on irc.freenode.net to ask for further assistance.
<wgrant> jml: That's slightly unpleasant of it.
<jml> wgrant: yes. the capitalization is egregious.
<wgrant> Both the fact that there's an error and the text content of the error
<wgrant> Yes
<wgrant> At least it doesn't say FreeNode
<wgrant> Let's see.
<wgrant> Slow ec2...
<wallyworld> wgrant: can you run a query on dogfood for me?
<wgrant> wallyworld: Sure
<wallyworld> https://pastebin.canonical.com/65435/
<wallyworld> it's for a job
<wallyworld> so not 100% critical is blazingly fast
<wallyworld> replace the policy ids
<wgrant> Ubuntu?
<wallyworld> sure, why not :-)
<wallyworld> it's to select all bugs a person is subscribed to but doesn't have a grant for (including via a team)
<wgrant> I expect it will be very slow, but let's see.
<wallyworld> i may need to use a CTE
<wallyworld> i have a bunch of tests passing, just need to optimise :-)
<jml> wow
<jml> it's so nice to have so many Australians around
<wallyworld> hi jml :-)
<allenap> lifeless: Do you have time to talk in about 10 or 11 hours, about MAAS architecture?
<jml> wallyworld: hi
<bigjools> hey jml!  Ah bugger, I'm not really Australian
<wallyworld> jml: how are they hanin', mate?
<jml> bigjools: bloody immigrants.
<wallyworld> bigjools: fuck off
<jml> wallyworld: all good, thanks
<bigjools> wallyworld: I'd be quiet if I were you or I'd tell everyone you were listening to Britney today.  Oops, too late :)
<jml> ow ow, it hurts to laugh
<bigjools> jml: bastards eh :)
<wallyworld> bigjools: lies, all lies
<bigjools> and also you had Lady Gaga on
<bigjools> I bet you wear a dress when nobody's looking
<wallyworld> no, just panties
<bigjools> with the peep hole
<wallyworld> that way i can wear them when people are looking
<wgrant> wallyworld: I'd spell it http://paste.ubuntu.com/964307/
<wgrant> wallyworld: Which is fast
 * wallyworld looks
<wgrant> It satisfies my subscriptions to Ubuntu in 30ms
<wgrant> ubuntu-crashes-universe in 1010ms hot
<wgrant> Which is not bad.
<wallyworld> wgrant: i didn't realise btf has access_grants
<wgrant> That doesn't have the restriction to Ubuntu because i am forgetful
<wgrant> But it's easy enough to add into the query
<wgrant> wallyworld: Yeah, it has all the access info so searches can be fast.
<wgrant> And this is a search :)
<wallyworld> wgrant:  does && mean AND? i've not seen thhat before
<wgrant> wallyworld: && is array intersection
<wallyworld> ok. is that postgres specific? or standard sql?
<wgrant> The array stuff is a postgres extension
<wgrant> AIUI
<wallyworld> i bet storm doesn;t support it, so i will have to hand code the query :-(
<wgrant> It's 3 lines to define &&
<bigjools> suck it up
<wgrant> And I think we already have one in the codebase
<wgrant> Bug subscriptions use it
<wallyworld> pointer?
<wgrant> lib/lp/bugs/model/structuralsubscription.py:    oper = "&&"
<wallyworld> bigjools: i've said it before and i'll say it again: fuck off
<wgrant> I'd move that into lp.services.database.stormexpr with the rest
<wgrant> wallyworld: However
<bigjools> wallyworld: muahaha :)
<wgrant> wallyworld: Those access checks are the same as the ones in bugtasksearch
<wgrant> wallyworld: You can probably reuse get_bug_privacy_filter
<wgrant> See how the access checks are done in there
<wgrant> It's not pretty yet, but eventually.
<bigjools> night all
<wgrant> Fuck off bigjools :)
<bigjools> wgrant: wallyworld gets away with it but not you
<wgrant> Aw
<bigjools> get back to your basement
<wallyworld> wgrant: i'll have a look. i may need to refactor what i've done. i've defined an indirect_bugs_filter helper method in accesspolicy.py
<wgrant> wallyworld: get_bug_privacy_filter also handles public bugs, which I omitted from that query
<wgrant> Mostly because I forgot
<wallyworld> ok. thanks. will have a look
<jml> :)
<wgrant> jml: Ah
<wgrant> jml: You still have that ec2 instance?
<jml> wgrant: yes.
<wgrant> jml: I suspect it will unbreak if you s/$INSTANCE_IP/*/ in /etc/apache2/sites-available/local-launchpad
<jml> hmm.
<wgrant> jml: ec2 demo forces it to listen on the external interface, when testopenid.dev will point to localhost
<wgrant> So the local appserver can't talk to the provider.
<jml> wgrant: ok, will try that.
<wgrant> launchpad-developer-dependencies only suggests libapache2-mod-wsgi. I guess it should depend now.
<wgrant> Ah
<wgrant> No
<wgrant> It also only suggests apache2, so I guess ec2 installs that manually.
<jml> yeah, haven't looked sorry.
<jml> wgrant: ok. Now I get "unknown email address" when I try to log in / register.
<wgrant> jml: admin@canonicalcom?
<wgrant> admin@canonical.com
<jml> wgrant: that gives me the following: http://pastebin.ubuntu.com/964360/
<wgrant> jml: Try clicking Log In / Register again
<wgrant> I suspect your session might have died.
<jml> wgrant: ooh, I only just noticed that when I got that error, the log in / register link was gone and replaced with Foo Bar (~name16)
<wgrant> Aha
<jml> which is weird but workaroundable.
<wgrant> Retrying with my fixes now.
<jml> wgrant: thanks.
<wgrant> Fourth time lucky...
<jml> wgrant: are your fixes to lp-dev-utils or to launchpad?
<jml> wgrant: I'm going to leave now â I have a lunch appointment. My IRC proxy will be connected.
<wgrant> jml: lp-dev-utils
<jml> wgrant: ah, thanks. I was going to do a patch for the '*' thing but will await yours.
 * jml really off
<wgrant> Ah
<wgrant> Neds an image rebuild to pick up the new package
<wgrant> That's why it's not working.
 * wgrant will commit and publish a new image.
<wgrant> Thanks for letting us know.
<StevenK> Hmm, then poolie can drop 523. Which might be difficult to organise.
<wgrant> jml: lp-dev-utils is fixed, and there's an updated image
<wgrant> so demo should work properly now
<wgrant> Also, buildout is slow
<StevenK> News at 11
<jml> wgrant: thanks.
* jcsackett_ changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: jcsackett | Firefighting: - | Critical bugs: 3.47*10^2
<jml> can someone please land lp:~jml/lp-dev-utils/public-ip
<jml> abentley approved it
<abentley> jml: done.
<jml> abentley: thanks!
<jml> running a dev instance via 'ec2 demo', looks like the private xmlrpc server isn't running
<jml> nothing is listening on 8097
<jml> how do I start that up?
<jml> hi
<achuni> g'morning :)
<rick_h_> sorry jml, not sure there. Looking at the makefile not sure which bits start up the xmlrpc
<jml> fwiw, we're getting https://pastebin.canonical.com/65462/ when we try to do an xmlrpc request
<jml> achuni: is that right?
<jml> and are being redirected from http to https.
<achuni> jml: that's just hitting the xmlrpc endpoint in a browser, I can try an xmlrpc request in a bit
<jml> achuni: ah ok.
<achuni> http->https redirect, yep.  afaict we should be using https anyway
<jml> *ahem*
<jml> 8087
<jml> never mind
<jml> it is running, and I'm a fool.
<jml> achuni: I get http://pastebin.ubuntu.com/964783/
<achuni> jml: looks good
<jml> achuni: good? is that what you expect?
<achuni> (does launchpadlib support proxies?  Can I ask it to use one?)
<achuni> jml: well, up until the 403 response, yep
<jml> achuni: if it didn't hurt so much, I would laugh bitterly
<achuni> :D
<jml> nothing on the help wiki
<achuni> jml: you think that's authorizing by IP just? we don't do any kind of authentication for that call afaict
<jml> it just uses httplib2 though
<jml> so it's probably whatever that uses
<achuni> right
<jml> googling doesn't reveal much
<jml> :(
<achuni> httplib2 supports proxies via the socks module
<jml> as in, by writing code
<achuni> yeah, checking if I can ask launchpadlib to pass in the right args
<jml> nothing in the stack seems to have environment-based http proxy support.
 * achuni hacks /usr/lib/python2.7/dist-packages/httplib2/__init__.py
<jml> :(
<jml> achuni: just double checking, you're not waiting on me for any questions, right?
<achuni> jml: nope, thanks
<achuni> jml: kind of success.  It seems to have worked, I now get Permission denied: '/var/tmp/launchpad_mailqueue/tmp/1336057390.27818.domU-12-31-39-08-04-1D.1557865906
<jml> achuni: hmm. that might be an fs thing.
<jml> odd.
<jml> achuni: I've made those writeable. Try again?
<achuni> weird, let me pastebin
<achuni> jml: https://pastebin.canonical.com/65469/
<achuni> jml: still a Perm denied failure, but now on an unspecified location
<jml> effing Python errors
<jml> how many hours have been lost because the 'os' module doesn't raise errors with function arguments
 * jml takes a deep, slow breath
<jml> achuni: oh, my bad.
<jml> achuni: there were more directories that needed permission changing
<jml> (my criticism of Python is still valid though)
<jml> achuni: try again?
<achuni> jml: I haz creds :)
<jml> achuni: \o/
<jml> achuni: what's next?
<achuni> jml: standup, but I'll try those creds out next, in a bit
<jml> achuni: heh, thanks.
<benji> bac: all clear
<adam_> hi there, was wondering if anyone could help me figure out why a few mails from the openstack list are occasionally silently vanishing into a bitbucket instead of reaching me?
<adam_> I receive most of them
<adam_> and have checked my spam folder
<czajkowski> adam_: are they showing up on the archive?
<adam_> czajkowski: yes
<adam_> e.g. https://lists.launchpad.net/openstack/msg11012.html
<czajkowski> adam_: any issue with your email as nobody else has said there are any LP mail issues
<adam_> czajkowski: not aware of any other issues or dropped mail from other sources, no
<angeloc> hi guys, I'm writing an ubuntu accomplishment that should test if a person had personally verified an SRU bug to earn the tester trophy. I'm using python api for launchpad, any idea how it could be done?
<adam_> czajkowski: don't suppose you'd be able to pinpoint that mail in the MTA logs and see if it was accepted by the MTA my end?
<czajkowski> adam_: I can ask and see
<adam_> if it was then I'll know I have to fight battles with our corporate email team ...
<adam_> thanks!
<czajkowski> adam_: and your email address it would have gone to is ?
<adam_> aspiers@suse.com
<czajkowski> adam_: checking
<adam_> awesome, thanks :)
<jcsackett> sinzui: thanks for helping out with the review queue. :-)
<adam_> czajkowski: gotta go now, but I'll stay logged in to catch anything you might say ... thanks again!
<czajkowski> adam_: I'll be gone shortly and no update yet
<adam_> czajkowski: ok, I'll be around tomorrow too
<czajkowski> ok
<adam_> of course I'm reachable on that email address too ... well, hopefully ;-)
<sinzui> jcsackett, I forgot it was Thursday
<jcsackett> sinzui: it's all good. that was a sincere thank you. that branch was the largest in my queue. :-P
<adam_> czajkowski: UK timezone, that is
<sinzui> jcsackett, :)
<adam_> czajkowski: hopefully we'll overlap by at least one hour but you never know on this channel ;-)
<czajkowski> adam_: look like it did make it to that end
<lifeless_> allenap: I'd be delighted to
<lifeless> allenap: I'm going back to bed for a bit (0600 here) but will ping when I'm on deck
 * deryck reboots, will brb
<jml> does anyone know why launchpad_mailqueue is unwriteable on ec2test?
<jml> ec2test/testrunner.py:426
<benji> jcsackett: if you have time for a small review, I have a bite sized one for you: https://code.launchpad.net/~benji/launchpad/bug-992692/+merge/104599
<jml> how do I create new users on a demo instance?
<lifeless> jml: there is a script
<lifeless> though we should really get demo talking to real openid
<jml> oh wait, 'make-lp-user'?
<lifeless> I believe so
<jml> I think I wrote that.
<jml> Anyway.
<jml> achuni: you've got shell access, right?
<achuni> jml: yep, logging in now
<achuni> jml: where's the make-lp-user script?
<jml> /var/launchpad/test/utilities/make-lp-user
<achuni> neat
<achuni> jml: https://pastebin.canonical.com/65498/
<achuni> jml: kind of close?
<jml> achuni: annoyingly, setting the email address does some gpg stuff
<achuni> jml: I'd skip that, except it's the only way the user can log in, I think?
<jml> achuni: probably best to run again without --email and then use the web UI to set  that.
<jml> achuni: no, elachuni@example.com
<achuni> ahhhh
 * achuni tries
<achuni> jml: "A confirmation message has been sent..." d'you think that'll arrive eventually, or should I retrieve it locally from the instance somewhere?
<achuni> jml: this was when I added an email via the web UI
<jml> achuni: tbh, I don't really know. Let's look in the mailqueue shall we?
<achuni> ok
<achuni> jml: ah, hm, and I'll probably need to stick my finger in the DB to make the openid for that user match my real life one
<achuni> sigh
<jml> achuni: psql launchpad_dev
<achuni> yep
<jml> achuni: sorry this is so hard. :(
<achuni> np
<jml> achuni: nothing in /var/log/mail.log
<achuni> jml: once I've gone that far, I might as well straighten out the emails in the db
<jml> achuni: it's 9pm here. I need to cook some food and start doing what I had meant to start at 7pm. Will still be around, just laggy.
<achuni> jml: go, get some rest.  I'll fiddle with this and let you know how it went in an email
<lifeless> gary_poster: do you think the test suite is robust enough for devs to use (tip) testr --parallel + lxc themselves and not expect egregious fallout ?
<allenap> lifeless: My wife needs me so I have to go. I'll send an email about maas. It should be reasonably short. Sorry if you'd carved out some time for me.
<lifeless> allenap: doh forgot to ping you back
<lifeless> only about 20m wasted :(
<allenap> lifeless: No worries :)
<lifeless> allenap: am here if you get time later, otherwise yes email is cool
<allenap> lifeless: Cool, okay.
<gary_poster> lifeless, we're still seeing test suite failure 1/3 of time.  We closed a bunch of the known errors today so maybe tomorrow stats will be much better.  33% failure seems a bit egregious to me but that's subjective.
<gary_poster> We're also starting to see repeats, which is good.
<gary_poster> seeing a new, different failure every test run was a bit disheartening.
<lifeless> gary_poster: yeah, I can well imagine
<lifeless> gary_poster: I *knew* this would be a long tail :(
<achuni> jml: \o/ success
<jml> achuni: yay
<jml> achuni: that was with non-commercial, right?
<achuni> jml: that was with non-commercial, right.  I'll try with commercial next for completeness.  I'll let you know if I got email
<jml> achuni: thanks.
<achuni> jml: I got the 'proof of purchase' email from sca, but none from LP so far
<achuni> jml: then again, I never got one for confirming my email either
<jml> achuni: I reckon it's just disabled for the demo instance
<achuni> yep
<jml> achuni: I guess we can verify that behaviour when we check against staging?
<jml> (does staging send email?)
<achuni> yep, and... I don't know
<achuni> jml: success with the commercial one too, I didn't expect that to fail, should I have been looking out for something?
<jml> achuni: I don't think so.
<achuni> jml: once again, got the email from sca but none from LP
<jml> achuni: I guess if we had email, you should have received some for the non-commercial and not received any for commercial
<jml> achuni: I just wanted to be extra sure.
<achuni> yep
<jml> achuni: so I guess now you mark the MP as Approved and we ask a Launchpadder to land it?
<achuni> jml: so, the other bit I didn't test was creating users via the xmlrpc api
<jml> achuni: oh right.
<jml> achuni: I think the only way this change could affect that is by some sort of quantum entanglement
<jml> achuni: can you test it?
<achuni> jml: I can give the MP my +1 fwiw, I wouldn't vouch for general launchpaddyness of the code :)
<jml> achuni: oh, that's been done already
<achuni> neat
<achuni> jml: I'm happy to check that on staging
<jml> achuni: ok, thanks.
<achuni> jml: yep, I don't think it's likely to be affected by this change
<achuni> jml: which is the MP?
<jml> achuni: https://code.launchpad.net/~jml/launchpad/drop-special-commercial-permissions/+merge/104270
<achuni> txs
<achuni> jml: your plan is to land the 'shut_up' bit next to get it deployed together with this change?
<jml> achuni: no, they'll probably be deployed separately
<jml> achuni: LP deploys at the drop of a hat, pretty much
<achuni> jml: wouldn't that mean that users that purchase software would get unwanted email for a while?
<jml> achuni: no, because you'll still set commercial, right?
<jml> achuni: it still governs email, it just doesn't grant any special permissions.
<achuni> ah neat
<jml> achuni: we'll work with you closely as we deprecate it too, probably having a period of supporting both
<achuni> jml: and, approved
<jml> achuni: great, thanks.
<achuni> jml: I'll see if I can add sca to the teams that own those two rogue ppas right away
<achuni> jml: when d'you think this would be deployed, approx?
<achuni> jml: btw, happy for you to disappear any time, I realize it's way past eod for you :)
<jml> achuni: +10 hrs minimum
<jml> achuni: it's OK, I'm at my laptop writing a speech
<achuni> k
<jml> achuni: http://lpqateam.canonical.com/ says that there's only one revision waiting to be deployed right now.
<achuni> neat
<jml> hello hello calling anyone with Launchpad commit privs
<jml> Please land https://code.launchpad.net/~jml/launchpad/drop-special-commercial-permissions/+merge/104270
<achuni> jml: that's fine, I was wondering if it would be today or next week :)
<jml> achuni: oh, I should double check
<jml> achuni: you can qa against qastaging readily enough, right?
<jml> (as opposed to staging)
<achuni> ehhhh
<achuni> jml: probably :)
<jml> achuni: hmm, ok.
<achuni> jml: there's nobody around from ISD atm to set up Ubuntu Pay if that needs changing
<jml> achuni: ah.
<jml> achuni: hmm.
<jml> seriously, does no one actually hang out on this channel any more?
<jml> achuni: I'll make a note on the MP then.
<maxb> [6/goe~
<maxb> oops
<lifeless> jml: who do you need ?
<achuni> jml: good news is, it seems those commercial ppas that aren't owned by commercial-ppa-uploaders aren't published by sca
<jml> lifeless: someone to land https://code.launchpad.net/~jml/launchpad/drop-special-commercial-permissions/+merge/104270 but ideally such that we can qa it on staging.launchpad.net rather than qastaging.launchpad.net
<lifeless> jml: you can't ?
<jml> lifeless: I abandoned my commit privileges when I left the team
<jml> lifeless: also, I don't exactly know how things get to staging.lp.net any more :)
<lifeless> I don't think you did
<lifeless> they get to staging in the same way
<lifeless> either trickle down from stable or direct from db-devel
<lifeless> whats different is that we never ever deploy code from staging now
<lifeless> only from stable
<lifeless> to do what you want, land on stable, note in the bug for qa that you need it on staging, wait, then qa on staging.
<lifeless> what do you need from staging specifically?
<lifeless> jml:  https://launchpad.net/~canonical-launchpad-emeritus/+members#active
<jml> lifeless: we want to test the whole chain of buying software through the USC. That has to involve Ubuntu Pay.
<lifeless> jml: go on
<jml> lifeless: there's a well-established way of testing against staging LP
<jml> lifeless: testing against qastaging is not so well-established, will probably require some kind of change to Ubuntu Pay, and no one with access will be around to do so.
<lifeless> staging runs the next schema, oop, gotta run cynythia waking up
<jml> lifeless: although on those last points, achuni could speak better.
<achuni> lifeless: yep, basically sca's staging instance is configured to talk to LP staging
<achuni> so testing against staging LP is something we do every rollout of our own
<achuni> lifeless: testing against qastaging would need config changes in sca staging, which would mean I wouldn't be as sure if something goes wrong about if it was the LP change that broke things, or a bad config update
 * jml tries ec2 land.
<jml> you know what
<jml> 'ec2 land' from 3g is pretty unreliable.
<lifeless> achuni: you should make those changes when you can
<lifeless> achuni: LP qastaging is the best place to test interop
<achuni> lifeless: ack
<jml> lifeless: I can't ec2 land from this network
<jml> lifeless: would appreciate you landing
<lifeless> my landing story is horked just now
<jml> achuni: may I kill that instance now?
<lifeless> I will arrange a volunteer
<jml> lifeless: ah ok.
<jml> lifeless: thanks.
<achuni> jml: sure, thanks!
 * lifeless calls for a volunteer to ec2land jml's https://code.launchpad.net/~jml/launchpad/drop-special-commercial-permissions/+merge/104270 branch
<jml> g'night all!
 * wgrant lands it
<wgrant> Night jml
<lifeless> wgrant: thanks
<SpamapS> Hi! I want to make https://code.launchpad.net/~openstack-ubuntu-testing/charms/precise/rabbitmq-server/trunk the branch for lp:charms/rabbitmq-server .. I also don't want this to be stacked on any other branches anymore...
<SpamapS> is there an easy way to override the stacking somehow?
<StevenK> wgrant: I think status uses the name, IE ' status FIXCOMMITED', but I can't see any tests.
<lifeless> SpamapS: bzr reconfigure can do it
<lifeless> SpamapS: but why do you care ?
<SpamapS> I've been warned against messing w/ stacking in the past
<SpamapS> lifeless: its stacked on oneiric.. which will some day go away
<lifeless> well, you'll have a system issue at that point
<lifeless> I wouldn't spend time special casing charms now
<SpamapS> The issue was caused by the branch-distro disconnect between trunk<->seriesname
<StevenK> wgrant: Ah, yes. The docs say ' status fixreleased'
<SpamapS> there was already a rabbitmq-server charm in precise.. but it was called 'trunk', so the branch-distro script missed it, and brought forward a really old and busted one
<lifeless> I though branch-distro followed the metadata
<lifeless> not the name
<SpamapS> It does a duplicate check using the name
<lifeless> anyhow
<lifeless> stacking doesn't affect the output of 'bzr branch'
<lifeless> setting the series branch is important though, havge you managed to do that yet ?
<SpamapS> Alright. There was also an instance where we wanted to delete the stacked-upon branch
<lifeless> reconfigure again
<StevenK> wgrant: http://pastebin.ubuntu.com/965852/
<wgrant> StevenK: That looks a bit more sensible, indeed.
#launchpad-dev 2012-05-04
<StevenK> wgrant: Good enough, or you'd prefer other changes?
<wgrant> Looks quite reasonable to me.
<wgrant> Does it work?
<StevenK> The tests all pass
<StevenK> And I think I wrote enough of them. :-)
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/bugs-information_type-mail/+merge/104483 has been updated.
<wgrant> StevenK: r=me
<mwhudson> jcsackett, benji: ayt?
<mwhudson> oh never mind
<wallyworld> wgrant: when do you expect use_flat to be ripped out?
<wgrant> wallyworld: Of get_bug_privacy_filter? Undefined.
<wgrant> Hopefully within a week, but it's a bit hard to tell.
<wallyworld> wgrant: ok. i want to use get_bug_privacy_filter() but it has no use_flat paparm; i'll add one
<wgrant> wallyworld: I have one added in one of my local branches, but it's like two lines and your branch will probably land before mine, so go ahead.
<wallyworld> rightio
<wallyworld> or if yours lands first, i'll deal with it. i have a few more branches to go yet
<wgrant> Yeah
<wgrant> It's going to be a pretty trivial conflict.
<wallyworld> wgrant: it seems the gbpf doesn't have a term which queries on bugtaskflat.access_policies, only access_grants
<wgrant> wallyworld: Right, at present it doesn't use access_policies, because there are no policy grants.
<wgrant> There's a card in Next about making it do that
<wgrant> It's 3 lines of code, and if that function's used everywhere it'll apply everywhere
<wallyworld> are there really no policy grants?
<wgrant> There are no policy grants yet.
<wgrant> That's an All permission
<wallyworld> on qas there are :-)
<wgrant> Which don't exist on prod, because the UI isn't write-enabled.
<wgrant> True
<wallyworld> wgrant:  so if i want to use gpbf i will need that extra functionality. so i will have to pick up that card from the board and do that
<wgrant> wallyworld: Why?
<wallyworld> i need it for the remove subscriptions job
<wgrant> No.
<wgrant> The current definition of access to bugs doesn't currently include policy grants.
<wallyworld> yes, it was even in the query you pasted
<wgrant> It was, yes.
<wgrant> That'll be the final thing.
<wallyworld> sure, but it's still correct to use it
<wgrant> But if you're going to use the existing function, you don't have to care about that stuff.
<wallyworld> i will be writing tests which require it or else trhey will fail
<wgrant> Ah, right, it could make tests slightly awkward, indeed.
<wgrant> You can steal the clause pretty much verbatim from the query I pasted yesterday.
<wgrant> And stick it into the function.
<wallyworld> yep
<wgrant> And it will Just Workâ¢
<wallyworld> should do
<wallyworld> hopefully
<wallyworld> wgrant: is there a bug number?
<wgrant> Don't think so
<wallyworld> ok, will raise one
<wgrant> You can probably find a bug search test to extend to cover APGs.
<wallyworld> hope so :-)
<wallyworld> wgrant: i can't find any bug search tests which create an artifact grant to give visibility; all the current tests seem to use subscription and reply on the triggers to update the artifact grant table. is that correct?
<wgrant> wallyworld: That's correct. Everything still assumes that subscriptions are the only things that confer access to bugs.
<wgrant> Now that legacy search is gone that can start to change.
<wallyworld> wgrant: ok. i've found a doc test in bugtask.txt which i can add to i think
<wgrant> (there was no performant way to respect grants/policies in legacy search, so flat search was a prereq for all this)
<wallyworld_> wgrant: can you see why this test isn't working? the new one is the one at the bottom https://pastebin.canonical.com/65516/
<wallyworld_> it fails because it still returns just the 3 bugs. perhaps it's not using the flat search?
<wgrant> wallyworld_: Non-flat search doesn't exist any more.
<wgrant> So it's probably using flat search.
<wgrant> wallyworld_: The bug isn't PROPRIETARY
<wgrant> wallyworld_: So the grant doesn't grant access to anything.
<wallyworld_> ah, USERDATA
<wallyworld_> doh
<wallyworld_> thanks
<wgrant> You'll want to ensure/find that, rather than create.
<wgrant> But yes.
<wgrant> USERDATA
<wallyworld_> yep
<StevenK> wallyworld_: Are you reviewing today?
<wallyworld_> StevenK: yes
<StevenK> wallyworld_: I have an MP up
<StevenK> It's only 221 lines, disappointing.
<wallyworld_> StevenK: sorry, wasn't running thunderbird
<StevenK> wallyworld_: Given your recent RAM issues ... :-P
<wallyworld_> yep :-(
<StevenK> I don't see why.
<wallyworld_> StevenK: i have to run out for 30mins to get kid from school
<wallyworld_> and buy ram on the way home
<StevenK> I have 4GiB of RAM with 64-bit, and I'm only 300MiB into swap with firefox, thunderbird, quod libet, gvim
<StevenK> wallyworld_: Given wgrant has wedged the deployment pipeline, it's not urgent.
<wallyworld_> ok. back soon.
<StevenK> I don't think I'll finish the garbo job this afternoon, sadly.
<StevenK> I wonder if the branch garbo job is doable using set
<wgrant> It should be, es.
<wgrant> yes
<wgrant> Since everything will just go to USERDATA
<StevenK> Everything?
<StevenK> It's either PUBLIC or USERDATA
<wgrant> Although I don't know if set() respects limits these days
<wgrant> Everything private.
<wgrant> Hm.
<wgrant> There may be few enough branches that a manual UPDATE is quick enough, too.
<StevenK> 500k on DF when I checked
<StevenK> And lifeless will smack us
<StevenK> Not sure how to use .set() against what I get back :-(
<StevenK> wgrant: Is "CASE WHEN Branch.explicity_private IS NOT NULL OR Branch.transitelvy_private IS NOT NULL THEN USERDATA ELSE PUBLIC" expressible in Storm?
<wgrant> StevenK: No, use an SQL().
<wgrant> Also, that's redundant.
<StevenK> Oh?
<wgrant> explicitly_private implies transitively_private
<wgrant> And they're not likely to be NULL
<wgrant> They're boolean.
<StevenK> Oh, so just check transitively_private IS TRUE ?
<wgrant> s/ IS TRUE//
<StevenK> So if I need to use SQL, I need the entire statement. Bleh.
<wallyworld_> StevenK: you will export IBranch.transitionToInformationType() later?
<StevenK> wallyworld_: After it does more than just blindly setting the information_type, yes.
<wallyworld_> ok
<wallyworld_> StevenK: i'm wondering if there needs to be a test like test_information_type_private_team to check that info type = public when required
<StevenK> wallyworld_: There is, that's why there is a few self.assertEqual(InformationType.PUBLIC, ...) scattered around the place.
<wallyworld_> ok
<wallyworld_> r=me, thanks
 * wallyworld_ sighs. sometimes lp is so slow at updating diffs :-(
<StevenK> 528? When did that happen?
<wgrant> Mine, last night.
<wgrant> 526 527 were already taken
<StevenK> Ah
<wgrant> I suspect by stub test images, but that's merely a guess
<stub> didn't one of them go live?
<StevenK> wgrant: Do you think it's worthwhile adding self.log.debug() to the garbo job about progress?
<wgrant> StevenK: The looptuner should be enough output
<wgrant> At DEBUG2
<wgrant> I think we run it at DEBUG2, too
<StevenK> Right, then I'll bug wallyworld_ again. :-)
<wgrant> stub: Your 525 was live.
<wgrant> wallyworld_: r=me
<wallyworld_> thanks!
<StevenK> Haha, 3 underscore Ian
<StevenK> wallyworld_{,__}: I have another one for you.
<wgrant> wallyworld_: A pretty quick dev-only one for you, if you still have time: https://code.launchpad.net/~wgrant/launchpad/easier-remote-access/+merge/104684
<wgrant> Heh
<wallyworld___> me finds a screwdriver to add extra ramyou're so kind
<wallyworld___> sure
<StevenK> wgrant: You can review the garbo job branch, if you wish, it's +38
<wgrant> I don't want to steal wallyworld________'s new non-swapping thunder.
<wgrant> But I might anyway.
<StevenK> Hahaha
<wgrant> StevenK: Your linebreaks are criminal, sqlvalues is evil, and you don't test public.
<wgrant> Also self.transaction wut?
<wallyworld___> wgrant: the branch looks ok but i'm not really sure i know enough to give it a proper +1
<wgrant> The worst it can do is break new dev installations :)
<wgrant> And it doesn't break new dev installations.
<wallyworld___> WCPGW
<wallyworld___> r=me
<wgrant> Thanks.
<wallyworld___> np
<StevenK> wgrant: Without sqlvalues, it puts in 'User Data' which doesn't help
<wgrant> StevenK: Use the ?
<wgrant> SQL("foo ? bar ?", params=('whatever', 'oh look something else unsafe'))
<wgrant> Hm
<wgrant> Although it's possible that won't work for a dbenum
<wgrant> In which case I guess sqlvalues will have to do.
<StevenK> I can try ? if you wish
<wgrant> Or you could just use params= with .value directly. That's probably best.
<wgrant> Oh good
<wgrant> There's a third copy of those hostnames in lp-dev-utils
<StevenK> Hahaha
<StevenK> wgrant: http://pastebin.ubuntu.com/966263/
<StevenK> wgrant: If it isn't duplicated it might be forgotten!
<wgrant> StevenK: Your linebreaking in the first hunk is still likely to draw criminal charges.
<StevenK> wgrant: http://pastebin.ubuntu.com/966266/
<wgrant> Deindent line 13 and you're good
<StevenK> wgrant: Diff updated, please have a look.
<wgrant> StevenK: Branch.information_type is unindexed.
<StevenK> Hmmm
<wgrant> I sense a 2-statement DB patch in your fu=ture
<StevenK> Haha
<StevenK> wgrant: Two?
<lifeless> index + metadata
<wgrant> CREATE INDEX and INSERT INTO LDR
<StevenK> Yah
<jtv> I think devel's versions.cfg still relies on some zope versions that are too old for Precise or something.  I can't build on Precise, without bumping a bunch of versions in versions.cfg to the ones I actually have.
<wgrant> jtv: What's the error?
<wgrant> jtv: Perhaps it breaks if you have some old versions installed through apt (for maas)?
<wgrant> Otherwise you should use LXC.
<jtv> Error: Couldn't find a distribution for 'zope.publisher==3.12.0'
<jtv> etc.
<wgrant> Sure your download-cache is up to date?
<jtv> No
<jtv> I only have newer versions than what's in devel's versions.cfg.
<wgrant> wgrant@lamuella:~/launchpad/lp-branches/devel$ bzr ls download-cache/dist | grep zope.publisher
<wgrant> download-cache/dist/zope.publisher-3.12.0.zip
<wgrant> download-cache/dist/zope.publisher-3.12.1.tar.gz
<wgrant> wgrant@lamuella:~/launchpad/lp-branches/devel$ bzr ls download-cache/dist | grep zope.publisher
<wgrant> download-cache/dist/zope.publisher-3.12.0.zip
<wgrant> download-cache/dist/zope.publisher-3.12.1.tar.gz
<wgrant> Your download-cache must be bad
<jtv> Bad in what way?
<jtv> Bear in mind that I've been running Precise since January, and so haven't been able to run any Launchpad setup since then.
<jtv> But I seem to get past this if I edit versions.cfg and bump z3c.ptcompat from 0.5.3 to 0.5.5; zope.app.component from 3.8.3 to 3.8.4; and so on.
<StevenK> stub: Can you have a look at https://code.launchpad.net/~stevenk/launchpad/db-branch-information_type-index/+merge/104689 ?
<StevenK> I can't seem to request a review of lifeless, but it's one index.
<wgrant> jtv: Which zope.publisher is in download-cache/dist in the branch you're trying to build?
<wgrant> jtv: Perhaps try make clean
<wgrant> jtv: It may still think it's using 2.6 or something
<jtv> Crap â even âmake cleanâ fails
<jtv> rm: cannot remove `sourcecode/pygpgme/gpgme/*.so': Too many levels of symbolic links
<stub> StevenK: r=stub. That should go live?
<wgrant> jtv: I think your download-cache, eggs, sourcecode etc. symlinks might be a little screwed.
<StevenK> stub: As long as backups aren't in our way, yup.
<jtv> They may well be.  I haven't been able to run the rocketfuel scripts for so long.
<wgrant> They haven't changed since 2009, but maybe.
<jtv> The scripts maybe, but I'm sure the data has!
<jtv> load_cache fails: No JSON object could be decoded
 * StevenK stabs Level3
<jtv> Trying more make clean & version bumps.
<StevenK> jtv: make clean and then cd into your download-cache and bzr up
<jtv> The download-cache in the branch?
<StevenK> Which is a symlink
<jtv> $ bzr up
<jtv> Tree is up to date at revision 465 of branch bzr+ssh://bazaar.launchpad.net/%2Bbranch/lp-source-dependencies
<jtv> Let's see what âmakeâ does now.
<StevenK> stub: DB patch is on db-devel r11569.
<stub> ta. I've added it to my todo for when the backups complete
<StevenK> stub: Thanks.
<StevenK> wgrant: ^ Can haz approve for garbo job *now*? :-)
<jtv> Wow, I managed to have a conflict in sourcedeps.cache.
<StevenK> Unsurprising
<jtv> Really?  And I was so proud.
<wgrant> StevenK: Why db-devel?
<wgrant> StevenK: Live patches go to devel
<StevenK> Bleh
<StevenK> wgrant: Merge it after Ian's FDT, then?
<wgrant> I guess.
<jtv> \o/ I can âmake.â  Thanks guys.
<wgrant> It's the first step toward recovery :)
<jtv> Our Makefile really needs work again.  I had it doing parallel tasks, and not rebuilding mailman and css/js every time you rebuild the db schema at one point.  :(
<StevenK> jtv: wallyworld___ has added things to prevent mailman building
<jtv> I so hope I wasn't wrong in bug 994410.
<wallyworld___> StevenK: what did i do?
<wgrant> Hmm
<wgrant> Do I want to teach ec2 to preserve the buildout egg cache?
<noodles785> Hi people, is a tag added to an LP bug as soon as it is deployed to staging or qastaging? (we'd like to do integration testing as soon as bug 992691 lands on staging).
<_mup_> Bug #992691: Special permissions for 'Archive.commercial' are not needed <tech-debt> <Launchpad itself:In Progress by jml> < https://launchpad.net/bugs/992691 >
<wgrant> noodles785: Yes, it will have qa-needstesting added a few minutes after it's on either qastaging or staging. In this case you'll need to wait up to 12 more hours for it to hit staging, since it'll be on qastaging first.
<noodles785> wgrant: the egg cache is left pristine isn't it? I don't see why not.
<noodles785> wgrant: perfect, thanks!
<wgrant> However, the branch failed ec2 a couple of hours back. It's trivial to fix.
<wgrant> Also, you've been incremented by 10?
<noodles785> 10? 10 credits?
<wgrant> Ypi
<wgrant> Bah
<wgrant> You've traditionally been noodles775
<noodles785> Hah, oh.
<wgrant> noodles775: So, hopefully jml will be around to fix the test failure tonight. If not, I'll do it. Either way I'll reland it this evening, and it will be on staging when you start on Monday.
<noodles775> wgrant: great, thanks!
<wgrant> It's about 7-12 hours from landing to qastaging, and then another 13-20 after that until staging.
<wgrant> This'll hopefully be down to 90 minutes and 120 minutes once parallel testing is complete.
<adeuring> good morning
<czajkowski> aloha
<cjwatson> jtv: was bug 994410 supposed to be on LP rather than MAAS?
<jtv> cjwatson: absolutely
<jtv> And I work so hard to catch chromium at that.
<jtv> Thanks.
<jtv> Reviewer needed!  https://code.launchpad.net/~jtv/launchpad/bug-994410-stop-using-latest_message/+merge/104703
<czajkowski> jtv: is there anything we need to be aware of when adding a new language in Ubuntu/lp
<jtv> Yes: there are several very different things that people call âadding a language.â
<czajkowski> https://answers.launchpad.net/launchpad/+question/195970
<czajkowski> wanted to add BRX
<jtv> czajkowski: these people have really done their research.  It answers all questions I might have; you're ready to go.
<czajkowski> so the next bit is I've not added a langauge before and don't know where in lp I do this
<czajkowski> so not sure I can  also :/
<jtv> I don't think there's an ISO 639-2 code, so the 639-3 is fine.  It's an Individual language code, not a Collective which we can't accept.
<jtv> This is properly "adding a language" in Launchpad.  Do you see the option just under âAvailable languages in Launchpadâ?  https://translations.launchpad.net/+languages
<czajkowski> jtv: yes
<jtv> Well there you go then.  You've got the plural expression, language code, English nameâ¦
<czajkowski> thank you
<jtv> Please convey to them my appreciation for the care and effort they put into this request.
<jml> hello
<jml> I'm around to fix the test failure.
<wgrant> jml: Did you get the email?
<jml> wgrant: I think so.
<wgrant> I didn't realise my suggestion was so offensive to the test suite :(
<czajkowski> jtv: thank for the help.
<jtv> no worries
<jtv> czajkowski: don't forget to enter the plural-form data!
<jml> wgrant: yeah.
<jml> wgrant: I like that we have auto-generated tests for IPersonRole, but that we don't auto-generate it ourselves.
<jtv> czajkowski: there's 2 plural forms, and the formula is ân != 1â (I think entering it in that form will work)
<wgrant> jml: That is pretty cool, yeah.
<jtv> stub: I want to drop a column from POFileTranslator, and stop the TranslationMessage trigger from writing it.  Would it be best to do that in one patch?  Or a sequence of smaller steps?
<czajkowski> jtv: am trying but I keep getting an oops
<jtv> Then I suppose the format isn't quite right.  Have a look at other languages; see if you can crib the format.
<wgrant> jtv: One patch is fine, as long as the appservers have already stopped going near it.
<jtv> OK thanks
<jtv> Speaking of which, here's my MP for achieving that.  :)  https://code.launchpad.net/~jtv/launchpad/bug-994410-stop-using-latest_message/+merge/104703
<stub> jtv: There is no problem doing both the column drop and the trigger change in one patch. Both are fast operations.
<jtv> Thanks.  Just wondering if there were multi-server rollout implications.
 * jml fixes now
<stub> jtv: We *might* be able to twiddle the trigger live. The column drop has to go through fast downtime. Easier for everyone if it can go in one patch.
<czajkowski> jtv: done
<jtv> stub: very well
<jtv> czajkowski: looking good
<czajkowski> jtv: sorry about that peksy oops :)
<jtv> Oh, the code change?  Didn't take long to figure out.  :)
<jtv> stub: I suspect https://dev.launchpad.net/Database/LivePatching is far out of date
<jml> hey
<jml> are there any webservice tests written as unit tests?
<jml> is there any non-historical reason for writing them as doctests?
<jml> they clearly don't function as documentation for the actual users of the webservice.
<stub> jtv: Was this an example of a CRITICAL (genuine) bug drowned out by CRITICAL (policy) bugs?
<cjwatson> jml: lib/lp/soyuz/browser/tests/test_archive_webservice.py comes to mind
<cjwatson> (Quite a few others, 'find -name \*webservice\*')
<jml> cjwatson: thanks.
<jtv> stub: arguably â in any case, we never actually seem to have made a decision to drive critical bugs to zero; we made a decision to allocate them some time, we announced that that would drive them to zero, and then we instated a policy for creating new ones at a faster rate.  :)
<jml> :(
<wgrant> cjwatson, jml: Note that those tests use launchpadlib, and launchpadlib tests are *very* slow.
<wgrant> Most of the doctests avoid launchpadlib. I'm not sure if there are non-doctests that do.
<jml> Hmm.
<stub> Just thinking we knew this was a timebomb and we let it explode creating more work (cleanup)
<jtv> stub: absolutely.
<jtv> And the flood of policy-critical bugs gave us easier things to fix instead.
<jtv> If you want to fill a bucket of dirty water with clean water, you can pour it out first and then refill it; or you can put it under the tap.  But the latter is something you only do if you can't stand to be without a full bucket for a while.
<jml> wgrant: ok, that's fixed. r15203
<stub> jtv: Are these tests you removing going to have to be rewritten when we maintain the latest translator asynchronously?
<jtv> There will be other tests, but much simpler.
<jtv> There won't be any complications from how data got where it is; it's either there or not there.
<jtv> Dragging the data through the same changes as other data is what makes it difficult.
<wgrant> jml: It's in devel.
<jml> wgrant: oh, you just landed it directly?
<wgrant> jml: Yeah
<wgrant> No point re-ec2ing it
<jml> wgrant: I guess not.
<jtv> Thanks for the review stub
<StevenK> Bah. Forgot the --incremental for r15204. :-(
<jtv> Erâ¦ what's the current landing procedure?  I've been away for a while.
<wgrant> jtv: bin/ec2 is now in lp:lp-dev-utils
<wgrant> So grab that, put it in your PATH, 'ec2 land'
<jtv> Thanks.
<jtv> Grar.  The bugs that come out of the woodwork when you start simplifying POFileTranslator.  Should've done it years ago.
<jml> jtv: I know what you mean.
<jtv> You've looked at it?  Or seen the same thing elsewhere?
<jml> jtv: I don't think I've ever thought, "man, I'm glad I postponed cleaning that up, that made everything go so much better and faster"
<jml> jtv: same thing elsewhere.
<jtv> Heh.
<jtv> To be honest, I think by nature _not_ cleaning up pays off beforehand, unlike cleaning up which pays off afterwards.
<jtv> But I don't think we've been sorry about having cleaned things up very often.  :)
<jml> jtv: well, practically every time I've got some code and my understanding doesn't quite match it (or there's a clean up I can see), whenever I do finally fix it up, I *always* think I should have done it earlier.
<jml> mostly because it would have made everything between the realization and the cleanup easier.
<jtv> Then I wonder: if you adjust to that and clean up earlier, what would be the signal that you've overshot?  Or is there no such thing?
<jtv> (I'm not entirely un-selfish in asking)
<cjwatson> Sometimes cleanup is blocked on something else, and waiting means the cleanup is "delete big pile of stuff" rather than "spend two weeks fixing other stuff and then delete stuff".
<jtv>  My current situation in a nutshell.  :)
<jtv> But with a twist: I found a kind of circular dependency.
<jtv> Which reminds me that I should keep dpm and Deryck updated.
<jml> jtv: hmm.
<dpm> heya jtv, what's up?
<jtv> Speak of the  :)
<jml> jtv: perhaps decreased certainty about what to clean up?
<jtv> dpm: I'm just dealing with some breakage during the Quantal opening.
<jtv> jml: something to keep an eye out for next time.  :)
<jtv> dpm: no worries, we'll get it done.  But I've gone off on a bit of a tangent cleaning things up that made it hard to fix these problems we've been having.  And I found some things.
<jml> jtv: sometimes I get a certain circular mental motion / paralysis when I'm trying to clean things up when I really should just solve the damn problem.
 * dpm worries that "things" are not "good things"
<jtv> dpm: Well the bad things have already happened and now we can work on fixing them.  Some deeply obscure PLPSQL code looks to me like this would happen:
<jtv> jml: also a good thing to keep an eye out for then.  I'll try to remember.
<jtv> dpm: say you have two POFiles that share.
<jml> sorry, figuring this out myself now
<jtv> And there's a record of your latest translation in one of them, but not in the other.
<jml> I think the tighter the loop, the less cost in overshooting either way
<jtv> Probably.
<jtv> dpm: now you enter a new translation, in either of them.
<jtv> dpm: the obscure code is the code that keeps track of who's contributed to which POFiles.
<jtv> And it says âoh good, I've just updated an existing contribution record (POFileTranslator is the actual table); that means I don't need to create a new one.  I'm done.â
<jtv> And so it never creates a similar record for that other POFile â even if that was the one you were translating in in the first plcae!
<jtv> *place
<jtv> dpm: sound familiar?
<jtv> This is what users have been complaining about that we just couldn't figure out.
* bac changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: bac | Firefighting: - | Critical bugs: 3.47*10^2
<dpm> jtv, oh, so if I understand it correctly that's the bug we recently talked about?
<jtv> dpm: I think it just might be.
<rick_h_> anyone know what's up with Error: Couldn't find a distribution for 'pytz==2012c'.
<rick_h_> ?
<rick_h_> pulled source-deps, not there, reran buildout but can't find it
<rick_h_> anyone know where I'm supposed to have gotten it from?
<wgrant> rick_h_: download-cache
<wgrant> bzr up it
<rick_h_> hmm, did but said it was up to date
<rick_h_> Tree is up to date at revision 465 of branch bzr+ssh://bazaar.launchpad.net/+branch/lp-source-dependencies
<rick_h_> ?
<wgrant> 2012c is in there for 2.6 and 2.7
<rick_h_> ok, yea see it there, just buildout/make hating on me I guess.
<rick_h_> ok, thanks for verifying it's where it should be
<rick_h_> wgrant: doh, sorry was updating download-cache in ~/launchpad/ not realizing it wasn't symlinked into devel but was it's own copy
<wgrant> rick_h_: Hm, that's meant to be symlinked.
<rick_h_> yea, that's what I thought. I know I did some hackery due to precise install fun so probably my fault
<wgrant> The default setup is that devel/download-cache is symlinked to ~/launchpad/lp-sourcedeps/download-cache, and all the other branches have theirs symlinked to devel
<rick_h_> right
<jml> tabs :(
<wgrant> jml: Ew, where?
<jml> wgrant: soyuz/configure.zcml
<wgrant> Just the one.
<wgrant> Still, destroy.
<jml> wgrant: with enthusiasm
<deryck> allenap, hi. You around?
<deryck> ah, see the lunch afk message now.  sorry, catch you later.
<allenap> deryck: Hi.
<allenap> deryck: I forgot to update my status.
<deryck> adeuring, rick_h_ -- sorry, got busy chatting and missed it was standup time.
<deryck> adeuring, rick_h_ -- be right there.
<jml> https://code.launchpad.net/~jml/launchpad/commercial-means-shut-up/+merge/104739
<jml> patch to rename IArchive.commercial
<rick_h_> deryck: https://code.launchpad.net/~rharding/launchpad/editstatus_timeout_874250/+merge/103912
<bac> jml: looking
<jml> bac: thanks.
<deryck> rick_h_, just ping me when that MP is good to go.  I'm not watching it, so will need the reminder.
<wgrant> rick_h_: It's midnight on Friday so I can't really give a proper review, but the _init_new_task thing strikes me as more than a little awkward. Have you considered merging that into createManyTasks, and having createTask just delegate to createManyTasks? That's the pattern we use elsewhere.
<rick_h_> deryck: all good
<rick_h_> wgrant: ah, good point. I can take a peek at that once I find why my MP isn't working right
<wgrant> It should remove a lot of duplicated default stuff, and eliminate the reasonably hideous getattr stuff that this structure requires
<rick_h_> yea, gotcha. started out smaller but grew over time, curse of taking too long
<wgrant> Yeah, that often happens, particularly when setifying ancient code.
<rick_h_> ok, need bzr help, I removed lib/lp/bugs/doc/bugtask.txt
<rick_h_> so the file is gone, but hte MP says there's conflicts in it
<rick_h_> and doesn't show the file deleted
<rick_h_> my bzr branch shows no conflicts, file is gone, push and push --overwrite no helpy, any other ideas?
<rick_h_> ah, sec...see this in the MP diff  renamed file 'lib/lp/bugs/doc/bugtask.txt' => 'lib/lp/bugs/doc/bugtask.txt.THIS'
<jtv> Should downtime database branches still branch off db-devel?
<rick_h_> jtv: I've yet to do one, but I do believe so
<wgrant> jtv: Yes, but it doesn't really matter since there's rarely more than in progress at a time.
<jtv> Naturally.  Thanks.
<wgrant> fastdowntime patches -> db-devel, live -> devel
<bac> hi jml -- your branch looks mostly good but i have a policy question about changing the signature of 'createPPA' and removing 'commercial' from being exported.  since those items were introduced in beta, we need to continue supporting them in beta and 1.0.
 * wgrant sleeps.
<jtv> nn wgrant
<rick_h_> see wgrant
<jml> bac: we (consumer-apps) are the only users permitted to do that
<jml> bac: I myself added the 'commercial' parameter a week or two ago.
<wgrant> Yeah, API stability is not a concern here.
<wgrant> Possibly for reading Archive.commercial, but meh.
<wgrant> That flag wasn't even around when 1.0 was released, I'm pretty sure
<wgrant> Nobody but CA has any cause to use it.
<jml> bac: and we're OK with the compat break, so I don't think it's a practical concern
<jml> wgrant: thanks :)
<jml> wgrant: sleep well & have a good weekend.
<wgrant> jml, rick_h_: Thanks, you too.
<czajkowski> rick_h_: please read all tweets today laced with thick irish accent - will make it funny for you , and hair pulling experience for me :)
<rick_h_> czajkowski: :)
<bac> jml: ok, so the createPPA signature change since beta was to add an optional parameter...so removing it is a-ok.  sounds reasonable
<jml> bac: thanks.
<wgrant> gary_poster: Argh, sorry. That cookie-authentication.txt failure is my fault.
<wgrant> gary_poster: I missed a space when I updated the domain list in setuplxc
<bac> jml: approved but your test is slightly broken
<jml> bac: oh?
<gary_poster> wgrant, oh cool.  good to know thx
<gary_poster> will look after babysitting/lunch time, starting now :-)
<wgrant> Heh
<wgrant> pqm-submitting a fix
<jml> bac: have pushed a fix for the test, as well as doc fixes that james_w recommended. Can you please land?
<jml> note, I expect test failures.
<bac> jml: would you set the commit message on the MP please.
<jml> bac: done.
<bac> jml: so you want me to 'ec2 land' but you don't think it'll make it?
<jml> bac: pretty much. renames in Launchpad never work the first time.
<bac> jml: alrighty
<jml> bac: I've done what I can, but grep only takes one so farr.
<bac> jml: is there a bug for this work ?
<jml> Hmm.
<jml> No. I can file one if you'd like.
 * jml wishes lp had rev-based QA
<bac> jml: where i guess there should be some qa, to ensure the UI doesn't break.  so a bug would be needed.
<bac> s/where/well
<jml> https://bugs.launchpad.net/launchpad/+bug/994644
<_mup_> Bug #994644: IArchive.commercial is misleading name  <Launchpad itself:New> < https://launchpad.net/bugs/994644 >
<jml> linked up
<rick_h_> ok, going to take bzr out back and shoot it...any masters around know how to trick bzr into removing the THIS file it's creating for it's own evil puposes?
<rick_h_> https://code.launchpad.net/~rharding/launchpad/editstatus_timeout_874250/+merge/103912
<rick_h_> tried creating a new branch and merging the old in, creating and removing the file itself,
<rick_h_> nothing in the conflict docs in bzr land is pointing me at any hints I can tell
<jml> rick_h_: there's only a .THIS? no .OTHER?
<rick_h_> jml: yea, I did at one point have a conflict I resolved there
<rick_h_> but that .THIS isn't in tree anywhere, it's only in LP's MP head for some reason
<rick_h_> so I can't do any sort of force delete, etc
<jml> rick_h_: so the current head on LP has got most recent devel merged in & conflicts resolved?
<rick_h_> jml: rgr
<rick_h_> if I try to merge with devel is says nothing to do, push, nothing to push, pull my branch, nothing to pull
<jml> rick_h_: what happens when you change into devel and merge your branch in to that?
<jml> i.e. manually on your own system
<rick_h_> but the MP says there's a conflict and refuses to delete the file
<rick_h_> hmm, haven't tried that, what I did do was create a fresh branch off of devel, and merge these changes into that new branch
<rick_h_> as soon as I pushed it up to create a new MP...it had this same problem
<rick_h_> so I'd wager that merging into devel itself would have this issue
<jml> rick_h_: because merging into devel is exactly what LP does, AIUI
<jml> rick_h_: that rename is *really weird*
<rick_h_> right, so figuring by some bzr voodoo I need to get rid of that .THIS which should never have been there after I did the bzr resolve on my conflicted file
<rick_h_> but that voodoo is escaping me
<cjwatson> Perhaps the file-ids for that file differ in the two branches
<cjwatson> bzr ls --show-ids would tell you
<rick_h_> looking
<jml> that would account for the original conflict but wouldn't account for committing the rename
<cjwatson> Well, it would suggest the simplest fix which would be to 'bzr rm --keep' the file with the wrong file-id and 'bzr add --file-ids-from=../devel' it with the same contents
<cjwatson> (If true)
<rick_h_> right, but the fiel isn't there at all, so I can't rm anything with it
<rick_h_> file that is
<rick_h_> there is no bugtask.txt.THIS
<cjwatson> Oh, hmm
<jml> underscores. sheesh.
<cjwatson> I'm stumped then.  If it were me, at some point I'd probably give up, make a fresh branch, and apply the patch ...
<rick_h_> so just manual diff/manually patch the trees vs trying to merge them with bzr itself
<rick_h_> yea, guess that makes sense, the commit history isn't important with this atm so works for me
 * jml tries.
<stub> gary_poster: That nl_  helper occasional failure is special.
 * rick_h_ cheers on jml
<gary_poster> stub, heh, yeah. :-)  benji is looking at it
<jml> watch this space: https://code.launchpad.net/~jml/launchpad/editstatus_timeout_874250/+merge/104763
<rick_h_> will do
<benji> stub: when I repeatedly run the test it fails about 1 in 30 times, when I edit the doctest that it's in to just include the failing test I can't get it to fail at all (in more than 100 runs)
<stub> benji: Is this with PG 9.1 or 8.4?
<benji> stub: 8.4
<jml> ok same prob.
<rick_h_> jml: ok thanks for trying. I'll feel better knowing it's not me just having a case of fridays not 'getting' something and work around it
<benji> stub: I have some reason to believe that the bad behavior is generated in these two lines:
<benji>         p = plpy.prepare("SELECT to_tsquery('default', $1) AS x", ["text"])
<benji>         query = plpy.execute(p, [query], 1)[0]["x"]
<stub> benji: I'd consider trying PG 9.1. If the problem goes away, we can blame plpythonu. It received significant updates in 9.0 and 9.1
<stub> benji: If there is some sort of memory allocation area, bind variables like you point out would be where they are at.
<stub> c/area/error/
<benji> stub: I quake in fear of doing a postgres upgrade
<stub> Its trivial really, and I just emailed saying devs will need to upgrade soon anyway :)
<benji> once more into the breach
<stub> benji: Have you managed to get debug information about what 'query' is being passed into plpy.execute from a failed run?
<benji> stub: yep, I couldn't figure out how to enable/locate the output of plby.debug so I just transmuted those calls to write to a file; everything looks good (even when the tests fail) through the "15" debug output (just before the lines I quoted earlier) so I added a "16" line just after and when I got a failure the logged query was messed up as you would expect
<stub> If the query string going in is correct, but the results from plpy.execute wrong, then I think we must be triggering a bug in plpythonu or tsearch2.
<abentley> rick_h_: So the conflict makes sense to me.  This is a case where one side wants to modify a file, and the other side wants to delete it.  It's a "contents conflict" rather than a "text conflict", because the file was deleted.
<stub> jtv: The 'IF EXISTS' clauses are likely 9.0 specific syntax
<stub> (and are unnecessary)
<abentley> rick_h_: Bazaar keeps the file around in case you choose to resurrect it.  But it renames it to foo.THIS because it's part of a conflict.
<jtv> stub: they are probably unnecessary, yes.  They're more to avoid development hassle than anything else.
<benji> stub: I'll try upgrading PG and see what happens
<stub> jtv: yer, I like them too (but I'm currently writing 9.1 specific code ;) )
<jtv> Grr.  Can't land: pqm says it can't verify my gpg sig.
<stub> ec2 land or bzr lp-land ?
<abentley> rick_h_: You can do diff -u lib/lp/bugs/doc/bugtask.txt.BASE lib/lp/bugs/doc/bugtask.txt.THIS to see what changes were made.
<jtv> stub: I did pqm-submit.
<jtv> (After ec2 land had the same problem)
<jtv> Meanwhile, looks like âIF EXISTSâ is 9.x only for the ALTER TABLE DROP COLUMN part of my patch.
<jtv> benji: if you're still on PG 8.4, maybe you could try out a âmake schemaâ on my branch?
<jtv> I'd sleep easier knowing that that worked.  :)
<jtv> It's lp:~jtv/launchpad/db-994410
<rick_h_> abentley: right, but I marked it as resolved so I have no files in tree named bugtest.txt*
<rick_h_> and I can't 'get' the file so I can see it by pulling/merging
<jtv> Definitely time to go home.
<abentley> rick_h_: Right, when you mark a conflict resolved, it deletes any files you had hanging around to help you resolve your conflict.
<benji> jtv: I'm trying your branch now.
<jtv> Thanks!
<rick_h_> abentley: right, so I follow the 'usual' process and thought I did it right from that standpoint, but no idea how to make the MP 'happy' at this point from this state
<abentley> rick_h_: You can retrieve the THIS version with "bzr cat lib/lp/bugs/doc/bugtask.txt"
<rick_h_> bzr: ERROR: u'lib/lp/bugs/doc/bugtask.txt' is not present in revision rick.harding@canonical.com-20120504145817-vw1o8a0212a8olkv
<jtv> benji: if it's not too much to ask, maybe run some arbitrary lp.translations tests to see if the trigger blows up on 8.4?
<benji> jtv: will do
<abentley> rick_h_: That would work in a devel branch, not in the editstatus_timeout branch.
<rick_h_> ah ok
<abentley> rick_h_: I don't think you'd have THIS in the editstatus_timeout.  It out to be OTHER in that case.
<abentley> rick_h_: And to retrieve BASE,  bzr cat -r ancestor:lp:~jml/launchpad/editstatus_timeout_874250 lib/lp/bugs/doc/bugtask.txt
<abentley> rick_h_: To make the mp "happy", you need to merge devel into editstatus_timeout_874250, fix the conflict, push.
<abentley> rick_h_: Since you can't push into jml's copy of the branch, you need to push your own copy, and resubmit the merge proposal to use your copy.
<rick_h_> abentley: right, but his branch is a copy of my branch which is already merged with devel, conflict resolved, and pushed
<abentley> rick_h_: When I merge his branch into devel, I get a conflict, so I don't think the conflict was resolved.
<rick_h_> ok, I merged devel with my branch and resolved it there
<rick_h_> but gotcha, it's still unhappy. I'll rework it and carry on. Thanks for looking into it abentley
<benji> jtv: no test failures in about 100 tests; unless you want me to run more I'll kill it now
<jtv> benji: fantastic.  Thanks.  Feel free to stop it.
<jtv> And with that, I'm off.  Have a good weekend everyone!
<benji> cool
<abentley> rick_h_: So, you merged 15195, but the file was changed in 15207.  That's why your merge didn't fix the conflict.
<dratone> hi all.
<benji> rick_h_: so, how did you fix http://paste.mitechie.com/raw/606/?  I'm upgrading to postgres 9.1 and have the same problem
<lifeless> benji: you need to enable the fti procedures, which is done as part of rocketfuel setup IIRC
<lifeless> benji: basically running the db setup part again
<benji> lifeless: thanks
<lifeless> (Thats a WAG, but I'm fairly sure about it)
#launchpad-dev 2012-05-05
<StevenK> wgrant: Sigh, self.findBranches()[:chunk_size] is bad -- https://bugs.launchpad.net/storm/+bug/820290
<_mup_> Bug #820290: resultset.set ignores slice, modifying data it shouldn't <Storm:New> < https://launchpad.net/bugs/820290 >
<wgrant> StevenK: Heh, I did mention that.
<mtaylor> hey guys -
<mtaylor> I'm getting an ssl handshake error when trying to connect to launchpad api
<mtaylor> httplib2.SSLHandshakeError: [Errno 1] _ssl.c:499: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
<mtaylor> I can login to staging, just not LPNET_SERVICE_ROOT
<mtaylor> httplib2.SSLHandshakeError: [Errno 1] _ssl.c:504: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
<mtaylor> gha
<elmo> mtaylor: looking
<mtaylor> elmo: thanks!
<mtaylor> elmo: fwiw, I get this on natty, oneiric and precise, and I hit it on both login_with() and login_anonymously()
<mtaylor> easy reproducable from a local laptop ... import launchpadlib ; launchpadlib.launchpad.Launchpad.login_anonymously('testing', launchpadlib.uris.LPNET_SERVICE_ROOT)
<elmo> mtaylor: sorry, got distracted by sabdfl, but someone else fixed it while I was talking
<elmo> mtaylor: is it working for you now?
<mtaylor> elmo: it seems to be fixed, thanks!
#launchpad-dev 2012-05-06
<lifeless> wheee
<lifeless> 17921  OOPS-10c6e2453422218f59fb3e8e621ea198  Person:+contactuser
<lifeless> wgrant: https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-bcba588b8f33bfa5da7c88e680763bf8
<wgrant> lifeless: Interesting, obviously a cold cache somehow.
<wgrant> lifeless: I suspect that GIN will be nice -- it's usually the FTI index that ends up cold.
<wgrant> Since the entire set of Ubuntu bugs is kept very hot by the regular full index scans.
<wgrant> lifeless: The only major repeatable timeouts are Person:+commentedbugs and Person:+bugs and their API equivalents for particularly prolific people (better than they used to be, but still bad), and things that use last_patch_uploaded, which I'm about to add to bugtaskflat.
<lifeless> cool
<lifeless> yesterday had 1.6K timeouts still
<wgrant> Yeah, mostly not bug-related, though.
<wgrant> Only a few hundred.
<wgrant> Most of them being +commentedbugs
<lifeless> kk
<lifeless> is the root bugs api one fixed ?
<wgrant> Ja
<wgrant> Overrode it to [], timeouts went away and nobody has complained.
<lifeless> hah
<wgrant> Hopefully DF will be back today so I can fix Distribution:+search's query
<wgrant> lifeless: All the +patches timeouts and most of Distribution:EntryResource:searchTasks will go away when I denorm latest_patch_uploaded.
<wgrant> StevenK: You didn't revert that bad rev..?
<StevenK> wgrant: I figured that since jml's QA was going to block it I could fix it better
<wgrant> StevenK: jml's thing will be deployable in 10 hours, so you'd best be swift.
<StevenK> wgrant: How best to fix it? self.findBranches().limit() ?
<wgrant> That won't work.
<StevenK> I didn't think so.
<wgrant> The Stormiest way to do it is to use a subselect.
<wgrant> Well, that's the SQLiest way to do it too
<wgrant> UPDATE doesn't support LIMIT
<StevenK> Ah, so it has to be WHERE id IS IN ...
#launchpad-dev 2013-04-29
<adeuring> good morning
<jelmer> moin moin
<mgz> hey jelmer
<jelmer> hey mgz, how's things in the far east?
<mgz> I'm the most east I've been for several weeks
<mgz> I liked your settling in message, was entertaining :)
<jelmer> heh, thanks :-)
<jelmer> what remote lands have you been hanging out ?
<mgz> the temperate rainforest of north western america, then our own wild north... well, middle
<mgz> have you yet picked up a cockney accent? can you be mistaken for a local lad?
<jelmer> mgz: sounds like fun :-) Portland?
<jelmer> mgz: The (true) locals are still incomprehensible to me
<czajkowski> jelmer: it never becomes any easier
<jelmer> czajkowski: luckily I don't interact with a lot of natives ;-)
<czajkowski> heh
<mgz> OOPS-5682d217c06be19eecf72b1b1b0eb6a4
<mgz> ...bot?
<czajkowski> oops :)
<wgrant> mgz: Looks like random long response times
<wgrant> Possible network/machine glitch.
<wgrant> eg. a SELECT * FROM Person WHERE id = foo; took 7s...
<mgz> o_O
<mgz> was a push to a branch after fiddling over sftp, was expecting something more dramatic
<mgz> but maybe just upset the caches in code hosting or something
<wgrant> No, that's just a random appserver latency-from-nowhere timeout
<mgz> joy.
<StevenK> cjwatson: I finally got around to reviewing your MP, if you want to chat about it, wgrant and I are in 210
#launchpad-dev 2013-04-30
<StevenK> wgrant: http://pastebin.ubuntu.com/5620771/
#launchpad-dev 2013-05-01
<StevenK> wgrant: http://pastebin.ubuntu.com/5620956/
<Ursinha> StevenK, https://bugs.launchpad.net/launchpad/+bug/1077257
<_mup_> Bug #1077257: DistroArchSeries.enabled not available over API <api> <lp-soyuz> <trivial> <Launchpad itself:Triaged> <https://launchpad.net/bugs/1077257>
<StevenK> Ursinha: lib/lp/soyuz/stories/webservice/xx-distroarchseries.txt
<cjwatson> Oh, yeah, please fix that :)
<cjwatson> There were some IRC discussions about that - wonder if I can find references
<cjwatson> 2012-11-10.log:00:30 <wgrant> cjwatson: The fixes for both are trivial; just s/architectures/enabled_architectures/
<cjwatson> 2012-11-10.log:00:36 <cjwatson> wgrant: Would it be better to export enabled_architectures on the API rather than export the enabled flag?
<cjwatson> 2012-11-10.log:00:36 <wgrant> cjwatson: Or possibly even to just replace architectures' with enabled_architectures
<cjwatson> from #launchpad-ops
<Ursinha> thanks cjwatson, I was wondering yesterday if exposing the enabled ones right away would be better, that helps :)
<StevenK> wgrant: http://pastebin.ubuntu.com/5623102/
<StevenK> wgrant: http://pastebin.ubuntu.com/5623139/
<StevenK> wgrant: http://pastebin.ubuntu.com/5623571/
<Ursinha> wgrant, https://code.launchpad.net/~ursinha/launchpad/fix-1077257-plus-enabled-architectures/
<Ursinha> wgrant, StevenK, https://code.launchpad.net/~ursinha/launchpad/fix-1077257-plus-enabled-architectures/+merge/161960
<Ursinha> guess I should check my pqm permissions?
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/nine-months-for-eva/+merge/161968
<StevenK> wgrant: http://pastebin.ubuntu.com/5624331/
#launchpad-dev 2013-05-02
<StevenK> Ursinha: http://lpqateam.canonical.com/qa-reports/deployment-stable.html is your friend
<Ursinha> StevenK, great, it's still used :D
#launchpad-dev 2013-05-03
<Ursinha> StevenK, https://code.launchpad.net/~ursinha/launchpad/fix-919679-copy-packages-show-ppa-id/+merge/162415
<Ursinha> StevenK, it needs review again because I pushed a change :)
<Ursinha> StevenK, https://code.launchpad.net/~ursinha/launchpad/fix-919679-copy-packages-show-ppa-id/+merge/162418
<StevenK> Ursinha: Approvinated
<Ursinha> thanks
#launchpad-dev 2013-05-04
<barki> hi
#launchpad-dev 2013-05-05
<barki> bzr launchpad-login
<mwhudson> the armhf ppa build queue looks fun
<maxb> hm. I think people are being a bit dishonest about the 10 builds/week thing. There is no way a daily builds PPA covering multiple series complies
#launchpad-dev 2014-04-28
<haobug> hi there,
<haobug> i am trying the make launchpad to run on my own server. but i didn't found a step by step guide to do so. can any on help me out ?
<wgrant> haobug: https://dev.launchpad.net/Running is just about all there is.
<haobug> wgrant:does bzr --no-plugins line download from remote again. i have already downloaded the source. can i make use of that?
<wgrant> haobug: To avoid redownloading it, run 'mkdir ~/launchpad; bzr init-repo ~/launchpad/lp-branches; bzr branch /PATH/TO/YOUR/LOCAL/BRANCH ~/launchpad/lp-branches/temp', then run ~/launchpad/lp-branches/temp/utilities/rocketfuel-setup and follow the instructions from 'Building' on that page.
<haobug> wgrant: i know know so much about bazzer, it seems 'bzr branch/checkout' are making a copy of the original directory. that looks strange.
<haobug> sorry, i don't know so much about bazzer.
<lifeless> What is bazzer ?
<haobug> lifeless: sorry again, i mean Bazaar. miss-typed.
<wgrant> haobug: That's right. I suggested that so it could pre-seed ~/launchpad/lp-branches with the data you downloaded earlier, to minimise the volume that rocketfuel-setup must download.
<haobug> i have another question about Karma, what mathematics modal are based on the build that? it is the most intention of me to try out the launchpad all in one system.
<haobug> let me make my question clean and short(for my poor English skill): what mathematics modal Karma point are based on?
<cjwatson> As I understand the PostgreSQL docs, if I have "CONSTRAINT livefs__owner__distroseries__name__key UNIQUE (owner, distroseries, name)", I don't also need to do "CREATE INDEX livefs__owner__distroseries__name__idx ON LiveFS (owner, distroseries, name);" - is that right?
<cjwatson> (http://www.postgresql.org/docs/9.1/static/indexes-unique.html)
<wgrant> cjwatson: You shouldn't need to, no.
<wgrant> The alternative is to "CREATE UNIQUE INDEX livefs__owner__distroseries__name__key ON LiveFS (owner, distroseries, name);" instead of using CONSTRAINT in the table definition.
<wgrant> But it makes no difference for a new, empty table.
<cjwatson> OK, thanks.  I'm really not very experienced at all with this.  Does http://paste.ubuntu.com/7351944/ look remotely sensible?
<cjwatson> I don't know whether indices of lots of columns for this sort of thing (updated maybe tens of times per day) are useful
<cjwatson> LiveFS.builds actually hits a couple more columns for ordering, which I don't know whether/how I should include
<wgrant> It totally depends on the sorts of queries you're doing. Indices are pretty much just B-trees, so they have the performance characteristics you'd expect: early index columns should approximate the common query filters, and the query's sort columns must be a suffix of the index's columns.
<wgrant> Do you have some example queries>?
<cjwatson> http://bazaar.launchpad.net/~cjwatson/launchpad/livefs/view/head:/lib/lp/soyuz/model/livefs.py lines 97-146
<wgrant> +-- LiveFS.getMedianBuildDuration
<wgrant> +CREATE INDEX livefsbuild__livefs__das__finished__idx
<wgrant> +    ON LiveFSBuild (livefs, distroarchseries, date_finished);
<wgrant> That's usable, but would be more efficient with a DESC on the date_finished
<cjwatson> Probably right in general, although the current code (which I cloned-and-hacked from sourcepackagerecipebuild) will not make effective use of that
<cjwatson> http://bazaar.launchpad.net/~cjwatson/launchpad/livefs/view/head:/lib/lp/soyuz/model/livefsbuild.py#L248
<wgrant> Ah, indeed, no DAS
<wgrant> oh
<wgrant> doesn't use last_completed_build?
<wgrant> Er
<wgrant> Does SPRB really do that?
<cjwatson> You think it should just use the last one as an estimate?  That doesn't seem very robust in cases where we have builders of different speeds
<cjwatson> It does
<cjwatson> I assume that should be done with a postgres expression ...
<wgrant> I don't even
<wgrant> It shouldn't be done at all
<wgrant> That's an unbounded set.
<wgrant> At least take the last ten or something.
<wgrant> Doing it in postgres (or just pulling back date_finished and date_started, rather than instantiating entire objects) would be more efficient, but it's still pointlessly inefficient.
<wgrant> That first build 5 years ago isn't very interesting.
<wgrant> The median of the last dozen might be.
<cjwatson> mm, right.  can I slice in the usual way after doing an order_by?
<wgrant> Yeah
<wgrant> I'd store.find((LiveFSBuild.date_finished, LiveFSBuild.date_started), blah blah blah).order_by(Desc(LiveFSBuild.date_finished))[:10] or so
<cjwatson> Right, makes sense
<wgrant> Just pull back ten sets of two datetimes, rather than instantiating 3462 full SourcePackageRecipeBuilds just to pull those two fields out.
<wgrant> Oh that's right, I was meant to be looking for sane indices, not crying over bad existing code.
<wgrant> last_completed_build (if it should exist in the first place) wants (livefs, date_finished DESC). builds wants (livefs, GREATEST(date_started, date_finished) DESC, date_created DESC, id DESC). The requestBuild one is fine.
<cjwatson> I was thinking of using last_completed_build from cdimage, but maybe that's wrong and it should be looking up a particular build
<wgrant> getMedianBuildDuration post-rewrite wants (livefs, distroarchseries, date_finished DESC) WHERE date_finished IS NOT NULL
<wgrant> last_completed_build doesn't know about DASes.
<wgrant> So I don't think it's useful for cdimage.
<cjwatson> A reasonable point
<wgrant> I try.
<cjwatson> Why wouldn't builds want archive in there too?
<cjwatson> (Oh, in fact I think I came up with last_completed_build before it occurred to me that it might be clever to add version to LiveFSBuild ...)
<wgrant> Because we're not filtering to a small subset of the archives, so adding the column to the index would serve mostly to make it useless for the sort. If I have an index on foo(bar, id) and do SELECT * FROM foo WHERE bar = 1 ORDER BY id LIMIT 10, the DB can just seek to bar in the index and read the first ten entries.
<wgrant> If I have foo(bar, baz, id) and do SELECT * FROM foo WHERE bar = 1 AND baz != 2 ORDER BY id LIMIT 10, the DB has to seek to bar, then read through the index for every value of baz that isn't 2, compile a list of all of the rows, and then sort in memory to find the latest ten.
<cjwatson> We are filtering to a small subset, namely one ...
<wgrant>             LiveFSBuild.archive_id == Archive.id,
<wgrant>             Archive._enabled == True,
<wgrant> That's more than one.
<cjwatson> Oh, er, idiot
<cjwatson> Yes
<cjwatson> OK, that makes sense then
<wgrant> Though you probably do want a method that filters by archive.
<wgrant> In that case, you'd always want one with (livefs, archive, SORT) instead of just (livefs, SORT)
<cjwatson> I'm not sure where such a method would actually be exposed.
<wgrant> But for the general case it's far more efficient to read the first maybe 20 rows until you find ten that have valid archives, rather than having to read every row for the interesting archives.
<wgrant> A good question.
<wgrant> LiveFS.getBuilds, probably. I'm not sure the builds property is actually valuable.
<wgrant> getBuilds can take an optional archive, das, pocket, etc.
<cjwatson> It's an API convenience, I think.
<cjwatson> cdimage will probably just be looking at the one it gets back from requestBuild.
<cjwatson> I haven't quite decided what LiveFS:+builds should look like yet
<cjwatson> It could perhaps reasonably display .builds[:10] or so
<cjwatson> Or just batchnav the whole of .builds
<cjwatson> We'll certainly want to be able to go back and look at old logs, but I don't know that the effort of adding fancy filtering forms and the like would be worthwhile
<cjwatson> Maybe by status and architecture, like DS:+builds, I guess
<cjwatson> I guess I can fill in indices needed for browser code in a -1 DB patch
<wgrant> Right, and we can apply indices live
<wgrant> So you could do them in 40 separate patches and it wouldn't be a major stab-worthy inconvenience.
<cjwatson> OK, so I'll do the file URLs API method, nuke last_completed_build, do the reasonably clear parts of the indices above, and then I think I should have something worth reviewing.  I plan to defer the garbo job to a subsequent branch since it isn't actually vital until we start doing a non-trivial number of builds.
<wgrant> Yep
<wgrant> And this branch is already quite large enough
<cjwatson> Indeed
<cjwatson> Hm, I wonder if I need to do some archiveuploader work here, since there's no .changes file
<wgrant> cjwatson: Not archiveuploader, but something similar, yeah.
<wgrant> This isn't a package upload, so there's no reason to involve archiveuploader.
<wgrant> But we need to do the librarian upload asynchronously (as we should with TTBJs, but we sorta get away with it now because they're so rare).
<cjwatson> Doesn't it need to go through process-upload --builds?
<cjwatson> Which is basically a shim over archiveuploader ...
<wgrant> Why does it need to do that?
<wgrant> process-upload is used to process package uploads.
<cjwatson> I thought that was responsible for pulling everything off the buildds
<cjwatson> Or rather dealing with all the stuff the buildds have pushed to builddmaster
<wgrant> It's up to the BFJB
<wgrant> The base implementation of BFJB._handleStatus_OK (which really should be in some other mixin) pulls the files off the buildd into a temporary directory, performs some sanity checks, then throws them into incoming/ for process-upload to handle.
<wgrant> It then sets the build status to UPLOADING
<wgrant> process-upload eventually runs, processes the upload, and sets the status to FULLYBUILT
<wgrant> TTBJs skip UPLOADING, and just go from BUILDING -> FULLYBUILT inside the BFJB
<wgrant> We can't really do that for LFSBs, because the librarian upload will take A Whileâ¢.
<wgrant> So we need some other async job.
<wgrant> But there's no reason to shoehorn that into archiveuploader, unless you really like changes files.
<cjwatson> Right, which means I need something very like process-upload except that it processes all the files in the upload rather than needing a .changes file
<wgrant> It's similar to the very top level of process-upload, indeed.
<wgrant> But it has basically none of the checks.
<cjwatson> It seemed that it might make more sense to make the .changes dependency in process-upload be dependent on the BFJB
<cjwatson> But maybe that's ridiculously hard
<wgrant> It just runs listdir to find pending uploads, then for each it: finds the build from the dir name, listdirs the upload dir, uploads the files to the librarian, and sets the build status.
<wgrant> You could possibly abstract process-upload to only invoke archiveuploader in some circumstances, but that seems like a lot of refactoring for little gain.
<cjwatson> It would be nice not to have to basically reimplement BFJB._handleStatus_OK except for the incoming dir name
<cjwatson> I guess I could make archiveuploader ignore livefs jobs
<wgrant> It also wouldn't be fatally hard to refactor _handleStatus_OK slightly to let you customise that easily.
<cjwatson> True
<wgrant> Note that this will change a bit once we have the 1SS buildd-master satellite. BFJBs will be thoroughly neutered, and all expensive work must be deferred to a later asynchronous cronjob. At that point I suspect that _handleStatus_OK will become totally generic and just dump files into a directory, and there'll be something like process-upload running over a directory of all types of build results. But I'm not sure it's worth you doing that ...
<wgrant> ... restructuring as part of the already complicated livefs work.
<wgrant> I'd use a separate cronjob on alphecca, probably using some refactored bits of process-upload's build integration.
<wgrant> The only bit of archiveuploader that is relevant at all is the build handling bit of process-upload.
<cjwatson> OK
<cjwatson> Hopefully my to-do list will shorten at some point :)
<wgrant> Indeed
#launchpad-dev 2014-04-29
<haobug> https://dev.launchpad.net/Running here tells to dist-upgrade, what the hell.
<haobug> after lp-branches/temp/utilities/rocketfuel-setup, i got there: http://bpaste.net/show/IcMIAb7WlsuTtuyjG3M1/
<cjwatson> That seems reasonable.  It's the simplest way to make sure you're current with the PPA you just added.
<cjwatson> That looks like update-sourcecode hasn't been run.  Try utilities/rocketfuel-get.
<haobug> cjwatson: i got the idea. it downlaod thing to ~/launchpad, not the working directory of running it.
<cjwatson> Right
<haobug> cjwatson: it seems hard-coded...
<haobug> cjwatson: i got another problem on start the web-server. it keep report "Adress already in use", even if i stop the apache.
<cjwatson> haobug: I think you need somebody a bit more experienced than I am, sorry
<cjwatson> I trust you're doing all this in an LXC instance
<haobug> er... | cjwatson
<cjwatson> It isn't worth the pain of trying to do it in other environments, IME
<haobug> LXC instance? what is that ?
<cjwatson> linked from the top of the page you're following
<cjwatson> https://dev.launchpad.net/Running/LXC
<haobug> oh sorry/
<cjwatson> Using an LXC container means that you don't get things like server port clashes with other things on your system.
<haobug> i didn't configured the LXC. i left out the `dist-upgrade` line from the guide.
<haobug> Downloading ubuntu precise minimal ...
<haobug> I: Retrieving Release  <-- it says, i noticed it installed qemu, does that means i am going to install another ubuntu within my current one?
<haobug> cjwatson: can i use the offline iso image to speed up this process?
<cjwatson> No idea
<cjwatson> And yes, an LXC container involves a nested installation
<cjwatson> It's manageable even on my terrible internet connection so it shouldn't take all that long.
<haobug> cjwatson: i am in a intranet with about 50K band-limitation...
<cjwatson> Gosh, even worse than mine.
<haobug> cjwatson: it's a bit annoying to install another ubuntu on top an already installed one.
<cjwatson> LXC containers are fantastically convenient once you have it set up.  Perhaps persuade your intranet administrator to give you a local mirror
<cjwatson> It's probably possible to mangle /usr/share/lxc/templates/lxc-ubuntu somehow to get packages from an ISO but it might still take more effort than just setting it off and doing something else while it completes
<cjwatson> Assuming by 50K you mean 50Kbytes, my network connection is only about five times faster than yours, typically
<cjwatson> So not like orders of magnitude better or anything
<haobug> cjwatson: ..... are there another way or docs i can read through and figure out a way myself to solve the 'already in use' problem?
<cjwatson> I really can't recommend it.
<cjwatson> I tried to avoid the LXC setup for a while for similar reasons, and it was too much effort to be worth it.
<haobug> wait a second, you are in this channel, you must be someone develop on it or just use it, okay i got that, you installed the LXC...
<cjwatson> I'm a Launchpad contributor, yes
<cjwatson> Even on an awful connection, in anything more than the extremely short term it's worth it to accept the one-time cost of setting up a container and save painful debugging effort later
<haobug> then should should be familiar with some 'internal' or hacks, that you can tell me to fix this ;)
<cjwatson> It might be possible to set MIRROR=file:///path/to/unpacked/iso in the environment when creating the container
<cjwatson> I'm only familiar with what I've needed to do myself, which I've explained to you
<haobug> cjwatson: can i mount the iso directly ?
<cjwatson> Yes
<cjwatson> Even if that works for creating the container, though, you'll still need to set /etc/apt/sources.list back to a proper mirror later; the ISO won't have enough packages to actually run Launchpad
<haobug> cjwatson: oh shit, my iso is amd64.
<cjwatson> Is that a problem?
<haobug> is it ok to change the i386 to amd64?
<cjwatson> Gosh, why does that say i386
<wgrant> RAM usage
<wgrant> But if you're only running one instance it's not usually a problem.
<lifeless> LXC ftw :)
<cjwatson> Oh, I just changed it to amd64 ...
<cjwatson> Revert it if you think that was wrong :)
<cjwatson> It's not like we test Launchpad on i386.
<wgrant> All my non-buildd installations are still i386 for that reason, actually. Though my desktop has 24GiB of RAM now so that's possibly less necessary than it once was.
<cjwatson> Do you think I should change the docs back and add an explanatory note, then?
<wgrant> Meh.
<wgrant> Most people aren't me, so they won't run several instances simultaneously, so it probably doesn't matter either way.
<haobug> wow! the mounted iso works: http://bpaste.net/show/NASyuEoXKIH4iSrGwqHk/
<cjwatson> Eh, might as well I guess, I've added a note
<cjwatson> Since otherwise I'll probably forget again later
<cjwatson> Hah, turns out my container is i386 too, I'd entirely forgotten that
<haobug> wgrant: does the i386 matters? i am using amd64; cjwatson: are you the one maintains that page?
<wgrant> haobug: The page was basically written by the two of us, I think.
<wgrant> The architecture doesn't matter for functionality.
<wgrant> But i386 uses less RAM
<cjwatson> I think I tweaked it a bit but can't claim to have written it.
<haobug> wgrant: okay that should be a question to right person ha?
<cjwatson> Well, you have an answer now anyway
<wgrant> Oh yeah, I thought you made more substantial changes than you actually did, it seems.
<haobug> yes. i am perfer to use i386, i am now using amd64 because i got an iso of that from local network in my intranet.
<haobug> i have asked question about the Karma mathematics modal, can you give me some explanation?
<wgrant> haobug: There's not much to it. You can see the algorithm in cronscripts/foaf-update-karma-cache.py
<wgrant> Each karma-granting action has a value assigned. An action's value decays linearly to 0 over the year after it was performed.
<wgrant> Karma is categorised by app: bugs, code, translations, etc. The total value of the karma actions in each category is scaled to be roughly equal. So if there are 1000 points of actions in the last year in Code, but 10000 in Bugs, Code points are worth 10x as much as Bugs.
<wgrant> The karma value displayed on each user's page is just the sum of the scaled, decayed actions.
<haobug> the guy ask me to setup the launchpad told me the launchpad has the measurement system all of aspects of contribution. some LUG members and me discussed about a good way for measuring contribution of members. then i run into this :D.
<cjwatson> I hope you aren't planning to actually run LP for real on a system with 50KB network access; that doesn't sound like fun at all
<haobug> cjwatson: the internat locally are connected at speed  about 100MB(1000Mb? i am not sure). i can download the internal file at speed of 90MB+.
<haobug> "scaled to be roughly equal", i don't get that all. 1000 points in code, and the 10000 points in bugs are all contributed by one or the whole system sum up?
<wgrant> haobug: System-wide.
<haobug> oh yeah, that's a currency system as we discussed at LUG meeting.
<haobug> wgrant: the points drops as time goes, because there are new contributions included. that's a clever idea.
<wgrant> haobug: Each action's value decays to 0 over a year because karma is designed to show recent activity. Someone who hasn't participated in the project for a year will have no karma, so karma can to some extent be used to judge how active someone has been lately.
<haobug> wgrant: what does that mean, i can't read python by now, the karma can drop without new contribution, just because time goes?
<wgrant> haobug: Right. If all I've done in Launchpad is file a single bug, that might be worth 10 points. So I'll have 10 karma immediately after filing that bug. Six months later, I'll have 5 karma. Another six months after that, I'll have 0 karma. Each action only has value for a year.
<haobug> wgrant: that sound strange. with the contributions grow as time goes, the point of each individual should drop. why do you introduce such decaying?
<cjwatson> If contributions actually grow with time, then they will gain new karma.
<cjwatson> But five-year-old contributions aren't as interesting as recent ones from the point of view of getting a basic idea of who's currently most active.
<haobug> wgrant: the whole worth of the launchpad system is dropping too, if the contribution activity is steady.
<cjwatson> Karma isn't intended to be a total measure of all contributions ever.  It's meant to be a rough indicator of who's active at the moment.
<haobug> oh..... i almost got the idea. it is not a currency system as i thought minutes ago.
<cjwatson> No; or if it is, it's a highly deflationary one.
<cjwatson> But you can't exchange karma for anything so it's not currency by any sensible understanding of the word.
<haobug> the whole worth of the system drops, how can that be a currency system?
<haobug> >you can't exchange karma for anything
<haobug> we are planning to make it can do exchange with the reality world in our LUG, with some sponsors.  but it is just a plan in mind by now.
<cjwatson> Deflationary currencies are still currency, even if they're generally pretty awful ones.  But not being able to exchange it for anything definitely makes it not currency.
<cjwatson> Karma wasn't designed as a currency, so you'll need to make design-level changes if you want to try to use it as one.
<haobug> cjwatson: er... is that changes big?
<cjwatson> I don't know
<cjwatson> That would be your job to determine if you plan to do this :-)
<cjwatson> Since it kind of depends on what changes you want
<haobug> i don't know exactly. but i think the total worth of the system should not drop.
<haobug> do you know some other alternatives system like launchpad manage all of this and has a ready to use currency system?
<cjwatson> I don't; it's not one of my interests.
<haobug> So do I...
<SpamapS> Hi! I'm making login.launchpad.net OOPS via OpenID .. wondering if anybody is up for trying ot fix it?
<SpamapS> OOPS-a74e777c522c4ea69b4f537c946d667a
<SpamapS> caused by
<SpamapS> AUthOpenIDAXRequire email http://axschema.org/contact/email .
<SpamapS> AuthOpenIDAXUsername email
<SpamapS> hm
<cjwatson> SpamapS: That's not an OOPS in Launchpad proper; you want SSO support
<cjwatson> (Can't remember if there's an appropriate IRC channel, sorry; the support link just leads me to https://forms.canonical.com/sso-support/)
<SpamapS> cjwatson: Ah right.
<lifeless> cjwatson: SpamapS: there was #canonical-isd at one point
<cjwatson> That could be it, thanks, memory didn't serve
<SpamapS> I sent an email
<SpamapS> got an rt response
<SpamapS> I'll set the effort aside to see what they say
#launchpad-dev 2014-04-30
<cjwatson> wgrant: Do we have any editor widgets that might be vaguely suitable for editing LiveFS.metadata?
<cjwatson> I can always just make people enter JSON directly, but I figured that if there was something already available for structured key/value pairs then I should use that
<wgrant> cjwatson: Nothing really like that. The closest thing we have is workitems, and that just parses text
<wgrant> Same with recipes
<cjwatson> OK, I guess the general approach of recipes isn't especially terrible given how rarely it'll be used
<wgrant> Yeah, will anyone ever actually seriously use it?
<wgrant> Doesn't seem critical to have good usability there :)
<cjwatson> I basically just want something minimal that won't explode.  As I say, if something had existed then I'd've used it, but I'm certainly not going to go writing new widgets for it.
<wgrant> cjwatson: I'll try to get to your code branch some time today, but it's going to require a bit of time blocked off.
<wgrant> Yep
<wgrant> That sounds fine.
<cjwatson> Yeah, no problem, I understand it's chunky
<cjwatson> Thanks
<cjwatson> I'm just trying to get through as much of the remaining bits as I can before something turns up to steal my attention
<wgrant> Yeah, good idea
#launchpad-dev 2014-05-01
<cjwatson> Zope: a technology for generating enormous tracebacks
<StevenK> cjwatson: TAL seems quite good at that, as well.
<cjwatson> Yup
<cjwatson> wgrant: Given something that currently looks like http://people.canonical.com/~cjwatson/tmp/lp-livefs.png, is there a reasonable way to turn the distroseries name there into a link to the distroseries with a +edit icon/link beside it?  I've had great difficulty finding an example of anything in LP with an editable distroseries that I might be able to use as an example.
<cjwatson> I've got http://paste.ubuntu.com/7374012/ so far
<wgrant> cjwatson: Hm, what's the problem with just a distroseries/fmt:link next to a normal menublahblah/fmt:icon?
<cjwatson> wgrant: Ah, thanks, that looks plausible.  I was just lost in a maze of complicated but promising-looking widget implementations.
#launchpad-dev 2014-05-04
<haobug> hi hi, got problem again. http://pastebin.com/XxaqxmbJ after about 3 days of waiting...
<cjwatson> haobug: You don't need to redo it all from scratch, at least, but there's some manual work to do to recover from a failure there without re-downloading - it should be possible to look at /usr/share/lxc/templates/lxc-ubuntu and search for "Installing updates" to find where it got to, but you'll have to trace the function call sequence up to there for yourself
<cjwatson> http://people.canonical.com/~cjwatson/tmp/lp-livefs-built.png   progress!
<cjwatson> five-ish to-do items remaining, but pretty close.
<cjwatson> unfortunately will need another lp-buildd deployment.
<cjwatson> wgrant: Perhaps we could try to squeeze a launchpad-buildd update in with RT#69868, since the risk of buildd upgrades seems to be more strongly correlated with how many of them we do rather than how much they contain?  I'd like to include https://code.launchpad.net/~cjwatson/launchpad-buildd/abort-race/+merge/218222 if possible, as well as the fairly obvious stuff that's in trunk.
<_mup_> Bug #69868: Contrast setting is bad <gxine (Ubuntu):Invalid> <https://launchpad.net/bugs/69868>
<cjwatson> (I haven't yet done a QA pass on the whole lot.)
<cjwatson> I've been meaning to do that abort-race stuff for months, but doko keeps running into it so I thought I'd better make an effort to sort it out ...
<wgrant> cjwatson: Looking
<wgrant> cjwatson: Is that screenshot of a full test run with a local buildd? Nice.
<cjwatson> It is indeed.
<cjwatson> Not that that took ages to set up or anything.
<cjwatson> Remaining to-dos: some bits of view need to be disabled if the feature flag is off; latest builds table for +livefs; UI for requesting a build; a way to delete a LiveFS (much like for recipes)
<wgrant> I should probably rewrite the docs, as they don't really work well when not running LP in a heavyweight VM.
<cjwatson> I'm afraid I ignored the docs as I was fairly sure I knew which bits I needed
<wgrant> Do we need UI for requesting a build?
<wgrant> Deletion is probably easy, as the LiveFS can just take all the LiveFSBuilds and their files with it.
<cjwatson> I expect to eventually get to the point where some builds don't require cdimage, at which point a UI would be useful
<cjwatson> deletion - I was expecting to do what we do with recipes, i.e. leave the builds intact but disconnected
<wgrant> Why?
<wgrant> It's basically impossible to find a build if you don't know the livefs.
<cjwatson> just seems odd to delete builds that happened
<wgrant> We avoid deleting SPRBs because that would leave SPRs without a signature and with no way to trace the owner.
<cjwatson> well, ok, I guess I can do either way
<cjwatson> my local setup was basically sampledata, initseries trusty, run the initialize job, feed in the production chroot, install launchpad-buildd listening on a port not one of those already in sampledata, add that builder
<wgrant> What value does a LiveFSBuild with no LiveFS provide? How would you use it, or even find it (except by looking through all the builds for an archive or builder)?
<wgrant> Hm, you were able to get it working in a container?
<cjwatson> wouldn't be any good for normal package builds, but it worked for this
<cjwatson> I had to make the container allow arbitrary mounts, with some help from hallyn
<wgrant> Ah, right.
<cjwatson> almost exploded my system on the first attempt since it enabled qemu-x86_64 in my host :-)
<wgrant> Heh
<wgrant> I normally just run a KVM instance which lives on lxcbr0.
<cjwatson> LiveFSBuilds> I was thinking builder history views and the like, but you're probably right.  I didn't know there was a specific reason for SPRBs.
<cjwatson> The use of old LiveFSBuilds is their logs.
<wgrant> Right, but if you want the logs you probably shouldn't be deleting the thing that lets you find them :)
<cjwatson> Perhaps I should have a way to disable a LiveFS
<cjwatson> Not deleted, but it wouldn't allow new builds any more
<cjwatson> Basically to avoid stupid mistakes
<cjwatson> Given that I'm not even sure we particularly need deletion
<wgrant> We need to be able to delete builds individually, and also the LiveFS and its builds as a whole unless there's a good reason not to
<cjwatson> What's the point of deleting individual builds?
<cjwatson> I don't think any other build types are individually deleteable, are they?
<wgrant> There doesn't need to be UI or API for it
<wgrant> but I'm working on the other types as we speak.
<cjwatson> Just the principle that all objects should be deleteable?
<wgrant> Though those are just being neutered, rather than deleted, since they produce artifacts that are referenced elsewhere.
<wgrant> Mostly for proper PPA deletion
<cjwatson> A LiveFSBuild associated with a PPA *could* just be neutered, since it doesn't actually build something *in* the PPA as such, it just uses it as a source; but there's probably not much use for such a build
<wgrant> Right
<wgrant> We delete where we can, neuter where we can't.
<wgrant> Keeping useless data around isn't something we should be doing.
<cjwatson> But I thought the database was too small
<cjwatson> I guess we do erase evidence of builds that happened in some other situations albeit in different ways, e.g. cancel/retry
<cjwatson> Anyway, thanks.  Bed then bank holiday, I think
<wgrant> Right. It's theoretically nice to have a full history of everything, but in practice it's somewhat onerous and not terribly useful.
<wgrant> It's rare that I need to look way back in a builder's history.
<wgrant> And for short-term stuff I mostly look at logs.
#launchpad-dev 2015-04-27
<wgrant> cjwatson: the pack protocol we use internally is extensible with a dict of strings
<wgrant> like git's normal host field, but not restricted to host
<wgrant> cjwatson: Morning.
<cjwatson> wgrant: Hi.  How's Malta?
<wgrant> cjwatson: Cooler than last year, but sprint going well so far.
<wgrant> cjwatson: I can't push to a locally mojo'd turnip today.
<wgrant> turnipserver.py still works on my host, so I wonder if the split is buggy.
<wgrant> I haven't debugged too fa yet.
<cjwatson> wgrant: Pushing to qas still works, and qas has the split ...
<cjwatson> (e.g. https://code.qastaging.launchpad.net/~cjwatson/binfmt-support/+git/binfmt-support which I just pushed)
<wgrant> Hm
<wgrant> cjwatson: I have a subordinate alternate branch proposed, and apparmor stuff working for git subproesses but won't propose that until I have turnip itself confined.
<cjwatson> wgrant: You definitely have all the listeners up?
<wgrant> cjwatson: Yeah, it looks like git receive-pack --stateless-rpc just immediately dies for some reason.
<wgrant> I can see it starting, and processEnded is called, but outReceived and errReceived are not :/
<wgrant> I'll poke harder.
<cjwatson> That's a peculiar failure mode for the stuff I was doing, unless I missed some detail in which the TACs differ from turnipserver.py.
<wgrant> Indeed, but I think you caught it all.
<cjwatson> The only one of those I noticed was that the backend server hadn't been updated to run hookrpc, but I fixed that.
<cjwatson> Hopefully the logging is less hopelessly inconvenient to play with locally now :-)
<cjwatson> Like, timestamps.
<cjwatson> I stuffed a draft production stage into our mojo spec branch too, if you want to have a look.  Mostly the same as qas.
<cjwatson> wgrant: alternates: I wonder if we need to force receive.autogc to false before doing this
<cjwatson> wgrant: Otherwise branch removal + next push might corrupt cloned-from repositories.
<cjwatson> (And we'll have to be very careful on repack, but we knew that.)
<wgrant> cjwatson: How can it corrupt cloned-from?
<cjwatson> wgrant: Sorry, I mean corrupt cloned.
<cjwatson> I think my comment in the MP was clearer.
<cjwatson> wgrant: I've been trying to reproduce the behaviour from http://paste.ubuntu.com/10909945/ in the test suite and can't (if I .encode the URL I pass to test_traverse, something in the traversal machinery decodes it again for me.  Is there anything specific to the test suite you know about that would cause that?
<wgrant> But we're cloning the cloned-from, so even deletion of it won't affect anything.
<cjwatson> Oh
<wgrant> cjwatson: A GC in the cloned-from can't touch the packs that we've hardlinked into the alternate.
<cjwatson> Yeah, OK, I'm confused then.
<wgrant> That's the entire reason we create the separate repo.
<cjwatson> Disabling autogc might be good for other reasons, but you're right.
<wgrant> Otherwise, yes, disaster would strike.
<wgrant> Indeed.
<cjwatson> wgrant: Have you had a chance to look at the couple of pending LP git branches I have?
<cjwatson> https://code.launchpad.net/~cjwatson/launchpad/git-mp-collection/+merge/257145, https://code.launchpad.net/~cjwatson/launchpad/git-mp-ref-proposals/+merge/257374
<wgrant> cjwatson: Looking.
<cjwatson> Ah.  So BasicLaunchpadRequest.__init__ decodes the PATH_INFO.  I wonder why that isn't done for API requests as well?
<cjwatson> Oh, except it doesn't, it decodes and re-encodes.  Must be somewhere else.
<wgrant> Isn't that all done in Zope?
<wgrant> Are you seeing it as a bytestring somewhere?
<cjwatson> zope.publisher.http:sane_environment is the bunny.
<cjwatson> Yes, in an API requesthttp://paste.ubuntu.com/10909945/
<cjwatson> er http://paste.ubuntu.com/10909945/
<cjwatson> I wonder if lazr.restful should be decoding the path to match.
<wgrant> :/
<cjwatson> How have we not come across this before?  Are there just lots of little model hacks to decode?
<cjwatson> I mean I know it's only pure-Storm stuff that's likely to care, but.
<wgrant> It's very odd.
<cjwatson> I had to handle this in GitTraverser, which is probably why.
<cjwatson> Never tracked it down properly at the time.
<cjwatson> I wonder if this is the else case of SimpleFieldMarshaller.marshall_from_request.
<cjwatson> If it's posting a non-JSON-encoded bytestring there, that case doesn't decode it.
<wgrant> Ah, yes.
<cjwatson> Will have a go at that after lunch.
<cjwatson> Ah, no, the problem is that lazr.uri returns bytestrings in URI.path (arguably reasonably) and we need to decode those in URLDereferencingMixin.dereference_url.
<cjwatson> Aha.  I think lazr.uri 1.0.3 is the thing.
<cjwatson> Far be it from us to have upgraded to our own software released three years ago.
#launchpad-dev 2015-04-28
<wgrant> blr: Evening.
<blr> wgrant: morning!
<wgrant> blr: I saw that your default clone_from thing landed, testing it now.
<wgrant> Did you make any progress on the init with alternates?
<blr> wgrant: merged your recent changes into the alternates branch as well now, which is much better - only change in the view is docstrings for repo:repo
<wgrant> Great.
<blr> the only thing I'm a little uncertain about is that there is now obvious place to cleanup the ephemeral repo - I've added a repo_cleanup() method to both diff functions, but it might be better to have a sweeper do this later
<blr> s/now/no/
<wgrant> I think that using a context manager makes sense.
<wgrant> eg. "with open_repo(blahblah) as repo:" would ensure that it's always cleaned up at the end.
<wgrant> The autocloning stuff works well, with one unforeseen issue.
<wgrant> The connectivity check on push uses rev-list --not --all, but --all only looks in the local ref.
<wgrant> s
<wgrant> So my first push to ~wgrant/ubuntu/+source/linux took a bit over a minute, as it ended up walking the entire tree when it had no known-good refs.
<wgrant> It may be worth patching git to let rev-list consider alternate refs as well.
<wgrant> But it's just a bit slow on first push, not a serious problem.
<wgrant> Particularly not for sensibly sized repos.
<wgrant> blr: What d oyou think of the context manager approach to ensure the ephemeral repo is always cleaned up?
<blr> wgrant: yep that sounds sensible, I'll have another pass
<wgrant> Thanks.
<wgrant> Let me know how it goes.
<blr> wgrant: open_repo() is a contextmanager now, that should be ready for you
<wgrant> blr: Let me see.
<wgrant> lifeless: I think you can do away with repo.ephemeral and cleanup_repo now, since you only call cleanup_repo in the ephemeral case.
<wgrant> Er
<wgrant> blr: ^^
<wgrant> This ssh latency is really awful for typing.
<blr> wgrant: quite, that's a fairly useless function now.
<blr> ok pushed
<wgrant> blr: Couple of other trivial comments, then I think it's all good.
<blr> wgrant: hmm actually, in get_commit repo=None is used to prevent us re-initialising repositories in get_commits
<wgrant> Ah, that's right.
<blr> not certain how to perserve that nicely in the context of the context manager
<wgrant> blr: Maybe get_commits and get_commit should call a private method which just takes a repo that must already be open.
<blr> that sounds reasonable, I'll sort that out in the morning, getting late
<blr> thanks wgrant
<wgrant> Night.
<cjwatson> wgrant: https://code.launchpad.net/~cjwatson/launchpad/git-fix-clone-from-owner-default/+merge/257622 if you get a minute, little bit of cleanup
<cjwatson> wgrant: Creating MPs on the webservice now works, though https://code.qastaging.launchpad.net/~cjwatson/germinate/+git/germinate/+ref/exp/+merge/254302 will OOPS in the web UI for now.
<wgrant> cjwatson: Nice, how's the testing of the remaining KLOC going?
<cjwatson> wgrant: Making progress, just thinking about how to mangle TestRegisterBranchMergeProposalView suitably.
<cjwatson> wgrant: Have you heard anything from IS?
<wgrant> cjwatson: We're yellow.
<cjwatson> Ah yes, thanks for the score bump.
<cjwatson> wgrant: Is there any equivalent of SQLObjectVocabularyBase for Storm?
<wgrant> cjwatson: I don't think so.
<cjwatson> I wonder if I should write one.  I just noticed that GitRepositoryVocabulary is wrong.
<wgrant> It would not be a huge task.
<cjwatson> https://code.launchpad.net/~cjwatson/launchpad/storm-vocabulary-base/+merge/257655 https://code.launchpad.net/~cjwatson/launchpad/git-mp-basic-browser/+merge/257674 https://code.launchpad.net/~cjwatson/launchpad/git-mp-register/+merge/257676 are all the MP stuff I have so far, except for my in-progress stuff on preview diffs.
<cjwatson> Should be (barely) enough to go live with.
<xnox> cjwatson: did you see recent discussions about git / merge proposal workflows on debian-devel et.al.?
<xnox> cjwatson: it would be cool if launchpad would support emailing in git-bundle(1) to create mp just like one can with bzr bundles
<cjwatson> xnox: I noticed the thread, but we have a fair bit still to do before getting to that kind of point
<xnox> cjwatson: sure. plus i believe git-bundle has insufficient information as usual in the git world. E.g. bzr bundle encodes public/parent/submit branch locations, and git bundle does not as far as i remember....
<wgrant> bzr bundle slso hasn't worked since 2009
<xnox> wgrant: the email interface or the bundles themself?!
<cjwatson> Yes, I wondered why I couldn't find any of this alleged-to-exist code
<xnox> i sure used bundles as recent as 2013-2014 e.g. generate it somewhere throw into usb-stick or email and pull from it on the other end....
<wgrant> bundles never worked with 2a
<wgrant> so the LP side was removed
<xnox> wgrant: just tested works here.... created --unchanged commit, branched, created --unchanged commit, bzr bundle ../first-branch, then branch first branch again and pulled from the bundle
<xnox> got matching revision-ids out of it....
 * xnox is sad =(
<wgrant> ah yes, the unchanged test :P
<wgrant> that crestes little new 2a data
<xnox> wgrant: well i did merges and the like... and it didn't fail me.
<xnox> althoguht plenty of things are broken with ghost revisions
<wgrant> hmmm
<lifeless> wgrant: I can? thanks
<wgrant> lifeless: 500ms sucks
<lifeless> aye, where art thou?
<wgrant> Malta
<lifeless> nice
<wgrant> after Austin and London
<lifeless> I don't miss those trips
<wgrant> it's certainly a new experimebt in blood caffeine content
<blr> this looks like it might be useful https://github.com/libgit2/pygit2/pull/525
<cjwatson> blr: I thought about backporting it, but reckoned I could probably just reuse the stuff in LP (actually bzrlib I think) for now and no rush to backport.  But I might change my mind if that turns out to be non-trivial :)
<cjwatson> (That is, it looked like it'd be less work to reuse)
<blr> right
<cjwatson> We'll see - I'm going to try to finish the LP side of preview diffs tomorrow
<lifeless> wgrant: whats em1.rapid.c.c ?
<lifeless> cjwatson: ^
<lifeless> thomi: ^
<lifeless> blr: ^
<lifeless> any two-letter nicks around ?
<cjwatson> lifeless: replacement for batuan, if you remember that
<lifeless> cjwatson: no; I've just got hold of the ops VG so we're good
<lifeless> cjwatson: thanks
<cjwatson> VPN endpoint for misc crap
<wgrant> lifeless: what was it spamming you with?
<lifeless> wgrant: git requests. see #openstack-infra. blahdeblah is on it
<thomi> hmmm?
 * thomi makes whooshing motion above his head
<wgrant> hmm
<lifeless> thomi: outbound NAT gateway for OIL it appears
<cjwatson> not totally implausible
<thomi> I know what some of those words mean, so yay!
<wgrant> for moat of the weird non-IS-managee labs
<thomi> wgrant: how's Malta? I'm surprised you're awake
<wgrant> awake is pushing it
<cjwatson> there's still a load of CI jenkins stuff behind rapid isn't there?
<wgrant> the CI lab, all the ex-CTD stuff, servenab, hwcert
<lifeless> contra-terrene devices?
<wgrant> CTS, that is
<cjwatson> right.  not LP anyway :)
<lifeless> no
<lifeless> I was pinging clued up people in the most convenient channel :>
<cjwatson> heh
<wgrant> batuan has ll the stuff you don't want to think about
<thomi> lol... and you pinged *me* :D
<lifeless> thomi: penance for using the wrong channel :). Actually its the CI nature of it that led me to ping you
<thomi> ahh
<lifeless> non-CI doesn't need many many many accesses to the same git repos
<wgrant> yep
<thomi> blissfull ignornace is my motto...
<lifeless> thomi: and I hear you've done some CI stuff
<wgrant> anyway, night avain
<lifeless> wgrant: night, thanks. cjwatson ditto
#launchpad-dev 2015-04-29
<lifeless> wgrant: don't you love release bugs ;0
<lifeless> wgrant: I've asked ttx to grab some timing data, it looks like milestone assignment on bugs may be rather slow - we'll try to create a decent bug report
<lifeless> wgrant: he's saying ~2s to dereference the bug, assign a milestone, and call lp_save()
<wgrant> lifeless: On OpenStack projects? Yes.
<wgrant> You have thousands of structsubs.
<lifeless> wgrant: still, should be deferrable work
<wgrant> I am aware.
<lifeless> wgrant: heh
<lifeless> wgrant: ok, well we'll gather the data
<wgrant> It's on the list for this year, as it's becoming quite unpleasant.
<lifeless> since assigning milestones is relatively rare I figure it doesn't show in the % failure rates much
<wgrant> Most other things are performing OK nowadays.
<cjwatson> Re discussion from yesterday, Diff.generateDiffstat works just fine for git diffs; it just needs a small tweak to run in -p1 mode.
<cjwatson> (We could change the diff format from turnip, but I'd rather leave that as close as possible to the default git style.)
<wgrant> Nice.
<wgrant> Yeah, let's notdiverge pointlessly.
<cjwatson> wgrant: And it's just the format_diff stuff in lp.app.browser.stringformatter that should need to be tweaked otherwise, right?
<cjwatson> In which case I'm nearly done with preview diffs, just need to have the alternate diffs stuff in place in turnip and to finish the job changes.
<wgrant> cjwatson: I think so. The rest is just bsed on the classes.
<wgrant> I'll get to your two remaining not-quite-mega-branches soonish.
<cjwatson> https://code.qastaging.launchpad.net/~cjwatson/germinate/+git/germinate/+ref/exp/+merge/254302 renders now.
<cjwatson> Thanks.
<wgrant> Ah excellent.
<wgrant> cjwatson: https://code.launchpad.net/~wgrant/launchpad/do-not-reject-ddebs/+merge/257758
<cjwatson> wgrant: r=me, thanks
<cjwatson> wgrant: That's hopefully all the MP stuff proposed now.  I'm moving on to see if I can sort out at least a little bit of repo listing display.
<cjwatson> wgrant: Should I shove contextlib2 into turnip-dependencies and land Kit's branch?  (Though, that's only worth it if you're going to get all the way up my review stack today, which may be a bit to ask ...)
<wgrant> cjwatson: Sounds reasonable to me.
<cjwatson> wgrant: This is basically awful: http://people.canonical.com/~cjwatson/tmp/git-target-listing.png
<cjwatson> I think the best way to improve that would be to provide a list of branches in all associated repositories sorted by last-modified and privileging the default repository, or something
<cjwatson> OTOH this was quick.  Any suggestions that might be doable short-term?
<cjwatson> Deploying Kit's turnip branch for cross-repo diffs now.
<cjwatson> Had to "rm -r /srv/mojo/mojo-stg-ue-launchpad-git/trusty/qastaging/build/storage-ceph/" so that the new auto-repo stuff stopped being confused
#launchpad-dev 2015-04-30
<cjwatson> wgrant: I'm looking into the reported (non-timeout) bug unsubscribe oops.
<cjwatson> Unless you are already.
<wgrant> cjwatson: Fixed on DF already.
<wgrant> Just writing tests for the untested cod
<wgrant> e
<cjwatson> Heh, OK.
<cjwatson> wgrant: Did you have any thoughts on http://irclogs.ubuntu.com/2015/04/29/%23launchpad-dev.html#t16:48 ?
<wgrant>  "Default" is ew, the rest makes some sense.
<cjwatson> Mm, maybe I can just drop that and the singular is enough.
<wgrant> In an ideal world that would be where the repository selector was, I think.
<wgrant> But "default" should definitely not be the first word on the page!
<cjwatson> repository selector> how do you mean?
<wgrant> Well, we need some way to see the other repositories, probably near the top of the page.
<cjwatson> Right.
<wgrant> Maybe even above the "You can browse the source code"
<cjwatson> Maybe just "Git branches".  It's "branches associated with the default Git repository", but "Git repository branches" just sounds like we're confused.
<wgrant> That would make it clear which repo is selected.
<wgrant> Yeah, "Git branches" makes sense-ish.
<cjwatson> The page is annoyingly spread out over product-branches.pt and product-branch-summary.pt right now, which makes it awkward.
<wgrant> dammit nvidia
<wgrant> number_of_duplicates
<wgrant>     1156
<wgrant> other_users_affected_count_with_dupes
<wgrant>     2865
<wgrant> cjwatson: https://code.launchpad.net/~wgrant/launchpad/git-fixes/+merge/257977
<cjwatson> wgrant: r=me
<wgrant> cjwatson: Thanks.
<cjwatson> I think we can NDT without that, though - I was going to sort that out after this call
<cjwatson> if that's OK
<cjwatson> since at least ASCII-only MPs work
<wgrant> yep
<wgrant> sounds good to me
<cjwatson> wgrant: LPSed/RTed; feel free to amend both if your rev is ready before webops take that
<cjwatson> https://code.qastaging.launchpad.net/~cjwatson/grub/+git/qas/+ref/fstest-test/+merge/254312 was the ASCII-only case I mentioned, BTW
<wgrant> Yep, saw it in the OOPS.
<wgrant> I poked around locally and found all the obvious crashers and fixed them.
<cjwatson> Yep.  Thanks for that.
<cjwatson> I think anything further from our side is gravy at this point.
<cjwatson> Is there anybody at the sprint who might be able to help chase down the nova scheduling thing tomorrow?
<wgrant> If it's not solved overnight, I will harrass someone.
<wgrant> I'm talking to designy people tomorrow morning, will try to get something useful out of them. But I think we just need something like your screenshot, target/owner repo listings...
<wgrant> And GitRef:+activereviews would be nice and might be easy.
<wgrant> All of which are quick to do something roughly as dirty as the bzr equivalent.
<cjwatson> Yeah, there's a huge pile of misc views to clean up.
<cjwatson> Layouts of the worst cases of combining git and bzr listings that also show a useful set of information about git branches would be good.
<cjwatson> In some ways I'd like to ram them all into a single table.
<cjwatson> Partly because multiple similar-content tables on a single page look bad because their column widths don't match.
<cjwatson> I haven't touched the package views yet.
<wgrant> The only place we really need to show both simultaneously is on Person:+branches
<wgrant> Since a target (or an owner-target) has a primary VCS defined.
<cjwatson> The project and package views could get away with having a mode switch, indeed.
<cjwatson> Once we model such a thing.
<wgrant> Person:+branches already has a relationship filter.
<wgrant> (it was several separate pages a few months ago, but it's all one now)
<cjwatson> I'd been focusing on the target partly because I'd like to have a rule that the target page for a target with a default repository always shows at least something about the default repository, so that we can switch the canonical URL over to shortened_path rather than unique_name
<cjwatson> since I really don't like the canonical URL for default project repositories being /~owner/project/+git/project
<wgrant> Yeah, definitely. And that's the only bit that really requires thought.
<wgrant> The others are, well, tables.
<cjwatson> so I think that even if we have a mode switch we always need to show *something* about the non-default mode
<cjwatson> (if there's any content for it)
<wgrant> yep
<cjwatson> we could certainly have a radically different summary, though
<cjwatson> very hard to get the summary right without a mode switch - I tried, it's a mess as you saw
<cjwatson> "you can clone a git repository!  oh and by the way you can push a bzr branch!"
<wgrant> Mhm.
<cjwatson> but perhaps I could go and try Person:+branches for a while, because that involves a lot of the same kind of infrastructure but has a fairly minimal summary
<cjwatson> and, I think, should probably just show repositories rather than refs
<cjwatson> maybe?
<cjwatson> although "last commit" is meaningless for repositories, so perhaps that's wrong
<cjwatson> perhaps repositories should have an expander for branches
<cjwatson> worth asking design if they have any thoughts there
<wgrant> I don't really know. I like seeing all the recent work, but it might get drowned out by eg. a push --mirror
<cjwatson> Yeah.  And status is meaningless for repositories *and* refs, too.
<cjwatson> (To the extent that it's meaningful for branches ...)
<wgrant> I would not complain if the status column accidentally went missing tomorrow.
<wgrant> mm
<wgrant> I guess Merged is useful
<wgrant> But meh
<cjwatson> In the default view, which excludes it :-)
<wgrant> yep
<cjwatson> If we're using committer_date (which we do right now in ref listings), then push --mirror is less of a problem
<cjwatson> One of my branches in progress fixes ref listings to sort by descending committer_date
<cjwatson> Which makes them a lot more sensible
<wgrant> blr: Do you want to experiment with these couple of views and see if you can get something vaguely sensible?
<cjwatson> http://paste.ubuntu.com/10956927/ is what I have today, if you want to crib anything from that
<blr> wgrant: sure - thanks cjwatson
<cjwatson> (also I have http://paste.ubuntu.com/10956930/ lying around from an older branch that tried to do more with targets, but I suspect more of that branch is crap)
<wgrant> I'll work with designy people tomorrow, but it'll be a while before we have anything useful, and we just need something vaguely adequate for tomorrow.
<blr> cjwatson: will move push-instructions to a partial, it is used in two places
<blr> and add the git specific copy we discussed last night
<cjwatson> The fixed sorting is the top bit of the patch to lib/lp/code/browser/gitrepository.py, and we likely want that regardless
<cjwatson> (from the first paste)
<cjwatson> I'm inclined to say that we may want to drift away from trying to have a single view for Person again
<wgrant> Yeah
<cjwatson> Person:+branches is recent work, but there could also be a view that lists all your repositories
<cjwatson> Then Person:+branches can just mix in refs with the committer date as "last modified"
<wgrant> A push --mirror with a billion branches seems not hugely likely.
<wgrant> Can always be adjusted if it doesn't work out.
<cjwatson> right, but even if somebody does that, a lot of those branches are going to be old
<wgrant> Yep
<cjwatson> in terms of committer date
<wgrant> So Person:+branches intersperses branches and refs.
<cjwatson> we miiiight have a perf problem, but I don't think it's a fundamental UI problem if you end up with a bazillion branches off in batch 87
<wgrant> Product:+branches shows one or the other, and additionally the default git repo if git is selected, plus a VCS switcher.
<wgrant> I don't see a perf problem here, though it will be awkward to implement.
<cjwatson> GitCollection lalala
<wgrant> It may be better to have a VCS switcher for Person:+branches in the first iteration.
<wgrant> No fundamental perf problem, then :)
<cjwatson> like another dropdown?
<wgrant> It's a simple matter of code.
<wgrant> Could be, yeah.
<cjwatson> Or just Person:+repositories, if it's going to have totally different semantics anyway
<wgrant> We can't currently dynamically change the default view for a particular object.
<wgrant> Only an interface.
<wgrant> But I guess having it as a secondary page for the first iteration is OK.
<cjwatson> right, but we could have another slot in PersonBranchesMenu
<cjwatson> which already has "Branches", "Active reviews", "Source package recipes"
<cjwatson> the only concern would be whether we're painting ourselves into a corner, but redirects are a thing ...
<wgrant> Right, it works short term.
<wgrant> Anyway, I need to sleep.
<cjwatson> I'm not convinced we can fix the canonical URL of target default repos by tomorrow, but even making it possible to find them via a Person view would be a small step up
<cjwatson> yeah, me too
<wgrant> blr: Experiment with what you can today, and Colin and I can hopefully bikeshed over whatever you have into something releasable in our morning.
<cjwatson> (conceivably, Person:+repositories could list all the projects you've worked on in code terms, whether that be bzr or git - take advantage of the bzr repository metaphor even if it isn't precisely identical
<cjwatson> )
<wgrant> True
<cjwatson> that sort of thing might help address people's issues with code
<cjwatson> and after all e.g. https://code.launchpad.net/~cjwatson/launchpad is a thing, so we *sort of* have that notion already
#launchpad-dev 2015-05-01
<blr> wgrant: what does "@@+" mean in a tal path?
<wgrant> blr: @@ means a view, as opposed to an attribute access.
<wgrant> So you look that up in configure.zcml
<wgrant> (I think @@ is meant to look like eyes)
<blr> hah okay
<rpadovani> wgrant, cjwatson congrats for the git release, and thanks :-)
#launchpad-dev 2015-05-02
<blr> wgrant: do we have any docs on updating sprites?
#launchpad-dev 2020-04-27
<cjwatson> Could I have a review of a small build system fix?  https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/382990
<cjwatson> (Might fix CSS on qastaging, not sure if it's the only thing)
<cjwatson> Not to mention CSS on staging and API on (qa)staging, I think
<cjwatson> ilasc: ^- maybe?
<ilasc> cjwatson: looking at it as we speak :)
<cjwatson> Ta
<ilasc> cjwatson: looks good to me
<cjwatson> Thanks
<ilasc> cjwatson: on the Edit OCI Push Rules screen, looks like with the current approach (replace credentials part of the rule with a link to the appropriate OCIRegistryCredentials:+edit view - screenshot on page 12) if User has no Push Rule defined there is no way for user to edit their Credentials
<ilasc> might need to do something different than just the link per rule
<cjwatson> ilasc: Aren't we still planning a tab on the person's index page too?
<ilasc> yes, that's still there (Display Creds only), allow Edit Creds there was part of the first version, but we said we want to keep Edit Creds with Edit rules. Are you thinking go back to enabling Edit Creds on the Person's index page ? (that's what I'm thinking at the moment)
<cjwatson> ilasc: I was expecting both
<cjwatson> credentials views accessible from your index page, and also the ability to edit credentials directly from a push rule if they're linked to one
<cjwatson> I don't think I intended to say that credentials editing should *only* be accessible from push rules
<cjwatson> if I implied that, I apologise
<ilasc> ok, makes sense to do it that way
 * ilasc implements both 
<cjwatson> Should just be a matter of the list view having some edit links
<cjwatson> https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/383000 (Fix determination of bug target roles)
<pappacena> This is what you were discussing on #launchpad some minutes ago?
<cjwatson> Yep
<pappacena> Awesome. Let me review it
<cjwatson> Thanks
<cjwatson> And I must owe you at least one review, so let's see
<pappacena> hahaha! Thanks! :)
<cjwatson> pappacena: Speaking of which, were you planning on landing https://code.launchpad.net/~pappacena/launchpad/+git/launchpad/+merge/382233 ?  Looks like it's ready to go
<pappacena> Is the DB patch applied to production already?
<cjwatson> It is
<pappacena> Ok! I'll top-approve it. Thanks
<cjwatson> pappacena: Conflict again, by the looks of it
<pappacena> Yep. Merging master into the branch here.
<SpecialK|Canon> pappacena: (Implicit) question in Freenode/#ubuntu-release ("whoever is accepting NEW packages") - is that something your archive admin audit work should be able to answer, out of interest?
<pappacena> uhm, I'll join this channel. Anyway, I guess this information is present at the archive admin audit trail, yes.
<SpecialK|Canon> pappacena: How could I view that for a given package?
<pappacena> At the publishing history page of that package (like https://launchpad.net/ubuntu/+source/man-db/+publishinghistory)
<pappacena> At the main page of the package, it shows as "Publishing details" when you click on the arrows at the bottom part too
<pappacena> like this https://usercontent.irccloud-cdn.com/file/67n2jGD1/image.png
<SpecialK|Canon> pappacena: Ok thanks will take a look!
<cjwatson> In this case it would actually be on the queue pages (https://launchpad.net/ubuntu/groovy/+queue)
<cjwatson> pappacena: FWIW the error on buildbot is easily reproducible locally with bin/test -vvc --layer=AppServerLayer
<cjwatson> ValueError: In version devel, IOCIRecipeBuildRequestEntry_devel.builds is a collection of IOCIRecipeBuild, but version devel of the web service does not publish IOCIRecipeBuild as an entry. (It may not be published at all.)
<cjwatson> is the key bit
<cjwatson> i.e. essentially the problem is that IOCIRecipeBuildRequest.builds is exported but the interface type that it's exporting a collection of isn't
<pappacena> Ah, got it. I'll take a look in a minute
<pappacena> Thanks!
<cjwatson> wgrant: Could you DB-review https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/383021 please?
#launchpad-dev 2020-04-28
<wgrant> cjwatson: db=me
<wgrant> er no, actually
<cjwatson> Hm yes, I suppose it does
<cjwatson> I wonder if we could contrive explicit tests for common kinds of FKs that need indexes
<wgrant> cjwatson: Mostly for deletions, I mean, rather than anything relating to the new feature
<cjwatson> Yeah
<wgrant> And yes, a linter here would be good
<cjwatson> Could just be in the test suite rather than linting the SQL as such - less convenient, but probably much easier to write, and would still run before stuff gets as far as an FDT
<wgrant> Oh yeah, that's what I mean, sorry
<wgrant> We already have a lot of the tools required
<wgrant> Used by person merge and librarian GC
<cjwatson> Right
<cjwatson> Anyway, index pushed
<pappacena> For the folks working on OCI, I've added the screenshot of proposed UI for OCI recipe list on the doc (https://docs.google.com/document/d/1hwFnrmEKfg7rimkjj2TC9J5i5gdzUPMZJTUUTFGmlR4/edit#heading=h.bgo6hibhf4m). Let me know if it looks ok that way, so I can implement tests, preloads and performance checks...
#launchpad-dev 2020-04-29
<tomwardill> morning all
 * ilasc emerges from the Debugger, morning tomwardill 
<tomwardill> o/
 * tomwardill is working from the conservatory today
<tomwardill> it's a lovely view, but it's pretty damn cold in here
<ilasc> nice! good excuse for plenty of hot coffee/tea :)
<cjwatson> https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/383121 (Ignore unparsable Built-Using for now)
<tomwardill> cjwatson: +1
<cjwatson> Thanks, landing
 * tomwardill does some reading about this new-fangled kubernetes thing
<tomwardill> I guess I should learn what some of these terms actually mean
<tomwardill> searchOCIProjects on distribution, seems reasonable to search by 'name' and/or 'owner', rather than the free-text search type link searchSourcePackages
<pappacena> strange... some operations (like requestBuilds and newPushRule) for OCI Recipe on dogfood are not available on the API, although they are decorated with @export_factory_operation. Only newWebhook is available...
<cjwatson> pappacena: probably stale cached wadl by your local lazr.restfulclient.  it's not great at refreshing
<pappacena> any easy way to clear that?
<cjwatson> pappacena: rm ~/.launchpadlib/api.dogfood.paddev.net/cache/*.wadl*
<cjwatson> we should really find and fix the bug that makes it get this wrong though
<pappacena> Aha! There it is! Maybe we could at least have a "lp.clear_caches()" method as a shotcut for this `rm`...
<pappacena> I'll put in my list to check that. Is there a bug reported for this problem?
<cjwatson> I dislike fossilising workarounds like that; it would probably be just as easy to fix it properly
<cjwatson> I don't see an existing bug :(
<pappacena> I can create one. https://launchpad.net/launchpadlib, right?
<cjwatson> pappacena: Yes, though I suspect the bug is actually further down the stack, either lazr.restfulclient or wadllib
<cjwatson> pappacena: All good to deploy then?  I see everything's green
<cjwatson> To production, I mean
<pappacena> Yes, just marked it green. Seems to be all good
<cjwatson> OK, great, will get going on that
<pappacena> Here is the bug, btw: https://bugs.launchpad.net/launchpadlib/+bug/1875917
<pappacena> Thanks, cjwatson
<mup> Bug #1875917: Outdated cache makes new named operation unavailable <launchpadlib :New> <https://launchpad.net/bugs/1875917>
<tomwardill> pappacena: I may or may not have aliased lp-shell in my bashrc to remove all the wadl files ;)
<tomwardill> (I did make muself a todo list to file that bug, so bad me for not doing it)
 * pappacena One day we will fix that. One step at a time...
<tomwardill> heh, yes :)
<tomwardill> oh whoops, ociProject doesn't have an owner
<tomwardill> so trying to search by it isn't going to help...
<tomwardill> add a searchOCIProjects to distribution: https://code.launchpad.net/~twom/launchpad/+git/launchpad/+merge/383157
 * pappacena I'll review it
<tomwardill> pappacena: oh, good point with the missing tests, I'll add them in the morning
 * pappacena ð
#launchpad-dev 2020-04-30
<tomwardill> morning all
<ilasc> morning Tom
<tomwardill> o/
<tomwardill> worst thing about emptying my office is having to reset my office chair again
<ilasc> oh... are you moving back in the office today ?
<tomwardill> yep
<tomwardill> all set up again, just got no shelves or pictures up yet
<ilasc> :) wow that was fast!
<tomwardill> we started decorating this room on Saturday :)
<tomwardill> been doing the lounge and the office alternately waiting for things to dry, etc
<ilasc> u guys should definitely come visit (our office needs a lot of TLC) :D
<tomwardill> hah, this is my decorating allocation for the year
<ilasc> lol
<tomwardill> I'm not doing any more after this!
<ilasc> :)
<tomwardill> It's mostly just to remove the wallpaper from the previous owners
<ilasc> right, yeah.. that makes sense
 * ilasc now loves Storm Vocabularies 
<ilasc> but as cool as they are, they don't yet implement the telepathy feature - only thinking I'm gonna need you to behave this way for tokens with a certain structure doesn't mean the Vocabulary actually will unless you code that in...
<cjwatson> :) they're nice once you get your head around them
<ilasc> agreed, yes
<tomwardill> back in 10 mins, need to remove a radiator.
<tomwardill> and back
<tomwardill> turns out double radiators hold a surprising amount of water, there was panicking
<cjwatson> I have that every damn time I try to unclog a washing machine filter
<tomwardill> yeah, the feeling of 'this pipe is only 3 feet long, how has it produced 40 litres of water'?
 * cjwatson tries to work out how process-job-source interacts with dbuser
<tomwardill> I think I've tried and failed at that
<tomwardill> cjwatson: distribution API tests are in doctests, I need to add a new test for searchOCIProjects. Is it worth adding a new doctest, or creating a new unit test case for it?
<tomwardill> I think I prefer the latter on the grounds of 'ergh, doctests'
<cjwatson> New unit test if that isn't too painful
<tomwardill> sure
 * tomwardill attempts the thing
<cjwatson> I've been known to add more stuff to an existing doctest when it was just too much work to get the necessary infrastructure set up
<cjwatson> tomwardill: but also, lp.registry.tests.test_distribution.TestWebService exists, although it's a bit weirdly-named and uses the slow launchpadlib test glue
<tomwardill> oh, so it does! missed that
<cjwatson> tomwardill: maybe a preliminary patch to port that off AppServerLayer/launchpadlib, and then add to it?
<tomwardill> cjwatson: I was just about to suggest the same, but reversed
<tomwardill> (do this one first, then transcribe the existing tests into it)
<cjwatson> Wouldn't be my preference, but either way works
<cjwatson> Hm.  process-job-source demonstrably does manage to set the dbuser.  But how?
<cjwatson> To the pdbmobile
<cjwatson> Aha, of course, it uses the generic script startup code
<tomwardill> pappacena: can you look over https://code.launchpad.net/~twom/launchpad/+git/launchpad/+merge/383157 again now I've added a test :)
<tomwardill> (first time writing an api test like that)
<ilasc> nicely done tomwardill, never seen an api test like until now either
<tomwardill> ilasc: I was just copying pappacena's work with the OCIRecipe API :)
<ilasc> :)
<cjwatson> Could I have reviews of https://code.launchpad.net/~cjwatson/launchpad-buildd/+git/launchpad-buildd/+merge/383183 and https://code.launchpad.net/~cjwatson/launchpad-buildd/+git/launchpad-buildd/+merge/383184 please?  Simple preliminary py3 stuff
<cjwatson> wgrant: Roughly how risky do you think it might be to extend process-job-source to be able to run multiple job sources with different dbusers?  (This is at the root of why process-job-source-groups spams lots of different expensive subprocesses.)  We have code that should be able to do the store reconnection work, but AFAIK using it is unprecedented outside the test suite.
<cjwatson> OTOH it seems to work very well in the test suite.
<wgrant> cjwatson: Doesn't celery do it?
<wgrant> I don't actually know how
<wgrant> Since switch_dbuser is IIRC in lp.testing
<cjwatson> task_init it seems
<cjwatson> Looks like it assumes a single store
<cjwatson> Maybe that would be OK here too
<cjwatson> transaction.abort(); store = IStore(Job); getUtility(IZStorm).remove(store); store.close(); dbconfig.override(dbuser=dbuser, isolation_level='read_committed')
<wgrant> Ah right
<wgrant> Will invalidate all objects, I guess.
<wgrant> Not invalidate but rather break
<cjwatson> Probably OK between job sources - there'll be no cache either so they'll get new objects
<cjwatson> Assuming the top level is sensible
<cjwatson> I might be inclined to extract that bit from task_init to somewhere else and put a bit more caution and docstringery around it
<cjwatson> I started writing code in process-job-source to run multiple job sources as long as they have compatible dbuser, but it's kind of ugly and annoying and doesn't help nearly as much as doing actual dbuser switching would
<wgrant> Yep, I don't think that really makes sense.
<cjwatson> pappacena: hold a minute on https://code.launchpad.net/~pappacena/launchpad/+git/launchpad/+merge/383094 - I have a few comments based on the screenshot, looking through the diff now
<pappacena> Ok! I was not planning to merge it right now
<tomwardill> landing https://code.launchpad.net/~twom/launchpad/+git/launchpad/+merge/383157
 * pappacena ok!
<cjwatson> pappacena: all right, commented now :)
<pappacena> Thanks, cjwatson! I'll adjust accordingly
<cjwatson> Another few buildd py3 patches: https://code.launchpad.net/~cjwatson/launchpad-buildd/+git/launchpad-buildd/+merge/383219 https://code.launchpad.net/~cjwatson/launchpad-buildd/+git/launchpad-buildd/+merge/383222 https://code.launchpad.net/~cjwatson/launchpad-buildd/+git/launchpad-buildd/+merge/383224
 * pappacena I'll check
 * pappacena done!
<tomwardill> do we have a prefered way to JSON a DateTime (for a named_get operation)
<pappacena> You mean on tests to check the returned JSON value?
<tomwardill> oh, it may not need to be JSON, just a quoted string, according to this other test I've found
<tomwardill> pappacena: no, to use a datetime as an argument to a named_get call
<pappacena> Yes... I guess it's using datetime_object.isoformat()
<tomwardill> that would make sense
 * pappacena I'm not 100% sure about it, tho
<cjwatson> .isoformat() is fairly usual, yes
<cjwatson> See lazr.restfulclient._json.DatetimeJSONEncoder
<cjwatson> Or indeed lp.services.webapp.batching.DateTimeJSONEncoder which seems to be a near-copy
<tomwardill> test transcribing for better webservice tests: https://code.launchpad.net/~twom/launchpad/+git/launchpad/+merge/383228
<cjwatson> tomwardill: Can you check whether the transaction.commit calls are necessary?  They can often be removed as part of this sort of work, since the webservice calls are now in-process
<tomwardill> ah, right
<tomwardill> checking
<cjwatson> (And removing them speeds things up, if it can be done)
<tomwardill> cjwatson: it does not need them, so removed and tidied imports
<cjwatson> tomwardill: r=me, thanks
<tomwardill> ta, landing
<tomwardill> pappacena: I'm doing some trello farming, is the asyncRequestbuilds work all landed?
<pappacena> Sorry for the delay... I went for lunch after the all-hands.
<pappacena> Yes, it is landed, Tom. Thanks for cleaning up Trello
#launchpad-dev 2020-05-01
<tomwardill> wgrant if you're about: when I do run `make run`, I get a zserver instance that is running 4 http servers. Do we do that in prod, or is there a separate process for each?
<wgrant> tomwardill: I think prod runs only two: the main one and xmlrpc-private
<wgrant> two of the dev ones are debuggers I think?
<tomwardill> ah, yeah
<tomwardill> so, I just got the main http server to run under talisker with gunicorn
<tomwardill> without zserver
<wgrant> tomwardill: Ooh, very nice.
<SpecialK|Canon> \o/
<SpecialK|Canon> weekly plz
<tomwardill> hah, no
<tomwardill> it is far, far, far from being ready for anything outside of my desktop
<tomwardill> :P
<SpecialK|Canon> Yes
<SpecialK|Canon> You have succeeded at a prototype milestone
<tomwardill> added
<tomwardill> SpecialK|Canon: I updated the OCI Build artifacts doc with a couple of comments pointing out which bits didn't get implemented (cross-registry mount),b ut it otherwise appears current
<SpecialK|Canon> tomwardill: what is the OCI Build artifacts doc?
<SpecialK|Canon> (Thank you!)
<tomwardill> so, unsurprisngly, my extremely naive implementation doesn't actually do anything useful...
