#launchpad-dev 2010-04-26
 * thumper afk and gym willing, back alive later
<mwhudson> huh, codebrowse for private branches is broken on staging...
<mwhudson> and maybe in production?
<mwhudson> surely someone would have noticed by now
<thumper> mwhudson: you would have thought so
<thumper> mwhudson: how broken?
<mwhudson> thumper: entirely
<thumper> do you know where in the chain it is broken?
<thumper> mwhudson: found this out testing your changes?
<mwhudson> thumper: yeah
<mwhudson> thumper: it's also important to change codebrowse to use safe_open
 * thumper nods
<thumper> well, on the plus side, I'm not dead
<mwhudson> thumper: hooray!
 * thumper back after dinner to review mwhudson's branches
<adeuring> good morning
<mrevell> Morning
<thumper> phew
<thumper> that's the last five reviews done for today
<thumper> hi mrevell
<bigjools> yo thumper
<thumper> hi bigjools
<thumper> bigjools: can you please chase the bzr-builder issue today
<thumper> bigjools: once we have that sorted, we can enable build to archive on dogfood
<bigjools> thumper: yeah, I'm going to speak to lamont later
<thumper> bigjools: and do some real tests
<mrevell> Hi thumper
<thumper> I'm done, but mrevell, we should have a chat sometime this week
<thumper> not now
<thumper> mrevell: could you do wednesday evening?
<thumper> mrevell: that's your local time
<mrevell> thumper, My Wednesday evening should be fine. What time are you thinking?
<thumper> mrevell: 8:30pm for you is 9:30am for me, which would be good
<thumper> mrevell: before that I have the stand up
<thumper> mrevell: but could push it the other way if you like
<mrevell> thumper, 9pm UK time or after works better for me
<thumper> mrevell: 9pm UK time is good for me
<mrevell> great, speak to you then thumper
<thumper> ok
<thumper> night all
<deryck> Morning, all.
<jml> good morning all.
<mrevell> Hey, intellectronica do you have a moment?
<mrevell> or deryck
<intellectronica> mrevell: sure, what's up?
<mrevell> Hi intellectronica. the bug-heat-view test is failing on my bug heat help branch. Mind if I pastebin you the output? I'm unsure how to adjust the test to take account of my changes.
<intellectronica> mrevell: sure, paste away
<mrevell> intellectronica, Cheers! http://pastebin.ubuntu.com/422705/
<intellectronica> mrevell: i guess the output changed a bit because now you're surrounding it with an anchor or something?
<intellectronica> mrevell: try changing "for img in soup.span.contents:" to "for img in soup.span.a.contents:"
<mrevell> intellectronica, Oh, cool, thanks. I've been trying to work out how to make it happy with the anchor. Cheers, I'll re-run the test now.
<huf> hi. is anyone successfully using the lp software other than you guys? :)
<Ursinha> huf, well, ubuntu? :)
<huf> i meant a separate installation
<Ursinha> ah
<Ursinha> that I don't know
<Ursinha> bigjools, hi
<maxb> huf: No-one's ever mentioned successfully creating another running installation. The image license terms are probably a notable part of that
<jml> oh
<jml> I just saw a comment from mwhudson, "all subversion-via-cscvs imports seem to be failing"
<jml> gary_poster: Is there any discernable difference between a read-only webservice API and a static JSON file?
<jml> gary_poster: I'd ask leonardr, but he doesn't seem to be here.
<jml> auth & wadl, I guess.
<gary_poster> jml, leonardr is out today.  Will be back tomorrow.  So, I'm not crystal clear on your question, but the only distinction I'd make is that, of course, a query string will quite probably matter for a webservice API, but be ignored if a file is server statically.
<jml> gary_poster: ok, thanks.
<deryck> hmmm, the bugtask table is missing its colors for status/importance now.
<deryck> sinzui, related to your CSS refactor maybe? ^^
<sinzui> deryck, I am sure it is
 * sinzui starts to investigate
<deryck> sinzui, was just about to ask if you're doing follow up fixes, or if we need to have it scheduled. :-)
<sinzui> I will be landing fixes everyday until I am certain all is well
* salgado changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 3 of 10.04 | PQM is open | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes | staging is down!
<abentley> mars, I have a bootstrapping problem: http://pastebin.ubuntu.com/422829/
<mars> abentley, looking
<mars> abentley, did you try 'make clean && make' ?
<mars> abentley, and if this was not created with rf-branch, then I assume you ran utilities/link-external-sourcecode ?
<abentley> mars, same problem.
<mars> a bit confused as to why 'elisa' is in that build chain?
<abentley> mars, this is the directory that link-external-sourcecode would use, as its target, so no.
<mars> abentley, ah, so this is running 'make' in devel then?
<abentley> mars, yes.
<abentley> stable, actually.
<jtv> intellectronica: we won't get around to that lazr-js upgrade this cycle, I'm afraid
<sinzui> deryck, did you report a bug about the colour issue?
<deryck> sinzui, I did not.  I can, though.
<intellectronica> jtv: that's a shame, it's not that much work, actually.
<mars> abentley, 'make' works in devel?
<jtv> intellectronica: I was wondering about that... do we have it documented?
<sinzui> I will report the bug. The issue is simple and universal. I moved the anchor colour rules to the new style sheet. I need to move all the colour rules new so that they take precedence.
<sinzui> deryck, ^
<deryck> sinzui, ok, thanks.  if I had filed a bug, this would be the sort of thing to file against launchpad-web?
<intellectronica> jtv: i don't think we do (and i must admit i don't remember it exactly to describe it), but we can look it up together if you want.
<abentley> mars, I don't have a devel on that box.
<sinzui> deryck, In this case it is.
<deryck> ok, thanks
<jtv> intellectronica: it's getting late here, but I would appreciate a quick collaborative look...
<intellectronica> jtv: excellent, let's do that tomorrow my morning?
<mars> abentley, was it upgraded to lucid recently?
<jtv> intellectronica: let's!  Thanks.
<abentley> mars, this box was upgraded to lucid a while ago.  What's new is I ran update-sourcecode for the first time (instead of rsyncing from devpad), and I think it deleted twisted.
<abentley> mars, Okay, I went into pymodules and moved elisa out.  That seems to have fixed it.
<mrevell> sinzui, Hi! Has your branch changing the CSS files landed?
<sinzui> It did
<sinzui> mrevell, merge away
<mrevell> thanks sinzui!
<sinzui> mrevell, I just started a branch to move all colour rules. (this will fix an issue on edge). Does your branch move colours?
<mrevell> no sinzui
<mars> abentley, :/
<mars> abentley, ok, so that was a code isolation failure of some sort.  Love to know where that came from.  I'll keep an eye out for similar failures in the future.
<mars> gary_poster, jml, looking at the build-infrastructure list, and I am sorely tempted to use the Wishlist priority (much to sinzui's chagrin :)
<jml> mars, priority schmiority -- all I care about is whether they are fixed or not :)
<jml> mars, but which ones in particular?
<gary_poster> mars, naah.  your wishlist is somebody else's low.  The question is what *you* do.  Other people might tackle some of the other ones you think of as wishlist
<mars> gary_poster, true true
<sinzui> mars, feature tag + low priority
<mars> jml, anything that says "make it ...", "want", or "should" would be fair game
<mars> jml, faults first, improvements after
<jml> mars, I agree with the second thing you said
<mars> jml, sinzui, that really does describe the problem best: looking at this list of bugs, I do not know which are faults and which are features
<sinzui> mars, :( My example of a bug is one where we know the rules, this situation violates the rules, so it is a bug. Otherwise this is a feature to introduce new behaviour.
<mars> but then you end up with the semantic problems that Gary stated.  Oh well.
<sinzui> mars, I think that is an accidental great incite into triage. the semantics are really essential-now, important-next, useful-future
<mars> sinzui, yes
<sinzui> or insight
<mars> sinzui, a fault is anything that keeps it from running today.  Anything else is an improvement or feature.
<mars> it becomes easy to decide the grey-area ones then
<mars> "Buildbot should be public" - does that keep it from working today?  No?  Improvement!
<mars> "Sanitize and document logins used in in pagetests" - Improvement
<mars> "LaunchBag.developer is not updated when ftest login() is called" - probably a fault
<sinzui> I distinguish priority 1 from 2 based on the severity of the blockage. severity 2 can be worked around, while 1 cannot
<mars> interesting
<sinzui> An example of that would be ec2 land is broken, but ec2 test with -s works
<mars> right
<mars> so I call that is a high priority fault
<sinzui> I have rated items with work arounds are 1 because the solution requires arcane knowledge
<mars> if there were no workaround, it would be critical
<mars> hehe
<mars> sinzui, I like your idea for three classifications.  I'll see if I can work with that to prioritize.
<mars> flacoste, were the stylesheet problems on edge last week fixed?
<flacoste> mars: yes
<mars> flacoste, thanks.
<mars> awe... sinzui, ping.
<sinzui> hi mars
<mars> hi sinzui, are the links to bug statuses on this page blue for you? https://bugs.edge.launchpad.net/launchpad-foundations/+bug/570380
<mup> Bug #570380: ec2test sometimes hangs on the first windmill test <build-infrastructure> <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/570380>
<sinzui> mars, not in the branch I just pushed.
<sinzui> do you want to review it?
<mars> sinzui, I would not feel confident doing so
<sinzui> mars: bug 570259 is the global issue
<mup> Bug #570259: css colours are wrong after style sheet reorganisation <css> <launchpad-web:In Progress by sinzui> <https://launchpad.net/bugs/570259>
<sinzui> mars, all the colours in the old style sheet (mostly enums) lost precedence when I moved the anchor rules to style 3.0
<mars> mwhudson, what do you think of this?  https://dev.launchpad.net/Debugging/GDB
<mars> mwhudson, I added a section for backtrace.py to the bottom.
<mwhudson> mars: looks good, you don't need the .gdbinit file though
<mars> mwhudson, oh, so "the .gdbinit should also work" was saying that backtrace.py does roughly the same thing as the GDB macros?
<mwhudson> mars: yes
<mars> mwhudson, updated
<thumper> morning
<mars> morning thumper
<mars> It would be nice to have an easy way to copy the short "http://launchpad.net/bugs/IIII" link, instead of having to type it out by hand.
 * persia remembers an exceeding contentious change that removed just that feature
<persia> I believe the argument was that if a user was already on a page, there was no point in having a UI element that would link back to the *same* page, and people who wanted to copy could always do it from the location bar.
<mars> persia, fair enough.  https://bugs.edge.launchpad.net/launchpad-foundations/+bug/570380 is not fun to copy-n-paste though :)
<mup> Bug #570380: ec2test sometimes hangs on the first windmill test <build-infrastructure> <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/570380>
<mars> persia, I am also a fan of StackOverflow's titles:  stackoverflow.com/QESTIONNUMBER/some-human-readable-question-title
<persia> Worse yet, because of the embedded "edge" it fails completely to be useful for lots of folks.
<mars> :)
<persia> This is part of why the bugbots do what they do, so one can usually say bug #570380 and get a useful response.
<mup> Bug #570380: ec2test sometimes hangs on the first windmill test <build-infrastructure> <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/570380>
<persia> Doesn't help so much with mail though.
<jml> persia: the change I remember more is when we removed the non-linked bug number
<jml> persia: which caused an outcry because there was no longer an easy way to copy-and-paste it.
 * jml disappears
<wgrant> persia: Fails completely? edge works for everyone.
<wgrant> (but yes, it's suboptimal)
<persia> wgrant: Fails completely *to be useful*
<persia> This isn't "fails completely"
<persia> jml: I thought I remembered complaints about it changing both times (and yes, current behaviour matches very old behaviour)
#launchpad-dev 2010-04-27
<thumper> gary_poster: ping
<jelmer> gary_poster: hi
<thumper> anyone know where the request.publication is set?
<poolie> hi jelmer, thumper
<thumper> hi poolie
<thumper> poolie: I have an interesting bug for you
<thumper> poolie: if I can find
<thumper> it
<thumper> bug 568740
<mup> Bug #568740: lp:ubuntu/sun-java6 isn't pulled, apparently due to MemoryError <branch-puller> <Bazaar:New> <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/568740>
<poolie> ok
<poolie> so can you give us any more information about it?
<poolie> or do you think pulling it directly should reproduce it?
<poolie> i've been meaning to do something about giving better debug info for MemoryError
<poolie> so i might try it in that context
<poolie> is this one important or just interesting?
<thumper> well, I don't know where it is failing apart from MemoryError
<thumper> it is important for someone
<mars> thumper, the request.publication stuff is a function of an IRequestPublicationFactory instance.  We have a few in c.l.webapp.servers
<thumper> mars: thanks, I'm going through the doc test for publication right now
<mars> thumper, the factory produced request/publication pairs, which the Zope machinery puts together, then gives to zope.publisher.publish()
 * thumper nods
<cody-somerville> Hey
<cody-somerville> I'm making a field read only and setting up a mutator for it. This causes tests to break because the field is used in forms. How do I get around this?
<cody-somerville> Ah, I see branch's privacy field is an example.
<thumper> mwhudson: what have we broken? http://launchpadlibrarian.net/45588750/maxb-django-trunk-bzr-svn.log
<mwhudson> thumper: huh
<thumper> mwhudson: it is a branch
<thumper> mwhudson: for svn
<thumper> mwhudson: so I'm wondering if we have broken something else
<mwhudson> bzr log -l 2 svn+http://code.djangoproject.com/svn/django/trunk seems to be doing something
<mwhudson> thumper: however, bzr log -l 2 http://code.djangoproject.com/svn/django/trunk does not work
<mwhudson> thumper: at this point i suggest the usual course of action
<mwhudson> thumper: which is asking jelmer
<thumper> mwhudson: bug jelmer?
<mwhudson> perhaps the code import system could prepend svn+ to http urls for svn imports
<nigelbabu> I'd like to work on fixing bug 569447
<mup> Bug #569447: Documentation for searchTask tag is wrong <api> <trivial> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/569447>
<nigelbabu> Is it something easy to do?
<mwhudson> nigelbabu: should be
<mwhudson> nigelbabu: the documentation for the api doc comes from the docstring in the interface
<nigelbabu> mwhudson: I read the docs and it said I should first discusss about it and then start
<nigelbabu> if it comes from the interface there is a problem.
<nigelbabu> the documention is corrent for the interface and wrong for the api
<mwhudson> nigelbabu: why?
<mwhudson> ah
<mwhudson> i wonder if there's a way of overriding/changing the doc for the api...
<mars> mwhudson, nigelbabu, grep-find?
<mars> since you already know the string you are looking for...
<nigelbabu> its not about finding.
<nigelbabu> mwhudson: can poke me when you find a way out?  I'm ready to work on it if it needs fixing :)
<mwhudson> nigelbabu: i'm not sure that you can override the string used in the api doc
<mwhudson> nigelbabu: so maybe a bug on lazr.restful asking for that is the way to go
<nigelbabu> ah, ok :)
<thumper> mwhudson: how come the postmortem debugger isn't giving me any content for listings?
<nigelbabu> what should I say? "There should be a way to override the interface strings for API docs"?
<thumper> mwhudson: I just get [EOF]
<mwhudson> thumper: um, the path in the pyc files might be wrong?
<thumper> :(
<thumper> doesn't seem to be
<mwhudson> someone has requested a mercurial import of netbeans
<mwhudson> good luck with that
<noodles775> G'day all.
<spm> yo
<mwhudson> must be time for dinner
<mrevell> Morning.
<wgrant> Is it just me, or have the JS status/importance widgets lost their custom colouring recently?
<mwhudson> baaah
<mwhudson> exactly one failure on no-hosted-area
<jml> mwhudson: :)
<mwhudson> fortunately it's rather trivial
<mwhudson> jml: good morning
<jml> mwhudson: hi
 * mwhudson fires up the ec2 land of doom
<mwhudson> " 96 files changed, 2640 insertions(+), 2916 deletions(-)"
<jml> mwhudson: that's a big change
<jml> mwhudson: are much of those insertions added documentation?
<bigjools> thumper: still around?
<jml> I'm off out of the house for a bit. Expect to be online shortly.
<jml> bigjools: let's talk today
<bigjools> jml: grab me when you can
<jml> bigjools: ok. thanks.
<jml> wgrant: you've probably already figured it out by now, but sinzui's recent CSS refactoring broke them. Looks like there's a fix up already.
<wgrant> jml: Yeah, I thought that would have to be it.
<thumper> bigjools: no
<bigjools> thumper: ok, then I can't tell you about the new PPA for bzr-builder
<thumper> bigjools: email still works :)
<bigjools> thumper: when kmail is not crashing, yes :)
<deryck> Morning, all.
<mwhudson> jml: no
<gary_poster> thumper jelmer pong
<bigjools> ah gary_poster, just the man I need to talk to
<gary_poster> what's up bigjools
<bigjools> gary_poster: I might have been a bit naieve in expecting a webservice method to work when returning a collection of ITextLine
<bigjools> there's no adapter to do that by the looks of it
<bigjools> is there any easy solution to returning a list of strings?
<gary_poster> bigjools: sadly, I'm not sure.  Leonard will probably be on soon and I'll ask him to seek you out.
<bigjools> gary_poster: ok thanks
<wgrant> Do you actually need to do returns_collection_of there?
<wgrant> If you omit it, lazr.restfulclient will just treat it as JSON, so they will come out as strings.
<wgrant> (we already do a similar thing for dicts in a few places)
<bigjools> wgrant: ah, got an example?
<maxb> Are there no CHR people this week?
<wgrant> bigjools: There are methods like Branch.composePublicURL and BugNomination.canApprove that return single objects without declaring their types (since they are basic JSON types).
<wgrant> I'm not sure of any that return lists.
<wgrant> It does just work, but I'm not sure how legal it is.
<bigjools> yeah it's the list aspect that is problematic
<wgrant> If you don't tell it otherwise it will just decode the JSON and be done with it.
 * bigjools tries and sees what happens
<wgrant> Technically it's fine.
<wgrant> Whether it will make a Leonard appear and attack you, I do not know.
<bigjools> a Leonard can change its spots
<bigjools> wgrant: BTW did you check out the sizes of those log files ?
<wgrant> bigjools: Yes.
<wgrant> I am scared.
<bigjools> yes thought so :)
<bigjools> has anyone done a webservice test using lplib yet?
<wgrant> People have tried.
<wgrant> Bugs have been filed.
<wgrant> I think some of them might have been fixed.
<wgrant> eg. Bug 569189
<mup> Bug #569189: Authenticated users in launchpadlib tests have no permissions <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/569189>
<wgrant> Bug 569101
<mup> Bug #569101: Protocol Error when calling some launchpadlib methods within the test environment <qa-untestable> <Launchpad Foundations:Fix Committed by bac> <https://launchpad.net/bugs/569101>
<wgrant> So it probably works a bit.
<bigjools> let's see
<bigjools> wgrant: win!
<wgrant> bigjools: You have a string collection exported and tested with launchpadlib in the test suite?
<bigjools> yarp
<wgrant> Win indeed!
<bigjools> so now I can land this branch that exports your p3a URLs
<wgrant> I bet that doesn't respect the OAuth token's privacy setting, though.
<bigjools> it does
<bigjools> the new method is on IPerson and it'll throw a 401 unless you've got lp.edit
<wgrant> But if I tell Launchpad that my token shouldn't be able to access private data, will it let me?
 * bigjools re-reads and sees you were talking about OAuth tokens
<wgrant> Ah.
<bigjools> not sure it does respect it, you're right, I'm using WRITE_PUBLIC
<leonardr> bac, refresh my memory
<leonardr> you were having some problems with the launchpadlib tests in launchpad
<leonardr> i gave you a little one-line-or-so fix, which you applied and it worked
<leonardr> is that right?.
<leonardr> have you landed that?
<leonardr> bigjools, gary told me of your woe
<leonardr> an ICollection is only for IEntry objects that have their own urls. you should be able to return an IList of ITextLine
<Ursinha> bigjools, hello :)
<Ursinha> bigjools, I pinged you yesterday, but  have no idea if you answered because my internet was worse than crap
<bigjools> Ursinha: hi
<bigjools> leonardr: yeah I'm just returning the list and it seems to work well, thanks
<jml> allenap, is that test process still going?
<jml> allenap, if so, could you please attach a strace to it and see what it's doing?
<allenap> jml: Sorry, I killed it. It was in a tight select() loop though; I did an strace before I killed it.
<jml> allenap, ok. thanks.
<allenap> jml: select(0, NULL, NULL, NULL, {0, 10000}) = 0 (Timeout)
 * jml raises an eyebrow
<bac> leonardr: yes, i landed the patch that involved uppercasing of the request method
<bac> leonardr: i also discovered this bug: https://bugs.edge.launchpad.net/launchpad-foundations/+bug/569189
<mup> Bug #569189: Authenticated users in launchpadlib tests have no permissions <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/569189>
<allenap> jml: Fwiw, it's in lp:~allenap/launchpad/early-batching-bulk-updates-bug-509223 when run with the test command in the email. If I run with "bin/test -vvc lp.bugs.scripts.checkwatches" it does not hang. This might have something to do with when zope.testing spawns a sub-process when the layer doesn't tear down.
<leonardr> james_w, i just released a lazr.restfulclient that fixes bug 568301
<mup> Bug #568301: Invoke a named operation on a collection, and the collection is fetched. <performance> <lazr.restfulclient:In Progress> <https://launchpad.net/bugs/568301>
<leonardr> want to give it a try and see how the performance improves?
<james_w> thanks
<james_w> I'll try later
<leonardr> sure
<leonardr> james_w: when you have time to test things i changed, also take a look at bug 561521
<mup> Bug #561521: Success of PATCH request dependent on dict iteration order <Launchpad Foundations:Fix Committed> <lazr.restful:Fix Released> <https://launchpad.net/bugs/561521>
<wgrant> deryck: Hi. Are you aware of some horrifyling slow BugJob-related queries caused by status changes?
<wgrant> They are making some Soyuz things time out.
<deryck> wgrant, I have just started to become aware of some BugJob-related timeouts.  Didn't realize we had some with status changes, though.
<Ursinha> bigjools, so, ping me when you're not busy, re. my bot and your bugs :)
<wgrant> deryck: Accepting queue entries can close bugs, and they've recently started timing out with several hundred milliseconds of BugJob query per bug.
<deryck> wgrant, ah, that makes sense.  Anything that qualifies for a bug change will touch that code.  gmb, see wgrant's comments. ^^
 * gmb looks
<deryck> gmb, so we need to look at ways of speeding this up.  it's causing timeouts when filing bugs on ubuntu, too.
<gmb> wgrant, deryck: Ah, so at a guess this is happening when we're looking at BugJob to see if there's already a CalculateBugHeatJob for that bug, right?
<wgrant> gmb: So I'm told, although I obviously can't see the full picture. OOPS-1575B227 is an example.
<maxb> allenap: If you've still got that hung test run around, could you check whether there actually is a related subprocess hanging around? Otherwise, I'm not sure why it would still be blocked. You could try a 'lsof -p thatpid' to see if there's anything interesting there
 * gmb looks
 * deryck looks at OOPS, too
<deryck> Yeah, this is the same as ubuntu filebug timeouts.  and yes, as gmb notes, it's the select.
<deryck> gmb, See bug 539382, too.
<wgrant> The query doesn't look like it has to be terrible.
<wgrant> Unless the indices are.
<gmb> wgrant, Yeah, I wonder if we're missing an index somewhere.
 * gmb looks
<gmb> wgrant, deryck, I wonder if there should be indices on Job.lease_expires and .scheduled start, since that's what's getting hit in the subquery.
<wgrant> My DB says seq scans on Job and BugJob.
<gmb> ?!
<wgrant> That would do it.
<gmb> So we need indices on both tables then.
<deryck> yup
<deryck> simple to fix then. :-)
<gmb> Indeed.
<wgrant> ...
 * wgrant looks at BugJob's indices.
<gmb> (Well, simple to *say* anyway)
<wgrant> Or lack thereof.
<deryck> gmb, do you think you could find some time today to cowboy a patch to staging to confirm, and if so, get a branch for that?
<gmb> deryck, Sure, I'll add a card for it. Should get me away from this endlessly frustrating refactoring, too.
<deryck> gmb, thanks!  Sorry to sideline you with it, but it is becoming more and more of a problem.  You can use bug 539382 to track it, I think.
<deryck> wgrant, thanks for pinging me about this.
<gmb> deryck, cool
<wgrant> deryck: Thank you for swiftly investigating.
<deryck> np
<bigjools> Ursinha: HELLEAU
<maxb> Hi, are there no CHR people this week?
<jtv> Puzzle for the day.  Why would Store.of(x) return None, given that x is not None and was retrieved from the database, other than that x has been deleted somehow?
<wgrant> jtv: Because you are a BranchJobDerived, not a real BranchJob.
<jtv> wgrant: brilliant!
<jtv> wgrant: for the bonus round, why would BuildQueue, which has no clue WTF I am or am not besides the fact that I implement a particular interface, want to destroy me?
 * bigjools removes the third massive bee from the office today
<jtv> Actually, BranchJobDerived delegates to BranchJob, so I'd guess that would include destroySelf.
<wgrant> jtv: It tries to the destroy the BuildQueue, the Job and the link between the Job and the real object.
<jml> jtv: humanity has a long history of destroying things based on perceived type
<wgrant> jtv: But this doesn't use destroySelf.
<jtv> jml: yes!  I thought we were finally getting good at it.
<wgrant> jtv: And bigjools vetoed my change to use it.
<wgrant> In favour of noodles' big refactoring.
<jtv> arrrr
<jtv> Hear that faint drone?  That's Baby Jesus crying.
<noodles775> wgrant: the big refactoring is based on your ideas too :)
<jtv> noodles775: that's as may be, but right now, BuildQueue does The Wrong Thingâ¢ with my object and what recourse do I have?
<wgrant> https://code.edge.launchpad.net/~wgrant/launchpad/bug-549340-fix-buildqueue-destruction was mine, FWIW.
 * noodles775 looks at the branch and jtv's issue.
<bigjools> jtv's branch is not a storm object is it?
<bigjools> s/branch/object/
<wgrant> bigjools: Right.
<wgrant> It delegates to one.
<jtv> bigjools: if you want to argue over whether there should be a separate TranslationTemplatesBuildJob table, go argue with jml who insisted that I use BranchJob instead.  :)
<jml> jtv: citation needed
<bigjools> lol
<jtv> Why not define a method in IBuildFarmJob for "this build is done, do whatever you want for historical purposes, or delete yourself, I don't care which"?
<wgrant> The Code team in general wants us to use Jobs.
<bigjools> yeah and that unfortunately hurt us in the buildfarm :(
<wgrant> Well, there was discussion last week that they were unhappy with the new design, since it doesn't involve Jobs.
<jtv> What, we're selling out to Apple?
 * wgrant throws some iPhone shards at jtv.
<jtv> Oh, `Job`s.  Sorry.
<noodles775> I'm not removing the jobs at the moment, and I think once the refactoring is finished we'll be in a much better position to see if we want to either (1) talk about removing the links to Job or (2) turn BuildFarmJob into a services Job with the correct job-type.
<wgrant> I think the new model could pretty trivially be altered to integrate well with Job, without the mess that is the current situation.
<noodles775> I agree.
<jtv> Actually, removing a BranchJob will also destroy the Job.
<jtv> Anyway.  I do need an interim measure, and fast.  How about this idea of delegating "go destroy yourself or something" to the IBuildFarmJob?
<bigjools> the only reason to use Job is if we intend to use the Job runner
<bigjools> that will never ever happen
<wgrant> Code disagrees.
<wgrant> Feel free to argue with them :)
<bigjools> they would :)
<bigjools> I'm not arguing, I'm stating a fact.
<wgrant> jtv: So you're actually thinking of deploying these jobs in 10.04?
<jtv> wgrant: or be damned near ready to.
<wgrant> Excellent.
<jtv> At the very least I want to be able to move on to the next failure, it's been long enough!
<bigjools> heh
<wgrant> Well, they worked perfectly for me a little over a month ago. I'm not I had any fixes that remain unmerged, apart from this one.
<wgrant> So barring the inevitable massive production issues, it probably just about works.
<jtv> Nice quote, that.
<noodles775> jtv: so that would be wgrant's branch above right? (IBuildFarmJob.destroySelf())
<jtv> noodles775: no, bigjools had one strong argument against thatâit's SQLObject.  My own would be: it's the IBuildFarmJob's own damn business what it wants to do after it retires.  It doesn't _have_ to die right there.
<jtv> Just call it something different and presto.  :)
<noodles775> Right.
<bigjools> jtv: I think you're simplifying it too much.  We have a BuildQueue, which has generic info, and an associated IBuildFarmJob which has specific info.  BuildQueue records are destroyed when done with, which means the IBuildFarmJob must be too.  If you want persistence you need a separate table.
<bigjools> which I'm pretty sure is what I said ages ago :)
<jtv> bigjools: why does it mean IBuildFarmJob must be destroyed?  All you need is for it to stop referring to the Job.
 * wgrant would really like to delay any persistence changes until after The Refactoring.
<bigjools> jtv: because that's how it's designed to work, changing it now would be very invasive
<wgrant> Particularly since it makes the transition much easier if we only have one type of job to migrate...
<jtv> bigjools: I can see how the design always assumed it, but where does it actually rely on it?
<bigjools> jtv, think of it as the difference between data about the job, and data about the build itself
<jtv> Yes, I'm thinking of it... go on!
<mrevell> noodles775, Hey, got five minutes?
<noodles775> mrevell: sure.
<bigjools> jtv: so it was designed like that so that we could throw away data that was of no use once the job was done.  Deletion is something that doesn't happen enough in LP!
<noodles775> jtv, bigjools: jfyi, after the refactoring IBuildFarmJob *won't* be destroyed, but all the other queue-related items will be (as they are now - I'm not touching their behavior). That is, BuildPackageJob delegates to IBuildFarmJob... the BuildPackageJob (and related job) will die, but not the BuildFarmJob.
<jtv> bigjools: but you don't know what the data _is_.  The IBuildFarmJob is not data; it's something abstract that helps you define how to do the job, not a database record.
<wgrant> jtv: That's about to change.
<noodles775> It will be, but that doesn't help you now.
<wgrant> noodles775: So BuildQueue/Job are staying as they are for now, just all the stuff on top of them is being fixed?
<bigjools> jtv: I didn't mean IBuildFarmJob as such, but the derived tables, but as they say it's going to change
<noodles775> wgrant: yes, I'm not touching the BuildQueue, just ensuring it still works as it does now. I'm just generalising so that all *Build models have common info in the same place.
<jtv> bigjools: that's fine with me, but as an implementer of IBuildFarmJob, I still say it's my own business how I take care of this.
<noodles775> (basically your diagram minus the queue related attributes)
<bigjools> jtv: I completely disagree :)
<jtv> Well, then fix BuildQueue to do it in a way that works for non-ORM classes!
<bigjools> making the build-manager work for different job types was fucking hard
<bigjools> the direction we took solved a lot of problems while not breaking too much of the existing functionality
<jtv> kudos for that, and I'm not married to the "the IBuildFarmJob doesn't have to die" idea.  Can we at least delegate the deletion somehow?
<bigjools> you're going to need a separate table that has persistent data on it, for the builder history page
<bigjools> anyway chat to noodles775, he might be able to help more than me since he's deep in that code at the moment :)
<jtv> We've got a ticket open for persistent history.  My very immediate concern is this failure that puts the builder into a very bad state.
<noodles775> jtv: I think delegating the deletion would be fine... happy to chat about it, or review an MP :)
<cody-somerville> Can I get some help with my branch? lp:~cody-somerville/launchpad/bug-444266/
<jtv> noodles775: great, thanks. Call it cleanUp() or something, put the Store.of(self).remove() in the default implementation, and override it in TranslationTemplatesBuildJob?
<noodles775> jtv: sounds good.
<jtv> noodles775: ok, I can have that waiting for you tomorrow morning.
<jtv> assert jtv.eod < time.now()
<cody-somerville> I created a IHasBugSupervisorEditSchema class to provide an editable field for bug_supervisor in IHasBugSupervisorEditView since bug_supervisor is defined as readoly in the IHasBugSupervisor interface.
<jtv> noodles775: thanks again.
<bigjools> jtv, noodles775: can we determine if the object is Storm or SQLObject and DTRT?
<noodles775> cody-somerville: the On Call Reviewer (on #launchpad-reviews) is usually around for that kind of think if no-one here takes you up :)
<jtv> bigjools: this one isn't either
<bigjools> ?!
<jtv> but afaic the default implementation could try
<bigjools>  /o\
 * bigjools -> OTP
<flacoste> can you believe it's snowing over here!
<jml> yes
<jml> you live in an arctic wasteland
<noodles775> bigjools: you can (I've had to create the following property on IBuildFarmJob as an intermediate measure: http://pastebin.ubuntu.com/423413/)
<jtv> jml clearly fancies those, or he wouldn't have moved so much closer to them
<cody-somerville> flacoste, Don't say that. That means its only a matter of time before it starts snowing here :P
<bigjools> it's hot here
<jml> jtv, no, I just _really_ like grey.
<jtv> "the night sky over the planet Krikkit is the least interesting sight in the Universe."
<jtv> bigjools: hot here too.  Still otp?
<jtv> Wow, it's getting so difficult to pick out a T-shirt to wear now that they have mobs in all colours.
<jtv> I loved the green ones.  Waving identical little flags with fixed, almost hopeful little smiles...  Very Pyongyang.
<gmb> deryck, wgrant: So, adding indices doesn't seem to solve the seqscan problem with bug 539382.
<gmb> (I haven't added indices for *everything* yet, maybe that'll work)
<gmb> However, it may be simpler to have CalculateBugHeatJob.create() just create a new job regardless and then have the garbo process make sure that only one update gets run for that bug.
<deryck> gmb, are you trying this locally?  My experience is that local indexes may not get used when they will on staging.  Unless you're working with a bit of BugJob data locally.
<gmb> deryck, Ah, good point. I didn't realise that indices might not get used when data volume is low. I'll try and scare up a LOSA.
<deryck> gmb, though you're right that ultimately it might be still be better to push the work to CalculateBugHeatJob.  But indexes will help regardless of that.
<jtv> In fact you'll get _completely_ different plans on realistic data than you will at home.
 * jtv is out.
<gmb> deryck[lunch], wgrant: So, indices FTW. Will put a DB patch together.
<deryck> gmb, good news, thanks
<mrevell> Night all
<leonardr> gary or barry, any suggestions on how to mock up a test for time.sleep? i have a semi-vague idea but it seems like something you'd know well
<gary_poster> leonardr: I've simply used the Grand Hack monkeypatch approach myself.  If I control the pertinent module(s) I sometimes use a "from time import sleep" within it so that I can just locally mokeypatch sleep in the module.  It's possible that mocker would give you an alternate approach, but then you are adding a dependency
<leonardr> gary, what you're saying is similar to what i was thinking, but i'm not sure what you mean by 'the module'. do you have time to work this out in person?
<gary_poster> sure
<salgado> leonardr, maybe this could be of help: http://labix.org/mocker
<salgado> Trivial mocking of any external module (e.g. time.time()) via proxy replacement.
<gary_poster> that was my second suggestion above, yeah.  But it's adding a dependency.  If you need it (because you don't control the module that uses sleep) it might be a particularly good choice
<leonardr> gary: it also might be a good choice since lazr.restfulclient already splits out test dependencies from regular dependencies
<gary_poster> true
<leonardr> gary: also, i now understand what you said originally. i'll try that
<gary_poster> cool
<mwhudson> The size of the diff (9540 lines) is larger than your specified limit of 1000 lines
<mwhudson> woop!
<mars> mwhudson, lol
<wgrant> mars: Hi. Do you know why we still have PlotKit and slimmer in the three?
<wgrant> AFAICT they have been unused since 3.0 or earlier.
<wgrant> s/three/tree/
<mars> wgrant, sorry, I do not know.  I would have to hunt for leftover traces of them in the source.  sinzui would know better than I if they are still used.
<wgrant> mars: I've grepped, there's nothing.
<sinzui> wgrant, I wanted to remove plotkit a few months ago. I cannot remember what stopped me
<sinzui> wgrant, I give you an rs=sinzui to remove them. I am certain no tests will break because they predate js tests.
<wgrant> haha.
 * sinzui thinks a few bug reports after the fact will spur engineers to get rid of cruft
<wgrant> I have a few other deletions of unused code, but the branch hit 10000 lines (all deletions) so I might split it.
<sinzui> wgrant, I can look at it. I love reviewing deletes
<wgrant> sinzui: https://code.edge.launchpad.net/~wgrant/launchpad/remove-obsolete-js-stuff/+merge/24264, if you have a moment.
 * sinzui looks
<sinzui> wgrant r=me
<sinzui> wgrant, Do you want me to land this now?
<wgrant> sinzui: Yes please.
<wgrant> sinzui: Oops, forgot to delete another copy of PlotKit.
<sinzui> okay
<sinzui> I was just looking for an old plotkit bug so I had not pressed enter yet
<wgrant> Ah. Pushed, and already mirrored.
<sinzui> bomb away
<wgrant> sinzui: Were you thinking of your attempt to remove MochiKit, which PlotKit needed?
<sinzui> yes I was
<sinzui> there is still some complicated script using mochikit
<thumper> mwhudson: talking with flacoste solved my problem for yesterday
<thumper> mwhudson: a 2005 XXX comment
<thumper> :)
<mwhudson> thumper: woot
<thumper> mwhudson: can I move a pipe in a pipeline?
<thumper> mwhudson: I forgot to add --before to my add-pipe command
<mwhudson> thumper: not directly afaik, but you can just remove it and add it again?
<thumper> nm
<thumper> yeah, that's what I did
<wgrant> sinzui: I'm confused -- what is left to do for bug #44052?
<mup> Bug #44052: Auto-suggest affected project based on affected package <bridging-the-gap> <package-link> <Launchpad Registry:Triaged> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/44052>
<wgrant> It has done so for years.
<mwhudson> any bugs developers here?
<mwhudson> some bugwatch failures in db-devel
<mwhudson> looks like http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/10789 conflicts with a database patch in db-devel
<thumper> flacoste: I've got the removal of the CustomMachine running through ec2
<thumper> flacoste: with that done, my test_traverse works in the harness now
<thumper> sinzui: around?
#launchpad-dev 2010-04-28
<mwhudson> thumper: want to review a testfix branch?
<thumper> sure
<mwhudson> thumper: https://code.edge.launchpad.net/~mwhudson/launchpad/db-devel/+merge/24270
<thumper> mwhudson: done
<mwhudson> thumper: thanks
<thumper> mwhudson: I have my breadcrumbs working
<mwhudson> thumper: woo
<thumper> mwhudson: and I'm going to refactor one of the existing tests
<mwhudson> thumper: i'm sure there's a joke to be made about a fairy tale in here somewhere...
<thumper> :)
<mwhudson> time for lunch
<mwhudson> ... and breakfast, which somehow got lost today
<thumper> :(
<thumper> my breadcrumbs works as long as you don't look for a person or product :)
<mwhudson> ah poo, it seems like there's a database patch in db-devel before my no-hosted-area landing, so i'll have to wait for a db restore before i can qa it :(
<wgrant> mwhudson: Why does that require a DB restore? Wouldn't that just need the patch to be applied when it next updates?
<mwhudson> oh, do we do that?
 * mwhudson hopes so
<wgrant> It would make a lot of sense.
<wgrant> The last DB reimport was on 9274->9275, which had no schema changes, so it probably isn't triggered by schema changes.
<thumper> mwhudson: I was under the impression that we were at least going to try that
<thumper> mwhudson: best to ask a losa though
<wgrant> Have we lost lots of tests again, or has the test suite really dropped to around 3:15 on EC2?
<mwhudson> wgrant: it
<mwhudson> wgrant: it is a lot faster since stub removed the commits from the test factory
<wgrant> I knew that made it faster... but I didn't think it would be nearly 25% faster.
<wgrant> Impressive.
<thumper> :(
<thumper> mwhudson: my breadcrumb change has found problems in other people's breadcrumb tests
<thumper> now to find out if they are real or not
<thumper> wgrant: yes, it is all those unneeded transaction.commits
<thumper> wgrant: they slowed down test.tearDown a lot
<wgrant> Yeah, I guess they would.
<mwhudson> thumper: somehow i am not very surprised
<thumper> down to 1 failure and 4 errors
<thumper> 3 translations 2 bugs
 * thumper grrs
<thumper> TranslationUnavailable: Translations for this release series are not available yet. is one error
<thumper> NotFound: Object: <lp.translations.model.potemplate.POTemplateSubset object at 0xed26750>, name: u'template'
<thumper> is the other two template errors
 * thumper digs
<thumper> I'm now ignoring the traversed objects they pass in
<thumper> and determining it from the url
<thumper> so not entirely surprising
<thumper> can I land a deprecation error?
<thumper> as in will it kill the test suite?
<wgrant> It will kill things if it shows up in script output. salgado supressed lots of those this morning for the 2.6 migration.
<wgrant> But if it lands somewhere that doesn't have its stderr checked, the test suite will be happy.
<wgrant> (only the people will be unhappy)
<thumper> I want people to be unhappy
<mwhudson> thumper: for the allow-import-password thing, what do you think of this text? http://people.canonical.com/~mwh/subversion-text.png
<mwhudson> i'm not very happy with it
<thumper> You can inclue a username and password as part of the url, but this will be
<thumper> displayed on the branch page.
<thumper> what about that?
<mwhudson> yeah, that seems better
<thumper> One of the breadcrumb bugs is just plain wrong test
 * thumper shakes his head
 * thumper is sad
 * thumper wants to delete the bugs breadcrumb privacy test
<mwhudson> hmm
<mwhudson> bzr+ssh://bazaar.launchpad.net/~launchpad-pqm/dulwich/devel/ has 129 inconsistent parents
<poolie> spiv, I wish Launchpad gave out xbox-style achievements
<poolie> like when you file 50 bugs and get someone else to fix them
<poolie> or do 50 reviews
<thumper> poolie: and you want it submitted for facebook for you?
<poolie> wbn
<poolie> but just showing them inline would be enough
<poolie> you could almost build it using apis and feeds
<poolie> but it might be a bit anonying
<thumper> :(
<thumper> I found a bug in the canonical_url of bug comment
<thumper> I wish I could say skip and inform the bug team
<thumper> poolie: I was hoping to get this finished before our call but it doesn't look like it is going to happen
<thumper> poolie: want to do a call now?
<poolie> sounds good
<mwhudson> spm: ping
<spm> mwhudson: yo
<mwhudson> spm: is it possible for you to guess for me how long it will take r9306 of db-stable to get onto staging?
<spm> mwhudson: no idea. < 48 hours, all going well :-)
<mwhudson> spm: staging is currently on r9302 but r9303 includes a database patch
<spm> then yeah, at least a full day.
<mwhudson> spm: i leave the launchpad team in 48 hours and no-hosted-area _really_ needs serious QA
<mwhudson> spm: is there anything we can do to speed it up? :/
<spm> mwhudson: this week? not so far as I'm aware of. :-/
<mwhudson> oh well
<adeuring> good morning
<jtv> noodles775: good morning!  Delegating deletion of BuildFarmJobs is complicated a bit by the introduction of BuildFarmJobDerived.
<noodles775> jtv: in what way?
<noodles775> jtv: btw, let me know if a call is easier.
<jtv> noodles775: well I wouldn't even have noticed if it weren't for the fact that buildfarmjobbehavior.txt does builder.currentjob.destroySelf()
<jtv> noodles775: IRC is easier because chances are that at some point wgrant will notice and solve it all in one offhand remark.  :-)
<noodles775> lol, yep.
<jtv> What builder.currentjob.destroySelf() does in the current codebase is delete the BuildQueue entry (the currentjob); then Store.of(currentjob.specific_job).remove(currentjob.specific_job); and finally delete the Job.
<jtv> I replaced the Store.of(...).remove(...) with a call to a BranchJob method that does Store.of(self).remove(self), and in this test, _that_ now fails with exactly the error I had in practiceâi.e. self has no store.
<jtv> Does that mean I'm supposed to delegate to IBranchJobDerived instead of IBranchJob?
<noodles775> jtv: I don't understand 'I replaced... with a call to a BranchJob method'? Can you point me to a branch or WIP MP?
<jtv> Yup, that does resolve the test failure.
<noodles775> Great.
<jtv> noodles775: haven't done up the MP yet... I was just unaware of the BranchJobDerived change.
<wgrant> So you're going to have double delegation?
<jtv> wgrant: no, I'm just moving the cleanup method from BranchJob to BranchJobDerived.
<noodles775> But you shouldn't normally be delegating to IBranchJobDerived (I wouldn't think). That's a code thing (the pattern of which I'm using for the BuildFarmJob stuff).
<wgrant> Ah.
<noodles775> Ah... that makes more sense.
<jtv> noodles775: do I need to make provisions for IBranchJob implementations that aren't BranchJobDerived?
<noodles775> jtv: I didn't create the BranchJobDerived classes (I'm just using the pattern), but afaiui, you should inherit from BranchJobDerived if you want to automatically *delegate* IBranchJob (overriding anything as necessary).
<noodles775> So I think your question implies that you're creating an implementation of IBranchJob that doesn't delegate?
<jtv> noodles775: any idea who did create the BranchJobDerived classes?
<jtv> No, I'm asking if there are any implementations of IBranchJob that don't delegate.
<noodles775> jtv: I assume one of the code guys, but annotate should show you.
<jtv> yes
<noodles775> jtv: I'd assume that there aren't any though... why do you ask?
 * noodles775 has a quick look.
<jtv> Because I need to know whether there is any point in my design allowing for it.
<jtv> Or whether there _would_ be, rather.
<noodles775> jtv: so the only mention of IBranchJob I can see is your IBuildFarmBranchJob (implemented by TranslationTemplatesBuildJob).
<jtv> noodles775: ironically, "bzr annotate" shows just "michael" for the Derived changes...  I guess that's mwhudson then.  :)
<noodles775> yep (well, for the BranchJobDerived changes) :)
<jtv> Did I say BranchJobDerived?  I meant BuildFarmJobDerived.  Sigh.
<noodles775> jtv: ok, that is the one that I created (using the same pattern as BranchJobDerived).
<jtv> Confusion reigns.
<jtv> That's what I meant to ask, sorry.  But BranchJobDerived is so much easier and more familiar that it seems to have taken over my brain and fingers.  :)
<noodles775> Aha. So is it working now that you've moved your method into BuildFarmJobDerived?
<jtv> noodles775: so, to fix all that up...  Would there be any point in the delegation of BuildFarmJob cleanup accommodating specific_jobs that were not BuildFarmJobDerived?
<jtv> Right.
<noodles775> jtv: Nope.
<jtv> Great.  Saves me some other potential complications as well.
<noodles775> Excellent.
<bigjools> morning
<jtv> noodles775: something puzzling here... why did getByJob move from a utility class into IBuildFarmJobDerived?
<noodles775> jtv: it was only used by classes inheriting from IBuildFarmJobDerived... from memory.
<jtv> noodles775: but doesn't that imply that you need an IBuildFarmJobDerived in order to find one?
<noodles775> jtv: it's a class method.
<jtv> noodles775: not in IBuildFarmJobDerived it's not.
<jtv> Interface-wise, it went from "class provides" to "implements."
<jtv> Which means that it's suddenly not accessible from the utility.
<noodles775> jtv: Sorry, it is on the implementation (BuildFarmJobDerived)... I'd forgotten to indicate that on the iface.
<noodles775> Just otp... back in a few mins.
<noodles775> jtv: ok, back. So the mistake was that the IFace does not define it as a classmethod, or was there more?
<jtv> noodles775: frankly I don't even know whether an interface _can_ specify a class method in implementing classes... you may need a separate interface, and classProvides it instead of implements it, like it was before.
<noodles775> jtv: sorry, I'm not seeing what the problem is? It is still provided by the class as it was before, it just moved from being on a separate interface (and using classProvides()). The branch that that change landed with still used getUtility(IBuildQueueSet).getByJob().
 * noodles775 tries that in a harness.
<jtv> noodles775: this isn't the IBuildQueueSet.getByJob though, I think.
<jtv> (I must admit I get easily confused with this stuff)
<noodles775> yeah, my brain is swimming atm.
<jtv> I worked around it by invoking getByJob directly on the class, without going through the utility.
<jtv> Right now I'm hitting a very different problem: when the changes from BuildQueue.destroySelf get flushed, BuildQueue still holds a fk to Job.
<jtv> Which is odd because the BuildQueue did delete itself first.
<noodles775> jtv: looking at the diff where I updated getByJob, i didn't modify any call-sites, so it seems the class has always been used to access the method, rather than a utility?
<jtv> noodles775: maybe it's the conflation with BranchJob confusing things again...
<noodles775> aha. Did you sort out your foreign key issue?
<jtv> Not yet  :(
<wgrant> Hmm. db-devel loggerhead crashes when I try to log in.
<jtv> noodles775: it's as if Store.of(branchjob).remove(branchjob) is calling branchjob.destroySelf(), which in turn deletes the Job before its time.  :(
<noodles775> jtv: can you create a WIP MP?
<jtv> noodles775: OK
<wgrant> Hmmm. I was going to add a print statement to debug this issue. But one is present in db-devel (lib/launchpad_loggerhead/app.py)! This does not bode well.
<jtv> noodles775: https://code.edge.launchpad.net/~jtv/launchpad/bug-569108/+merge/24297
<jtv> noodles775: note how I can't call BranchJob.destroySelf, because that tries to destroy self.job, which we can't destroy yet because BuildQueue still refers to it at that point.
<jtv> but wait... the order to delete the BuildQueue comes in even before that.
<noodles775> jtv: just otp again, but will look straight afterwards.
<jtv> thanks
<wgrant> allenap: Hmm, so that's twice it's vanished?
<allenap> wgrant: Yes. I'm a bit grumpy about that. There seems to be a lot of this about at the moment.
<wgrant> allenap: Indeed :(
<wgrant> lp:~wgrant/launchpad/fix-mailman-testrot also probably disappeared, although it's so long ago that I don't remember.
<thumper> jtv: I'm being nice to translations
<jtv> thumper: ?
<thumper> jtv: your breadcrumb test sucks less than that others
<jtv> thumper: strong praise
<jtv> do we have one?
<allenap> wgrant: I'll submit that too, see if one of them sticks :)
<jtv> or are you saying we don't?
<wgrant> allenap: Thanks.
<thumper> jtv: you do, and they are reasonably good
<thumper> jtv: I've refactored the base breadcrumb test case
 * jtv tries to swap the subject back in
<jtv> noodles775: found it
<noodles775> jtv: great... what was it?
<jtv> noodles775: this is why strong typing is sometimes a good idea, and naming things after their type is often a bad idea
<jtv> buildqueue wasn't a BuildQueue.
<noodles775> Ah.
 * thumper mops brow
<thumper> that's all the breadcrumb tests fixed
<thumper> in the vetinary sense
<jtv> thumper: neutered..?
<thumper> put down :)
<jtv> thumper: hope your kids aren't there to hear that...
<thumper> ââ¹
<thumper> I should have stopped by now :(
 * thumper is done now
<thumper> branch is in ec2
<thumper> 1145 lines? no wonder if felt long
<thumper> no wikkid hacking for me tonight
<thumper> need to sleep before the TL call
<wgrant> jtv: I think you might be confused about the slave's buildid handling.
<wgrant> 'build ID' may be an overloaded term, but 'chroot checksum' is not among its meanings.
<jtv> wgrant: but the one I'm pointing at is definitely the chroot checksum.
<jtv> So I'm saying it's _too_ overloaded in this case, probably by accident.
<wgrant> jtv: The only thing I can see xmlrpc_build calling with the chrootsum as an argument is BuildManager.initiate.
<wgrant> Which takes a chroot checksum in the expected position.
<jtv> wgrant: exactly
<jtv> ...and stores it in self._buildid
<jtv> ...after which it gets used all over the place, sometimes as an arbitrary id, sometimes in dealing with the chroot tarball.
<wgrant> jtv: _buildid is set only in __init__(), not initiate()
<jtv> wgrant: then I may be wrong about how it gets there, but look at the logs: the chroot hash is all over the place
<wgrant> And xmlrpc_build calls __init__ like "self.slave.startBuild(self._builders[builder](self.slave, buildid))"
<wgrant> jtv: Hm, they've always looked fine for me. I haven't used it in the new Era of Cookies, but it all used to look sane.
<jtv> wgrant: where do the BuildManager classes get instantiated anyway?
<wgrant> jtv: The expression that I quoted three lines ago.
<jtv> wgrant: actually it's quite possible that I'm wrong about this...  but then that means we're seeing the cookie all over, which would probably be a bad thing
<wgrant> jtv: You should expect to see the cookie in quite a few places.
<deryck> bigjools, ping
<bigjools> deryck: helleau
<deryck> BjornT, ping.
<BjornT> hi deryck
<Ursinha> danilos, sure :)
<danilos> Ursinha, here are the people who can probably help you more than I can :)
<bac> reviewers meeting beginning
<jelmer> danilo: I wasn't the one updating lp:~launchpad-pqm/bzr-git/devel, so I guess that must've been mwhudson
<jelmer> danilos: In that case, all that's left to do is update sourcedeps.conf
<danilos> jelmer, right, it must simply be very approximate timing then :)
<danilos> jelmer, cool, can you do that or should I take care of it?
<jelmer> danilos: If you can do it that'd be great, I've got enough pending branches as is.
<danilos> jelmer, heh, who doesn't :) sure, I'll take care of it then
<jelmer> danilos: it should be a matter of updating utilities/sourcedeps.conf to point at the latest tip and firing that off to ec2
<danilos> jelmer, yeah
<mrevell> Night
<jml> thumper: I thought you might like to look at http://www.wikimatrix.org/
<jml> james_w, I've got your wadllib patch down as something I'd like to look at
<jml> james_w, but realistically, that's unlikely to happen for another three weeks.
 * jml stops for the day
<wgrant> Is PQM really open?
<mwhudson> wgrant: yes
<wgrant> I've two branches that were rejected during last night's testfix... and one that disappeared in EC2 (\o/)
<thumper> wgrant: I had one ec2 run disappear too
<thumper> wgrant: although I caught it before it killed itself
<thumper> wgrant: stuck in windmill tests
<thumper> wgrant: pqm closes in approx 24 hours
<wgrant> Hm, I thought it was closing earlier.
<thumper> wgrant: that is earlier
<thumper> wgrant: normally it closes in approc 48 hours
<wgrant> Ah. oops.
<wgrant> mwhudson: Hi.
<mwhudson> wgrant: hi
<wgrant> mwhudson: So, I tried to get codebrowse running on db-devel last night... and it was crashing when I tried to authenticate. So I looked in launchpad_loggerhead and tried to add a debug print... but it was already there (app.py:113).
<wgrant> This has me suspicious that it's actually broken, and you were trying to debug it, and then it landed broken with the debug print.
<mwhudson> wgrant: ah yes
<mwhudson> wgrant: i think the problem is that testopenid.dev is a bit lame
<mwhudson> you're right that the debug print shouldn't be there though
<mwhudson> wgrant: afaict testopenid doesn't support the sreg extensions codebrowse needs
<mwhudson> wgrant: so, uh, i hope it'll work on staging
<wgrant> Sounds easy enough to fix.
 * wgrant tries.
<wgrant> But yes, hopefully it will work on staging...
<mwhudson> wgrant: browsing private branches is totally broken on prod right now
<wgrant> I wondered if that was related.
<mwhudson> (i'm surprised that noone's tried to burn my house down)
<mwhudson> wgrant: it's not, it's a launchpad branch that removed an api launchpad-loggerhead used
<wgrant> Ah.
<wgrant> https://code.edge.launchpad.net/~wgrant/launchpad/fix-mailman-testrot/+merge/23475 and https://code.edge.launchpad.net/~wgrant/launchpad/bug-565739-dont-retry-superseded-builds/+merge/23622 passed EC2 overnight (the latter took three tries to not disappear in EC2...), but were rejected in the testfix. allenap said he'd pqm-submit them, but it looks like he didn't get around to it. Can someone please deal with them, if they have a moment?
<wgrant> https://code.edge.launchpad.net/~wgrant/launchpad/delete-more-stuff/+merge/24280 also disappeared in EC2 overnight, so it would be nice if someone could try it again...
<mwhudson> wgrant: ok
<StevenK> wgrant: I had the same problem yesterday with EC2 instances buggering off.
<mwhudson> wgrant: two pqm-submits done
<wgrant> mwhudson: Thanks!
<wgrant> StevenK: Yeah, it's been happening for a while :/
<mwhudson> wgrant: and the ec2 land is firing up
<StevenK> wgrant: But now the branch has failures in windmill tests ... :-(
<wgrant> mwhudson: Even better.
<wgrant> mwhudson: Where are you escaping to, BTW?
<mwhudson> wgrant: can't say in public yet
<wgrant> Ah.
<mwhudson> wgrant: i'm sure you've been paying enough attention for you to know what that means though
<wgrant> You're not as leaky as you used to be, although I have some ideas :P
#launchpad-dev 2010-04-29
<thumper> wgrant: here is my review pedantry: doctests should have four spaces before >>>
<wgrant> thumper: But existing code has 2, 3 or 4. I am never sure whether to go with consistency, fix it, or be inconsistent.
<wgrant> Which should I choose?
<thumper> fix the test
<wgrant> OK, will do so in future.
<thumper> thansk
<thumper> I'm sure we have this documented somewhere
<thumper> wgrant: if I'm using launchpadlib to load something
<thumper> wgrant: what is the base url I give it?
<wgrant> thumper: In a test, or in the real world?
<thumper> QAing against edge
<wgrant> https://api.edge.launchpad.net/$VERSION/
<wgrant> Or you pester people to fix the bug.
<wgrant> Bug 524775
<mup> Bug #524775: Launchpad.load doesn't like relative urls <Launchpad Foundations:Triaged> <launchpadlib :Triaged> <https://launchpad.net/bugs/524775>
<thumper> :)
<thumper> what version should I be using? 1.0?
<wgrant> 1.0 shouldn't have any new APIs, although it does.
<wgrant> Use devel.
<lifeless> I gotta say, isn't my hack correct?
<lifeless> so fixing == write a test to confirm?
 * mwhudson afk briefly to post a letter
<Ursinha_> how do you suggest testing scripts that use launchpadlib?
<lifeless> Ursinha_: by prayer
<Ursinha_> lol
<Ursinha_> lifeless, that was my fear
<wgrant> Maybe someone needs to write some infrastructure to do it.
<wgrant> What with LP being open source and all, there's no reason it can't be done.
<lifeless> the challenge is that launchpadlib is all dynamic
<wgrant> Hm?
<lifeless> Ursinha_: there are two basic things; 'are you using launchpadlib as it expects to be used' and 'does your code do what you wanted it to, still'
<lifeless> Ursinha_: the latter, you can use mocks for fairly effectively, if you like mocks.
<lifeless> the former, you cannot ever really test without using live launchpad, because it can change without warning.
<wgrant> There's no reason we can't have an easy way to write launchpadlib script tests that run against a local LP instance.
<lifeless> wgrant: indeed, but that will be heinously slow
<Ursinha_> lifeless, I'm considering using mocks for the second, but I was wondering if it was possible to test the former
<wgrant> lifeless: Better than having dodgy tests.
<wgrant> (slightly)
<lifeless> I'd consider using the same introspection launchpadlib does - compiling the wadl - to compile a set of fake objects
<lifeless> be nontrivial to do
<lifeless> wgrant: many of my projects complete their *entire* test suite in the time it takes to start up launchpad.
<lifeless> bzr is the notable exception.
<lifeless> Ursinha_: pragmatically, today, I'd have a suite of non-default tests that talks to staging
<lifeless> and mock everything else.
<Ursinha_> lifeless, hm, right
<mwhudson> grunk, one ec2 test run stalled
<mwhudson> and two which have failed in lp.code.model.tests.test_diff.TestDiffInScripts.test_fromFile_withError
<mwhudson> for a grand total of 0 out of 3
<mwhudson> wgrant: it's yours which has hung in ec2 fwiw
<mwhudson> ~wgrant/launchpad/delete-more-stuff
 * mwhudson shoves wgrant's branch back into lp
<mwhudson> ec2 rather
<wgrant> mwhudson: EC2 seems to really love my branches at the moment.
<wgrant> One that you landed earlier took three attempts, the other one two, this one at least two...
<mwhudson> it seems to be getting worse indeed
 * thumper is sad that staging hasn't updated yet
<thumper> given my late night hacking last night
 * thumper EODs
<wgrant> staging's been updating for more than 24 hours now... it must be almost done.
<spm> wgrant: you optimist you
<wgrant> I mean, it's not dogfood.
<stub> Is someone loadtesting against login.staging.launchpad.net?
<adeuring> good morning
<stub> Do we have anything available to parse an ISO8601 timestamp into a Python datetime?
<mrevell> Morning all
<jml> As a great man once said
<jml> Hello, world!
<jml> stub: you mean YYYY-MM-DD HH:mm:SS?
<jml> stub: I think so
<wgrant> The size of the diff (16938 lines) is larger than your specified limit of 1000 lines
<wgrant> What?
<jelmer> wgrant: when do you get that?
<wgrant> db-devel r9315
<wgrant> Which doesn't seem like it should be so absolutely gigantic.
<wgrant> Hmm.
<wgrant> The sampledata ordering is all different.
<wgrant> But I thought the script ordered it sanely!
<jtv> allenap: I think I'm probably hitting the same segfault you are
<jtv> No output from EC2, and running locally with output redirection loses the critical tail of the output.
<allenap> jtv: That sounds similar.
<wgrant> I've had five instances disappear in the last week.
<allenap> jtv: The way I confirmed it is to pdb.set_trace() in spawn_layer_in_subprocess() to get the exact command line args for the sub-process, then run that from the command line.
<jtv> allenap: do you know what test(s) it happens in?
<allenap> jtv: I triggered the segfault in my branch; it was from new code. I don't know about existing code.
<jtv> allenap: strange...
<jtv> allenap: running some local "make check"s to see where the problem happens
<stub> jml: Yer, but I'm using an interval now so no worries. Closest I found was the datetime type I added to the OptionParser that knew about various formats, which we could factor out one day.
<jml> hello
<mwhudson> jml: good morning
<jml> mwhudson, hi
<jml> mwhudson, I cannot help but notice that you are here
<mwhudson> jml: emma and i are sitting on the sofa, a laptop on our laps each
<mwhudson> it's very sad :)
<jml> ahh, the married life
<ajmitch> heh
<jml> I just got back from over an hour of internet downtime because the phone rang and the router was physically inaccessible for rebooting
<jml> mwhudson: how's the hosted/mirror thing panning out?
<mwhudson> jml: staging is terrible
<mwhudson> jml: it might update soon
<mwhudson> jml: but it's landed and, uh, i haven't found any problems with it yet
<jml> mwhudson: it's pretty exciting
<mwhudson> jml: yeah, i'd hoped for more than 1 day to qa it
<mwhudson> i hope i don't find too many problems ...
<jml> mwhudson: and a strong testament to avoiding bduf
<mwhudson> jml: bduf?
<jml> mwhudson: big design up front
<mwhudson> jml: yes
<jml> sinzui, bac, EdwinGrubbs: does this look like a fairly complete list of features? https://dev.launchpad.net/FeatureChecklist/Registry
<jtv> allenap: about the problem you ran into... any idea what caused the bad data to be fed to storm?
<jtv> PQM just cleaned out all of a sudden... did we go into testfix?
<allenap> jtv: User error :)
<jtv> allenap: do I need to resubmit?
<allenap> jtv: No, that was the answer to your earlier question.
<jtv> Ah
<jtv> Well, we do have a failed compile...  I'm having a look
<jtv> bigjools: I think your fix fell out of PQM somehow
<jtv> PQM> Conflicts during merge: Text conflict in utilities/sourcedeps.conf
<jtv> flacoste: did your sourcedeps.conf update for loggerhead get caught in a stable/db-devel merge conflict?
<didrocks> jml: you are coming to RMLL in France? :)
<jml> didrocks: I think I'm speaking at it
<jml> didrocks: but I've been a bit lazy w/ my submission
<didrocks> jml: well, I see we have a conference the same day ;)
<jml> didrocks: does it clash w/ something at Canonical?
<didrocks> jml: oh no, I'm going there on holidays, giving two conferences (bughugger and quickly) and being at ubuntu-fr booth
<didrocks> well, if I say that I'm going there "for" bughugger, maybe rick will accept that I don't go there only for holidays ;)
<wgrant> bigjools: Um, won't PPA deletion not work until 10.04 is on germanium?
 * bigjools taps side of nose sagely
<wgrant> Heh.
<bigjools> I cherrypicked the backend
<wgrant> Ah.
<wgrant> Handy.
<bigjools> gives a few days of edge fun
<deryck> BjornT, ping
<bigjools> wgrant: btw did you manage to QA any of your stuff this cycle?
<wgrant> bigjools: 'fraid not. Far too much uni assessment this week... but it's all over now, so I'll organise that tomorrow.
<bigjools> wgrant: no worries, I'm going to start hitting the QA trail of doom this arvo
<BjornT> hi deryck
<jml> didrocks, ahh cool
<jml> didrocks, I wonder how much French I can learn between now and then.
<didrocks> jml: we can train you at UDS ;)
<jml> didrocks, je parle francais comme un vache espagnole
<didrocks> jml: that's a typical French expression, it's perfect. Sure you can even order food with that :)
<jml> haha
<bigjools> jml: mais, ou sont les baggages!
<didrocks> bigjools: I hope jml won't need that sentence ;)
<bigjools> didrocks: oui!
<bigjools> Ma grandmere est flambÃ©e!
 * jml chuckles
<didrocks> hum, "une crÃªpe" can be "flambÃ©e", you don't tell that for your "grand mÃ¨re" ;)
<bigjools> You could learn more if you watch Eddie Izzard's videos :)
<sinzui> :)
<sinzui> bigjools, didrocks: http://www.youtube.com/watch?v=x1sQkEfAdfY
<didrocks> let me have a look :)
<didrocks> "room #42" ;)
<didrocks> we have the same in French with "where is my umbrella?" and "where is Brian?" :)
<bigjools> heh
<bigjools> ou est la plume de ma tante?
<didrocks> yeah, that's funny :)
<flacoste> mthaddon: what happened to staging restore?
<mthaddon> hmm, has been creating the slave since 2010-04-28 18:23:11
<barry> leonardr, gary_poster i can't remember where i posted about this but i use a fake datetime factory that allows me to produce predictable dates when in testing mode.  maybe something like that is useful?  (it's in the mailman 3 source code if you want to look)
<leonardr> barry: is this your way of telling me you've discovered a horrible date-related test failure? :)
<gary_poster> on call but sounds interesting barry
<barry> leonardr: no, just a clever way to mock datetimes so you can do fun stuff like fast-forwards and stuff
<leonardr> ah, cool
<leonardr> i can think of a place where that would be useful, actually
 * barry searches for a link
<barry> leonardr: http://bazaar.launchpad.net/~mailman-coders/mailman/3.0/annotate/head%3A/src/mailman/utilities/datetime.py
<Ursinha> Chex, gary_poster, rockstar, bigjools, danilos, sinzui, allenap: production meeting on #launchpad-meeting @ Freenode in 40 minutes
<leonardr> barry: thanks
<bigjools> Ursinha: let;s talk about bots
<Ursinha> bigjools, no, you ignored me the whole week :P
<bigjools> Ursinha: did not!
<Ursinha> bigjools, can we discuss that after the meeting? I'm kinda preparing for that now :)
<bigjools> Ursinha: it'll be too late after the meeting, unless you make it a quick meeting ;)
<Ursinha> bigjools, you should have answered when I first pinged you about it :P
<Ursinha> bigjools, we can talk about that tomorrow, is that ok for you?
<bigjools> Ursinha: I'm pretty sure I did, but you ignored me :)
<bigjools> anyway, yes, what time?
<Ursinha> 1h30 earlier than now? :)
<bigjools> Ursinha: it's a date
<Chex> Ursinha: thank you for reminder
<Ursinha> Chex, no problem :)
 * bigjools cries at an extra Popen in a test in wgrant's branch
<wgrant> Is there a better way to test scripts like that?
<Ursinha> hm, just saw sinzui's email, EdwinGrubbs, are you able to attend the prod. meeting?
<EdwinGrubbs> Ursinha: yes, is it a conference call or on irc?
<Ursinha> EdwinGrubbs, on irc, #launchpad-meeting
<Ursinha> EdwinGrubbs, thanks :)
<mars> jml, just looking at the nifty datetime factory barry created above.  twisted and bzr have some nifty stuff too.  Thought you might know: is there a central resource for all these cool python testing power tools?
<jml> mars, launchpad.net/pyunit-friends
<mars> jml, thanks!
<jml> mars, and the lp:testtools project specifically.
<jml> barry, mars, what's the point of using that rather than just passing in 'now' as an optional parameter?
<barry> mars, jml maybe even the testing-in-python mailing list
<jml> oh yeah
<jml> definitely t-i-p
<barry> jml: cleaner apis :)
<jml> barry, because now they take a mandatory date factory?
<barry> jml: no, nothing takes a factory.  any code that needs a datetime imports the factory and uses that to create it instead of datetime.datetime.now()
<jml> :(
<jml> barry, using mutable globals isn't cleaner
<jml> although I'm willing to admit it's a matter of taste
<barry> jml: neither is passing now arguments up and down a call stack.  i'm not saying it's pretty but it's a trade-off.  one i like only slightly more
<jml> I really like functions and parameters.
<barry> jml: eibti
<jml> yeah, that's what I mean
<Ursinha> Chex, gary_poster, rockstar, bigjools, danilos, EdwinGrubbs, allenap: production meeting on #launchpad-meeting @ Freenode in 15 minutes
<barry> i get that.  not (totally ;) disagreeing with you.  still, it's better than monkeypatching datetime :)
<jml> barry, yes, it's better than monkey patching datetime
<gary_poster> Ursinha: I've asked matsubara-lunch to stand in for me
<gary_poster> matsubara-lunch: that's your confirmation ;-)
<Ursinha> gary_poster, sure, thanks for the info
<gary_poster> thank you
<Ursinha> Chex, matsubara, rockstar, bigjools, danilos, EdwinGrubbs, allenap: production meeting on #launchpad-meeting @ Freenode now :)
<danilos> Ursinha, thanks
<cody-somerville> I'm getting lazr.restfulclient.errors.HTTPError: HTTP Error 503: Service Unavailable when I try to ec2 land my branch.
<cody-somerville> weird. I'm getting the launchpad offline page in response to whatever ec2 land is trying to fetch from launchpad.
 * cody-somerville tries again.
<leonardr> bac, gary: bug 569189 cause has been revealed
<mup> Bug #569189: Authenticated users in launchpadlib tests have no permissions <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/569189>
 * bac awaits answer with suspense
<leonardr> bac: sorry, there was an implicit 'check out the bug'
<bac> did
<bac> thanks
<leonardr> bac: as a workaround, you could grant your user WRITE_PRIVATE access. that should make the test work
<bac> ok, i'll try that.  thanks
<gary_poster> bac, leonardr I'm inclined to mark https://bugs.edge.launchpad.net/launchpad-foundations/+bug/569189 as Invalid on the basis of Leonard's evaluation.  Alternatively, we could convert it into a bug for the code team, to change their permissions somehow or other, but my preference is for Invalid (again, unless I misunderstand).
<mup> Bug #569189: Authenticated users in launchpadlib tests have no permissions <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/569189>
<leonardr> gary: mark it invalid, but let's also let the code team know about it so they can fix it if they want
<gary_poster> leonardr: ok will do
<jml> gn'ight all
<leonardr> barry, do you have a minute to see whether i'm crazy?
<leonardr> i need someone to branch lp:~leonardr/lazr.restfulclient/delete-entry, buildout, run the tests, and tell me if you get any failures
<EdwinGrubbs> Ursinha: can you send me the distributionmirror-prober error message?
<Ursinha> EdwinGrubbs, it's in the script-failures mailing list
<Ursinha> EdwinGrubbs, just a moment
<sinzui> EdwinGrubbs, Ursinha there was no error. It is running late.
<Ursinha> sinzui, this explains why I'm not finding the email on the lp-error-reports list
<lifeless> mmmm bacon
<lifeless> thumper: so, shall we chat ?
<mwhudson> staging is down :(
<mwhudson> why?
<mwhudson> losa ping
<lifeless> thumper: ping
<Chex> mwhudson: hi there
<mwhudson> Chex: what's up with staging?
<lifeless> thumper: ping
<mwhudson> lifeless: he's probably doing the school/kindy run now
<lifeless> mwhudson: isn't that 'load catapult(); fire();' ?
<thumper> lifeless: the foundations behind my problem with using queued too much right now is that there isn't the correct security checks around it
<thumper> lifeless: I really want to split out the reviewed_status from the queued_status
<thumper> lifeless: with your current branch, anyone could take something from needs review to merge failed
<thumper> lifeless: from there they could queue it
<thumper> bypassing the approval
<lifeless> thumper: I don't think thats true. I'll add a test to prove it happily.
<thumper> lifeless: it is true
<thumper> lifeless: setStatus(merge_failed) has no checks other than launchpad.edit
<thumper> lifeless: which the proposer has
<thumper> lifeless: and the source branch owner
<thumper> we really need to split out the reviewed status
<thumper> I don't think it is too much work (fingers crossed)
<thumper> lots of fiddly bits though
<thumper> I think what I want to do is to remove all the special case set status methods
<thumper> and have two new ones: setReviewStatus and setQueueStatus
<lifeless> thumper: there is a check that only official reviewers can queue things
<thumper> lifeless: or if it has been approved
<lifeless> right
 * lifeless digs up the patch, vm starting
<cody-somerville> Is anyone else getting ImportError: No module named tickcount on make?
<lifeless> thumper: so, I don't want to increase the scope right now; happy to participate in optional cleanups.
<lifeless> thumper: your primary concern is 'non reviewers should not be allowed to set merge failed', right ?
<thumper> lifeless: that's good enough for now
<thumper> lifeless: I'd like to get this fixed soon
<lifeless> because merge failed permits toggling back to queued
<thumper> right
<lifeless> ok, my patch enforces that
<lifeless>      elif (next_state in (code_approved, queued) and
<lifeless> -          from_state not in (code_approved, queued)
<lifeless> +          from_state not in (code_approved, queued, merge_failed)
<lifeless> in the existing state machine transition check
<lifeless> first hunk in lib/lp/code/model/branchmergeproposal.py
<lifeless> thumper: ^
<lifeless> I'll triple check though
<lifeless> ok, its not quite right; adding test and fixing.
<thumper> ah, missed that
<lifeless> is it fair to say the branch owner is an implicit reviewer ?
<lifeless> thumper: ^
<thumper> target branch owner should be
<lifeless> yes
<lifeless> just checking my language
<thumper> lifeless: you should be aware that we are likely to break the way that PQM deals with queued proposals
<thumper> lifeless: in the changes we want to do
<lifeless> keep me in the loop, I'll update as needed
<lifeless> what sort of damage do you have in mind ?
<thumper> api breaks most likely
<lifeless> thumper: I have a small question about the tests
<lifeless> assertAllTransitionsGood - that seems to test with a non reviewer
<thumper> sounds right
<lifeless> but
<lifeless> (checking stuff)
<thumper> lifeless: this isn't paged in right now, and I'm on a call, and about to run out to an appointment
<lifeless> sure
<lifeless> anyhow, I think the existing tests were making an odd claim - that any user can switch wip to any state
<lifeless> thumper: I have fixenated and pushed
<lifeless> ok -> .au
#launchpad-dev 2010-04-30
<wgrant> "Add a proportion of the maximum bug heat to a bug's heat for every day since the bug was created.
<wgrant> "
<wgrant> Isn't that the opposite of what one would normally want?
<thumper> wgrant: :)
<thumper> the proportion could be negative
<wgrant> That's true.
<wgrant> In fact it seems to add a quarter of the max heat divided by the number of days since creation.
<lifeless> thumper: ping
<thumper> lifeless: pong
<lifeless> hey
<lifeless> I'm on airport wifi
<lifeless> do you have any other issues with the patch ?
<lifeless> I realise its not a priority, but i have a plane flight where I can polish it, if more is needed
<thumper> I've not looked at it, chasing other issues
<lifeless> ok
<wgrant> lifeless: Running back already? :P
<lifeless> wgrant: speaking at devopsdays tomorrow morning
<lifeless> wgrant: then UDS
 * wgrant should probably work out what needs doing before UDS.
<lifeless> 'stuff'
<lifeless> wgrant: are you going to be at UDS ?
<wgrant> lifeless: I am.
<lifeless> wgrant: cool. And have you applied for the code team job position yet ?
<lifeless> :)
<wgrant> Heh, no.
<lifeless> -> .au - ciao.
<persia> .nz -> .au again already?  What it that uninspiring?
<wgrant> Heh.
<wgrant> Hm, Maverick doesn't exist yet :(
<persia> Probably won't until next week, I'd think.  Toolchain stuff is happening in PPAs (fear ye all who plan to run ports during early maverick)
<cody-somerville> what have I done wrong? why is this test failing like this? http://pastebin.ubuntu.com/424952/
<StevenK> cody-somerville: The test isn't expecting /beta/ in the resource links
<wgrant> No.
<wgrant> security_contact_link is in the wrong place in the test.
<wgrant> It should be before self_link, but you've put it after.
<wgrant> The rest of the diff is not real.
<StevenK> Oh, that bloody ... versus real-text garbage?
<wgrant> Yes.
<wgrant> It is worse than stupid.
<cody-somerville> wgrant, ty
<thumper> wgrant: how do I stop the automatically stated librarian and other thingy?
<thumper> wgrant: for tests
<wgrant> thumper: bin/kill-test-services
<wgrant> But if you want to run the whole test suite, you probably have to unset LP_PERSISTENT_TEST_SERVICES too.
<wgrant> I've found some of the Foundations tests check that the librarian shuts down and such, so fail when it's on.
<wgrant> Why do most project pages have a warning sign next to "Submit code", which I can do nothing about?
<mwhudson> probably because they "do not officially use launchpad for codehosting"
 * mwhudson is guessing
<thumper> wgrant: I don't know how you work with soyuz
<thumper> wgrant: example of warning?
<wgrant> thumper: See the involvement portlet on https://edge.launchpad.net/soyuz
<wgrant> You may have to be unauthenticated.
<wgrant> What is troubling you about Soyuz?
<wgrant> It's a whole lot better now than when I first worked it out, when nobody had run much of it locally for years...
<wgrant> The sole source on https://dogfood.launchpad.net/~ubuntu-chumby-hackers/+archive/ppa/+packages was a test recipe build, right?
<wgrant> I cannot find the log for it anywhere around that time in the librarian.
<wgrant> Was it not stored properly?
<wgrant> Grgh. There's no solution to http://launchpadlibrarian.net/46467230/wgrant-lintian-master.log, is there?
<mwhudson> no, i don't think so :(
<mwhudson> well apart from slapping bzr around a bit to be less picky
<wgrant> Hmm, I might do that rather than using git.
<persia> Should https://dev.launchpad.net/Getting work on lucid now, or would it be better to do it in an older chroot/VM ?
<mwhudson> persia: i've been developing on lucid since just after beta1 and not had any real issues
<wgrant> It will work fine on Lucid.
<wgrant> LP even works fine on Python 2.6 now, although it's not the default.
<persia> Thanks.
<thumper> I think we should force it to python 2.6
<thumper> so we can stop importing with from the future
<StevenK> from __future__ import braces
<wgrant> thumper: We can't until Lucid is in the DC.
<wgrant> But yes, that's the plan.
<thumper> ah
<thumper> ok
<cody-somerville> wtf
<wgrant> More doctest diff screwyness?
<wgrant> Or something more sinister?
<cody-somerville> lets say you have a bugtask that is assigned to a suspended user
<cody-somerville> lets say you have a script to generate a report about bugs in a project
<wgrant> Yes, you have to catch the 410.
<cody-somerville> running said script on project that has a bugtask that has an assignee that is suspended == FAIL
<wgrant> Yes, this is probably stupid.
<cody-somerville> PLUS I get a nice text/html response
<wgrant> Indeed.
<wgrant> thumper: So, what was your Soyuz objection?
* BjornT changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 3 of 10.04 | PQM is in release-critical mode | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes | staging is down!
<thumper> wgrant: I was reading tests
<wgrant> thumper: Ahahahahahahahahah fool.
<wgrant> Big mistake.
<thumper> wgrant: I had to fix a test
<wgrant> This is Soyuz.
<thumper> I wasn't doing it out of choice
<wgrant> I occasionally try to rip apart one of the multi-thousand-line doctests.
<wgrant> But then I realise that it's all intricately woven together, and removing just one bit will make the whole thing collapse into a steaming pile of Soyuz.
<wgrant> What?
<wgrant> You're killing off buildd-slavescanner.txt?
<wgrant> Really really really bad idea
 * wgrant looks.
<wgrant> buildd-slavescanner.txt is the main test that tells us that the house of cards is still standing :(
<wgrant> What was the failure?
<thumper> wgrant: it was a subtle text match failure
<thumper> wgrant: which I didn't have time to fix
<thumper> I'm not killing it off
<thumper> just disabling it briefly
<thumper> I expect a soyuz person to fix it
 * cody-somerville frowns.
<persia> Isn't that the bit that wedges all the buildds every so often?
<persia> Or rather, the workaround therefor?
<wgrant> persia: It is the thing that does all the talking to the buildds.
<wgrant> It tests that builds happen and work.
<wgrant> It is about the only test that does so :/
<persia> That's what I thought.
<cody-somerville> Well, I hope it doesn't mean Soyuz will be broken after the next deployment.
 * noodles775 feels he missed an important conversation?
<cody-somerville> noodles775, LP #572005
<mup> Bug #572005: Disabled test buildd-slavescanner.txt <Soyuz:Triaged> <https://launchpad.net/bugs/572005>
<StevenK> cody-somerville: It's a failed test, it doesn't mean the world is ending. And PQM is still open for testfixes
<thumper> plz disable soyuz, kthxbye
 * thumper EODs (for now at least)
<adeuring> good morning
<mrevell> Hello
<wgrant> bigjools: Morning.
<bigjools> g'day
<wgrant> bigjools: So, we need a script to populate lots of SPR rows with their changelogs. http://pastebin.ubuntu.com/425094/ is my first pass, and it seems to work OK. But I'm not sure how Storm is going to handle the ResultSet mutating over the several days that it will likely take to run.
<bigjools>  /o\
<bigjools> wgrant: I suggest you use process_in_batches and partial commits
<bigjools> can't hold txn locks for days
<wgrant> bigjools: I commit after each SPR.
<bigjools> ah ok - I can't read
<bigjools> if it bails out half way, will it restart in the right place?
<wgrant> I should probably read process_in_batches at some point.
<wgrant> Er, yes, if I hadn't omitted a clause from my rewritten query.
<wgrant> It was meant to have a changelog=None, so it will skip any that already have it set.
<bigjools> :)
<bigjools> we should ask stub to take a look
<wgrant> Probably.
<wgrant> http://pastebin.ubuntu.com/425099/ has the changelog=None.
<wgrant> stub: Does that script make you cry?
<wgrant> BjornT: A couple of weeks ago, Zopeless email was changed to use raw_sendmail, which appears to use the mail delivery utility, so I think it probably is being used to send email (re. bug #572077)
<mup> Bug #572077: Scripts are starting up the queued mail runner <current-rollout-blocker> <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/572077>
<wgrant> db-devel r9071.2.33, lib/lp/services/mail/sendmail.py
<wgrant> Actually, the db-devel r9205 diff for that file is better.
<BjornT> wgrant: oh, thanks. it's unclear whether it's it was an intentional change, or if he wanted to change test dev environment only.
<wgrant> I was suspicious that exactly this would happen, but I was assured it had been checked, so it might be irrelevant.
<BjornT> bigjools: can you take a look at bug 572077, and comment on whether this affects soyuz? at least in the past, soyuz used the fact that mail were sent even if the transaction was aborted.
<mup> Bug #572077: Scripts are starting up the queued mail runner <current-rollout-blocker> <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/572077>
<stub> wgrant: Ideally, it would be using DBLoopTuner to be database friendly etc. This would alleviate concerns with iterating over a huge result set for several hours too
<stub> wgrant: Looks fine apart from that. We can stick a time.sleep() to make it db friendlier if you can't be arsed switching it to looptuner format
<stub> wgrant: This is a run-once script or is it going to become a cron task?
<bigjools> BjornT: that is almost certainly the case
<wgrant> stub: Run once.
<wgrant> bigjools, BjornT: process-upload.py still relies on it to send rejection emails.
<wgrant> stub: New uploads are populated by the upload processor.
<bigjools> BjornT, wgrant: see https://bugs.edge.launchpad.net/launchpad-foundations/+bug/29744
<mup> Bug #29744: In Zopeless, mails should be sent only when the transaction is commited <email> <infrastructure> <tech-debt> <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/29744>
<bigjools> and
<bigjools> lib/lp/archiveuploader/uploadprocessor.py at line 409
<bigjools> BjornT: so basically we will lose upload rejection emails
<bigjools> BjornT: the change will have to be backed out
<wgrant> The change was only there in the first place to allow testing of email-generating scripts.
<wgrant> Which is a really good thing, but not if it breaks this...
<wgrant> s/testing/dev testing/
<BjornT> bigjools: agreed
<deryck> Morning, all.
<cody-somerville> who ever manages the launchpad-dependencies package should add postgresql-plpython-8.3 as a dependency.
<henninge> wgrant: ping
<wgrant> henninge: Hi.
<henninge> wgrant: Hi! ;)
<henninge> wgrant: I am trying out the build slave locally again, driging it through RPC
<henninge> I got as far as this:
<henninge> >>> proxy.status()
<henninge> ['BuilderStatus.WAITING', 'BuildStatus.OK', '1-1', {'translation-templates.tar.gz': '5e385d8a01320b5f9ee59d68c7e63229a252daed'}, '']
<wgrant> The build has completed successfully.
<henninge> wgrant: I'd like to actually retrieve the file from the slave now. Any idea how to try that out.
<henninge> ?
<wgrant> henninge: Try http://localhost:8221/filecache/5e385d8a01320b5f9ee59d68c7e63229a252daed
<henninge> oh, that easy ;)
<henninge> wgrant: great, that worked. Thanks a lot! ;)
<wgrant> Great.
<lifeless> cody-somerville: why? we're moving to 8.4 / have moved to 8.4
<noodles775> Why does running `make newsampledata` on a fresh db-devel (after make schema) result in an 8000line difference between current.sql and newsampledata.sql?
<wgrant> noodles775: Yesterday or the day before, 16000 lines of sampledata reorderings were landed.
<wgrant> I do not know why, since the script is meant to order them deterministically.
<wgrant> I say clobber.
<wgrant> Perhaps somebody dumped them manually.
<noodles775> Yeah, I'll create a separate branch that does exactly that so I can see just my diff.
<noodles775> Thanks wgrant
 * wgrant disappears.
<noodles775> Night!
<noodles775> Hi abentley! Is there a way I can insert a pipe at the start of my pipeline?
<abentley> noodles775, no.
<abentley> noodles775, Why isn't it working?
<deryck> intellectronica, hi.  Was there a branch for bug 567379?
<mup> Bug #567379: Recalculate bug heat more frequently <story-bug-heat> <Launchpad Bugs:In Progress by intellectronica> <https://launchpad.net/bugs/567379>
<noodles775> abentley: I'm not sure what you mean... bzr pipeline is working perfectly for me, I just didn't know if there was a way to insert a pipe *before* my first pipe... but I can get around it another way.
<abentley> noodles775, use --before
<noodles775> abentley: thanks (sorry, I just checked the bzr help pipeline, but forgot to check bzr help add-pipe)
<cody-somerville> lifeless, okay, who ever manages the launchpad-dependencies package should add postgresql-plpython-8.4 as a dependency. :P
<lifeless> cody-somerville: Isn't it in the production deps list already? [ there are multiple dep packages ]
<stub> noodles775, wgrant: You get huge diffs between different PG releases as the dump order changes
<noodles775> stub: Ah, thanks... but we should all be using the same PG version right? (I'm still wondering why I see such a big diff for a clean db-devel locally).
<stub> noodles775: Some people are on PG 8.4. That process got stalled a bit.
<jml> intellectronica, mars: https://code.edge.launchpad.net/~jml/launchpad/still-more-tests-for-ec2/+merge/24503 -- we discussed this the other day
<intellectronica> jml: great, thanks for making this change.
<jml> intellectronica, np
<mars> jml, botched merge, or wrong merge target branch?
<jml> mars, Launchpad botched the merge.
<allenap> danilo_: I see you replied to the blog post <http://blogs.gnome.org/johannes/2010/04/29/downstream-bug-reports-fail/>. I still get a server error, so I can't post my reply.
<jml> mars, intellectronica: the diff is fixed if either of you would like to review it.
<mars> ah, much saner +/- numbers
<mars> "FlagFallStream", nice object name
<jml> thanks. actually, I think I should delete it since it's not being used any more.
<intellectronica> mars: so are you picking this up? if not i'm happy to review it later.
<mars> intellectronica, to be honest, I would be happier if you reviewed it.  I'm lack confidence at the moment.
<mars> apparently I'm also lack grammar
<intellectronica> :)
<intellectronica> jml: i'll give it a go soon, i just want to finish something first
<jml> intellectronica: thanks.
<james_w> losa ping, please can we create source package branches for maverick?
<james_w> https://lists.launchpad.net/launchpad-dev/msg03227.html for how
<james_w> could someone also check the SQL within? Max suggested it was not what we wanted.
<Chex> james_w: sure, looking
<james_w> thanks
<intellectronica> jml: approved
<jml> intellectronica: thank you.
<jml> oh crap
<jml> I'm not allowed to land things, right
 * jml sads
<jml> g'night all.
<deryck> bdmurray, ping
<bdmurray> what's up?
<bdmurray> deryck: pong
<mars> bac, do you think your branch for Adi was eaten by the same ec2 hang bug?
<bac> mars: dunno.  probably.  but i have no proof
<mars> ok
<mars> thanks
<sinzui> bac: ping
<bac> sinzui: yes?
<sinzui> I am trying to make sense of this: https://edge.launchpad.net/ubuntu/maverick 1. I did not link a52dec 4 hours ago. I may have a few days ago
<sinzui> bac: I suspect the count of packages is odd because there SPPH will not exist until maverick is fully build
<sinzui> built
<sinzui> bac: mayby the copy script ran 4 hours ago and I was the last person to link a lucid package?
<bac> sinzui: perhaps.  too bad https://edge.launchpad.net/ubuntu/lucid/+source/a52dec doesn't tell you who/when it was linked
<sinzui> bac: The data is in the db. I can see it when it gets to staging
 * sinzui looks a lucid to see if he is the last user
<sinzui> I think the first 20 on +needs-packaging links actually needs project registration too
<sinzui> I was not the last user
<sinzui> bac: I am going to assume the series setup script was run 4 hours ago and it is a coincidence that my name is in the list.
<bac> ok
<sinzui> I verified that the last packages linked in lucid are also linked in maverick
<sinzui> That was 5 months to verify the script really works in production
<flacoste> sinzui: nice blog post about the proposed changes! are you going to send out an email to launchpad-dev pointing to the blog post
<flacoste> sinzui: and this will be a very worthwhile enhancement
<sinzui> I am on Monday. I see the pics are missing the private bugs toggle
#launchpad-dev 2010-05-02
<thumper> morning
 * thumper needs coffee
<thumper> mwhudson: do you know what you're doing yet?
<mwhudson> thumper: reading stuff
<thumper> :)
<mwhudson> gosh edge seems slow currently
<thumper> so say our users too
<wgrant> It's a lot better than it has been for a few days!
<wgrant> Most of my scripts aren't even timing out any more.
<thumper> launchpad speed makes me sad
<thumper> or lack thereof
<thumper> mwhudson: it seems my use of the zope mailer that was in my branch caused may of our problems
<wgrant> Particularly since it has just about no load.
<thumper> mwhudson: BjornT reverted that part of my branch over the weekend
<thumper> mwhudson: I wish zope email config wasn't such an arse
<thumper> mwhudson: we shouldn't have to care in the code
<thumper> mwhudson: how do we go about upgrading an import branch to 2a?
<thumper> mwhudson: do we have that written down anywhere for the losas?
<thumper> hmm... seems that the bzr_lp builder is working again
<thumper> 10 failures, 55 errors
<mwhudson> thumper: get a losa to run "bzr upgrade" from one of the import slaves
<thumper> mwhudson: is that all?
<mwhudson> thumper: yep
<thumper> mwhudson: which branch do they end up upgrading?
<thumper> ERROR: Error (No GeoIP DB found. Please install launchpad-dependencies.)
<thumper> from the bzr builder
<mwhudson> thumper: argh
<mwhudson> launchpad-developer-dependencies isn't installed?
<mwhudson> thumper: they should upgrade sftp://hoover@escudero/srv/importd/www/${BRANCH_ID_IN_HEX}
<thumper> it seems that way
<thumper> mwhudson: do you remember the bug about oopses and that when testing for a generated oops it can fail over a day boundary?
<thumper> mwhudson: I have a test that needs to check for an oops, so I thought I might fix it at the same time
<mwhudson> thumper: https://bugs.launchpad.net/launchpad-code/+bug/567257
<mup> Bug #567257: Race condition in TestDiffInScripts.test_fromFile_withError <spurious-test-failure> <tech-debt> <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/567257>
<thumper> mwhudson: ta
#launchpad-dev 2011-04-25
<jtv> lifeless: I see you were playing with the idea of sacrificing bug-count accuracy for private bugs.  I sometimes wonder if it makes sense to treat them as the same thing in the first place.
<jtv> UI-wise, that is.
<jtv> If we had, say, private-bug portlets then we'd get more caching, less low-level decision-making ("which bugs should I count here exactly?") and maybe even less of the narrow-deep/wide-shallow query antipattern as a matter of course.
<jtv> In that example (private-bug portlets), I imagine bug counts etc. including only public items, and logged-in users with access to private items getting something extra on the side to show those items.
<jtv> In some ways that'd be worse UI, but in some ways it might be better: less "oh you can't access this it's private" and more confidence that private things really are private.
<lifeless> jtv: its an interesting discussion to have for sure
<lifeless> jtv: doesn't really impact the coding though - its easy enough to union in on search generation to avoid shallow-wide issues
<lifeless> jtv: caching of public stuff doesn't really interest me
<jtv> shallow is rarely an issue when it comes to db query performance.  :)
<lifeless> jtv: users seem to notice stale data
<lifeless> so I'm very keen on use ripping out caches that aren't keep fresh
<jtv> Of course.
<jtv> But how often do we need e.g. the Ubuntu bug count?
<lifeless> dunno :)
<jtv> In fact...
<lifeless> I can tell you of course
<jtv> this is way too generic, but
<jtv> just a thought:
<jtv> imagine we had support for short-lived caches with optional aggressive pre-seeding.
<jtv> So basically on-demand caching with short lifetimes, but separately from that imagine we could pick the most popular items and refresh them before they expire.
<lifeless> there is a common antipattern that turns up there
<jtv> That'd be a way to head off the thundering herd.
<lifeless> which is that the generation time is high
<lifeless> said generation time is then the lowest latency for updates to the cache
<jtv> Why would a high generation time be the lowest latency for updating the cache?
<lifeless> either the data is incrementally updatable
<lifeless> in which case a cache is a bit of a misleading description (its more a memo that can be edited - like my fact table)
<lifeless> or the data has to be regenerated after a change occurs
<lifeless> in which case the shortest latency between change and inserting the updated cached item is the time it takes to generate the the item from scratch
<jtv> So you're really saying the problem there is we lose sight of the time it takes for a change to become visible to the user?
<lifeless> yes
<lifeless> the key is to build data structures that let us do /all/ the work we need to do efficiently
<lifeless> e.g.
<lifeless> 'update the bug'
<lifeless> 'update the aggregates the bug appeared/now appears in'
<jtv> lifeless: ISWYM
<jtv> lifeless: got a moment to discuss something else?  Pre-imp, basically.
<lifeless> sure
<jtv> Thanks.  It starts at this page: https://wiki.ubuntu.com/NewReleaseCycleProcess
<jtv> Lots of manual steps on there, many of which could be automated, when a new Ubuntu distroseries is created.
<jtv> Some of those steps apply to other distros as well.
<jtv> I've been asked to automate steps 12 & 14.
<jtv> Basically, two script runs for a new distroseries.
<jtv> AFAICT those script runs can be done right after initialise-from-parent (step 10).
<jtv> For Debian series, it basically doesn't matter whether we do that script run or not.
<jtv> And I think for the foreseeable future, every series that matters is going to have a parent to initialise from.
<jtv> The way I think I'd like to handle that is:
<lifeless> man, that whole thing should be automated
<jtv> Exactly.
<StevenK> We don't follow that process for Debian at all, since we don't publish it
<jtv> And I don't want my solution to get in the way of that, but also not go overboard on the design.
<jtv> StevenK: no, I'm just using that page as a starting illustration here.
<jtv> StevenK: as I said earlier, many of those steps are not Ubuntu-specific on the one hand, and on the other we can take or leave steps 12 & 14 for Debian.
<jtv> What I have in mind for a solution when it comes to just steps 12 & 14, is to have a DistributionJob type that is handled by a runner on cocoplum.
<jtv> If it fails, I'm thinking it should notify the distro owners who can then sort it out with us LP people.
<lifeless> jtv: so, what about the cron job disable/enable?
<lifeless> seems you need to fit in with that / make it unneeded
<wgrant> jtv: Debian's archive won't have its publish flag set, so you won't try to publish it.
<jtv> What cron job disable/enable is that?
<lifeless> step 5 and 15
<lifeless> that bracket the step syou're automating
<jtv> wgrant: AIUI it doesn't matter whether we run this or not for Debian series.
<wgrant> jtv: Debian has nowhere to publish to, so it will fail if you try.
<jtv> wgrant: you mean it doesn't have a publisher config?
<wgrant> jtv: Indeed, that is the case now.
<jtv> lifeless: sorry if I left that bit out -- I was thinking to create this job in initialise-from-parent, so the run won't happen before the cron jobs are disabled.  We'll want a notification to the distro owners in the case of success as well then, as one of the preconditions for re-enabling them.
<wgrant> jtv: We need to automate the disablement.
<jtv> wgrant: you mean "yes it doesn't have a publisher config" or "yes it does have a publisher config"?  (Asking because where I live, people tend to answer negative questions with "yes")
<wgrant> jtv: That is the case: it doesn't have a publisher config.
<wgrant> A month ago that wasn't the case.
<wgrant> Because there were no publisher configs.
<jtv> wgrant: we ought to automate the whole bally lot of this.  But that's for another day.  I'm trying to solve a specific problem in such a way that it doesn't stand in the way of a wider solution.
<wgrant> So it would have happily published into completely the wrong place.
<jtv> That's fine too, as long as it's not a place used by someone else.
<jtv> I do wonder though whether a new non-Ubuntu, non-Debian distroseries will have a publisher config at this point.
<jtv> If not, there's not much point to any of this.
<wgrant> Not right now, no.
<wgrant> But as soon as DDs are in use, yes.
<lifeless> as a DD, I object to that
<lifeless> we don't get 'used'
<wgrant> Ha ha.
<jtv> Before I make the joke none of us wants to see me make... what's a DD?
<lifeless> debian developer
<jtv> I don't think that's what wgrant meant...
<wgrant> I believe I meant Derivative Distros, but now I am not so sure.
<jtv> Well this is _for_ derived distros (if you don't mind changing your words a bit) so will a new distroseries for a new derived distro have a publisher config right from the outset?
<jtv> mind *me* changing your words, that is
<wgrant> jtv: A new distro had better have a publisher config before its series is initialised, yes.
<jtv> If that job is not on the board yet, perhaps it should be.
<wgrant> Mm, not sure that's the case.
<wgrant> It's one of those things that can/will be done at the end once everything else is worked out.
<wgrant> It doesn't affect much else.
<wgrant> Unlike the other stuff that people seem to be pushing through excessively quickly :P
<jtv> By the way, I suppose that the disabling/enabling the cron jobs can in future be done for just the distro.
<wgrant> Can't it already?
<jtv> Don't think so, no.  All the locking is still per-script.
<wgrant> Ah, assuming you're publishing multiple distros on a single machine, yeah.
<jtv> Well otherwise there's not much difference, is there?  :)
<wgrant> What do you mean?
<wgrant> There is one instance of each job for each distribution.
<jtv> Yes, so disabling the cron jobs for all distributions on a machine would be pretty much the same thing as disabling the cron jobs for the one distribution on the machine.
<jtv> By the way, we do have a ticket for making the scheduling of publisher jobs db-driven.
<jtv> That'd be helpful in managing this.
<wgrant> Have you actually discussed how the scheduling is going to work?
<wgrant> And what the requirements for setting up a new distro are?
<jtv> No, the scheduling thing is a lower-priority task.
<jtv> The requirements for setting up a distro is effectively what we're discussing right now.
<jtv> I imagine the Ubuntu procedure is about the best template we have for that.
<wgrant> The best template for what is necessary for creating a new series in an existing distribution.
<wgrant> Not for what we want creating a new series to be like, let alone creating a new distribution.'
<lifeless> grag
<lifeless> whats the bug # for users being confused about login.launchpad.net
<lifeless> jtv: I just remembered a great example about cache update latency
<lifeless> google used to run their search index in readonly mode
<lifeless> they would process newly scanned pages into a new index
<lifeless> and then run around their datacentres pivoting the new index into place
<lifeless> this was a fortnightly or something even
<lifeless> *event*
<lifeless> so a new page wasn't findable for weeks - as an expected thing
<lifeless> then, they made it transactionally updatable (they did a whitepaper on this, adding transaction abstraction to bigtable); and now scan pages directly into the live index, with sub-minute latency on event driven changes (like pingthesemanticweb) and 1/2-refresh-frequency for polled content
<lifeless> all about data structures
<jtv> wgrant: yes, that is a big question right there.  I have a feeling we'll either have to do another round of design (which will probably still miss things, I guess) or create a manual list similar to this one as we learn, and automate as a separate process.
<jtv> lifeless: that's very much down to investing effort into the specific case though.  There's often _also_ a lot to be gained from more generic partial solutions: partly as a stopgap while you direct the effort elsewhere, and partly as an actual solution for the more mundane problems.
<lifeless> jtv: perhaps.
<jtv> For instance, caching can be bad in that it hides worst-case latency problems, but it can be good for reducing load-driven latency and a stopgap for average-case latency.
<lifeless> so, the latter two cases do make sense for judicious use of caching
<lifeless> however
<lifeless> identifying them is harder than you might think
<jtv> wgrant: have you had a chance to look at the pythonic publish-ftpmaster again by the way?  I'm really quite dependent on your judgment when it comes to Q/A.
<lifeless> jtv: at a guess, not till wed
<lifeless> jtv: its easter
<lifeless> + anzac day
<wgrant> jtv: lifeless speaks the truth.
<wgrant> When are we going to switch over to it?
<wgrant> I don't really have much idea of the timeframe of this whole thing.
<jtv> anzac?  aus&nz asomething csomething?
<wgrant> Other than suspicions that everyone's being very optimistic about timing.
<jtv> wgrant: totally concur there, but I suspect the answer depends largely on "when wgrant is satisfied that it works properly."
<lifeless> jtv: australia new zealand armed core
<lifeless> jtv: gallipoli remembrance basically
<jtv> Some kind of independence movement?
<jtv> Joking, joking.
<jtv> Perhaps not a good idea to joke about Gallipolli with the antipodeans...
<StevenK> I thought it was Army Core?
<lifeless> StevenK: probably
<wgrant> YUou mean Corps?
<lifeless> wgrant: I do
<jtv> That makes slightly more sense.
<lifeless> <- tired
<StevenK> At 6:45pm?
<lifeless> StevenK: late night + early kitty-wake-up
<jtv> A full corps would be a sensible unit for those population sizes, I guess.
<jtv> lifeless, StevenK, wgrant: thanks for your help with this.  The salient points have been recorded in notes.  Which after an urgently approaching meal I shall start to turn into code.
<wgrant> jtv: I wish we had Julian this week :(
<jtv> Having Julian is always good.
<jtv> wgrant: do you think anyone would mind terribly if I added an --all-suites option to publish-distro?
<wgrant> jtv: As opposed to omitting the '-s' option..?
<jtv> wgrant: well I would have _expected_ omission to mean "all," but I don't see that in the code.
<wgrant> It does.
<wgrant> It leaves allowed_suites empty.
<jtv> Is it hidden in getPublisher somehow?
<wgrant> Which means everything is allowed.
<jtv> *somewhere
<wgrant>     def isAllowed(self, distroseries, pocket):
<wgrant>         return (not self.allowed_suites or
<wgrant>                 (distroseries.name, pocket) in self.allowed_suites)
<wgrant>                 if (self.allowed_suites and not (distroseries.name, pocket) in
<wgrant>                     self.allowed_suites):
<wgrant>                     self.log.debug(
<wgrant>                         "* Skipping %s/%s" % (distroseries.name, pocket.name))
<jtv> That's not in publishdistro.py, is it?
<wgrant> etc.
<jtv> wgrant: intriguingly, I see different code for isAllowed.
<jtv> Oh wait, you've got stuff after the return... what is that?
<jtv> And more importantly, don't the publish-distro invocations in the Ubuntu release procedure manually specify all Ubuntu suites?
<jtv> That plus the publishdistro.py code made me think that omitting -s couldn't mean "all suites"
<wgrant> jtv: The stuff after the return is from another method, doing a similar thing.
<wgrant> jtv: NRCP runs publish-distro for all suites in the new series.
<wgrant> Not all suites.
<jtv> It passes "-sÂ DSNÂ -sÂ DSN-updatesÂ -sÂ DSN-securityÂ -sÂ DSN-proposedÂ -sÂ DSN-backports" -- what's missing?
<wgrant> DSN-1, DNS-1-updates...
<jtv> Ah OK.
<wgrant> Please don't imbue it with the knowledge of series.
<wgrant> It should operate on suites opaquely as far as is practical.
 * jtv scribbles note
<jtv> By the way, the distsroot that's passed for the partner archive looks to me like it's exactly what we would now get (thanks to archive configs, I take it) without the -R option.
<wgrant> We don't use .new?
<jtv> Not there, no.
<stub> How to I get from a project page to its PPA? Do we have a link yet?
<stub> nm. projects can't specify an official ppa yet
<lifeless> and due to w----aacky UI choices its hard to  it now
<deryck> Morning, all.
<jtv> hi deryck
<bac> morning deryck
<danilos> mrevell, hi, we should have all the bits ready on production for feature review to start (a bit late because qa took longer than expected)
<mrevell> danilos, Superb. I'm working my way through the feature review notes at the mo. Ursinha should be around soon, too.
<danilos> mrevell, cool
* benji changed the topic of #launchpad-dev to: We have a 9s timeout!!! (2 months early!) | https://dev.launchpad.net/ | On call reviewer: benji | https://code.launchpad.net/launchpad-project/+activereviews
<benji> gary_poster: two will enter, one will leave
<benji> heh, but I will enter the wrong room
<gary_poster> lol
<magcius> where did jam go? :(
<jelmer> magcius: it's a holiday here in .nl
<gary_poster> not in US
<jelmer> gary_poster: jam moved to Eindhoven in March
<gary_poster> oh, cool!
 * gary_poster mildly jealous :-)
<jelmer> gary_poster: there's no second easter day in the US? Do you at least get judgement day off?
<gary_poster> heh, there's no first easter day off in the US.  judgement day is still up in the air.  we'll probably have off for the heat death of the universe.
<wgrant> In .au we have Tuesday off this year as well :D
<gary_poster> heh :-P
<jelmer> wgrant: a third easter day? Don't they only have that in very catholic countries?
<wgrant> jelmer: ANZAC Day falls on Easter Monday this year, the so its public holiday is shifted to Tuesday.
<wgrant> So two Easter days plus ANZAC Day.
<jelmer> ah
<jtv> gary_poster: I think I want a lib/lp/archivepublisher/configure.zcml, where there currently is none.  Could you help with this?
<gary_poster> jtv, sure, 1 sec, trying to tie something up
<jtv> Thanks.  I find one shoe at a time is easier.
<gary_poster> :-)
<jtv> (In this vague and poorly thought-out pun, I am the other shoe)
<gary_poster> heh
<gary_poster> jtv, ok, what's up
<gary_poster> that is to say, what do you need help with?
<jtv> Hi.  I'm adding a job class in archivepublisher, and it has its own utility.
<gary_poster> ok
<jtv> So I reckon there ought to be a configure.zcml.
<jtv> There is not.
<gary_poster> lol, end of thought?
<wgrant> Get your dirty XML away from that innocent code!
<jtv> gary_poster: I just discovered the zcml directory though.
<jtv> I guess it goes in there now.
<gary_poster> I don't think so
<gary_poster> but maybe I'm wrong
<gary_poster> I know the machinery, but I don't know our policies
<jtv> Ah
<jtv> Well there's a lib/lp/archivepublisher/zcml/configure.zcml that already has a utility...
<gary_poster> oh *that* zcml directory
<jtv> So at a minimum I suppose I wouldn't be making things substantially worse.
<gary_poster> yeah that sounds fine
<gary_poster> I thought you meant the top-level zcml directory
<jtv> Whoooah no
<gary_poster> :-)
<jtv> Funny how these things always get rolling when you're explaining the problem.
<gary_poster> :-)
<gary_poster> ok, so you have found a happy home for your ZCML snippet, and all is well in this small corner of the world?
<jtv> Yup.  So thanks for, er, listening.  :)
<gary_poster> heh, yeah, np
<gary_poster> benji, very small MP when you have a chance, please: https://code.launchpad.net/~gary/launchpad/bug770217/+merge/58958
<benji> gary_poster: will do it now as I'm in the middle of a deep review that will take a while
<gary_poster> ok thanks benji
<benji> gary_poster: if I only apply the patch to the test (and leave out the change to lib/lp/bugs/javascript/subscription.js, the new test doesn't fail
<gary_poster> benji, huh, did for me
<gary_poster> that's how I developed it anyway :-)
<gary_poster> Lemme see...
<benji> I wonder if I did something wrong, double-checking.
<benji> gary_poster: my fault: I was running lib/lp/bugs/javascript/tests/test_subscriber.html instead of lib/lp/bugs/javascript/tests/test_subscription.html
<gary_poster> benji, great.  Thank you for checking
<benji> gary_poster: review done
<gary_poster> thanks benji
<benji> my pleasure
<beuno> hello hello
<beuno> Launchpad seems to be doing funny things in code hosting: http://paste.ubuntu.com/598761/
<sinzui> beuno: I just confirmed I can push a new branch. Can you try pushing you branch with a letter at the start of the branch name
<beuno> sinzui,  I can, but branches i nPQM are failing for us now
<sinzui> I wonder if this related to thumper's change to branch ids to permit stable renames
<sinzui> beuno: I do not know how to test this kind of issue :(
<sinzui> jcsackett: are available to mumble about question activity and email. My next step is pretty uncertain
<jcsackett> sinzui: sorry, was brewing up coffee, missed your ping. i can mumble.
<sinzui> fab
<sinzui> jcsackett: can you hear me?
<jcsackett> sinzui: i can hear you, i believe that you cannot hear me. i am rebooting mumble.
 * sinzui too
<deryck> has anyone online run windmill tests on natty?
<deryck> I know wallyworld has, but anyone else?
<gary_poster> benji, have a moment for https://code.launchpad.net/~gary/launchpad/bug770287/+merge/58979 ?  Should be another short one.
<gary_poster> (no)
<benji> gary_poster: sure
<gary_poster> thanks benji
<gary_poster> sorry, the "no" was to deryck
<gary_poster> ("not me" would be more accurate :-) )
<deryck> heh, thanks for answering gary_poster :-)
<gary_poster> :-)
<deryck> I'm guessing no one has.
<gary_poster> I'm guessing they are badly broken?
<deryck> gary_poster, a couple reporting on jenkins.  I just want to verify a fix locally, but can't get it to run.
<gary_poster> gotcha :-/
<deryck> No default or local profile error for windmill running FF 4.  FF 4 doesn't have a profile template anymore.
<benji> gary_poster: done
<gary_poster> benji, yippee! :-) thanks
<sinzui> jcsackett: ping
<jcsackett> sinzui: pong.
<lifeless> morning
<jcsackett> sinzui: hi, what's up?
<sinzui> jcsackett: did you update hide-comment.py to work with questions?
<jcsackett> sinzui: i did locally, but it's periodically failing for reasons i haven't diagnosed; thanks for the reminder. i will prioritize that for the rest of today.
<jcsackett> if you are encountering question spam, just assign to me.
<sinzui> I am updating my copy of the script now to close two questions. You may have already done this since I do not see spam comment on https://answers.launchpad.net/ubuntu/+question/144090
<jcsackett> sinzui: yes, i whacked spam from that earlier.
<sinzui> How do you know which comment to deactivate?
<sinzui> hide
<jcsackett> i believe the question that points out the spam for that is already assigned to me, isn't it? i can update.
<jcsackett> sinzui: right now? you have to count the messages. which is terrible. the ui work needs to add numbers to the message (a la bug comments).
<sinzui> okay. I will run my script
<sinzui> jcsackett: This is my updated script. Question comment silently fails: http://pastebin.ubuntu.com/598898/
<jcsackett> sinzui: fails as in you go to the page and the comment is still there?
<jcsackett> if that's the case, wait a bit. i've found i was seeing delays between firing off setCommentVisibility and seeing it removed.
<sinzui> jcsackett: comment hiding is queued?
<jcsackett> sinzui: i don't believe so. i haven't determined the cause of the delay.
<sinzui> jcsackett: were you anonymous...if so you get the cached page from the proxy
<jcsackett> sinzui: i was not anonymous.
<sinzui> I was viewing the page logged in, then  using an anon browser with a query string appended to ensure I do not get the cached
<sinzui> page
<jcsackett> sinzui: ah, i see what you mean.
<lifeless> bug 691352
<jcsackett> sinzui: so, you are not observing anything happening after a delay, i take it.
<sinzui> jcsackett: that is correct
 * jcsackett resists the urge to swear.
<sinzui> jcsackett: I can also state the comment is number 10000
 * jcsackett ceases resisting the urge.
<sinzui> I am pretty sure there are no questions with 10,000 comments
<sinzui> I already swore.
<jcsackett> the latter is especially vexing, as you should get an index error on trying to do anything to comment 10000.
<sinzui> I have decided that I need to make questions faster so I will make that my top priority today
<sinzui> I have confirmed the script gets the correct question too
<jcsackett> sinzui: i'll start a branch in a sec to see if the setCommentVisibility stuff became unbolted by something.
<jcsackett> i will probably do my analysis far away from people, perhaps in a cave somewhere.
<lifeless> sinzui: hey
<sinzui> hi lifeless
<lifeless> I'm trying to find a bug
<lifeless> it was about how users get confused by login.launchpad.net
<lifeless> I think bug 770107 is a dup of it
<sinzui> canonical-identity-provider
* lifeless changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: benji | https://code.launchpad.net/launchpad-project/+activereviews
<lifeless> sinzui: thats the project
<lifeless> I thought we had a task on LP
<lifeless> but I can't find the darn thing
<lifeless> I was wondering if you remembered such a bug?
<sinzui> I am not seeing it either
<sinzui> But I think I can find a question that links to it
<sinzui> lifeless: I am not hunting my email. This is infuriating
<lifeless> sinzui: s/not/now/ ?
<sinzui> yes
<lifeless> :) makes more sense
<sinzui> lifeless: I give up. I cannot find a login.launchpad.net  bug in c-i-p or a login.ubuntu.com bug in launchpad.
<lifeless> thanks for looking
* benji changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<lifeless> flacoste: ping
<thumper> sinzui: my change wouldn't have effected beuno's branch like that
<sinzui> fab
<lifeless> I really wish we had full bug federation
<lifeless> sinzui: did you mean to /part #launchpad ?
<sinzui> No, emapathy is being a complete and utter prick
<lifeless> :(
<lifeless> its not being empathetic :)
<sinzui> Well I cannot workout where to report bugs about that launchpad-bugzilla, so I unconfigured the bugtracker. But since we are the maintainers, shouldn't we be tracking bugs in Lp?
<sinzui> Looks like empathy want me to be in one freenode channel at a time
<sinzui> lifeless: http://pastebin.ubuntu.com/598960/
<sinzui> ^ My thoughts are really scattered
<lifeless> Seems reasonable
<lifeless> the snapshot things seems overly generic to me
<lifeless> its one of those attractive until you use it patterns
<sinzui> lifeless: I prefer putting the snaptshot/copy of the current question into the event. Let a subscribed function queue a job with the question and the event as metadata.
<lifeless> sinzui: sure; I think that the json should be quite targeted though.
<sinzui> lifeless: I am unsure now if QuestionNotification needs to change if I build something that Implements IQuestion, but is acutally using values instead of references.
<lifeless> sinzui: I'm confused about the implementing of IQuestion
<sinzui> lifeless: I was looking a PersonTransferJob that requires json-compatible values. Instead of keep a reference to the reviewer that is optional, the job has a property that instanties the user from the id in the metadata
<sinzui> lifeless: I do not want to build a schema for the job that  create lots of references to other objects. I think the question plus a dict of values will work most of the time. Some of those values need to turned back into users, bugs, faqs and messages.
<sinzui> lifeless: I cannot put the snapshot into the db, so I need a light way to store the changes
<lifeless> sinzui: AIUI other deferred mailers
<sinzui> I think create a ro-implementation of question that uses values instead of objects to make the email may work
<lifeless> they generate the message
<lifeless> or a template for th message
<lifeless> and a list of subscribers
<lifeless> and they shove that into the queue to be processed on the backend
<sinzui> Well I want to put an end to timeouts and thoses tasks you mentioned are part of the time out
<lifeless> right
<lifeless> I'm just saying that /most/ or even /all/ of the objects you're mentioning are not needed
<sinzui> personMemebershipJob an PersonMergeJob do all the email building, themselves
<lifeless> but perhaps I misunderstand the problem
<sinzui> I think you do
<sinzui> As I said, I cannot get my thoughts in order
<lifeless> AIUI the work required is:
<lifeless>  - update the db to show the event
<lifeless>  - build a message to send to each of [long list of subscribers], and its customised per subscriber
<lifeless>  - spool and send the message
<sinzui> How do I queue the event
<sinzui> QuestionNotification does th last two points perfectly
<lifeless> I was thinking you wouldn't queue the event at all
<lifeless> but generate the common message text /once/
<lifeless> and then queue a job to do a mail-merge of the subscribers & the mail template
<lifeless> but I haven't looked at this code
<lifeless> if what I describe sounds harder or worse, you should ignore me :)
<sinzui> lifeless: That is doable by extracting the QuestionNotification.getBody() implementation to the subscriber function I envision. I think there are a few nuances  though. headers and footers assume the current question state is correct, but infact, the state of things like assignee or target may have several changes in the same jobqueue
<sinzui> Thus I started to envision a very elaborate way of serialising the existing state.
<lifeless> wouldn't the state of things like assignee or target be captured when building the template to use for all subscribers ?
<lifeless> not as explicit fields, but as things injected to said template
<sinzui> The build op needs to create the common body, common header and common footer. We merge the to addresses, and the header/footer reasons later>
<sinzui> ?
<sinzui> lifeless: I think the X-Launchpad-Question and the body must be queued with the question since those describe the current state. Other headers, and the footer can  created using the current question and its current subscribers
<lifeless> sinzui: sounds reasonable to me
<sinzui> So I think the schema is (question, body, header) header can be a string at this moment. It could be a serialised dict for extension.
#launchpad-dev 2011-04-26
<sinzui> wallyworld: I head you
<lifeless> pop quiz
<lifeless> should we count both open tasks in aggregates
<lifeless> e.g if a distro has 1 bug with 2 tasks on different source packages
<lifeless> should we should 'open bugs 1' or 'open bugs 2'
<LPCIBot> Project windmill build #215: STILL FAILING in 1 hr 4 min: https://lpci.wedontsleep.org/job/windmill/215/
<sinzui> wallyworld: maybe this bug next https://bugs.launchpad.net/launchpad/+bug/757248
<lifeless> 51 /    0  RootObject:index.html ?!
<wgrant> lifeless: Probably tests from gac before memcached was fixed last night.
<wgrant> Which prefixes?
<lifeless> indeed
<lifeless> A/B etc
<lifeless> gac is out of rotation now though
<wgrant> Oh, why?
<wgrant> (memcached was fixed before it was put into rotation)
<lifeless> internal xmlrpc failures showing up on bzr ops
<wgrant> I wondered about those.
<lifeless> theres an rt
<lifeless> with comments about it
<wgrant> It wasn't a bad haproxy config?
<lifeless> no idea
<lifeless> till tom comments I can't really tell
<wgrant> Ah, you bring that up later on, I see.
<wgrant> lifeless: Seen the RootObject:+login timeouts?
<lifeless> yes
<lifeless> oh you mean the new ones
<wgrant> The ones that show the OpenID association begin taking 10s.
<wgrant> As expected.
<lifeless> I filed a bug about pageid not working
<lifeless> aren't you on leavce today?
<wgrant> I am, but I couldn't leave #launchpad's commercial query go unanswered.
<lifeless> fair enough
<lifeless> https://bugs.launchpad.net/launchpad/+bug/770647
<lifeless> stuartm has indicated that isd can't spare anyone to work on this for weeks or months
<lifeless> so I'm going to talk to flacoste about that a little
<lifeless> and probably escalate the bug about the pageid: scope not working for this page
<wgrant> i'm intrigued that their primary consumer is not worth the time, but OK.
<lifeless> not a matter of worth
<wgrant> I was going to look at the scope issue on Wed anyway.
<lifeless> same trap LP was in
<lifeless> every-dev-developing at capacity
<lifeless> LP was equally totally unresponsive to not-quite-crisis situations until we reorged
<wgrant> Isn't this a crisis?
<lifeless> its intruiging that its so precisely 10 seconds
<wgrant> The point of splitting SSO out was to make it HA :(
<beuno> to be fair, u1 is a pretty huge consumer (payments, sso)
<lifeless> wgrant: 50/300000 requests are slow
<lifeless> wgrant: how was that ever the point ?
<lifeless> this 10 second thing is -damn -weird
<wgrant> Checked python-openid for the 10s request timeout it probably has?
<lifeless> beuno: this is unlikely to be -not- affecting them
<lifeless> wgrant: it appears to be working
<wgrant> Or SSO for its 10s statement timeout?
<lifeless> wgrant: a 10s fail open auth logic would scare me
<lifeless> (but no, I haven't)
<wgrant> lifeless: Fail open?
<wgrant> Howso?
<lifeless> wgrant: I may be misreading it
<lifeless> wgrant: but it looks to be completing the handshake when it hits the LP timeout
<wgrant> lifeless: OpenID is a bit special.
<wgrant> It will probably fall back to stateless mode if stateful setup fails.
<wgrant> (in stateful mode the OP signs the response so the RP can verify it itself; stateless requires the RP send the response to the OP for verification)
<lifeless> \o/
<lifeless> total bug aggregates, with privacy, in 50ms
<lifeless> 223ms for the tag counts
 * lifeless thinks this will do
<LPCIBot> Project windmill build #216: STILL FAILING in 1 hr 5 min: https://lpci.wedontsleep.org/job/windmill/216/
<jtv> lifeless: https://dev.launchpad.net/Foundations/JobSystem seems to use bug heat calculation as its prime example, but the cron script it refers to doesn't exist.
<StevenK> Yes, bug heat used to be done via a job, and isn't any more.
<StevenK> Bug heat is now done via a garbo script
<lifeless> stub: hi
<stub> lifeless: yo
<lifeless> lp:~lifeless/launchpad/bug-758587 has my bugsummary schema
<stub> Is there a MP up on that?
<lifeless> I'm still agonising over trigger vs appserver
<stub> Don't worry, in hindsight you will decide you did it the wrong way no matter which choice you make.
<lifeless> stub: I'm wondering if you could have a peek at the patch in it (-63-) and see if any direction pops out as better from that, IYO
<stub> k
<lifeless> the query table ends up with 1/2 million rows but can answer tag summaries for all of lp in 250ms
<lifeless> (with privacy)
<stub> Merging in now. Its good to have a merge proposal flagged WIP so there is always a diff ready.
<lifeless> loggerhead can dothat on the fly
<stub> Your indentation sucks :-) viewedby should be viewed_by
<lifeless> http://bazaar.launchpad.net/~lifeless/launchpad/bug-758587/view/head:/database/schema/patch-2208-63-0.sql
<stub> That orrible CHECK constraint reminds me we really need a BugTarget table to do that multiplexing instead of having that logic in BugTask, BugSummary and whereever else we do similar. But that is a separate issue.
<lifeless> yeah
<lifeless> I considered doing that first
<lifeless> but decided against it
<lifeless> http://bazaar.launchpad.net/~lifeless/launchpad/bug-758587/view/head:/database/schema/patch-2208-63-0.sql
<lifeless> rename done
<stub> lifeless: We have to have a primary key on the table or replication explodes.
<lifeless> stub: can that be the unique dimension tuple ?
<stub> I don't think we have a sane candidate, so likely will need an id column
<lifeless>     COALESCE(product, (-1)),
<lifeless>     COALESCE(productseries, (-1)),
<lifeless>     COALESCE(distribution, (-1)),
<lifeless>     COALESCE(distroseries, (-1)),
<lifeless>     COALESCE(sourcepackagename, (-1)),
<lifeless>     COALESCE(viewed_by, (-1)),
<lifeless>     COALESCE(tag, ('')),
<lifeless>     status,
<lifeless>     COALESCE(milestone, (-1)));
<stub> Can't do that sorry
<stub> Need a set of NOT NULL columns that are unique.
<stub> Maybe that is too strict - I'll need to look up the docs
<lifeless> http://www.postgresql.org/docs/8.4/static/ddl-constraints.html
<lifeless> (looking laready)
<lifeless> ok, I'll add a noddy id
<lifeless> unless we coalesce on inserts
<lifeless> and make that simply non-null
<stub> lifeless: So I'm thinking we should use triggers to cope with simultanous updates. When two threads do UPDATE ... count=count+1, postgresql does all the locking for us. But if out code does SELECT count; update ... set count=:old_count+1, that only works if we are in a serialized transaction (by raising a serializable exception for one transaction and making us retry)
<stub> appservers would be fine, but scripts run in read committed isolation by default (because they normally don't need the extra isolation and 'retrying the transaction' is non-trivial for many scripts)
<lifeless> stub: won't triggers end up with relaxed isolation too ?
<stub> lifeless: No. Triggers run in the same transaction as the rest of the connection.
<lifeless> stub: thats what I meant
<lifeless> stub: I should rephrase, how do triggers differ from python code doing read, add, write
<stub> ahh... but your trigger will do UPDATE BugSummary SET count=count+1 rather than SELECT count FROM BugSummary INTO old_count; UPDATE BugSummary SET count=:old_count + 1
<stub> With Storm code, you will likely get the latter structure. So you either need to somehow force storm to do the former, or use raw SQL.
<lifeless> stub: it needs to be upsert and delsert
<stub> yucky.
<lifeless> stub: in that we may not have a row for a given aggregate (no row has count 0)
 * stub wishes PG had an UPSERT extension
<lifeless> stub: and that if the count drops to 0 we need to delete the row (at some point, not necessary to do it immediately)
<stub> So a proper upsert and delsert in a trigger will do the right thing.
<stub> One that loops over INSERT and UPDATE until one of them succeeds, such as the example in the PG reference.
<lifeless> should that be a trigger or a helper function?
<stub> You usually hand roll the upsert logic in the trigger function. I've not seen a generic upsert that takes a tablename etc.
<lifeless> http://www.postgresql.org/docs/current/static/plpgsql-control-structures.html#PLPGSQL-UPSERT-EXAMPLE
<stub> That is the one
<stub> If you need multiple triggers doing the upsert, a helper would be best yes.
<lifeless> bug
<lifeless> bugtask
<lifeless> bugmessage
<lifeless> bugtag
<lifeless> bugsubscription
<lifeless> are the input tables
<stub> What can create new rows though? BugTask and Milestone I would have thought are the only tables that can trigger creating new rows.
<lifeless> a new subscription to a private bug
<stub> k
<lifeless> toggling a bug from duplicateof to not-a-duplicate
<lifeless> adding a bugtag
<stub> oh right... one row per tag.
<lifeless> stub: how do you normally test trigger based stuff
<stub> So far, at the higher level. Create an object foo from Python and confirm the FooCache is updated in the following transaction. That way the tests aren't wedded to the trigger implementation.
<lifeless> ok
<lifeless> bit late to really fiddle, but I'll get on it tomrrow morning
<stub> I can't recall if we have UPSERT or similar at the moment, but if we do, we are not trying to engineer a race to test that code path.
<lifeless> how would decrement-or-delete look like
<stub> feel free to hand ball it if you want me to work on the triggers. Only thing with a deadline at the moment is the deployment scripts
<lifeless> decrement; delete from where count=0; return ?
<stub> I'd just do decrement and have the garbo delete 0 count rows.
<lifeless> ok
<lifeless> stub: if you feel like helping with this, awesome
<stub> I think 'if count == 1 then delete row else update set count=count-1' would work
<lifeless> stub: just refresh my branch as I pushed that variable rename
<lifeless> stub: and push your branch before you EOD; I'll pull it back into mine and work on it during my day.
<stub> ok
<lifeless> stub: if I don't have mail from you I'll assume you didn't get to it and start on the triggers from scratch
<stub> right
<lifeless> wgrant: why is ppa privacy linked to copyable-to-mainability?
<wgrant> lifeless: lalalalalalalalalala
<wgrant> lifeless: You do not want to go there.
<wgrant> But it's because private->public copies trigger the delayed copy mechanism.
<wgrant> Which applies overrides and sends announcements.
<wgrant> Direct copies do not, and delayed copies cannot be explicitly requested.
<lifeless> so we should always do delayed copies?
<wgrant> No, delayed copies should die since they are now slower than direct copies.
<lifeless> This came up because doko wants to let folk see his oni packages
<wgrant> We just need to port a couple of things into the direct copy logic.
<wgrant> And then we can abolish delayed copies.
<wgrant> And the world will be a less nauseating place.
<lifeless> such as overides and announcements?
<wgrant> Right.
<wgrant> And de-restricting the files.
<wgrant> (currently done by reuploading, but we can just toggle the flag now)
<wgrant> None of which are at all hard.
<wgrant> Since the refactoring.
<mrevell> Hi!
<adeuring> good morning
<jtv> hi adeuring
<adeuring> hi jtv!
<jtv> wgrant: did you say that publish-distro needs the overrides symlinks?  They're still dangling when we do the original publish-distro run, no?
<wgrant> jtv: It needs them for proper functionality.
<wgrant> So yes, the first run will be bogus.
<jtv> Ah
<jtv> Anyway, no matter actually: I schedule the initial publish-distro runs during i-f-p
<jtv> Any reviewers out there?  https://code.launchpad.net/~jtv/launchpad/db-bug-766807/+merge/59027
<jtv> gmb, are you OCR today?
<wgrant> jtv: Why are we passing in what appears to be a shell command-line?
<wgrant> Our internal script interfaces should not require such terrors.
<jtv> wgrant: you mean, to the publish_distro invocation?
<wgrant> Yes.
<jtv> For the umpteenth time, it doesn't use our internal script interface.  :)
<jtv> I'd love to clean that up, but not as part of a large feature branch.
<wgrant> I mean, it doesn't really make sense to call a script from a job.
<wgrant> It should call some publisher thing with nice arguments, and publish-distro.py is just a wrapper around that.
<jtv> Well by modern convention we have 3 layers: script, script object, and underlying logic.  In this case, there's only script and underlying logic.  So I call the underlying logic.
<wgrant> The underlying logic takes getopt-style arguments?
<jtv> Yes.  :/
 * wgrant dies violently.
<gmb> jtv: I am, yes. Give me a short while to get sorted and I'll pull your branch from the queue first.
<gmb> Oh, unless wgrant has dealt with your branch already.
<LPCIBot> Project windmill build #217: STILL FAILING in 1 hr 4 min: https://lpci.wedontsleep.org/job/windmill/217/
<wgrant> gmb: Sorry, I'm still on leave, was just doing a quick sanity check of that scary MP.
<gmb> wgrant: Well, I may punt it yet since I'm just coming back myself. Anyway, ta for checking it
<gmb> Now bugger off :)
<gmb> jtv: So, I'm not actually mentally capable of reviewing that yet (and like wgrant, I find it scary). I may take another stab at it in my afternoon, though, but if you find someone else to review it in the meantime that's fine.
* gmb changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: gmb | https://code.launchpad.net/launchpad-project/+activereviews
<jtv> gmb: I doubt I can find anyone else today...  if it's any consolidation, this feature is turned off by feature flag so there's time yet to have Julian pass verdict on the design choices.
<gmb> jtv: Okay. I'll hopefully be back up to speed this afternoon anyway so I'll give it a shot then.
<jtv> maverless
<gmb> I'm just struggling to remember how Python works at the moment.
<wgrant> Oh no.
<jtv> this sounds bad.
<gmb> Then I need to re-learn Zope.
<wgrant> No Julian during release-week.
<jtv> gmb: okay, so we're talking post-lunch then?
<wgrant> After the biggest refactoring in history, too :(
<gmb> jtv: Pretty much :)
<gmb> wgrant: That sounds like a big, scary, un-fun...
<jtv> wgrant: the critical stuff is all behind feature flags and unlanded crontabs branches though, right?
<wgrant> jtv: Right, but distroseries initialisation has still changed massively.
<wgrant> I guess we'll still have to use the manual script for now, though?
 * jtv nods unsmilingly
<wgrant> And just hope it works?
<jtv> wgrant: which manual script do you mean?
<wgrant> initialise-from-parent.py
<jtv> wgrant: I haven't landed any chances to that one, personally...  wouldn't know what others might have done to it.
<wgrant> It's been mostly rewritten.
<jtv> ah
<jtv> So that old-school non-LaunchpadScript script is probably still needed, rather than the job runner?
<wgrant> I don't trust that the jobrunner is going to be usably tested by Thursday.
<wgrant> Unless you know otherwise?
<jtv> Nope
<jtv> In fact there's a ticket marked "PLEASE QA THIS WHILE I'M AWAY."
<wgrant> Forboding.
<jtv> wgrant: any objects to me restarting the df app server?
<wgrant> jtv: None.
<jtv> wgrant: where was our wiki page full of recipes such as "what to pay attention to when restarting the app server" again?
<wgrant> https://wiki.canonical.com/Launchpad/Soyuz/CommonTasks
<jtv> Thanks.
<jtv> I was searching for "Routine."
<LPCIBot> Project windmill build #218: STILL FAILING in 41 min: https://lpci.wedontsleep.org/job/windmill/218/
<lifeless> night
<deryck> Morning, all.
<wgrant> rvba: I've replied to your email with something hopefully not entirely unhelpful.
<rvba> wgrant: I'm reading it ... thanks a lot ... since I'm relatively new to all this, having as much information as possible about the problem is really helpful.
<rvba> wgrant: besides, I reckon you're right and the whole subject is very much involved ... it's a good thing to try to see the various aspects of the problem before jumping into the actual coding :)
<wgrant> rvba: Yeah, it's a complex issue with unclear constraints.
<wgrant> Nobody knows what they want.
<wgrant> Particularly us.
<rvba> :)
<Ursinha> bug 757426 bug 750445
<_mup_> Bug #757426: Distribution:+bugs timeout with combinator ALL <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/757426 >
<_mup_> Bug #750445: MaloneApplication:+bugs timeout on 'any tag {pcert, blockshwcert}' <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/750445 >
<bac> hi gmb, could you have a look at https://code.edge.launchpad.net/~bac/launchpad/bug-770248/+merge/59047 when you have a moment?
<gmb> bac: Sure
<gmb> bac: is FeatureFixture({'foo': 'bar'}) now our preferred way of doing things throughout LP?
<gmb> (well, in tests, that is)
<bac> gmb: last week benji and i looked at some tests using "with feature_flags" where some stuff was done in setUp inside a context manager that terminated.  we were surprised at the way it worked.
<bac> i find the FeatureFixture context manager to be unambiguous and unsurprising
<gmb> Ah, right. Good to know.
<bac> so i'd be +1 for making it the preferred way to do things
<gmb> bac: It might be worth telling launchpad-dev about it, then (you may already have done this; I've been skipping over not to-me emails this morning).
<bac> gmb: good point
<gmb> bac:  test_bug_mute_for_individual_structural_subscription() needs a comment explaining the expected behaviour. Otherwise, r=me.
<bac> gmb: thanks, will do
<deryck> adeuring, I lost you?
<adeuring> deryck: strange. I can see you in mumble
<adeuring> I'll restart mumble
<deryck> adeuring, ok
<deryck> let me restart too
<deryck> adeuring, I'm having issues, sorry
 * adeuring too...
<wallyworld> gmb: hi. i have a large mp for review. the reason it's so many lines is that there's ~500 lines of new tests and cut'n'paste refactoring of some content. the actual real changes aren't too scary I hope
<wallyworld> https://code.launchpad.net/~wallyworld/launchpad/filebug-loses-data/+merge/58410
<gmb> wallyworld: Hmm. Is there any way you can split that up? Does the refactoring have to happen in the same branch for the tests to pass?
<gmb> 1700+ lines is more than I can get done today, I think.
<wallyworld> gmb: sadly the tests won't pass without the refactoring. they were specifically written to check tat the new stuff works. the exisiting tests were relied on for regression testing. no prob about not doing it today. i just saw you as oc and thought i'd given you a reason why it is so long instead of you just seeing it and going wtf
<gmb> wallyworld: Fair enough. You may be better off arranging with someone on your squad to review the branch, given its size (That's what we Yellow squadders did with the 1800-odd line chunks of JS that were written for the advanced structural subscriptions work).
<wallyworld> gmb: good idea :-) i'll do that. thanks.
<gmb> np
<rvba> gmb: Hi, could you take a look at https://code.launchpad.net/~rvb/launchpad/deriv-ui-fixes/+merge/58793 ?
<rvba> gmb: this is a tiny branch that fixes a few UI gliches on the pages related to the derivation work, I deliberately did not file a bug for every single one of these bugs because I'm not sure it's worth the bother, but tell me if you think otherwise
<gmb> rvba: Sure, I'll take a look at that now.
<rvba> thanks
<gmb> rvba: I know that bigjools suggested it in the bug, but I think "Commence Initialization" isn't a good choice of wording for the button. It sounds pretty formal (and a bit silly). It would make more sense to have the help text read something like:
<gmb> "Note that initialization is not instantaneous; initializing with many thousands of packages is likely to take hours to complete."
<gmb> And leave the button as it is already.
<gmb> rvba: How's that sound?
<rvba> gmb: sound good to me!
<rvba> s/sound/sounds
<gmb> rvba: Cool. I'll comment in the merge proposal but otherwise this is r=me with that change.
<rvba> gmb: thanks a lot, I'll make the change and land this.
<MTecknology> hardest part about recipe builds..... versions
<MTecknology> 1.0.0-0ppa1 is the latest LP but the next version could be 1.1.0 or 1.0.1 or 2.0.0... either way, this snapshot will always be ahead of that..
<MTecknology> 99.0.0-0~{revno}+{time}  seems kinda reasonableish
<MTecknology> or s/99/10/ ?
<MTecknology> idk...
<sinzui> jcsackett: do you have time to mumble?
<jcsackett> sinzui: sure.
<jcsackett> sinzui: have a moment to mumble again?
<benji> gmb: I have a quick review for you when you get a chance: https://code.edge.launchpad.net/~benji/launchpad/bug-771269/+merge/59090
<gmb> benji: Sure
<gmb> benji: r=me.
<benji> thanks
<gary_poster> gmb, https://code.launchpad.net/~gary/launchpad/bug731099/+merge/59098 if you have a chance, though I am running to lunch now.  back in a bit
<gmb> gary_poster: I'll take a look now. Nice way to finish my day.
<MTecknology> When it comes to building recipes; why is there not a prebuilt environment with packages already installed that are always needed, and why are they removed on exit; would perhaps it not be best to just purge this rather than remove pieces from the chroot type system and then remove all the same?
<MTecknology> just thoughts..
<gmb> gary_poster: r=me.
 * gmb goes off call; EODs.
* gmb changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<NCommander> wgrant (or anyone knowable in Soyuz) how far implemented was NativeSourceSync?
<LPCIBot> Project windmill build #219: STILL FAILING in 1 hr 5 min: https://lpci.wedontsleep.org/job/windmill/219/
<allenap> Does anyone have 5 minutes to review a 38-line proposal, all Javascript? https://code.launchpad.net/~allenap/launchpad/deriveDistroSeries-json-weirdness-bug-753249/+merge/59120
<gary_poster> allenap, I'll do it.
<allenap> Cheers gary_poster :)
<gary_poster> allenap, r=me, with some maybe-lint blather
<allenap> gary_poster: Thanks.
<gary_poster> np :-)
<lifeless> jcsackett: g'day
<jcsackett> lifeless: 'ello.
<lifeless> jcsackett: just touching base to make sure my follow-up on your bug-comments-seen-by-registry did reach you
<lifeless> jcsackett: and to note that the same issue applies to questions & visibility ;)
<jcsackett> lifeless: no i didn't see it. reading now.
<jcsackett> lifeless: noted on the collection bit. i can see the possibility of follow up branches or bug report for this, but i think it's lower priority than finishing (an already falling out of scope) piece of work before we go off maintenance.
<jcsackett> i *hope* to be in a place where all of this is shiny by the end of the week, in which case i'll be in a good spot to address the collection thing.
<lifeless> jcsackett: I leave those calls to you and sinzui
<lifeless> jcsackett: I don't intend to make the scope bigger
<jcsackett> lifeless: oh, no worries. it did that all by itself. :-)
<lifeless> jcsackett: can you make sure its captured one way or another though?
<jcsackett> lifeless: absolutely.
<lifeless> great, thanks
<jcsackett> per the notes on the tests: good points, all, and thanks.
<lifeless> sinzui: how does one create a project group ?
<sinzui> /projectgroups/+new
<lifeless> sinzui: could we link that to <only authorised> folk in the links-to-new-pillars-portlet ?
<sinzui> it isn't
<sinzui> ?
 * sinzui loods
<sinzui> looks
<lifeless> its not in https://launchpad.net/
<sinzui> https://launchpad.net/projectgroups
<sinzui> lifeless: The lp frontpage is an example of something posted to fail.blog
<lifeless> ha, ok
<lifeless> so
<lifeless> https://launchpad.net/projects
<sinzui> It does little that anyone need
<lifeless> the portlets on the side are a little weird
<lifeless> they *look* like the let you create anything
<lifeless> but really its just teams + the collection-you-are-on ?
<sinzui> such as locate project groups, users, teams, distributions, projects Or work with questions, bugs, blueprints, code, translations.
<sinzui> We should throw that page away tomorow
<sinzui> lifeless: We changed the front page days before releases 3.0. the intent before that page was to link to the top level collections on two axises. Apps and pillar/person collections. You would be able to move between the pillar/person collections once you were in one
<lifeless> anyhow, Ihave helped our user and will remember I hope enough to do so next time
<lifeless> sinzui: maybe its just me, but isn't a facet based search nicer for users? /?filter=projects&projectgroups
<sinzui> Sure. Someone just need to commission  the development.
<sinzui> I wanted to extend the generic page search and object lookup with an expandable form that that revels specific search for pillar, person, an packages
<lifeless> sounds useful
<lifeless> I'm not sure we're allowed to do that :P
<sinzui> I thinks so. That aspect was not in scope when we added google in 2009, but it was in our minds
<sinzui> I am pretty sure such as feature needs user testing *before* we build it. That is a large barrier to JFDI
<lifeless> I think user testing would be needed for some aspects of it
<lifeless> but others - like a single filterable collection for people.teams.projects.groups.ppas.archives.builders would be jfdiable I think
 * maxb wonders if anyone actually uses the bzr-lpreview-body package?
<maxb> Given that it apparently has never actually installed in a way that it will work
<huwshimi> Morning
<lifeless> lolololol
<lifeless> https://bugs.launchpad.net/beeseek-peer/+bug/182821
<_mup_> Bug #182821: Honeybee does not show some pages correctly <BeeSeek Peer (obsolete):Fix Released by andrea.corbellini> < https://launchpad.net/bugs/182821 >
<lifeless> wallyworld: ^
 * wallyworld looks
<lifeless> wallyworld: I was looking at bug https://bugs.launchpad.net/launchpad/+bug/2672
<_mup_> Bug #2672: Bug tasks are displayed against inactive upstreams <lp-bugs> <Launchpad itself:Fix Released> < https://launchpad.net/bugs/2672 >
<wallyworld> lifeless: i think the lp bug above is fixed with the mp i did recently. sounds like a dup. i'll check and can mark it as such
<lifeless> wallyworld: I've closed the bug
<lifeless> wallyworld: I was laughing at the display
<lifeless> wallyworld: I *thought* we were going to show the tasks to registry admins
 * wallyworld looks again
<lifeless> wallyworld: when I look at the bug, I see zero bugtask rows.
<wallyworld> lifeless: ah i see now. we need a better display of info when there are no matching bug tasks don't we
<lifeless> wallyworld: Well, I thought the idea was to show them to registry admins
<lifeless> I'm not sure whats up ;)
<lifeless> wallyworld: anyhow, I'm not suggesting you do anything, just lolllling
<wallyworld> lifeless: the BugTaskSearch of whatever it's called is used and i don't think it does show them to admins
<lifeless> yup
<lifeless> separate problem, *if* we care to fix it
<wallyworld> so whatever that bit of infrastructure does under the hood is what we get :-) i see the why you were amused
#launchpad-dev 2011-04-27
<wallyworld> yep. sorry, bit slow on the uptake there. brain took a minute to context switch
<sinzui> wgrant: mumble?
<lifeless> anyone know if https://bugs.launchpad.net/launchpad/+bug/2810 is actually fixed now ?
<_mup_> Bug #2810: buildd <-> buildmaster protocol is undocumented <lp-soyuz> <soyuz-build> <Launchpad itself:Triaged> < https://launchpad.net/bugs/2810 >
<LPCIBot> Project windmill build #220: STILL FAILING in 1 hr 4 min: https://lpci.wedontsleep.org/job/windmill/220/
<jelmer> lifeless: that's a low bug number :) There wasn't anything like that when I was in the Soyuz team, if that's a useful data point.
<lifeless> jelmer: thanks
<lifeless> poolie: morning! you might like to do bug https://bugs.launchpad.net/launchpad/+bug/83928 yourself, given you know the ropes :)
<_mup_> Bug #83928: register launchpad url scheme <lp-code> <Launchpad itself:Triaged> < https://launchpad.net/bugs/83928 >
<poolie> hi lifeless
<poolie> is it urgent?
<poolie> oh, are you just reassigning tim's things?
<sinzui> wallyworld: I am back
<wallyworld> sinzui: ok
<lifeless> poolie: I'm nuking the use of wishlist in the lp bugs db
<poolie> benji: thanks for the mail refactoring review
<benji> my pleasure
<poolie> good
<wgrant> NCommander: It will be implemented soon as part of the Derived Distros work.
<NCommander> wgrant: handy, I was looking at LP and seems a far chunk was implemented since the last time I looked
<wgrant> Yeah, some progress has been made.
<lifeless> wgrant: https://bugs.launchpad.net/launchpad/+bug/2810 - has that changed recently?
<_mup_> Bug #2810: buildd <-> buildmaster protocol is undocumented <lp-soyuz> <soyuz-build> <Launchpad itself:Triaged> < https://launchpad.net/bugs/2810 >
<wgrant> lifeless: No, it got even worse when I deleted the old docs (which were completely wrong, but still)
<sinzui> wallyworld: lp:lp-production-crontabs/trunk
<lifeless> hmm
<lifeless> odd urlification here - https://bugs.launchpad.net/launchpad/+bug/409665/comments/2
<_mup_> Bug #409665: reviews lose commit message context <lp-code> <Launchpad itself:Triaged> < https://launchpad.net/bugs/409665 >
<lifeless> see the 'lp)'
<wgrant> Tags: lp-code
<wgrant> Martin Pool wrote on 2007-02-08: 	#1
<wgrant> Oops.
<poolie> ?
<poolie> mispasted?
<wgrant> Yes.
<wgrant> StevenK: Does initialise-from-parent still work?
<StevenK> It ought to. Why?
<wgrant> Well, we sort of need it tomorrow.
<wgrant> So it would be nice if it did.
<wgrant> And a lot of that stuff has changed.
<StevenK> wgrant: Trial run on DF?
<wgrant> Might try that.
<lifeless> wallyworld: ping
<lifeless> re mp 58410
<wallyworld> lifeless: hi
<lifeless> wallyworld: the permissions are per-project
<lifeless> wallyworld: so I'm wondering why you say they don't need updating
<wallyworld> lifeless: i wasn't aware of that. i'll recheck. i may need to change the xhr call as you said
<lifeless> wallyworld: on https://bugs.launchpad.net/launchpad-project/+filebug
<wallyworld> lifeless: the sample data used for testing is very limited so it was hard to get a full view as to what was updated when the project changed
<lifeless> on dtdparser - submit something, expand extra options, no status/importance/assignee etc
<lifeless> change the dropdown to lp
<lifeless> same thing, but when extra options is expanded, all the fields are available
<lifeless> wallyworld: sure, I'm giving you a demo is all
<wallyworld> lifeless: ah, excellent. i really appreciate the input. tha's helps a lot
<wallyworld> sinzui: ^^^^^ based on the above, i have some fixing to do before you finish the review. but you can always take an initial look
<wgrant> I'm guessing nobody added the new prefixes.
<wgrant> lifeless: Could you do that, with your newfound superpowers?
<lifeless> they are already in the system
<wgrant> We have very few OOPSes.
<wgrant> Hm.
<lifeless> \o/
<wgrant> Although at least 150 hwdb OOPSes are missing.
<wgrant> And they're from loganberry...
<StevenK> Haha. "That's a small number, so it can't be right."
<wgrant> Or maybe they unbroke checkbox.
<lifeless> diogo put A-Z into the system a while back
<wgrant> Ah.
<lifeless> I checked this yesterday :)
<lifeless> and just now again to be sure
<lifeless> wgrant: bug 770107
<wgrant> I assumed you would have, but the numbers were implausible.
<_mup_> Bug #770107: launchpad relies on login.ubuntu.com but does not sync email addresses <Launchpad itself:Triaged> < https://launchpad.net/bugs/770107 >
<wgrant> Yes :(
<lifeless> wgrant: I'm positive there is a bug about user confusion and login.launchpad.net
<lifeless> neither sinzui nor I could find it yesterday
<wgrant> There is, but I can never find it.
<lifeless> grag
<wgrant> I'm sure I've seen it.
<wgrant> Has Google always mangled bug page titles?
<wgrant> I don't recall seeing this before.
<lifeless> ref ?
<wgrant> Some of them are now "$SUMMARY - Launchpad Bugs"
<wgrant> eg search for 'site:bugs.launchpad.net login.launchpad.net'
<wgrant> Some of the results have the real titles, others don't.
<lifeless> whee
<wgrant> (not unreasonable, given our titles suck so much)
<lifeless> I wonder if that will affect our google-api-search results
<cody-somerville> hmm... is it known that if you expand a bug task for editing, the milestone field is set to '(no value)' instead of the milestone the task is currently assigned to if the milestone is deactivated? So if you go and edit a bug assigned to an old, deactivated milestone it'll remove it from the milestone and to fix it you have to go reactivate the milestone. total PITA
<lifeless> cody-somerville: sounds like a bug
<wgrant> lifeless: Ahem.
<wgrant>         rs = (store
<wgrant>                 .find(FeatureFlag)
<wgrant>                 .order_by(
<wgrant>                     FeatureFlag.flag,
<wgrant>                     FeatureFlag.scope,
<wgrant>                     Desc(FeatureFlag.priority)))
<wgrant> Spot the bug.
<wgrant> This explains why none of the overrides are working.
<StevenK> Hah
<wgrant> So, we are *actually* running with a 9s timeout.
<lifeless> wgrant: scope should be after priority
<wgrant> Without exceptions.
<wgrant> lifeless: Right.
<lifeless> \o/
<lifeless> right, lets clean those out
<lifeless> leave the rootlogin one behind
<lifeless> but the rest are clearly tolerable
<wgrant> I don't trust the OOPS counts, but yeah, we can add them back.
<wgrant> 11:18:36 < lifeless> MTecknology: runs in a trusted context; not at all easy to change
<wgrant> Huh?
 * StevenK tries to remember the shell magic for 'show me everything that is in file1 that isn't in file2'
<StevenK> comm, I guess ...
<StevenK> Argh, ENOSPONGE
<lifeless> StevenK: diff ?
<StevenK> Suppose
<StevenK> comm did the job, so ... :-)
<StevenK> And I've not used it for a while, so it's good to dust off the memory
<MTecknology> so.. are recipes and packages not build in the same restrictive manner?
<lifeless> MTecknology: not identical, no
<wgrant> MTecknology: They are.
<wgrant> Well, recipes are run outside the chroot, but that doesn't matter.
<MTecknology> I just assumed they were so I didn't get why allowing recipes to run commands was dangerous - since deb/rules could easily be changed prior to an upload
<StevenK> It isn't just debian/rules that's an issue
<MTecknology> is debian/rules an issue?
<StevenK> MTecknology: You just said 'since debian/rules could easily be changed prior to an upload' -- I think there are far more sinister things one could do if running external commands was allowed
<MTecknology> StevenK: I just assumed you guys handle it if someone tried to do something in debian/rules; so I'm not understanding why recipes are different - or.. why they have to be so different that shell commands in there has to be so dangerous
<cody-somerville> Recipe builds aren't executed in a fresh VM?
<wgrant> cody-somerville: They are.
<cody-somerville> So I guess I have the same question as MTecknology.
<wgrant> Hmmmmm.
<wgrant> lifeless: wgrant@carob:~/logs$ grep -l -r -e 'Exception-Type: \(LaunchpadTimeoutError\|RequestTimeout\)' */2011-04-26 | wc -l
<wgrant> 452
<wgrant>  * 228 Time Outs
<wgrant> Does not compute.
<lifeless> wgrant: I have a theory
<wgrant> wgrant@carob:~/logs$ grep -l -r -e 'Exception-Type: SoftRequestTimeout' */2011-04-26 | wc -l
<wgrant> 408
<wgrant>  * 105 Soft Time Outs
<wgrant> Computes even less.
<lifeless> wgrant: histogram by machine?
<wgrant> wgrant@carob:~/logs$ grep -l -r -e 'Exception-Type: SoftRequestTimeout' */2011-04-26 | cut -d/ -f1 | uniq -c
<wgrant>      55 chaenomeles
<wgrant>      63 gac
<wgrant>     134 soybean
<wgrant>     156 wampee
<wgrant> (this is also only lpnet, not edge, so the numbers should be *lower*)
<StevenK> But none of those can be combined to make 105
<wgrant> You are correct.
<wgrant> Maybe with edge, though...
<wgrant> Let me find the edge logs.
<StevenK> As my father used to say, "He must have gone to a different school."
<sinzui>  /join #launchpad
<wgrant> lifeless: /win 55
<wgrant> Argh.
<lifeless> yes?
<wgrant> lifeless: https://code.launchpad.net/~wgrant/launchpad/bug-770576/+merge/59155
<lifeless> wgrant: why is FeatureFlag.scope still there?
<lifeless> (flag, priority) is a unique
<wgrant> True.
<wgrant> Forgot that bit.
<MTecknology> I wish I could understand what makes running commands in recipe builds dangerous... If it's a fresh vm and is basically (from what I can see), a locked down chroot; what makes that more insecure than upload builds where a person could put anything at all in debian/rules (with the exception that those uploads also provide an easy way to download packages [Build-Depends:])
<MTecknology> Sorry to be such a pain too... I just don't get it. :(
<wgrant> MTecknology: It isn't dangerous.
<wgrant> It's forbidden for other reasons.
<wgrant> That are not entirely clear.
<MTecknology> Is there any way to make those reasons more clear?
<wgrant> I'm not sure.
<wgrant> But still, you shouldn't have to run shell commands.
<wgrant> Why do you want to?
<MTecknology> https://code.launchpad.net/~nginx/+recipe/nginx-nightly
<wgrant> That seems mildly insane.
<MTecknology> indeed
<wgrant> lifeless: That's fixed.
<MTecknology> this is the recipe I'd like to use.. http://dpaste.com/536105/
<MTecknology> the nest-part only used because ..... i'd definitely do that part of the code differently
<spiv> MTecknology: that strikes me as a kinda odd thing to want to do
<MTecknology> spiv: changes is generated by hand on every release... about the only non-nest option would be with wget and that'd probably be an ugly option..
<spiv> MTecknology: I'm curious why the lp:nginx/stable branch isn't something that you could simply do a regular merge of, rather than picking just those files.
 * spiv suddenly realises part of what this is trying to do
<MTecknology> spiv: because I want to make this something fully automated so I don't need to do a merge any time a branch changes
<spiv> MTecknology: I'm not sure abusing nest-part to copy a file from a branch into a tree of the same branch is something we want to support
<MTecknology> so far, it feels like recipes are pretty much worthless...
<spiv> The point of the "merge" directive of recipes is exactly so that you don't need to do a merge yourself!
<MTecknology> Why don't you tell me how I should do this so I set it up once and don't touch it again until a new stable exists?
<StevenK> MTecknology: Then don't use them?
<MTecknology> If I could just use 'run' then I wouldn't need to do the copy..
<StevenK> As in 'run wget' ?
<MTecknology> 'run cp'
<lifeless> MTecknology: why would you need to do a merge every time the branch changes ?
<spiv> MTecknology: Perhaps this is a naÃ¯ve question, but why not just use the files where they are, rather than cp'ing them?
<MTecknology> when a version is released, there's a whole series of crap that takes place. Part of that is an auto tool that moves stuff around and compiles docs. the README file isn't just docs/text/README, if I could use run, I'd use cat and a redirection to actually build that file correctly.
<MTecknology> but it's also not something the debian packaging should need to accomodate; because when a version is released, that stuff is in place
 * spiv wonders if "nest-part deb lp:foo/debian debian; merge tweak-deb lp:foo/tweaked-debian" works (where lp:foo/tweaked-debian is a branch off lp:foo/debian)â¦
<spiv> So the problem is you want to use the debian packaging info designed for releases, but what you have are branches not releases, and these branches are almost but not quite laid out the same way?
<MTecknology> yup
<spiv> I'd probably try making a branch of the debian packaging then that accomodates those differences, and merge that in after the nest-part of the debian packaging branch.
<spiv> I'm not sure if that would be too ugly.
<MTecknology> would that work after things change in the packaging?
<spiv> I can imagine a 'cp' or 'mv' directive in recipes being reasonable to add, but for this problem in general I don't think those would always be sufficient.
<spiv> MTecknology: it should, if the changes inside debian/ don't conflict.
<spiv> MTecknology: I'm not sure that it will work, but it ought to :)
<MTecknology> so debian packaging is lp:nginx/debian and no debian/ dir; using nest it fits in just fine; the merge will need to be of the change, with a debian sub directory?
<MTecknology> can I have just the debian/rules file in there?
<lifeless> is there a sensible bugtracker for our moinmoin sites? https://bugs.launchpad.net/ubuntu/+source/ubuntu-docs/+bug/685482
<_mup_> Bug #685482: Error logging in via launchpad openid to edit community wiki <openid> <Launchpad itself:New> <ubuntu-docs (Ubuntu):Incomplete> < https://launchpad.net/bugs/685482 >
<MTecknology> spiv: or would that probably not work?
<spiv> MTecknology: just a branch of lp:nginx/debian with a change to its rules or whatever to do what you need
<spiv> MTecknology: no renames of files in that branch or new directories, unless you really want to rename files in debian/ or make new directories in debian/ in the resulting recipe.
<MTecknology> I meant- I was wondering if that branch could have only the changed files.. but i imagine not
<MTecknology> then it'd be a crappy merge..
<spiv> MTecknology: right, just make the changes you need, and don't worry about the other files you don't need to change.
<MTecknology> heh...
<MTecknology> 'run make misc/GNUmakefile snapshot'
<MTecknology> that'd be enough to replace all of this
<StevenK> Then your rules file only needs one extra line
<spiv> MTecknology: given that make rule does "svn export", probably not :)
<MTecknology> well... I'd still need nest-part to work on files..
<MTecknology> ah.. missed that
<spiv> I don't think "svn export . $(TEMP)/$(NGINX)" will work well in a bzr tree :)
<MTecknology> probably not :P
<spiv> ("bzr export . temp-dir" would work alright in an svn tree if you have bzr-svn installed, though ;)
<MTecknology> doesn't that only eliminate the .SVN directories while copying to a new location?
<spiv> And only copy versioned files, I'd expect
<spiv> Which probably doesn't make a difference in a fresh tree.
<MTecknology> I'm adding the extra stuff... setup: with setup added to .PHONY
<MTecknology> looks like they actually did automate the release process, just too automated for a nightly snapshot liek this..
<LPCIBot> Project windmill build #221: STILL FAILING in 1 hr 5 min: https://lpci.wedontsleep.org/job/windmill/221/
<MTecknology> now we see if it works....
<MTecknology> spiv: bzr: ERROR: Branches have no common ancestor, and no merge base revision was specified.
<MTecknology> it doesn't want to merge with the packaging branch, it wants to merge with the source branch
<MTecknology> spiv: I 100% agree that nest-part shouldn't be used to copy files; but in this event, it doesn't seem to be possible
<MTecknology> being unable to use run has kept me from using recipes other places too...
* jtv changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | On call reviewer: jtv | https://code.launchpad.net/launchpad-project/+activereviews
* lifeless changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jtv | https://code.launchpad.net/launchpad-project/+activereviews
<MTecknology> heck... even having to make the series stable is a work around hack that could be avoided by use of run..
<spiv> MTecknology: ah, of course, nest-part doesn't set a parent revision in the tree :(
<spiv> MTecknology: I think that bzr-builder could be improved to make that case work
<spiv> MTecknology: but that doesn't help you today, of course :/
<MTecknology> spiv: nah... I already accepted that it's not going to work today
<MTecknology> probably not this week, or next month
<MTecknology> but after that... i can hope :)
<MTecknology> spiv: for a daily build.. this is probably the best option - http://dpaste.com/536138/
<spiv> MTecknology: one question I guess is why does upstream's vcs layout differ from the tarball layout like that.  It seems a bit unnecessary.  But how recipe builds can cope nicely with packaging designed for release tarballs rather than vcs snapshots is an interesting problem in general.
<MTecknology> spiv: there's a makefil that makes the source ready for a release or snapshot - but it's not really feasible to try to call a makefile inside of a recipe - especially because they require other apps be installed
<spiv> MTecknology: oh, one last trick
<spiv> MTecknology: to workaround the âBranches have no common ancestorâ
<spiv> MTecknology: you could try appending something like "-2..-1" to merge line
<MTecknology> hm?
<spiv> This is for the "merge in a tweak to the debian/ dir added via nest-part" approach
<MTecknology> what would the line look like for the merge?
<spiv> What did you try that gave âBranches have no common ancestorâ¦â?
<spiv> I think just append â -2..-1â for a crude workaround
<spiv> (To say just take the changes in the newest revision of that branch)
<spiv> So e.g. if you had âmerge deb-tweak lp:blah/debian-tweaked" then âmerge deb-tweak lp:blah/debian-tweaked -2..-1â
<MTecknology> I had 'lp:nginx' 'nest packaging lp:nginx/debian debian' 'merge tweaks lp:nginx/debian-tweaks'
<MTecknology> something close to that
<spiv> Ok, so add ' -2..-1' to that tweaks line
<MTecknology> k- i'll have to try that tomorrow.. it's getting too late for me
<MTecknology> what's that do?
<StevenK> Only merge changes from the latest revision
<MTecknology> it was complaining about merging lp:nginx/debian-tweaks into lp:nginx
<spiv> MTecknology: takes changes from the 2nd-last revision (-2) to the last revision (-1)
<spiv> MTecknology: i.e. the changes in the most recent commit
<spiv> MTecknology: It's actually merging into a tree, not a branch
<spiv> MTecknology: a tree that already has the debian/ dir added
<MTecknology> hm.. are commits made at every step of the recipe?
<spiv> No, no commits at all
<spiv> It's like using bzr merge --force, which allows doing a merge on a changed-but-uncommitted tree
<MTecknology> oh
<spiv> Hmm, you're using nest not nest-part to embed the debian/ dir, so really that merge ought to have just worked.  I think I know why it didn't, I'll file a bug.
<MTecknology> thanks, want to subscribe me to it?
<wgrant> Bug #723417
<_mup_> Bug #723417: Further information input area shrunk making it difficult to input and review details <bugs> <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/723417 >
<wgrant> 5 lines seems a bit short for most description fields.
<wgrant> I think we should probably bump to 10/15.
<wgrant> Any objections?
<spiv> MTecknology: will do
<wgrant> jtv: Hi.
<MTecknology> spiv: thanks, i'm gonna try tofall asleep over the noise of my neighbors that need to be murdered..
<jtv> hi wgrant
<wgrant> jtv: Nice short review for you: https://code.launchpad.net/~wgrant/launchpad/bug-751982/+merge/59162
<MTecknology> spiv: hopefully I wasn't too much of a pita
<spiv> MTecknology: no, not at all
<jtv> wgrant: I'm working on a big one, but if this one's short then I'll SJF.
<wgrant> jtv: Diff against target:31 lines (+11/-0) 2 files modified
<jtv> wgrant: that reply actually took longer than it did to read the diff.  :)
<jtv> wgrant: done
<wgrant> Thanks.
<spiv> MTecknology: oh, I'm being silly, merging into nested branches is already supported:
<spiv> MTecknology: you just indent the 'merge' line under the 'nest' line.  https://help.launchpad.net/Packaging/SourceBuilds/Recipes#Nesting
<MTecknology> heh... easy
<MTecknology> makes sense too...
<spiv> :)
<MTecknology> I'll play with that tomorrow; then when the bug is taken care of, I should be able to make this work :)
<MTecknology> although.. using 'run' would be a better solution to merging changes from another branch
<MTecknology> but either way- it'd work :)
 * MTecknology is going to sleep for real this time
<jtv> wgrant: I wonder if maybe you could review my branch for initializing distroseries indexes.  The regular reviewer chickened out.
<wgrant> jtv: So I saw.
<wgrant> I will have a look.
<jtv> Great, thanks.
<jtv> He's right though: the reviewer I could otherwise get on my own squad hasn't just returned from two weeks' leave.  Somehow that's not helping.  :-)
<wgrant> jtv: Reviewed.
<wgrant> lifeless: Still around to mentor?
<jtv> wgrant: that's fantastic, thanks.  Didn't expect it so soon.
<lifeless> fsvo
<lifeless> I'm trying to finish these triggers off
<lifeless> which is tricky
<wgrant> Ah, right.
<wgrant> Fun fun.
<lifeless> so if its non trivial, not tonight.
<wgrant> Sure.
<wgrant> lifeless: Bug #771527
<_mup_> Bug #771527: 'Answers' link greyed out on Ubuntu nattry translation page <Launchpad itself:Triaged> < https://launchpad.net/bugs/771527 >
<wgrant> lifeless: DistroSerieses don't have Answers.
<lifeless> wgrant: but distros do
<wgrant> Sure.
<lifeless> wgrant: so the link can sensibly and trivially go to the distro answers
<wgrant> The header is already confusing enough :(
<StevenK> jtv: https://code.launchpad.net/~stevenk/launchpad/pdsd-cleanup/+merge/59172 when you're able
<jtv> StevenK: up to my ears atm... maybe in half an hour?
<StevenK> jtv: At some point before you EOD is perfectly fine.
<jtv> StevenK: got you scribbled down.
<jtv> Jotted.  That's the word I was looking for.
<wgrant> lifeless: Hi.
<StevenK> jtv: I note it's been 45 minutes.
<jtv> StevenK: I meant remind me in 30.  :)  I'll start right now.
<jtv> StevenK: "don't have any special fields set" is a bit weird as a requirement... how about "don't have their details filled out"?
<jtv> StevenK: reviewed.
<StevenK> jtv: Thanks for the +1, I'll take your suggested wording
<jtv> Cool.
<lifeless> wgrant: hi
<wgrant> lifeless: 11634.2.9
<jtv> wgrant: in your review of my branch, you say to say "careful indices mode."  I note that the actual script doesn't have such a thing... shall I add the wording to the run_publisher option?
<wgrant> lib/lp/bugs/model/bug.py
<wgrant> lifeless: Second hunk of lib/lp/bugs/model/bug.py
<wgrant> lifeless: Any ideas why you changed that?
<lifeless> wgrant: isn't that just the bottom of linkAttachment ?
<wgrant> lifeless: Yes, I am a fool. I mean lib/lp/bugs/browser/bugtarget.py instead.
<lifeless> wgrant: +        :param send_notifications: Control sending of notifications for this
<lifeless> +            attachment. This is disabled when adding attachments from 'extra
<lifeless> +            data' in the filebug form, because that triggered hundreds of DB
<lifeless> +            inserts and thus timeouts. Defaults to sending notifications.
<wgrant> Well, *that*'s not in the commit message.
<wgrant> I see.
<lifeless> wgrant: so I think I had a dirty branch
<wgrant> jtv: -A is careful indices.
<jtv> wgrant: it'd be nice if the script called it that then!
<wgrant> jtv: -P is careful file publishing, -A is careful index publishing -SOMETHING is careful domination, -C is all of the above.
<lifeless> wgrant: but yeah the motivation was to stop sending a notification-per-attachment in  apport bug filings
<lifeless> wgrant: if it was a dirty branch this may have only been half realised
<wgrant> lifeless: We have a critical regression bug on that now :(
<jtv> wgrant: the script currently calls it "careful apt" and says it makes the apt-ftparchive run careful.
<lifeless> wgrant: #?
<wgrant> lifeless: Bug #667604
<_mup_> Bug #667604: new bug reports with attachments added generate blank email instead of including links to attachments <lp-bugs> <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/667604 >
<wgrant> jtv: Right, and apt-ftparchive generates the indices.
<wgrant> jtv: (it's still a bit wrong now, since NMAF is also an option)
<jtv> NM?
<wgrant> NoMoreAptFtparchive.
<wgrant> Native index generation.
<jtv> Argh that
<jtv> Looks to me like it should be hidden in the publisher TBH
<wgrant> It is.
<wgrant> The docs of -A are wrong.
<lifeless> wgrant: so, sucky fail.
<wgrant> It will trigger index publication regardless of the used mechanism.
<lifeless> wgrant: but yes, reverting that would just break apport bug filing
<jtv> wgrant: I have no idea what that means, but it's obvious to me that it needs some updating.
<lifeless> wgrant: you bisected to that rev?
<wgrant> jtv: Index generation uses either a-f or NMAF. -A will trigger whichever one is appropriate, despite its description referring solely to a-f.
<wgrant> lifeless: annotated to it.
<lifeless> wgrant: I don't understand why it doesn't generate a notitifcation with the summary and description
<wgrant> lifeless: Hm?
<lifeless> wgrant: this doesn't suppress notifications for the bug/task creation
<lifeless> wgrant: and is (IIRC) only in the apport bug filing form.
<wgrant> lifeless: Two emails are sent.
<wgrant> lifeless: One about the bug filing.
<wgrant> And one about the subsequent comment and attachments.
<lifeless> wgrant: oh and a stupid empty one ?
<wgrant> Except the comment is empty, and the attachment notifications are suppressed.
<wgrant> Right.
<wgrant> I guess we could just suppress that email, but it would be nice to unbreak notifications.
<lifeless> so, two bugs; shouldn't send mail if there is nothing to say. And separately tell folk about the attachments.
<lifeless> I was intending to aggregate the warnings or something I think
<lifeless> but the problem is that we use a bloody pub-sub mechanism for sending mail about bugs
<jtv> wgrant: thanks for clarifying that.  I'll update the help string.  What I meant earlier is that the decision which index generation method to call looks as if it should be hidden inside a single method call.
<wgrant> jtv: It is.
<lifeless> which is a totally fine system *if* you have enough granular control
<wgrant> lifeless: Yeah, ideally it would queue subscribers, I guess.
<wgrant> Or queue the inerts.
<lifeless> wgrant: IIRC this was when we had 17 second bug filings.
<wgrant> Yeah.
<wgrant> i recall.
<wgrant> I just didn't see any relevant commits.
<wgrant> And indeed there were not any :P
<lifeless> yeah, sucks to be working with me :)
<jtv> wgrant: not the way I mean -- I'm talking about the "if ...: publisher.C_doAptFTPArchive(...) ; else: publisher.C_writeIndexes(...)
<wgrant> jtv: Ah, right.
<wgrant> jtv: Well, it is encapsulated. At the wrong level, but meh.
<jtv> shudder.
<wgrant> This. Is. SOYUZ. and all that.
<jtv> I noticed.
<adeuring> good morning
<LPCIBot> Project windmill build #222: STILL FAILING in 1 hr 13 min: https://lpci.wedontsleep.org/job/windmill/222/
<lifeless> stub: evening
<stub> Morning
<stub> I forgot I had to do a Visa run, so just back from Cambodia.
<jtv> hey stub, all sorted?
<jtv> Presumably you are now registered on both sides as a spy.
<stub> Yup. new stamp
<stub> Nah... at Poipet it would be compulsive gambler. Spys cross at Si Saket and now Surin.
<jtv> Note the pattern by the way: the fighting starts in response to "possible" action from somewhere in the middle (last year the sound of smallarms fire, this year a flare)... around Chinese New Year.
<jtv> Ah yes, I forgot about the spies who were actually caught.
<lifeless> stub: I'm too lazy to do a trial run
<lifeless> stub: what happens if you specify a row in an update WHERE clause and both the row and the constraint are NULL
<lifeless> e.g. update bugsummary where .... and product=NULL and productseries=NULL
<lifeless> stub: I'm guessing because null!=null it will update no rows.
<jtv> stub: I think there's probably a single kid unknowingly starting wars every year by lighting fireworks outside his house.
<stub> Nothing happens, because NULL=anything == NULL
<stub> And null is equivalent to false when you end up with that in the WHERE clause.
<jtv> lifeless: how can you update a null row?
<lifeless> jtv: there is a row identified by a tuple (product, productseries, distro, distroseries, viewed_by, status, tag, milestone)
<jtv> Nice key!
<lifeless> jtv: adapting an UPSERT function for it, and of those tuples all but status are permitted to be NULL
<stub> Unfortunately it isn't a key ;)
<lifeless> if we coalesce them with -1
<lifeless> we could make it a key
<lifeless> stub: so I had a ratsnest day
<lifeless> stub: I've got the basic structure for triggers sketched
<jtv> wgrant: don't know if I got around to mentioning it... followed up to your review. Now to find a mentor.
<jtv> hey bigjools!
 * bigjools is not really here
<jtv> good
<wgrant> Morning bigjools.
<wgrant> jtv: Let's see.
<bigjools> hello wgrant
 * stub pulls lifeless' branch
<wgrant> jtv: Looks good. And I agree with DISTROSERIES.
<lifeless> stub: I've just pushed another rev
<lifeless> stub: adding the sketched upsert and decrement functions
<lifeless> stub: remaining bits are
<lifeless>  - tests
<lifeless>  - upsert handling for nulls
 * stub pulls it again
<lifeless>  - generating the vector of rows to increment or decrement in the various cases
<stub> lifeless: I can fill in the blanks in the _inc and _dec methods easily enough
<lifeless> stub: its late here, but I'm guessing you'd have to reengineer the bugs of the other functions. If I put some prose in them would that help with that process?
<stub> And reasonably sure what needs to go into the other stubs
<lifeless> s/bugs/guts/
<stub> Prose is good if you have time - no need to assume
<lifeless> I'll do that now
<stub> k
<stub> My DB script is up for review so I can work on this now.
<lifeless> stub: its reviewed
<mrevell> Hello
<stub> Just saw that
<lifeless> stub: pushing
<lifeless> stub: ok its up there
 * stub pulls
* jtv changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<stub> jtv: I found a B250 tablet pc cover that makes a great Kindle cover in Fortune. I need to pick up another one for Kirsten - do you want one too?
<jtv> stub: sure, thanks
<jtv> wgrant: any objections on your end to my running populate-dsd on df's maverick?
<stub> mthaddon: Just landing https://code.launchpad.net/~stub/launchpad/db-deploy/+merge/59065
<stub> mthaddon: This should improve Launchpad DB updates.
<mthaddon> interesting...
<stub> mthaddon: The preflight.py script does the painful checks for long running transactions, replication lag, sync working etc. for you. The full-update.py is just a wrapper around the 4 update scripts to run them in sequence for faster turnaround.
<mthaddon> "One of the checks it makes is to confirm there are no non-system connections open to the active databases" \o/
<mthaddon> stub: how does full-update.py handle logging? we'd just output the logs for that to a single logfile rather than having separate logs for each step?
<stub> mthaddon: Yes, it will log all the steps to a single log file.
<mthaddon> k
<lifeless> mthaddon: stub: there is an RT open to kill everything properly on staging
<jtv> Any reviewers here who could mentor this review by wgrant?  https://code.launchpad.net/~jtv/launchpad/db-bug-766807/+merge/59027
<wgrant> jtv: We tend to do that on staging now, but sure.
<wgrant> jtv: Hmm.
<wgrant> jtv: Although it's not going to work really well on DF.
<jtv> Hmm?
<wgrant> jtv: Since it doesn't have all the changelogs.
<jtv> What isn't?
<wgrant> populate-dsd
<jtv> ah
<wgrant> staging's all fine in that regard.
<jtv> OK
<wgrant> I'd suggest running it there, unless you have a good reason to not.
<jtv> May be faster as well.
<wgrant> Myes.
<jtv> DB access. :)
<wgrant> Yeah.
<jtv> And most of all perhaps, less pain from figuring out whether it has your change or not!
<wgrant> My change?
<wgrant> Or perhaps your change.
<wgrant> But staging should be rather up to date.
<wgrant> Given that we are coming from a drought of failures.
<wgrant> In fact we've not had a test failure in a week!
<wgrant> This is unprecedented.
<spiv> wgrant: it's an Easter miracle!
<wgrant> spiv: Shh, don't spoil it.
<LPCIBot> Project windmill build #223: STILL FAILING in 1 hr 5 min: https://lpci.wedontsleep.org/job/windmill/223/
<jtv> wgrant: pushing fix, and thou-sands of bytes per second
<deryck> Morning, all.
<jtv> hi deryck!  Approved your branch.
<deryck> jtv, hi.  I had something up for review?
<jtv> Any takes for a very simple review?  https://code.launchpad.net/~jtv/launchpad/db-bug-771727/+merge/59203
<jtv> *takers
<jtv> deryck: oops no, not you sorry
<deryck> no worries
<jtv> I'm in a jostling car on a 2G connection and it's distracting!
<jtv> Reviewer needed!  https://code.launchpad.net/~jtv/launchpad/db-bug-771727/+merge/59203
<jtv> Review mentor needed: https://code.launchpad.net/~jtv/launchpad/db-bug-766807/+merge/59027 - any takers?
<gary_poster> stub, hey.  I ran ec2 test on my branch and there is some code cleanup I need to try to do from fallout of those new constraints (particularly the unique index, it seems)...*if* we actually make those changes.  Is it worth my time to try to clean things up, or should I wait for your review to see if I actually need to go another entirely different direction?
<gary_poster> (This is re https://code.launchpad.net/~gary/launchpad/bug753000/+merge/59139)
<stub> gary_poster: I need to rework your constraints. We should be using partial unique indexes to avoid NULL issues.
<stub> gary_poster: So CREATE UNIQUE INDEX structuralsubscription__product__subscriber__key ON StructuralSubscription(product, subscriber) WHERE product IS NOT NULL
<gary_poster> stub, ok, cool.  I'd love to understand why (I thought constraints didn't have NULL issues since NULLs are considered non-equivalent) but understand if you don't have time
<gary_poster> stub, and does that mean that my one UNIQUE INDEX I have already is OK, so I can fix code bugs related to it?
<stub> gary_poster: it minimizes the size of the index. I think you are correct though that the indexes work as expected in this case.
<gary_poster> stub, ah!  I see
<stub> So I would consider splitting that into two - one WHERE sourcepackagename IS NULL and another WHERE sourcepackagename IS NOT NULL
<stub> But you can argue for your syntax too...
<gary_poster> stub, I don't really care :-P
<gary_poster> whichever you prefer is cool by me
* danilos changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: danilos | https://code.launchpad.net/launchpad-project/+activereviews
<adeuring1> danilos: could you have a look at this mp: https://code.launchpad.net/~adeuring/launchpad/bug-277118/+merge/59194 ?
<danilos> adeuring1, sure thing, I'll do it after my call
<adeuring1> danilos: thanks!
<stub> gary_poster: I think the new 'windowing function' stuff in 8.4 could make those deletions nicer, but no point changing what we have already if it is fast enough (should be - we don't have that many dupes)
<gary_poster> stub, sounds cool, but yeah, the dupes are few
<danilos> adeuring, hum, shouldn't the copyright be updated to state both years? eg. "2009, 2011" or "2009-2011"
<adeuring> danilos: really? I thought we just mentioned tzhe year of the last change. But I am not 100% sure
<danilos> adeuring, that would be very weird, I've never seen it done like that as far as copyrights go (they should be the years of content creation)
 * gary_poster wonders who mrevlel is and what he did with mrevell
<mrevlel> heh, I'm the bizarro mrevell
<adeuring> danilos: ok, I'll change it
<danilos> gary_poster, that's called improvement (or evolution)
<danilos> adeuring, thanks :)
<gary_poster> :-)
<danilos> adeuring, hum, dsp_url was not used anywhere at all?
<adeuring> danilos: yes, lint reminded me about it
<danilos> adeuring, cool
<danilos> adeuring, I like the "no_bug_racking" test, but it still looks like a typo ;)
<adeuring> danilos: argh... I'll fix it
<danilos> adeuring, all else good, r=me
<adeuring> danilos: cool, thanks. Just one question: I can't find "no_bug_racking" -- typo on your side?
<danilos> adeuring, sorry, "bugracking", line ~108 of the diff
<adeuring> danilos: ah, right! thanks again
<danilos> adeuring, yw
<stub> gary_poster: The 'subscriber IS NOT NULL' in the WHERE clause of the partial index was unnecessary as the column is NOT NULL (but I'm reworking it anyway, so don't mess with your copy)
<gary_poster> stub, ack, thanks
<stub> gary_poster: So you can't subscribe to a (distroseries, sourcepackagename) ?
<gary_poster> stub, not ATM, according to the code patterns I'm looking at in structuralsubscription.py.  Robert says we want to support this eventually, and I filed a bug for it, but AFAIK what I said is right.   That said, I'm fine with the constraint allowing it now
<stub> gary_poster: We allow people to be subscribed to a (distribution) as well as a (distribution, sourcepackagename) ?
<gary_poster> stub, yes, the first is a subscription to an entire distribution; the second is a subscription to a package in that distribution.
<stub> So the second is redundant, or can it have different settings?
<stub> (or different notifications)
<gary_poster> stub, it could have different settings
 * coffeedude waves to deryck
 * deryck waves back
<stub> gary_poster: Reviewed. I think we need a BugTarget - this multiplexing of targets is irritating (lifeless has a patch like this too)
<gary_poster> stub, thank you very much.  Wouldn't a BugTarget have the same problem, just abstracted away one layer?  It would only seem to be a win if we could share that abstraction across many uses--is that what you mean?  I've wanted a way to reference targets generally in the past (Zope ObjectId sort of thing) so that would be cool.
<deryck> so has *anyone* got Windmill running on Natty?  maybe sinzui or wgrant?
<deryck> with FF 4, I mean.
<LPCIBot> Project windmill build #224: STILL FAILING in 1 hr 4 min: https://lpci.wedontsleep.org/job/windmill/224/
<wgrant> deryck: wallyworld is the only success story I've heard of.
<deryck> wgrant, ok, thanks.  I'll just mail him then.
<deryck> it's fixed in Windmill trunk, but I really don't want to upgrade windmill again at this point.
<wallyworld> deryck: you need to create a symlink from something like /usr/lib/mozilla to the ff rofile
<wallyworld> profile
<wallyworld> let me find the exact details
<deryck> hey there's wallyworld!
<wallyworld> hiya
<deryck> You Australians never sleep.
<wallyworld> deryck: gotta get this %@%^!^ branch finished
<deryck> ah, the love of the painful branch
<deryck> I symlinked to FF 3 but still get "no local or global profile" or whatever the ff 4 error is.
<deryck> wallyworld, ^^
<stub> gary_poster: Same problem, but in one place.
<wallyworld> deryck: yeah, still looking trying to find what i did. one sec
<gary_poster> stub, yeah, sounds good if it can be shared
<stub> gary_poster: rather than the same problem, all over the place.
<gary_poster> :-)
<deryck> wallyworld, no worries.  Thanks for the help!
<wallyworld> deryck: make a directory /opt/firefox/defaults
<wallyworld> deryck: then inside that dir symlink profile ->/etc/firefox/profile
<wallyworld> deryck: i can't recall if ff4 puts that profile dir in /etc/firefox/profile or if i just copied a profile dir there
<wallyworld> deryck: the windmill global_settings.py file looks for the profile dir in well known locations. /opt/firefox is the first place it tries
<deryck> wallyworld, ok, thanks man.  trying now.....
<wallyworld> deryck: the tests pass except for a couple which rely on onkeyup handlers to do some work. these are never invoked
<wallyworld> deryck: maybe fixed in trunk? i did try trunk a few weeks back and had no luck
<jcsackett> danilos: any chance i could get a review on https://code.launchpad.net/~jcsackett/launchpad/comment-display-rules-questions/+merge/59104?
<deryck> wallyworld, it is fixed in trunk.  I saw the commit message.  uses a different mechanism to create the windmill profile now.
<wallyworld> deryck: i hadn't documented my workround since i was still experimenting and i thought it would be best to upgrade to trunk before natty released and save a lot of stuffing around
<danilos> jcsackett, there is _some_ chance, let me calculate what it is for you :)
<wallyworld> deryck: the startup maybe but not sure about the failing tests
<jcsackett> danilos: anything greater than 50% works for me. :-)
<danilos> jcsackett, I'll be back in 32h with the chance value :)
 * jcsackett laughs.
<deryck> wallyworld, right
<deryck> wallyworld, and we really need to decide if we fix windmill or move forward with something else.  I'm honestly not sure at this point.
<danilos> jcsackett, reviewing (fwiw :))
<jcsackett> danilos: thanks. :-)
<wallyworld> deryck: pain either way. but we need the tests to stop bas shit reaching prod. surely it would be easier to tweak windmill than porting a shit load of tests to something else?
<danilos> jcsackett, if the typos in the MP are to prove that a reviewer has read the MP, I did :)
<deryck> wallyworld, yes, it is easier.  definitely.  but we've been doing this "crap it breaks, fix it" dance for two years now.  At some point we have to move off, I think.
<jcsackett> danilos: no, i can say with certainty the typos are to prove that i shouldn't code without ample supplies of coffee. :-)
<deryck> wallyworld, but maybe not.  maybe this is the last fix to it and it's stable going forward ;)
<danilos> jcsackett, a question, we are not batching (visible) messages anywhere?
<jcsackett> danilos: nope, prior to the visible_messages thing i added, we were just grabbing question.messages and rendering.
<jcsackett> at least, i didn't see us doing anything else with them.
<danilos> jcsackett, cool
<danilos> jcsackett, just making sure we don't prefetch all messages if we only need a single batch of them, all is fine :)
<wallyworld> deryck: we can only hope. maybe fix it and do a proof of concept with something else so we can move if/when it breaks again
<jcsackett> danilos: actually, that reminds me that lifeless asked me to file a bug regarding some inefficiencies in both question and bug messages.
 * jcsackett goes to file bug
<deryck> wallyworld, yeah, I think that's the right way.  get it working now.  but I would tweak that to say, proof to decide how to move off and plan to move, rather than wait til it break again.
<deryck> wallyworld, I have some ideas about what is better actually.  but we can take that up later.  I'll let you get back to that branch you love :-)
<wallyworld> deryck: yes, agreed. bad phrasing on my part
<deryck> gotcha
<rvba> danilos: Hi, could you take a look at this https://code.launchpad.net/~rvb/launchpad/dsd-api-bug-766158/+merge/59186 ?
<danilos> jcsackett, r=me
<danilos> rvba, sure thing :)
<rvba> danilos: thanks.
<jcsackett> danilos: thanks!
<danilos> rvba, btw, perhaps you should have proposed the branch to be merged into devel/db-devel (whatever the target is) and used Steve's branch as a pre-requisite branch (there's a hidden option for that)
<danilos> rvba, (mentioning this if you are unaware of it :))
<rvba> danilos: I'm not aware of this :)
<rvba> danilos: ho ... wait, yes I am ...
<rvba> danilos: let me correct that ...
<danilos> rvba, heh, ok, so when you go to file a MP through the web UI, there's "extra options" or something, and you can put a prereq branch there as well
<danilos> rvba, don't worry about it, it's just that MP won't show conflicts or such that might appear with the real target branch
<danilos> rvba, (you can't change it once MP is up, so you'd have to create a new one I think)
<rvba> danilos: yeah, looks like it ...
 * rvba creates a new mp
<jelmer> "Resubmit this MP" allows you to create a new one based on the old one without having to discard comments, description, etc.
<rvba> danilos: steven's branch targets db-devel but mine does not involves modifications to the db ... is that possible for me to target devel?
<deryck> wallyworld, I got it working, thanks!  I had downloaded ff 3, so had to link into that download dir, not /etc/, but otherwise worked fine.
<danilos> rvba, btw, any reason you are not using assertContentEqual in assertSameDiffs?
<rvba> jelmer: thans, I'll keep that in mind for next time (I already deleted my mp)
<deryck> wallyworld, Now I can work on fixing windmill in my copious free time. :-)
<danilos> rvba, it's possible, but the correct thing would be to target it to db-devel as well
<rvba> danilos: ok
<rvba> danilos: you're right, it probably could use assertContentEqual
<wallyworld> deryck: excellent that it works. i know what you mean about free time. sure it's not worth seeing if trunk is any good?
<danilos> rvba, you can at least get rid of the sorted() calls
<rvba> danilos: that's right. Here is the new mp https://code.launchpad.net/~rvb/launchpad/dsd-api-bug-766158/+merge/59227
<deryck> wallyworld, oh, sure.  I'd probably do that first.  I have a patch in my branch that has to be applied, unless that is fixed now, too.
 * deryck doesn't recall what the patch fixed now, though
<danilos> rvba, cool, will be commenting there
<danilos> rvba, another tiny bit in that first test in the diff: "ds_diff_ws" should probably be "ds_ws" or "derived_ds_ws"
<rvba> danilos: well spotted :)
<danilos> rvba, shouldn't difference_type and status be `Choice`s instead of `TextLine`s? and child_version_higher could probably be a `Bool`
<danilos> rvba, fwiw, I am not completely sure how important it is to be precise with operation_parameters, but when it's not hard, I'd go for it :)
<danilos> rvba, and parent_series could probably be more specific as well (all in interfaces/distroseries.py)
<rvba> danilos: you're definitely right.
<rvba> danilos: I guess I was a little bit too quick to submit this one :)
<danilos> rvba, well, considering the tests pass, this seems to be mostly related to annotations for wadl file generation, so probably not a big deal :)
<rvba> danilos: yes, but the test is pretty minimal. I was advised not to duplicate all the tests for the api version of a function that is tested extensively elsewhere.
<deryck> henninge, oops, I suck.  I forgot our call.  Two minutes to warm coffee and then meet you in mumble?
<rvba> danilos: perhaps I could add one more test with parameters. Not to test the behaviour itself but parameter passing.
<danilos> rvba, btw, you _do_ have a conflict now (line ~306)
<danilos> rvba, yeah, that'd probably be good
<rvba> danilos: this conflict just appeared!
<rvba> easy to fix though.
<danilos> rvba, yeah, though note that once you merge db-devel, your diff between Steve's branch will grow until he merges db-devel as well
<rvba> danilos: right
<danilos> rvba, marked it as needs-fixing (added a comment about what we discussed here, and one whitespace-in-a-string issue I noticed which I don't care about too much :)
<rvba> danilos: thanks a lot for the review!
<danilos> rvba, you are welcome, I'll be going off OCR now, but if you get this done in <1h, I might have a time to look at it again, so just ping me
* danilos changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<rvba> danilos: all right ... for the whitespace issue: if you took to time to mention it, I'll take the time to fix it :)
<danilos> rvba, heh, thanks
<deryck> henninge, connection troubles?  Shall we chat now?
<henninge> sinzui: Hi!
<sinzui> hi henninge
<henninge> sinzui: I want to move Authorization into lp.app.authorizationbase because I want to add a unit test and do not want to add a file to canonical.launchpad.
<henninge> sinzui: would you agree with that course?
<henninge> sinzui: sorry "AuthorizationBase"
<henninge> sinzui: I am still wondering where IAuthorization should go in that case ...
<sinzui> henninge: other apps have their ow security.py. I think I would expect to see the classes that I extend to be in the same module name
<sinzui> eg I think lp.bugs.security will import from lp.app.security
<henninge> sinzui: so you are saying I should use lp.app.security? I am fine with that, it was actually my first thought, too.
<sinzui> henninge: Most importantly, I am happy to give you +1 to move those classes.
<henninge> sinzui: thanks. The interface, too?
<sinzui> yes please
<henninge> cool
<jcsackett> \o/ another thing gets moved into the lp tree!
<sinzui> jcsackett: do you have time to mumble?
<jcsackett> sinzui: sure.
<jcsackett> sinzui: https://code.launchpad.net/~jcsackett/launchpad/bloody-question-api-redirect/+merge/59248
<jcsackett> and if you have something you would like me to review, now is a good time for it.
<sinzui> jcsackett: https://code.launchpad.net/~jcsackett/launchpad/bloody-question-api-redirect/+merge/59248
<jcsackett> sinzui: that's my MP. did you mean to paste in yours?
<sinzui> jcsackett: I did. It is approved to land
<sinzui> I will find my own
<sinzui> jcsackett: https://code.launchpad.net/~sinzui/launchpad/question-email-0/+merge/59247
<jcsackett> sinzui: in IQuestionEmailJob i see body defined twice. based on description i think you mean subject for one?
 * sinzui looks
<sinzui> oops
<sinzui> I am incompetent. I do indeed mean subject
<jcsackett> sinzui: r=me, assuming typo fix. I mentioned in the comment that you might prefer assertContentEqual over having to sort lists when doing assertEqual, but it's hardly a necessary change.
<sinzui> jcsackett: thank you. Yes I think I do want assertContentEqual. I will certainly need it in my next branch/
<deryck> I have an easy 13 line diff for review.  Who wants it? :-)
<sinzui> me
<deryck> sinzui, thanks so much for claiming my review!
<deryck> sinzui, you're the perfect person for this actually.
<sinzui> I am because there was a test and I think someone should remove the cache purge instructions
 * sinzui is looking
<deryck> ah, cool
<deryck> grep and test regex was complete fail for me
<sinzui> Since i am not seeing it. I wonder if it was deleted. It was in a page test and I personally take every opportunity to delete them
<jcsackett> sinzui: i'm pretty sure that's increasingly becoming the rule for everyone.
<deryck> jcsackett, sinzui -- as it should be!  Though hopefully with replaced coverage. ;)
<ymail> who ymail
<sinzui> jcsackett: I am suddenly feeling ill. I need to step away form my next for a few minutes.
<jcsackett> sinzui: ok, i will try to keep an eye on #launchpad for a bit.
<jtv> Any reviewers in today?  Got a very simple one:  https://code.launchpad.net/~jtv/launchpad/db-bug-771727/+merge/59203
<deryck> Later on, everyone.
<sinzui> lifeless: flacoste: One approach to allowing mlist posting is to treat it as an entry point for registration. Send a reply to the user confirming the post, state it is queued, and the user can register at this link to forward it to the list's moderation queue
<flacoste> sinzui: that doesn't avoid the spam backscatter
<flacoste> which would make IS cringes
<flacoste> we had that problem with launchpad registration
<sinzui> I did not know that
<flacoste> and it puts us on the 'spammers haven' list of big provider
<lifeless> lp registration backscatter?
<flacoste> yeah
<lifeless> how did that work ?
<flacoste> people complaining to google/yahoo that they get spam by launchpad
<flacoste> because someone entered their email address on the registration page
<sinzui> okay, now I connect the events
<sinzui> That still happens though. Users do not know that registration is really with ubuntu?
<lifeless> flacoste: ah
<lifeless> flacoste: OTOH there is a robert collins who thinks my gmail address is his and signs up with hotels and banks and so on and I get all his mail.
<lifeless> flacoste: *because* these folk don't do a handshake.
<lifeless> If I wanted US credentials I could have had them several times over now
<flacoste> lifeless: with a very bad credit score ;-)
<flacoste> you probably don't want his credentials!
<lifeless> :>
<lifeless> well, he flys to london reasonably regularly
<lifeless> so can't be too poor :>
<flacoste> lifeless: should I tally up the /api/ requests with the api.launchpad.net ones (in ppr)?
<lifeless> flacoste: no
<flacoste> or have two api categories?
<flacoste> so keep /api/ with the related web requests
<lifeless> flacoste: /api is part of the user interactions
<lifeless> api. is scripts
<sinzui> flacoste: lifeless: I suppose the captcha is an effective barrier to users wanting accounts, and that addressed the spam backscatter. the captcha was not effective against spammer who want to directly exploit Lp
<lifeless> I think its only slightly interesting to count /api as a breakout from web requests, and not useful to combine with api.
<lifeless> sinzui: you'd send a mail once with a captcha link?
<flacoste> lifeless: no, you need to file a captcha to register
<lifeless> oh right
<flacoste> which means that no email is sent unless a captcha is passed
<lifeless> having uid 2 I haven't seen our rego process in a long long long time
<lifeless> so, some options:
<lifeless>  - make a very clear page saying 'we discard mail from non LP users'
<lifeless>  - start sending backscatter *if* we think its not spam from a non-lp user
<lifeless>  - change nothing
<lifeless> as sinzui says, things like dkim could aid in the 'we think its not spam' assessment, but the prevalence of spam from compromised accounts means that it won't be a very useful metric
<flacoste> we should do a) for sure
<lifeless> as a data point
<lifeless> google groups have unmoderated groups which accept mail from anyone according to the docs (though there may be a rego-step in there, I can't tell without finding a random group and testing)
<lifeless> my google friends are asleep atm sadly, or I would ask around
<sinzui> I think google has a fantastic spam checker in front of its apps
<flacoste> we could put spamasssassin in front of ours
<flacoste> that would reduce it a bit
<flacoste> probably not as good as google though
<lifeless> we could run crossassassin too if we wanted
<lifeless> though with only 2K mailing lists it may be too small to benefit yet.
<lifeless> anyhow
<lifeless> right now, the key thing is user surprise
<lifeless> lets not give ourselfs months of speculative design
<sinzui> This could be treated as a switch. I declare my list is open, and I am willing to moderate the queue. those I approve get Lp profiles.
<lifeless> I'd like to reopen that bug though flacoste - I think there is /plenty/ of data that casual users sending mail in is important to communities
<lifeless> and we are mandated with building communities
<flacoste> lifeless: i say open a different bug: discarding email from non-LP users is surprising
<sinzui> I think so to. We can mark it incomplete an then we have 60 days to conclude the the merit of the proposal
<flacoste> i think Won't fix for that bug is the accurate current state
<flacoste> the design constraints around spam still stand
<flacoste> sinzui: given the Low priority this bug is going to expire
<flacoste> no point discussing a design for it
<lifeless> flacoste: I agree that there is no point discussing a design while its low pri
<lifeless> I would like the bug open (phrased differently perhaps) because I think this will come back repeatedly over time, *particularly* once our docs are accurate.
<lifeless> flacoste: I will also open a bug about the docs
<lifeless> flacoste: if you feel very strongly about this, we can leave it closed
<flacoste> what's another Low open bug :-)
<flacoste> iow, i don't feel strongly about it :-)
<lifeless> https://bugs.launchpad.net/launchpad/+bug/772016
<_mup_> Bug #772016: users are surprised when mail from non-lp users is silently discarded by launchpad hosted mailing lists <Launchpad itself:Triaged> < https://launchpad.net/bugs/772016 >
<lifeless> I've marked high
<lifeless> but it should be easy
<flacoste> thanks
<lifeless> I have my monthly allergy shot today
<lifeless> will be out middle of the day
<lifeless> ok, time for injection... back in a few
<huwshimi> Morning all
<LPCIBot> Project windmill build #225: STILL FAILING in 1 hr 1 min: https://lpci.wedontsleep.org/job/windmill/225/
<LPCIBot> Project devel build #681: FAILURE in 2 hr 12 min: https://lpci.wedontsleep.org/job/devel/681/
<sinzui> does anyone one know if the number of builders is low either for the release of because OEM wants them back?
<elmo> sinzui: it's not OEM who own them, but rather Ubuntu Platform Services
<elmo> sinzui: but the answer is both
#launchpad-dev 2011-04-28
<wallyworld> sinzui: huwshimi: wgrant: standup?
<sinzui> I think my mic is dead. I will restart
<huwshimi> sinzui: wallyworld: was just on the phone, back now.
<poolie> good morning
<huwshimi> poolie: Morning
<wallyworld> wgrant: want a review? https://code.launchpad.net/~wallyworld/launchpad/poppy-sftp-gpgconf/+merge/59154
<wgrant> Not of that, but OK.
 * wgrant looks.
<wallyworld> wgrant: you don't have to if you don't want, just asking :-)
<wgrant> No, no, I'll do it, it's just a bad problem with no good solution.
 * wallyworld nods
<wgrant> lifeless: Could you mentor https://code.launchpad.net/~wallyworld/launchpad/poppy-sftp-gpgconf/+merge/59154?
<wallyworld> wgrant: with the time.sleep() in the test, we need that because we need to give the job a chance to trigger and execute. simply setting the times in the past won't test that aspect
<wgrant> :( OK'
<lifeless> wallyworld: while you're looking at it
<lifeless> the inner function isn't needed, just make it a normal instance function and call self.touch..
<lifeless> secondlyt
<lifeless> you appear to do this unconditionally whether its in a twisted environ or not
<lifeless> that seems likely to fail hilariously, or perhaps I misunderstand the context
<wgrant> Isn't that OK?
<lifeless> lastly, should touch the dir as well, just to be fireproof
<wallyworld> lifeless: we do touch the dir as well. wrt the inner function, i didn't want to expose the code on the object. is there a reason why you don't like the inner. wrt twisted, yes you are right but i though twisted was part of our deployment and hence could be assumed?
<lifeless> wallyworld: in reverse order
<lifeless> wallyworld: we use gpghandler in the zope environment too, with no reactor running, and we create them transiently
<wallyworld> lifeless: ok. no twisted. will fix
<lifeless> wallyworld: you are basically creating a [slow] memory leak of incomplete looping calls per instance with the current code
<lifeless> wallyworld: in fact, thinking more about it
<lifeless> you need to cancel the looping call
<lifeless> at some point as well
<lifeless> on the function
<lifeless> uhm, style. I prefer not to curry things that don't need currying
<wallyworld> lifeless: the looping call is cancelled in the atexit handler or when the job is rescheduled. is that not enough?
<lifeless> it makes them more testable
<lifeless> oh, and you ar makeing _touch_home_call a mixed attribute
<lifeless> set it in __init__
<lifeless> -or-
<lifeless> assign to it via self.__class__._touch_home_call
<wallyworld> lifeless: i guess it's a matter of opinion. i prefer to enforce encapsulation in this case. i don't think the inner hinders testability here but will move it out
<lifeless> so in terms of cancellation, the following will leak:
<lifeless> ahehm
<lifeless> how does the buildmaster use this
<lifeless> as a secured utility
<wgrant> buildmaster doesn't.
<lifeless> or via instances ?
<wgrant> poppy does.
<lifeless> blah yes that thing
<wgrant>             sig = getUtility(IGPGHandler).getVerifiedSignatureResilient(
<wgrant> securedutility
<lifeless> ok
<lifeless> so, we shouldn't be leaking
<wgrant> That's what I thought. There's only ever one instance.
<lifeless> but the touchiness won't work on zope
<lifeless> wgrant: one per thread, no ?
<wgrant> Hmm, I guess it probably does use a thread-local site manager, true.
<wallyworld> i didn't know that. i though sm was global to a zope instance
<wallyworld> but then again my zope foo is not that high
<lifeless> wallyworld: it is global
<lifeless> but its allowed to follow any old policy for what it hands out
<lifeless> I'm pretty sure the default which we use is thread local utilities
<wallyworld> ok
<lifeless> so that folk don't have to deal with concurrent mutation of instance attributes
<lifeless> anyhow
<lifeless> we're solving a bug in the twisted environment
<lifeless> we should only run the loop in the twisted environment
<lifeless> but always in the twisted env
<lifeless> I've no idea how to make that true.
<wallyworld> lifeless: i could just do it another way without twisted and remove the dependency
<wallyworld> but you'd think there would be an api to see if a reactor were running
<wallyworld> in the current environment
<lifeless> One way would be a different class (e.g. a subclass) for the twisted case and either different zcml or ask for a different utility Iface
<lifeless> wallyworld: there is an api to see if there is a running reactor, but do we know that the reactor is running when zcml is parsed during startup of poppy-sftp
<wallyworld> lifeless: so if i did getUtility() with  a different interface, i'd only need to do it in the one place - lib/lp/poppy/twistedftp.py  ??
<lifeless> yes
<wgrant> INotBrokenGPGHandler?
<lifeless> and perhaps any tests that are testing lib/lp/poppy/twistedftp.py
<wallyworld> cool. i'll do that. keeps all the other pgphandler instances nice and clean. and i hadn't forgotten about any twistedftp tests :-)
<lifeless> hmm
<lifeless> wonder if a class action against sony is appropriate (for forcing disclosure of birthdate to sign up and then not protecting it properly per UK data protection act rules)
<wallyworld> wgrant: perhaps best to just change the known broken use case initially?
<wgrant> lifeless: They're already being sued in the US.
<wallyworld> i don't think i'll use your suggested iface name :-)
<wgrant> Not sure if it's class action, though.
 * wallyworld hates sony even more than apple if that's possible
<wgrant> wallyworld: Hmm. I think they're pretty similar.
<wgrant> Apple has some good aspects, but they are also far more effective at their evil than Sony is.
<wgrant> But Sony has no good aspects...
<wgrant> So hm..
<wallyworld> i don't think apple has any good aspects either. i hate their dumbed down, locked down, retarted pos ipads, iphones and app store
<wgrant> They turned KHTML into an awesome WebKit.
<wgrant> That's at least one good thing.
<wallyworld> and the rapid fanbois whole bend over for steve jobs like he is some sort of deity
<wallyworld> i guess so
<lifeless> wallyworld: its his massive..innovations they like
<wallyworld> lifeless: you say innovations, i say merely refining concepts developed by others
<lifeless> wgrant: is it a bug or feature that the upload works after gpg verification fails
<lifeless> 760 timeouts
<lifeless> reality sets in
<wgrant> lifeless: Both.
<wgrant> lifeless: It's intentional that they're let through for now, in case it breaks.
<wgrant> lifeless: I wonder how many of those timeouts are from edge.
<wgrant> This OOPS report is indeed much more plausible.
<lifeless>  17 https://bugs.launchpad.net/ubuntu/+bugtarget-portlet-bugfilters-stats (Distribution:+bugtarget-portlet-bugfilters-stats)
<lifeless>        OOPS-1943AQ100, OOPS-1943AZ499, OOPS-1943BA451, OOPS-1943CD170, OOPS-1943CK442
<lifeless>       6 https://bugs.edge.launchpad.net/ubuntu/+bugtarget-portlet-bugfilters-stats (Distribution:+bugtarget-portlet-bugfilters-stats)
<lifeless>        OOPS-1943EDGEA955, OOPS-1943EDGEB1120, OOPS-1943EDGEB1422, OOPS-1943EDGEE1002, OOPS-1943EDGEE897
<lifeless>       3 https://launchpad.net/ubuntu/+bugtarget-portlet-bugfilters-stats (Distribution:+bugtarget-portlet-bugfilters-stats)
<lifeless>        OOPS-1943AO372, OOPS-1943DT447, OOPS-1943L444
<lifeless> so more from edge as a %, less as an absolute count
<wgrant>   53 SELECT BugTask.assignee, BugTask.bug, BugTask.bugwatch, BugTask.date_assigned, BugTask.date_close ... e AND BugTask.bug = Bug.id )) ORDER BY BugTask.importance DESC, BugTask.id LIMIT $INT OFFSET $INT:
<wgrant>     GET: 53 Robots: 0  Local: 1
<wgrant>       23 https://api.edge.launchpad.net/1.0/ubuntu (Distribution:EntryResource:searchTasks)
<lifeless> find the user. Point them at prod.
<lifeless> ok
<lifeless> I'm going to nose down on these triggers
<lifeless> unless anyone needs more from me?
<wgrant> sinzui: Hi.
<wgrant> lifeless: Hmm, 99% under 1.45.
<wgrant> lifeless: That's not bad.
<lifeless> cool
<jtv> lifeless: thanks for that review -- you're a lifesaver.  Our OCR system has basically collapsed.
<lifeless> :(
<lifeless> I have a killer headache
<lifeless> a freaking sore arm
<lifeless> and triggers to write
<lifeless> I think today will be a writeoff
<jtv> lifeless: a sore arm is a warning.  Heed it!
<lifeless> jtv: I was injected with 1ml of stuff I'm highly allergic too
<lifeless> jtv: not rsi
<lifeless> s/too/to/
<jtv> Okay... a warning not to let people do that then I guess.  Wow that sucks.
<StevenK> lifeless: That sounds like a punishment. What did you say to Lynne? :-P
<lifeless> http://en.wikipedia.org/wiki/Allergen_immunotherapy
<StevenK> Ugh
<StevenK> Just reading that article makes my skin crawl
<wgrant> Hi stub.
<stub>     UploadFailed: Server said: 200
<stub> Yo
<stub> Haven't seen that failure before in buildbot, but subsequent builds worked so looks spurious
<wgrant> stub: Yeah, known spurious failure.
<wgrant> stub: Did SSO fork the emailaddress table?
<stub> So occasionally the librarian succeeds, which is considered a failure? :-)
<wgrant> Yup.
<stub> wgrant: Yes. They have their own email address list.
<wgrant> stub: It is full of cruft :(
<wgrant> stub: It contains eg. team email addresses.
<wgrant> Which don't make sense in SSO.
<wgrant> And cause crashes (since it assumes all emailaddresses have an account)
<stub> I guess they need to clean it up or ask me too.
<wgrant> Their table even still has a person column.
<stub> wgrant: Have you opened a bug?
<wgrant> No.
<wgrant> Should I?
<stub> Yes. canonical-identity-provider I think.
<wgrant> That's the right project, but I wasn't sure if it was worth a bug.
<stub> Crashers are worth bugs :)
<wgrant> I guess it means their schema is buggy, true.
<stub> Their schema is a pile of cruft because they inherited it from us and only needed 10% of the fields.
<stub> They also have a number of Django defined tables which is the stuff  they created.
<wgrant> Yeah.
<stub> effort/reward for fixing the crufty schema is low (high effort due to high availability requirements), but cleaning out bad data is little trouble.
<lifeless> stub: hi
<stub> lifeless: Morning
<lifeless> stub: I only got a tiny bit done so far - monthly allergy shot messed me up :(
<stub> lifeless: Sure. Want me to continue working on my branch or pull in yours again? I wasn't sure if using the BugSummary composite type was a good idea or not.
<lifeless> stub: I think it is
<lifeless> stub: have your morning coffee, check for dba bugs, reviews etc
<lifeless> I have a bit of structure I want to stab at this
<stub> cool.
<stub> I gotta book me a flight to Dublin.
<stub> Not sure exactly why I'm needed at a UI focused sprint, but Ireland should be cool :)
<lifeless> cause abstractions leak :)
<stub> Who you calling an abstraction???
<lifeless> I don't know, do you leak ?:P
<lifeless> stub: http://www.slideshare.net/selenamarie/managing-terabytes-when-postgresql-gets-big may interest you
<lifeless> stub: 400K tables
<jtv> lifeless: selena may want to consider not using WordPress MU then.  :)
<wgrant> Anyone want to review https://code.launchpad.net/~wgrant/launchpad/bug-761439/+merge/59317?
<stub> pg gets really sucky with huge numbers of tables, but some people do it anyway - just cloning the one set of tables over and over for every customer
<jtv> wgrant: loading
<StevenK> jtv: I was about to approve it
<jtv> StevenK: then don't let me stop you
<wgrant> StevenK: You managed to not vomit?
<StevenK> Only just.
<stub> Geez this current flash player sucks
<jtv> lifeless: perhaps you're up for mentoring a review by william?  I need an interretur(*) for https://code.launchpad.net/~jtv/launchpad/db-bug-766807/+merge/59027
<jtv> (*) Which I hope is Latin for something like "imprimatur" except for landing instead of printing.  Hope it doesn't mean "let it be buried."
<jtv> Does Latin even have terms for landing?  Control towers?  Re-entry shields?
<StevenK> jtv: cronscripts/create-distroseries-indexes.py is pointless, run_jobs.py can do it
<jtv> StevenK: does run_jobs.py run on the right host?
<StevenK> jtv: Note that IDistroSeries.deriveDistroSeries() doesn't use i-f-p, so you'll need to create a job at the end of IDS, not i-f-p
<StevenK> jtv: Er, what? How is that even related?
<jtv> I don't know.  You started it.
<StevenK> jtv: I see the comment in it, but the generic run_jobs.py cronscript can handle it, and it stops our tree getting tons of small mostly-identical cron scripts.
<jtv> Okay, how do I use the generic run_jobs.py cronscript?
<wgrant> StevenK: Oh, you mean like add it to a new source group and run run_jobs over that group on cocoplum?
<StevenK> jtv: You add the interface to schema-lazr.conf and 'run_jobs.py create_distroseries_indexes'
<StevenK> As much as I hate perpetuating that antipattern, it's what we have.
<jtv> Never mind, I'll look for it in the wiki.
<StevenK> jtv: I could create a diff quicker than this argument :-P
<jtv> StevenK: I'm not sure what argument we're having, but if a diff is the easiest way to explain what you're trying to tell me, then please do!
<jtv> No documentation on the wiki.  :(
<StevenK> jtv: There is a generic job running cronscript: cronscripts/run_job.py
<jtv> Okay, that part I had figured out by now.
<jtv> I'm now looking for documentation.
<StevenK> jtv: It is almost identical to the new cronscript you've added in your branch.
<StevenK> I'm not sure it is documented.
<jtv> It doesn't seem to be.
<StevenK> jtv: http://pastebin.ubuntu.com/600158/
<jtv> StevenK: so this is part of, but not mentioned in, the job-system documentation?
<jtv> Would be worth updating that page.
<StevenK> jtv: It's certainly part of the job system, but I doubt it is documented.
<jtv> Well there's no mention anywhere on the dev wiki.
<jtv> Two places in the test suite mention the script.
<StevenK> One of them would be the IDSJ job runner
<jtv> Yes.
<StevenK> Since I ripped out it's cronscript when I learnt about run_jobs
<jtv> And there's 1 mention in the production crontabs.
<jtv> What's the point in writing reusable code if there's no way for people to find out about it?
<StevenK> Then fix the documentation? :-)
<jtv> The documentation is just bad enough that I'm afraid to touch it.
<StevenK> Yes.
<jtv> By the way I am grateful to you for pointing this out.
<lifeless> stub: jtv: cheatsheet - how do you declare a variable to how a set of rows ?
<lifeless> s/how a/hold a/
<lifeless> jtv: sorry, not just now (review); I can do it later, or any other dev can mentor wgrants review; I'm not picky
<stub> 'setof record' or 'setof bugsummary'
<stub> oh... you want an array I think
<lifeless> declare tags setof bugtag; ?
<stub> record[] or bugsummary[]
<lifeless> a setof is being returned from a helper function
<stub> So returning from a helper, you want RETURNS SETOF bugsummary
<stub> Then in the helper, you want to do 'RETURN NEXT a_bugsummary;' which works like yield in python generators
<lifeless> yes, I have that
<stub> I think I had this in my stub bugsummary_sources()
<lifeless> this is in the function that calls that
<lifeless> http://pastebin.com/cVajbkq3
<lifeless> is the helper
<stub> So you can just loop over the result of that doing RETURN NEXT is one way. Or else you should be able to assign the result of that method to a 'bugsummary%ROWTYPE[]' array and iterate over that
<lifeless> grwat, thanks
<stub> Or we could do this in Python :)
 * stub wonders if we care about the speed hit
<stub> plpythonu I mean
<lifeless> psql:test.sql:157: ERROR:  syntax error at or near "["
<lifeless> LINE 6:     tags bugtag%ROWTYPE[];
<lifeless> stub: pushing now; got time for a weekly catchup call?
<stub> lifeless: sure.
<stub> lifeless: you got the syntax working?
<lifeless> stub: I did some reading
<stub> what is the syntax?
<lifeless> looks like an immutable cursor or some such is a better fix
<lifeless> *fit*
<lifeless> it really wants just 1 row (select into) or a cursor usage.
<poolie> i'm getting a proxy timeout on +filebug, ie 'please try again' with no oops
<lifeless> stub: however I'm barely a trained monkey here
<wgrant> Oh no not again.
<wgrant> poolie: How long does it take?
<wgrant> And what'
<wgrant> s the URL?
<lifeless> stub: I've achieved what I wanted which is a structure that is auditable and parallels the loader.
<poolie> it was /bzr/+filebug
<lifeless> stub: its pushed; skype ?
<poolie> it worked on the 3rd attempt
<stub> foo integer[]; works fine. bugsummary%ROWTYPE[] should too... but it doesn't.
<stub> skype - sure
<poolie> uh, it took quite a while
<wgrant> lifeless: Do we have visibility into queue depth yet?
<poolie> maybe 20s?
<lifeless> wgrant: no; losa time
<wgrant> spm: Hi.
<poolie> the successful run says it completed in 0.64s, and it was probably really a couple of seconds
<lifeless> poolie: 30 seconds would be haproxy
<spm> ...
<wgrant> spm: What's the queue depth like on the haproxies?
<poolie> is there anything that distinguishes different types of failure short of an oops?
<poolie> ie haproxy vs apache ?
<poolie> i mean, anything visible in the page
<wgrant> There are slight differences in the HTML. But otherwise not really, AFAIK.
<spm> wgrant: currently 0 down the line, but has had a max of 65. banana.
<wgrant> Hmm. Not so bad.
<poolie> perhaps something subtle would generate better bug reports? i don't know
<wgrant> The appserver request graphs are a little empty.
<danilos> rvba, r=me on your branch from yesterday
<danilos> rvba, not sure about str(enum), so please check with someone else (though this seems right)
<rvba> danilos: great, thanks ... the str(enum) works, but I wonder if it's the Right Way to do it. I'll ask around.
<lifeless> a'voir (maybe back later)
<rvba> danilos: I'll wait a bit to land this branch since StevenK's branch which is depends on is still in the works ...
<danilos> rvba, understood :)
<StevenK> rvba: I was going to mention that to you. One column, using an enum
<adeuring> good morning
<mrevell> Morning
<jtv> hi mrevell
<mrevell> hey jtb
<mrevell> er, jtv
<jtv> :)
<lifeless> poolie: hi
<lifeless> poolie: an unthemed 503 is a complete fail in the zope stack (error handler barfing)
<lifeless> poolie: a themed oops is a regula fail in the zope stack
<lifeless> poolie: a themed 'could not connect' is haproxy/apache (I forget the exact difference, 504/503 I think respectively
<lifeless> poolie: and a browser-themed page implies a networking /ssl error
<lifeless> still a little excessive 1183  OOPS-1943EC114  POFile:+translate
<jtv> StevenK, lifeless: I updated the jobs documentation.  If I got anything horribly wrong then please let me know, that future generations may be entertained and edified.  Look for "run_jobs."  https://dev.launchpad.net/Foundations/JobSystem
* allenap changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: allenap | https://code.launchpad.net/launchpad-project/+activereviews
<jtv> allenap: could I trouble you for a review?  It's large, but wgrant's already gone over it once and StevenK also made suggestions.  The only _really_ new part compared to what they looked at, apart from their suggestions, is some documentation for JobCronScript.
<jtv> It's this one: https://code.launchpad.net/~jtv/launchpad/db-bug-766807/+merge/59027
<allenap> jtv: Sure :)
<jtv> Great, thanks
<allenap> jtv: Just revision 10479 then?
<allenap> Ah, no revision 10477 too.
<jtv> allenap: well william's review also needs mentoring.
<allenap> jtv: Okay.
<jtv> BTW I updated the job-system docs on the wiki to match what I did here, so you may not want to trust it too much as validation for this branch.
<poolie> can you no longer request another review from someone who already commented?
<poolie> or was it always thus?
<poolie> also, could i get another review on <https://code.launchpad.net/~mbp/launchpad/mail-cleanup/+merge/58867>
<rvba> wgrant: Hi! Steven suggested me to request your opinion/suggestions about a task I'm working on ... would you have some time to help me with that?
<rvba> I need to change the perm check for syncing packages (from the +localpackagediff pages) from lp.Edit on the series to lp.Append on the archive.
<wgrant> rvba: Sure.
<rvba> The tricky part is that this check cannot be done in a security adapter because it requires the pocket and component to be available and it's not the case in a security adapter.
<rvba> great, thanks
<wgrant> lp.Append on the archive does not use the pocket or component, and is not what you want.
<rvba> wgrant: here a (very much in the works) mp https://code.launchpad.net/~rvb/launchpad/change-perm-sync/+merge/59341
<wgrant> You mean you want it to respect actual upload privileges?
<rvba> it's how I understand it yes
<rvba> the tickets says 'syncing packages should be lp.append on the archive, not lp.edit on the series'
<wgrant> Right. We currently use launchpad.Append in parts of the code as a hack to very roughly approximate that.
<wgrant> Ah.
<wgrant> So, probably easiest to use lp.Append to start with, if that's all Julian wants for now.
<wgrant> That doesn't need the pocket or component.
<wgrant> Which bug is this?
<rvba> but he also told me that I will need pocket and component ... and this won't be able to use a security adatper.
<rvba> there is no but for this yet.
<wgrant> He probably didn't mean launchpad.Append, then :/
<rvba> that's wht I figured
<wgrant> You may want to look at package branches.
<rvba> wgrant: perhaps you could have a look at the partial diff on the MP and tell me what you think
<rvba> I guess the tricky part is that I not only have to figure out a solution for this ... but also what the problem is :)
<wgrant> rvba: Do you have a quote of what Julian said? I saw a bug to use launchpad.Append some time ago, but this sounds like far more than that.
 * rvba scans the logs
<rvba> avril 27 10:46:55 <bigjools>    rvba: and the permission change
<rvba> avril 27 10:47:09 <bigjools>    but that one is hard because of the problem we encountered before
<rvba> avril 27 10:47:28 <bigjools>    (we need more info than available in the security adapter)
<rvba> avril 27 10:49:04 <bigjools>    I think we can do it, but it will mean not doing it in the security adapter
<wgrant> Right, so it sounds like he wants it to go all the way.
<rvba> and also (from an email)
<wgrant> rvba: You'll want to make use of Archive.canUploadSuiteSourcePackage.
<rvba> The security adapters only have the context archive and the user.  Far more is
<rvba> needed to ascertain upload privileges. So no, it isn't.  ;)
<rvba> It can be worked around by doing the security in the model layer though (not
<rvba> the browser layer - so it also works for the API).
<wgrant> rvba: That takes a person, suite and source package name, and does the rest for you.
<rvba> wgrant: I reckon this is sort of what I did ... but I called directly Archive.checkUpload ... which is what canUploadSuiteSourcePackage seems to do
<wgrant> rvba: Also, this probably needs to be flagged. syncSource must not be used to copy into a primary archive yet.
<wgrant> rvba: You're using the source component instead of the destination component, but apart from that it looks OK.
<rvba> wgrant: how can I get the destination component?
<wgrant> rvba: canUploadSuiteSourcePackage has a reasonable example.
<rvba> wgrant: (flag) all I do here is add a perm check to existing code ... and this perm check will only be performed if a not None person is passed, which is something that will be done only in flag protected code
<rvba> wgrant: so I guess I should be covered don't you think?
<rvba> component = sourcepackage.latest_published_component
<wgrant> rvba: Ah, you're not also doing this through the API? Sure.
<rvba> wgrant: not yet, but I've got another task waiting to expose the sync operation on the api
<wgrant> rvba: syncSource is already exposed.
<wgrant> (but restricted to launchpad.Append)
<rvba> yes, I got confused: I meant doing the same check for the api call
<wgrant> That's a long way off.
<rvba> and remove the lp.Append protection
<wgrant> It needs to do overrides and announcements and respect the queue first.
<rvba> I see
<rvba> wgrant: I think I'll wait for Julian for the api stuff then. At this stage I just wanted to make sure that my changes will be compatible with this future api refactoring.
<rvba> wgrant: I think I'll change the call from checkUpload to canUploadSuiteSourcePackage ... looks like it's more consistent to use most high "level code".
<rvba> wgrant: thanks a lot for you help ... I guess I'll have a good time writing tests for this :).
<rvba> s/you/your/
<wgrant> rvba: It looks reasonable. Although I wonder how it's going to behave for anonymous requests; self.user will be None, so person will be, so it will skip the check.
<rvba> wgrant: well spotted ... I'll have to find a way around that ...
<rvba> wgrant: in my experience (that is with other systems) the classic work around is for the infrastructure to pass a singleton AnonymousPerson and not None when dealing with anon requests
<wgrant> rvba: Yeah :(
<rvba> wgrant: I'll find a way ... thanks again for you time ... I might be back tomorrow with more questions ;)
<rvba> sigh s/you/your/
<wgrant> Heh.
<rvba> StevenK: you might want to have a look at the above conversation :)
<jtv> allenap: still reviewing?
<gary_poster> allenap, hi.  https://code.launchpad.net/~gary/launchpad/bug771232/+merge/59302 could use a review if you have time.  If not, lemme know; I can ask someone on yellow to look at it.
<allenap> gary_poster: Sure I'll look at it after I've finished jtv's review.
<gary_poster> stub, forgive my paranoia, but buildbot blessed my db branch 15 hours ago and it is on db-stable but not on staging yet.  Should it be there already--is this indicative of a potential problem in the patch?
<gary_poster> allenap, thanks!
 * stub shrugs
<gary_poster> uh, ok :-)
 * stub has a look at the staging logs
<gary_poster> thank you
<gary_poster> I could do that too
<stub> Might be faster... my net is the suck.
<stub> I think some twats using our data centre have released an operating system or something.
<gary_poster> heh
<gary_poster> allenap, another favor I could ask you is to go to https://launchpad.net/launchpad/+subscriptions and tell me if you see any subscriptions now that you are in ~yellow (you definitely should see the one from now Launchpad Bug Contacts, and if you don't that is a bug).  You could also take a look at https://launchpad.net/launchpad-project/+subscriptions .
<stub> gary_poster: What was the patch number? Looks like the update happened today but there were no patches to apply.
<gary_poster> stub, 65 I think.  checking
<stub> Not there. Last patch applied was -61 on 24th.
<gary_poster> stub, ok, I'll try to hold off on the paranoia some more than.  Thank you!
<stub> r10480 of db-stable
<jtv> allenap: no questions so far... is that good or bad?  :)
<allenap> gary_poster: On launchpad/+subs I see the ~launchpad-bugs subscription, and on launchpad-project/+subs I see my "Triage" subscription, so all seems well. Thanks for following up on that.
<allenap> jtv: I have some comments, but no breakage.
<jtv> Sweet.
<gary_poster> allenap, yay!  thanks for looking
<allenap> jtv: I started out doing a full review because I'm not used to mentoring, but I'm just skimming through now.
<jtv> allenap: one design worry is that we haven't got the full initialization procedure together yet for derived distros, so we're not sure yet how to slot this in there.
<jtv> The actual code should be distro-agnostic though.
<jtv> The main concern for the procedure is that there be no concurrent regular publisher runs.  We'll probably develop different ways of solving that anyway.
<wgrant> jtv: The great more-extra overrides mystery is solved.
<jtv> ?
<wgrant> jtv: The symlinks are indeed dangling for the first run, and the dists/ comparison ignores the headers that are missing because of it.
<wgrant> So we can just leave them dangling and everything will be fine.
<jtv> Good to know.  (I don't think it'd have affected my code because that's only kicked off after the links have been created).  Might be worth noting somewhere if this wasn't previously known.
<henninge> allenap: Hi!
<allenap> henninge: Hi!
<henninge> allenap: Can you please review my branch? ;)
<henninge> allenap: https://code.launchpad.net/~henninge/launchpad/devel-766955-fix-forwardCheckAuthenticated/+merge/59364
<allenap> henninge: Sure.
<allenap> henninge: I've just started a review for Gary, so I'll do yours right after that.
<henninge> allenap: thanks!
* jcsackett_ changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: allenap, jcsackett | https://code.launchpad.net/launchpad-project/+activereviews
<jtv1> thanks allenap!
<allenap> jtv1: No worries.
* barjavel.freenode.net changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: allenap | https://code.launchpad.net/launchpad-project/+activereviews
 * henninge relocates
<gary_poster> mrevell_ (_?) hi .  You assigned yourself https://bugs.launchpad.net/launchpad/+bug/770315 .  Thank you! Will you have a chance to look at this today or tomorrow?  You are on holiday Monday, right?
<_mup_> Bug #770315: Help for bug muting uses unfamiliar terminology <exploratory-testing> <story-better-bug-notification> <Launchpad itself:Triaged by matthew.revell> < https://launchpad.net/bugs/770315 >
<mrevell_> gary_poster, Hi! Not sure why I have an underscore. Yeah, I'll deal with that this afternoon.
<gary_poster> awesome, thank you mrevell_
<jcsackett> henninge: can i bug you for some info on reviewing translation imports?
<henninge> jcsackett: sure ;)
<jcsackett> henninge: excellent. i am looking at https://translations.launchpad.net/+imports/5469865, and trying to verify the project is set up appropriately.
<jcsackett> the page tells me there is no translatable series, but i see that it is targeting a series. i believe i am confused by what that all means.
<henninge> jcsackett: "translatable series" refers to a series for which a template has been imported.
<jcsackett> ah, so approving this would create a translatable series, henninge?
<henninge> jcsackett: so this tells you that this is the first template import ever for this project.
<henninge> jcsackett: yes
<jcsackett> henninge: okay, that makes sense.
<jcsackett> henninge: there is also a thing in the wiki saying that i should look at the product under translations, and check links for release-series to make sure things are setup.
<jcsackett> in this case, looking at translations.launchpad.net/libravatar, i see nothing that looks like what the wiki is talking about.
<mrevell> danilos, Hey, do you have a moment to talk about bug 770345?
<_mup_> Bug #770345: When muting a bug your name disappears from the subscribers list but doesn't re-appear when unmuting <exploratory-testing> <story-better-bug-notification> <Launchpad itself:Triaged by yellow> < https://launchpad.net/bugs/770345 >
<danilos> mrevell, sure thing, though note that gary_poster is writing a comment as well :)
<danilos> mrevell, want to do voice?
<danilos> mrevell, (though, mumble is broken for me, so skype would be better)
<mrevell> danilos, I'll wait and see what Gary's comment says :)
<danilos> mrevell, I also think you've raised most of the important comments in 770342 as well, fwiw
<mrevell> danilos, Did I? Oh, good. /me tries to work out what.
<gary_poster> danilos, mrevell, I made my comment.  I don't like the situation at all, but I've said what I thought. :-) https://bugs.launchpad.net/launchpad/+bug/770345
<_mup_> Bug #770345: When muting a bug your name disappears from the subscribers list but doesn't re-appear when unmuting <exploratory-testing> <story-better-bug-notification> <Launchpad itself:Invalid by yellow> < https://launchpad.net/bugs/770345 >
<gary_poster> mrevell, we can tackle the internal change, but IMO we need approval from Jono and Francis to do so.
 * gary_poster needs to run to dr appt.  biab
<jcsackett> sinzui: you triaged a bug right out from under me. :-P
<snowdream> i want to uninstall launchpad ,what can i do?
<danilos> mrevell, if you still want to chat about it, just ping me :)
<snowdream> anybody can help me unistall launchpad
<henninge> allenap: thanks for the review. How strongly do you feel about assertVectorEqual?
<allenap> henninge: I'm not going to fight if you really want to land it :) I agree it's probably useful to get the full state of something when a test breaks, rather than just the first failure, but I don't think assertVectorEqual is a good way to go about it.
<mrevell> danilos, Okay, ta
<henninge> allenap: So, you are ok with agreeing to disagree on that point?
<allenap> henninge: Something like that :)
<henninge> allenap: thanks ;-)
<henninge> Nooo!
<jcsackett> sinzui: have a few to mumble?
<jtv1> henninge: what happened?
<henninge> I lp-landed my branch but I meant to ec2-land it ...
<jtv> henninge: if you "ec2 test" it now, you'll probably still be just ahead of buildbot to patch up any failures.
<henninge> jtv: yes, I was thinking about that, too.
<henninge> I have 3 hours 38 minutes
<jtv> run, henning, run!
<jtv> You can also do a local run for the most likely suspects, of course.
<jcsackett> henninge: did you already get pqm notification of success? i don't see it in the queue, which implies it's already through, but could mean it hasn't hit it quite yet?
<henninge> jcsackett: it's through
<jcsackett> blast.
<henninge> jtv: problem is I won't be here then.
<henninge> I don't expect much to happen.
<henninge> I'll start the ec2 test run now
<jtv> henninge: I'd also start that local run.  May just catch something before you leave.
<henninge> jtv: good point
<wgrant> Oh good, it'll fail just in time for my start-of-day :P
<jtv> wgrant: while you're still here (and I know you're not), is this accusation I just heard true?
<wgrant> Hwhich accusation?
<jtv> Do some Australians spread that marmite-clone they have (or however you spell it) right next to nutella?
<wgrant> I've never heard of that being done seriously, fortunately.
<wgrant> Or I would have to abandon the nation.
<jtv> wgrant: thank you, that is an immense relief.  I will pass the news on.
<jtv> wgrant: I do have an eyewitness here, but that may have been a freak madman, not representative of the wider gene pool.
<jtv> It happened in Hobart, for what it's worth.  Lots of inbreeding there?
<wgrant> Indeed.
<danilos> allenap, hi, do you think you'll have time to look at https://code.launchpad.net/~danilo/launchpad/subscribers-list-js/+merge/59387? it's mostly JS tests :)
* jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: allenap, jcsackett | https://code.launchpad.net/launchpad-project/+activereviews
<jcsackett> danilos: i can grab it. not sure why my attempt to set topic this morning failed.
<danilos> jcsackett, excellent, thanks :)
<mrevell> allenap, jcsackett: If either of you have time for https://code.launchpad.net/~matthew.revell/launchpad/bug770315-bug-muting-help/+merge/59388 I'd be most grateful :)
<jcsackett> mrevell: if allenap doesn't take it before i finish one for danilos, i'll get it.
<adeuring1> allenap: got a few minutes for a pre-imp call? (about duplicate bugs)
<allenap> mrevell: Sure, I'll take that.
<mrevell> thanks
<allenap> adeuring1: Sure. Skype?
* allenap changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jcsackett | https://code.launchpad.net/launchpad-project/+activereviews
<adeuring1> allenap: ok, let me just power on a machine where skype is installed.
<allenap> adeuring1: Or Mumble is okay.
<adeuring1> allenap: then let's mumble ;)
<henninge> jtv: I now have a local run for most likely suspects going plus a full ec2 run.
<jtv> Very well.
<adeuring1> allenap: https://bugs.launchpad.net/launchpad/+bug/769293
<_mup_> Bug #769293: Timeout trying to set bunch of dupes <Launchpad itself:In Progress by adeuring> < https://launchpad.net/bugs/769293 >
<jcsackett> danilos: this is all my own javascript ignorance, but i see that remove_user_link calls for two params, subscriber and is_dupe. in places you call it with only one argument, and i don't understand if/how is_dupe is set to a default. is there js magic i don't get?
<sinzui> jcsackett: I can talk. I was distracted by the flood of water and felt compelled to bail
<jcsackett> sinzui: http://people.canonical.com/~jc/images/mark-as-spam-0.png
<danilos> jcsackett, yeah, the magic is that it gets set to "undefined"
<danilos> jcsackett, thus, I explicitely check for "true" when I want it to be used, I assume everything else means "no"
<jcsackett> danilos: ah, ok. that makes sense. thanks!
<danilos> jcsackett, np
<henninge> Do OOPS ids expire?
<henninge> I have an old bug report that refers to an OOPS but the utility cannot find it.
<danilos> henninge, they do not expire afaik, but they are sometimes manually purged by LOSAs/QA team I think; worth checking with matsubara/Ursinha
<Ursinha> hi
<wgrant> oops-prune removes old unreferenced OOPSes.
<henninge> Hi Ursinha! ;)
<Ursinha> henninge, there's a script, oops-prune, that removes old oopses
<Ursinha> hi henninge! :)
<henninge> How does it know they are unreferenced?
<jcsackett> danilos: r=me.
<danilos> jcsackett, thanks
<Ursinha> henninge, what do you mean? it doesn't :( it deletes old oopses without any checking
<henninge> Ursinha: ah, wgrant mentioned "unreferenced". Thanks for clarifying.
<henninge> so, no use putting OOPS urls in bug reports, just paste the oops itself.
<henninge> I really have a hard time reproducing the bug now
<Ursinha> :/
<henninge> jtv, jcsackett, wgrant: My local run did not have any troubles. I am quite confident that it will be fine. ec2 still running, of course. I will check back later.
<jtv> Splended
<jtv> *did
<henninge> bye for now
<gary_poster> mrevell, hi, I'm back and available for call if you'd like to have one; OTOH, I don't have much to report that you haven't already experienced first hand :-)
<gary_poster> s/much/anything/
<mrevell> gary_poster, Hi :) Thanks. I think we've worked pretty closely this week, so I'm happy to skip the call.
<gary_poster> cool thanks mrevell.  ttyl
<jcsackett> sinzui: can i put you on help contact, or are you still bailing water?
<sinzui> yes. I continue to forget
<sinzui> jcsackett: can you review https://code.launchpad.net/~sinzui/launchpad/question-email-1/+merge/59414
<jcsackett> sinzui: yes. it will give me something to do while my lp-dependencies re-install.
<sinzui> thanks
<lifeless> moin
<jcsackett> sinzui: r=me.
<benji> lifeless: if I get the impression that changes to database/schema/security.cfg don't have to go onto the DB branch; is that correct?
<benji> (I'm giving the processmail user access to the table where the "I don't want to get messages about things I do" flag is stored.)
<benji> s/if//
<lifeless> benji: thats correct
<lifeless> security.py --no-revoke is run before nodowntime deploys
<benji> cool, thanks
<wallyworld> lifeless: can you please +1 on https://code.launchpad.net/~wallyworld/launchpad/poppy-sftp-gpgconf/+merge/59154
<lifeless> wallyworld: when I get a browser back, sure
<lifeless> you'll need to remind me after I disconnect and reconnect
<wallyworld> lifeless: np. your system broken?
<lifeless> nz ubuntu mirror was horked
<lifeless> currently in terminal only while my last -> natty upgrade completes
<wallyworld> good luck :-)
* jcsackett changed the topic of #launchpad-dev to: https:/â/âdev.launchpad.net/â | On call reviewer: - | https:/â/âcode.launchpad.net/âlaunchpad-project/â+activereviews
<lifeless> wallyworld: url me up
<wallyworld> lifeless: turns out curtis did the +1 just after i asked you.  thanks anyway
#launchpad-dev 2011-04-29
<lifeless> lk
<poolie> lifeless, hi there,
<poolie> we're just cleaning out some bugs like bug 722043 where bzr users report ssl timeouts getting to launchpad
<lifeless> hi poolie
<_mup_> Bug #722043: bzr crashed on launchpad-login <amd64> <apport-crash> <natty> <running-unity> <bzr (Ubuntu):New> < https://launchpad.net/bugs/722043 >
<poolie> obviously there is one easy bug here which is that bzr should not give a traceback
<poolie> assuming we are not doing something dumb that gives a short timeout or causes a spurious failure
<poolie> and also assuming it's not a local network problem
<poolie> i guess it may be an lp issue
<poolie> but, probably just operational or intermittent, and perhaps not worth fliing as a bug
<sinzui> wgrant: mumble?
<lifeless> poolie: this is the thing that that thomas guy from openstack whas having issues with, Ithink
<poolie> there are a few dupes
<lifeless> mmm
<lifeless> so launchpad-login talks to xmlrpc
<lifeless> the get one is talking to a different vhost
<lifeless> and not using the xmlrpc proxy
<lifeless> perhaps its the same, perhaps not
<lifeless> anyhow
<poolie> so, soon if not already, we should cease using xmlrpc on trunk
<lifeless> we want to know of these issues so we can fix, where the fix is at all doable by us
<poolie> (maybe not for anonymous access?)
<wgrant> sinzui: mumble mumble sure.
<poolie> but the one talking to the webapp will still exist
<wallyworld> sinzui: we can hear you
<lifeless> yup
<wgrant> lifeless: I eliminated packagebuild last night.
<wgrant> buildfarmjob will hopefully die today.
<wgrant> Apart from a couple of Storm bugs it was fairly simple.
<wgrant> (inherited References == breakage)
<lifeless> \o/
<lifeless> wgrant: and bpb ?
<wgrant> lifeless: PB has been flattened into SPRB and BPB.
<lifeless> oh right
<lifeless> I mismentallyhandled your change
<wgrant> They both now reference BuildFarmJob, which I will hopefully flatten into them today,.
<wgrant> But it's slightly harder.
<lifeless> so bfj is all that remains to flatten
<wgrant> Since we query based on it at times.
<lifeless> it has three users
<wgrant> Yeah.
<wgrant> There are about two or three methods that use BuildFarmJob in a generic way, so I just need to fix them.
 * wallyworld off to doctor for blood test
<lifeless> hmm
<lifeless> we needs a deploy
<wgrant> We do.
 * wgrant QAs process-accepted.
 * wallyworld shakes fist at windmill... again
<wgrant> wallyworld: Oh?
<wallyworld> why dies waits.forElement(xpath=xxx) work but asserts.assertNode(xpath=xxx) fail
<wallyworld> same xpath
<wgrant> Hah
<wallyworld> haven't looked at the code, but you would think it would work
<wallyworld> anyhoo, another test passes
<wallyworld> again
<wgrant> wallyworld: Which? The broken one?
<wallyworld> till next time something breaks
<wgrant> StevenK: Is canonicloud broken?
<wallyworld> test_filebug_extra_options
<StevenK> Likely suffering from the release DDoS
<wgrant> Yeah.
<wgrant> wallyworld: Hmm.
<wallyworld> wgrant: plus the xpath was actually wrong
<wallyworld> wgrant: i fixed it and then was wondering why it still failed
<StevenK> wgrant: Yes, tis broken.
<wgrant> StevenK: Great.
<wgrant> As long as it's not our problem.
<StevenK> Ha. Those nodes are going to take a WHILE to come up.
<wgrant> Oh?
<wgrant> a.u.c overloaded?
<StevenK> Yup
<lifeless> hmm
<lifeless> now I have unity on my primary machine, more quirks emerge/
<wgrant> Oh?
<lifeless> the machine scrollbars don't show up reliably
<lifeless> I'm having to do a binary chop along the length of the window to find the horizontal scroll handle
<wgrant> Hah
<lifeless> wgrant: you don't have that problem?
<wgrant> lifeless: It seems to only appear next to the thumb.
<lifeless> wgrant: thumb?
<wgrant> lifeless: The orange/filled bit of the scrollbar.
<lifeless> I just have a thin grey line along the bottom of the screen
<wgrant> Oh, not sure about horizontal scrollbars.
<lifeless> Ok, squinting I can just make out that there is a difference ther.
<lifeless> I think this sill last until I get a headache
<lifeless> :(
<wgrant> I quite like the new scrollbars.
<wgrant> Except the whole "1px should be enough for everybody" thing.
<lifeless> I can't make out the thumb on the vertical on my CRT at all
<wgrant> Hmm, it's fairly obvious here.
<wgrant> But I don't use a CRT.
<lifeless> I have to move the mouse downthe window  carefully to stay in the grab region width - until the grab appears
<lifeless> to figure out how far through the document I am
 * StevenK tries to work out how to get from a ResultSet to a hash
<lifeless> a hash?
<StevenK> Er, sorry, Perl-ism
<StevenK> dict
<lifeless> your perl is showing ...
<StevenK> :-)
<lifeless> map/dict/whatever you want
<lifeless> the default is to give you objects
<StevenK> Hm, which aren't really keyable
<StevenK> But the code looks it up anyway
<lifeless> perhaps some context would help
<StevenK> lifeless: I'm fixing populate_distroseriesdiff to use DistroSeriesParent
<StevenK> lifeless: DistroSeriesParent is a table that shows the relationship between a parent and a derived series -- please note that this is a Many-to-1 relationship WRT to the child.
<StevenK> lifeless: So I'm fixing the code to deal with multiple parents, but there is code that looks up all derived series -- this used to be simple, since we assumed that if IDistroSeries.parent_series was set, and wasn't the same Distribution as the child, they are derived.
<lifeless> StevenK: isn't it just a join now ?
<StevenK> Since I deal with multiple parents, I need to loop, which is fine, but I'm trying to work out how to store the mapping of child to parent(s) -- a dict of {'child': ['parent1', 'parent2']} sounds like a good solution
<lifeless> child_distroes = list(store.find(distroseries, distroseriesparent.childID==distroseries.id, distroseriesparent.parentID==self.id))
<lifeless> do you need transitive children or direct children ?
<StevenK> Actually, this is seperate code, and is wanting to find all derived distroseries, regardless of parent
<lifeless> if you need transitive children you want a with clause and a recusrsive query.
<lifeless> StevenK: so you want all distroseries that are derivatives?
<StevenK> (IE, self.id makes no sense)
<lifeless> derivatives = list(store.find(distroseries, distroseriesparent.childID==distroseries.id))
<StevenK> lifeless: I can find that easily enough -- I'm trying to figure out how to store the information that doesn't result in a massive loop over a ResultSet
<lifeless> derivates = set(store.find(distroseries, distroseriesparent.childID==distroseries.id))
<lifeless> ?
<StevenK> But how does that help the multiple parents case?
<StevenK> I'd also like to get the parents at the same time -- pre-loading them
<lifeless> StevenK: you didn't say you wanted the parents :)
<StevenK> Oh, sorry. I do. :-)
<lifeless> derivatives = set(DecoratedResultSet(store.find((DistroSeries, DistroSeriesParent), DistroSeriesParent.childID==DistroSeries.id), operator.itemgetter(0)))
<lifeless> mmm
<lifeless> .parents will be a join
<lifeless> so you'll need a more complex one - you need a cached property on DistroSeries that can remember the parents for you
<lifeless> then
<lifeless> def eager_load(rows):
<lifeless>     for distroseries, parent in rows:
<lifeless>     cachedpropery_manager(distroseries)._parents.append(parent)
<lifeless> [with the obvious tweak to handle the first parent]
<lifeless> and the query becomes
<lifeless> derivatives = set(DecoratedResultSet(store.find((DistroSeries, DistroSeriesParent), DistroSeriesParent.childID==DistroSeries.id), pre_iter_hook=eager_load)
<lifeless> )
<jtv> I finally realized why people wear trousers at work.
<StevenK> Just now?
<jtv> Yes.  Sometimes coffee mugs just break at the handle, and I just realized how much worse things could have been.
<jtv> I knew there had to be a reason, since so many people were doing it.
<jtv> But does that imply a good reason?  Many people use Windows.
<jtv> So quite an epiphany.
<jtv> *However there is the corollary of multi-touch trackpads, Unity gestures, and electricity-conducing liquids not mixing well.
<jtv> No amount of trousing is going to help with that.
<jtv> Better shut down and let this dry up... no way I'm getting anything done like this.
<StevenK> lifeless: Where is cachedproperty_manager defined? My grep didn't find it.
<lifeless> from lp.services.propertycache
<StevenK> Ta, I shall dig there to make sure of _ placement and so on
<lifeless> StevenK: I've given you a brief sketch; if you have trouble shout and I'll explain missing bits
<lifeless> StevenK: this idiom is in reasonably widespread use
<lifeless> FWIW
<StevenK> Do you mean cachedproperty(distroseries)._parent, by the way?
<StevenK> lifeless: ^ I don't think you do, since cachedproperty is a decorator
<lifeless> right
<lifeless> get_property_cache
<lifeless> lets you get at the cache
<lifeless> .parents should be the attribute, if the cachedproperty will be 'def parents'
<lifeless> and you want the regular join attribute to be on _parents
<lifeless> so you have
<lifeless> @cachedproperty
<lifeless> def parents(self):;
<lifeless>     return list(self._parents)
<lifeless> which will support folk doing late evaluation of the parents
<StevenK> I see where you're going -- adding a decorated method onto IDistroSeries
<lifeless> when eager loading you inject into the property
<lifeless> evening stub
<lifeless> I haven't made any trigger progress
<lifeless> had a day of copying vms around basically ;<
<stub> yo
<stub> ok
<poolie> hi all
<poolie> that '-' bastard is a slack reviewer
<StevenK> Haha
<StevenK> poolie: There is no OCR for Friday APAC
<poolie> StevenK, i don't recall often seeing one in apac timezones
<poolie> in fact i had kind of thought the concept was dead entirely
<poolie> but apparently not in eu/us
<StevenK> poolie: https://dev.launchpad.net/ReviewerSchedule
<poolie> what should i do to get https://code.launchpad.net/~mbp/launchpad/mail-cleanup/+merge/58867 going again?
<wgrant> APAC reviewers tend to defect to other parts of Canonical.
<StevenK> Haha
<poolie> oh huh
<StevenK> poolie: Given the time, I'd suggest waiting for adeuring and asking him.
<poolie> realistically i may not be able to land it and catch fallout before i leave for uds etc
<poolie> k thanks
<StevenK> I think wallyworld should become a reviewer*
 * StevenK drops thumper and leonardr from the table
<poolie> good idea
<StevenK> And rockstar :-(
<stub> poolie: That MP looks approved to me
<poolie> because of the 'approve code'?
<StevenK> Yes, benji approves
<poolie> lp is a bit unclear about votes vs mp status
<wallyworld> StevenK: i sort of already am, but maybe not "formally"
<poolie> so i can set the state to approved?
<StevenK> poolie: If you feel it has been reviewed enough, sure.
<poolie> i would actually like a response to my comments
<poolie> or a sanity check of them
<wallyworld> StevenK: but i see i'm not listed. i'll get that fixed
<StevenK> wallyworld: I'd suggest you talk to bac
<wallyworld> StevenK: ok. i originally talked to tim
<StevenK> poolie: Right, if you're not sure, then I suggest you do what I first suggested and discuss your concerns with the OCR today, which is Abel.
<wallyworld> StevenK: i'm rooming with bac at the summit and uds, so i'll bring it up in person :-)
<stub> poolie: I'm looking over it now. First thing, is bare execpts are always evil and should be except Exception:, or else you cause grief with Ctrl-C handling etc.
<StevenK> Or you could wait for stub to finish. :-)
<StevenK> wallyworld: Sounds like a plan. Then he can't escape. :-)
<poolie> thanks stub
<wallyworld> yup :-)
<poolie> the bare except was already there, but i don't mind changing it
<poolie> since it's generating an oops, arguably it should be catching internal errors
<stub> It would be nice. I consider it a bug unless there are big fat comments explaining why BaseException exceptions need to be caught too.
<poolie> but as i said, recovering from that gets messy
<stub> From my POV, anything that it doesn't catch should either not log an oops, or will mean the environment is so unstable you probably can't log an OOPS.
<stub> So +1 for the big fat comment, but I don't think it justifies the bare except in this case but haven't thought it through as much as you might have :)
 * stub wonders how this interacts with the OOPS handling built into the logging system now
<stub> I guess it isn't invoked since you are not calling log.exception()
<stub> And you need to do it yourself to email the OOPS id.
<lifeless> y'know, friday night.
<lifeless> I might have a *beer*
<stub> You rebel.
<stub> poolie: Do we want an expiry time for the email we stuff in the Librarian? Or is the default 72 hour stay of execution enough?
<wgrant> lifeless: Fixed your machines yet?
<poolie> lifeless, you promised me a beer last week
<poolie> stub, i'm just moving the exception code around not changing it
<poolie> similarly for the oops logging
<poolie> if people are finding 72h enough so far perhaps that is enough
<wgrant> Yes, but we have so much bad code it's always good to be pedantic about stuff as we see it :)
<stub> ok
<lifeless> poolie: I did
<lifeless> poolie: if you're ok with the odd 'one sec while I turn the potatoes' we could chat ~now
<stub> poolie: You can fix a long standing bug here if you like by converting the domain to lowercase around line 415
<stub> second review in anyway, not that it is needed :)
<lifeless> poolie: so, did you want skypebeer ?
<lifeless> wallyworld: sorry mate, bad news for you
<poolie> thanks stub i appreciate it
<adeuring> good morning
<StevenK> adeuring: Morning!
<adeuring> hi StevenK!
<poolie> hi adeuring
<adeuring> hi poolie!
<poolie> i put up a tiny mp at https://code.launchpad.net/~mbp/launchpad/feature-scope-cleanup/+merge/59460
<poolie> edging up on more scope work
<poolie> i'd appreciate a review
<poolie> also a meta review on whether it is too small of a chunk to land
<poolie> (otoh it is probably all i will do on that today, so it's worth pushing anyhow)
<wallyworld> lifeless: ?? bad news about?
<lifeless> wallyworld: gpghandler branch
<wallyworld> lifeless: yeah, but i redid it and it's back in ec2
<lifeless> wallyworld: your redo tries to run twisted code in the main appserver stack
<lifeless> wallyworld: I assure you that thats not a good idea
<wallyworld> oh. why?
<lifeless> because its zope
<lifeless> not twisted
<wallyworld> zope is an app server framework, no? so surely it can handle services eunning within it?
<lifeless> its threaded
<lifeless> zope is callbacks in a single thread
<lifeless> entirely different models
<wallyworld> :-(
<lifeless> not like the EJB environments you may be used to
<wallyworld> yeah, looks like i found ou tthe hard way
<lifeless> imagine running tomcat specific code in a jboss container
<wallyworld> so i'll cancel the ec2 run
<lifeless> it might work
<lifeless> or it might appear to work and crash and burn after several days uptime
<lifeless> I also noticed another bug
<wallyworld> well, the tests which instantiate the zope layer "work" in isolation. so perhaps the latter
<lifeless> yah
<lifeless> (and tomcat and jboss are near identical when you consider the zope / twisted differences)
<lifeless> I've commented on the other bug
<wallyworld> lifeless: perhaps the best solution is to create a cron script which just touches anything under /tmp/gpg-* ??
<lifeless> we have several hours analysis to do I suspect
<StevenK> wallyworld: *Ew*
<wallyworld> ok, i'll look at the comment, after i play soccer
<lifeless> wallyworld: Enjoy the weekend
<lifeless> wallyworld: I think the right thing to do is to disable this 'feature' for now.
<wallyworld> StevenK: i know, but atm i'm out of ideas
<lifeless> this is actually non-trivial engineering to fix.
<lifeless> the gpg check on upload is nice but not -required-
<wallyworld> lifeless: so it seems
<lifeless> it was added as part of migrating away from the zope ftp stack
<lifeless> conflating two things
<lifeless> I would like to see this work properly
<wallyworld> lifeless: so, i'll back out my changes and just disable the check on upload?
<lifeless> wallyworld: yeah
<wallyworld> ok
<lifeless> I don't like the idea of a maintenance squad disappearing down this rats nest at the moment
<wallyworld> yeah
<lifeless> we can have a high bug open to reinstate it
<lifeless> and note that its a couple days work because having a static dir leads to conflicts, having dynamic dirs needs this if there is a tmpreaper, but 'this' needs to be conditional on the specific appserver thats running
<lifeless> and AFAIK we don't have 'if' statements in our zcml
<lifeless> I suspect that we process zcml so early in the piece you can't tell if its a twisted thing or not reliably. IMBW
* adeuring changed the topic of #launchpad-dev to:  https:/â/âdev.launchpad.net/â | On call reviewer: adeuring | https:/â/âcode.launchpad.net/âlaunchpad-project/â+activereviews
<rvba> wgrant: Hi! As promised, I'm back with a question if you have a few minutes :)
<rvba> I finally decided to use Archive.checkUpload and not Archive.canUploadSuiteSourcePackage because if the user does not have the right perm, I need to be able to display the reason why not.
<rvba> But my question is related to what you mentionned yesterday: the fact that in my first version I was using the source component instead of the destination component. Even looking at how this is handled inside canUploadSuiteSourcePackage I'm not really sure what to do: what component to pass to checkUpload that it.
<rvba> More precisely (please correct me if I'm wrong):  the sync can be on a package that might not already exist in the destination series (so the sourcepackage.latest_published_component trick and the usage of a suitesourcepackage is not possible)
<wgrant> rvba: Yeah, that's difficult. At the moment any component (but not packageset) uploader can upload a new package.
<wgrant> It's not a good solution.
<wgrant> But it is at least consistent.
<rvba> this strict_component parameter seemed kinda hacky indeed
<wgrant> Hacks? Surely not.
<rvba> ho?
<wgrant> I would replicate the hacky check for now.
<rvba> So I guess my next question would be: why is this  strict_component needed?
<rvba> (I suppose it's a use case I don't know about)
<wgrant> rvba: So, originally only component-based upload privileges existed.
<wgrant> Packageset- and sourcepackage-specific privileges are a new addition.
<rvba> I see
<wgrant> So initially it just took a component and person, basically.
<wgrant> strict_component is used when the package is new.
<wgrant> So there's no component to check.
<rvba> # strict_component is True because the source package already
<rvba> # exists (otherwise we couldn't have a suitesourcepackage
<rvba>  # object) and nascentupload passes True as a matter of policy
<wgrant> Since there's no component to verify privileges against, and the package will end up in the NEW queue anyway (where it will be reviewed by an archive admin), it was decided to let any uploader upload it.
<rvba> # when the package exists.
<wgrant> Right.
<rvba> Not sure I understand what you're saying when I look at the comments I just pasted ...
<rvba> (comments com from canUploadSuiteSourcePackage)
<wgrant> That's a new method, from after the new permission scopes were added.
<wgrant> Have a look at Archive.verifyUpload.
<wgrant> It is a little confusingly structured.
<rvba> I thought that strict_component=True would be used when a package already exists (and thus is in a specific component)
<wgrant> But it first checks if you can upload to the package or its packageset: if so, success. Then it checks if you can upload to any components: if not, failure.
<wgrant> Right.
 * rvba looks at verifyUpload
<wgrant> That's right.
<wgrant> Then, if strict_component is set and the uploader doesn't have privileges on that component, failure.
<wgrant> Otherwise success.
<wgrant> 18:17:52 < wgrant> strict_component is used when the package is new.
<wgrant> By 'used' I mean 'changed from the default', ie. set to False.
<rvba> ok, makes sense ... if thought "used" = True.
<wgrant> Sorry, yeah, I was really unclear.
<rvba> so ... what component could I use for the sync to make sense ... I guess I have to do 2 different things. One if the package exists in the dest series and one other if it does not
<wgrant> Right. If it does exist, use the destination component. If it doesn't, strict_component=False.
<rvba> makes sense ... but how can I figure out what the dest component is?
<wgrant>         rejection_reason = self.policy.archive.checkUpload(
<wgrant>             uploader, self.policy.distroseries, source_name,
<wgrant>             self.changes.dsc.component, self.policy.pocket, not self.is_new)
<wgrant> This is what the uploader does.
<wgrant> dsc.component is the old component, if it's not new.
<wgrant> You can probably just pass None if it's new.
<wgrant> You need to get the SourcePackage object (distroseries.getSourcePackage(sourcepackagename)), then use latest_published_component.
<rvba> ok, makes sense. Thanks for the clarification, I really needed it :).
<StevenK> self.changes.dsc.component likely won't exist for what you want to check.
<StevenK> Ah
<StevenK> Maybe I should keep reading
<rvba> I think so :)
<poolie> adeuring, could i have an OCR of https://code.launchpad.net/~mbp/launchpad/feature-scope-cleanup/+merge/59460
<adeuring> poolie: sure
<poolie> thanks
<poolie> is not very big
<adeuring> poolie: r=me
<poolie> adeuring, thanks!
<deryck> Morning, all.
<henninge> Hi deryck!
<henninge> deryck: great to hear you are ok
<deryck> henninge, thanks!  it's been pretty rough here in Alabama the last day or so.  lots of loss.
<henninge> yes, it's been on the news here, too.
<deryck> My town was pretty hard hit.  I have friends who lost homes and loved ones.
<henninge> sorry to hear :(
<henninge> jtv: Hi!
<henninge> jtv: can you help me with triggering gettext validation errors?
<henninge> danilos: Hi! ;)
<henninge> danilos: maybe you can help me?
<danilos> henninge, hi, how about 'msgid "Blah %d"\nmsgstr "Foo %s"'? :)
<henninge> danilos: yes, I got that. I am actually looking for that than format specifiers mismatch.
<henninge> danilos: do you know any other way?
<henninge> danilos: I am trying to see if bug 385086 can or needs to be fixed.
<_mup_> Bug #385086: Non-ascii in error messages causes OOPS in +translate <lp-translations> <oops> <Launchpad itself:In Progress> < https://launchpad.net/bugs/385086 >
<henninge> danilos: the problem is that the OOPS is gone and I am missing clues.
<danilos> henninge, imho, if you can't reproduce it, close it, and if it re-appears, we should re-open it with a more recent OOPS if we can find it
<henninge> danilos: yes, that's what I was thinking.
<danilos> henninge, fwiw, looking at gettext source code, it seems to try very hard to not include any non-ASCII in error messages anymore, so closing it seems a good idea
<henninge> danilos: thanks
<danilos> henninge, gettext-tools/src/format-invalid.h for reference :)
<henninge> danilos: close as "Invalid"?
<danilos> henninge, yeah
<gary_poster> staging is dead.  doesn't appear to be my fault. :-P
<gary_poster> no stub around. :-/
<gary_poster> flacoste, fyi ^^
<gary_poster> lifeless, I assume/hope you are weekending, but just in case, also ^^
<gary_poster> will ask losas on ops if they have any ideas
<dpm> hi all, could someone tell me where I could check when the staging database was last synced with production? I know there is a url where I can see that, but I cannot find it right now
<benji> jcsackett: will you be able to QA bug 769173 today?  There are a few things behind it that I'd like to get deployed.
<_mup_> Bug #769173: "Hidden" comments should be visible to registry experts <qa-needstesting> <Launchpad itself:Fix Committed by jcsackett> < https://launchpad.net/bugs/769173 >
<wgrant> dpm: https://staging.launchpad.net/successful-updates.txt -- last was restored on Sunday from a Friday or Saturday backup.
<dpm> awesome, thanks a lot wgrant
<wgrant> dpm: (of course it's down at the moment, but with luck will be back in a couple of hours)
<dpm> wgrant, that's fine, the info you gave me was exactly what I needed
<jcsackett> benji: i see that i got a ping from you about qa, but my alert only showed part of the message. if it was about bug 769173, it's done.
<_mup_> Bug #769173: "Hidden" comments should be visible to registry experts <qa-ok> <Launchpad itself:Fix Committed by jcsackett> < https://launchpad.net/bugs/769173 >
<benji> jcsackett: great, thanks!
<flacoste> dpm, wgrant: i usually look at https://lpstats.canonical.com/graphs/StagingRestoreDurations/
<flacoste> a green bar shows a successful production restore/upgrade
<flacoste> and red bars are successful partial upgrade
<danilos> adeuring, hi, would you have time to look at https://code.launchpad.net/~danilo/launchpad/subscribe-unsubscribe/+merge/59511?
<adeuring> danilos: sure
<dpm> flacoste, ah, great, thanks!
<danilos> adeuring, it's a very short JS branch, and unfortunately, lacks tests because I am also working some more on the refactoring of that code
<adeuring> ok
<wgrant> flacoste: Ah, I was using successful-updates.txt before I had access to lpstats. Didn't know about that one.
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #682: FIXED in 5 hr 27 min: https://lpci.wedontsleep.org/job/devel/682/
<adeuring> danilos: nice work. Just one minor remark: for me, the variable name "person_links" sounds more like a sequence, but it contains the number of links. What about renaming it to num_person_links?
<adeuring> but anyway, r0me
<adeuring> r=me
<deryck> back shortly.... swapping locations so grandparents can watch kids.
<danilos> adeuring, heh, I know, but I'll have to wrap the line then :)
<danilos> adeuring, thanks, I'll fix it
<adeuring> danilos: I see your point ;) leave it if you think it looks nicer ;)
<sinzui> bac: adeuring: can either of you review my branch https://code.launchpad.net/~sinzui/launchpad/question-email-2/+merge/59433
<adeuring> sinzui: I'll look
* bac changed the topic of #launchpad-dev to:  https:/â/âdev.launchpad.net/â | On call reviewer: adeuring, bac | https:/â/âcode.launchpad.net/âlaunchpad-project/â+activereviews
<bac> thanks for the reminder sinzui
<adeuring> bac: could you please have a look at this mp: https://code.launchpad.net/~adeuring/launchpad/bug-769293/+merge/59496
<bac> adeuring: sure
<adeuring> thanks!
<rvba> adeuring: could you please take a look at this MP: https://code.launchpad.net/~rvb/launchpad/change-perm-sync/+merge/59341 ?
<adeuring> rvba: sure, let me just finish another one
<rvba> adeuring: great, thank you.
<bac> hi adeuring
<adeuring> hi bac
<bac> i saw this comment "# and one common target, i.e., 2*N+3 targets for N duplicates." and find it a little confusing, given the values used in the following asserts
<bac> i, perhaps naively, thought the expected value would follow that formula
<adeuring> bac: doesn't it? let me check...
<bac> can you embellish your comment to explain why
<adeuring> ouch.. you're right..
<bac> adeuring: otherwise i think the branch looks fantabulous
<adeuring> bac: that should be N+3
<adeuring> just a second...
<adeuring> bac: and thanks :)
<adeuring> bac:         # For N duplicates, we have N distinct targets, we have
<adeuring>         # the targets for old master bug and for the new master bug,
<adeuring>         # and one common target, i.e., N+3 targets for N duplicates.
<adeuring> loks more sane, I think
<bac> great
<adeuring> ...and i changed "for old master bug" to "for the old master bug"...
<adeuring> sinzui: r=me
<sinzui> thank you adeuring
<bac> adeuring: done.  thanks for the nice branch.
<adeuring> bac: thanks for the review!
<LPCIBot> Project windmill build #226: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill/226/
<adeuring> rvba: overall, a very nice branch. But i have  one concern: checkCopy() takes person as an optional parameter. This means that it is quite easy to call the method without it -- and the whole check is simply not done.
<adeuring> I have no clue about other callsites, but couldn't we add person as a required parameter?
<rvba> adeuring: indeed, the method is used in other places ... where AFAIK no such check should be done.
<adeuring> rvba: that's a pity...
<rvba> adeuring: but quite frankly, to give you a proper answer, I should ask someone who knows soyuz more than I do ...
<adeuring> rvba: well, there is another option ;)
<rvba> my goal was to only modify the fonction when used inside the part that we're working on right now. The derived series stuff that it.
<adeuring> rvba: file a bug about it. At least I am really used to rely on permission checks by the zope mechanism. When this doesn't work, I'D _like_ to have some kind of surrogate
<adeuring> but if this isn't available, we can add it later.
<rvba> adeuring: all right, can just make a small comment about that in the MP? I'll ask a soyuz-man just to make sure and then file a bug.
<adeuring> rvba: great, together with an XXX and a bug?
<rvba> adeuring: sure, that would be great.
<adeuring> rvba: cool, so r=me
<rvba> thank you adeuring. Give the rather sensitive nature of the modifications I'll have the branch double checked by a soyuz expert on monday.
<adeuring> rvba: yeah, that would make sense
<rvba> s/Give/Given/
<wgrant> Hmm.
<wgrant> Has the internal ircd died?
<wgrant> I can't connect.
<adeuring> wgrant: I just got a "You are connected" message
<rvba> I can see lots of people login in and out right now ...
<benji> Yeah, I think it died.  I'm back in now.
<wgrant> Me too.
<rvba> wgrant: by the way, if you're still around (and still in the mood) ... but this really can wait ... perhaps you could have a look at https://code.launchpad.net/~rvb/launchpad/change-perm-sync/+merge/59341
<wgrant> Hm. It's nearly 1:30. I'd completely failed to notice the time, since Unity decided to go crazy and replace my clock display with apparently uninitialised memory.
<wgrant> I may do that on Monday, if you don't mind.
<rvba> I really don't, have a good weekend wgrant :)
<wgrant> Thanks!
<benji> wgrant: you just haven't learned how to read the new clock yet, I'll email you the 45 page manual
 * henninge gives up
* adeuring changed the topic of #launchpad-dev to:   https:/â/âdev.launchpad.net/â | On call reviewer: bac | https:/â/âcode.launchpad.net/âlaunchpad-project/â+activereviews
<deryck> bac, hi.  I have something for review, if you're available.
<bac> sure, deryck
<deryck> bac, thanks!  https://code.launchpad.net/~deryck/launchpad/xss-deleting-ssh-key-740160/+merge/59535
<bac> nice deryck.  r=bac
<deryck> bac, awesome, thanks!
<bac> hi deryck, you still around?
<deryck> bac, yup.  what's up?
<sinzui> jcsackett: do you have time to mumble?
<jcsackett> sure. one moment.
<jcsackett> sinzui: apparently mumble was uninstalled when i finished updating.
<jcsackett> i am reinstalling.
<sinzui> really? Who is that I see in the channel?
<sinzui> I think you are still logged in from your faster computer
<jcsackett> yes, that is in the other room. forgot about it. one sec.
<deryck> Later on, everyone.  Have a nice weekend.
<benji> bac: I don't know how much longer you have until EOD, but I have a medium-ish branch up for review if you have time: https://code.edge.launchpad.net/~benji/launchpad/bug-771231/+merge/59555
<bac> benji: i'll look now.
<benji> cool
<bac> benji: looks good.  thanks.
<benji> no, thank you
* bac changed the topic of #launchpad-dev to:   https:/â/âdev.launchpad.net/â | On call reviewer: - | https:/â/âcode.launchpad.net/âlaunchpad-project/â+activereviews
#launchpad-dev 2012-04-23
<wgrant> wallyworld_: How can you deserialise an enum?
<wallyworld_> wgrant: using the lazr.json lib
<wgrant> But you can't know what type it is.
<wgrant> So it's impossible.
<wallyworld_> the type is encoded in the serialisation
<wgrant> Ah
<wgrant> That sounds absolutely disastrous.
<wgrant> ly exploitable
<wallyworld_> it's how most serialisation scheme i've seen work
<wgrant> And most software is fucking terrible.
<wgrant> What's your point? :)
<wallyworld_> my point is i'm using common patterns and not reinventing any wheel
<wgrant> wallyworld_: Ew
<wgrant> Are you trying to turn JSON into pickle? :)
<wallyworld_> wgrant: i'm not going to argue with you. the approach has been approved by curtis
<wgrant> Letting user-specified JSON import arbitrary things, double-encoding stuff, embedding code paths in JSON...
<bigjools> wallyworld_: I haven't seen any suggestions from wgrant either
<wgrant> I don't know if there is a good solution.
<wgrant> JSON is too untyped to do this cleanly.
<wgrant> We could potentially do something with the enum name. But embedding the class path is extremely onerous and completely inappropriate.
<wallyworld_> we need to be able to handle arbitrary complex objects - it's not just about enums
<wgrant> JSON isn't for deserializing directly into complex objects.
<wallyworld_> it provides hooks to do so
<wallyworld_> and we use a pattern where we shove arbitrary stuff down rabbit
<wallyworld_> so we need to be able to handle that
<wgrant> That pattern was simply convenient.
<wgrant> We have not studied its validity or correctness.
<wgrant> My inclination would be to fix that.
<wgrant> Rather than turn JSON into pickle.
<wgrant> With the various security and compatibility concerns that that entails.
 * lifeless notices the conversation
<lifeless> someone want to give me a quick overview?
<wgrant> '{"BaseItem,lazr.enum._enum.Item": "{\\"type\\": \\"Foo\\", \\"name\\": \\"BAR\\"}"}'
<wgrant> lifeless: ^^
<lifeless> that looks a little like JSON
<lifeless> but whats the story around this?
<cjwatson> I would love a review of https://code.launchpad.net/~cjwatson/launchpad/init-packageset-fixes/+merge/103036
<wgrant> lifeless: lazr.json is designed to serialize and deserialize arbitrary types from JSON.
<cjwatson> (and QA suggestions as per my MP comments)
<wgrant> lifeless: The initial case is lazr.enum.
<wgrant> lifeless: It does this in the format shown above.
<lifeless> wgrant: why are we doing this? wallyworld_ ^ ?
<wallyworld_> sorry, what's the question?
<lifeless> wallyworld_: I'd like to understand why we're doing arbitrary type encapsulation in JSON; its rather antithetical to JSON and thus has me confused.
<wallyworld_> lifeless: give me a sec otp
<wgrant> cjwatson: You could just say Packageset.distroseries_id.is_in(...)
<cjwatson> Oh, that works?  OK, will try
<cjwatson> Ah, yes, I see that pattern elsewhere
<cjwatson> nukes a circular import too
<wgrant> Yay
<cjwatson> wgrant: Updated
<wgrant> cjwatson: Approved, thanks.
<cjwatson> Wonderful.  'ec2 land --incremental' is right for this, isn't it?
<cjwatson> I haven't actually checked the state of the DB here
<wgrant> Was going to tell you to use incremental, yep
<wgrant> Since we should also add DB constraints
<wgrant> Later
<cjwatson> Right
<cjwatson> OK, off to EC2 then
<wgrant> Great.
<cjwatson> With any luck we can squeeze in a deployment pre-Q-opening ...
<wgrant> It's nodowntime now, so we'll hopefully have two or three before release.
<wallyworld_> lifeless: with the json thing - we are currently serialising arbitrary objects and if there's a particular object class that is not supported, we oops. enum is one but there will be others. the approach taken is to allow object encoders/decoders to be plugged in and used to handle the cases json doesn't know about
<cjwatson> wgrant: on init-packageset-fixes, do you know if running initialize-distroseries on dogfood is going to be a saneish thing to do?
<cjwatson> sub-day runtime, that sort of thing :P
<wgrant> cjwatson: It will take forever. We could initialise/create a small distro and do it there
<wgrant> Or we could run on qastaging, actually
<wgrant> Will still take a while, but probably 100x faster or so
<cjwatson> I'll need help for that, but that sounds like a better plan
<cjwatson> IIRC it's ten minutes or so on production
<wgrant> Didn't we use the web UI last time?
<cjwatson> good point, I believe we did
<wgrant> So we shouldn't need webops to run stuff manually, hopefully.
<cjwatson> that's what the NewReleaseCycleProcess notes say, anyway
<wgrant> Unless the runner isn't cronned, which is plausible.
 * wgrant checks.
<cjwatson> Might still need somebody to deal with creating a test series on qastaging for the purpose
<cjwatson> Though I suppose we could just use q-series
<wgrant> I'd just rename q-series to Quarrellous Quokka and use that, yeah.
<cjwatson> Does it need a rename?
<StevenK> wallyworld_: Are you still OTP?
<cjwatson> I mean, in order to test this
<wallyworld_> StevenK: no
<StevenK> wallyworld_: Can haz mumble, then?
<wgrant> cjwatson: Not really, true.
<wallyworld_> sure
<StevenK> wallyworld_: http://pastebin.ubuntu.com/941907/
<lifeless> wallyworld_: why are we serializing arbitrary objects?
<lifeless> wallyworld_: and serialising arbitrary object doesn't imply *deserialising* them.
<lifeless> wallyworld_: why is deserialising needed? what are we doing with this json ?
<wallyworld_> lifeless: you tell me and we'll both know. poor design? i think we are listening to object modified events and shoving the result down rabbit
<wallyworld_> lifeless: if we don't handle deserialisation, tests will fail. and i do think we deserialise at the other end of the mq?
<lifeless> wallyworld_: We don't have Launchpad at the other end of the mq
<lifeless> wallyworld_: which tests will fail if we don't deserialise?
<wgrant> (there's an ObjectModifiedEvent hook on the Storm baseclass which shoves diffs into RabbitMQ)
<wgrant> (this should probably be stopped0
<lifeless> wgrant: shoves to rabbit for what? txlongpoll ?
<wallyworld_> lifeless: i would have to check and get back to you, can't recall exactly
<wgrant> lifeless: Yes.
<wgrant> lifeless: Currently only one thing is listened to: MP diff changes.
<lifeless> wallyworld_: so, I'm going to raise a red flag here, I think this is heading into a world of pain and information disclosure.
<wgrant> Specifically the job status, I think
<wgrant> YAY
<lifeless> wallyworld_: can you please schedule a call with curtis, you and I (more optional, but we three must be there) to review this.
<wallyworld_> lifeless: another potential use case - sending arbitrary data to jobs via rabbit
<wallyworld_> lifeless: so, you don't agree we should support serialising arbitrary objects? json or not
<lifeless> I don't agree.
<wallyworld_> lifeless: so the only alternative is to make each bespoke use case do it's own flattening
<lifeless> or rather, I don't think adding an additional typed-encoding in json makes any sense; mapping from rich objects to basic types is something we already do in the API via the interfaces.
<wallyworld_> lifeless: how about we have a call tomorrow after our standup
<wallyworld_> say around 9am AEST
<wallyworld_> or you could join our standup
<wallyworld_> starts at 8am AEST
<lifeless> wallyworld_: there is an existing protocol for flattening objects; if we do want to do this, that would be the right way forward; AFAICT though, we don't /want/ this, we've stumbled on it via adding something to an existing code path
<lifeless> wallyworld_: I will be here at both times, ping me :)
<wallyworld_> lifeless: ok, will do. talk tomorrow
<wgrant> (it also pretty much destroys our compliance to any of JSON's goals, and is likely to be insecure)
<StevenK> wallyworld_: http://pastebin.ubuntu.com/941914/
<wallyworld_> wgrant: well we need to decide if there's a use case for arbitrary object deserialisation. most enterprise systems support it and provide mechanisms to do so. it's a very common pattern.
<wallyworld_> not sure why there's so much hostility given it's not exactly a new idea
<wgrant> I am not completely averse to the entire idea.
<wgrant> But this is without a doubt not the right way to do it.
<lifeless> wallyworld_: well, AIUI you're solving an OOPS we're seeing
<lifeless> wallyworld_: I think the OOPS has uncovered us sending confidential object change metadata to arbitrary unauthenticated clients on the internet
<wallyworld_> lifeless: yes, and in a way that handles arbitrary objects, which will happen, not just the oops we are seeing
<lifeless> wallyworld_: not in practice, but in potentia, we have a dangerous API
<lifeless> writing a system to let that dangerous API be more powerful isn't a great use of time IMNSHO
<lifeless> trimming that system back to be less dangerous, which will also avoid the OOPS, will be.
<lifeless> the question of 'can we serialise everything' is tangential.
<lifeless> serialise-everything opens inappropriate-disclosure holes; deserialise-everything opens bad state injection holes
<wallyworld_> lifeless: sure, there's a balance to be achieved. we can discuss tomorrow on the call. too hard via irc
<lifeless> yup
<lifeless> totally agree, thats why I suggested a call :)
<wallyworld_> lifeless: btw, the arbitrary objects thing - the stuff you mention just above is solved in the decoder/encoder classes, so it's not as bad as you probably are thinking
<lifeless> wallyworld_: I don't see how its solved
<wallyworld_> lifeless: an arbitrary object is only serialised if there's been a decoder registered for it's type (and it will oops if there's none). i guess it is possible to register a  fake decoder and inject bad data, but that's an issue not unique to this case and we would need to deal with that i guess
<lifeless> consider a privileged field
<lifeless> how do you deserialise it *and* trust the result.
<lifeless> secondly, object graphs
<wallyworld_> lifeless: the implementation handles object graphs just fine
<wallyworld_> that's the point of it
<lifeless> I'm sure it does
<lifeless> but thats where the vulnerabilities in these systems come in
<StevenK> lifeless: I've been on Mumble with wallyworld_ for 30 minutes now, and you keep interrupting every 3 minutes!
<StevenK> I would have been done by now!
<lifeless> StevenK: hey, I'm just replying. Tell wallyworld_ to stop :P
<StevenK> I have been!
<wallyworld_> i will now. till tomorrow :-)
<StevenK> Now I'm tell you BOTH to stop.
<lifeless> in soviet russia, words speak you
<wallyworld_> StevenK: how goes the yui test writing?
<StevenK> wallyworld_: Trying to muddle along
<wallyworld_> sounds like fun
<StevenK> wallyworld_: Firebug complains YUI isn't defined.
<StevenK> And make lint is complaining about undefined entity in the HTML.
<wallyworld_> hmmm. what are you trying to load?
<wallyworld_> sounds like a syntax error in the html
<wallyworld_> you may have stuff from the template left over
<StevenK> Let me just get my current thought out of my head and into Vim and I'll repaste the diff
<wallyworld_> ok
<StevenK> wallyworld_: http://pastebin.ubuntu.com/942011/
<wallyworld_> one sec
<wallyworld_> StevenK: there's nothing obviously wrong with the html and test js that i can see. are you trying to load the html into your browser?
<StevenK> Yah
<wallyworld_> one thing is that you may not have included all the necessary dependencies in the html
<wallyworld_> all the js files required need to be specified manually
<wallyworld_> so you are missing choiceedit.js and friends i thibk
<StevenK> It's just strange it's manifesting as YUI being undefined.
<wgrant> Dpesm
<wgrant> Er
<wgrant> Doesn't one normally use LPJS from firebug?
<StevenK> This is a YUI test.
<wallyworld_> StevenK: if yui can't load all the dependencies it won't initialise
<StevenK> And therefore isn't defined. Right.
<wallyworld_> so if you say to load information_type.js and that required xxx.js and xxx.js isn't available, boom
<wallyworld_> so it's a pita but you need to add in the things you think you need by hand
<wallyworld_> to the html test harness
<StevenK> That's choiceedit, privacy and bugtask_index. Now to work out what else I need.
<StevenK> wallyworld_: I guess base == yui.js itself, what about oop, node, event and io-base?
<lifeless> wgrant: https://bugs.launchpad.net/bugs/+bugs?field.searchtext=foo+bar
<lifeless> wgrant: and https://twitter.com/#!/pipesec/status/194202104582258688
<lifeless> wgrant: look at only one
<wgrant> lifeless: What about it?
<wgrant> Works fine for me.
<wgrant> Returns a plausible number of results
<wgrant> etc
<wgrant> The FTI index will take a while to page in on all 3 nodes.
<lifeless> wgrant: times out for me, 3 out of three times
<wgrant> Maybe you got a bad slave.
<lifeless> https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-f28e0087ffa2474538d8d21683092d1d
<wgrant> Just got one in 8.6s rather than the normal 2.1s
<wgrant> So one slave may just be almost there.
<lifeless> could be
<wgrant> Reliably <2s now
<wgrant> (note that this is still GiST... will improve with GIN in a couple of weeks :))
<wallyworld_> StevenK: you need node etc in your test js, not the html as yui as a whole is loaded but you still need to specify the dependencies in the js as like with all yui modules
<wgrant> lifeless: Currently the tiebreak sort key is always bugtask ASC or bug ASC. I've currently got the indices set up to handle the default direction for each of the common sort orders, but they doesn't work for the reverse because the tiebreaker is the wrong direction.
<StevenK> wallyworld_: http://pastebin.ubuntu.com/942063/
<wgrant> lifeless: Do you think the tiebreak order is significant enough that I should double the number of indices, or can I reverse the tiebreak order in that case?
<wgrant> (currently reversing the order doesn't reverse the tiebreaker)
<wallyworld_> StevenK: so you've included overlay.js in your html test harness?
<StevenK> wallyworld_: http://pastebin.ubuntu.com/942064/ is the current test
<wallyworld_> StevenK: /build/js/yui/yui/overlay/overlay.js is bogus
<wallyworld_> it's more like lp/app/javascript/overlay/overlay.js
<wallyworld_> /build/lp/app/...even
<StevenK> That's there too
<StevenK> A few lines down
<wallyworld_> ah so it is. you don't need any yui stuff in there though
<wallyworld_> looks like you're missing anim
<wallyworld_> probs also extras
<wallyworld_> you sort of need to look at the requires bits of the things you are importing and import those dependencies also
<StevenK> wallyworld_: Right. I was unsure if I needed to care about the YUI bits included too, like oop and event.
<wallyworld_> not in the html, just the requires of the test js
<wallyworld_> if required
<StevenK> wallyworld_: But overlay only has: "requires": ["oop", "overlay", "event", "widget", "widget-stack", "widget-position"] ?
<lifeless> wgrant: so you have foo bar quux id all asc, and reversed foo desc bar desc quux desc id desc ?
<wallyworld_> StevenK: from memory, choiceedit requires those other things i mentioned
<wallyworld_> yes, lp.anim
<wgrant> lifeless: Currently we default to (importance ASC, bugtask DESC). When you reverse that listing, you get (importance DESC, bugtask DESC) rather than (importance DESC, bugtask ASC)
<wallyworld_> StevenK: and lp.anim requires lp.extras
<StevenK> wallyworld_: But choiceedit isn't complaining, and it's imported above overlay?
<lifeless> wgrant: ok, thats a bug
<wgrant> lifeless: Arguably.
<wallyworld_> StevenK: such are the mysteries of yui. i'm not sure why that would be the behaviour
<wgrant> So I can change it?
<lifeless> yes
<wgrant> Thanks.
<lifeless> reversing the sort should reverse all the rows, which it won't be atm
<StevenK> wallyworld_: http://pastebin.ubuntu.com/942069/ is a new diff
<wallyworld_> StevenK: looks reasonable, but only way to know is to try
<StevenK> wallyworld_: It still gives the YUI not defined errors
<wallyworld_> hmmm. still missing dependencies i would guess. what's the branch?
<StevenK> wallyworld_: Let me commit this test and push
<wallyworld_> ok
<StevenK> wallyworld_: lp:~stevenk/launchpad/bugs-information_type-ui-portlet
<wallyworld_> ok, let me save my current branch
<wallyworld_> StevenK: you appear to have one to many ../ in your js paths
<wallyworld_> s/to/too
<StevenK> wallyworld_: All of them?
<wallyworld_> all the ones referencing build/***
<wallyworld_> my IDE told me that immediately :-P
<StevenK> Bah
<StevenK> I copied from the lp/app/testing/testrunner.js line
<wallyworld_> StevenK: our javascript dirs are entirely 100% consistent
<wallyworld_> StevenK: also, you will need
<wallyworld_>       <script type="text/javascript"
<wallyworld_>           src="../../../../../../build/js/lp/app/effects/effects.js"></script>
<wallyworld_>       <script type="text/javascript"
<wallyworld_>           src="../../../../../../build/js/lp/app/lazr/lazr.js"></script>
<StevenK> effects I added
<wallyworld_> definitely lazr.js then also
<wallyworld_> and i think that solves the initial bootstrapping of the test
<wallyworld_> so now you just have to make everything pass :-)
<StevenK> wallyworld_: But I need to drop a '../' from every src line involving build/js ?
<wallyworld_> yes
<wallyworld_> StevenK: but not the lp/app ones
<wallyworld_> the template is correct for those includes
<StevenK> Or the YUI one?
<wallyworld_> correct
<StevenK> Looks like only bugtask_index.js then
<StevenK> Bah, I still get YUI is not defined
<wallyworld_> ah, sorry i meant the top 2 lp/apps lines
<wallyworld_> id didn't notice the ones near the bottom were lp/app
<wallyworld_> should 5 5 changes
<wallyworld_> be
<wallyworld_> 1 bugs one and 4 lp app ones
<StevenK> You mean the one under Dependencies?
<wallyworld_> yes
<wallyworld_> the top 2 lp/app lines under dependencies are ok
<wallyworld_> the next 4 aren't
<wallyworld_> the next 2 are ok
<wallyworld_> and then bug task isn't
<StevenK> wallyworld_: So extras, anim, effects, lazr.js, overlay and expander?
<wallyworld_> StevenK: https://pastebin.canonical.com/64716/
<wallyworld_> i should have pasted the diff in the first place
<StevenK> Haha, yes.
<wallyworld_> sorry
<wallyworld_> got myself confused
<wallyworld_> the only correct lines were overaly and expander
<StevenK> Oooh, it loads the test now.
<StevenK> Y.lp.testing.mockio is undefined.
<wallyworld_> should be 5 ../
<wallyworld_> you may also find you need to include the subscription_list from lp/app
<StevenK> wallyworld_: It's actually calling the JS test now.
<wallyworld_> yay :-)
<StevenK> Testing completed at Mon Apr 23 2012 16:13:21 GMT+1000 (EST).
<StevenK> Passed:1 Failed:0 Total:1 (0 ignored)
<wallyworld_> progress!
<StevenK> Mind you, the test doesn't assert anything yet. :-P
<StevenK> But yes, progress!
<wgrant> lifeless: Barring massive issues I was aiming to turn it on for ~launchpad-beta-testers tomorrow morning.
<wgrant> But maybe earlier is good.
<stub> lifeless, wgrant: But I don't think perfect behaviour justifies doubling the number of indexes. Do real searches actually use the tiebreaker, or is it only to keep things perfectly stable for the test suite?
<wgrant> stub: Real searches use them.
<wgrant> stub: eg. Launchpad's default bug listings
<wgrant> stub: We only use three statuses, so the tiebreaker is used for the majority of the sorting.
<stub> ok
<wgrant> s/statuses/importances/
<lifeless> wgrant: tomorrow is a bad day to make changes
<lifeless> wgrant: as wed is a pub hol
<wgrant> Mmmm, I guess.
<wgrant> Will run some more tests and do it this evening.
<StevenK> wallyworld_: I can't see how to grab hold of the choicesource after I simulate the click on the edit link.
<wallyworld_> StevenK: bit more info?
<StevenK> wallyworld_: I've got an edit link in the test HTML, which has href='#'. That is passed into setup_information_type_choice() so it should bring up the choicesource overlay when it's clicked.
<StevenK> wallyworld_: I want to click one of the options on the choicesource
<wallyworld_> right, i see. look at test_shareetable.js where i test the choicesource popups for changing info type for a sharee
<wallyworld_> StevenK: test_permission_update_click
<wallyworld_> lines 277-282
<StevenK> wallyworld_: Right. I'm doing the same thing
<wallyworld_> figured as much :-)
<StevenK> var choicesource = Y.one('.yui3-ichoicelist-content');
<StevenK> And choicesource is null
<wallyworld_> the click to open the choice source popup must be wrong
<StevenK> Hmm, I think I'm missing a div
<wallyworld_> have you looked behind the couch?
<StevenK> Hah
<wallyworld_> one of my favourite t-shirts is "I found Jesus. He was hiding behind the sofa"
<StevenK> Right, that gets the choicesource working
<cento> hi
<cento> is possible to browse launchpad project by program language?
<lifeless> you could use the API, but we don't have a built in thing for that AFAIK
<cento> lifeless, and through web page?
<cento> for example, to list all Python project
<wallyworld_> StevenK: the test passes?
<StevenK> wallyworld_: I have it working, I just need to figure out how to get the content of the <strong> so I can assert against it.
<wallyworld_> node.get('text')
<wgrant> StevenK: btw, while you're there, care to fix the capitalisation in BugSecrecyEditView.label?
<StevenK> Which capitalisation?
<wgrant>         if bool(getFeatureFlag(
<wgrant>             'disclosure.show_information_type_in_ui.enabled')):
<wgrant>             label += 'Information type'
<wgrant>         else:
<wgrant>             label += 'visibility and security'
<wgrant> The first has a capital, the second does not.
<wgrant> Neither should.
<StevenK> Right
<wgrant> lifeless: Meh, shall I play dangerously and switch it on now?
<wgrant> I can't see any issues, apart from the rare index usage due to tiebreaker, but it's still strictly better than the old code.
<lifeless> wgrant: for all beta +1
<wgrant> Thanks.
<lifeless> wgrant: lets hit it fully on thurs.
<wgrant> Sounds good.
<wgrant> We may see particular issues with searching by structsub or tag, but it should be slightly better than the old one.
<StevenK> wallyworld_: Y.lp.bugs.bugtask_index.portlets is undef :-(
<StevenK> And adding "lp.bugs.bugtask_index.portlets" to requires didn't help
<wallyworld_> StevenK: did you include subscription_list?
<StevenK> wallyworld_: Yup
<wallyworld_> hmmm. must be something else. i'll see if i can see what it is
<wallyworld_> StevenK: lp.app.errors, lp.app.confirmationoverlay
<StevenK> For portlets?
<wallyworld_> lp.bugs.javascript.subscription
<wallyworld_> i think portlets renders subscriptions?
<wallyworld_> haven't checked
<wallyworld_> but if so, then subscriptions uses those things
<StevenK> So I should include lp/bugs/subscription.js too?
<wallyworld_> yes
<wallyworld_> subscribers_list is the other one I was thinking off
<wallyworld_> of
<StevenK> wallyworld_: Firebug showing NOT loaded is stuff I'm including and don't have to?
<wallyworld_> can't remember. example?
<wgrant> There's one thing that always appears there
<wgrant> widget-position-ex or something
<StevenK> yui: NOT loaded: lp
<StevenK> yui.js (line 4009)
<wallyworld_> that's spurious, ignore it
<wallyworld_> the lp one shouldn't appear
<wallyworld_> that means there is a missing dependency
<wallyworld_> StevenK: i'm off to soccer, back later. i'll check in to see how you're going
<StevenK> wallyworld_: I should stop working anyway
<wallyworld_> you left off " and start drinking"
<StevenK> Hah
<wallyworld_> i might have a few after soccer
<StevenK> I've been dealing with JS all day, that's *implied*.
<wallyworld_> yeah, i'm stuck in js as well atm
<StevenK> wallyworld_: Besides s/drinking/drinking heavily/
<wallyworld_> almost got it sorted
<wgrant> Didn't you hear, we're rewriting LP in node.js.
<wallyworld_> luckily i know you're joking
<adeuring> good morning
<wgrant> stub, lifeless: https://code.launchpad.net/~wgrant/launchpad/bugtaskflat-more-indices/+merge/103063
<wgrant> Heh
<wgrant> The backup completed 5s before I proposed the live index changes.
<wgrant> Handy.
<lifeless> stub: you've got that ?
<stub> yer
<stub> Will ordering by date_last_updated without a tie-breaker break tests? I guess fixing the tests would be better.
<stub> We could fake date_created by using id instead
<stub> I wonder if this would be better accomplished by dropping the IS NOT NULL on the existing partial indexes? Not for this live update, but that might work.
<stub> No obvious errors. No point me questioning the utility of the indexes - I understand they are driven by the searches we allow
<wgrant> stub: date_last_updated and date_created have always been regarded as unambiguous. We've never used a tiebreaker with them, despite the fact that they could technically be ambiguous.
<stub> We have, but only to keep tests happy and that was probably a mistake.
<wgrant> The existing bug search code did not use a tiebreaker.
<stub> better data is a better option
<wgrant> So the new one does not.
<stub> Yer, that is all fine.
<wgrant> stub: Dropping the IS NOT NULL doesn't work reliably.
<wgrant> These indices are just for sorting.
<wgrant> All the existing ones work marvellously :)
<stub> k. Would be nice if they did. My guess would be they would often be used, but occasionally odd stats would cause them to be skipped. But empirical results override my assumptions :)
<wgrant> stub: Dropping the NOT NULL doesn't help contextless sorts, as the index will still be primarily ordered by context.
<stub> oh, right
<wgrant> Unfortunately COUNT(*) over Ubuntu's open bugs still takes slightly over a second.
<wgrant> But better than the 4s it used to take.
<czajkowski> jtv: ping
<stub> wgrant: At least it isn't EOVERFLOW yet
<stub> wgrant: Those indexes are live on staging & qastaging
<jtv> hi czajkowski
<wgrant> stub: Let me do a few checks.
<wgrant> Marvellous, https://bugs.qastaging.launchpad.net/bugs/+bugs in 1.5s
<stub> Anything stopping us taking this live?
<wgrant> Just checking Ubuntu.
<wgrant> But it looks good.
<czajkowski> jtv: sorry I'm ssure it's late over there but a bit unsure about https://bugs.launchpad.net/launchpad/+bug/987199 dpm reported on translations
<wgrant> Yeah, all more than fine. Enlivenate.
<_mup_> Bug #987199: Automatic translation exports fail to commit latest translations  <Launchpad itself:Confirmed> < https://launchpad.net/bugs/987199 >
<jtv> czajkowski: I don't think it's that changes weren't detected.  More likely is that the exports are breaking.  You'll want to watch the launchpad-error-reports mailing list for error output from the translations-to-branch export script.  Often it's a lingering branch lock.  Another possibility is that imports and exports cross, in which case the export script backs off, for fear of overwriting something.
<wgrant> There's only one perma-locked branch.
<wgrant> The export script hasn't complained about ubuntu-doc ever AFAICR
<wgrant> nautilus-pastebin is the only regular victim.
<czajkowski> hmm need to check folders for launcpad-error-reports ml
<wgrant> czajkowski: You're not subscribed by default.
<czajkowski> nope I'm not
<czajkowski> just checked all my folders
<jtv> czajkowski, wgrant: I suppose it's not unthinkable that the translations branch was broken _for a while in the past_, and changes accumulated during that time.
<wgrant> I'm embarrassingly ignorant of how this stuff works.
<jtv> If you know anything at all, that's far from embarrassing.  As I recall, the exporterâat least at the time I wrot eitâlacked a way to see whether a particular change had already been exported.
<jtv> Instead, it just exported changes from some fixed time window â even if that meant occasionally exporting something twice.
<jtv> The branch is clearly getting translation updates committed to it.
<jtv> Another possibility is might be that the translations are activated in such a way that their timestamps aren't updated.
<jtv> czajkowski, wgrant: errrâ¦ there _was_ a discrepancy, right?  Because I see 337 untranslated strings on both the Ubuntu side and the upstream side now.
<wgrant> I trusted the bug, didn't check myself.
<jtv> Oh, need to check the branch contents apparently, not the translation stats.
<jtv> Should've known; David's very thorough.
<wgrant> That's why I didn't bother checking :)
<jtv> Good call, as it turns out.  Yes, the branch has a bunch of untranslated strings that are translated in Launchpad.
<jtv> So it sounds as if sharing is to blame.
<jtv> Not certain, of course.
<czajkowski> nods
<czajkowski> thanks jtv
<jtv> So the suspicion is that a string can become current, without updating timestamps in such a way that the translations-to-branch exporter notices that anything new needs exporting.
<dpm> jtv, yes, as mentioned on the bug, it seems the stats in LP are correct, it's just the branch that does not seem to get them committed.
<jtv> Hi dpm
<dpm> jtv, hey, thanks for looking at this :)
<jtv> I *think* it's the timestamp on the POFile that the translations-to-branch exporter looks at, and which most likely does not get updated.
<dpm> jtv, could it be that translations have been done offline, uploaded, and LP hasn't noticed there have been some changes?
<jtv> Shouldn't make a difference in itself.
<dpm> ah, good to know
<jtv> Any idea whether those changes were made in the Precise series itself, or made in e.g. Oneiric and shared with Precise?
<dpm> in Precise, afaik, which is what the French team were translating
<jtv> Because I don't _think_ (mostly guesswork for me at this point) that we update the POFile timestamp when we share translations.  It's actually pretty costly to figure out which POFiles might be affected.
<jtv> Hmmâ¦ if the work was done in Precise itself, you'd expect that timestamp to update.
<dpm> they might have been doing the work on the upstream project too
<jtv> Ah
<jtv> That'd do it.
<dpm> what do you mean exactly?
<jtv> Well, sharing an individual string's translation (a TranslationMessage) between upstream and the Ubuntu package is a matter of setting an extra âthis is the current translation for Ubuntuâ flag on the TM.
<dpm> what's a TM and who or what sets that flag?
<dpm> ah, TranslationMessage
<dpm> but the sharing seems to happen, just the commit doesn't seem to
<jtv> TranslationMessage.  LP sets that flag to make it the current translation in a given context; each TM has one such flag for current-upstream and current-in-Ubuntu.  Setting that flag may affect Ubuntu-side POFiles for the same language in whose POTemplate the TM's POTMsgSet participates, _and_ which don't have another, newer translation to that language _and_ which don't have a diverged translation for that language.  With me so far?
<jtv> Never mind, I'm pulling your leg.  :)
<jtv> It's a bit complicated, and that's exactly my point.
<jtv> When LP sets that flag, it's doing something relatively simple in itself.  But once you start looking at all the POFiles that it might possibly affect, it gets a bit complicated.
<jtv> And so I suspect we don't really seek out those affected POFiles to update their timestamps.
<jtv> And if we don't update their timestamps, chances are the translations-to-branch exporter doesn't notice that anything's changed in its version of the POFile.
<dpm> oh, I thought the point of having translation sharing between upstreams and Ubuntu was to be able to commit Ubuntu translations to the upstream projects. Do I understand it correctly that that actually does not happen by design? Or is it rather a bug?
<jtv> It goes both ways, with exceptions.
<jtv> (The exceptions are: Ubuntu translations only go automatically into upstream if the reviewer happens to have review privileges upstream as well; diverged messages, which override a message's translation in just one upstream release series or in just one Ubuntu release, stay diverged; and I think changes on either side don't override those on the other side unless the translation on the other side is either missing or identical to the old translatio
<jtv> All that seems to work just fine, but it all clouds our view of which POFiles just received a new translation.  It's a lot of work to get the exact set.
<jtv> And so, I don't think we actually do all that work.
<jtv> A fresh translation upstream will probably automatically go into all Ubuntu releases of the package that have the same English string, but probably without updating POFile.date_changed for all those places it ends up.
<dpm> Ok, so the summary is that I should stop recommending maintainers to use sharing + automatic exports and tell them to request tarball exports instead. Does that sound reasonable?
<jtv> It'd be a sad step back.  :(
<dpm> absolutely, and I'll be the first to regret it :/, but now I understand that sharing + exports does not work as expected, which I didn't know before.
<jtv> Another equally sad way to cope would be to make some arbitrary change to the Ubuntu-side POFile.
<jtv> I didn't know it either.  I'm not even entirely sure.  But I did just verify that POFile.date_changed is what drives the exports.
<dpm> jtv, thanks a lot for the research and the help. I think it might make sense to add the relevant parts of the conversation to the bug. I'd volunteer for it, but you're the one who knows how everything fits together, would you mind adding your findings so far?
<jtv> OK
<jtv> dpm: I've updated the bug.  I hope it's clear enough; if not, please say.
<dpm> thanks a lot jtv, it looks clear enough to me for the parts I can understand :)
<jtv> OK.  Remember, if you really need one of these exports to happen, just make an arbitrary change and then re-active the original translation.
<jtv> Not very nice, but it should do the trick for that POFile.
<dpm> jtv, gotcha, that's good to know, as a workaround, thanks.
<czajkowski> jtv: marked triage and low, should it be higher?
<jtv> czajkowski: I would call David's problem a high priority; the one about the wrong credits a low priority.
<czajkowski> nods
<czajkowski> jtv: fixered cheers
<jtv> ok
<czajkowski> rick_h: are you alive over there yet? if not ping when you come on thanks.
<rick_h> czajkowski: yep, I'm alive
<czajkowski> rick_h: did you do stuff with OAuth mails last week ?https://answers.launchpad.net/launchpad/+question/194281
<rick_h> czajkowski: yes I did
<rick_h> czajkowski: I'll reply.
<czajkowski> lovely jubbly
<czajkowski> cheers and happy monday to you :)
<rick_h> you too!
<rick_h> czajkowski: answered
<rick_h> and doh, newline fail on my twitter reply :/
<czajkowski> rick_h: :) I did say please after all
* benji changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: benji | Firefighting: - | Critical bugs: 3.47*10^2
<rick_h> czajkowski: yea, supposed to be a nice bash response: command not found
<czajkowski> sounds like monday
<czajkowski> rick_h: good weekend ?
<rick_h> busy, but yea can't complain
<rick_h> now monday...she's really starting to get irritating
<deryck> Morning, all.
<jml> hi
<jml> gary_poster: fwiw, I haven't had a chance to look at bug 986421 and bug 986434, and probably won't get a chance soon.
<_mup_> Bug #986421: In parallel tests, global tags eventually become misapplied <Testrepository:New> < https://launchpad.net/bugs/986421 >
<_mup_> Bug #986434: global tags unnecessarily inflate concurrent test results <testtools:New> < https://launchpad.net/bugs/986434 >
<gary_poster> jml, hey.  np.  the inflation one is no biggie.  The global misapplication is actually not a showstopper for us either right now but we may want to investigate it and propose something if we make progress elsewhere.  In terms of official release of the packages, I think the global tag one should be addressed before a release.  The file inflation one might be invalid as far as you are concerned.
<jml> well, I think a strong case can be made for it being a genuine bug.
<gary_poster> cool
<abentley> wgrant: is bug 983420 really fix released?
<_mup_> Bug #983420: Celery branch scan jobs intermittantly fail to show revisions for new branches <branch-scanner> <fast-slow-lane-jobs> <qa-needstesting> <Launchpad itself:Fix Committed by abentley> < https://launchpad.net/bugs/983420 >
<abentley> wgrant: Oh, nm.
<wgrant> abentley: Yeah, sorry, misclicked and reverted.
<abentley> wgrant: np.
<czajkowski> sinzui: did you just spring clean license review?
<sinzui> czajkowski, I send a license-conflict email to one that was in review
<czajkowski> sinzui: ah that was the one I'd left to check with you
<czajkowski> just wondered refreshed page and was gone
<czajkowski> good day :)
<rick_h> deryck: ping
<deryck> rick_h, yo
<rick_h> deryck: so I think the issue is that the distro source pkg isn't 'published' so using the factory.makeSourcePackgePublishingHistory
<rick_h> deryck: do you know if I should make sure it's published for each series then?
<deryck> rick_h, honestly, I don't know.  sorry.
<rick_h> I guess does that click/make sense
<rick_h> ok, will test it out thanks
<deryck> rick_h, yeah, I'd guess you need it in each series first. but don't know for sure.
<StevenK> rick_h: I'm on my way to bed, but an SPPH is per series and has a status. I think the default is PENDING.
<StevenK> The enum is in soyuz or archivepublisher. Probably the former.
<rick_h> StevenK: ok, thaks.
<abentley> benji: Could you please review https://code.launchpad.net/~abentley/launchpad/celery-everywhere-4/+merge/102918 ?
<benji> abentley: sure, how urgently do you need it?
<abentley> benji: not urgent.
<benji> k
<gary_poster> jelmer, if you have a minute, we need a build of jml's subunit filter branch in our PPA, and we are not having success with our recipe.  We thought we copied all the relevant bits from your recipe (https://code.launchpad.net/~testing-cabal/+recipe/subunit-daily) to ours (https://code.launchpad.net/~yellow/+recipe/subunit-daily) but debhelper is not installed for precise or lucid.  Can you take a glance and give us a
<gary_poster> ny hints as to what we're doing wrong?  If you don't have time, np, understood of course.  can try someone else
<jelmer> gary_poster: you're missing python-testtools
<jelmer> gary_poster: you'll need an updated version - you can either also add the testtools recipe, or at the testing-cabal PPA as a dependency for yours
<gary_poster> jelmer, ah ok.  to add the testing-cabal PPA as a dependency, I need to fork your packaging branch, right?
<jelmer> gary_poster: no, that should be a setting in your PPA
<cjwatson> If you're using debhelper, you should explicitly build-depend on it, anyway
<cjwatson> (Or your packaging branch should
<gary_poster> jelmer, I see.  We do have a recent python-testtools in our PPA already though (0.9.14-bzr267~ppa40~precise1 in https://code.launchpad.net/~yellow/+archive/ppa , from the April 19)
<cjwatson> )
<gary_poster> cjwatson, yeah that's what I was wondering
<gary_poster> so we could fork jelmer's packaging and add it explicitly and try that...
<gary_poster> will try
<cjwatson> The only things you're allowed to assume are Essential: yes + build-essential
<cjwatson> i.e. libc6-dev, gcc, g++, make, dpkg-dev
<gary_poster> ack
<gary_poster> thanks
<jelmer> cjwatson, gary_poster: there already is a dependency on debhelper >= 9 in the subunit package
<cjwatson> Dependency or build-dependency?
<jelmer> sorry, build dependency
<jelmer> as far as I can tell this is merely pbuilder-satisfy-depends giving a confusing error
<jelmer> gary_poster: does your testtools ship python3-testtools?
<gary_poster> jelmer, no.  we found that part of the build to be broken
<cjwatson> Ah, yeah
<gary_poster> maybe we disabled it prematurely
<jml> we really need better CA stuff.
<jelmer> gary_poster: I suspect that's the real cause of the error - the subunit python3 package depends on the testtools python3 package, and the source package depends on it too
<jml> CI
<gary_poster> ah :-/
<gary_poster> jelmer, ok so we need to go back to why python-testtools was not building for us
<gary_poster> on oython 3
<jelmer> gary_poster: do you actually care about ubuntu releases all the way back to lucid?
<jelmer> because if you do that might be the cause
<gary_poster> jelmer, it is necessary to have this available in precise and it is convenient to have it available in lucid.  don't care about the rest.
<gary_poster> jelmer, so for python-testtools your suggestion is to change that to only precise, and return to the original packaging branch, and see if that works, yeah?  trying that
<jelmer> gary_poster: that's what I was going to suggest, but looking at the testing-cabal recipe now..
<jelmer> it looks like testtools is broken for python3 at the moment because it uses some python2-specific code in trunk
<jelmer> jml: ^
<jelmer> https://launchpadlibrarian.net/102600248/buildlog_ubuntu-natty-i386.python-testtools_0.9.14%2Bbzr254~ppa37~natty1_FAILEDTOBUILD.txt.gz
<jelmer> gary_poster: you can either add a dependency to the testing-cabal in your PPA, copy the testtools package or your own recipes (perhaps based on the ~testing-cabal ones) that disable the python3 support
<jelmer> or of course contribute a fix for the python3 build issue in trunk :)
<gary_poster> jelmer, :-) I'll plan to contribute fix but the other ideas help us *now* which is always nice.  But we already have a working python-testtools build that copies the cabal packaging branch and recipe and disable the python 3 support; I guess you mean we wouls also then need to disable the python3 support in subunit to go down that road?
<gary_poster> it struck me that we could also be gross as a temporary fix
<gary_poster> well doesn't even have to be gross
<jelmer> gary_poster: ah, yes - if you have a working testtools (that's new enough) then just disabling the python3 support in the subunit package should be sufficient too
<gary_poster> but we coud have our own testtools branch for now with the fix.  OK, I see some roads out of this.  Thanks jelmer!
<sinzui> jcsackett, are you aware of BaseBreadcrumbTestCase
<sinzui> I think we want to extend it if it does not work to test your changes
 * jcsackett looks
<jcsackett> yeah, i remember taking a look at this, and discarding it for some reason.
<jcsackett> i'll double check why i discarded it, and update it.
<sinzui> I think other developers will be confused if they use this test, then spend hours to discover it does not support their needs.
<sinzui> jcsackett, you have been frustrated by cases like this in the past. I think I should be able to add the view to the list of traversed objects to get breadscrumbs
<sinzui> jcsackett, I think getBreadcrumbsForUrl() is the method that needs improvement
<jcsackett> sinzui: yeah, that seems right.
<sinzui> jcsackett, I think getBreadcrumbsForObject() might work, but it is not obvious that the object can be a view.
<sinzui> I think I passed the view in lp.registry.browser.tests.test_breadcrumbs to verify some views were correct.
<abentley> james_w: I'm trying to migrate all Jobs to be able to run under Celery, but it seems like CopyArchiveJob is unused.  Is that correct?
<abentley> benji: thanks for your review.
<benji> abentley: my pleasure
<lifeless> morning
<abentley> benji: Could you please review https://code.launchpad.net/~abentley/launchpad/celery-everywhere-5/+merge/103161 ?
<benji> abentley: sure
 * sinzui has exceeded his writing about non-interesting problems again
 * sinzui considers the last beer as crutch
<lifeless> flacoste: ping
<lifeless> flacoste: my calendar claims now is our catchup
<benji> abentley: done with your latest review
<abentley> benji: Thanks.
<benji> np
<flacoste> lifeless: yeah, i apologize, i had a conflicting appointment with my osteopath, do you mind if we catch-up tomorrow instead?
<lifeless> sure
* benji changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 3.47*10^2
<wallyworld_> StevenK: lib/lp/bugs/javascript/bug_subscription_portlet.js
<StevenK> wallyworld_: Thanks.
<lifeless> wallyworld_: sinzui: let me know when you want me
<wallyworld_> lifeless: will do, soon :-)
<sinzui> lifeless, we are on mumble
<lifeless> wallyworld_: lib/lp/services/longpoll/adapters/storm.py
<lifeless> wallyworld_: end of the file
<sinzui> jcsackett,  r=me
<lifeless> wallyworld_: http://www.ubuntu.com/devices/android
<StevenK> wallyworld_: Branch updated, lp:~stevenk/launchpad/bugs-information_type-ui-portlet
<wallyworld_> cool. will test
<StevenK> wallyworld_: Ta. Just pastebin me the output of bin/test -vvt test_information_type_choice
<wallyworld_> yup
<wallyworld_> StevenK: https://pastebin.canonical.com/64795/
<wallyworld_> StevenK: doing a make resulted in this error though
<wallyworld_> ERROR: LP_BUGS_BUGTASK_INDEX_PORTLETS is not mentioned in the meta file
<wallyworld_> make: *** [combobuild] Error 1
<StevenK> That is probably me.
<StevenK> wallyworld_: Do I need to worry about the NOT loaded messages?
<wallyworld_> don't think so
<StevenK> wallyworld_: I've fixed the issue that caused the LP_BUGS_BUGTASK_INDEX_PORTLETS error.
<wallyworld_> cool
<wallyworld_> yes, now make is clean
<StevenK> Yeah, if you reference stuff that doesn't exist in requires, convoy will scream.
#launchpad-dev 2012-04-24
<StevenK> wgrant: Are you for reviewing https://code.launchpad.net/~stevenk/launchpad/bugs-information_type-ui-portlet/+merge/103193 ?
<wgrant> StevenK: Probably not right now, (see -ops)
<StevenK> wallyworld_: Since wgrant is distracted by *redacted*, can you review my branch?
<wallyworld_> StevenK: ok, just give me a few minutes before i look. i've found a flaw in our person vocab i'm in the middle of fixing
<StevenK> wallyworld_: Still distracted?
<wallyworld_> StevenK: just writing up a mp, almost done'
<StevenK> Should I prod lifeless so I can have three people who are too busy to review my branch? :-P
<wallyworld_> it's complicated so i wan tto do it before i forget what i need to say
<bigjools> wallyworld_: at your age, it's easy to forget I suppose
<wallyworld_> bigjools: fuck off
<bigjools> exactly my predicted response :)
<wallyworld_> StevenK: looking now
<wallyworld_> StevenK: test_proprietary_hidden - i don't think the test is sufficient. proprietary should only be visible if the bug is targetted to a commercial project right? and the test doesn't do that so i think it will succeed even without the feature flag
<StevenK> wallyworld_: We don't currently limit it like that.
<wallyworld_> we are supposed to i think. it's limited like that on the +sharing page
<StevenK> wallyworld_: For the moment, we hide it in the UI everywhere via the feature flag
<StevenK> I'm just making sure it isn't leaking into the JS
<wallyworld_> ok. so long as we don't accidentally toggle the flag and have proprietary exposed when it shouldn't be
<StevenK> wallyworld_: I know PROPRIETARY needs more work.
<wallyworld_> yeah. hopefully between the 5 of us we won't do something dumb by mistake
<wallyworld_> it wouldn't be much work though to add the extra check
<StevenK> wallyworld_: So it also dealt with correctly on Bug:+secrecy and a few other places
<StevenK> It needs careful thought and planning
<StevenK> So I don't want to go off and just do this, I'd like the five of us to discuss how we handle that and do it properly
<StevenK> (In a new branch
<StevenK> )
<wallyworld_> so we must enable disclosure.proprietary_information_type.disabled on prod before landing this then
<StevenK> It doesn't matter, it won't get hit because show_information_type_in_ui is off
<wallyworld_> it would be good to turn it on though as a safety measure
<StevenK> Right
<wallyworld_> just to ensure it's there when we do toggle the show in ui flag
<StevenK> I wasn't planning on setting anything on prod yet
<StevenK> And have all 3 flags set on qas on Thursdasy
<wallyworld_> ok
<wallyworld_> StevenK:  if (privacy_link && LP.cache.show_information_type_in_ui) {..... do you think a nested if() would be better?
<wallyworld_> if( privacy_link) { if(show_in_ui) ... else ....  }
<StevenK> I can, I didn't think it was worth it
<lifeless> -> dentist
<wallyworld_> StevenK:  i think it's cleaner. i'll leave it up to you
<StevenK> wallyworld_: I was trying to avoid indenting the inner block another level so the diff isn't massive
<StevenK> Since the current JS code there is about 150 lines
<wallyworld_> figured as much :-) i don't mind an extra few lines in the diff if the code is cleaner
<StevenK> wallyworld_: I'd personally prefer a smaller diff :-)
<wallyworld_> why?
<wallyworld_> surely final code clarity is most important?
<wallyworld_> i couldn't really care less about the diff size
<StevenK> wallyworld_: TBH, I'd rather not touch that 150 line massive block at all -- it's untested, and I can hopefully delete it in two weeks or so.
<wallyworld_> ok
<wallyworld_> StevenK: 120	+ title: "Change Information Type"    should be "Change information type"? not sure
<StevenK> Hmm
<StevenK> wallyworld_: Which do you prefer?
<wallyworld_> whatever the coding standards say
<StevenK> I just fixed Bug:+secrecy to say Change information type
<wallyworld_> i think it's "Change info type"
<StevenK> So let me fix this too
<wallyworld_> cool.
<wallyworld_> StevenK: also, information_type_edit.on("save", function(e) {...}) -- the guts of the function should be broken out into a separate method which can be unit tested
<StevenK> Er, what guts? I set up a bunch of variables and then call named_post?
<wallyworld_> so you want to separate out the concerns. hooking into a choice selection and doing the xhr call are separate things. in the sharing stuff where i've used a mvc approach, the onsave published an event which the controller subscribes to and these aspects are separately tested. here,  a separate method would be sufficient
<wallyworld_> out code has too much of this stuff in it - it's an anti pattern which would be good to eliminate
<wallyworld_> another argument - it's very much like visual basic eeewwww
<StevenK> wallyworld_: Sure, but I don't see what value that gets us. The guts are in ChoiceSource and the save() function is quite small.
<wallyworld_> how do you test the save action then without invoking the popup? you can't
<wallyworld_> your unit tests are compromised
<StevenK> But how can I, it needs the data from the popup
<wallyworld_> you test the separate method by passing in data you provide and such data is as per what the popup would provide
<StevenK> So you want more tests?
<wallyworld_> one more would be needed if the logic were separated out
<StevenK> And I here I was thinking that two days writing JS tests was enough.
<wallyworld_> depends what you want. spagetti ish code of properly constructed mvc style code
<wallyworld_> s/of/or
<wallyworld_> it would be good if we could start avoiding some of the mistakes of the past for new work
<wallyworld_> if i were writing this, i wouldn't even be using a namespace, i'd use proper yui widgets
<StevenK> That last comment makes me feel SO much better
 * StevenK goes to grab lunch
<wallyworld_> sorry, i was just trying to say that it's ok to compromise on some things, but if we do let's get the structure right
<wallyworld_> i only just recently started using yui widgets as they were intended myself
<lifeless> hah
<lifeless> https://launchpad.net/~lifeless/+participation is buggy
<lifeless> shows owner as member
<lifeless> now, is the bug filed
<lifeless> flacoste: we might want to consider removing https://launchpad.net/~rosetta-alpha, the distinction vs beta is perhaps unneeded?
<wgrant> lifeless: There's also ~launchpad-recipe-beta
<StevenK> LOL
<lifeless> indeed
<StevenK> Sigh, now the tests say no JS
 * StevenK stabs setup_privacy_notification for being so crap
<StevenK> wallyworld_: Hai
<StevenK> Three days on JS tests. /wrists
<StevenK> wallyworld_: MP updated.
<wallyworld_> StevenK: thanks, looking
<wallyworld_> StevenK: looking good. test_information_type_save_success has stuff commented out. plus
<wallyworld_> test_save_information_type - you need to provide a success response and check that save_success is called
<wallyworld_> i can point you at some examples
<wallyworld_> test_perform_update_information_type still has the mockio calls which aren't needed there - you just need to ensure that the save is called
<wallyworld_> same technique as checking save_success called in test_save_information_type
<wallyworld_> see _test_delete_confirmation in test_bugtask_delete for an example of monkey patching a method to ensure it's called
<StevenK> Damn it, I thought I had finished with the tests
<StevenK> And then you point out I still had stuff commented out :-(
<StevenK> And I'm back to null isn't an object.
 * StevenK slams his head through his desk a few times
<wallyworld_> sorry :-(
<StevenK> I'm sick of this branch
<wallyworld_> i don't blame you
<StevenK> var node = this._node; and node is null? :-(
<wallyworld_> which code is that from?
<StevenK> anim.js
<wallyworld_> that means that one of the nodes or selectors passed to choiceedit is not defined and so the resulting node is null
<StevenK> It's calling display_privacy_notification() and dying
<wallyworld_> let me find that method
<wallyworld_> initial thought - one way to solve this is to add some dummy nodes to the html harness
<wallyworld_> the ones which display_privacy_notification tries to find and flash
<wallyworld_> or stub out the call to that method and just check that it has been called
<StevenK> I thought I'd done enough, but I can't figure out what node the privacy notification is trying to grab and failing
<wallyworld_> one thing it looks for is a node with class "global-notification"
<wallyworld_> that is probably a div or something
<StevenK> wallyworld_: Would you prefer I stub it out, or fix it to work?
<wallyworld_> easiest to just add the fake nodes to the html harness
<wallyworld_> less work
<StevenK> I think it's easier to stub the fucker out, given I've been fighting with this for hours.
<wallyworld_> to stub it out, you need to monkey patch it and then add it back at the end of the test, so about 6 or 7 lines of js
<wallyworld_> vs one line of html per node
<wallyworld_> of which there are perhaps 2 or 3
<wallyworld_> StevenK: want me to finish off the branch for you?
<StevenK> I've done the stubbing out for save_information_type
<StevenK> And I've added the div to HTML
<StevenK> Node is still null after adding div class="global-notification"
<StevenK> I'm tempted to stub it out. We don't want to test it here.
<StevenK> If it's called and the body has a private class, we're good.
<wallyworld_> ok, whatever is easiest at this stage :-) i'm handwaving a bit
<StevenK> And working.
<StevenK> wallyworld_: Diff updated.
<StevenK> wallyworld_: Pushed another change to completly drop mockio from test_perform_update_information_type.
 * wallyworld_ looks
<StevenK> wallyworld_: Still looking?
<wallyworld_> StevenK: yeah, bigjools distracted me
<StevenK> wallyworld_: Oh, did you want me to add back the library_exists test? I removed it because I thought it was a stub test just to show how to write one.
<wallyworld_> StevenK: theoretically it should be there
<StevenK> wallyworld_: I'll resurrect it
<wallyworld_> it ensures the module is correctly declared/defined
<wallyworld_> StevenK: test_perform_update_information_type - the function stub should assert that the passed in value is as expected
<wallyworld_> Y.Assert.areEqual(xxx, value)
<StevenK> wallyworld_: Okay, anything else?
<wallyworld_> don't think so. i'm about to approve :-) sorry to be picky about it earlier. i think the tests are great now
<wallyworld_> i would ask for a save success test where the privacy banner were hidden but you'l lkill me :-)
<wallyworld_> to compliment the one where it is shown
<StevenK> Grrr.
<wallyworld_> it should just be a cut'n'paste where s/show/hide :-)
<StevenK> And hasClass('public')
<wallyworld_> yeah
<wallyworld_> r=me anyway. fix if you want :-) if you do fix, the tests then provide good coverage
<wallyworld_> and the extra test ensures that we have covered private being turned on and off and the page is updated as expected
<StevenK> Yes, I've just finished adding it
<wallyworld_> thanks. i bet you can't wait to do another javascript branch!!!
 * StevenK wraps his mouse cord around his neck and pulls very very tightly.
<StevenK> Oh, damn it!! My mouse is cordless!
 * bigjools hands StevenK some rope
 * nigelb hands StevenK some noodles.
<StevenK> Noodles? How do they help?
<StevenK> And gee, two people helping me commit suicide.
<StevenK> I'm not sure if I should be touched or disgusted.
<nigelb> You probably don't need help. You work on LP :P
 * nigelb runs away
 * StevenK points nigelb to https://dev.launchpad.net/Contributions
<nigelb> Dammit.
<nigelb> Wat. I'm still 4th. Needs fixing.
<StevenK> 3 more landing
<StevenK> s
<nigelb> 3 more landings which delete code. That's a bigger challenge.
<StevenK> Not really
 * StevenK finally throws portlet at ec2
<bigjools> I am currently working with Django.  It might be worse than LP.
<wgrant> In parts.
<nigelb> bigjools: You don't work on LP anymore?
<nigelb> (MAS?)
<bigjools> nigelb: haven;t  been since January, I was doing MAAS.  Start part of LP team though.
<bigjools> Start?  Still*
<nigelb> bigjools: Oh. Interesting. Did not know.
<czajkowski> StevenK: have you seen this weeks desktop image the chunkey has been removed
<StevenK> czajkowski: I have fear.
<adeuring> good moorning
<czajkowski> StevenK: less scary
<czajkowski> adeuring: ello
<czajkowski> best tool ever evented - listadmin
<StevenK> czajkowski: How so? (Re: G+)
<czajkowski> StevenK: https://plus.google.com/photos/102921374554385564572/albums/5730819334465556225/5734582582300875890  is niec and innocent looking without the scary chunkey
<StevenK> czajkowski: I meant your PPA comment
<czajkowski> StevenK: how ws I gonna work that out now really :)
<czajkowski> StevenK: each time I get to figure out an answer for one, a new one comes up and wgrant points out its the user not the ppa
<czajkowski> but until that happens I go around in cirlces wondering wtf is up with the ppa
<wgrant> The trouble with letting people run code as part of a feature is that they come up with infinitely many different failures to blame on us :)
<czajkowski> wgrant: like the one last night whouc pointed out WRONG ANSWER
<deryck> Morning, all.
<rick_h> morning deryck
<deryck> I've had all I can take of these doc tests.
<deryck> They will die today.
<rick_h> hah
<czajkowski> deryck: and good morning to you
<stub> ImportError: No module named convoy.meta
<StevenK> Your deps are out of date.
<rick_h> StevenK: are you avail next monday night for our convoy call? I need to reset that back up after the utter failure last time
<StevenK> rick_h: I don't see an issue with that
<rick_h> StevenK: ok, will send out a new email shortly then and see if I get any replies.
<StevenK> rick_h: Perhaps you should chat to the others first. I'm happy to fit in with them.
<StevenK> And if that doesn't work, get deryck to poke their managers until they squeal.
<rick_h> hah will do
<StevenK> rick_h: However, convoy 0.2.4 is in our PPA
<rick_h> StevenK: is it? last I knew I still broke the build
<rick_h> with my makefile that does make test first
<StevenK> I fixed that.
<rick_h> ah, I'll look then, ty much
<StevenK> Your Makefile is sort of pointless, but meh
<rick_h> StevenK: yea, but I've got vim shortcuts and such to auto make test and things
<rick_h> so standard + place + handy
<sinzui> mrevell, ping
<sinzui> jcsackett, ping
<jcsackett> sinzui: pong.
<sinzui> jcsackett,  how goes the day?
<jcsackett> sinzui: alright. looking into cleaning up code mess i noticed on last week's branches. you?
<mrevell> hey sinzui. Thanks for the email and the LEP updates. I won't be able to look at them in detail until tomorrow afternoon.
<sinzui> mrevell, I think https://dev.launchpad.net/LEP/PrivateProjects need a total rewrite. It talks mostly of what other leps do...my team cannot read that and know what to build?
 * sinzui hopes he is not the only one who really knows what private projects are
<mrevell> sinzui, I believe I have a good grasp of what we're looking to achieve with private projects, so I can do that rewrite.
<sinzui> jcsackett, I am torn to write again, which depresses me, work on an entitlement queue mechanism, which depresses me, or return to ie8 which depresses me
<jcsackett> sinzui: ugh.
<jcsackett> sinzui: if it's just a case of not wanting to step on toes, there's the "Share None" card -- i don't think any of us are working on that.
<sinzui> mrevell, I can delete the non-relevant parts of page, but i would limit myself to two hours to put what should be in the page.
<sinzui> jcsackett, I think we can undertake it know that bogtaskflat is in place
<jcsackett> sinzui: yeah, i think it's ready. i was just presenting it as something else you could do if you need a break from the three things you've been doing for weeks.
<sinzui> I try not to make myself a part of the critical path because I need to be available/distractable at all times
<jcsackett> sinzui: fair.
<sinzui> jcsackett,  put a card on the board to describe the clean up.
<jcsackett> sinzui: sure thing, thought i had.
<gary_poster> jml, hey.  Hopefuly helpful heads up that does not bother us right now: when you install the current subunit trunk as built in a PPA, it builds subunit-filter to run with python3, which is broken.  subunit is not importable by python 3.  Do you want me to file this trunk packaging bug as a subunit software bug, or just leave this with you here on IRC, or...?
<jml> gary_poster: trunk bug pleaose
<gary_poster> ack jml will do
<kirkland> is it possible to apt-mirror a private-ppa?
<kirkland> I'm trying to use my private url with the name:pw embedded in it
<kirkland> but that's not working
<kirkland> nevermind, I think I found a hack
<abentley> deryck: There's no OCR.  At your leisure, could you please review https://code.launchpad.net/~abentley/launchpad/archivejob/+merge/103331 ?
<deryck> abentley, sure.
<abentley> deryck: thanks.
<deryck> np!
<gary_poster> deryck, I'm seeing errors trying to import convoy (and thus breaking make) in a fresh build of r15141.  Do you know anything about this? I seem to recall you and maybe rick_h involved with convoy?  Probably completely wrong :-P but if you have any info I'd appreciate it
<rick_h> gary_poster: it should be in the deps
<deryck> gary_poster, rick_h and I are totally to blame :)
<rick_h> no new issues with it in some time, last change was > month ago
<rick_h> gary_poster: any more details on error?
<gary_poster> rick_h, deryck :-)
<gary_poster> this is in parallel testing, which has similarly been running fine for some time
<gary_poster> for some definition of fine ;-)... *this* has been fine
<deryck> heh
<rick_h> gary_poster: gotcha, I know SteveK did update the PPA build last night to fix the build
<rick_h> so guess there is a change recently, I lied
<deryck> gary_poster, yeah, maybe update packages?
<gary_poster> deryck, this is as of a completely fresh ec2 instance created and initialized from scratch about 4 hours ago.  rick_h this doesn't tell me much new :-) http://pastebin.ubuntu.com/944439/
<gary_poster> so is this supposed to be installed via debs?
<rick_h> looking
<rick_h> gary_poster: yea, it's pulled from lp's ppa
<gary_poster> ok, lemme run with that for a second...
<rick_h> gary_poster: so yea, just looks like it's not installed atm.
<dhillon-v10> hi all, I have a quick question about launchpad PPA's: so i just finished compiling a custom kernel and the whole process resulted in 2 debian packages, the kernel contains a debian folder with all the make files and stuff, now if I upload the patched source to launchpad will it build the kernel for me and let me download the debs?
<gary_poster> rick_h, appears to be! http://pastebin.ubuntu.com/944443/
<abentley> deryck: Could you please run "select count(*) from archivejob" against the staging DB?
<gary_poster> this is in on lucid (that is, in a lucid lxc container) fwiw
<rick_h> gary_poster: hmm, yea so that's the right version and eveyrthing
<deryck> abentley, sure.  give me two seconds.... new connect command IIRC and haven't run it yet.  consulting mail....
<rick_h> gary_poster: can just a normal python shell can you import convoy.meta? Guessing path issue?
<gary_poster> rick_h, no, you can't
<gary_poster> http://pastebin.ubuntu.com/944449/fwiw
<gary_poster> heh http://pastebin.ubuntu.com/944449/
<rick_h> gary_poster: ok, give me a sec to update my local install and see if it's something I can get due to recent package update
<gary_poster> thx
<deryck> abentley, 0 results.
<abentley> deryck: Thanks.  So the odds are that there are no rows on production either...
<rick_h> doh, not getting issue getting an updated python-memcache? Anyone else seen this via rocketfuel-get?
<rick_h>   Getting distribution for 'python-memcached==1.49DEV-r57'.
<rick_h> sorry Error: Couldn't find a distribution for 'python-memcached==1.49DEV-r57'.
<deryck> abentley, yeah, good odds.
<deryck> rick_h, update download cache?
<gary_poster> rick_h, suspicious? http://pastebin.ubuntu.com/944455/
<rick_h> deryck: got it, yea don't know what that didn't update
<rick_h> gary_poster: yea, seems to be missing the python bits :/
<gary_poster> exactly
<rick_h> guessing the StevenK update package lost some files
<deryck> rick_h, cool
<gary_poster> seems like it
 * rick_h goes to see if it's possible to see what changed
<rick_h> gary_poster: so I'll work with stevek on that. Is it possible for you guys to cheat and pull the python package just to unblock you?
<rick_h> http://pypi.python.org/pypi/convoy/0.2.4
<gary_poster> rick_h, may I leave this with you for now?  Please ping me if you'd like me to help, and also please ping me if you find a resolution.  Also, if you verify the problem we probably should send a note to launchpad-dev so others know what is going on.  pulling the Python package: not easy because we're testing LP trunk, and rebuilding every time :-P If it starts to be really painful we can make our own branch of trunk
<gary_poster> I guess
<gary_poster> but it is kind of a big deal to us
<gary_poster> not gigantic
<gary_poster> but not small :-)
<rick_h> gary_poster: yea, I'm just not 100% sure how this is setup since SteveK has been doing it from the start.
<gary_poster> gotcha rick_h
<rick_h> I've handled all the python and he all the packaging
<gary_poster> cool
<rick_h> gary_poster: definitely, I'll send an email to -dev
<gary_poster> cool thanks rick_h.
<rick_h> thanks for the heads up, sorry for the borkage
<gary_poster> np, welcome
<abentley> deryck: Could you also please review https://code.launchpad.net/~abentley/launchpad/celery-everywhere-6/+merge/103346 ?
<deryck> abentley, sure.
<deryck> abentley, I'll get them before I EOD.
<abentley> deryck: okay.
<deryck> sinzui, ping
<deryck> sinzui, unping, I just found it.
<deryck> weird how to happens. :)
<deryck> s/to/that/
<flacoste> lifeless: time for a chat?
<lifeless> flacoste: still doing just-got-up-thing, will ping you in a couple of minutee
<lifeless> s
<flacoste> lifeless: ack
<deryck> abentley, r=me on both MPs.
<abentley> deryck: Thanks!
<deryck> abentley, np!
#launchpad-dev 2012-04-25
<wgrant> StevenK: Erm
<wgrant> StevenK: Why does https://code.launchpad.net/~launchpad/+recipe/launchpad-convoy use nest-part rather than merge?
<nigelb> Ah. It's a holiday today?
<wgrant> nigelb: Yeah, ANZAC Day for AU/NZ.
<StevenK> wgrant: :-(
<nigelb> wgrant: Ah. I get worried when this channel has very little activity ;-)
<G> nigelb: haha, especially since it's the middle of the week :)
<nigelb> G: Exactly
<adeuring> good morning
<czajkowski> aloha
<gary_poster> Hey StevenK.  The convoy packaging problem is an issue for parallel testing because our automated system to initialize ec2 instances for tests breaks.  It looks like you worked on it but didn't get it working, seemingly because of a trivial version-already-exists-in-ppa issue.  Should I just try making a trivial change to convoy and firing off the recipe to see if that helps?
<wgrant> gary_poster: Perhaps we should revive the old known-good version and try fixing it in a staging PPA or something?
<gary_poster> wgrant, yeah, that sounds reasonable.
<wgrant> gary_poster: Do you know how to do that?
<gary_poster> wgrant, no, but it didn't sound too daunting & I was going to give it a try.  hints much appeciated.
<gary_poster> r
<wgrant> gary_poster: Delete the current version from the PPA, then copy the last known-good versions back from the grave (search for convoy with "Any status" on +copy-packages, select the last good version for each series, copy with binaries)
<gary_poster> wgrant ack.  Then I have good versions in the PPA.  Then we can temporarily change the current recipe to build into a different PPA while we work out the kinks?
<wgrant> Looks like 0.2.1-0~21+r21 is the version to go for, I think.
<wgrant> gary_poster: Right, you can manually ask the recipe to build into a different PPA
<wgrant> So we don't break everyone.
<rick_h> is there any way to get the old recipe? I broke things by adding a Makefile to help run tests and upload to pypi, I can change that back if I knew what the old working recipe was
<rick_h> that's what I was trying to figure out yesterday
<wgrant> The recipe probably hasn't changed. You might have changed the packaging, which is fully versioned in the bzr branch.
<wgrant> It's likely that the makefile broke it
<wgrant> Simply by existing
<rick_h> yea, I know that broke it, but then he went in and 'fixed' it and I couldn't tell what his 'fix' was
<wgrant> Since the debian/rules just uses dh $@, which tries to autodetect the build system.
<rick_h> right
<gary_poster> wgrant, rick_h, I believe I have done the right temporary fix in the PPA.  Verification appreciated.
<wgrant> gary_poster: Looks good
<wgrant> Heh
<gary_poster> cool thank you
<wgrant> Yeah, seems OK
<gary_poster> :-)
<rick_h> wgrant: so where would this branch be Stevek had going with the debian dir? https://code.launchpad.net/~launchpad/+recipe/launchpad-convoy
<wgrant> Should be published in a couple of minutes.
<gary_poster> cool
<rick_h> ok, cool
<rick_h> sorry gary_poster, forgot the holiday
<wgrant> rick_h: The second branch referenced in the recipe
<wgrant> https://code.launchpad.net/~launchpad/convoy/trunk
<rick_h> ah, gotcha. Sorry, read that as convoy/trunk
<wgrant> (neither StevenK nor I are here today, but we can look tomorrow)
 * rick_h drinks more coffee
<wgrant> Yeah, that's a bit confusing.
<wgrant> Should probably s/trunk/packaging/
<rick_h> gary_poster: so if I can bug you a sec, where did you make a change that's now building?
<gary_poster> rick_h, I went to https://launchpad.net/~launchpad/+archive/ppa
<gary_poster> clicked on the delete link on top right
<gary_poster> deleted convoy
<gary_poster> went back to https://launchpad.net/~launchpad/+archive/ppa
<rick_h> ah ok, I thought you were making a 'trivial' change to it somewhere and didn't see an edit
<gary_poster> clicked on copy link
<rick_h> gotcha, thanks
<gary_poster> searched for convoy with Any status and copied the one that seemed like it might not be brokacool np
<StevenK> I did question rick_h adding a Makefile, I felt it was pointless, but I utterly missed that dh would try it first. :-/
<wgrant> A Makefile is handy here
<rick_h> I've gotten too used to having make test and make upload, make sdist for all my setup.py etc needs
<wgrant> Yup
<rick_h> didn't realize I was going to bork things, I need to find some time to get back into playing with packaging bits
<wgrant> Lots of our stuff has Makefiles for that purpose.
<rick_h> yea, since LP I've gone through the Make oreilly book and gone nuts
<rick_h> https://github.com/mitechie/Bookie/blob/develop/Makefile bwuhahahaha
<StevenK> rick_h: I tend to put .PHONY at the end with everything
<StevenK> Since make parses it in one-shot
<rick_h> StevenK: yea, I started going that, but trying to remember/find them in teh list got to be a pain
<rick_h> so I've gone the python "explicit" route
<deryck> Morning, all.
<dobey> morning deryck
<dobey> did some part of the LP DBs get rolled back at all last night or earlier this morning?
<wgrant> No.
<wgrant> What's the issue?
<dobey> https://code.launchpad.net/~ralsina/ubuntuone-control-panel/unique_in_ubuntu/+merge/103337
<dobey> this branch had 2 reviews last night, and even a couple of comments from tarmac failing to land it
<dobey> but now it has nothing
<dobey> oh weird
<dobey> nevermind
<dobey> i see the problem
<wgrant> It was probably deleted and recreated or something.
<wgrant> We don't just randomly revert bits of the database :)
<wgrant> Hmm
<wgrant> I indeed have emails from 11 hours ago about that MP
<dobey> no, it looks like he made 2 proposals at the same time
<wgrant> But it has a different ID
<wgrant> So yeah
<dobey> sorry :)
<wgrant> 103336 is the one tarmac attacked
<cjwatson> Some people get very confused by the MP interface and think they need to re-propose any time they make a change
<wgrant> In this case it looks like a swift double-click of the Propose Merge button
<wgrant> The two were created seconds apart.
<dobey> cjwatson: nah, this wasn't that case. more likely twitchy finger. i've made it very clear to our team to not do re-submit all the time as well :)
* abentley changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: abentley | Firefighting: - | Critical bugs: 3.47*10^2
<abentley> adeuring: I'm the OCR.  Could you please review https://code.launchpad.net/~abentley/launchpad/celery-everywhere-7/+merge/103483 ?
<adeuring> abentley: sure
<adeuring> abentley: r=me
<abentley> adeuring: Thanks
<gmb> deryck, I've bumped https://bugs.launchpad.net/launchpad/+bug/924378 up to Critical; it's now caused two build failures on db_lp in as many days (though not on consecutive runs, AFAICT).
<_mup_> Bug #924378: buildbot spurious failure UncleanReactorError for TestPullerMasterIntegration <Launchpad itself:Triaged> < https://launchpad.net/bugs/924378 >
<gmb> Just wanted to give you the heads up.
<gmb> deryck, If you don't have resources enough to tackle it, gary_poster has suggested that ~yellow take it on.
<gary_poster> Because we only have 8 other bugs in the queue now :-P
<deryck> gmb, thanks. We can add it to our board, but it will likely be a bit before we can get to it.
<gary_poster> thanks deryck.  We'll coordinate via the bug if we tackle it
<deryck> gmb, but we could disable the test and wait on one of our squads to get bandwidth to fix it properly.
<deryck> gary_poster, ok, cool.
<gary_poster> deryck, +1 on disabling fwiw :-) although the bug makes reference to some reason why that's a bad idea
<deryck> gary_poster, I think that was speculation from flacoste back when I filed it.  if no one recalls, we could disable and see the fallout to be sure ourselves. :)
<gary_poster> :-) fair enough
<mrevell> Apologies jcsackett, you weren't on the calendar entry. I've invited you now.
<mrevell> sinzui, jcsackett, danhg, flacoste, matsubara, czajkowski: https://plus.google.com/hangouts/_/extras/talk.google.com/better-privacy
<jcsackett> mrevell: having a bit of fun trying to get to the hangout. be there in a moment.
<mrevell> jcsackett, no worries
<sinzui> rick_h, do you know what I can do to convoy so that I can run Lp locally?
<rick_h> sinzui: so gary_poster updated the ppa I thought with an older version that should work
<rick_h> so update the packages?
<sinzui> already done
<sinzui> ImportError: No module named convoy.meta
<gary_poster> rick_h, sinzui that will only work if you have not updated, or on a fresh install, IIUC
<sinzui> ^ still present
<gary_poster> Because all I did was remove the newer package and reinstate the old
<rick_h> can you remove/reinstall?
<sinzui> gary_poster, you mean visit lp and locate an older package, then lock the version?
<gary_poster> Alternatively, sinzui, you could do (not exact spelling) sudo easy-install convoy and that would probably fix things up at the expense of a mess in your system python
<gary_poster> sinzui, no I mean in the PPA, I deleted the new package
<gary_poster> then copied the old one back into the PPA
<sinzui> easy install is crack in that case
<gary_poster> If I understand you, I agree that it would be nasty, but I think it would work
<sinzui> I see precise is 4 version behind
<gary_poster> yes
<rick_h> I think r20 will still work for you
<rick_h> the path stuff is only needed on production and I don't think we use it on dev, but not 100% sure
<gary_poster> jml, just a heads up in case you are around: the test failure I emailed you about is also affecting our production usage, and therefore blocking our progress.  We will be working on it.  Any insights from you will of course be welcome.  benji, could you file a bug for this so we have a coordination point?
<benji> gary_poster: sure
<gary_poster> thanks benji.  benji, I doubt it affects us, but there are two subunit patches in new, recent bugs for subunit that I noticed.
<gary_poster> bug 987490
<_mup_> Bug #987490: Test failures with Python 2.6 <subunit:New> < https://launchpad.net/bugs/987490 >
<gary_poster> bug 987938
<_mup_> Bug #987938: subunit trunk packaging breaks subunit-* commands <subunit:New> < https://launchpad.net/bugs/987938 >
<gary_poster> no not the last one
<gary_poster> bug 987514
<_mup_> Bug #987514: Regressions in support for Python 3 <subunit:New> < https://launchpad.net/bugs/987514 >
<gary_poster> that one
<jcsackett> is there a good/easy way to get +/- LoC for a branch out of bzr? like some magical incantation to pass to diff, maybe?
<jelmer> jcsackett: you can feed the diff through diffstat
<jelmer> "bzr send -o - | diffstat"
<jcsackett> ah, awesome.
<jcsackett> thanks, jelmer.
<jelmer> we should add a lp-dev-tools script that does that :)
<abentley> deryck: Could you please review https://code.launchpad.net/~abentley/launchpad/celery-everywhere-8/+merge/103545 ?
<deryck> abentley, sure.
<abentley> deryck: thanks!
<deryck> abentley, np!
<deryck> abentley, looks good to me. r=me.
<abentley> deryck: Thanks.
<deryck> abentley, np.  I've done enough of these now.  It's easy to spot the pattern. :)
<lifeless> jcsackett: bug 988510 confuses me
<_mup_> Bug #988510: Bug supervisor should not be subscribed on ubuntu bugs when transition from security to userdata <disclosure> <regression> <Launchpad itself:In Progress by jcsackett> < https://launchpad.net/bugs/988510 >
<lifeless> jcsackett: I thought we'd checked that the vanilla policies around information types defined would work for ubuntu ?
<jcsackett> the policies work, it's a matter of handling auto subscriptions.
<lifeless> auto or implicit?
<lifeless> is this perhaps a mixed-mode issue until we stop doing auto subscriptions entirely?
<flacoste> sinzui: you did send a sketch on how to implement agreements in Launchpad on launchpad-dev a while back right?
<flacoste> "Proposed team agreement" feature
<sinzui> flacoste, I did
<flacoste> gary_poster: who is chairing this week, you didn't update the wiki page after last week meeting
<gary_poster> sorry flacoste, lemme find the email
<gary_poster> flacoste, sinzui
<sinzui> flacoste, That is the email thread I started. I cannot determine the real dates of when it happened because Shitehawk^hThunderbird does not honour my date formats. I cannot tell months and days apart
<flacoste> sinzui: i found it in the archive, thx, it was 2011-11-08
<sinzui> I see a sensible iso approved big-endian date just like I configured my desktop to use.
<gary_poster> sinzui, you are still ok with hosting team lead call?
<gary_poster> lifeless, you are unavailable so we do not need to try and invite you right?
<bigjools> morning
<sinzui> gary_poster, I am, I wish my wife would stop blocking me from doing it
<gary_poster> ok
* abentley changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 3.47*10^2
<timrc> It feels like its taking excessively long to do ppa2ppa binary copies
<timrc> still pending, wtf :(
#launchpad-dev 2012-04-26
<wallyworld_> StevenK: is only workaround to downgrade convoy? ImportError: No module named convoy.meta
<wgrant> sudo apt-get install convoy/precise
<wgrant> should do it
<wallyworld_> so pkg has been updated in repos?
<wgrant> No, downgraded.
<wgrant> That should still downgrade it
<wallyworld_> ok,will try thanks
<wallyworld_> all good
<StevenK> wallyworld_: I'm working on it.
<wallyworld_> np. i downgraded to the previous version
<StevenK> wgrant: I don't think I need Provides: convoy, right?
<wgrant> StevenK: Hopefully only launchpad-developer-dependencies depends on it, indeed.
<wgrant> So just C/R
<wgrant> sinzui: I see your progressive-enhancement-ftw branch touches the inline-picker.pt, which has a div inside a span, which is illegal. Any chance you could look at fixing that?
<wgrant> Not sure what it's doing there, but it's probably simple to eliminate.
<wgrant> I hope.
<StevenK> Woo
 * StevenK rips out five self.assertEquals() and replaces it with MatchesStructure()
<wallyworld_> wgrant: is it preferred to use a CTE or subsquery to force a distinct to be evaluated before a sort?
<lifeless> wallyworld_: trick question, there's no point evaluating distinct right before a sort
<lifeless> wallyworld_: as doing distinct requires sorted data
<wgrant> You also have to be careful about doing that with resultsets, as Storm will apply the limit to the outside.
<wgrant> You need to play around to see what performs well.
<lifeless> wallyworld_: whats the big picture ?
<wallyworld_> such an optimisation was done for a query on the accesspolocygrantflat table already though
<wallyworld_> lifeless: i am looking at a query which returns a number of rows which when distinct is applied will be far less
<wallyworld_> and i need the result to be ordered on one of the columns which distinct will operate with
<wgrant> Need to know specifics.
<wallyworld_> lifeless: wgrant: https://pastebin.canonical.com/64992/
<wallyworld_> i'm experimenting atm with a couple of approaches
<wallyworld_> a single query and a bit of python to loop over the result set vs 2 queries
<lifeless> https://pastebin.canonical.com/64993/
<wallyworld_> the sql pasted would be for the single query approach. the will be many rows with SOME and these need to be reduced down to 1 per grantee
<lifeless> your current query conflats distinct columns and calculate results
<wallyworld_> yes it does, i hadn't finished it yet, just threw up something quickly cause you asked :-)
<lifeless> indeed, so the thing is you asked about forcing an operaiton order; but that sort of optimisation always happens after you can express what you want basically :)
<wallyworld_> so, without the distinct, the result set might be largish. and order by is done first right?
<lifeless> no
<lifeless> order by is done depending on the plan
<lifeless> it may use an index
<lifeless> it may sort in memory
<wallyworld_> ok. there was a bit of sql already refactored and the comment was that we wanted to force the distinct to be done first since the sort by time dominated the query time
<lifeless> it may be different on each execution depending on the params
<wallyworld_> so a sub query was used
<wgrant> You need to test with both Ubuntu and Launchpad.
<wgrant> as targets
<lifeless> wallyworld_: for this case or another case? Tuning is a vicious cycle of iteration and experimentation
<wallyworld_> lifeless: another case
<lifeless> in which case, ignore it. Optimise this separately.
<wallyworld_> ok. i wasn't sure if we were dealing with a postgres quirk or not
<wallyworld_> s/quirk/generalisation
<wgrant> postgres quirks are completely situation-dependent.
<wgrant> You have to test.
<lifeless> I have a couple of thoughts
<lifeless> what do you want to show when both an artifact and  non-artifact grant are present ?
<lifeless> or do we prevent that ?
<wallyworld_> we show ALL
<wallyworld_> but i now need to know of there are SOME also
<wallyworld_> whereas before we didn't need to know that
<wallyworld_> so previously the query i pasted used MIN(COALESCE(artifact, 0)) and a group by
<wallyworld_> if i keep that i need a second query
<wallyworld_> or i remove the MIN and have one query with a distinct and order by and a bit of python logic
<wallyworld_> lifeless: actually, i take it back. i don't need the 'distinct on', just plain old distinct.
<wallyworld_> for my use case
<wgrant> You also don't care about order there, do you?
<wallyworld_> wgrant: it makes the post processing easier but is not strictly necessary. if the distinct is run first, the order by is trivial
<lifeless> what about https://pastebin.canonical.com/64994/
<wallyworld_> hence my wanting to run distinct first
<lifeless> you'll get up to two rows for any grantee,policy pair
<lifeless> one with the min null and True
<lifeless> one with a min value and False
<wallyworld_> it's missing the group by but perhaps
<lifeless> why would it need one ?
<wgrant> We need to know (person, person, artifact grant?, policy grant?)
<wgrant> s/person, person/person, policy/
<lifeless> well, this gives you that but not folded into one row
<lifeless> if you are paginating, you might need the fold
<lifeless> but if you are paginating, you might want to avoid entire-context processing, which group by will need
<wgrant> This is post-pagination
<wallyworld_> lifeless: the group by is needed because there's a min() in the select
<lifeless> wallyworld_: oh duh, of course.
<wallyworld_> my original query works. wgrant, could you possibly run https://pastebin.canonical.com/64992/ on dogfood, adjusting ids for ubuntu?
<wgrant> 90ms for me + ubuntu-crashes-universe
<wgrant> So very slow
<lifeless> wgrant: how does mine run ?
<wgrant> Buggily
<lifeless> wgrant: with an appropriate group by
<wallyworld_> wgrant: perhaps https://pastebin.canonical.com/64995/
<lifeless> wgrant: does qas have appropriate data ?
<wgrant> qas has appropriate data
<wgrant> lifeless: Your thing has no chance of running without a rewrite
<wgrant> It uses artifact in both an aggregate and a non-aggregate.
<lifeless> what are the right constants for you + u-c-u ?
<wgrant> wallyworld_: 120ms
<wallyworld_> so even worse :-(
<wgrant> lifeless: SELECT id FROM accesspolicy WHERE distribution = 1
<wgrant> And (21997, 1324721)
<wallyworld_> wgrant: maybe a subselect?
<wgrant> Well
<lifeless> 72ms
<lifeless> wgrant: all specific grants atm right ?
<wgrant> For those constraints, yes.
<wgrant> Ah
<lifeless> SELECT DISTINCT grantee, policy, min(artifact), artifact is NULL FROM AccessPolicyGrantFlat
<lifeless> WHERE AccessPolicyGrantFlat.grantee IN (21997, 1324721) AND AccessPolicyGrantFlat.policy IN (65,66)
<wgrant> The old query is 80ms for the same data on DF, so the 90ms before wasn't too bad.
<lifeless> GROUP BY AccessPolicyGrantFlat.grantee, AccessPolicyGrantFlat.policy, artifact is NULL
<lifeless> ORDER BY AccessPolicyGrantFlat.grantee, AccessPolicyGrantFlat.policy;
<wallyworld_> so what's the concensus?
<wallyworld_> one was run on dogfood, the other qas so can't really compare
<wgrant> SELECT AccessPolicyGrantFlat.grantee, AccessPolicyGrantFlat.policy, NOT bool_and(artifact IS NOT NULL), bool_or(artifact IS NOT NULL)
<wgrant> FROM AccessPolicyGrantFlat WHERE AccessPolicyGrantFlat.grantee IN (21997, 1324721) AND AccessPolicyGrantFlat.policy IN (130423, 130464) GROUP BY AccessPolicyGrantFlat.grantee, AccessPolicyGrantFlat.policy
<wgrant> SELECT AccessPolicyGrantFlat.grantee, AccessPolicyGrantFlat.policy, bool_or(artifact IS NULL), bool_or(artifact IS NOT NULL)
<wgrant> FROM AccessPolicyGrantFlat WHERE AccessPolicyGrantFlat.grantee IN (21997, 1324721) AND AccessPolicyGrantFlat.policy IN (130423, 130464) GROUP BY AccessPolicyGrantFlat.grantee, AccessPolicyGrantFlat.policy
<wgrant> better
<wgrant> 78ms on DF
<wallyworld_> i haven't seen bool_or before
<wgrant> I've never had cause to use it before, either.
<wgrant> But I think it fits well here.
<wallyworld_> what's it do?
<wgrant> It's an aggregate function that computes the boolean OR.
<wallyworld_> ah right, nice
<wgrant> Which is, I think, exactly what we want here.
<wallyworld_> well, before the queru produced column values 'ALL' or 'SOME' not true or false
<wgrant> Yeah
<wgrant> But that doesn't really make sense any more
<wallyworld_> so i could CASE it
<wgrant> Since we may need to return both.
<wallyworld_> sure
<wallyworld_> wgrant: so there's so order by there. that would make post processing easier
<wallyworld_> but if it adds too much overhead
<wgrant> You can just stick an ORDER BY on the end
<wgrant> Doesn't affect performance in this case.
<wgrant> Since it has to do the hashagg first.
<wallyworld_> rightio
<wallyworld_> wgrant: seems to work ok locally too. thanks. updating code now....
<wgrant> Great.
<wgrant> Hm
<wgrant> So it looks like tag searching becomes much faster if I remove all the INTERSECT crap
<lifeless> wgrant: the bool_or is redundant there
<wgrant> lifeless: How?
<lifeless> foo is null is boolean already
<lifeless> ditto is not null
<wgrant> I use them for the aggregateness
<wgrant> Not the boolness
<wgrant> Hm. I guess since we paginate before this, we could possibly just get DISTINCT AccessPolicyGrantFlat.grantee, AccessPolicyGrantFlat.policy, artifact IS NULL.
<wgrant> Since we don't need a single row per person
<lifeless> wgrant: huh, please enlarge
<lifeless> (on the aggregateness)
<wgrant> lifeless: I want the aggregate
<wgrant> Directly requesting a non-distinct bool is not an aggregate.
<lifeless> oh right, I see
<lifeless> handy
<wgrant> But a distinct bool is probably sufficient here, since we don't care about pagination in this query
<wgrant> EXPLAIN ANALYZE SELECT DISTINCT AccessPolicyGrantFlat.grantee, AccessPolicyGrantFlat.policy, artifact IS NULL
<wgrant> FROM AccessPolicyGrantFlat WHERE AccessPolicyGrantFlat.grantee IN (21997, 1324721) AND AccessPolicyGrantFlat.policy IN (130423, 130464) ORDER BY AccessPolicyGrantFlat.grantee, AccessPolicyGrantFlat.policy;
<wgrant> is pretty much the same speed and less ugly, but will return up to two rows per person
<wgrant> And is sort of the obvious way to go.
<wgrant> wallyworld_: ^^
<lifeless> indeed
<lifeless> its roughly, except cleaner, what I originally suggested.
<wgrant> And working :)
<wgrant> lifeless: Have you reworked the tag searching stuff at all?
<lifeless> no
<wallyworld_> wgrant: i really want one row per person. the query with bool_or is better
<wgrant> k
<wgrant> plans are identical
<wgrant> So just query prettiness, really
<wallyworld_> wgrant: and the order by is done outside this bit by the batching stuff i realised
<wgrant> Well, yeah, (person.id, policy.id) is hardly going to be a user-friendly sort. I assumed it simplified your code somehow.
<wallyworld_> it would have but now we have one row per person so it doesn't matter
<wallyworld_> ah never mind, forget my comment about batching - different query
<wgrant> lifeless: https://pastebin.canonical.com/64996/ is what ALL tag search queries look like now
<wgrant> lifeless: Replacing the EXISTS ( ... INTERSECT ... ) with EXISTS ( ... ) AND EXISTS ( ... ) is about 30 times faster.
<wgrant> I guess it plans better from the tag counts.
<wgrant> I guess I'll do it behind a flag and see what happens.
<wgrant> Because it, like, makes Ubuntu multi-tag searches not take 10s.
<wgrant> (although it was 20s in the old schema)
<lifeless> wgrant: I tweaked it for the old schema at one point
<lifeless> wgrant: I'm glad you can get it better
<lifeless> wgrant: did you end up making an array for tags ?
<wgrant> lifeless: Not in this round.
<adeuring> good mornin
<stub> Grrr... lifeless stole my namespace with lazr.postgresql.
<stub> What do we have besides lazr? Need to package a Python library containing PG helpers.
<wgrant> lazr.postgresql, maybe? :)
<stub> :-P
<stub> Lot of renaming to move that db patch buildout application
<lifeless> stub: huh, shove your stuff in there.
<lifeless> stub: its what its there for
<stub> And the obvious name for lazr.postgresql application has now been taken - might be interesting. http://pypi.python.org/pypi/pgmigrate2/1.2.2
<lifeless> hah
<lifeless> stub: so seriously, put your helpers in lazr.postgresql; theres plenty of namespace within that
<lifeless> stub: I just hadn't bothered moving stuff it didn't need.
<lifeless> (into it)
<stub> lifeless: Merging the buildout driven application and my helper library looks like a packaging mess
<lifeless> stub: how so?
<lifeless> stub: which helper library ?
<stub> One I'm going to assemble so I can get common CLI, helpers like ConnectionString etc. available for our various scripts (backups, database report) without pulling in a Launchpad tree or cargo culting.
<stub> Been cargo culting so far and finally got sick of it.
<lifeless> stub: great; buildout should be fine for that.
<lifeless> stub: its got a very small dep footprint
<lifeless> stub: and will play nice with e.g. pip or virtualenv (which btw we probably want to do a mass migration to)
<stub> I'm thinking a .deb
<lifeless> stub: still no problem, I wanted lazr.postgresql to end up deb'd too with a primarily CLI interface.
<stub> Python packages into .deb seem not hard if you are using setup tools. Buildout?
<lifeless> stub: its still setuptools
<lifeless> stub: don't overthink it :>
<lifeless> stub: buildout is for local dev mode; you can totally ignore it as far as delivery of debs is concerned.
<stub> I'm used to using buildout assembing a tree
<lifeless> separate problems.
<lifeless> buildout will give you the dev environment.
<lifeless> The deb stuff will be a separate packaging tree anyhow.
<stub> Yer, just as long as I can get 'python setup.py install' to install mystuff my limited packaging skills should suffice.
<lifeless> which buildout doesn't affect ;)
<lifeless> also you might like to use pkgme for deb stuff
<lifeless> stub: all that said, the -ops are good at getting buildout trees onto machines via deploymgr, so you might be good to go just as-is, with no packaging effort.
<stub> It will end up being used by stuff like lp:losa-db-scripts. Getting helpers on the standard PYTHONPATH seems best to me.
<wgrant> While you're here, I think there is an issue with your assignees-must-have-visibility policy.
<wgrant> lifeless: ^^
<lifeless> stub: sure
<lifeless> wgrant: whats issue ?
<wgrant> lifeless: I've fixed lots of bugs in my company's private project.
<lifeless> stub: we should do a catchup; after wgrant grills me ?
<wgrant> lifeless: But I've just left the company :(
<stub> And the extracted database report, which is what I actually want to be working on.
<stub> lifeless: Sure, or tomorrow if it is late for you.
<lifeless> wgrant: clearly none of the bugs are your responsibility any more
<wgrant> lifeless: Now all these historical bugs have to be molested, since I no longer have accss.
<lifeless> stub: it is late, but today is better.
<stub> k
<wgrant> lifeless: That argument might apply to an In Progress bug.
<lifeless> wgrant: I don't see a conceptual problem: open bugs *should* be updated. Closed bugs the assignee is just vanity, and its in the changelog for the bug.
<wgrant> But it probably doesn't apply to a Fix Released one.
<lifeless> if you consider assignee a next-actor field its clearly irrelevant for F-R bugs
<lifeless> arguably it should not permit being set there anyhow.
<wgrant> That's not how the field is defined at present.
<wgrant> And it has never been used that way.
<lifeless> (not something we could do today, but if we were starting from scratch...)
<lifeless> wgrant: actually, it has been used that way.
<wgrant> Some Ubuntu process bugs use it that way.
<wgrant> use/used
<wgrant> But nothing widespread.
<lifeless> wgrant: I forget which team, but there is/was one that religiously updates it that way.
<lifeless> wgrant: so I admit its a little awkward, but it is, I think, the best of a bunch of bad options.
<lifeless> wgrant: allowing data that is inconsistent means we have to have safeguards and doublechecks throughout the system; having assignee imply notification means having it imply visibility (by definition)
<lifeless> wgrant: if we take any one piece out, the system gets harder to explain and use.
<lifeless> wgrant: I find it implausible that users will be functionally affected if the assignee is cleared when someone leaves; consider the following identical cases:
<lifeless>  - the user deletes their LP account
<lifeless>  - the user merges their LP account
<lifeless> wgrant: another way of thinking about it is that having a single identity for interacting with multiple partitioned visibility areas has an inbuilt tension.
<lifeless> wgrant: if you left canonical, for instance, your google docs would be migrated and the account and all its track record nuked.
<lifeless> stub: so, skype?
<stub> sure
<wgrant> lifeless: Assignee doesn't have to imply notification any more than structural subscription does.
<StevenK> danhg: O hai!
<danhg> hey stevenk
<StevenK> danhg: I'd like to get a text review if you have a moment?
<danhg> yeah sure
 * StevenK stabs gpg for being difficult
<jml`> hey
<jml> the bug listing page seems to have no favicon
<jml> never mind.
<deryck> Morning, everyone.
<rick_h_> morning
<abentley> deryck: morning.
<abentley> rick_h_: morning.
<StevenK> rick_h_: Did you see I fixed convoy?
<rick_h_> StevenK: no, didn't see it. thanks!!!
<rick_h_> StevenK: I'll pull/check it out and reply to the -dev email then
<StevenK> rick_h_: It's also on asuka at least
<rick_h_> StevenK: awesome
<StevenK> rick_h_: And we set caching on asuka for +combo
 * rick_h_ assumes asuka is a dev server
<rick_h_> or something
<StevenK> asuka is staging/qastaging's asppserver
<rick_h_> cool
<StevenK> *appserver
<StevenK> rick_h_: We might be in a position to turn on the feature flag on qas and see how much breaks
<rick_h_> StevenK: very cool then
* rick_h_ changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: rick_h* | Firefighting: - | Critical bugs: 3.47*10^2
* jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: jcsackett, rick_h* | Firefighting: - | Critical bugs: 3.47*10^2
<awilkins> Is there a statechart of the Launchpad bug statuses ; I have the list of states, but not the transitions. Or is there a particular piece of the source I can look at to work it out?
<deryck> awilkins, not sure what you mean by the transitions.
<deryck> awilkins, what are you wanting to understand?
<awilkins> deryck, Which state leads to which other state : I got an answer in the non-dev channel, apparently you can move to any state if you have the privileges
<awilkins> I just want to compare it to Bugzilla, etc
<deryck> awilkins, ah, ok.  yeah, there's no set order.
<jml> looking at MPs now
 * deryck lunches
<jml> benji, bac: have replied to MPs. am available for the next hour to chat.
<sinzui> rick_h_, jcsackett: do you have time to review https://code.launchpad.net/~sinzui/launchpad/progressive-enhancement-ftw/+merge/103733
<rick_h_> sinzui: sure thing
<rick_h_> sinzui: ping, ? for you
<sinzui> hi rick_h_
<rick_h_> sinzui: so you've wrapped the <a> contents with invisible-link css for many cases
<rick_h_> in a non-js browser situation then, is there any method for these to show up not hidden?
<rick_h_> since this is to replace no-script use cases
<sinzui> ah
<sinzui> invisible-link exists the Zope testbrowser, not humans
<sinzui> the sprite class instead renders an icon instead of text for the human
<rick_h_> ok, ic
<sinzui> rick_h_, there is a nuance we need to consider
<sinzui> linx is non-js and non-graphical, so crucial cases of reporting a bug must work without graphics or script. +filebug is such as case. We really want users to see those extra options. There are not edit icons on them
<rick_h_> ah, gotcha
<rick_h_> yea, it's the eternal debate of JS flash to hide things vs hiding by default
<rick_h_> sinzui: so with my icon fonts I've used this: http://pictos.cc/articles/using-icon-fonts/
<rick_h_> which helps make sure if you don't get the font you get the text
<rick_h_> I wonder if this could be adapted for a pure css solution
<sinzui> ah
<sinzui> I tried something like that a few years ago
<rick_h_> yea, I think part of this works because it's a font vs a sprite, but haven't tried it out
<sinzui> it was rejected as a programmer hack. I think we could get it accepted on the grounds that we care about accessibility.
<rick_h_> yea, it works pretty well ime in my side project and if we can praise it for accessability and lack of JS driven flashing UI it's a win
<sinzui> Anything that properly sites in the uncode ranges and that we can add beauty too via a web font would be grand
<sinzui> s/sites/sits
<rick_h_> yea, unfortunately I think most icon fonts aren't sticking to unicode land because of unidoce issues people end up with
<abentley> rick_h_: Could you please review https://code.launchpad.net/~abentley/launchpad/celery-everywhere-9/+merge/103723 ?
<rick_h_> abentley: will do
<rick_h_> sinzui: ok, commented, also wanted to bring up alt vs title text for these links since again we're really targeting accessability here
<sinzui> thank you.
<rick_h_> thank you, look forward to noscript dying
<sinzui> rick_h_, I already know that we cannot change the broken span in a div markup easilly
<sinzui> We tried this 18 months ago :(
<rick_h_> sinzui: ok, it looked like it might be a much bigger task, but wanted to note it
<sinzui> It is quite complicated. I can be done, but that is not the issue that needs to be solved in this branch
<abentley> gary_poster: ISTM that Storm does not maintain a mapping of table names to classes.  Does that seem correct to you?
<rick_h_> sinzui: understand
<sinzui> rick_h_. we stopped adding inline editing to the project page because th divs did bad things in element that had to be inline
<gary_poster> abentley, it's been a long time since I looked at that code, but I believe you are correct.
<rick_h_> sinzui: can the reverse happen though, can the spans be turned to divs with inline or inline-block styles?
<sinzui> No.
<rick_h_> sinzui: ok
<sinzui> Lp and HTML in general wants form elements to work in line
<abentley> gary_poster: I had always assumed there would be one.  I also thought having two classes with the same table would be impossible.  But if Storm can't tell that there are two classes with the same table, it shouldn't care, either.
<gary_poster> heh
<sinzui> rick_h_, consider that div can never go in a paragraph
<rick_h_> sinzui: ah right ok.
<sinzui> rick_h_, you are right about the title, I see I was not consistent
<sinzui> I will work on this now
<rick_h_> sinzui: np, I'll be around laterish today and can ok any time. Thanks for looking into it.
<gary_poster> abentley, it must reliably be able to determine what class to use to instantiate records from the db; looking for a sec; suspect that this is configuration
<abentley> gary_poster: There's a complicated derivation dance used in most Jobs classes; but if BranchScanJob, BranchUpgradeJob etc all were ORM objects with the same table, life would be simpler.
<abentley> gary_poster: I think that it relies on the caller of Store.fine to specify what class to use, and class specifies table.
<abentley> s/Store.fine/Store.find
<gary_poster> abentley, yes, I think you are right!  A very interesting idea
<gary_poster> Crazy idea abentley: you could maybe even pass a (single) factory, depending on how loose the contract is for the first argument to store.find
<gary_poster> if it just has to be a callable with a __storm_table__ ...
<gary_poster> you could create different objects depending on one or more columns
<gary_poster> May or may not be a horrible hack :-)
<gary_poster> __new__ might also lead down interesting, horrible paths
<gary_poster> if the contract is tighter than "callable + __storm_table__"
<abentley> gary_poster: So, Store.find does appear to defer to the caller: http://pastebin.ubuntu.com/947827/
<gary_poster> cool
<abentley> gary_poster: Unfortunately, my current work kinda assumed that I could serialize an IRunnableJob by writing out its table and id.
<gary_poster> ah
<deryck> jcsackett or rick_h_ -- I've got a branch killing doctests for review.
<rick_h_> deryck: cool, almost done with abentley's stuff and I can get right onto that
<deryck> rick_h_, cool, thanks!  It's at: https://code.launchpad.net/~deryck/launchpad/refactor-editemail-doctest-363916/+merge/103718
<rick_h_> deryck: ok, loading up now. Man, where's that achievement system from jono. I want a reviews badge for today lol
<deryck> heh
<rick_h_> deryck: #2 is just for my own curiosity as searching for doc gets me a ton of stuff
<deryck> rick_h_, so #1 -- I can underbar the create methods. I only did the assert one differently, so it wouldn't be mistaken for any self.assertXXX methods provided by the test case classes....
<deryck> rick_h_, #2 is due to the error message being an instance in one case, and unicode in all the others.
<deryck> rick_h_, #2 cont. I think some of the form magic calls doc on the widget to get the error string.
 * deryck is re-reading #3 now :)
<rick_h_> deryck: ok yea wasn't sure what .doc came from that seemed to imply it converted the input to unicode
<rick_h_> deryck: I've been confused on #1 before as usually personally I'd _private things that weren't tests but were used but then again things like setUp/tearDown and all that aren't as well so I guess it goes either way
<rick_h_> deryck: #3 just says that each unit test should start with a comment about what's being tested according to the testing style guidelines
<lifeless> tearDown shouldn't be used :)
<deryck> rick_h_, yeah, that's why I don't normally underbar something unless I need to signal it is somehow different.
<rick_h_> and wasn't in this case for the record, just comparing the names of non test_methods
<deryck> rick_h_, but if you want me to change either for consistency, I don't mind.
<deryck> hi lifeless  :)
<rick_h_> right, this only struct me because it did both, I see your point though now deryck. Up to you there
<lifeless> deryck: o/
<deryck> rick_h_, so can you point me at a line in that wiki your referring to. We're inconsistent about comments in unit tests, and I followed the example of the other tests in that file.
<rick_h_> deryck: under "Python test cases" #3
<deryck> rick_h_, ah, see it now thanks.  I can update the other tests in this file then too.
<deryck> rick_h_, and you know you're a new reviewer. ;)  That's a lot of wiki doc to read thoroughly. :)
<lifeless> I think that line is too proscriptive; 'test intent should be understandable' is a much better rule
<rick_h_> deryck: so jcsackett says I'm going to be set free for tues
<rick_h_> lifeless: come on, I'm a fan of the comment string for a nice readable line in the error reporting :P
<rick_h_> don't rain on my parade
<lifeless> rick_h_: in the runner output? All my runners disable that as an antifeature
<rick_h_> lifeless: yea
<deryck> I feel like we should have something akin to "You might be a redneck" jokes sometimes....
<deryck> "You might have a crazy test system if...."
<rick_h_> hah
<deryck> it requires multiple huge wiki docs to explain how to write good tests.
<deryck> :)
<rick_h_> well test names can be a bit shorter if you have a nice sentence on wtf just blew up in that first comment line. I like that as someone that blows up other peoples testes a lot lol
<deryck> I really don't mind adding a comment.  Just kidding around.
<rick_h_> I'm just sticking up for my anti features :)
<sinzui> rick_h_,  I replied and pushed a change to verify that widget-position-ext is not needed: https://code.launchpad.net/~sinzui/launchpad/progressive-enhancement-ftw/+merge/103733
<rick_h_> sinzui: saw that, so the two items are the alt tags and the hidden by default text of the links right?
<rick_h_> did you want to worry about the hidden text bits as a seperate branch/item?
<rick_h_> and I can approve it as is with the understanding the alt text can get added in?
<sinzui> rick_h_, I updated my reply. alt tags never EVER worked on anchors
<sinzui> rick_h_, title is universal and the only true mechanism to use in the 2000's
<rick_h_> sinzui: ah ok, sorry I was looking at examples at the start wich are tests anyway
<sinzui> :) That is my trick
<rick_h_> sinzui: thanks for the updated comment. Marked ok
<sinzui> fab.
 * sinzui reports bug to remove widget-position-ext source-dep
* rick_h_ changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: jcsackett | Firefighting: - | Critical bugs: 3.47*10^2
<sinzui> rick_h_, the http://pictos.cc/font/ is really great. I want to build something with it
<abentley> jcsackett: Could you please review https://code.launchpad.net/~abentley/launchpad/efficient-universal-source/+merge/103768 ?
<jcsackett> abentley: it's in my queue. might be a bit, it's not a short queue. :-)
<abentley> jcsackett: That's a second one.
<jcsackett> abentley: got it, just letting you know i'll get to it asap.
<abentley> jcsackett: Cool.
<rick_h_> sinzui: so I changed over to an open source version: http://fortawesome.github.com/Font-Awesome/
<rick_h_> sinzui: I paid for pictos though
<sinzui> That is nice too
<rick_h_> yea, only reason I switched was license, but yea it's nice to use the font idea like that
<jcsackett> abentley: r=me on that branch, in case you missed the email.
* jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 3.47*10^2
<abentley> jcsackett: thanks.
<StevenK> -            'text': '** Visibility changed to: Public',
<StevenK> +            'text': '** Information type changed from User Data to Public.',
#launchpad-dev 2012-04-27
<wallyworld_> wgrant: why did you remove the check for the bug created response at line 120?
<wgrant> wallyworld_: That was previously the only success verification
<wgrant> wallyworld_: Since it used /bugs to obtain the ID.
<wgrant> But now I check the body of the response, so the status code check isn't too important.
<wallyworld_> ok, thanks
<wgrant> wallyworld_: Thanks
<wallyworld_> np
<wallyworld_> lifeless: the new celery jobs infrastructure - if i were to write a new celery job, would there be any need to provide a db table? i would think not, since the celery/rabbit stuff would provide the necessary guaranteed delivery/execution?
<lifeless> wallyworld_: that aspect is unchanged to date.
<lifeless> wallyworld_: we don't run rabbit in HA mode, its only good for things we're willing to drop on the floor in times of failure.
<wallyworld_> lifeless: ok, so i need to create the db table etc
<lifeless> yes
<wallyworld_> just wanted to check, thanks. i figure new stuff should be done using celery
<lifeless> it should
<wallyworld_> thanks
<lifeless> whats the bug # for 'apport bugs generate multiple notifications'
<nigelb> 424849?
<nigelb> bah. that one's fixed.
<lifeless> possible 667604
<lifeless> hah 274537 should be dead easy
<lifeless> wgrant: I have a suspicion you fixed 626542 ?
 * wallyworld_ off to see doctor. bbiab
<wgrant> lifeless: Not sure. All of that stuff sort of blurs together.
<lifeless> wgrant: any idea on the bug # I'm looking for ?
<lifeless> before I wont-fix all of LP ?
<StevenK> lifeless: That sounds a little drastic?
<StevenK> "BAH, I can't find the dinner plate I want, so I'm going to burn the house down."
<wgrant> lifeless: 667604 is the only one I know about
<wgrant> But that's about new bugs.
<wgrant> Bug 424849
<_mup_> Bug #424849: Launchpad should batch attachment notification emails <lp-bugs> <story-better-bug-notification> <story-better-notification-sending> <Launchpad itself:Fix Released by gmb> <apport (Ubuntu):Invalid> < https://launchpad.net/bugs/424849 >
<wgrant> Ah
<wgrant> The end of that bug reveals all
<wgrant> Bug #768997
<_mup_> Bug #768997: Batch multiple comments from the same person/same bug <feature> <Launchpad itself:Triaged> < https://launchpad.net/bugs/768997 >
<wgrant> lifeless: It's a dupe of that
<lifeless> wgrant: good call, I went past that and ignored it.
<lifeless> now, where is the damn one about mailing list subscriptions -> user terrible confusion
<lifeless> (that https://bugs.launchpad.net/launchpad/+bug/816373 is a dupe of)
<_mup_> Bug #816373: Launchpad sends duplicate bug change notifications <Launchpad itself:Triaged> < https://launchpad.net/bugs/816373 >
<StevenK> Bah, bug change
<lifeless> wgrant: fwiw I'm seeing nearly 3 second bug searches on LP
<lifeless> 423ms 	
<lifeless> SQL-main-slave: SELECT COUNT(*)
<lifeless> 681ms 	
<lifeless> SQL-main-slave: SELECT BugTaskFlat.bugtask
<lifeless> so, most of it elsewhere
<wgrant> Yeah, they're rarely below 2s now.
<wgrant> Partly because of slow COUNT(*)s, but it's mostly out of the DB Now
<wgrant> Yay, only major bugtaskflat timeouts were already a problem. Tag union searches, +subscribedbugs, +commentedbugs. But some obscure sort orders have regressed... I think most of them would be OK if we didn't have stupid COUNT(*).
<wgrant> But it seems I can sensibly denorm a few more cols onto BugTaskFlat to sort out sorts.
<StevenK> AssertionError: Name "+secrecy" is not registered as a view or navigation step for "Bug" on "bugs".
<StevenK> LIES!
<StevenK> Oh, it's on the bugtask
<StevenK> *stab*
<wgrant> Goddammit
<wgrant> I hate partner
<StevenK> Everyone does.
<wgrant> Not as much as me.
<StevenK> True.
<nigelb> You hate having a partner?
<nigelb> (j/k I know what he means)
<StevenK> nigelb: I love having a partner. I can't speak for wgrant. :-P
<nigelb> StevenK: Haha :)
<wgrant> Hah
<wgrant> Maybe nobody cares about searching for partner bugs :)
<nigelb> I'm very tempted to start launchpadmemes.tumblr.com in the footsteps of mozillamemes and webkitmemes
<nigelb> Though, I suspect all it will have are rage faces ;-)
<wgrant> I think it could be pretty well summed up by the original rageface :)
<nigelb> heh
<nigelb> For April fools, lp should at some point show rage face icon for ~launchpad-dev
<spm> you could have a yellow rubber duck, and a cowboy hat
<wgrant> webops have the monopoly on amusing icons
<spm> s/amusing/accurate/
<wgrant> True
<nigelb> heh
<hloeung> canonical-sysadmin has a pigeon icon
<nigelb> so we bribe spm? Or is he LOSA and not webops
<spm> nigelb: s/losa/webops/ basically
<spm> just a rename a while ago
<wgrant> Back in my day, canonical-sysadmins had the default icon!
<nigelb> Ah
<spm> losa was getting a trifle overloaded.
<wgrant> But indeed, it is a pigeon now. Odd
<wgrant> Aw
<wgrant> Launchpad/Landscape/ISDU1/CA/EA OSA? :)
<nigelb> heh
<spm> launchpad, landscape, ubuntu one, consumer apps, isd etc operational systems admin
<nigelb> admin ALL THE THINGS
<spm> wgrant: right
<hloeung> pes, engineering
<hloeung> pretty much A-Z OSAs
<spm> oh god. I forgot about them
<wgrant> Oh right, PES now too
<wgrant> Must need to take on some more of CS soon
<nigelb> See, there's enough material in here to make a meme site.
<nigelb> Hah, the pigeon reminds of @feral_pigeon
<spm> webops is better than losa too. WeBops and use a Cyndi Lauper song as our theme; or Wh00psies! etc
<nigelb> "We didn't start the fire" is a better theme song for ops teams ;-)
<spm> nigelb: expect when we did
<spm> except*
<wgrant> A theme song doesn't have to be truthful :)
<spm> a good mate has a long running joke: he fixes everything with fire. as one of the UQ Datacentres had a fire in it - on his watch (he was networks dude at the time).
<spm> I'd prefer "Burning Down The House" myself, anyway.
<nigelb> heh
<bigjools> spm: We Are The Cham^WWebops?
<spm> bigjools: more like We're Too Sexy for LP Developers, too sexy by faaaarrr
<bigjools> spm: now you're doing Mission Impossible
<spm> heh
<nigelb> haha, well played bigjools.
 * nigelb slow claps
<StevenK> wallyworld_: Are you back?
<lifeless> spm: we're too sexy for bops ?
<spm> I think my brain just abort retry ignored?
<wallyworld_> StevenK: yes
<wallyworld_> ah a review
<wallyworld_> StevenK: TestBugTaskInterestingActivity - that test looks a little out of place. where are the other similar tests for the bug activity stuff on _index? a doc test?
<StevenK> wallyworld_: Yes, a doctest. And I didn't really want to add to it.
<wallyworld_> StevenK: is it worth porting the doc test to a unit test? as it stands now, there's a bit here and a bit there if you know what i mean
<wallyworld_> the BugVisibilityChange class - that is only relevant if the show info type in ui feature flag is off right, and when turn on the ff for all, we delete that class?
<StevenK> wallyworld_: BugVisibilityChange and BugSecurityChange and related gubbins will all die when we drop the UI feature flag and switch to information_type fully.
<wallyworld_> cool, just wanted to check - should we put in a XXX so we can keep track?
<StevenK> I can mark the two classes as such, if you wish.
<wallyworld_> with bug-change.txt - i'm all for ripping it out so long as there's coverage elsewhere
<StevenK> wallyworld_: I'd love your opinion on that -- compare bug-change.txt to test_bugchange.py
<wallyworld_> ok, let me look
<wallyworld_> StevenK: perhpas if we raise a bug for the work to rip out the legacy stuff when the ff is on that would be good and we could put the relavant XXXs in as we find them
<wallyworld_> StevenK: ffs, there's also test_bugchanges - note the extra 's'
<StevenK> Haha
<StevenK> wallyworld_: To be fair then, I'd be putting in XXXs all over the place.
<wallyworld_> ok, lets' leave it then. so long as we make sure we eventually deleting everything required
<StevenK> wallyworld_: I like the idea of XXXs for BugVisibilityChange and BugSecurityChange -- pretty much everything else will fall out due to tests failing.
<wallyworld_> right. my fear was that those classes would hang around, not be used, and we'd forget them
<StevenK> wallyworld_: test_buchange.py doesn't exist, it's only test_bugchanges, sorry
<wallyworld_> StevenK: lib/lp/bugs/adaptors/test/test_bugchange
<wallyworld_> StevenK: so, about the fact that this branch introduces a new unit test but everything else is in a doc test (bug activity stuff). my personal preference is to be consistent otherwise it's too hard to keep track of what's where. so add the new test as a doc test or port the doc test to unit tests. do you share that view?
<wallyworld_> StevenK: with bug-change.txt vs test_bugchanges.py: they are similar for sure but bug-change.txt mainly exercises the addChange() API which is a lower level one and is invoked indirectly in test_bugchanges when those tests are run. so if addChange() is broken, so too will be test_bugchanges bug there is an argument that all the current tests are required since strictly speaking they do test sifferent aspects
<StevenK> wallyworld_: Right.
<wgrant> wallyworld_: If you have time once you're done with StevenK, could you please look over https://code.launchpad.net/~wgrant/launchpad/bug-899776/+merge/103806?
<wallyworld_> sure
<StevenK> wallyworld_: I can move that one unit test into xx-bug-activity.txt if you wish
<wallyworld_> StevenK: it would be great if you could because then everything is together
<wallyworld_> StevenK: and the test_bugchange, test_bug_changes, bug-change.txt can stay as is for nowe
<StevenK> wallyworld_: Aye
<wallyworld_> StevenK: r=me but with an additional comment about the bug-change.txt deletions
<StevenK> wallyworld_: I just remember the other reason I didn't want to use xx-bug-activity.txt
<StevenK> Ran 1 tests with 1 failures and 0 errors in 24.564 seconds.
<wallyworld_> it failed as is ot with your additions?
<StevenK> wallyworld_: No, look at the time!
<wallyworld_> yeah, slow
<wallyworld_> so are manu unit tests due to setup of db
<StevenK> wallyworld_: I'll revert the drops in bug-change.txt with a little sadness.
<wallyworld_> StevenK: i share the sadness but we would be removing test coverage
<wallyworld_> you will need to wrap with a ff check as well
<StevenK> I will? It won't be set, they'll be fine unchanged.
<wallyworld_> StevenK: ah yes sorry. the classes are constructed directly
<wgrant> wallyworld_: Thanks. There are some other existing tests, but they're mostly at the browser layer because apparently people hate doing things properly.
<wallyworld_> wgrant: np. my main worry was that we have the test coverage to ensure any regression is caught
<wgrant> Yeah
<wallyworld_> seems like we do hopefully
<wgrant> And my new test covers most interesting caes.
<adeuring> good morning
* adeuring changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: adeuring | Firefighting: - | Critical bugs: 3.47*10^2
<stub> One more benefit of moving to PG 9.1 is that the enum types no longer suck, and we can use meaningful terms instead of cryptic integers at the DB level with things like our partial indexes.
<wgrant> stub: Ah yeah, sorry, I just assume everyone knows those two
<stub> Oh, I'm used to that. Just seeing the index go past prompted me to confirm being able to add values to the enumerated type got into 9.1.
<stub> wgrant: I assume this new index needs to go live?
<angeloc> Hi guys, i'm writing some scripts for ubuntu accomplishments, until now i solved all my problems reading online api documentation, but I'm now in stuck
<angeloc> i have to find all packages uploaded by a person, can you suggest me the best way?
<lifeless> stub: yes it does :)
<stub> lifeless: the index?
<lifeless> yes
 * StevenK stabs gpg for being obtuse and terrible.
<wgrant> stub: Yes please. Don't think backups are done yet, though.
<stub> lifeless, wgrant: My network is just far to bad. webops will need to do it if you don't want to wait for my SEA -> Los Angeles link to stop sucking.
<stub> Got as far as typing 'scree' and waiting for the 'n' to come though :-/
<wgrant> stub: It's not urgent; the rev won't be deployed until Monday or so.
<stub> k. I'll stick it on my list and try later.
<wgrant> Thanks :)
<stub> Except my reminder list is 'in the cloud' and accessed via LA too...
 * stub grabs a pen
<stub> Looks like all of level 3 from my POV... did we break level 3?
<StevenK> stub: Haha, it's possible.
<stub> Is it sucking from AU too?
<StevenK> I've not seen any problems, what are you having trouble reaching?
<stub> StevenK: Starts happening Level 3 in Los Angeles. Probably the Singapore -> LA link or Level 3 in LA.
<rick_h_> bwuhahaha, https://launchpad.net/+combo/rev15149/?yui/yui/yui-min.js
<StevenK> rick_h_: Indeed!
<StevenK> rick_h_: I suggest you and deryck find some time to get the flag enabled on qas and then hit up every bit JS to see what breaks.
<StevenK> Actually, while I think of it.
<wgrant> We also probably need to get a reasonable subset loaded early.
<StevenK> Right.
<wgrant> stub: Thanks.
 * wgrant lands.
 * StevenK declares victory over gpg.
<czajkowski> StevenK: every small victory counts!
<StevenK> czajkowski: I've been fighting with it for a few hours spread over the last few days.
<StevenK> You'd think moving a private key to a different keyring is hard or something.
<jml> hey guys
<jml> so, uhh, I left a couple of Launchpad ec2 instances running somehow
* bac changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: adeuring, bac | Firefighting: - | Critical bugs: 3.47*10^2
<bac> morning adeuring
<adeuring> morning bac
<bac> hi beuno
<beuno> hiya bac!
<bac> adeuring: you're not reviewing ian's MP are you?  if not, i'll grab it now.
<bac> adeuring: nm, done.
<adeuring> bac: , right, I did had a look at the mp. (sorry for the late answer: had lunch...)
<adeuring> erm i did _not_ have a look at the mp
<bac> adeuring: perfect
<deryck> Morning, folks.
<adeuring> morning deryck
<angeloc> StevenK: here I am!
<StevenK> angeloc: Right, so as I said in #launchpad, the first step is to gather your requirements. What exactly do you want to search by, and what results do you want to get back. Once you have that, we can see.
<angeloc> i have to verify that a person has uploaded at least one package to assign him a trophy
<angeloc> StevenK: sorry for delay
<StevenK> angeloc: To a PPA? To the main Ubuntu archive? As the uploader itself, or is it okay if his upload was sponsored?
<angeloc> StevenK: sorry for not being explicit... as a first package uploaded, I can think of a sponsored one, I think I can create another tropy for people that has packages on their ppas
<StevenK> angeloc: Right, so I can't recall any method exported over the API that will do that. If you're not afraid of diving in and writing some Python, some people here will probably answer any questions you have. As for me, it's time for bed.
<angeloc> StevenK: thank you so much! Wich package I have to download to dig inside launchpad api implementation?
* bac changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: adeuring | Firefighting: - | Critical bugs: 3.47*10^2
* adeuring changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: 3.47*10^2
<czajkowski> gmb: photo walk plan ?
<gmb> czajkowski, Will email to you by Monday - bug me if I don't! Trying to find out if some of the suggested routes that I've found are actually a) sane b) interesting c) not liable to get us mugged.
<czajkowski> gmb: cool ask pleia2  if you like
<czajkowski> ahh you sent the mail to uds cool
<czajkowski> was working on that saves em time :)
<gmb> czajkowski, Ah, sorry, didn't realise I was duplicating effort. Oh well, you can relax now. Have a beer, knock off early... :P
<czajkowski> hehe
<czajkowski> :)
 * deryck lunches afk for a bit, to enroll the youngest in kindergarten.
<rick_h_> ooh, send the kids away!
<abentley> bac: Are you OCR?
<abentley> rick_h_: There's no OCR.  Up for a review?
<rick_h_> abentley: aure thing
<abentley> rick_h_: Thanks.  https://code.launchpad.net/~abentley/launchpad/remove-create-merge-proposal-job/+merge/103929
<rick_h_> np
<abentley> rick_h_: It is long, but 90% deletion.
<rick_h_> abentley: what's this --\x20 in line 619 of the diff?
<rick_h_> some sort of unicode foobar there with the mp maybe?
<abentley> rick_h_: that's a lint fix.  It's the encoding for a space.  Having an actual space caused a lint complaint.
<abentley> rick_h_: Because we don't like having trailing whitespace.
<rick_h_> abentley: ah, ok that's inside of some message body
<rick_h_> abentley: thanks
<abentley> rick_h_: np.  Should have mentioned that in the description.
<rick_h_> woot! no more code* my poor chrome auto complete is so confused
<SpamapS> Hrm, I need the opposite of source_package.setBranch .. I need to delete the official branch pointer
<cjwatson> SpamapS: .setBranch(None)
<SpamapS> That easy?
<SpamapS> Why didn't I think of that. :)
<cjwatson> well, branch=None obv.
<cjwatson> according to the code, yes
<cjwatson>         else:
<cjwatson>             # Delete the official DSP if there is no publishing history.
<cjwatson>             self.distribution_sourcepackage.delete()
<cjwatson> maybe try it on staging first
<SpamapS> this is for the 'charms' distribution :)
<SpamapS> ERROR:can't determine Launchpad branch from bzr branch
<SpamapS> oh wait
<SpamapS> thats my problem..
<SpamapS> cjwatson: first test did not result in its removal... Hrm
<SpamapS> is there a way to get launchpadlib to spew out what it is doing? Some kind of logging decorator?
<cjwatson> import httplib2; httplib2.debuglevel = 1
<SpamapS> man
<cjwatson> but that may not help if I made a mistake in interpreting what's happening on the server side
<SpamapS> I *just* fell far enough down the well to find that in pydoc :)
<SpamapS> yeah I just want to see that it sends None :)
<SpamapS> pocket=Release&ws.op=setBranch&branch=null
<SpamapS> Yeah, so I get a 200 OK with that, but it doesn't seem to actually be deleting the source package record
<cjwatson> OK, maybe I'm wrong then
<SpamapS> cjwatson: I'm looking at the same code.. but I haven't gone deeper to see what .delete() does
#launchpad-dev 2012-04-28
<SpamapS> interesting
<SpamapS> getBranch() does in fact show a cleared branch
<SpamapS> aha!
<SpamapS> it did work
<SpamapS> curse you caching!
<Laney> How should I be going about setting up lp on precise? launchpad-developer-dependencies isn't installable there so rocketfuel-setup barfs out.
<lifeless> Laney: it should be, it may be uninstallable temporarily
<lifeless> Laney: generally I recommend folk use a lucid lxc container to do LP dev
<lifeless> since it deploys on Lucid, this avoids a bunch of friction.
<lifeless> In the next few months we'll migrate to deployed-on-precise, when this will change to recommending a precise lxc container :)
<Laney> yeah, my container got borked due to some lxc bug so I recreated a precise one to see how it went
<Laney> I think I hacked around the uninstallability now, will see how it goes
#launchpad-dev 2012-04-29
<Laney> yeah I gave up and went back to a lucid container :P
<lifeless> :P
<wgrant> Things seem to be a lot smoother with a fresh late-Precise-generated Lucid container
<alexh> hi
<jelmer> hi alexh
<alexh> any hints on why the bug activity API returns an empty entries set but a non-zero total_size?
<alexh> e.g. https://api.launchpad.net/1.0/bugs/123920/activity
<_mup_> Bug #123920: Bluetooth Logitech Dinovo Keyboard/Mouse don't work <cft-2.6.27> <iso-testing> <linux-source-2.6.22> <metabug> <patch> <Bluez-utils:New> <bluez (Ubuntu):Triaged> <linux (Ubuntu):Invalid> <linux-source-2.6.22 (Ubuntu):Won't Fix> <udev (Ubuntu):Invalid> < https://launchpad.net/bugs/123920 >
<alexh> happens with every bug I've tried, not just that one, fwiw
<Laney> Is there a make harness that lets me do doctest stuff?
<Laney> (or, what do I import to get that stuff?)
<lifeless> Laney: what do you mean?
<Laney> I'm trying to make a doctest and it'd be good to be able to test this interactively
<Laney> but I don't know what to import into the harness repl to get those functions
<Laney> it doesn't really matter; I'm feeling it out without this.
<lifeless> ah
<lifeless> so we consider doctests one step up from the spawn of evil
<lifeless> you might like to not make one
<Laney> yeah?
<Laney> I thought they were the way to test UI changes
<lifeless> yah
<lifeless> very much deprecated
<lifeless> having testable *documentation* is still good, and if you are writing docs you expect folk to read, feel free to make them a doctest
<lifeless> but such are not replacements for unit tests or integration tests
<cjwatson> Laney: unfortunately the codebase is not always a good indicator of modern practice :-/
<cjwatson> another approach is: make your test fast enough that instead of running things interactively in a harness, you can just repeatedly run the test
<Laney> cjwatson: I don't know how to test the UI then :P (unless I am misusing the terminology and interactive tests are something different from doctests)
<Laney> the reason I want to use the harness is because I don't really have a view of the test interfaces yet so it's most efficient to just be able to bash on stuff
<Laney> fyi, the file I am proposing to edit is lib/lp/soyuz/stories/soyuz/xx-distributionsourcepackagerelease-pages.txt
<cjwatson> doctests are noninteractive
<Laney> the repl-style tests I mean
<cjwatson> well, anything you can do in that can be recast as a unit test fairly straightforwardly
<cjwatson> just a matter of learning the various self.assert* helpers
<Laney> indeed. I just thought these were the way that UI tests were done.
<lifeless> so a UI test could mean:
<lifeless>  - JS tests
<lifeless>  - html content checking
<lifeless>  - non-ajax form checking
<lifeless>  - ajax integration test
<lifeless> perhaps a better question is, what change are you making ?
<lifeless> also, morning all
<lifeless> bbiab, shopping run
<mwhudson> lifeless: did you see http://cortesi.github.com/pathod/
<lifeless> no, looking
<lifeless> nice
<lifeless> fault injection test server. woot.
<lifeless> nice - 200@100k,ascii_letters
<wgrant> alexh: You're unauthenticated.
#launchpad-dev 2013-04-22
<StevenK> wgrant: margin:auto doesn't seem like a magic bullet. text-align:center on the container div and display:inline-block on the calender div just results in the widget looking squashed and still on the left of the overlay
<StevenK> wallyworld: ^ Can you give me some clues about the above?
<wallyworld> StevenK: sadly my css foo is not that good
<wallyworld> curtis is your best bet
<StevenK> wallyworld: Fair enough, I might hit up a few friends of mine
<wallyworld> sorry
<nigelb> StevenK: Is whatever you're trying to fix, online?
<nigelb> I mean, can I see it?
<StevenK> Not really
<StevenK> It's a LP branch, but it requires you have a dev environment
<nigelb> argh. Mine's broken for now :/
<wgrant> StevenK: Are your latest changes pushed?
<StevenK> wgrant: Nope
<wgrant> Please do so :)
<StevenK> wgrant: I have it working now anyway
<wgrant> I will poke
<wgrant> Ah, good
<wgrant> What'd you need to change?
<StevenK> I needed to apply 'margin: 0 auto' to .yui3-calendarbase
<wgrant> Ah
<wgrant> Screenshot?
<wgrant> What remains to be fixed? Wiring up the selection events properly?
<wgrant> Which is not CSS, so it's trivial :)
<StevenK> No, done
<StevenK> wgrant: http://wedontsleep.org/~steven/new-calendar-4.jpg
<wgrant> StevenK: So it writes the correct format to the textbox, and only when the tick is clicked?
<StevenK> wgrant: Yeah, it will only close the overlay when you click the tick
<StevenK> In the case of showing time as well
<wgrant> Right.
<StevenK> wgrant: If it looks good, I can push it up
<wgrant> StevenK: Sounds good
<wgrant> StevenK: Were you going to integrate that CSS?
<wgrant> Into combo.css
<StevenK> wgrant: Well, yeah
<StevenK> It gets pulled into combo.css as part of combine-css
<wgrant> StevenK: Then why'd is referenced in launchpad-widget-macros.pt?
<StevenK> Oh
<StevenK> Right
<StevenK> Sorry, I thought you meant the other bits I changed in css/modifiers.css
<wgrant> Ah, no
<StevenK> wgrant: So, I can. Not a great deal of pages use the calendar
<wgrant> StevenK: Sure, but it's pretty small and it sucks to have the four extra requests like that.
<wgrant> StevenK: Also, you've split the .inline and .hidden rules, but the order between them matters, so they should ideally always be together.
<wgrant> And those skin overrides will want a comment detailing their motivation.
<wgrant> StevenK: Do you also want to destroy the overlay and calendar when the overlay is hidden?
<wgrant> At present you only destroy them on cancel
<StevenK> Hmm, I should refactor that out
 * StevenK stabs combine-css for not including the calendar css
<StevenK> combine-css really dislikes yui/assets/skins/sam/calendar-base.css
<wgrant> Howso?
<StevenK> wgrant: http://pastebin.ubuntu.com/5591677/
<wgrant> StevenK: What if you include the -core and -skin separately?
<wgrant> It may not like the pre-minfied version.
<StevenK> combine-css really dislikes yui/assets/skins/sam/calendar-base.css rather than yui/assets/skins/sam/calendar-base.css works
<StevenK> Bleh
<StevenK> Including yui/calendar/assets/calendar-core.css, yui/calendar/assets/skins/sam/calendar.css and yui/calendar/assets/skins/sam/calendar-skin.css rather than yui/assets/skins/sam/calendar-base.css works
<wgrant> StevenK: Right, that's what I meant.
<wgrant> I wonder if it's a bug in YUI's minification, or a bug in cssutils.
<StevenK> Except I need calendar-base too
<wgrant> Oh
<wgrant> Misread
<StevenK> xml.dom.SyntaxErr: PropertyValue: No match: ('CHAR', u':', 2, 588)
<wgrant> That calendar.css should be redundant with calendar-skin.css and calendar-core.css
<wgrant> foo-core.css + foo-skin.css = foo.css
<wgrant> So you want to include calendar.css, calendar-base-core.css, calendar-base.skin.css
<StevenK> wgrant: http://pastebin.ubuntu.com/5591691/
<wgrant> Try that
<wgrant> +    'yui/calendar/assets/calendar-core.css',
<wgrant> +    'yui/calendar/assets/skins/sam/calendar.css',
<wgrant> +    'yui/calendar/assets/skins/sam/calendar-skin.css',
<wgrant> +    'yui/assets/skins/sam/calendar.css',
<wgrant> 2 and 4 are the same
<wgrant> And 2 and 4 both include 1 and 3
<wgrant> calendar and calendar-base are separate components
<wgrant> They each have -skin.css, -core.css and .css files
<wgrant> The .css file (without -skin or -core) is the combination of -skin and -core.
<wgrant> Minified.
<StevenK> wgrant: However, now combine-css croaks, so one of the unminified files causes the same thing
<wgrant> StevenK: Hopefully you can narrow it down.
<StevenK> Just doing so
<StevenK> Both yui/calendar-base/assets/skins/sam/calendar-base.css and yui/calendar-base/assets/skins/sam/calendar-base-skin.css
<wgrant> StevenK: Remember that one of those files is a superset of the other.
<StevenK> Oh, -skin is unneeded
<wgrant> No
<wgrant> -skin is the source
<wgrant> calendar-base-skin.css is a subset of calendar-base.css
<wgrant> So if you want to continue isolating, the next step is to drop to just the -skin
<StevenK> xml.dom.SyntaxErr: PropertyValue: No match: ('CHAR', u':', 12, 19)
<StevenK> wgrant: Line 12 of yui/calendar-base/assets/skins/sam/calendar-base-skin.css
<wgrant> That's what I guessed from looking at the file.
<wgrant> StevenK: btw, if you want to purge yui 3.5.1, it's probably safe now
<wgrant> StevenK: http://mail.python.org/pipermail/python-announce-list/2009-August/007679.html
<wgrant> Search for DXImage
<wgrant> >>> parser.parseString(".yui3-skin-sam .yui3-calendar-content {\nfilter: progid:DXImageTransform.Microsoft.gradient( startColorstr='#f9f9f9', endColorstr='#f2f2f2',GradientType=0 ); /* IE6-9 */\n}").cssText
<StevenK> I see that
<wgrant> WARNING	Property: Unknown Property name. [2:1: filter]
<wgrant> Seems to work with that enabled :)
<wgrant> '.yui3-skin-sam .yui3-calendar-content {\n    filter: progid:DXImageTransform.Microsoft.gradient(startColorstr="#f9f9f9", endColorstr="#f2f2f2", GradientType=0);\n    /* IE6-9 */\n    }'
<StevenK> wgrant: I need the .set line?
<wgrant> StevenK: Yes, before you instantiate the CSSParser
<wgrant> Or you may be able to set it on the CSSParser itself
<wgrant> Not sure
<wgrant> (we are using the latest cssutils as of last week, btw, so an upgrade won't help)
<StevenK> wgrant: http://pastebin.ubuntu.com/5591715/
<wgrant> StevenK: Looks reasonable. Push and I'll rereview?
<StevenK> Oh, wait, I can go back to the original CSS
<StevenK> wgrant: Sorry, was on the phone since :08
<StevenK> I'll check if the original CSS works
<wgrant> StevenK: good point
<StevenK> It does
<StevenK> wgrant: Changes pushed
<StevenK> wgrant: And diff updated
<wgrant> StevenK: + 'yui/calendar/assets/skins/sam/calendar.css',
<wgrant> Why aren't you using the one in yui/assets/skins/sam like the others?
<StevenK> wgrant: Good point
<StevenK> wgrant: Pushed that too
<StevenK> wgrant: Diff updated
<StevenK> wgrant: We no longer build 3.5.1, or do you want it dropped from sourcedeps?
<StevenK> % grep '^YUI_VER' Makefile
<StevenK> YUI_VERSIONS := 3.9.1
<wgrant> StevenK: r=me
<wgrant> Oh, did you already do that?
<wgrant> I forgot
<wgrant> So you did
<wgrant> :)
<StevenK> r16573
<StevenK> % bzr di | diffstat -s
<StevenK>  80 files changed, 1 insertion(+), 26238 deletions(-)
<StevenK> That felt good.
<wgrant> :)
<StevenK> wgrant: The checkwatches cronspam was python warnings?
<wgrant> StevenK: Yeah
<wgrant> There's also one Trac instance that returns truncated lines
<wgrant> But the main spam was from the RT deprecation warnings
<StevenK> Truncated lines is just awesome
<wgrant> That was the third most common avoidable cronspam on launchpad-error-reports, behind the process-upload encoding issues (which I fixed last week) and the process-upload build for superseded source exceptions (which are as-yet unfixed, but tolerable)
<StevenK> So it's still a firehose?
<wgrant> Much much better now that the process-upload encoding issue is fixed
<wgrant> StevenK: I wouldn't land the YUI2 removal until the new calendar is deployed and we're happy with it
<wgrant> Just in case we need to revert
<StevenK> Sure
<StevenK> But I'd thought I'd put it up
<StevenK> Mostly so I can gloat
<wgrant> Heh
#launchpad-dev 2013-04-23
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/reset-session-on-full-restore/+merge/160265
<wgrant> StevenK: Oh, does staging still use stagingsetup?
<StevenK> Yeah
<adeuring> good morning
#launchpad-dev 2013-04-24
<StevenK> iharness loves me not: ImportError: No module named ipapi
<wgrant> StevenK: I may have broken that last week
<wgrant> Yeah, looks like it
<wgrant> StevenK: iharness is fixed
<StevenK> wgrant: Oh, your upgrade carnage caused it?
<wgrant> Yes
<wgrant> IPython 0.11 changed the API that we used
#launchpad-dev 2013-04-26
 * StevenK stabs BulkUpdate
<StevenK> wgrant: http://pastebin.ubuntu.com/5603182/ First bit is the query I wrote, second bit is my attempt to convert it into BulkUpdate, third bit is what BulkUpdate thinks of it
<mwhudson> StevenK: what do you expect dict([PreviewDiff.merge_proposal_id, BranchMergeProposal.id]) to do?
<StevenK> I was wanting that bit to expand to the SET bit in the query, but I'm not clear on what BulkUpdate actually wants there.
<wgrant> That's not valid dict syntax
<wgrant> dict() takes a sequence of pairs
<wgrant> IIRC that wants to be a mapping of columns to expressions
<wgrant> So dict([(PreviewDiff.merge_proposal_id, BranchMergeProposal.id)]), perhaps
<wgrant> But it's been a while since I wrote BulkUpdate
<wgrant> StevenK: And you might need to use values= to include BMP
<wgrant> And you want table=PreviewDiff, not table=BranchMergeProposal
<StevenK> wgrant: Yeah, I've just fixed it
<StevenK> Worked those bits out myself
<StevenK> But thanks
<StevenK> The query it's generating looks good
<wgrant> StevenK: r16467
<wgrant> This is similar to my BFJ flattening stuff
<wgrant> But if you've already got it working..
<StevenK> I have a permission error, but what it's excuting from the query log looks great
<StevenK> wgrant: So I think I might land the YUI2 destruction branch
<wgrant> StevenK: Sounds good
<StevenK> wgrant: So my garbo job works, now you want me to keep going and drop BMP.merge_diff before proposing?
<wgrant> StevenK: There's a sequence of branches that end with the dropping of BMP.merge_diff. I wouldn't land any of them until you've got the last branch working, so you don't land anything that won't actually work.
<wgrant> StevenK: https://lpstats.canonical.com/graphs/CodeImports/20130426/20130427/ is what all that codeimport spam was about, btw
<StevenK> Hmmm
<StevenK> How did this code ever work
<nigelb> StevenK: Second phase of debugging.
#launchpad-dev 2014-04-22
<cjwatson> wgrant: Is there a particular value in the approach of having binarypackagefile_id_seq rather than just having BinaryPackageFile.id be SERIAL PRIMARY KEY?  I'm wondering what I should be doing for the analogous LiveFSFile.
 * cjwatson wonders how anyone ever gets database/schema/security.cfg right short of sheer guesswork
<cjwatson> configure.zcml mistakes really are capable of generating ginormous tracebacks
<wgrant> cjwatson: That's the implicit sequence created by serial PRIMARY KEY
<cjwatson> Ah, so it's just what pg_dump produces?
<wgrant> Right. serial is shorthand for creating an integer column bound to a sequence.
<cjwatson> All right, that makes sense
<wgrant> But pg_dump can't tell that it was used initially.
<cjwatson> I'll probably be able to get at least some set of simple livefs tests working tomorrow.  But not tonight because argh Processor.
<cjwatson> (Spent most of today on it, up to 1100 loc or so ...)
 * cjwatson tries to figure out why status is apparently denormed across BuildFarmJob and some of the BFJ implementations
<cjwatson> Is the former deprecated or something?
#launchpad-dev 2014-04-23
<wgrant> cjwatson: BuildFarmJob mirrors relevant attributes of the implementations so there's one place to query for all types. The mixins handle that for you.
<wgrant> cjwatson: Any other issues with the liveFS arc?
<cjwatson> wgrant: Going pretty well at the moment.  I'm battling with traversal code.
<cjwatson> Still need to write behaviour tests.
<wgrant> Some complication with traversal, or just a part of the world you don't visit often?
<cjwatson> Just the latter.
<cjwatson> I think I should have something worth an initial review by the end of the week at this rate (I'm off this afternoon).
<cjwatson> Also still need to write webservice-y methods for grabbing files out of the LiveFSBuild.
<cjwatson> But (with the exception of whatever random traversal bug I have at the moment) I have a reasonable set of tests working, so over the hump.
<wgrant> Great.
<wgrant> Happy to help with the traversal bug, if you can't work it out.
<cjwatson> Is there a way to get pdb to break in a traverse method called in the back end of an AppServerLayer webservice test?
<cjwatson> It just hangs, so I'm obviously doing it wrong ...
<wgrant> cjwatson: Heh, are you trying to view /builders?
<cjwatson> No, /~owner/+livefs/blah
<wgrant> That's probably affected too
<wgrant> Hm.
<wgrant> Not in the existing implementations.
<cjwatson> Where blah is DISTRIBUTION/DISTROSERIES/LIVEFS so the traversal is a bit complicated to start with (I'm using a cut-down version of the branchnamespace thing)
<wgrant> But the testbrowser respects meta refresh, so loading /builders or /builder/whatever will hang by default.
<wgrant> Oh, you mean the pdb is hanging, not the test normally?
<cjwatson> Right, the test normally is failing with a too-many-redirections error at the moment and I'm trying to debug it
<wgrant> It's a subprocess in AppServerLayer, so that indeed won't work. It's probably easier to reproduce manually on the dev appserver.
<cjwatson> hm, ok
<cjwatson> or I guess I could do the FakeLaunchpadRequest thing and test just the traversal that way, come to think of it
<cjwatson> but yeah, I'll bring up a dev appserver if that fails
<wgrant> Right, you could do that, and I think we should have more tests like that, but for debugging it's easy to do it outside the test suite
<cjwatson> If I can figure out how to log in ...
<cjwatson> (a) make run only runs on port 8085, is that right? (b) +login says the openid provider is unavailable
<wgrant> make run listens on several ports, but you should always access it through apache on 443
<wgrant> If you don't have apache set up, +login will fail when it can't talk to https://testopenid.dev/
<cjwatson> That gives me an ssl_error_rx_record_too_long error
<wgrant> Sounds like you don't have mod_ssl enabled, or your LP vhost config is broken.
<cjwatson> Oh, huh, no vhost
<wgrant> If you're using LXC, make sure you've run "make LISTEN_ADDRESS=* install" and then restarted apache
<cjwatson> Much better, thanks
<cjwatson> No luck getting lp-shell to get an oauth token though.  Trying the more lightweight test approach
<wgrant> Hm, should work fine with LP_DISABLE_SSL_CERTIFICATE_VALIDATIOn
<cjwatson> (I copied and pasted the URL into firefox on my host, but the token doesn't seem to be recognised inside the LXC guest)
<cjwatson> Yeah, I used that
<wgrant> I usually run lp-shell on the host, but it shouldn't matter.
<cjwatson> That seemed to get a bit further, but it got an ECONNREFUSED trying to send mail
<wgrant> Ah yes
<cjwatson> http://paste.ubuntu.com/7314405/
<wgrant> That will break unless you have an MTA installed
<wgrant> Install a local-only postfix in the container and restart lp-shell
<cjwatson> Ah, brilliant, that works now
<cjwatson> Just in time for me to EOD - will investigate more tomorrow
<wgrant> :)
#launchpad-dev 2014-04-24
<cjwatson> wgrant: traversal> turns out if I stop issuing pointless redirections then I don't get into a redirect loop.  amazing
<wgrant> cjwatson: Mysterious.
 * cjwatson wonders what subset of pending_builds/completed_builds/builds/last_build from sourcepackagerecipe is useful here
<cjwatson> maybe I just want builds and last_completed_build and I can add anything else as needed later
<wgrant> That sounds sensible.
<xnox> How to run "view smoke test" story selectively ?
<xnox> i want to execute ./lib/lp/services/oauth/stories/request-token.txt for example
<xnox> ./bin/test -vvct request-token
<xnox> worked
<xnox> \o/
<cjwatson> Urgh.  I have an AppServerLayer test where I need to change things directly in the database and then make a webservice request.  I'm calling transaction.commit(), but that doesn't appear to be enough for the webservice side to pick up the database changes.  Is there something else I need to flush?
<cjwatson> Oh.  Maybe ws_object isn't actually re-fetching the object
<cjwatson> aha, it's failing to do so because traversing one of the links in the returned document isn't working.  Right, making sense now
<wgrant> cjwatson: Is there a particular reason that you're using ws_object, which is launchpadlib-based and extraordinarily slow and awkward, rather than webservice_for_person, which is plain JSON, fast, and in-process?
<cjwatson> Because I spent a couple of hours trying to convert it to webservice_for_person, got stuck in interaction hell, and decided it was more productive to at least get something that works
<wgrant> Heh
<wgrant> You mostly just have to logout() before using any of the webservice object's methods.
<cjwatson> Yeah, and extract everything you need first
<cjwatson> I'll try the conversion again later
<wgrant> webservice_for_person does the logout, IIRC, then I normally use 'with admin_logged_in()' to pull stuff out later.
<cjwatson> But I wanted to get on with making this batch of tests at least minimally work so that I can move on to the behaviour tests
#launchpad-dev 2014-04-25
<cjwatson> wgrant: I'd like to create a launchpad-livefs-builders (or similar) team so that we can feature-flag livefs configuration/building to only be allowed for that team; that seems like the simplest way to avoid world+dog eating a ton of build time.  I guess you'd need to create that in order to have it under launchpad-leader?
<wgrant> cjwatson: Or ~launchpad could own it. It doesn't seem like it needs to be ultra-restricted.
<cjwatson> Mm, that would work.  Indeed, I was basically planning to put ~launchpad-buildd-admins and ~ubuntu-archive in it, or something to that effect
<wgrant> That would sound sensible.
<wgrant> (also, one can transfer a team to anyone, so you could create it either way)
<cjwatson> Oh true, duh
<wgrant> I tend to believe that ~launchpad-leader should own the sensitive privileged teams, but ~launchpad-livefs-builders is just to prevent abuse of resources.
<wgrant> AIUI
<cjwatson> Yep
<cjwatson> Created
<wgrant> cjwatson: Everything else going OK?
<cjwatson> I think so, yeah - I need to write a garbo job, and I need to go back and refactor those livefs tests, but the rest looks like it might be dangerously close to done
<cjwatson> er
<cjwatson> those webservice tests, I mean
<wgrant> :)
<cjwatson> oh and I still need to write the webservice-flavoured file download methods; that's the only other todo item I have left
<cjwatson> wgrant: Is the latest buildbot failure a known transient problem?
<cjwatson> http://lpbuildbot.canonical.com/builders/lucid_lp_lxc/builds/1149/steps/shell_9/logs/summary
<cjwatson> I'll retry on general principles ...
<wgrant> cjwatson: Yeah, that's our most frequent spurious failure atm.
<cjwatson> OK
#launchpad-dev 2014-04-26
<caryhartline> Has there been any discussion of branching out the Launchpad website such as a responsive or mobile site or as an app on Ubuntu, iOS, Android?
<jelmer> caryhartline: I haven't seen anything
<caryhartline> The mobile apps may be a massive undertaking, but retrofitting the website into something that is usable on a phone may be a good starting point.
<caryhartline> Although I'm surprised this has not been discussed before.  I'm not sure if enough people would be enthusiastic enough to start working on this.
<wgrant> caryhartline: We'd happily accept patches for that, but it's not something that's a priority for the Launchpad team to work on in the immediate future.
#launchpad-dev 2015-04-21
 * cjwatson starts trying to saw up his Git MP work from the plane into manageable pieces
<cjwatson> blr: You probably didn't see, but I got MPs working on Saturday evening \o/
<cjwatson> (minimally, anyway)
<cjwatson> wgrant,blr: Are we holding the team meeting at the calendared time tonight?
<wgrant> That time's a bit awkward for me this week but I guess we might be able to get away without it.
<cjwatson> It's exactly as awkward for me as it normally is :-)
<wgrant> Heh, true.
<cjwatson> How goes the release sprint?
<wgrant> The usual amusing crop of installer bugs.
<wgrant> Except all with added systemd fun this time.
<wgrant> eg. oem-config breaking because of a shadow bug from two years ago because systemd.
<wgrant> Only two last-minute sabdfl changes so far.
<wgrant> cjwatson: Are you comfortable enough with your MP changes to make schema review sensible at this stage?
<cjwatson> wgrant: Yeah, my threshold was "works locally", which it does.
<wgrant> Great, will look soon
<cjwatson> Do you think I need to ask thumper etc. if they're OK with removing BMQs?
<cjwatson> Or just assume that if they haven't finished it in the last 5+ years they're unlikely to now?
<wgrant> Meh.
<wgrant> They're unused, and have been disabled for like five years.
<wgrant> If someone wants to revive them, they can feel free.
 * cjwatson nods
<wgrant> VCSes are cool like that.
<cjwatson> :-)
<cjwatson> Vanishing.
<wgrant> cjwatson: https://code.launchpad.net/~wgrant/launchpad/changelog-formatting/+merge/256898
<wgrant> Quality tests.
<cjwatson> wgrant: Nice, r=me.
<wgrant> Thanks, bugged me for five years or so...
<jpds>  
<wgrant> cjwatson: Not oversized? I'm disappointed.
<cjwatson> wgrant: Hm?
<wgrant> Your next git branch is just under 800 lines!
<cjwatson> Ah :-)
<cjwatson> I have another smallish one, but then need to think a bit ...
<wgrant> The whole diff's 4kish, right?
<cjwatson> 4060 right now, though I'll need to write some more tests.
<wgrant> cjwatson: boom, btw
<cjwatson> Grr.
<cjwatson> So much for having carefully run all the code tests and thought that was probably enough ...
<wgrant> Heh
<cjwatson> Ah, maybe I didn't apply the changed security.cfg first.
<cjwatson> I'll just revert security.cfg, not worth the hassle.
<wgrant> Yep
<cjwatson> Testfix landing.
<cjwatson> wgrant: Do you know why the existing Branch:+subscribe etc. permissions are the way they are?  I was copying from those.
<wgrant> cjwatson: Probably from before private branches.
<wgrant> The lack of a ViewAndAnyPerson-ish thing has always been annoying.
<cjwatson> What would the difference be between that and just View here, given that an anonymous user can't subscribe to things?
<cjwatson> Oh, but View would *allow* an anonymous user to subscribe.
<wgrant> Exactly.
<cjwatson> I guess it's sufficient to just rely on you not being able to get at the Branch/GitRepository to subscribe to it in the first place if you don't have View.
<cjwatson> The portlets could be launchpad.View at least, though.
<wgrant> Yeah, that's the way it works now, but it's a bit unfortunate.
<wgrant> Particularly given that +subscribe is the access control view.
<cjwatson> I'll put something in Asana for it.
<wgrant> cjwatson: Can I upgrade DF?
<cjwatson> Sure.
<wgrant> cjwatson: Have you timed the new check constraints?
<cjwatson> wgrant: I don't have the scrollback, but that was what I was doing during the wrap-up when I accidentally failed to do it in a transaction :-)
<cjwatson> wgrant: IIRC they were something like 0.2 seconds each
<wgrant> Oh,heh.
<wgrant> Right.
<wgrant> It's not a big table.
<cjwatson> Yeah, on DF I get 153.458 ms, 95.952 ms, 94.037 ms, 69.866 ms, 125.432 ms
<cjwatson> So the whole patch might end up being as much as a second or two but shouldn't be much worse.
<wgrant> cjwatson: Does different_git_refs really handle a null dependent_git_repository proprely?
<cjwatson> wgrant: Should do.  ROW (NULL, NULL) != ROW (1, 1), e.g., and I tested without a prerequisite locally.
<wgrant> cjwatson: And one_vcs doesn't check that the VCS is consistent.
<wgrant> Just that each of source, target and dependent only has ne.
<cjwatson>  source_git_repository |   source_git_path    | target_git_repository |  target_git_path  | dependent_git_repository | dependent_git_path
<cjwatson> -----------------------+----------------------+-----------------------+-------------------+--------------------------+--------------------
<cjwatson>                      1 | refs/heads/ibm-nvram |                     1 | refs/heads/master |                          |
<cjwatson> wgrant: Hm, yes, that's true.
<cjwatson> Should be (source_branch IS NULL OR target_branch IS NULL OR dependent_branch IS NULL) != (source_git_repository IS NULL OR target_git_repository IS NULL OR dependent_git_repository IS NULL), I think.
<cjwatson> Er, s/IS NULL/IS NOT NULL/g
<cjwatson> Fixed.
<wgrant> cjwatson: I think that new one_vcs allows me to completely omit target or source, as long as one of the others is set.
<wgrant> But apart from that it looks reasonable now.
<cjwatson> Oh, that's true.
<cjwatson> Will fix after lunch.
<cjwatson> wgrant: How does that look to you now?
<cjwatson> ALTER TABLE BranchMergeProposal
<cjwatson>     ADD CONSTRAINT one_vcs CHECK (((source_branch IS NOT NULL AND target_branch IS NOT NULL) != (source_git_repository IS NOT NULL AND target_git_repository IS NOT NULL)) AND (dependent_branch IS NULL OR source_branch IS NOT NULL) AND (dependent_git_repository IS NULL OR source_git_repository IS NOT NULL));
<wgrant> cjwatson: That works.
<cjwatson> brushing off my boolean logic
<cjwatson> I want IMPLIES
<wgrant> A truth table would be more compact...
<wgrant> Yeah
<cjwatson> p â q â¡ Â¬p â¨ q, but.
<cjwatson> wgrant: Did you intend to review db-git-mp in the MP itself?
<wgrant> cjwatson: Er, yeah, if you've pushed that.
<wgrant> Sorry, distracted by OpenStack.
<cjwatson> I pushed that change, yes.
<cjwatson> How goes the restacking?
<wgrant> Finally got the x86 cloud back up with trusty+kilo, doing builds, next step is to see if nova-compute works on cameron.
<wgrant> And then weneed to hack up the imemaker bits so we can hulksmash them onto cameron for ppc64el only.
<cjwatson> works on cameron> Hopefully you won't have done the kilo upgrade for nothing ...
<cjwatson> Huh, I wonder why I made GitRepository.name readonly=True?
<wgrant> cjwatson: It'll at least be easier to fix on kilo!
<wgrant> cjwatson: name needs a setter, at least.
<cjwatson> Branch.name doesn't have one.  I assume you just get a constraint violation if you try to make it equal to an existing one.
<wgrant> Ugh.
<cjwatson> And trying to set GitRepository.owner crashes, bah.
<cjwatson> I'll be able to QA subscriptions eventually ...
<blr> cjwatson: excellent, great news :)
<blr> should have the cross-repo diff soon, yesterday was a bit of a write-off.
<cjwatson> I can imagine
<cjwatson> Not been hugely useful today myself ...
<cjwatson> wgrant,blr: Are we meeting?
#launchpad-dev 2015-04-22
<wgrant> cjwatson, blr: Oops, I remember seeing the notification on my phone, but apparently not enough to not fall asleep in the intervening ten minutes.
<cjwatson> wgrant: Could you have a quick look at https://code.launchpad.net/~cjwatson/launchpad/git-mail-fix-delta/+merge/257015 ?  I believe that will make it possible for me to clear the QA queue.
<cjwatson> (There's a couple of BMP preparation branches too, which should be reasonably trivial)
<cjwatson> wgrant: Ah, thanks :)
<wgrant> Yep
<cjwatson> wgrant: Oh, how's the apparmor stuff coming?
<cjwatson> wgrant: Our "Work out branch vs repo UI and model considerations" is essentially done now, isn't it?
<cjwatson> *task
<wgrant> cjwatson: Yep, I think so.
 * cjwatson tickyboxes
<cjwatson> I moved 'Make "Unmerged revisions" work' to later, and went through tasks adding dependency links in their descriptions
<cjwatson> Now that I've worked out how to do that
<wgrant> oh you can do that?
<cjwatson> wgrant: type @ and then you can search for a task name (among other things)
<cjwatson> It's only in prose, but a lot better than nothing
<wgrant> oh fancy
<cjwatson> Most of the MP code is up for review now.  There are a bit over 1K lines remaining, consisting of GitRef.getMergeProposals, TargetGitRepositoryWidget, BMP browser code of various levels of hackiness (I suspect resubmit might still be broken), and the new GitRef views.
#launchpad-dev 2015-04-23
<cjwatson> wgrant,blr: we're definitely going to need some pygit2 backports sooner rather than later - even disregarding merge_trees, conflicts handling in 0.22.0 is unusably bad for our purposes (you can find out that there was a conflict in a merge and which files it touched, but you can't get a merged version with conflict markers).  I'm going to sort out some backports, as the patches we need look fairly self-contained
<cjwatson> (why yes, I'm trying to put together the LP side of preview diffs because I can't sleep, why do you ask)
<blr> cjwatson: oh dear, hope you're not up all night!
<cjwatson> meh ...
 * cjwatson blinks at https://launchpad.net/~cjwatson/+archive/ubuntu/launchpad/+build/7350071.  How is that architecture-specific in scalingstack?
<cjwatson> https://github.com/libgit2/pygit2/pull/520 should fix the above, cherry-picking that
<blr> cjwatson: huh, in what instances were you seeing garbage in the diff?
<wgrant> cjwatson: Hum, I guess I was looking at master rather than 0.22.0 when I thought the conflict handling looked OK.
<cjwatson> blr: nothing we were using yet, but when trying to improve the conflict handling using merge_file_from_index, we ended up with garbage in place of the path in the conflict marker sometimes, because it wasn't holding onto a reference to that memory
<cjwatson> wgrant: right, it's fiddly but I do now have a workable algorithm which I'll implement on Friday when actually working
<cjwatson> basically walk over index.conflicts and do blob_oid = repo.create_blob(repo.merge_file_from_index(*conflict)); index.add(IndexEntry(path, blob_oid, GIT_FILEMODE_BLOB)); conflicts.remove(path)
<cjwatson> and keep a note of the conflicted paths and return them separately
<wgrant> cjwatson: What was in the merged tree in case of conflicts?
<cjwatson> wgrant: do you mean in diff_from_tree().patch?
<wgrant> cjwatson: yeah
<cjwatson> wgrant: a patch removing the conflicted file
<cjwatson> super-useful
<wgrant> ... oh
<wgrant> That's totally not what favor says it does.
<cjwatson> it soooooooort of makes sense internally
<cjwatson> in a very skewed way
<cjwatson> but yeah, I noticed when I tried to write a test and observed that it made NO SENSE
<cjwatson> (when adding info about conflicts to the compare-merge return value so that previewdiff can use them)
<cjwatson> wgrant: the cherry-picks for this plus merge_tree are in the LP PPA now, anyway.
<wgrant> Lovely, thanks.
#launchpad-dev 2015-04-24
<wgrant> cjwatson: Why "except Exception" in merge-conflicts?
<cjwatson> wgrant: Because pygit2 can raise a cluster of exceptions and I can't think of a better response to any of them than just leaving the file in a dodgy unresolved conflicted state.  Although I guess we could just decline to handle that exception at all and let it 500.
<wgrant> cjwatson: Crashing rather than giving a bogus diff seems reasonable to me, unless we discover otherwise later on.
<cjwatson> OK, done.
<cjwatson> Hm.  It's a bug that PreviewDiff.fromBranchMergeProposal doesn't set preview.prerequisite_revision_id, isn't it?
<wgrant> Quite likely.
<cjwatson> Seems like that means that we won't notice when a preview diff is stale due to a changed prereq.
<wgrant> Well, we don't consider diffs stale unless the source branch has changed, usually.
<cjwatson> I'm not actually sure stale affects anything much anyway ...
<cjwatson> The comments claim there's a stale-diff CSS class, but that's never set by anything.
<cjwatson> And I don't think whatever updates preview diffs has much to do with the stale property definition per se.
<wgrant> It may have been more relevant in the MAD days.
<cjwatson> MAD?
<wgrant> Merge Analysis Daemon
<wgrant> From the good old days when you had to run a script to generate preview diffs yourself because reasons.
<cjwatson> I don't remember that.
<wgrant> 2008ish, I think.
<cjwatson> wgrant: I think that https://code.launchpad.net/~cjwatson/rabbitfixture/work-around-stop-hang/+merge/257361 should work around quite a few of the pesky buildbot hangs, although we'd need to reclaim PyPI access.
<wgrant> cjwatson: Bah, I thought I'd caught all those packages apart from sidnei's.
<wgrant> bac: ^^ can we have rabbitfixture PyPI access?
<bac> wgrant: you want access to rabbitfixture on pypi?  what account?
<bac> wgrant: i had no idea i had access but i'm glad to add you.
<wgrant> bac: wgrant and cjwatson, pls.
<bac> okey doke.
<wgrant> bac: Thanks.
<cjwatson> wgrant: Ta.  subprocess32 was a nice discovery.
<wgrant> I had no idea of it.
<xnox> wgrant: are lazr.* & launchpadlib reclaimed as well?
 * xnox wants to release those on pypi
<wgrant> xnox: Yes, I have access to all lazr packages except two that nobody cares about.
<wgrant> Yell if you want some.
 * xnox checks which ones i care about
<xnox> lazr.restful, lazr.restfulclient, launchpadlib, lazr.delegates, lazr.authentication - you seemed to have access to all of them
 * xnox never did pypi releases
<xnox> so I'll ping you to do some of them ;-)
<xnox> unless it's automated
<cjwatson> twine IYF
<cjwatson> gpg --armor -b dist/foo.tar.gz && twine upload dist/foo.tar.gz{,.asc}
<cjwatson> assuming you already have a suitable .pypirc
<cjwatson> Right, rabbitfixture upgrade landing in devel, so hopefully that's the end of the spurious hangs.
<cjwatson> At least that relatively common species of them.
<cjwatson> wgrant: Have you had a chance to try out mojo locally recently?
<wgrant> cjwatson: I like having my containers working.
<wgrant> So no mojo since I left home; juju 1.23 is iffy enough
<cjwatson> Mm, I was trying to do the apache2/haproxy unhulksmashing task, but it's pretty tough to do blind ...
<wgrant> cjwatson: Does it not work?
<wgrant> Ah yeah
<cjwatson> Well I never solved the problem I had last Saturday.
<wgrant> Was that the share-netns?
<cjwatson> Yeah
<wgrant> Let's see if I can make it work ish.
<cjwatson> Hm.  Do we actually need apache2 at all if haproxy is doing SSL termination, I wonder?
<cjwatson> All it's doing is logging, proxying to turnip, and getting in the way
<cjwatson> (I ask because the stock apache2 charm doesn't support the relation stuff it'd need to be frontended by haproxy)
<wgrant> We don't currently make much use of it, indeed.
<wgrant> I suspect that we will in future, but it could be removed for now if it's awkward.
<cjwatson> What for?  Routing hacks or something?
<wgrant> Or serving any static content, redirecting / rather than returning uselessness, etc.
<cjwatson> It'd be tempting to just do that in turnip directly.
<cjwatson> We already have a few static bits there for cgit.
<cjwatson> (Which, OK, are kind of in the wrong place but it shows it's not hard)
<wgrant> Heh
<wgrant> Sure.
<cjwatson> The HTTP -> HTTPS redirection is reasonably easy with haproxy 1.5.
<wgrant> orly
<wgrant> I'm used to LP's haproxy which requires error pages to include HTTP headers...
<cjwatson> "redirect scheme https code 301 if !{ ssl_fc }"
<cjwatson> I think the only difficult bit is that we probably have to change the charm and the spec at once due to using relation-driven proxying, unless perhaps we add something to turnip's config.yaml to let it know whether it's being frontended directly by a modern haproxy
<cjwatson> (Because then haproxy has to listen on ports 80/443, not the same ports as turnip)
<wgrant> I'm not sure that using sensible relations here is sensible.
<wgrant> But do what seems sensible.
<cjwatson> Thanks for the detailed advice. :P
<cjwatson> I don't know.  Could put the redirect bit directly in the spec, but there isn't really a good way to override the smart-http port at spec level AFAICS.
<cjwatson> Thinking of something like http://paste.ubuntu.com/10879298/
<wgrant> ew
<wgrant> But that would indeed work, I think.
<cjwatson> It's kind of weird to have this in turnip.
<wgrant> The lack of relation config bites us again
<cjwatson> I guess an alternative is to put the 80/443 logic in the spec and have haproxy proxy to itself
<cjwatson> So haproxy:443 -> haproxy:9419 -> turnip:9419
<cjwatson> (Which I'm assuming one can do, but it's kind of shuffling bytes around for no production benefit)
<cjwatson> Seriously?  My random-buildbot-hang fix has failed buildbot twice in succession with different random errors?
<cjwatson> buildbot has a sense of humour.
<wgrant> It certainly does.
<cjwatson> Now it's failing in a different bit of rabbitfixture, but it doesn't look related
<cjwatson> Must do something about attaching that log as a test detail
#launchpad-dev 2015-04-26
<cjwatson> wgrant: mind if I pick up the "separate services for turnip frontend/virt/storage daemons" task from you?  I was going to start on cleaning up our logging tomorrow morning, but that's going to be easier if I first separate out all the services and sort out the whole twisted logging malarkey
<wgrant> cjwatson: oh sure
<cjwatson> ok, grabbed
<cjwatson> wgrant: I seem to remember you had some thoughts on adding a request id to turnip so that we could track log messages associated with failures sensibly.  Did you have any thoughts on how to do that in a useful way now that the services are separated?
<cjwatson> It would be nice not to have to extend all the various protocols involved to put a request ID at the front, but I suppose it's possible ...
#launchpad-dev 2020-04-20
<cjwatson> Could I have a review of https://code.launchpad.net/~cjwatson/lp-signing/+git/lp-signing/+merge/382431 (Run migrations in charm), please?
<tomwardill> cjwatson: you have migrations happening before service.configured, is that the right order?
<cjwatson> tomwardill: I think so - doesn't seem like a good idea to claim that the app is up before its DB schema is up to date
<tomwardill> fair
<tomwardill> have a +1 :)
<cjwatson> (In fact, gunicorn starts on service.configured, so we definitely need to avoid doing that early.)
<cjwatson> There is a certain amount of puzzle-box programming here to figure out exactly how to avoid tickling the OLS layers in the wrong way.
<cjwatson> Thanks
 * tomwardill fetches more coffee
<tomwardill> do we want users to be able to create credenti'als with the same url and username?
<tomwardill> I'm leaning towards we should check for that
<cjwatson> It seems likely to be confusing
 * tomwardill adds the check
<tomwardill> but first, lunch
<cjwatson> Ah that's right, didn't add a DB constraint because we weren't really sure enough of what form creds would take I think
<tomwardill> Yeah, still not sure. The first set is obviously going to be username and password, but tokens are a possibility later.
<tomwardill> Doing code checks for it makes the most sense, as weâll need to change the push code to handle anything other than None and username/password
<cjwatson> Not to mention the UI.
<tomwardill> Oh yes, that too.
<cjwatson> https://code.launchpad.net/~cjwatson/lp-signing/+git/lp-signing/+merge/382581   brown-paper-bag fix to DB setup, please review
<tomwardill> cjwatson: +1
<tomwardill> cjwatson: do we have any precedence for a 'createOrExisting' type method?
<tomwardill> I'm thinking Registry Credentials could use that
<tomwardill> maybe get_or_create is a better description
<cjwatson> tomwardill: Yeah, there are various
<cjwatson> tomwardill: Several getOrCreateFoo
<cjwatson> tomwardill: ensureFoo is also an option
<tomwardill> ah, excellent
<tomwardill> I shall try some variant of that
 * tomwardill blergs at the web service
<tomwardill> getting a 401 when I try to call my new method, when I am definitely the owner
<pappacena> Is there anything on the response's body?
<tomwardill> "(<lp.oci.model.ocirecipe.OCIRecipe object at 0x7f3fa1f8c250>, 'createPushRule', 'launchpad.Edit')"
<pappacena> strange... did you try `user.isOwner(the_oci_recipe_object)`, just to make sure?
<cjwatson> I usually find that some bit of ownership is not what I think it is ...
<tomwardill> is there a way to debug this?
<pappacena> On security.py, put a breaking point at EditOCIRecipe.checkAuthenticated
<cjwatson> Yep
<cjwatson> Alternatively, breakpoint on zope.publisher.publish.mapply is a bigger hammer if you need it
<cjwatson> That should be shortly before the traversed-to method is entered
<tomwardill> oh, maybe I'm doing a silly
<cjwatson> (But EditOCIRecipe.checkAuthenticated is an easier place to start, probably)
<tomwardill> narrator: he was doing a silly.
<tomwardill> (forgot the owner= on factory.makeOCIRecipe)
<cjwatson> 16:47 <cjwatson> I usually find that some bit of ownership is not what I think it is ...
<cjwatson> :-)
<pappacena> hahaha! Been there, done that...
<tomwardill> NoCanonicalUrl: No url for <lp.oci.model.ocipushrule.OCIPushRule object at
<tomwardill> ah, yes
<tomwardill> that
<cjwatson> pappacena: Could you have a quick look at https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/382592 ?
 * pappacena sure!
<cjwatson> Should fix an OOPS I hit on dogfood
<pappacena> Tom, URL tag on zcml... export the new entity on webservice.py...
<pappacena> Ahm... I didn't realise the signing process uses a different db user. Thanks for the MP, cjwatson
<cjwatson> Easy to miss
<cjwatson> pappacena: Did you notice http://lpbuildbot.canonical.com/builders/lp-devel-xenial/builds/1192/steps/shell_9/logs/summary ?
<pappacena> No... let me check
<cjwatson> and/or do you need help debugging it?
<cjwatson> Ugh, passes for me locally
<pappacena> I'll try to reproduce... but I don't recognize those tests...
<pappacena> uhm...
<pappacena> there was a timeout trying to start rabbitmq at the begining... could that be the cause?
<cjwatson> pappacena: Ah, the patch that triggered this was just adding DB tables, so buildbot is probably unfairly blaming you
<cjwatson> It's possible
<pappacena> Let me force build it again
<cjwatson> I've just retried, sorry for the noise
<pappacena> No problem :)
<tomwardill> testtools.matchers._impl.MismatchError: 200 != 404: Object: <lp.oci.model.ocirecipe.OCIRecipe object at 0x7f8c1bebfe10>, name: u'string:distribution-100004'
<tomwardill> you're not wrong there testtools, you're not wrong
<cjwatson> That string: is slightly suspicious
<cjwatson> Feels like a buggy <browser:url/>
<tomwardill> yeah, that would make sense
<tomwardill> I'm failing to define the correct url for a pushrule, apparently
<tomwardill> any ideas why "path_expression="string:+pushrule/${id}" is producing: 'http://api.launchpad.test/devel/~person-name-100000/distribution-100004/+oci/oci-project-name-100017/+recipe/oci-recipe-name-100018/+pushrule/None'
<tomwardill> the None there isn't exactly right...
<pappacena> Strange. Is it really commited to database? Did you check if the OCIPushRule object actually has an ID at that point?
<cjwatson> ids are typically serials which are generated by the DB in the process of adding the row, so you may need to flush after adding the object to the store in order to get the id back
<cjwatson> (commit would do it too but is overkill)
<cjwatson> Another small fix found while QAing the signing service: https://code.launchpad.net/~cjwatson/lp-signing/+git/lp-signing/+merge/382598
<pappacena> I'll check
<tomwardill> okay, that fixed my id, now to work out why I'm getting a 404 :)
<pappacena> New @stepthrough method on OCIRecipeNavigation ?
<tomwardill> yep, that's the one. Ta pappacena :)
 * pappacena Good! :)
<tomwardill> Total: 217 tests, 0 failures, 0 errors, 0 skipped in 2 minutes 55.683 seconds.
<tomwardill> well, that'll do. Cleaning up the mess I've made along the way is a job for FutureTom
 * tomwardill -> EOD :)
<cjwatson> https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/382606 (Add UI for editing Distribution.oci_project_admin)
 * pappacena I'll check
<cjwatson> I've done a bunch of QA and intend to do a rollout tonight; we need to get it out this week, but rollouts for the next few days will have to be extra-careful to avoid disrupting the Ubuntu release, so it has to be now
<cjwatson> (Not ideal timing, but there you go)
<cjwatson> Shout if you know of anything wrong
<pappacena> Ok... let me know if something doesn't work properly...
<cjwatson> Actually ... bother, I just realised that the comment about bug contacts in https://bugs.launchpad.net/launchpad/+bug/117752 still stands
<mup> Bug #117752: Any logged in user can delete any attachments <lp-bugs> <Launchpad itself:In Progress by pappacena> <https://launchpad.net/bugs/117752>
<cjwatson> Which we discussed briefly in the MP, but I'd missed that comment about retracers deleting attachments
<pappacena> retracers?
<cjwatson> Ubuntu has an automatic crash reporting system which in some cases creates bugs with core dumps attached, but those have to be private since they often contain lots of confidential information; there's a service that goes through core dumps attached to bugs in this way and extracts tracebacks from them, then deletes the core dumps, which makes it less of a privacy minefield
<cjwatson> But the "deletes the core dumps" bit is kind of crucial
<cjwatson> Sorry, I should have thought of that wrinkle earlier
<pappacena> Ahh... what permission should we use to allow them to delete too?
<cjwatson> bug_supervisor, but it involves looking at bug tasks
<cjwatson> Because that's on the pillar
<cjwatson> But then we have to take more care about performance because bugs might have lots of bug tasks
<cjwatson> EditBug._hasAnyRole is similar and would probably do
<cjwatson> Let me think for a bit about whether we can get away with taking that slow approach
<pappacena> Ok...
<cjwatson> pappacena: I think we can.  Mind if I take this?  We're in a bit of a rush and I think I can do it faster than I can explain it :)
<pappacena> Sure, no problem! :-)
<cjwatson> pappacena: https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/382608
 * pappacena checking
<cjwatson> Thanks, landing
<cjwatson> pappacena: Before I forget - nice work on the lp-signing integration so far.  Once I fixed all the things that were essentially my fault (charm bugs, incorrect firewall rule, missing escaping in openssl's command line), it all seems to work just fine.
<cjwatson> Though I don't think we should actually attempt to enable it until after the Ubuntu release, even if we have a production environment by then :-)
<pappacena> Thanks, cjwatson! And actually, thanks for all the reviews and the work on the infrastructure... my part was actually quite small compared to this.
<pappacena> And I agree to play it safe and wait for Ubuntu release before we enable this in production.
#launchpad-dev 2020-04-21
<tomwardill> first attempt at API for OCI Registry Credentials: https://code.launchpad.net/~twom/launchpad/+git/launchpad/+merge/382651
<SpecialK|Canon> cjwatson pappacena: Nice work both; very much agree on holding off going live unti lpost-release!
 * SpecialK|Canon moves that l a bit to the left
<cjwatson> https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/382484   quick trivial UI fix?
<tomwardill> +1
<cjwatson> And https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/381912 is a simple code tidy-up
<cjwatson> Thanks
<tomwardill> also +1
<tomwardill> I did get very confused by the title of the first one
<tomwardill> 'but why are we building images again?'
<cjwatson> ah, sorry :)
<cjwatson> tomwardill: and thanks, I'll look over oci-registry-credentials-api
<tomwardill> ta
 * tomwardill -> lunch
<cjwatson> tomwardill: All right, you have a pile of comments on https://code.launchpad.net/~twom/launchpad/+git/launchpad/+merge/382651 now
<cjwatson> ilasc: ^- You probably want to look over those as well, since they have some bearing on your UI work
<cjwatson> And this reminds me that I need to get the production config ticket in to generate a registry credential storage key pair
<tomwardill> sigh, that thing where pass-by-reference catches you out, again
<tomwardill> cjwatson, ilasc: https://code.launchpad.net/~twom/launchpad/+git/launchpad/+merge/382674
<ilasc> thanks tomwardill
<ilasc> looking now
<tomwardill> now without me copying and pasting cjwatson's comment into the middle of it :)
<tomwardill> if this push ever succeeds
<ilasc> :)
<ilasc> +1
<cjwatson> tomwardill: r=me with a few comments
<tomwardill> ta
<tomwardill> after a brief fight with tests, landing that branch
<cjwatson> Good stuff
<cjwatson> Could anyone have a look over https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/381234 (built-using-model)?  It's had an overall direction review from William, but it needs more detailed code review.  You may need to look further up the stack (anything on https://code.launchpad.net/~cjwatson/launchpad/+activereviews that mentions "built-using") for context.
<cjwatson> I'm still working on the top couple of layers of that, but I think the lowest-level model branch and probably also built-using-ui is pretty solid now.
 * tomwardill will have a look in a mo
<tomwardill> cjwatson: having now implemented that branch, I'm unsure how it helps... as `getCredentials` will still attempt a decrypt, so it doesn't help in the OCIPushRule API
<tomwardill> other than accessing _credentials directly
<cjwatson> tomwardill: Well, we don't need to export getCredentials.  We could have a username accessor property that just looks at the unencrypted bit
<tomwardill> ah, yes, that makes sense
 * tomwardill goes back to arguing with interface declarations
<tomwardill> cjwatson: https://code.launchpad.net/~twom/launchpad/+git/launchpad/+merge/382651 is up again
<cjwatson> Ack
<cjwatson> tomwardill: r=me with a few tweaks
<tomwardill> ta, looking now
<tomwardill> fixed, landing
 * tomwardill -> EOD, I'll pop back to poke buildbot if required
#launchpad-dev 2020-04-22
 * tomwardill pokes buildbot a bit more
 * ilasc is writing a book for the Support Wiki with answers she's getting from wgrant this week :)
<tomwardill> excellent :)
<ilasc> tomwardill: just checking I understood correctly from the buildbot emails I saw this morning, your Store Username merged & built successfully, then API for OCI Registry Credentials failed with the usual and you triggered a forced build which again failed with the usual  ?
<tomwardill> yep
<tomwardill> I've just triggered another one
<tomwardill> they were slightly different usuals each time, but in that group, yeah
<ilasc> ok cool, thanks! :)
<ilasc> tomwardill: code-wise this means I can now rebase / merge my UI branch to take in the Username changes from master
<tomwardill> yep, it should be merged
<ilasc> ok, great
<cjwatson> tomwardill: So if I upgrade dogfood, it should be possible to run an end-to-end test?
<tomwardill> errr, I think so? :)
<tomwardill> does dogfood run upload jobs automatically?
<cjwatson> We'd need to run that manually
<tomwardill> right
<tomwardill> yeah, I think we should be good to try it
<wgrant> ilasc: Thanks for continuing to write the manual that we should have written years ago :)
<cjwatson> tomwardill: dogfood is current now
<tomwardill> well
<tomwardill> better try a thing then :)
<tomwardill> huh, getOCIProject isn't exported on distribution
 * tomwardill takes notes
<tomwardill> https://dogfood.paddev.net/~twom/ubuntu/+oci/test-twom-2020-04-22-2/+recipe/test-twom-2020-04-22-2-recipe/+build/7
 * tomwardill forgot to add any credentials to the recipe
 * tomwardill does that
<tomwardill> in lp-shell, is there a way to load an object from an arbitary API call?
<cjwatson> Like lp.load you mean?
<tomwardill> oh, maybe that :)
<cjwatson> Hm, no OCIRecipeBuildJob created by your build.
<cjwatson> Ah, maybe you haven't actually created the push rule yet?
<tomwardill> no, I failed
<tomwardill> well, forgot
<tomwardill> trying to do that now
<cjwatson> On loading arbitrary objects, also note that where launchpadlib normally takes an object as a method parameter, you can in general also pass a URL path.
<tomwardill> newPushRule appears to be missing from lp.operations on a OCIRecipe
<tomwardill> oh, is this lp-shell caching WADL?
<cjwatson> So e.g. instead of foo.bar(distro_series=lp.distributions['ubuntu'].getSeries(name_or_version='focal')) you can say foo.bar(distro_series='/ubuntu/focal')
<cjwatson> Yes
<cjwatson> Very annoying and one of these days I'll fix it in launchpadlib
<tomwardill> how do I fix that again?
<cjwatson> rm -v ~/.launchpadlib/api.launchpad.net/cache/*.wadl*
<tomwardill> nope, still only have newWebHook
<tomwardill> hmm
<cjwatson> Er sorry, dogfood
<cjwatson> rm -v ~/.launchpadlib/api.dogfood.paddev.net/cache/*.wadl*
<tomwardill> ah, of course
<tomwardill> right, there we go
 * tomwardill finds his credentials
<cjwatson> (The bug is probably in lazr.restfulclient actually)
<tomwardill> raise RuntimeError("No public key configured")\nRuntimeError: No public key configured\n'
<cjwatson> You know I absolutely could have sworn I'd done that
<cjwatson> But no sign of it
<cjwatson> Please hold
<cjwatson> tomwardill: Should be fixed
<tomwardill> well, I've got a push rule
<tomwardill> starting a new build
<tomwardill> https://dogfood.paddev.net/~twom/ubuntu/+oci/test-twom-2020-04-22-2/+recipe/test-twom-2020-04-22-2-recipe/+build/8
<tomwardill> cjwatson: build done
<cjwatson> $ /srv/launchpad.net/codelines/current/cronscripts/process-job-source.py -v IOCIRegistryUploadJobSource
<tomwardill> {"repositories":["busybox","dogfood/test-twom-weget","pjdc/registry-2","registry","test-wget-image","twom/base_with_curl","twom-test/test-wget-image"]}
<tomwardill> well, there's a thing
<cjwatson> Seems happy.  Would be nice to have some UI indication in LP
<cjwatson> Also it's super-annoying that lp-shell no longer has good completion after upgrading to focal
<tomwardill> Status: Downloaded newer image for upload.image-registry.staging.canonical.com:5000/dogfood/test-twom-weget:edge
<cjwatson> Presumably a Python 3 thing
<tomwardill> I can docker pull it
<tomwardill> should probably try it with a more complicated docker image...
<cjwatson> tomwardill: You can probably move https://trello.com/c/uUeoPcJx/68-ociregistrycredentials-ocipushrule-api to done?
<tomwardill> done
 * tomwardill enjoys the confetti
<tomwardill> so yeah, we're done right?
<tomwardill> ;)
<cjwatson> This does basically look like an MVP: we need https://code.launchpad.net/~cjwatson/lp-production-configs/oci-registry-secrets-public-key/+merge/382733 merged, then a production deployment, then setting up celery and cron for IOCIRegistryUploadJobSource
<cjwatson> A whole bunch more UI would sure be nice
<tomwardill> yeah
<cjwatson> Looks like you can currently create push rules but not list them
<cjwatson> (on the API)
<tomwardill> yup
<tomwardill> I've got a couple of notes of missing things I just found
<cjwatson> I'm marking your commits green in deployable though since they basically work
<cjwatson> tomwardill: I'll work on the UI to show the state of registry uploads, if that's OK
<tomwardill> sure
<cjwatson> tomwardill: Is the error message handling in OCIRegistryUploadJob.run actually correct for pushing to OCI registries?  It seems to be fairly directly taken from snap store interaction, and the "messages" and "details" stuff there is quite specific to the store's API
<cjwatson> tomwardill: Do you know if the registry API ever returns multiple error messages for a single request?
<cjwatson> Just wondering if we can simplify this or rationalise it in some way
<tomwardill> cjwatson: hmm, it may not be. It seemed reasonably flexible for handling possible cases, but I'd need to reread the registry docs for it. I was also working on the assumption that a custom frontend might eventually be a thing
<tomwardill> https://docs.docker.com/registry/spec/api/#errors-2 the spec is not hugely enlightening
<cjwatson> tomwardill: The other thing is that in the store case error_message was mostly there for compatibility but you've replicated it; so trying to work out if we can get away with only one of singular vs. plural before fossilising the API
<tomwardill> cjwatson: yeah, it looks like it
<cjwatson> tomwardill: errors does generally seem to be possibly-plural there, so I think I'll just consider the plural case in what I'm doing
<tomwardill> export push_rules on the API: https://code.launchpad.net/~twom/launchpad/+git/launchpad/+merge/382771
<cjwatson> r=me
<tomwardill> ta, landing
<SpecialK|Canon> cjwatson: sometime between now and Tuesday it would be fab if you could add some status notes to https://trello.com/c/41wQZbTD/26-built-using-support please
<cjwatson> Yep, plan to
<cjwatson> Was hoping to get closer to "code all done locally" first
<cjwatson> tomwardill: did you get a chance to start having a look at built-using-model btw?
<tomwardill> cjwatson: gah, no, I forgot. Give me link again and I'll do it next?
<SpecialK|Canon> cjwatson: by all means, cheers
<cjwatson> tomwardill: https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/381234
<cjwatson> tomwardill: https://code.launchpad.net/~cjwatson/launchpad/+activereviews and grep for built-using has other useful context
<cjwatson> (As in, sometimes looking further up the stack may be helpful for understanding
<cjwatson> )
<cjwatson> Also https://docs.google.com/document/d/1HNwxF16tBlh2EjSexihmZj9qP9w5oeGDWF0Fygb8ols is the hopefully-not-too-bitrotten design doc
<tomwardill> i need to find a more effective way to review things, I really struggle with it just scrolling in a browser
<cjwatson> I sometimes add a remote and fetch locally
<tomwardill> cjwatson: it all looks reasonable to me, I'll go over it again in the morning, but I think I can follow it :)
<cjwatson> Finally landing removal of --system-site-packages now that it's unblocked
 * cjwatson looks forward to fewer incomprehensible pip bugs
 * pappacena \o/
<cjwatson> I did actually have something else blocked on that for some reason I never quite tracked down - a ZTK upgrade I think
<cjwatson> Something to do with ampoule child processes preferring the system zope.interface to the virtualenv's one because why
#launchpad-dev 2020-04-23
<tomwardill> cjwatson: I've had another look over the Built Using MP. I've left two (small) questions, but otherwise it looks good to me. I think I can follow the implementation and the model for it at least.
<tomwardill> there is possibly/probably detail bits I'm missing, but I can't see anything broadly wrong with it
 * tomwardill looks at the pile of reviews and suspects cjwatson of not sleeping again :P
 * SpecialK|Canon peers
<cjwatson> tomwardill: clarified, I hope
<tomwardill> cjwatson: ah right, I remember you talking about that now
<tomwardill> okay, +1 then :)
<cjwatson> Thanks
<tomwardill> is an exported property on distribution listing the ociprojects desirable?
<tomwardill> I can't find precedence in source packages or livefs, but it seems like something that would be wanted
<cjwatson> Properties are probably not great for unordered unbounded collections like that.  Maybe a search method instead?
<cjwatson> We want to be able to navigate down the tree, but maybe not by way of a full list
<cjwatson> (searchSourcePackages exists, for instance, though it's a very old design and shouldn't necessarily be taken as unqualified precedent)
<tomwardill> ah, interesting
<tomwardill> it is really hard to write OCI type documents when you want to use the word 'container' outside of an actual OCI container
<tomwardill> cjwatson: something that has just occured to me, is there any reason why there can only be one 'official' recipe for a project?
<tomwardill> for instance, if we had a project that relied on multiple images to be produced, they could be represented as multiple 'official' recipes
<cjwatson> tomwardill: well, the point of 'official' was to own a default bit of namespace
<cjwatson> We could consider a different presentational name for that idea
<tomwardill> ah yeah
 * cjwatson finally gets built-using-domination back into shape
<cjwatson> I think that and the rest of the built-using stack are now worth reviewing again
<pappacena> I'll land https://code.launchpad.net/~pappacena/launchpad/+git/launchpad/+merge/381683, with the permission check for Distribution.oci_project_admin on OCI Project creation. It was approved a couple of days ago, and I forgot to merge it.
<cjwatson> Yep, go for it.  I don't always remember either ...
<pappacena> Buildbot seems to be failing. Is anyone checking that?
<cjwatson> pappacena: Ah, built-using-model fallout, I'll look at it, thanks
<pappacena> Thanks!
<cjwatson> Looks trivial
<pappacena> If it's already EOD for you, I can check that
<cjwatson> It's OK, will take five minutes
<pappacena> Ok! :)
<tomwardill> how do I add a new dependency to launchpad?
<cjwatson> doc/pip.txt
<tomwardill> aha
<cjwatson> Some assembly required
<cjwatson> pappacena: https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/382878
 * tomwardill defenestrates python https://github.com/python-poetry/poetry/issues/1975
 * pappacena checking MP
<cjwatson> thanks
#launchpad-dev 2020-04-24
<tomwardill> the hardest part about retries is writing tests for them
<SpecialK|Canon> ahja
<tomwardill> https://github.com/jd/tenacity is neat though
<tomwardill> Add retries for ConnectionError in registry client: https://code.launchpad.net/~twom/launchpad/+git/launchpad/+merge/382904 and matching dep addition: https://code.launchpad.net/~twom/lp-source-dependencies/+git/lp-source-dependencies/+merge/382903
<cjwatson> Thanks, looking
<cjwatson> Oh, Julien Danjou, good
<tomwardill> it's a maintained fork of the retrying library
<tomwardill> the backoff library is currently unbuildable from source
<cjwatson> Question in there for you
<tomwardill> yep, we can shorten the retry, I just picked a number
<tomwardill> 5 seconds * 5 retries?
<cjwatson> 3 seconds maybe?  I'm not super-happy about 15 seconds either but it's hopefully tolerable at least until we figure out something better
<tomwardill> sure
<tomwardill> cjwatson: changed and pushed
<cjwatson> r=me with a few more comments
<tomwardill> cjwatson: oh good call on the sleep patch, that has massively simplified things
<cjwatson> Oh good
<tomwardill> in theory it'd be possible to remove the inner patch / exception raising, but I'm going to leave it so we're not _just_ trusting tenacity to tell us it's retried
 * tomwardill adds comments to that effect
<tomwardill> cjwatson: how do I land the dependencies branch, just top approve and wait for it to be merged?
<cjwatson> tomwardill: Yep
<tomwardill> excellent, landing then
<tomwardill> landing retry
<tomwardill> both merged
<SpecialK|Canon> So uh
<SpecialK|Canon> Automagic links on the OCI image build page for "we built it AND pushed so it's available at example.com/badger"
<SpecialK|Canon> Is that as painfully hard/unpossible as I suspect?
<tomwardill> Snap has something similar, iirc
<SpecialK|Canon> presumably easier there because they're only going to be pushing it to one place
<cjwatson> SpecialK|Canon: I started working on that the other day
<tomwardill> yeah, but we know the registry url, so we should be able to work it out from that
<tomwardill> 'should'
<SpecialK|Canon> cjwatson: <3
<cjwatson> At least to provide the status of pushes.  I expect I'd need help on figuring out the target URL
<cjwatson> tomwardill: I don't suppose it's something the registry tells us in a push response?
<cjwatson> If so, the push job could stash it.  (That's what we do for snap)
<tomwardill> I've not looked, but https://registry_url/image_name should just work
<tomwardill> as that's what the `docker pull` command would be (we have to assume there's no web front end on a registry)
<cjwatson> Reliably for all registries?  OK
<tomwardill> oh, for us it'd be https://registry_url/image_name:edge to perform a no-interaction pull
<tomwardill> where 'edge' is the result of OCIRegistryClient._calculateTag (which could move to OCIRecipeBuild reasonably easily)
<cjwatson> Or the build job could stash what it did
<cjwatson> Trade-off is that that puts the URLs in the DB so they're harder to fix if we get them wrong
<tomwardill> or that
<tomwardill> but is immune to changes in the calculation affecting past images
<tomwardill> given we know that image_name and tag is going to change soonish
<ilasc> cjwatson: acknowledging I saw your comments in the OCI UI doc and all makes perfect sense, working in that direction, thank you !
<cjwatson> ilasc: you might as well take advantage of my painful experience working out that permissions UI :)  my notes say spec started 2018-08-10, blog post about the completed feature posted 2019-01-10
<ilasc> :)
<cjwatson> though of course a lot of that was working out the correct underlying model
<cjwatson> OK, 2018-10-05 "started experimenting with git permissions UI"; 2018-10-31 proposed the MP
<cjwatson> that's probably slightly more representative
 * tomwardill nearly defenestrated that matching algorithm, many times
<tomwardill> and I actually could've done, because I had it printed out and covered in red pen
<ilasc> wow :) OK
<ilasc> yes, I am currently using a similar approach to parse the form data
