#launchpad-dev 2009-03-10
<cody-somerville> Moo
#launchpad-dev 2010-03-15
 * mwhudson takes a late lunch break
<wgrant> OOPS-1535ED144
<wgrant> https://edge.launchpad.net/ubuntu/karmic/i386/cpp-3.4 reproducibly OOPSing -- what's the traceback on that?
<lifeless> 10509/lib/lp/soyuz/browser/../templates/distroarchseriesbinarypackage-index.pt object at 0xe60c850>, 'isRedirectInhibited')
<lifeless>   Module zope.traversing.adapters, line 52, in traverse
<lifeless>    - __traceback_info__: (<zope.browserpage.metaconfigure.SimpleViewClass from /srv/edge.launchpad.net/edge/launchpad-rev-10509/lib/lp/soyuz/browser/../templates/distroarchseriesbinarypackage-index.pt object at 0xe60c850>, 'isRedirectInhibited', [])
<lifeless> LocationError: (<zope.browserpage.metaconfigure.SimpleViewClass from /srv/edge.launchpad.net/edge/launchpad-rev-10509/lib/lp/soyuz/browser/../templates/distroarchseriesbinarypackage-index.pt object at 0xe60c850>, 'isRedirectInhibited')
<wgrant> Oh, that one.
<wgrant> Thanks.
<lifeless> wgrant: de nada
<lifeless> popping out for a bit
<thumper> wgrant: know about that?
<thumper> wgrant: I had that on a team page to add a member
<wgrant> thumper: General issue with anything that isn't a LaunchpadView.
<thumper> ah...
<wgrant> I saw discussion about it somewhere.
<wgrant> It's known by foundations/registry, at least.
<thumper> cool
<mwhudson> thumper: i think i've unexpectedly fixed https://bugs.edge.launchpad.net/launchpad-code/+bug/340295
<mup> Bug #340295: exceptions.AttributeError: 'NoneType' object has no attribute 'write' <codehosting-ssh> <oops> <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/340295>
<thumper> mwhudson: cool. How?
<adeuring> good morning
<mrevell> Morning
<mwhudson> thumper: by fixing https://bugs.launchpad.net/bugs/335156
<mup> Bug #335156: "not allowed to execute..." message sent over ssh stdout channel <Launchpad Bazaar Integration:In Progress by mwhudson> <https://launchpad.net/bugs/335156>
<deryck> Morning, all.
<maxb>   File "bootstrap.py", line 151, in <module> PYTHONPATH=ws.find(pkg_resources.Requirement.parse('setuptools')).location)
<maxb> AttributeError: 'NoneType' object has no attribute 'location'
<maxb> wgrant: ^^^ Is that the python2.6 distribute intermittent error you have mentioned in the past?
<wgrant> maxb: That's one aspect of it, yes.
<wgrant> It manifests itself in around three different ways.
<wgrant> Or none at all.
<wgrant> Depending on $UNKNOWN.
<maxb> mm. lovely
<wgrant> maxb: Oh, 2.6? It happen(s|ed) with 2.5 sometimes too.
<maxb> ah. never had that happen to me so ar
<maxb> *far
<wgrant> I haven't had it happen for a while.
<wgrant> So maybe it's fixed now.
<maxb> yet I've just had it happen for the first time :-/
<wgrant> But 2.6?
<henninge> jtv1: while I have you here alive and kicking, can you look at this if it looks sane? Looks almost too simple to me  ...
<henninge> http://paste.ubuntu.com/395545/
<henninge> jtv: I moved findProductSeries into Branch
<henninge> jtv: also, I assume that these imports are *not* published elsewhere and set the branch.owner to be the importer.
<jtv> henninge: templates don't have the distinction... effectively they're always published.
<henninge> jtv: yes, I was just wondering about that ....
<jtv> I'd pass True, but purely as a matter of form.
<jtv> But yes, that looks pretty much alright...  assuming "tarball" is the string of contents of the tarball.
<henninge> jtv: it is
<jtv> henninge: the findProductSeries method isn't currently in IBranch, is it?
<henninge> jtv: yes, I moved it there. I found it odd importing the BranchJobSource here and it really is a method if the branch, isn't it?
<henninge> jtv: I did the same with ...
 * henninge looks it up
<henninge> providesTranslationFiles()
<jtv> henninge: to me it looks more as if it should be a method of ProductSeriesSet, if there is such a beast.
<henninge> jtv: hm, that is the other alternative, I admit
<jtv> There is such a beast.
<henninge> jtv:  findHasBranch ?
<jtv> It's going to be ugly pigeonholing this under one part of LP when it's clearly a bridge between Code and Translations, with one foot in Registry.
<henninge> findHasTranslationsBranch and leave out the parameter
<jtv> findTranslationImportersForBranch?
<henninge> huh?
<jtv> findTranslationConsumersForBranch?
<henninge> better, Importers has been used too many times
<henninge> findRelatedBranch ?
<henninge> findRelatedTranslationsImportBranch
<jtv> henninge: one part I probably just don't know enough about...  why do you have a seemingly generic function for finding productseries related to a branch, with a parameter saying what behaviour you want?  Is there more than this one use-case that needs to call the same function?
<henninge> jtv: not yet, but this is just following a simple database fk relation, so I found it a generic problem plus a filter.
<henninge> jtv: oh no, the parameter was there for a different reason!
<jtv> ?
<henninge> jtv: to enable "force translation imports" behaviour.
<jtv> oic
<henninge> jtv: we need to look at all series for that, not just those that have translation imports enabled.
<jtv> I still say that's the wrong thing to do though.
<henninge> and I inverted the meaning of the parameter to make it more general
<henninge> what?
<henninge> jtv: what is the wrong thing?
<jtv> I still say it's the wrong thing to let people request this on the productseries and then force it on any productseries that uses the same branch.
<henninge> ah
<henninge> hm, not sure it really works that way
<jtv> It should be either an action in the branch UI (not my favourite way to do it) or it should force imports only on the productseries you request it on.
<jtv> It doesn't currently work that way, and I'm saying that's wrong.
<henninge> that does sound wrong and I was not aware of that.
<henninge> I thought it did the filtering in code
<jtv> We discussed a few weeks back.
<jtv> The "force" flag really needs a productseries argument, not a boolean
<henninge> jtv: yes, you are right. i don't remember talking about it, though.
 * jtv resists temptation to also claim that henning borrowed money from him on that occasion
<henninge> lol
<jtv> It was weeks ago.
<henninge> jtv: ok, I don't want to fix both things in the same branch. I'll just try to avoid making it harder than necessary.
<jtv> IIRC a solution could be to make the browser code pass the productseries as an argument when creating the IRosettaUploadJob, have that productseries' id stored in the BranchJob.json_data instead of just the boolean flag, and then if it's present, only upload to the given productseries (regardless of setting)
<henninge> jtv: do we have a bug for that?
<jtv> I don't think so.
<jtv> No wait, I think I would've filed one.
<jtv> Hang on.
<henninge> yes, I would expect ... ;)
<jtv> bug 521095
<mup> Bug #521095: One-time import acts on all series for the branch <code-integration> <Launchpad Translations:Triaged> <https://launchpad.net/bugs/521095>
<henninge> jtv: so all we need is a method to get all product series that have imports enabled on a specific branch.
<jtv> henninge: yup, that's what I thin
<jtv> k
<henninge> jtv: and I agree, it would fit best on the the set.
<jtv> \o/ then we _all_ agree!  :-)
<henninge> jtv: I don't like to introduce a new term by using "Consumers"
<henninge> though
<jtv> there's that.
<jtv> findByTranslationsImportBranch?
<henninge> cool
<henninge> jtv: yup, we'll take that.
<henninge> jtv: I'll see if i can fix both bugs after all. I am just wondering about affected tests.
<jtv> henninge: if an existing test should break, hopefully all that's needed is to change the import setting on a productseries.
<deryck> adeuring, hi.  Since you did the patch sniffing work... is Bug #538219 easy to fix?
 * adeuring is looking
<adeuring> deryck: yes, that should be easy to fix.
<deryck> adeuring, ok.  I'll mark it high and stick on our easy bugs backlog then.
<adeuring> deryck: sure, it sould be easy to fix it this cycle
<deryck> adeuring, excellent.  thanks.
<wgrant> The quick fix is to change a hack in lp.services.mime which overrides .debdiff to text/plain.
* salgado changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 2 of 10.03 | PQM is in open | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes |
<kfogel> adeuring: for bug #534736 ("patch attachments portlet should appear above regular attachments portlet on a bug page"), all tests pass without any changes other than the code change itself.  Is this the sort of thing that needs a new test?
<mup> Bug #534736: patches portlet should appear above attachment portlet <trivial> <ui> <Launchpad Bugs:In Progress by kfogel> <https://launchpad.net/bugs/534736>
<kfogel> adeuring: Also, should I get UI review on an ordering change like this?
<kfogel> intellectronica: see above questions to adeuring; if you're there and can answer, please do :-).
<adeuring> kfogel: I think a small addition to an existing page test would be good for this change
<adeuring> kfogel: and I am not sure if this really requires a UI revoew ;)
<kfogel> adeuring: thanks.  I'll look for a test.
<kfogel> adeuring: hmmm.  I'm not sure how to test this.  Usually when testing a portlet, we use find_portlet().  But in this case, it's the order in which two portlets appear that we care about.
<adeuring> kfogel: something like peint extract_text(browser.contents), then ellipises (?) to skip the unintersting text?
<kfogel> adeuring: aaaah.  gotcha, that makes sense.
<kfogel> thanks
<deryck> gmb, hi.  Have you seen Bug #538097?
<mup> Bug #538097: +storeblob fails with "500 Internal server error" on production (works on edge) <apport-collected> <lucid> <Launchpad Bugs:Confirmed> <apport (Ubuntu):Invalid by pitti> <https://launchpad.net/bugs/538097>
<kfogel> adeuring: I'm still a little confused about the difference between doc/ tests and stories/ tests.
<deryck> gmb, related to the filebug store blob work maybe?
<kfogel> adeuring: they appear to be the same kind of thing.  What is the distinction?
<adeuring> kfogel: storeis are intended to, well "tell stories from a user's perspective". They are not necessarily comprehensive tests. But we have quite often tests of details there tooo
<kfogel> adeuring: oh, so the difference is purely conceptual -- not technical.
<adeuring> kfogel: mostly. In doc, you can more easily do also things like run tests with another DB user
<gmb> deryck: Ihadn't seen that. I'll look into it.
<deryck> gmb, I put a card on the board for you.  Since this is blocking Lucid bug reports we should move it through quickly... if you need help from me, just ping.
<gmb> deryck: Righto, thanks
<gmb> deryck: Interestingly, all the OOPSes I'm seeing so far for +storeblob today are of this nature: https://lp-oops.canonical.com/oops/?oopsid=OOPS-1535E1487
<deryck> gmb, looking....
<gmb> deryck: Security machinery gone maaaaaad?
<deryck> gmb, interesting.  so apport needs to set referrer header?
<gmb> deryck: Well, I can't be sure yet, need to test.
<gmb> But that's my working hypothesis, yes.
<deryck> gmb, ok.  and I'm curious too why this is an issue for lpnet but not edge?
<gmb> deryck: Still investigating whether it actually isn't an issue for edge too.
<deryck> gmb, ok, cool.  thanks for looking into it.  sorry to sideline your checkwatches work
<gmb> deryck: No worries. I think I might need to get an up-to-date Lucid vm running to test this properly; I'll get that started and go back to CW until it's ready.
<deryck> gmb, that works
<intellectronica> kfogel: i'm back. do you still need help?
<kfogel> intellectronica: no, adeuring helped me.  Thanks.
<kfogel> adeuring: having trouble getting this test text to match.  The new test is: http://paste.ubuntu.com/395636/   And the failure is: http://paste.ubuntu.com/395637/
<kfogel> adeuring: I'm fiddling with lots of different combinations of whitespace and "..."  but no luck yet.
<adeuring> kfogel: did you try to start the expected text with "Bug #11"?
<mup> Bug #11: Rosetta says there are untranslated strings, but it isn't <Launchpad Translations:Fix Released by carlos> <https://launchpad.net/bugs/11>
<adeuring> i mean; instead of the ellipsis
<adeuring> erm, before the ellipsis
<henninge> kfogel: what he means is: You cannot start with an ellipsis, there must be some test first.
<henninge> adeuring: :-P
<kfogel> adeuring, henninge: thank you!
<sinzui> didrocks: ping
<didrocks> sinzui: hey
<sinzui> didrocks: gpgkeys are not visible over anon api.
<sinzui> didrocks: Is this intentional?
<didrocks> sinzui: no, it's not. as jml finished the change and I was able to see them in +apidoc, I don't know what happened
<danilos> kfogel, that initial text can be as simple as "<..." (i.e. opening < of a <html> tag)
<kfogel> danilos: *nod*
<danilos> (#11 bug mention pinged me :)
<kfogel> henninge, adeuring, danilos: test passes now, thanks.
<kfogel> danilos: heh
<sinzui> didrocks: It needs a anon security function and a 5 line test to verify anon users can access it
<didrocks> sinzui: oh, to anynomous gyus you mean?
<didrocks> sinzui: but still, I'm logged into LP and https://edge.launchpad.net/+apidoc/ is only has we were anonymous?
<didrocks> (I don't see it there)
<sinzui> didrocks: I do not think I should have to log in in my script to get a gpgkey that I could get by scraping the web site while not logged in
<didrocks> sinzui: I agree, but my question wasn't that one:
<didrocks> sinzui: I'm logged into LP, and I don't see the api in +apidoc
<didrocks> (the gpg in the api)
<didrocks> is it because I'm considered as being anonymous?
<sinzui> interesting, I read the api from the page.
<didrocks> erf, ctrl + R for the win
<sinzui> didrocks: https://edge.launchpad.net/+apidoc/#gpg_key
<didrocks> sinzui: right, the cache wasn't refreshed apparently
<sinzui> didrocks: you can see the collection listed on https://edge.launchpad.net/+apidoc/#team
<didrocks> sinzui: well, I can do that if you point me to the right doc, but I would prefer to have my ssh patch which prevents me release Quickly 0.4 to be reviewed first and not really have the time to fix that before Friday
<sinzui> didrocks: I can do it, I just wanted to be certain that the anon was intentionally omited
<didrocks> sinzui: no no, it wasn't intentional
<didrocks> sinzui: same fix has to be done to the ssh key I guess
<sinzui> yes, I think every object that we have to write a browser:url directive for must get an anon security function, because this is the first time the object is published
<danilos> jtv, what was the page that describes setting up a local buildfarm for translations?
<jtv> danilos: it's linked from dev.launchpad.net/Translations
<jtv> under Technical topics
<jtv> last entry there
<danilos> jtv, https://dev.launchpad.net/Translations/BuildTemplatesOnLocalBuildFarm
<danilos> ?
<jtv> yup
<danilos> jtv, does that integrate any discoveries henning has made? :)
<jtv> danilos: I may have copied some small things, but mainly I refer to the page he's written.
<jtv> He documented how to get the slave running in a chroot, which imho is infinitely more useful than the local install we used to have.
<jtv> danilos: I'm not aware of any inconsistencies between the two
<danilos> jtv, cool, thanks
<kfogel> deryck: I see bug 538226 and bug 538219 in the backlog; they look like good ones for me to grab, when I get back to coding, so I'll just pick one when ready.
<mup> Bug #538226: "What is a patch?" links to wrong help page <help> <trivial> <ui> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/538226>
<gmb> deryck: I've just confirmed pitti's reproduction of the bug in https://bugs.edge.launchpad.net/ubuntu/+source/apport/+bug/538097/comments/29
<mup> Bug #538097: +storeblob fails with "500 Internal server error" on production (works on edge) <apport-collected> <lucid> <oops> <Launchpad Bugs:Triaged> <apport (Ubuntu):Invalid> <https://launchpad.net/bugs/538097>
<gmb> deryck: Doesn't occur on edge.
<gmb> deryck: At the moment I've no idea why that is, unless some security gubbins got CP'd to production.
<deryck> gmb, hmmm, ok, interesting.
<deryck> gmb, we should ask around about recent CPs.
<gmb> deryck: Or perhaps there's some redirect happening with request to https://edge.launchpad.net/+storeblob (guessing)
<gmb> deryck: Simple work around is to try posting to http://launchpad.net/+storeblob; the http->https redirect adds a referrer header.
<gmb> deryck: So we can pretty much confirm that that's the problem.
<salgado> gary_poster, ^
<gary_poster> reading back...
<salgado> looks like the change to not accept POSTs without a referrer broke apport: https://launchpad.net/bugs/538097
<mup> Bug #538097: +storeblob fails with "500 Internal server error" on production (works on edge) <apport-collected> <lucid> <oops> <Launchpad Bugs:Triaged> <apport (Ubuntu):Invalid> <https://launchpad.net/bugs/538097>
<gary_poster> salgado: ah :-(
<salgado> gmb, ^
<deryck> hey, at least we know what happened now.
<gmb> \0/
<deryck> salgado, thanks for pointing this out to us.
<sinzui> matsubara: I cannot find the bug that is marked as bad for the registry: http://people.canonical.com/~lpqateam/test-plan-report-10.03.html . I do not see anything related to my team on https://bugs.edge.launchpad.net/launchpad-project/+bugs?field.tag=qa-bad
<gmb> (But subdued, becase, hey, bug)
<gmb> gary_poster, salgado: I'll update the bug report; want me to re-target to foundations?
<gary_poster> gmb: yes, thanks.
<gary_poster> gmb: please don't go int ospecifics; this is a security bug after all
<gary_poster> '
<matsubara> sinzui, it's fix released, that's why it doesn't show up in https://bugs.edge.launchpad.net/launchpad-project/+bugs?field.tag=qa-bad
<matsubara> (i.e. default search doesn't show fix released bugs)
<gmb> gary_poster: Okay. Should I mark it as such? It' got a lot of subscribers already so there may not be much point.
<gary_poster> Although perhaps that's closing the barn door after the animals have all escaped...
<gary_poster> gmb: no.  just assign to foundations with no explanation, perhaps.  I'll deal with it...somehow. :-)
<gmb> gary_poster: Ok.
<deryck> gmb, and ping pitti off list/channel about it, just so he knows what's up
<gmb> deryck: I think gary_poster's already taking care of that.
<deryck> gmb, ok, cool
<deryck> gmb, and let's do our call 15 after.  To give us time to close out some of this.  (and me more coffee ;))
<deryck> gmb, cool?
<gmb> deryck: Sure. Besides, I need tea.
<gary_poster> deryck, gmb: I don't have a reply from pitti on irc.  Is he known to be around-ish (i.e., he will be back soon) or should I just shoot him an email and look for another resource?
<gmb> gary_poster: Have you tried on the internal server?
<gary_poster> yeah, he just got back thanks
<sinzui> ââmars: is this bug still valid bug 435832
<mup> Bug #435832: Launchpad email verification uses sso server template <post-3-ui-cleanup> <ui> <launchpad-web:Triaged> <https://launchpad.net/bugs/435832>
<adiroiban> EdwinGrubbs, sinzui: do you know what is the revision number where the referrer/next_url mixin was introduced?
<sinzui> adiroiban: no, but I think It landed last week
<adiroiban> ok. thanks. I will search the log and the code
<adiroiban> sinzui: do you have any hints how / where should I look for that mixin?
<salgado> adiroiban, tests are passing now; I've submitted your branch again
<adiroiban> salgado: thanks! Can you please run the test using screen as some tests were not related to âactiveâ attribute and they were still failing on ec2-test
<sinzui> adiroiban: ReturnToReferrerMixin
<salgado> adiroiban, too late, but you don't have to worry now -- if there are spurious failures I'll fix them and then land your branch
<sinzui> from canonical.launchpad.webapp.launchpadform.ReturnToReferrerMixin
<sinzui> ^ adiroiban
<adiroiban> sinzui: thanks!
<adiroiban> sinzui: ReturnToReferrerMixin is not working for my bug
<adiroiban> as the referrer url can be made invalid by the changes from the form
<sinzui> :(
<adiroiban> also looking at ReturnToReferrerMixin, it looks like it is not checking if the referrer URL is identical with the current URL
<adiroiban> and preventing a redirection to the same page
<kfogel> adeuring: https://code.edge.launchpad.net/~kfogel/launchpad/534736-patches-portlet-first/+merge/21377  (when you get a chance)
<adeuring> kfogel: sure, let me first do some organsational stuff for the mini sprint. I have to use a travel agency that is absolutely unresponsvie to emails ;)
<kfogel> adeuring: sure.  You know, I've really found that the travel agencies cost more time than they save.
<adeuring> kfogel: you have point there...
<adeuring> kfogel: r=me
<kfogel> adeuring: thank you
<kfogel> adeuring: I'm going to go with ui=none, unless you counsel otherwise.
<adeuring> kfogel: I think that's fine.
<kfogel> adeuring: I just got a testfix mode failure:
<kfogel> Command failed!
<kfogel> All lines of log output:'Commit message [[r=abel][bug=534736][ui=none] On a bug page,\n\tlist patches before non-patch attachments.] does not match commit_re [(?is)^\\s*\\[testfix\\]\\s*\\[(?:release-critical=[^\\]]+|rs?=[^\\]]+)\\]]'
<kfogel> adeuring: ...from PQM.  Any idea what's up?
<adeuring> kfogel: seems that buildbot failed recently
<adeuring> that's not related to your branch
<jtv> bigjools: I'm solving the UI's inability to deal with non-Soyuz buildfarmjobs by adding some interfaces and templates...  wgrant turned out to be thinking in the same direction, as per our discussions this weekend.  How would you feel if I added browser & templates directories to lib/lp/buildfarm?
<bigjools> jtv: I would be against that
<bigjools> but you could persuade me
<jtv> bigjools: there's not a lot of the stuff now (after some simplification) but it'd be a more appropriate place I think.
<bigjools> jtv: we already have lp.buildmaster, or are you talking about a new module?
<jtv> bigjools: sorry, lib/lp/buildmaster is what I meant
<bigjools> jtv: ok, the /builders and builder-related pages could all be moved there
<jtv> phew :)
<jtv> Not that I'm looking forward to the job, but the end result seems more desirable.
<jtv> This sort of validates my sanity.  :)
<bigjools> jtv: I'd like to see this discussed on the mailing list though
<bigjools> so I can see more details
<jtv> bigjools: sure.  Still waiting for responses to this weekend's design proposal though.
<jtv> (hint, hint ;-)
<bigjools> ah hrm yes
<bigjools> jtv: the refactoring of the "name" stuff is a good plan but I need to re-read your email to get a better handle on things
<jtv> I should add that the complexity of the existing design became a bit more explicit with a branch I just landed:
<bigjools> jtv: I don't know why that cross check is so complicated either
<bigjools> but it's there to protect against builders appearing and disappearing with random builds
<jtv> Heretofore it was assumed that one buildfarmjob's "self.build.id" might be completely unrelated to another's, since their self.build objects might be in different db tables.
<bigjools> and to prevent the wrong build being assigned to a source
<jtv> Well, the buildqueue check would do thatâbarring determined attack (which would have only a tiny chance of messing things up anyway aiui)
<bigjools> well buildqueue will be common
<bigjools> I can't remember the current format, it names directories buildid-buildqueue-id-something doesn't it?
<bigjools> I suspect it was originally belt and braces as you say
<jtv> that, grumble, depends on the fritzin' type of the confounded thing
<jtv> No directories are involved that I know of, but Build uses <Build.id>-<BuildQueue.id>
<jtv> Sorry, a binary package build uses that.
<jtv> Then there's other types that are encouraged to use <BuildBase.id>-<BuildQueue.id>, where BuildBase doesn't really have an id but any of its implementing leaf classes will.
<jtv> And then of course there's us folks who don't have a self.build
<jtv> ...so we use <Branch.name>-<BuildQueue.id> instead, mainly to help debug.
<jtv> I'm suggesting to use <BuildQueue.id>-<hard-to-guess-string>
<jtv> ...without going into any specifics of different build-farm jobs at all.
<jtv> bigjools: wgrant, bless his toes, said something like "this dates back to the beginning of time, remember."  I would like to state once again for the record that I am not that old.
<jtv> I did mention that I could ask barry though.
<bigjools> jtv: you really should have a build table
<bigjools> for history
<bigjools> I talked about this with noodles
<bigjools> that doesn't preclude this change though
<jtv> well, there's all this talk of different ways we could store history...  what I'm doing with the UI atm will facilitate that, I think.
<jtv> I'm using inheritance between interfaces to achieve a kind of polymorphism in displaying things, assuming I'll always get the view for the most-specific applicable interface.  Is that bad?
<bigjools> jtv: once your job is finished, there will be no permanent record of it.  You need a build table so we can see the history of a builder.
<jtv> bigjools: I know, yes.  But as you say that doesn't preclude the simpler slave build id, and I think the UI work is worthwhile regardless as well because it gives us a base for having specific UI (elements) for different types of build record.
<bigjools> jtv: did you discuss that with noodles?
<jtv> bigjools: no, not since the weekend
<jtv> bigjools: I'm not proposing to implement a full UI here, just taking away breakage from "buildfarm == soyuz" assumptions.
<jtv> But it does have useful little things like a link formatter for builds.
<jtv> Taking some logic out of the TAL.
<jtv> Hmm...  for the record, I did not mean to imply that the TAL needs to be less logical.
<bigjools> jtv: it could be more <something> though :)
<jtv> bigjools: you mean, less can be more?  :)
<bigjools> less is more - it's my favourite pager
<rockstar> salgado, hep me! I can't build. http://pastebin.ubuntu.com/395727/
<rockstar> Or maybe anyone from foundations can help. ^^
<salgado> rockstar, I know absolutely nothing about gpgme. are you on Lucid?
<rockstar> salgado, nope,  Karmic.
<salgado> rockstar, looks like you don't have launchpad-dependencies installed
<rockstar> salgado, Oh, crap.  I wasn't in my development chroot.
 * rockstar facepalms.
 * rockstar gets up to go on a walk and get re-focused
<james_w> how do I run all the tests in lib/lp/code/stories/branches/?
<beuno> james_w, ./test -tv code.stories.branches?
<beuno> maybe, I don't remember it super well
<james_w> beuno: nope, doesn't run any tests
<james_w> nor lp.code.stories.branches or lib.lp.code.stories.branches or lib/lp/code/stories/branches/
 * james_w tries a more brute-force way
 * mwhudson_ watches his other machine fsck
<mwhudson> bit odd though, it just rebooted
<Ursinha-afk> sinzui: hi
<sinzui> hi Ursinha
<Ursinha> sinzui: matsubara-afk asked me about that bad item that was showing up in registry count, it seems that the script was considering salgado in registry team, but I've checked the config file and he was in the proper team; I've stopped and started the script again and the bad item is in its right place
<sinzui> oh..yes I consider salgado being misclassified, but he wasn't last month so I assumed I was being confused
<Ursinha> sinzui: that's weird because I've put him in the same team he was last month
<Ursinha> this was spurious
<sinzui> okay
<Ursinha> mwhudson: hi, is bug 536486 really qa-ok? by your comment it seems it's still not working as it should..
<mup> Bug #536486: Svn import of ruby taking 2.7G RSS <qa-ok> <Launchpad Bazaar Integration:Fix Committed by mwhudson> <https://launchpad.net/bugs/536486>
<mwhudson> Ursinha: i think i hijacked the bug to link a branch to it inappropriately
<mwhudson> (it turns out)
<mwhudson> Ursinha: i'll fiddle things around
<EdwinGrubbs> sinzui: ping
<Ursinha> thanks mwhudson
<sinzui> Hi EdwinGrubbs
<jml> g'night floks
<EdwinGrubbs> sinzui: on the mailing list, interest was expressed in removing the "Register a blueprint" link from the involvement portlet. The official_blueprints boolean appears to only be useful for showing/hiding that one link on the index page. Can I remove it from the form so it's impossible to edit, since "Register a blueprint" is always visible on blueprints.lp.net?
<sinzui> EdwinGrubbs: I do not think so. I still think most people were confused by your proposal
<sinzui> EdwinGrubbs: That link to enable blueprints is only for the project owner. He cannot find it to enable or disable it
<sinzui> EdwinGrubbs: So if we remove the proposal for the owner see how to control blueprints on his project, how will he ever accomplish his task?
<sinzui> EdwinGrubbs: I think the confusion is that jml and poolie thought you were going to let any person use that link
<jml> sinzui, that's not what I was confused about, I think.
<mwhudson> Ursinha: bugs fiddled, i think
<EdwinGrubbs> sinzui, jml: it sounded like going to the blueprints tab is the desired place for the "Register a blueprint" link since registering a blueprint is not an important enough activity to be on the index page.
<jml> EdwinGrubbs, yeah, that's what I meant.
<jml> EdwinGrubbs, it's orthogonal to the other things you were discussing.
<sinzui> jml:setting aside my dislike for blueprints, I see that some project like the app, and project owners should not have to hunt through the UI to enable it. conversely, project owners that are disappointed about the app should be able to disable it.
<jml> sinzui, I suggest that we make it a little hard to enable blueprints for now.
<sinzui> Why is blueprints different from ask a question or register a bug?
<sinzui> jml: it is pretty fucking hard right now
<jml> sinzui, because the app isn't very good.
<jml> sinzui, heh heh
<sinzui> In fact only someone who has setup a project before, probably in 2.0 can enable any application
<sinzui> jml: EdwinGrubbs: I am looking at https://blueprints.edge.launchpad.net/subunit
<EdwinGrubbs> it is odd how "Register a blueprint" appears twice
<sinzui> ^ blueprints is not an official app. the Involvment link to register a blueprint be removed from the blueprint page?
<sinzui> EdwinGrubbs: yes it is weird. I think is a UI inconsitency. The bugs team replaced the action buttons with involvement links. I updated the other apps to do the same, but I think the action link is more appropriate
<sinzui> EdwinGrubbs: The bugs team went farther last month when they removed the entire bugs page if the app is not official
<jml> yeah...
<jml> sinzui, did you see the registry bugs I filed recently?
<sinzui> jml: my email is littered with your email filings. I do not know which one you are referring to
<jml> sinzui, me neither :)
<jml> sinzui, don't worry about it then.
<sinzui> jml: you want to say your project does not have a source package in ubuntu?
<sinzui> jml: or The ajax ok/cancel button is rendered poorly when the primary context's title is edited
<sinzui> jml: Project description and summary field could explain formatting rules
<sinzui> which relates to a markup bug you moved to launchpad-web
<jml> sinzui, let me find it. it's none of those. I think it's to do with enabling bugs for a project
<sinzui> man, there are really old bugs related to that. I found them in malone
<jml> oh wait, that's a malone thing
 * sinzui assigned two to himself last week
<sinzui> jml: I have a prototype single bug configuration screen
<jml> anyway, we have a real inconsistency when it comes to enabling applications for a Launchpad project
<jml> it looks and feels really sloppy.
<sinzui> single screen that allows you to configure bugtracking
<jml> sinzui, that'd be awesome
<sinzui> I was very tempted to propose we abandon the bool states last week
<jml> sinzui, I'd like more elaboration before you do that, but it sounds like the right line of thinking to me.
<sinzui> jml: and it is hard to make apps behave consistently when some apps have lots of community involvement, and by the fact that two of our apps are not owned
<jml> sinzui, I'm not sure that's the reason.
<sinzui> How do you deal with requests from project owners to disable the branch tab and remove all the branches...including the ones not owned by the owner?
<jml> sinzui, now _that's_ a more compelling reason :)
<jml> sinzui, the short answer is "say 'no'"
<sinzui> That is what we say. The project owner thinks of launchpad has a project hosting service
<jml> sinzui, the longer answer is find out why they want it
<sinzui> jml: In most cases it is because the owner prefers git or subversion, in fact since he thinks launchpad is a project hosting service, he want the code removed
<jml> sinzui, you are saying that they don't actually have a benefit in mind when they ask for it?
<sinzui> jml: yes
<sinzui> I think some were concerned that launchpad always shows up first in google listing
<jml> sinzui, hmm
<sinzui> Others were worried about old code confusing users
<jml> sinzui, there's a blog post in that.
<sinzui> but the latter case is always addressed by switching to an improt
<sinzui> import
<jml> sinzui, I'm less concerned about the old code reason -- it's easy enough to update trunk to point at the right branch if you are a project owner.
<jml> sinzui, right.
<sinzui> EdwinGrubbs: I think there are two action regarding blueprints. One, do not add a link to configure blueprints. two I will find the bug about the blueprints tab and update it to include disabling the feature when it is not configured...and provide a link for the owner to enable it
<EdwinGrubbs> sinzui: do I move the "enable blueprints" boolean back to the +edit form, or do I just make it inaccessible by removing the "configure blueprints" link?
<sinzui> EdwinGrubbs: back to +edit.
<EdwinGrubbs> ugh
<sinzui> EdwinGrubbs: do you have a separate for for blueprints?
<sinzui> separate form?
<jml> anyway guys, I'm gone for real
<jml> good chatting.
<EdwinGrubbs> sinzui: yep, I have a separate form for all the services. Except for bugs, they are just one boolean. Though this is weird, I think it is better than searching through all the +edit fields.
<sinzui> EdwinGrubbs: that is fine, great in fact.
<EdwinGrubbs> sinzui: so, what do I do about the "configure blueprints" link and field then?
<sinzui> EdwinGrubbs: We can add a link to the blueprint app menu for product. Make sure the permissions are launchpad.Edit
<sinzui> EdwinGrubbs: It should be the first link in the list
<sinzui> EdwinGrubbs: I am looking at my own page: https://blueprints.edge.launchpad.net/gdp
<EdwinGrubbs> sinzui: although I am risking running the conversation in circles, it's odd to have the "configure blueprints" link have no effect on blueprints.launchpad.net, but it will show/hide the "Register a blueprint" link in the involvement portlet.
<sinzui> EdwinGrubbs: It does have an effect, it make the Register a blueprint link appear in the involvement portlet
<sinzui> EdwinGrubbs: And by landing your change, it make it easier to disable the blueprint root page like bugs does.
<EdwinGrubbs> sinzui: but only on the index page. The blueprints.lp.net/$project page always shows "Register a blueprint". I haven't done any work on disabling the blueprint root page. Has that already landed? Maybe I just need to merge in RF again.
<sinzui> EdwinGrubbs you are definitely confused is said "I" would find the blueprint bug and update it. You are NOT doing anything disabling the root
<sinzui> EdwinGrubbs: Do not render the configure link blueprints link in the involvement portlet as jml asks.
<sinzui> EdwinGrubbs: Do add the link as the first link the project blueprints menu
<sinzui> EdwinGrubbs: If the second point bothers you, you can put the official_blueprints back on +edit
<EdwinGrubbs> ok
<sinzui> EdwinGrubbs: want to keep the scope of this exception out of your work
<sinzui> Someone has to volunteer to change blueprints and maybe answers. We had no volunteers last month
<james_w> wow, branch-rewrite.py imports windmill
<mwhudson> james_w: launchpad is a tangle :(
<EdwinGrubbs> sinzui: was that a request for volunteers?
<sinzui> EdwinGrubbs: no, just a statement about the likely hood the blueprints or answers getting changed
<wgrant> Are we in testfix?
<wgrant> I have two "submitted to PQM" ec2 emails from 5 hours ago.
<wgrant> Neither of them have actually landed.
<wgrant> james_w: There's a bug for that branch-rewrite.py Windmill PATH issue.
<james_w> good
<wgrant> gary_poster: Since I guess Launchpad won't have bothered to email you, there is a new comment on https://bugs.edge.launchpad.net/launchpad-foundations/+bug/529348
<wgrant> james_w: There is a workaround that I've been using for a while on https://dev.launchpad.net/Translations/BuildTemplatesOnLocalBuildFarm
<james_w> I didn't need it to test this change, but thanks
<wgrant> Ah.
<gary_poster> wgrant: I had not received an email, oddly.  You know, you could have requested a patch.  Thanks for the comments.  The sarcasm is perhaps less warmly received.  :-)  I'll contact you privately, because I'm afraid I don't see how the webservice aspect can be addressed.
<wgrant> Yeah, sorry about that. I was pretty displeased.
<wgrant> gary_poster: (it's not odd that you haven't received an email; ~launchpad-security emails go through ~launchpad, which redirects all emails to launchpad-bugs@lists.c.c, which nobody reads)
<gary_poster> ah :-/
<wgrant> Why yes, this does mean that nobody gets automatically notified of new security issues.
<NCommander> BjornT: re: raw source changelogs, I don't mind rewritting the code to use librarian, but I fear hitting the librarian to pull it on the fly and stick it in the UI might really put a strain on loading an Soyuz page that needs the raw changelog
<lifeless> NCommander: why are you worried about that?
<lifeless> NCommander: as in, do you have a metric that says its a concern?
<NCommander> lifeless: no, but we keep the debian/copyright in the database. my knowledge of librarian is fairly limited (I'm still getting my around to LP source trees)
<lifeless> so the librarian is pretty fast
<lifeless> and there is a squid in front of it to boot
<NCommander> lifeless: ah. I didn't want to be the one that causes a change to land that causes LP to DOS its own librarian :-)
<lifeless> the librarian is just a big disk, with stuff named by hash, and a front end that takes urls in and serves the files from the disk.
<lifeless> NCommander: bug attachments, all packages (ever), crashdumps - they all go in the librarian
<wgrant> lifeless: But none of those are ever rendered in the UI.
<lifeless> wgrant: no, but the load on the librarian is related to use, of which rendering is one
<lifeless> the key questions are:
<lifeless>  - how many loads per second are you expecting
<lifeless>  - how many loads per second the librarian already does.
<NCommander> lifeless: well, ideally, the SRP in Soyuz will pull raw source changelog directly.
<NCommander> lifeless: we also need it for generating changelog diffs as part of native source sync
<lifeless> so source sync is low frequency
<lifeless> its (at most) one every 5 seconds - if every package changed every day
<dobey> is there a good way to getting bugs for API calls escalated?
<lifeless> dobey: the most effective way is to submit a patch
<lifeless> failing that try begging, and if that doesn't work, ask your manager :P
<dobey> is there good documentation on making changes to the REST API bits? :)
<lifeless> dobey: generally you don't need to change the REST API bits at all, just the object definitions. what is your bug?
<dobey> lifeless: i haaven't filed the bug yet, as i wanted to understand more process first. but I want date filtering on {person,team}.{getMergeProposals,getRequestedReviews}
<lifeless> ok, thats just an object change
<lifeless> reflecction is used to export interface defintions to rest
<dobey> ok
<lifeless> so the bug belongs on the relevant team
<dobey> not being a lp developer, i am not familiar with these things :)
<lifeless> in this case, launchpad-code
<lifeless> dobey: et moi
<lifeless> dobey: anyhow, launchpad-code, and if you haven't filed it, I wouldn't even consider escalation yet :P
<dobey> lifeless: i'm just wondering what the process is :)
<henninge> wgrant: your branches passed ec2 testing but did not land because of testfix. They are in pqm now.
<henninge> testfix mode
<wgrant> henninge: Ah. Thanks.
<elmo> I'm getting unexpected form data trying to update bugs in LP, specifically using +editstatus slider
<elmo> is that a  known thing?  this is on prod
<wgrant> elmo: You block referers?
<elmo> wgrant: yeah
<wgrant> elmo: LP now hates you.
<wgrant> https://bugs.edge.launchpad.net/launchpad-foundations/+bug/529348
 * mwhudson lunches
#launchpad-dev 2010-03-16
<thumper> rockstar: ping
 * thumper does TZ math
<rockstar> thumper, hi
<thumper> rockstar: it's getting late for you right?
<thumper> rockstar: bug 527450
<mup> Bug #527450: email unsubscribe link is fragile (broken-ish) <email> <notification> <qa-needstesting> <trivial> <Launchpad Bazaar Integration:Fix Committed by rockstar> <https://launchpad.net/bugs/527450>
<thumper> rockstar: are you able to qa this?
<rockstar> thumper, I just got back from a bike ride.
<rockstar> thumper, I think I can.  I have to get the imap mail thing working again, which is always a PITA.
<thumper> rockstar: can you put qa onto your TODO list for tomorrow morning?
<rockstar> thumper, yessir.
<thumper> rockstar: awesome, thanks
<rockstar> (sorry you had to remind me)
<mwhudson> thumper, rockstar: can i beg a couple of reviews off you?
<thumper> mwhudson: hit me
<mwhudson> thumper: https://code.edge.launchpad.net/~mwhudson/launchpad/fewer-spurious-partial-code-imports-bug-532402/+merge/21416
<rockstar> mwhudson, you can, but I'm upgrading to Lucid right so...
<mwhudson> thumper: https://code.edge.launchpad.net/~mwhudson/launchpad/back-off-failing-imports-bug-413637/+merge/21411
<mwhudson> (jml already sort of half reviewed the last one)
<mwhudson> thumper: ta
<mwhudson> thumper: can you approve the merge proposals too?
<mwhudson> or i can of course...
<Ursinha> Â²Â²Â±/16
<thumper> Ursinha: eh?
<Ursinha> thumper: nevermind my irssi fail
<thumper> Ursinha: you just had me confused
<Ursinha> thumper: sorry
<Ursinha> :)
<wgrant> jtv: Has there been any further discussion about what to do about generalised builder histories?
<wgrant> (in particular, has there been any discussion about making the model suck less?)
<jtv> wgrant: hi!  apparently the thinking right now is "just implement buildbase somehow"
<jtv> Maybe we should just start out by renaming BuildBase to BuildRecord and Build to PackageBuildRecord or something.
<wgrant> Hmmm. That seems silly.
<jtv> BTW you said that the builder UI also stumbles over recipe builds, right?
<wgrant> It won't crash any more (they have a canonical URL as of a few days ago).
<wgrant> I had a little play around with remodeling the model to remove duplication and actually allow generalised histories to work. Some of the output is at http://people.ubuntu.com/~wgrant/launchpad/buildfarm/
<jtv> wgrant: so the idea is to split Build into "build attempt," "binary build," and "source package build"?
<wgrant> jtv: I'd like to store attempts, but others probably don't, so there's one without that too.
<wgrant> (at the moment if a build fails, clicking "Retry" will reset and obliterate the build's history. this can make things very opaque)
<jtv> Oh, that's exactly the kind of situation where I'd have expected Build to be useful as an historical record.
<wgrant> We can't really create a new Build, I don't think.
<wgrant> Although that might work better
<wgrant> Hmm.
<wgrant> Excluding UI problems, I think the Soyuz model might actually cope with having duplicate Builds.
<jtv> Why have a BinaryPackageBuild.buildstate and a SourcePackageRecipeBuild.build_state?
<wgrant> Because I haven't resolved the unresolved issue.
<wgrant> And I apparently forgot to rename the attributes that I copied from the original classes.
<wgrant> But the whole result thing needs to be rethough. SPRB and BPB can share them. I'm just not sure where to put it.
<jtv> Another point is that what we are working on is not a "package" build.
<wgrant> "we"?
<wgrant> Do you refer to the template jobs?
<jtv> Right.
<wgrant> I don't see any implication in the current model or my experiments that it is.
<jtv> Well, none of PackageBuild would be applicable to us.
<jtv> Okay, "job" would.  :-)
<james_w> wgrant: hey, do you have a pointer to the object served as the root over the API? I'm looking to export a new collection, it's in the wadl, but I need to serve the link to it in the root.
<wgrant> jtv: That's why it's not connected to any of the Translations bits.
<jtv> wgrant: so if we wanted to keep history for the translations jobs as well, where would it fit into your model?
<wgrant> james_w: Which collection?
<james_w> wgrant: code-imports
<wgrant> jtv: TTB
<jtv> oh, I had been completely ignoring the lhs!
<wgrant> james_w: I don't think you should export that. Code imports exist under branches, and should be created from under productserieses.
<wgrant> I don't think exporting the root collection is useful.
<james_w> wgrant: productserieses/distributionsourcepackage?
<wgrant> jtv: Heh.
<jtv> wgrant: I now see why.
<wgrant> james_w: Hmmm.
<jtv> wgrant: I don't see any meaningful difference between TTB and TranslationTemplatesBuildJob.
<james_w> wgrant: what I want to achieve is to be able write a script that syncs Vcs-* with LP
<wgrant> james_w: Expose it on IBranchTarget or whatever it is, then.
<james_w> ok
 * james_w changes direction
<wgrant> jtv: TTBJ doesn't exist, does it?
<jtv> wgrant: does too!
<wgrant> I thought it was just Branch<->BranchJob<->Job<->BuildQueue
<jtv> In the database, yes.
<jtv> In terms of storage, a TTBJ is just a BranchJob.
<jtv> But it's also a BuildFarmJob.
<wgrant> james_w: Root collections should be avoided if at all possible. Both operations (retrieving and creating code imports) have logical parents, so they should be exposed there.
<james_w> wgrant: right
<wgrant> jtv: OK, so the meaningful difference is probably that TTBJs die when the operation completes.
<wgrant> And the current model has too many duplicated classes, so I want to kill it.
<jtv> wgrant: right, so the TTB would simply be a copy of the TTBJ
<wgrant> jtv: TTBJ ceases to exist.
<james_w> wgrant: so, code imports should be retrieved by getting a subordinate URI of the branch they import to?
<jtv> wgrant: TTBJ shares a table with all sorts of other jobs of different types, so I don't think we should dictate our lifetime semantics to it.
<mwhudson> james_w: my understanding was that someone tried that and lazr.restful stabbed them
<mwhudson> james_w: but noone's completely sure any more :-)
<wgrant> mwhudson: '@stepto("+code-import")'?
<wgrant> jtv: I mean, why does TTBJ need to exist if TTB does?
<wgrant> Is there any point in it sharing the table?
<wgrant> Same for Job itself.
<james_w> mwhudson: I am equipped with a stab-proof vest, my naivety, and a glass of wine, I hope to prevail
<mwhudson> wgrant: eh?
<wgrant> mwhudson: Do you recall what the problem was?
<mwhudson> wgrant: no
<wgrant> It seems like one could just add @stepto("+code-import") to BranchNavigation, and it would have to work.
<mwhudson> i don't think i ever knew
<wgrant> I recall I brought it up in Wellington one day, and nobody knew there either :(
<wgrant> Which is odd, since all of Code in recent memory was there.
<mwhudson> something to do with having an entry as part of another entry rather than as part of a collection
<mwhudson> james_w: please try, and if it fails, please explain what happens in the bug report
<james_w> I know the API well from the client-side, but know little about implementing it server side, so this is partly a learning exercise
<jtv> wgrant: wouldn't that mean that TTB represented a past, ongoing, or future build whereas the other Build types represent only past or ongoing ones?
<wgrant> jtv: The other Build types can never be removed.
<mwhudson> the bug btw https://bugs.edge.launchpad.net/launchpad-code/+bug/366102
<mup> Bug #366102: API for creating and managing code imports <api> <code-import> <package-branches> <udd> <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/366102>
<wgrant> If they succeed, they are deeply integrated into the terrible web that is the Soyuz data model.
<wgrant> They are never removed.
<wgrant> Oh, misread.
<wgrant> jtv: I don't see how the futureness is different.
<wgrant> A TTB will start to exist moments before it is queued.
<wgrant> SPRBs and BPBs do as well.
<jtv> wgrant: I thought that was in the same transaction that also dispatched it.
<wgrant> jtv: No. They are created by any one of a number of processes on several different machines.
<jtv> Does build candidate selection etc. use Build?
<wgrant> Yes.
<jtv> I see.  Then it's not really different.
<wgrant> Exactly.
<wgrant> No class in my non-current diagrams gets deleted.
<wgrant> (except when a branch gets deleted, but mrrhrhrrhrhr)
<wgrant> Whereas all the common classes in the old one do.
<jtv> That's the kind of case where I'd be in favour of either cascading deletes or a denormalized history tableâthe latter being something we should do only if there is a proven need.
<wgrant> I think we should probably just ban branch deletes in some circumstances.
<wgrant> What happens if I try to delete a branch referenced by a recipe referenced by sourcepackagerecipebuilds with dozens of sourcepackagereleases in dozens of archives?
<mwhudson> we have a kind of denormalized history for code imports
<mwhudson> completely unused :/
<wgrant> Heh.
<jtv> wgrant: so what?  If the recipe is basically there to service the branch, it should die.  The SPRs don't depend on the recipebuilds, do they?
<wgrant> jtv: SPR references SPRB.
<wgrant> Deleting an SPRB removes all accountability information for any SPR built from it.
<jtv> wgrant: and the reference from SPR to the branch does not go through the recipe?
<jtv> Because if it does, well, cascading nulls solve it very nicely.  :)
<wgrant> It does. But then you end up with a broken recipe.
<mwhudson> grar
<wgrant> Hm?
<mwhudson> i revoked ec2 lands oauth token and now it's broken
<mwhudson> do i have to clear some cache somewhere?
<wgrant> Hah.
<wgrant> Try removing the launchpadlib cache, i guess...
 * mwhudson finds /home/mwh/.launchpadlib/api.edge.launchpad.net/credentials/launchpad-branch-lander
<wgrant> jtv: Anyway, any more impressions of the model?
<mwhudson> and a few other similarly named files
<jtv> wgrant: contrary to all experience so far, emotionally I still have trouble believing that something so much simpler can work.  :-)
<wgrant> Haha.
<wgrant> The current one has too many arrows.
<wgrant> It makes me sad.
<mwhudson> oh what now
<jtv> wgrant: mwhudson is from a place where every place where you might conceivably want to cross a street has a big arrow saying which direction to look in.
<mwhudson> jtv: i think that's only london
<jtv> mwhudson: oh, London stops somewhere before the coast?  I never really noticed.
<jtv> wgrant: the current one has enough arrows to scare us out of cleaning up.
<wgrant> True.
<jtv> So I think it's a great thing you're doing, but it looks like one hell of a job.
<james_w> mwhudson: I'm starting to see the issue
<wgrant> jtv: Well, we need to get some centralised permanent history table for builder history to work.
<wgrant> And it's not that much more work to fix the rest up.
<mwhudson> james_w: :(
<wgrant> james_w: What's going wrong?
<james_w> you want to export an entry that is subordinate to another entry, so lazr.restful dutifully tries to create a link for you
<wgrant> "create a link"?
<james_w> oh no, I've confused myself
<james_w> it seems that wine was not my ally on this adventure
<wgrant> Heh.
<mwhudson> :)
<james_w> I'm stuck trying to work out how to export() code_import on IBranch though
<james_w> as it's just Attribute currently, apparently due to circular import issues.
<wgrant> code_import = exported(Reference(..., schema=Interface))
<james_w> I've seen workarounds for that elsewhere
<wgrant> Then patch it in _schema_circular_imports.py with the rest.
<james_w> ah, I was wondering if it had to be a *Choice
<wgrant> I don't believe so.
<wgrant> I think ReferenceChoices are only used for fields editable in the UI.
<james_w> ok
 * wgrant heads trainward.
 * mwhudson stops too
<wgrant> spm: How much RAM does crowberry have to be able to survive two >13GiB RSS processes!? The bug says 8GiB, but that can't be right...
<jtv1> hi al-maisan!  Say, do you know of any pagetests for the access check on build logs?
<wgrant> Access check?
<al-maisan> jtv: not off the top of my head ..
<al-maisan> page tests that verify the access to build logs I assume..?
<jtv1> wgrant: yes, the bit that hides build logs when private.
<wgrant> jtv1: The build itself is hidden, and the log is accessed by a proxying view on Archive. I don't see how the latter is relevant to your work.
<jtv1> wgrant: there are more places that access the log, without going through archive.
<jtv1> Builder page.
<wgrant> jtv1: All build details are hidden from the builder page when the build is private. There's no special handling for the log.
<wgrant> Oh, unless you mean logtail?
<StevenK> Ddoes anyone know off hand how to run the test suite for gina? test_gina.py is being unhelpful.
<wgrant> StevenK: IIRC I've just 'bin/test -vvt gina' in the past.
 * wgrant tries.
<wgrant> That looks like it's working.
<StevenK> I was trying bin/test -vvt test_gina, which just did nothing
<wgrant> Ah.
<wgrant> Most of them are doctests.
<wgrant> So that won't catch them.
<wgrant> You're fixing it too like multiple published versions of the same source?
<StevenK> Yeah
<wgrant> StevenK: Which archive admins use the queue web UI?
<wgrant> It's been requested that I fix it to show package sets, and I'm wondering who is likely to kill me if they disagree.
<StevenK> wgrant: The ones that don't have shell on cocoplum, so probably just ScottK
 * persia has also seen folk with shell use non-shell for trivial tasks
 * wgrant too.
<jtv1> wgrant: that's right, the logtail...  I'm moving that check into security.py.
<wgrant> jtv1: Ah. Ew.
<wgrant> (good idea)
<jtv1> wgrant: the two opinions seem ill-matched.  :)
<persia> wgrant: You might check with infinity also, although I haven't seen much activity from him in a while.
<wgrant> jtv: The current status is ew. Fixing it properly is a good idea.
<jtv> ah :)
<wgrant> persia: infinity has been gone for a long time now.
<persia> He's still listed as archive-admin though.
<wgrant> Hm. that seems odd.
<StevenK> And a member of ubuntu-cdimage
<persia> He's just no longer a buildd-admin
<jtv> wgrant: I'm making the view use View permissions for IBuildFarmJob, which I newly defined & provided with checks for private builds & branches.
<wgrant> jtv: It's probably untested.
<wgrant> I can't see any tests for it.
<wgrant> And it was Soyuz until recently.
<jtv> wgrant: oh well, then I guess I'll write some.
<jtv> (grumble grumble bloody nuisance)
<wgrant> persia: He is still a buildd admin, though.
<persia> Ah.  I thought he stopped being something.  Maybe I was misinformed.
<wgrant> An admin, perhaps.
<persia> perhaps.
<jtv> wgrant, al-maisan: xx-private-builds.txt tests for access.
<al-maisan> jtv: thanks for the pointer!
<jtv> I think I'll write up an MP now before I go mad with it all.  :-)
<wgrant> henninge: Thanks for landing those.
<henninge> wgrant: you are welcome
<adeuring> good morning
<stub> intellectronica, gmb: Is there anything to guarantee that AportJob rows eventually get removed? I need to know if the Librarian garbage collector should not collect blobs linked to AportJob rows, or if it should delete the corresponding AportJob and Job rows in addition to the blob.
<wgrant> A branch I'm about to propose for review has new config keys and a new script that needs to be run regularly. What special stuff do I need to know?
<bigjools> morning
<wgrant> Morning.
<gmb> stub: There's nothing pruning them at the moment. I can easily add something that does, though.
<gmb> stub: But if that table's bloating you can probably delete anything more than a day old.
<stub> gmb: If it is simple (delete any apportjob and job rows linked to a blob older than a day or three) I can do that in the librariangc
<stub> gmb: Otherwise it should go in garbo.py, and possibly need to be cherry picked too to avoid the Librarian exploding.
<gmb> stub: It is. Once the Job is completed the +filebug process will use the data stored in the ApportJobs json_data field, but after that it's never used again. Librarian gc should be sufficient
<stub> ok
<mrevell> Morning
<deryck> Morning, all.
<jtv> henninge: I got a little bit further with the FSM problem on the slave side: I think things go awry because the success code becomes None somehow.
<henninge> jtv: is "None" supposed to mean "success" of "failure"?
<jtv> henninge: neither.  AFAICT it's not supposed to happen.
<jtv> 0 means success.
<henninge> jtv: I know that
<henninge> jtv: what I mean is that anything != 0 is treated as a failure.
<wgrant> So the subprocess ends without a return code?
<jtv> The code treats it as a failure; it explicitly compares to 0.
<henninge> yes, looks like it
<wgrant> Huh
<jtv> Or _somehow_ twisted feeds us a None.
<henninge> jtv: which state was it again?
<jtv> I'll double-check.
<jtv> Oh, can't check that now.  IIRC it was UNPACK.
<henninge> jtv: I did notice that "None" in there but thought it was a sign of failure
<henninge> (and it does not happen in our code ... ;)
<jtv> Maybe it was.
<jtv> Nope, that's what makes it so infuriating.
<wgrant> An early stage that's used by two other managers? Really?
<henninge> wgrant: yup
<jtv> Well, I'm sure we're _causing_ it somehow, but...
<jtv> Does twisted get a timeout somewhere, after which it passes None?
<henninge> jtv: you always feel guilty
<henninge> ;-)
<wgrant> Does a recipe build work OK on the same machine?
<jtv> henninge: sorry.
<jtv> wgrant: I haven't tried that; is there documentation on how to test that?
<wgrant> jtv: Not really, no. But a binary build is well-documented on HowToUseSoyuzLocally, and just as useful.
<bigjools> james_w: did you do any QA on your fix for https://bugs.edge.launchpad.net/soyuz/+bug/89150 ?
<mup> Bug #89150: sync-source should not assume 0 is the lowest version number possible <qa-needstesting> <soyuz-ftpmaster-tools> <Soyuz:Fix Committed by james-w> <https://launchpad.net/bugs/89150>
<jtv> wgrant: the instructions don't say anything about signing, but dput complains that the package is not signed.  I've had to look this up before, but what was the option again?
<wgrant> jtv: Give debuild '-kKEYID'.
<wgrant> jtv: Normally if the changelog matches your key it won't be necessary, though.
<jtv> wgrant: I didn't have a changelog of my own handy
<jtv> wgrant: wait... you're saying I need to rebuild the package for this?
<jtv> I thought there was a way around that...
<wgrant> jtv: debsign blah.changes
<jtv> nope, that's not it
<jtv> that still complains when I'm not in the changelog.
<jtv> ahhh, -u does it.
<jtv> gah, unable to find mandatory field 'files' in the changes file.
<wgrant> jtv: Insufficiently signed.
<bigjools> yeah that error sucks
<jtv> okay, debsign -k<me@example.com> <foo>_source.changes
<jtv> debsign: Only a .changes, .dsc or .commands file is allowed as argument!
<jtv> one gets so tired...
<wgrant> Actually, you might be able to use a policy that allows unsigned uploads now. When I started looking it forbade PPA uploads, but I might have fixed that...
<jtv> ah, I had a space after the -k
<bigjools> -C absolutely-anything
<jtv> bigjools: ?
<wgrant> Yeah, absolutely-anything will work now.
<bigjools> it's a policy for process-upload
<wgrant> jtv: If you give that to process-upload.py, it should not require a sig.
<bigjools> wgrant: as opposed to "anything" which is not anything!
<wgrant> Yep.
<jtv> aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaah!
<jtv> and just when I got my changes file signed...
<bigjools> jtv: I made that policy explicitly for testing
<bigjools> see the comment on it :)
<jtv> bigjools: and very welcome it is too...  I've added it on the wiki page.
 * wgrant fixed it a few months ago, but then forgot.
<bigjools> though I'd like to rename it to "anything" and rename the existing "anything" to what it really is
<jtv> wgrant: not getting the slave to do much...  says it's still idle.
<jtv> maybe one of my failed attempts got in the way somehow.
<jtv> Or I missed a step.
<jtv> But it's definitely not getting to the point where the translations fsm is breaking.
<wgrant> jtv: The virtualisation setting matches between the PPA and builder?
<jtv> wgrant: uhhh...  I'll have a look.
<jtv> wgrant: ppa does not require a virtualized builder; builder is virtualized.
<wgrant> jtv: Flip the PPA's flag.
<wgrant> 'Does not require a virtualized builder' means 'Requires a non-virtualized builder'
<jtv> ah
<jtv> $ ./scripts/process-upload.py /var/tmp/poppy -C absolutely-anything -vvv
<jtv> 2010-03-16 11:01:45 INFO    creating lockfile
<jtv> 2010-03-16 11:01:49 DEBUG   Initialising connection.
<jtv> 2010-03-16 11:01:49 DEBUG   Beginning processing
<jtv> 2010-03-16 11:01:49 DEBUG   Checked in /var/tmp/poppy/incoming, found []
<jtv> 2010-03-16 11:01:49 DEBUG   Rolling back any remaining transactions.
<jtv> 2010-03-16 11:01:49 DEBUG   Removing lock file: /var/lock/process-upload-absolutely-anything.lock
<jtv> wgrant: ^^^ doesn't look right, does it?
<wgrant> jtv: You've not uploaded it?
<jtv> wgrant: I dput it, and dput said it was successful.  What else do I need to do?
<wgrant> jtv: Try uploading it again, I guess (you'll probably need to give dput '-f').
<jtv> wgrant: did that several times
<wgrant> Unless poppy is being stupid...
<jtv> with different changes files
<wgrant> Are you getting any errors in the terminal in which you started poppy?
<bigjools> why are you using poppy?
<wgrant> When it works it's a trivial way to get the directory structure right.
<wgrant> But since it doesn't seem to be working...
<jtv> wgrant: I have a lot of output mixed together... when exactly should the poppy error happen?  During dput?
<wgrant> jtv: Generally at the end of dput.
<wgrant> But as bigjools says, you could work out the directory structure and avoid poppy.
<bigjools> what is in /var/tmp/poppy?
<jtv> one item (dir+file) in failed, one (dir+file) in rejected.
<jtv> I dput rather more.
<bigjools> poppy is not putting stuff where you think it is then
<wgrant> Which generally means you're getting that client accept hook failure or whatever it is.
<wgrant> Which should be spewing tracebacks everywhere.
<jtv> maybe it's in a different shell then... I have to stop now.
<bigjools> jtv: just mkdir -p /var/tmp/poppy/incoming/jtv
<bigjools> then copy your upload there
<jtv> why jtv?
<bigjools> it doesn't matter
<bigjools> anything
<jtv> ah ok
<jtv> and my upload is the changes file, dsc file etc?
<bigjools> call it I_hump_donkeys
<bigjools> yes
<wgrant> Doesn't it need an upload path?
<bigjools> this is a PPA upload?
<bigjools> then yes
<jtv> I'm afraid I really have to stop typing now
<jml> wtf https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1536EA471
<bigjools> jml: foundations bug
<bigjools> they're working on it
<jml> oh good.
<bigjools> something to do with assuming all views are LPViews
<jml> heh heh
<jml> Unable to obtain lock lp-66556816:///~launchpad/lp-source-dependencies/trunk/.bzr/branch/lock
<jml> held by leonardr@bazaar.launchpad.net on host crowberry [process #5035]
<leonardr> jml: i'm committing right now
<jml> leonardr, so I gathered :)
<leonardr> jml: all done
 * jml parties like its 1999
 * bigjools sighs at lucid bugs
<maxb> lucid bugs?
<bigjools> laptop suspending at battery low instead of critical
<bigjools> laptop not resuming from suspend 50% of the time
<bigjools> fsck at boot freezing
<jtv> henninge: I filed bug 539499 about my FSM problem.  I'll go to my office tomorrow and see how things work out on a different machine.  If you have any notes, please add them!
<mup> Bug #539499: Build-farm slave FSM fails early on <buildfarm> <Launchpad Translations:New> <https://launchpad.net/bugs/539499>
<wgrant> jtv, henninge: I ran a template job through from the master a few days back (just as one of henninge's branches was landing), and it went through fine (except for not actually generating a template tarball, since that wasn't implemented yet)
<wgrant> So it's either something broken on your machine, or something very recently broken.
<jtv> wgrant: so intltool was actually run on your branch?
<wgrant> jtv: Yes.
<wgrant> It printed out the template paths and everything.
<jtv> wgrant: that's great news, thanks.  This was the Debian-based FSM, right?
<wgrant> jtv: Uh, not sure. I merged the henninge branch that you had linked to on the wiki page.
 * wgrant will check logs tomorrow.
<wgrant> But... sleep.
<jtv> thanks.  yes.  Very glad to hear this.
 * jtv really buggers off
<wgrant> (it worked sufficiently that I could just push a branch, run 'make sync_branches', and it would print out the template names.
<wgrant> Night.
<jtv> cool cool cool sweet dreams
<henninge> wgrant: yes, printing out template names is success. cool!
<henninge> g'night
<sinzui> salgado: ping
<salgado> hi sinzui
<sinzui> salgado: I am seeing several questions about how to change launchpad passwords. I think I should update the FAQs about this. Maybe you are planning to update the user profile page to explain that accounts/emails/passwords are managed at login.ubuntu.com?
<salgado> sinzui, I didn't have plans to do it, but I could do so.  also, it's only the password that is managed by login.lp.net; email addresses are still managed by LP
<sinzui> understood
<sinzui> salgado: Can we do something as simple as a sentence:
<sinzui> "Manage your Ubuntu account and password > login.launchpad.net"
<salgado> sinzui, I suppose so, but I think we should s/Ubuntu/Login Service
<sinzui> I think that will lead to confusion. Changing my Launchpad password is changing my Ubuntu password isn't it?
<james_w> bigjools: how would I QA sync-source.py?
<james_w> bigjools: or is it on production?
<salgado> sinzui, login.u.c and ubuntu accounts are not used/mentioned when logging into LP, but I'm not sure what's the plan on the SplitIt front -- can you check with flacoste?
<sinzui> I will
<flacoste> salgado, sinzui: they should be mentioned since they are the same thing
<flacoste> a Launchpad login service account is really a Ubuntu account
<sinzui> flacoste: I was thinking to using a sentence on user +edit sending the user to l.l.n. We want to update the header/footer of l.l.n to state the service is provided by l.u.c
<flacoste> sinzui: right, can you file a bug about this on the ISD project?
<sinzui> I will
<bigjools> james_w: I can run it on dogfood, I was just wondering if you had a test case or had already tested it
<james_w> I don't know one offhand, and I haven't tested it
<james_w> let me see if I can find one
<james_w> bigjools: I can't find anything waiting to be synced from Debian that has an appropriate version number
<bigjools> james_w: :(
<kfogel> vila: so I think I finally have it working.  What's weird is I successfully pushed a commit to a branch (up to a server using the kind of authn we were talking about), and on the server side I can't figure out where the actual commit data lives.  See http://paste.ubuntu.com/396199/ -- just out of academic curiousity, what file under .bzr has the commit?  I created a file named "fish.txt" with the text "Hello, world!", and it's hard to see, even with
<kfogel> compression, where that data is.
<kfogel> vila: mind you, it *is* working.  If I branch that branch again, the data is there.
<vila> kfogel: that's because there is no working tree here, try 'bzr co .'
<vila> kfogel: or 'bzr cat fish.txt'
<kfogel> vila: yeah, no working tree, but the data has to be *somewhere* in there.
<kfogel> vila: I understand the upstream doesn't have a working tree, but it obviously is remembering this data somewhere.
<vila> kfogel: what does 'bzr info -v' says ? maybe a shared repo up
<kfogel> vila: AAAAAAAAAAAAAAAAAAAH
<kfogel> vila: that's right, I did create a shared repo upstream
<kfogel> vila: thanks, that's it
<james_w> anybody recognise "RuntimeError: Property used in an unknown class" ?
<james_w> I'm just trying to use the factory
<james_w> an, using a project in place of a product
<jml> james_w, more context please
<james_w> jml: if you pass product=IProject to a factory method you get a cryptic error message
<mrevell> nytol
<mwhudson> good morning
<james_w> morning mwhudson
<jml> mwhudson, good morning
<jml> I've got buildout building subunit from the tarball and installing it in the parts/ directory
<jml> but I'm not sure how to get it into the PYTHONPATH
<jml> gary-lunch, when you're back, I'd appreciate some advice
<james_w> mwhudson: would you like to help me get https://code.edge.launchpad.net/~james-w/launchpad/export-code-import/+merge/21472 finished?
<mwhudson> james_w: probably
 * mwhudson looks
<mwhudson> (actually i kinda want to go back to bed already, but struggling on...)
<mwhudson> james_w: "This exports ICodeReview." ?
<james_w> ach
<maxb> salgado_: Uploading to the launchpad PPA with a Debian-lookalike version isn't optimal
<james_w> fixed
<salgado> maxb, you mean the -2 bit?
<maxb> yes
<maxb> Preferably the name or version of every package would contain 'launchpad' so that people can easily identify them as non-distro packages
<salgado> ah, right
<mwhudson> james_w: i don't think ICodeImport['owner'] is used anywere at all
<gary_poster> jml: yo
<gary_poster> jml: extra-paths might be what you are looking for
<gary_poster> see how mailman is done?
<salgado> maxb, I'll upload another one and delete the -2 one
<mwhudson> i'm not even sure registrant is
<james_w> mwhudson: yeah, I was a bit confused between registrant, owner and assignee
<james_w> you mean I should export the latter instead?
<maxb> salgado: So, given its effectively a backport from lucid, it should be named something like 0.2.0-1~BLAH
<mwhudson> james_w: all a legacy of over design
<gary_poster> jml: extra-paths = ${buildout:directory}/lib/mailman is what we have now.  You would add a newline and then an indented value for the other directory
<maxb> salgado: For example, look at what I did with the python-imaging backports in the LP PPA
<maxb> But really, the main problem with 0.2.0-2 is that's what the next upload to Debian itself will be! :-)
<james_w> mwhudson: one thing that I was unsure about is the information that seems to be duplicated with ICodeImport.branch, e.g. product
<james_w> especially since I would like to have code imports for package branches as well as product branches
<mwhudson> um
<mwhudson> that looks screwy
<mwhudson> i wonder if _that's_ used anywhere
<james_w> series I can kind-of understand, as that's not an attribute of branch
 * mwhudson is tempted to delete a few things from ICodeImport and run the tests...
<salgado> maxb, heh, good point.  thanks for the heads up; I'll re-upload with proper versions (following the pattern used in python-imaging)
<maxb> thanks
<mwhudson> james_w: i think you shouldn't export registrant, owner, assignee, product, series
<mwhudson> james_w: actually, registrant might be worth exporting
<james_w> deleted the export of owner, so now none of those are exported
<mwhudson> ok
<mwhudson> easier to add than remove an export i guess :-)
<james_w> it tends not to be exported elsewhere
<mwhudson> james_w: so when you say "help you finish", what does that mean?
<james_w> review if that is all it takes
<james_w> I expected there to be several more things involved though
<mwhudson> it looks like a start
<mwhudson> i'm not super familiar with api stuff
<mwhudson> james_w: have you tried accessing code imports on launchpad.dev ?
<james_w> adding exported() around things and testing that they are in the representation is easy
<james_w> it' the traversal stuff that I wasn't sure about
<mwhudson> with launchpadlib
<james_w> I'll try that now
<mwhudson> james_w: well, webservice.get() in the page test is testing the traversal stuff
<mwhudson> i presume $branch/+code-import will still 404 in the webapp as there are no pages registered
<james_w> mwhudson: well, there's working and then the "correct" way to do it :-)
<mwhudson> james_w: it looks ok to me
<james_w> thankfully there was a wiki page for that, so I took enough from there to get the tests to pass
<mwhudson> i wonder why the first people to try this couldn't get it to work
<james_w> the rest seemed to be presenting it in the webapp as you say, but we don't want that
<mwhudson> maybe lazr.restful bugs that are now fixed
<james_w> mwhudson: any idea who it was? We should get their review if so, as they may spot something I missed
<mwhudson> james_w: no, maybe i'm in fact making up that this problem was encountered :/
<mwhudson> if it works it works, there is limited potential for subtle traps here
<james_w> right
<mwhudson> <james_w> the rest seemed to be presenting it in the webapp as you say, but we don't want that
<mwhudson> james_w: i don't understand what you mean here
<james_w> the rest of the wiki page seemed to be about linking up pages to interfaces so that the webapp would render them
<mwhudson> ah ok
<mwhudson> i wasn't completely sure that 'rest' in what you said was the english word :)
<james_w> heh
<james_w> mwhudson: yep, it all shows up in +apidoc, +code-import 404s in the webapp, and lplib can query the code import in sampledata
<james_w> the only issue is that most attributes are writeable, I assume we don't want that
<mwhudson> oh right
<james_w> if I just mark everything as read-only will that break other code?
<mwhudson> james_w: i think it might break the +new form
<mwhudson> but that's fixable
<mwhudson> hm, no it shouldn't, should it?
<wgrant> james_w: So it all pretty much just worked?
<james_w> wgrant: seems like it
<james_w> once I worked out how to do canonical_url for ICodeImport and then traversal
<mwhudson> james_w: trying to set attributes through launchpadlib fails in a pretty strange way
<mwhudson> they should be read_only for now
<mwhudson> and any fallout that causes should just be fixed
<james_w> mwhudson: changed and ec2 test running to discover any fallout
<mwhudson> james_w: awesome
<mwhudson> that means i can ignore the issue for a few hours :-)
<james_w> you and me both
<maxb> Gaaah setuptools distribute pkg_resources buildout namespace-packages ARGH
<maxb> Clean bootstrap on lucid appears broken
 * thumper is back around
 * mwhudson lunches
#launchpad-dev 2010-03-17
 * thumper gets stuck into jobs
<james_w> when I log in to ec2 during postmortem, where can I actually find the LP tree it is testing?
<james_w> or am I misunderstanding how it works?
<thumper> james_w: I expect that you should be able to find it, but I've never tried personally, perhaps mwhudson knows
<mwhudson> james_w: /var/launchpad/test i think
<james_w> thanks
<mwhudson> yeah, that
<james_w> I have some failures in the initial branch, so I'll squash them now
<mwhudson> ok cool
<thumper> mwhudson: quick sanity check
<thumper> mwhudson: during my investigations I found a pile of stuff still in canonical/launchpad/event
<thumper> mwhudson: worth making a new pipe to move them to lp.code.event?
<thumper> mwhudson: or perhaps lp.code.interfaces.event
<james_w> dammit, failures in pagetests
<james_w>     - The code import has been updated.
<james_w>     + There is 1 error.
<james_w>     + Enter the URL of a foreign VCS branch.
<james_w> so something (the readonly change?) breaks +edit-import it seems
<mwhudson> james_w: oh that's not so surprising
<mwhudson> james_w: easy to fix though
<mwhudson> thumper: err
<james_w> care to enlighten me?
<mwhudson> what's in canonical/launchpad/event ?
<thumper> mwhudson: interfaces for events
<mwhudson> yay two conversations
<thumper> mwhudson: not a lot to be honest
<mwhudson> thumper: oh, lp.code.interfaces.event then i guess
<wgrant> If the field is writable, why should it be readonly on the API?
<mwhudson> wgrant: crack
<mwhudson> of the 'updateFromData' kind
<wgrant> mwhudson: But the import URL should be editable by some, shouldn't it?
<mwhudson> wgrant: yes, but it's not edited by going "import.url = foo"
<wgrant> Odd.
<james_w> mwhudson: could we allow certain people to do that over the API?
<mwhudson> james_w: this /should/ all be fixed to work like other objects
<mwhudson> but it's quite a lot of work
<james_w> right
<mwhudson> <drevil>millions</drevil> of $units of tech debt
<james_w> we can work towards that
<mwhudson> james_w: anyway, to make the edit form work again, you need to edit the EditCodeImportForm schema in lp.code.browser.codeimport
<james_w> for now I would like to fix the tests to make my work landable
<james_w> I've got it open
<mwhudson> james_w: to look like this: http://pastebin.ubuntu.com/396468/
<james_w> it's fairly opaque to me though
<james_w> aha!
<mwhudson> (warning: code typed directly into pastebin, probably right though)
<james_w> that's what we have tests for
<mwhudson> this is what i thought would break, btw
<mwhudson> if nothing else is broken, that's pretty good :)
<mwhudson> grar testfix mode grumble
<mwhudson> the testsuite seems to have problems with email at the moment
<james_w>     KeyError: 'cvs_root' on import
<mwhudson> james_w: oh haha
<james_w> yah
<mwhudson> s/IBranch/ICodeImport/ for the first three
<james_w> yay for familiarity
<james_w> beats me spending ages last night working why doctest was giving me syntax errors on
<james_w> import = factory.makeCodeImport()
<mwhudson> :)
<wgrant> What's the db-lp failure?
<james_w> same failures with that change
<wgrant> james_w: s/read_only/readonly
<james_w> thanks wgrant
<mwhudson> wgrant: some crap involving sending mail and not having an interaction :/
<mwhudson> wgrant: http://pastebin.ubuntu.com/396471/
<wgrant> That line is too long.
 * wgrant tries to read.
<mwhudson> wgrant: http://pastebin.ubuntu.com/396472/ better formatted
<wgrant> Ahh much better.
<mwhudson> it's in "Failure in test testSimpleRun (lp.soyuz.scripts.tests.test_lpquerydistro.TestLpQueryDistroScript)
<mwhudson> "
<mwhudson> which i guess i could try running locally
<wgrant> That looks like there's no MTA running.
<mwhudson> but on the whole launchpad tests don't actually send mail through an MTA do they?
<wgrant> Except for test-sampledata-setup.txt, they should not.
<wgrant> (well, it shouldn't either, but it does)
<james_w> thumper: https://code.edge.launchpad.net/~james-w/launchpad/export-code-import/+merge/21472 updated to pass tests. Seeing as you approved it would you now submit it for me please?
<thumper> james_w: I'll take a look
 * james_w has https://code.edge.launchpad.net/~james-w/launchpad/register-code-import/+merge/21505 on the go too, any suggestions appreciated
 * thumper goes to grab coffee
<wgrant> james_w: Why is IHasCodeImports separate from IBranchTarget?
<james_w> wgrant: good question
<james_w> wgrant: one good reason may be that code imports only support products so far
<mwhudson> because for better or worse we don't support code imports for !product
<james_w> wgrant: a less good reason is that I didn't know about IBranchTarget (though I now remember you mentioning it yesterday)
<wgrant> Why does a code import have any idea what it's on?
<mwhudson> good question!
<james_w> I want to fix that in a later branch, but perhaps it should be done earlier
<mwhudson> i really hope nothing uses ICodeImport['product']
 * james_w sends a branch nuking all those things through ec2 to get an idea of the fallout
<james_w> the only thing that might get in the way is ICodeImport['series'], as we can't really store that information on IBranch as far as I know
<mwhudson> ah right
<wgrant> Is that information used for anything?
<mwhudson> i doubt anything uses that either
<mwhudson> i'd certainly entirely forgotten about it
<wgrant> It's potentially useful except not really.
<wgrant> I've never seen any hint that it's used.
<mwhudson> it was probably used during the transition from the old to new system
<james_w> I'm nuking owner, product and series so that we can get an idea of where they are used
<mwhudson> james_w: oh, i was going to do that, but if you're on it already...
<james_w> assuming that anything that uses .owner should use .branch.owner, .product should be .branch.{product,sourcepackage}
<james_w> I believe .assignee is different?
<mwhudson> assignee is just not used
<mwhudson> remove it
<james_w> ok, bye bye
<wgrant> Where does ICodeImport['series'] come from? I can't see it on the table.
 * wgrant checks the code.
<wgrant> Oh.
<wgrant> It searches for the ProductSeries with the branch selected.
<james_w> oh, so it only applies to focus branches?
<wgrant> Yes.
<wgrant> So it's not critical.
<wgrant> So you could always leave it alone and it will just be None, and not break anything.
<wgrant> But better to remove it entirely.
<mwhudson> yes
<mwhudson> if it'
<mwhudson> s not used, kill it
<james_w> error: (4, 'Interrupted system call')
<james_w> oh, that's a great reason to error out from an ec2 test
<james_w> tests finally running
<james_w> I'll call it a day
<james_w> thanks for your help team
<lifeless> james_w: did you get the chance to mail me about bzr-builder?
<mwhudson> james_w: thanks for working on this!
<james_w> lifeless: no, slipped my mind, sorry, pls to be mailing me then I will remember
<lifeless> k
<mwhudson> thumper: what does 'reject' mean for the puller?
<thumper> umm
<thumper> what?
<mwhudson> thumper: http://dev.launchpad.net/Code/RemoveHostedArea?action=diff&rev1=5&rev2=6
<thumper> mwhudson: more context?
<mwhudson> sorry, not a very clear question
<thumper> mwhudson: hmm, can we not catch this while the user is pushing?
<mwhudson> i guess we need to be more careful about the way say codebrowse opens the branch
<mwhudson> thumper: not really, certainly not until we turn the vfs off
<thumper> mwhudson: oh yeah
<thumper> hmm...
<wgrant> Branches aren't accessible over HTTPS at all, are they?
<thumper> wgrant: no
<wgrant> Ah, good.
<thumper> at least I hope not
<thumper> wgrant: actually
<thumper> kinda
<wgrant> Eep.
<thumper> wgrant: from loggerhead anyway
<thumper> what do you mean accessible?
<thumper> branchable?
<wgrant> but that hopefully won't serve the files directly.
<wgrant> Since this will mean that bazaar.launchpad.net contains arbitrary user-generated files.
<lifeless> it will serve the content
<wgrant> And if that was HTTPS, then it could do horrible untold nasty things to the webapp and its cookies.
<mwhudson> right
<lifeless> via the bzr smart protocol, not as files on disk, AIUI
<mwhudson> well
<mwhudson> loggerhead _can_ serve branches, both as dumb and smart http
<lifeless> anyhow, personally I'd keep the mirrored area and scale it out.
<mwhudson> but it doesn't on launchpad
<mwhudson> argh
<thumper> mwhudson: yes?
<mwhudson> given that it would be nice to serve bits of launchpad over http rather than https...
<mwhudson> lifeless: what do you mean?
<wgrant> mwhudson: Don't worry, you would not be the first to be serving arbitrary files under launchpad.net.
<wgrant> It already happens, and worse.
<thumper> mwhudson: the apache rewrite rules make anything not in .bzr go to loggerhead
<thumper> mwhudson: although if someone put some random file in .bzr dir, it would be served
<mwhudson> thumper: i'm getting a bit worried by the fact that if we eliminate the mirrored area, people will be able to get launchpad to serve arbitrary content over http
<thumper> wgrant: yes?
<thumper> mwhudson: yeah...
<wgrant> eg. bug #420292
<mwhudson> thumper: yes, but what if people put crap in .bzr ?
<thumper> mwhudson: exactly
<thumper> hmm...
<mwhudson> currently what saves us here is the puller
<thumper> I guess it comes down to "do we care enough?"
<mwhudson> wgrant: heh, as soon as you mentioned the number, i thought "what about ..." and guessed the outcome
<stub> Can we add a 'bzr tidy' or similar that removes crap from .bzr directories that shouldn't be there, or is that too dangerous as users could be using plugins that store arbitrary stuff in there?
<mwhudson> stub: sounds difficult to get right to me
<stub> I guess we could, as bzr tidy would just remove stuff that the mirror process skips already.
<mwhudson> i think plugins do store stuff
<thumper> mwhudson: there is a bzr concept right now of extra baggage (terminology may be wrong)
<thumper> mwhudson: where push also pushes plugin extras
<stub> Is the goal of removing the mirror step to remove an unwanted copy, or to remove the time between upload and published?
<thumper> stub: to save space
<mwhudson> clearly what we need is a file system with cow snapshots at the directory level, you could make a clone of the directory on write access, then pull from the snapshot to the original then delete the snapshot
 * mwhudson falls out of the blue sky
<mwhudson> thumper: very much at the concept stage though, right?
<stub> So we could mirror, then replace the original with the mirrored copy
<stub> Which should just remove anything that shouldn't be there
<thumper> stub: that sounds relatively sane
<wgrant> mwhudson: Could you abuse stacking to do something like that?
<mwhudson> i guess that would reduce the time we're vulnerable for
<mwhudson> wgrant: hnngh
<mwhudson> wgrant: maybe
<mwhudson> you could probably do it at the transport level too i guess
<mwhudson> have writes go to one area, reads fall back to another
<wgrant> say, a fallback repository?
<mwhudson> all sounds a bit crazy though
<mwhudson> wgrant: i think the problem with doing at the stacking level is that you don't know whether a branch is going to be written to when its opened
<wgrant> True.
<mwhudson> so you could present every branch over a writable transport as being stacked on the real branch, but that doesn't sound very sane
<stub> Overwriting the original branch with the mirrored copy also gives us an opportunity for other fun, like pushing revisions into the stacking target (saving two thirds of the disk space used to store Launchpad code perhaps).
<wgrant> True.
<stub> Hmm... 3/4... dev branch, devel, stable, db-devel is four copies of each revision
<wgrant> stub: You can save just about all of the space except for whatever is in db-stable.
<stub> (for non private branches with a public stacking root)
<wgrant> since almost every branch will end up in db-stable, so their branch data is entirely redundant.
<stub> Yes.
<stub> Most projects you would only save 50% though (dev branch -> trunk)
<mwhudson> you could also do that in a garbo type job
<mwhudson> probably fairly slow to do it for 200k branches though
<wgrant> mwhudson: But most will end up with no revs after the first run, so they needn't be touched.
<stub> Well... mirroring would become non-time critical so yes it could be done in batches rather than on-push
<stub> (on-push = stick branch in queue, garbo == mirror and pack stuff from the queue)
<mwhudson> stub: i meant just anyway, independent of changing how mirroring worked
<stub> ok
<mwhudson> wgrant: point
<mwhudson> maybe i should just not worry too much about the arbitrary hosting thing and assume that sooner or later we'll be able to disable vfs-level access to branches
<wgrant> is that a valid assumption?
<wgrant> Hasn't it been happening reasonably soon for a long time now?
<mwhudson> i don't know, but it's not /really/ a risk as long as the cookie is marked secure
<wgrant> Right.
<mwhudson> and then write a branch opener that's suitably paranoid about stacking and so on and have everything use this
<mwhudson> (this will be easier once the hosted area/mirrored area distinction has gone away i guess)
<lifeless> mwhudson: I mean simply that I'd keep the separation between 'mirrored and anything goes' and 'not mirrored owners only'
<lifeless> I think its a nice data cleansing step
<lifeless> and because the mirrored area doesn't need to worry about stuck clients or anything, its arguably easier to scale/mirror/replicate
<mwhudson> lifeless: that does seem to imply using twice as much disk as we in some sense need though
<lifeless> mwhudson: we can probably save that several times over by doing a gc on stacked branches, encouraging deletes of merged branches, ...
<wgrant> lifeless: How much data is stored in a stacked branch with no revs of its own?
<lifeless> mwhudson: I mean, look at pg - we use several times the disk we need to there, as well :)
<wgrant> Deleting branches is bad.
<mwhudson> lifeless: we use considerable more disk for branches than for postgres, i think
<mwhudson> not as much as the librarian though
<lifeless> wgrant: it depends; a stacked branch that has been merged to trunk has all the data ever written to it still
<wgrant> lifeless: I mean after GC.
<wgrant> If that exists yet.
<lifeless> wgrant: deletion is a valid thing for users to do to their own data
<wgrant> lifeless: But it deletes MPs and invalidates recipes and blah blah, so it would be really nice to minimise it.
<mwhudson> deleting implies deleting the merge proposal which is a bit lame
<lifeless> wgrant: oh, about 800 bytes or so, butprobably 25 allocation units
<lifeless> wgrant: uhm, I *want* to delete stuff I'm done with. Its cognitive overhead I no longer care about.
<wgrant> lifeless: Ew.
<lifeless> wgrant: I'd like to make deletion easier, not harder.
<wgrant> Why do you want to?
<wgrant> what's wrong if they're hidden, as they are now?
<lifeless> wgrant: because it has no value anymore. The history is in bzr
<wgrant> Keeping MP history is useful.
<lifeless> wgrant: sometimes. Its not an absolute.
<lifeless> wgrant: 'trivial fix: doit' is not valuable historical data.
<lifeless> wgrant: big debate about the right way, which noone ever looks at, is also not valuable historical data.
<wgrant> lifeless: It also doesn't consume a significant amount of space.
<lifeless> *if* someone looks at it, its a different matter
<wgrant> And if we delete it, it can never be looked at.
<lifeless> wgrant: nonzero cost, nonzero benefits. You can do the math.
<wgrant> Should we delete bugs too?
<lifeless> anyhow, gc doesn't exist yet. It should be able to zerg stuff back to nearly empty in principle. for merged branches.
<wgrant> Right.
<lifeless> wgrant: there are many I'd like to, yes.
<lifeless> the name 'bubba' is floating in my consciousness right now
<wgrant> Heh.
<stub> Ooh... its about to bucket down on the protestors outside.
 * stub goes to switch on the TV
<mwhudson> stub: the current government seems to have been in charge for quite a while this time
<mwhudson> unless i've missed a revolution or two
<spm> mwhudson: blink and you miss 'em....
<stub> I've lost track
<wgrant> Can I do something like foo[0] in TALES without resorting to python: ?
<mwhudson> wgrant: no
<wgrant> :(
<mwhudson> ugh, tests are failing again in buildbot
<wgrant> Same one?
<mwhudson> basically yeah
<wgrant> Mine's the only untested rev, isn't it?
<wgrant> But it doesn't look at all relevant.
<mwhudson> wgrant: not testing any of yours in this build
<mwhudson> 10530-10532
<wgrant> The db-lp failure a few hours ago had me in the blamelist. Is this something else?
<wgrant> Ah.
<mwhudson> this is the other builder
<mwhudson> i guess it's worth looking to see what these tests do
<mwhudson> and, wow, i hadn't realised how much i used / to start a search in ff
<wgrant> Doesn't work in Chromium?
<mwhudson> nope
<StevenK> Ick
<lifeless> ctrl-F
<lifeless> breaks my fingers all day atm
 * StevenK hasn't found a compelling reason to switch
<lifeless> it is faster
<lifeless> more memory hungry though
<wgrant> And uses ugly widgets.
<lifeless> by faster, I mean lp bug pages don't feel like torture
<mwhudson> StevenK: firefox started segfaulting continuously for me, i didn't want to switch
<mwhudson> it seems mostly ok
<wgrant> lifeless: Surely you jest.
<mwhudson> it handles multiple windows in a rather strange way
<lifeless> wgrant: no, I don't. The js is a huge component of bug page load times.
<lifeless> and chromium is a lot fast at js
<wgrant> Ah, yes, particularly for multi-tasked bugs.
<lifeless> I haven't yet done my 'open 20 tabs by ctrl-clicking on a search' thing in chromium yet
<lifeless> but on ff that used to take 4 or 5 minutes
<StevenK> lifeless: Right, but then I hear people complain about things that I do constantly in a browser and I push looking at chromium further down the list
<stub> mwhudson, wgrant: I think the db-devel and current devel failures are spurious
<mwhudson> stub: hmm
<mwhudson> stub: they're intermittent, and i suspect not directly related to the test cases that are failing
<mwhudson> stub: but they've started happening every other run for the last week, so _something_ has changed
<mwhudson> it's annoying because i have some kind of memory of looking at a checkin and going "ooh, i wonder if that's a good idea" in this sort of area
<mwhudson> and now i can't find it :(
<stub> mwhudson: They seem Librarian related. The only change I could think of might have been BjornT's persistent-resources change to the test suite, making it possible to keep the Librarian and memcached running between test runs.
<mwhudson> stub: it's email related, really
<mwhudson> the librarian is just being a canary
<stub> Other branches were failing with Librarian issues anyway
<stub> In non-email sections
<stub> Looks to me that it is failing because the logging system is trying to upload the traceback into the Librarian and this is failing.
<stub> So we get a different error message than expected
<mwhudson> the current lp builder is definitely failing in an email related way
 * mwhudson grinds to a halt
<spm> night mwhudson
<mwhudson> for the test failure, i reckon some test is putting things in the mail queue and then sometimes the mails are handled before that test process exits
<mwhudson> and sometimes not
<mwhudson> and then the next test to activate the queue processing thread blows up
<mwhudson> maybe?
<mwhudson> it'
<mwhudson> s a guess anyway
<adeuring> good morning
<wgrant> Do I need a UI review for https://code.edge.launchpad.net/~wgrant/launchpad/show-package-sets-in-queue/+merge/21521? It's only used by a few people, and I have +1s from two of them.
<mrevell> Howdy
<jml> jelmer, take a look at https://code.edge.launchpad.net/~jml/launchpad/new-zope-testing/+merge/21539
<jml> mrevell, hello
<jml> mrevell, I'm in Worcestershire!
<jelmer> hey mrevell
<jelmer> jml: looking
<mrevell> jml, Welcome to everything that is great about England.
<mrevell> :)
<mrevell> hey jelmer
<mrevell> How long are you chaps sprinting?
<jml> mrevell, just until Friday
<jelmer> jml: yuck, hardcoded subunit version in buildout.cfg ?
<jml> jelmer, I guess. I mean, we have hardcoded versions in versions.cfg
<jelmer> jml: yeah, but it'd be nice to only have a hardcoded version in one place so that it's easier to update
<jml> jelmer, I agree. I don't really know how to do that though.
<jml> jelmer, anyway, did I end up deleting your code?
<jelmer> jml: would it be possible to just use a checkout of a bzr tag rather than downloading the tarball?
<jml> jelmer, not w/ the cmmi recipe
<jelmer> jml: yep - around line 60 in the diff
<jml> jelmer, but it's hard to tell. the docs might be incomplete because J1m only cared about writing tests.
<jml> jelmer, do you want me to add your bit back?
<jelmer> jml: If possible :-) But I guess it'd have to go into zope.testing now?
<jml> jelmer, I _think_ it's equivalent to "if os.getenv("SUBUNIT_FORMATTER"): options.append('--subunit')"
<jml> jelmer, the idea is that if the env var is defined, we do subunit, right?
<jelmer> jml: Yep, and we pipe our subunit output into the command set in that variable
<jml> jelmer, I'll have a look
<jml> :\
<allenap> jml: Building new-zope-testing fails for me, after merging into a fresh devel branch. Specifically, it breaks during the cmmi bit for subunit.
<jml> allenap, can you paste the error please?
<allenap> jml: I'm having trouble replicating it.
<jml> allenap, so it works now?
<allenap> jml: Not sure. I was being a muppet when trying to replicate. Having another go now.
<allenap> jml: http://paste.ubuntu.com/396612/
<jml> allenap, I guess you don't have CHECK
<jml> allenap, I wonder if we can disable that
<allenap> jml: Ah, "unit test framework for C"?
<jml> allenap, yeh
<jml> allenap, I guess that means I'd need to add that to deps or something
<allenap> jml: Yes, guess so. I'm seeing if I'm missing any other deps now.
<poolie> hello jml
<jml> poolie, hi
<allenap> jml: cppunit is missing too.
<deryck> Morning, all.
<jml> allenap, I see. Is that it?
<jml> deryck, good morning
<jml> mrevell, hi
<mrevell> hey jml
<allenap> jml: Yep, it's building fine now (for me, anyway).
<allenap> jml: Yes, built fine, and subunit.__file__ eventually resolves to parts/subunit/lib/python2.5/site-packages/subunit/__init__.py.
<allenap> jml: In new-zope-testing, --save-list is gone. Do you know of a good alternative? `bin/test --list` does not produce a reusable list (it needs a fair bit of munging).
<jml> allenap, bin/test --subunit --list-tests | subunit-ls
<jml> bigjools, what do you think about me moving a bunch of stuff to lp.poppy?
<allenap> jml: Oh, that's beautiful, thanks :)
<jml> I wonder what I need to do to get the check & cppunit stuff
<allenap> jml: maxb seems to be the expert on that.
<allenap> soyuz: Have you seen the test failures in buildbot? They don't seem to be transient, and they're in soyuz.
 * bigjools looks
<wgrant> Is this the same as the db-lp failure ~10h ago?
<allenap> wgrant: Possibly. testMissingAction and testSimpleRun?
<bigjools> this is the librarian issue again
<wgrant> allenap: Yeah. mwhudson was having a look at that.
<wgrant> it is probably not actually a Soyuz bug.
<allenap> bigjools, wgrant: Okay, sorry for the noise.
<stub> the current devel one is a weird traceback because librarian_client.remoteAddFile(...) is raising an AttributeError, but the exception handling code that is trying to stuff the traceback into the Librarian only deals with an UploadError.
<stub> oh... hmm... there is a catchall there...
<stub> That code should die anyway - I don't think we have a valid use case for stuffing exceptions in the Librarian any more. OOPS reports exist now for a start.
<wgrant> It's handy for Soyuz.
<wgrant> Since the upload processor sucks at reporting errors.
<wgrant> Sometimes you have to look at the traceback.
<maxb> allenap: hmm?
<allenap> maxb: jml might need to add "check" and "libcppunit-dev" to lp-dev-dependencies. You seem to be an ace at packaging :) That's all.
<maxb> wow, C code in launchpad? :-)
<jml> allenap, no. C code in our dependencies
<jml> maxb, allenap, maybe I should just manage subunit as a package dependency
<maxb> works for me, but I have a somewhat jaded view of buildout/setuptools/distribute :-)
<stub> Maybe the librarian is a red herring there in the failure... the test seems to be expecting a single ERROR line but is getting two ERROR's (plus extra traceback cruft from the handler)
<wgrant> When did this start happening?
<jml> maxb, how would I add subunit as a dependency?
 * jml has nfi
<maxb> First, you run 'rmadison python-subunit' and decide if the versions packaged across hardy-lucid are sufficient for your needs
<maxb> Then you'd backport packages as needed
<maxb> and lastly depend on it in launchpad(-developer)-dependencies
<jml> maxb, ok, thanks.
<jml> maxb, so I need to backport to hardy :\
<jml> maxb, it's just "subunit" fwiw.
<jml> which means I need to do more exploratory work
<jml> making changes to Launchpad is so frikking hard :(
<wgrant> leonardr: Why can't the apport fix go into lucid as well?
<leonardr> wgrant: i don't know anything about the apport fix, so i just preserved what was written about it earlier
<wgrant> Ah.
<leonardr> wgrant: looking at bug 538097, i don't even thing the apport developers have been told they need to make a change
<mup> Bug #538097: +storeblob fails with "500 Internal server error" on production (works on edge) <apport-collected> <oops> <Launchpad Foundations:Fix Released by gary> <apport (Ubuntu):Invalid> <https://launchpad.net/bugs/538097>
<wgrant> It seems like a one line fix in apport now would be quite a bit better than 5 years of hack in LP.
<leonardr> all right, i'm filing a bug against apport
<maxb> jml: Hmm. subunit depends on debhelper >= 7.2.9 for the dh --without option
<maxb> Some amount of packaging faff would be required. Not really all that hard, though
<jml> jelmer, from canonical.launchpad.sqlbase import ZopelessTransactionManager
<jml> jelmer, and then pass ZTM as txn
<jml> maxb, ok cool.
<jml> maxb, it will all be new ground for me anyway
<jml> jelmer, from canonical.database.sqlbase import ZopelessTransactionManager, sorry
<intellectronica> i have a problem logging in in my dev environment after updating. when i try to log in i get http://pastebin.ubuntu.com/396651/
<intellectronica> DiscoveryFailure: Error fetching XRDS document: <urlopen error (-5, 'No address associated with hostname')>
<intellectronica> anyone seen that, knows what this is about?
<wgrant> intellectronica: Add testopenid.dev to /etc/hosts.
<maxb> intellectronica: Have you got the new testopenid.dev vhost.....yes
<intellectronica> wgrant, maxb: cheers!
<leonardr> wgrant: bug 540212
<mup> Bug #540212: Send a Referer header when making requests to Launchpad <Apport:New> <https://launchpad.net/bugs/540212>
<wgrant> leonardr: Thanks.
<jml> oh look
<jml> there's a thing called lucilleconfig
<wgrant> Heh heh heh.
<wgrant> Two things, even.
<wgrant> One of which seems to be redundant.
<jml> :(
<jml> bigjools, want to review https://code.edge.launchpad.net/~jml/launchpad/lp-poppy/+merge/21551 ?
<wgrant> (DistroSeries.lucilleconfig appears to just list components, which would appear to also be the purpose fulfilled by ComponentSelection)
<bigjools> lucilleconfig is an abomination
<wgrant> Pfft, who needs a config in the config file.
 * wgrant wonders about lp.archivepublisher.library, which has been XXXed for removal for nearly four years, and has just about not changed since the start of 2005.
<stub> We should finally kill off ZopelessTransactionManager...
<kfogel> Ursinha: wow, I like this new QA process better
<Ursinha> kfogel: cool! what do you like best?
<Ursinha> kfogel: and what would you change, of course -- feedback is gold :)
<kfogel> Ursinha: so far nothing I would change.  I just like knowing that what I do is being captured by the system; also, knowing that there's an infallible way to do searches is great.  It's also interesting that we're organized by bug rather than by particular merge/commit now.
<Ursinha> kfogel: to be honest I don't trust launchpad searches
<kfogel> Ursinha: ?
<Ursinha> kfogel: and I mean using the UI
<Ursinha> kfogel: because there are default options that you may have no idea launchpad is using
<Ursinha> you can't even know what you were searching when in the results page
<Ursinha> (I've filed a bug for that a while ago)
<kfogel> Ursinha: oh yeah, that's a big one
<Ursinha> kfogel: but it's cool to be able to do that using the api
<kfogel> Ursinha: <3 lp APIs
<james_w> could someone give me some pointers towards writing a test for https://code.edge.launchpad.net/~james-w/launchpad/package-merge-proposal-permissions/+merge/21561 please?
<leonardr> james_w
<leonardr> the pagetests have different browser objects for differently-permissioned users
<leonardr> you could use two of them to show that some users can carry out an action and some can't
<leonardr> i can't be more specific than that, but that's the general idea
<james_w> and where would I find those tests?
<leonardr> james_w: i think the tests closest to what you want are in lib/lp/code/stories/
<james_w> ok
<leonardr> rockstar or someone might know which test you should add to
<james_w> I've found some permission tests in lp.code.tests.test_branch
<james_w> I'm tempted to do a parallel lp.code.tests.test_mergeproposal
<james_w> branchmergeproposal
<leonardr> james_w: yes, if you can do it, it would be better to test the permissions directly in a unit test
<james_w> great
<bac> reviewers meeting starting in #launchpad-meeting
<james_w> are they any docs on what goes on Forms?
<james_w> I need to provide a Product picker widget
<james_w> there's https://dev.launchpad.net/LaunchpadFormView, but it doesn't talk about the schema
<jml> james_w, no, there aren't documents
<jml> james_w, or if there are, it's perhaps best to pretend there aren't.
<jml> james_w, the standard way of doing this sort of thing is to find some other code that does something close to what you want to do.
<bigjools> james_w: lib/canonical/launchpad/doc/launchpadform.txt
<bigjools> jml hates doctests ;)
<james_w> wow
<james_w> deleting all the cruft from ICodeImport only gives a single failing test
<mars> james_w, you should get massive karma for that :)
<jml> I think thumper joked about giving extra karma for lines removed
<mars> I think salgado would win
<jml> at least for today
<mwhudson> james_w: yay
<mwhudson> james_w: want a review for that branch?
<james_w> well, I assume we should fix the failing test first :-)
<mwhudson> i guess
<james_w> plus, I can't actually kill owner without db modifications
<mwhudson> which test is it out of curiosity?
<mwhudson> right
<james_w> and presumably assignee
<james_w> pagetest for creating new code imports
<mwhudson> yeah, but you can just leave them lurking in the db
<james_w> my hacksaw job to remove the reference to product was a little too rash
<james_w> they have a non-NULL constraint
<james_w> and if anyone can decode the question on https://code.edge.launchpad.net/~james-w/launchpad/export-code-import/+merge/21472 for me I would appreciate it
<james_w> "Do we know if the security wrapper were properly defined for the exported field?"
<james_w> mwhudson: https://code.edge.launchpad.net/~james-w/launchpad/nuke-code-import-warts/+merge/21586 if you would
<james_w> oh, hang on, I need to set the prerequisite branch
<mwhudson> i was about to say
<james_w> mwhudson: https://code.edge.launchpad.net/~james-w/launchpad/nuke-code-import-warts/+merge/21587 should be better
<thumper> here's a reminder that I'm not in this morning
<thumper> james_w: you can't just make the code import status writable
<thumper> james_w: unfortunately it doesn't work like that :(
<thumper> james_w: mwhudson may be able to shed some light on it
 * thumper off to the museum
<james_w> but it has readonly=True?
<mwhudson> james_w: ignore thumper :)
<james_w> ok :-)
<mwhudson> james_w: the question about "<james_w> "Do we know if the security wrapper were properly defined for the exported field?"" isn't relevant now its all readonly
<james_w> ok
<EdwinGrubbs> mars: ping
<mars> hi EdwinGrubbs
<EdwinGrubbs> mars: is the Makefile supposed to be smart enough to run jsbuild when one of the files is modified? Actually, it may be just bin/test that does not update it automatically?
<EdwinGrubbs> mars: does base-layout-macros.pt still play any role in what gets combined by jsbuild?
<mars> EdwinGrubbs, I think base-layout-macros does
<mars> It is scrapped for the YUI dependencies
<mars> EdwinGrubbs, I don't know about jsbuild.  It was supposed to, but last I saw it didn't.  That was one of those "we should let make do the work" things IIRC.
<EdwinGrubbs> mars: does jsbuild scrape base-layout-macros for the dependencies? I don't see any mention of base-layout-macros in the Makefile.
<mars> looking, it calls a different script to scrape the template
<mars> EdwinGrubbs, utilities/yui-deps.py
<EdwinGrubbs> mars: just to confirm, nobody should be loading js files inside any other .pt file besides base-layout-macros?
<EdwinGrubbs> mars: hmmm, when I run yui-deps by hand, it doesn't list any of the files from lib/canonical/launchpad/javascript.
<mars> EdwinGrubbs, it should not list any files in lp/javascript.  I think those are handled elsewhere.
<mars> EdwinGrubbs, it is OK to list per-page javascript files, but the support for that is shaky right now.  Whatever you need will not be included in the rollup.
<mars> I guess it is OK for the purpose though.
<mars> The rollup is site-wide.   The per-page JavaScript is exactly that: files that are only needed by the given page.
<mwhudson> lunchtime!
#launchpad-dev 2010-03-18
<maxb> Purely ooi, and because I'm attempting to copy it for use in my own project, is it correct that db patches don't necessarily land in numerical order?
<wgrant> Correct.
<maxb> wgrant: Does Launchpad work with python2.6/lucid at all for you now? I am now getting the distribute pain on multiple lucid machines, and it looks like it ought to reproduce 100%
<wgrant> maxb: I forget if I still have everything downgraded.
 * wgrant tries.
<wgrant> I appear to have the latest version of python-distribute and python-pkg-sources.
<wgrant> er, *resources
<wgrant> There is still the zope.interface issue, but that's all I have held.
<maxb> Hmm. I wonder if I apt-get install python-distribute, it will work for me
<wgrant> And I did a tree build (not from scratch, but from a fresh branch) this morning.
<maxb> zope.interface issue?
<wgrant> I don't have python-distribute installed.
<maxb> huh
<wgrant> Builds were failing a couple of weeks ago for me because LP wants zope.interface 3.5.2, while Lucid has 3.5.3.
<maxb> how odd. I can't even run bootstrap.py without hacking ez_setup.py
<wgrant> What's the error you're getting now?
<wgrant> Right, I was in that situation for a while.
<wgrant> But it works now.
<maxb>     PYTHONPATH=ws.find(pkg_resources.Requirement.parse('setuptools')).location)
<maxb> AttributeError: 'NoneType' object has no attribute 'location'
<maxb> ugh, how mysterious
<wgrant> Yeah, that one...
<wgrant> It was never particularly reproducible.
<maxb> perplexing. ok.
<sidnei> maxb, i think gary was working on that issue. he helped me get past that on lazr-js
<maxb> I would imagine so. Poor gary gets stuck with all the buildout insanity :-)
<sidnei> maxb, if you fetch the bootstrap.py from lp:lazr-js it *might* work, though the command line options changed a little bit.
<maxb> that could be interesting
 * thumper drags attention back to email jobs
<thumper> utilities/ec2 land --dry-run lp:~james-w/launchpad/export-code-import
<thumper> mwhudson: do you remember the simple way we should be getting a store now? (in a test)
<mwhudson> thumper: IStore(ModelClass) ?
<thumper> ta
<thumper> oh FFS, why is it in canonical.launchpad.interfaces...
<wgrant> Is there a place for all that Foundations stuff to go?
<thumper> wgrant: like what?
<wgrant> thumper: Most of the Foundations stuff still lives in canonical.$NONLOGICALCONGLOMERATIONOFPACKAGES
<thumper> wgrant: well, specifics?
<wgrant> c.l.webapp
<thumper> wgrant: most foundational stuff should go into lp.services
<wgrant> The Storm utilities.
<thumper> we should probably have lp.services.storm
<thumper> lp.app.webapp maybe?
<wgrant> Probably.
<wgrant> Maybe.
<thumper> or lp.services.webapp
<mwhudson> and call it something else
<spm> lp.services.AAAAAAAAA
<spm> oops. did I type that in? sorry....  >:)
<mwhudson> gara!
 * mwhudson fights the database restore
 * wgrant is bemused by the existence of lib/canonical/not-used
<wgrant> It contains only an empty directory -- hctapi, of all things.
<spm> at least the naming is self-referentially correct :-)
<mwhudson> wgrant: i
<mwhudson> 'll review a branch removing it
<thumper> spm: heh
<wgrant> It's also tempting to kill canonical.doap and lp.archivepublisher.library. Both have been untouched and unused for at least four years.
<thumper> ââ¹
<thumper> silly team inconsistencies with the rest of the codebase
<thumper> .teamowner vs .owner for *everything* elase
 * thumper pushes up pipe 2 of 4 for email job work
<wgrant> thumper: Well, teamowner is special, given that it defines the type of the object.
<thumper> wgrant: yeah, I know
<thumper> wgrant: it could just be called owner and have the same impact
<wgrant> Possibly.
<spm> mwhudson: ref the DB restore; also triggered a nagios alert on "staging in restore mode for > 1 hour"; the actual "outage" part of a staging restore was 62 minutes. ouchies. ie sync new code; build it; (~45 mins) alter _new DB's and re-start, ~ 17mins.
<mwhudson> spm: ugh
<mwhudson> it would be really nice to ratchet make build time down some
 * thumper EODs the 1/2 day
<jtv> Is launchpad totally unresponsive (beyond the front pages) for anyone else?
<wgrant> jtv: WFM
<adeuring> good morning
<jtv> wgrant: for example, do you get https://code.edge.launchpad.net/launchpad/+activereviews ?
<jtv> hi adeuring
<adeuring> ji jtv!
<spm> jtv: I do/can
<jtv> hmm...
<jtv> I've got no problem with https to our wikis, or to the front page, but I can't access anything useful on LP
<spm> diff browser? and or sniff network traffic?
 * wgrant can too.
<jtv> time to check /etc/hosts
<jtv> /etc/hosts is fine.  :/
<spm> jtv: is "I can't access anything useful on LP" some sort of reference to the info hosted by LP? or a network access type of comment? :-P
<jtv> spm: I can't access code, bugs, or translations pages for any projects I've tried.
<spm> awesome
<spm> try a different browser?
 * wgrant deletes 10KLOC
<spm> wgrant: only 220K more to go!
<jtv> spm: heyy, that did the trick!
<jtv> so... why doesn't it work in chromium!?
<spm> jtv: seriously? oh my. in that case, flush your cache for starters/restart that browser; and if still no joy; zot cookies as well. if STILL no joy; rename that .<browser> dir and start afresh
<spm> jtv: you're the developer. debug it! :-P
<spm> man, I am *so* going to hell....
<jtv> I just did.  It was the browser I had the kanban board open in.
<jtv> spm: as opposed to the people who have to work with you?  I wonder what we're being punished _for_ though
<snaggen> I'm trying to get launchpad codereviews to work on my own launchpad setup. It currently works on the web and it is sending mails. But how is the replying to the mail part working?
<spm> jtv 2, spm 0.
<spm> snaggen: there's a processmail script that runs that processes incoming email
<jtv> spm: so the trick turns out to be, don't keep leankitkanban.com open any longer than you have to.  With chromium at least you don't need to restart the browser to undo its damage, so that's progress.
<spm> snaggen: cronscripts/process-mail.py fwiw
<snaggen> spm: but the reply goes to some mp+1@code.launchpad.dev. So I guess I have to have some mailserver running on that host. But is there some procmail to get the mails in to the database or how does that work? or does teh process-mail.py script access to the mailboxes?
<spm> jtv: I tried chromium, but with the number of tabs I typically have open; it (impressively!) eats even more memory than firefox
<wgrant> snaggen: process-mail.py looks in a mailbox.
<spm> snaggen: latter; pop3 I think...
<wgrant> See mailbox.txt.
<snaggen> wgrant: spm: Thanks for the pointers...
<spm> np
<spm> and on that note. dinner beckons. night all.
<mrevell> Morning
<bigjools> morning
<wgrant> Morning bigjools.
<wgrant> bigjools/noodles775: What do you think of https://code.edge.launchpad.net/~wgrant/launchpad/show-package-sets-in-queue/+merge/21521?
<bigjools> wgrant: looks great, my only question is did you look at the difference in the number of queries made?
<bigjools> that page is very, very sensitive to query count
<bigjools> i.e. it's close to the timeout limit
<wgrant> Hm, I thought that was only when accepting stuff.
<bigjools> it's displaying the page
<wgrant> It renders in 3 seconds SQL time on edge.
<bigjools> we fixed the acceptance issue by making it redirect when the txn finishes
<wgrant> Oh, it didn't before?
<bigjools> what I am interested in is the increase in queries needed for each row
<bigjools> it never used to
<wgrant> Crazy.
<bigjools> it was fixed just before karmic
<bigjools> yar
<bigjools> stub: when all references to a libraryfilealias have gone, the gc deletes the files automatically right?
<wgrant> bigjools: It adds just one query per source row.
<wgrant> And it's pretty trivial as well.
<bigjools> wgrant: ok that's fine then cheers
<wgrant> bigjools: Do you have a solution to build start time estimation with builder pools?
<bigjools> wgrant: nope
<bigjools> wgrant: btw can you get another reviewer to check out your +queue change please, I am sprinting
<bigjools> soryr
<wgrant> I've been wondering how to do it for multi-arch builders, and it seems to be a similar sort of problem :/
<bigjools> sorry, too
<wgrant> bigjools: Oh, I intended to.
<bigjools> yes it's kinda hard
<wgrant> Just wanted you to OK it.
<bigjools> cool thanks for asking :)
<wgrant> I can't see an obvious way to do it apart from a full simulation.
<bigjools> it's a *estimation* :)
<wgrant> jtv: The 'building' icon spacing on https://edge.launchpad.net/builders seems a bit broken. Is that related to your changes?
<jtv> wgrant: yes, it is.  How appropriate it is depends a bit on your perspective: it's the build status, so I'm displaying it as part of the link to the build.  It did simplify the code.
<jtv> wgrant: in fact this is the subject matter of a UI review that's currently somewhere in limbo.
<wgrant> jtv: The current spacing is definitely broken. I think the ordering is too, but that's up for debate.
<jtv> wgrant: I don't think I changed the ordering... did I?
<wgrant> It was '( ) Building _blah blah blah i386_'
<wgrant> It is now 'Building ( )_blah blah blah i386_'
<jtv> wgrant: oh, sorry, misunderstood you earlier.  That's the part I was talking about all along.
<wgrant> jtv: Why does it make the code simpler?
<wgrant> Because it was looking at the build status to generate the icon?
<jtv> wgrant: and because it was easier to have a link formatter for builds.
<wgrant> jtv: Argh, where's the template for that?
 * wgrant cannot find it.
<jtv> builder-index.pt
<wgrant> jtv: Looks like buildfarmbuildjob-current.pt
<jtv> ah yes, that's where the link happens now.
<jtv> That was another thing: we can't show that icon for a buildless job.
<jtv> So that's another way in which it belongs to the build, not the builder.
<wgrant> So, I think we should do what is done for the rest of the states.
<wgrant> And just use the currentlybuilding icon if there's a job assigned.
<wgrant> Regardless of state.
<jtv> That's fine by me.
<noodles775> jtv: did you update the MP with the instructions for demoing it that we discussed? I won't get to it while sprinting, but there are quite a few people around who can do it?
<stub> bigjools: yes, after while.
<noodles775> Although, I now it'll be a separate MP.
<jtv> noodles775: alas, no.  Catastrophic system failure & bad telephone line.
<noodles775> aha.
<jtv> Not fun, being negotiated down to 1mbps when you're paying for 12
<jtv> while your main system won't boot
<wgrant> Ow.
<bigjools> stub: ok ta
<wgrant> jtv: Ah, so you already hardcode that icon form non-build BFJs?
<jtv> wgrant: I _think_ I did that in an earlier iteration but it didn't seem appropriate.
<wgrant> It appears to be in devel's buildfarmjob-current.pt
<gmb> stub: Question for you re: checkwatches holding transactions open, if you've got a sec.
<stub> sure
<jtv> wgrant: there was a lot of exploring to get it all in a sort of minimal working shape... I don't mind moving things around a bit now that we have a working situation.
<wgrant> Yep.
<jtv> henninge: weird...  following the pbuilder instructions, when I exit my chroot and login to it again, I've lost my /etc/hosts patch that ntp needed.  Wasn't --save-after-login supposed to preserve that?
<henninge> jtv: yes, I thought so
<jtv> maybe --save-after-exec instead?
<henninge> but may --login takes the system files and overwrites them on each login?
<henninge> jtv: I have not tried that too many times, I admit.
<henninge> jtv: try putting it in your own /etc/hosts ...
<jtv> will the chroot talk to my local dns!?
 * wgrant again points out that you can change the NTP host.
<jtv> wgrant: you mean in /etc/ntp.conf?  Or by running an explicit ntpdate?
<jtv> Hmm... ntpdate isn't likely to work if it'll compete for a port with the host ntpd
<stub> Damn Internet is a yoyo. I must be wearing the wrong colour shirt.
<wgrant> jtv: By altering /etc/launchpad-buildd/default
<jml> http://paste.ubuntu.com/397153/
<jml> http://paste.ubuntu.com/397153/
<jml> http://paste.ubuntu.com/397153/
<jml> http://paste.ubuntu.com/397153/
<jtv> wgrant: ntphost?  That's set to pool.ntp.org btw
<jtv> wgrant: which is what I'd hope for, I guess
<wgrant> jtv: Then you shouldn't need stuff in /etc/hosts.
<henninge> jtv: that's true
<henninge> jtv: I suspect that pbuilder updates the /etc/hosts, /etc/hostname, /etc/resolv.conf on each --login. That's why I was suggesting putting it there.
<jtv> wgrant: I have to get some food soon...  could you file a bug for the build[er] status icon?
<wgrant> jtv: Will do.
<jtv> thanks.
<jtv> noodles775: how about we call wgrant's feedback a UI review?
<wgrant> Hmm. There is no appropriate project any more :(
<jtv> wgrant: yeah... I've been dealing with that by calling it Soyuz.
<wgrant> I guess the views live in soyuz until somebody bothers to move them, so it can go there.
<jml> mwhudson, are you still around?
 * jml wants to know in how many places we've implemented two-stage kill
<noodles775> jtv: yeah, it's definitely valuable, so add it to the MP (when you create it, it'll make it much easier for a ui-reviewer to approve it :)
<gmb> stub: Sorry; didn't see your reply. So, my question:
<bigjools> jtv: are you going to fix that icon problem on /builders?
<gmb> stub: We know that checkwatches has a habit of holding transactions open whilst it does network stuff, which is Bad. However, we're not sure under what circumstances the transaction killer will come in and stomp on us. As far as we can tell, the first time we access a Storm object a new transaction is started, is that correct?
<gmb> (assuming there isn't a transaction already)
<stub> gmb: yes
<gmb> stub: So if we have a storm object locally, even if we're not updating it, just accessing its attributes, we're essentially going to hold a transaction open whilst we do network stuff that might require data from that object, correct?
<stub> From the Zope transaction side of things, yes, from the DB side of things, maybe.
<stub> 'maybe' because it will depend if Storm decides it needs to refresh something from the DB for some reason.
<stub> Oh... not accessing the store... just the object...
<stub> 'maybe' on both counts in that case :)
<gmb> stub: Hmm. allenap did some testing last night and it appeared that Storm *always* refreshes from the DB if you access anything other than the ID. Is that not necessarily the case?
<allenap> stub: I'll forward you the example I wrote.
<stub> I don't know.
<gmb> allenap: Ta.
<stub> I used to be the one campaigning to not keep objects alive over transaction boundaries :)
<jml> heh
<jml> why isn't hitchhiker packaged in karmic, dammit
<stub> allenap, gmb: Look like it is refreshing, yes. Same behavior with aborting and committing the transaction. Not sure if this is Storm or Storm+SQLObject compatibility standard.
<stub> def commit(self):
<stub>         """Commit all changes to the database.
<stub>         This invalidates the cache, so all live objects will have data
<stub>         reloaded next time they are touched.
<stub> Looks like Storm default behavior
<gmb> Hmph.
<gmb> stub, allenap: So, looks like local caching is the way to go. Feels like a bodge, though.
<stub> Running checkwatches in autocommit mode might be the simplest solution. It doesn't seem like the sort of problem you need transactional integrity for.
<allenap> gmb: Right, the floor is covered in egg shells.
<allenap> stub: That sounds good.
<stub> Also, consider pulling your objects from the slave and casting them to IMasterStore(obj) before making changes.
<allenap> stub: That also sounds good. Will the transaction killer not kill transactions from slave stores?
<allenap> stub: And, what's the right way to get checkwatches to run in autocommit mode?
<stub> It will, but there is no reason not to use the slave given you are already using possibly out of date information.
<stub> We are less aggressive on the slaves, and can be more lenient if necessary.
<stub> Sorry... IMasterObject(obj)...
<stub> Hmm... it is twisted isn't it.
<stub> Oh... it is also using LaunchpadScript as a base. There is an argument in there you can set.
<stub> isolation=ISOLATION_LEVEL_AUTOCOMMIT argument to run()
<stub> the constant imported from canonical.database.sqlbase
<jtv1> bigjools: I'd like to fix that, yes
 * gmb -> rebooting for updates
<stub> allenap, gmb: Using the slave would be quite simple. Just install the SlaveDatabasePolicy(), and cast objects you need to write to using writable_obj = IMasterObject(object).
<stub> with SlaveDatabasePolicy(): [...]
<allenap> stub: You're awesome, thank you.
<jml> any keen reviewers for https://code.launchpad.net/~jml/launchpad/sftp-poppy/+merge/21627
<allenap> jml: I'll do it.
<jml> allenap, thanks, but I've already swapped with noodles. your comments are still welcome though.
<jtv> henninge, wgrant: http://pastebin.ubuntu.com/397192/  :-(
<jtv> slave goes through:
<jtv> Iterating with success flag 0 against stage INIT
<jtv> Iterating with success flag None against stage UNPACK
<jtv> Iterating with success flag 0 against stage CLEANUP
<wgrant> jtv: Your verifySlaveBuildID is probably buggy.
<wgrant> It's being told to abort.
 * jtv looks at the master log
<jtv> wgrant: yup, that looks plausible!
<jtv> 2010-03-18 18:13:05+0700 [-] Builder 'bob' rescued from 'bf-test-5': 'BuildQueue 5 is not available: Object not found'
<jtv> wgrant: I rather suspect that ended up being the job id instead of the buildqueue id...
<jtv> Did we mention that this stuff was unnecessary complex?  :-)
<jtv> Yup, it's the job id.
<wgrant> Heh.
 * jtv has to run off for a bit
<jtv> make that quite a bit
<jtv-afk> danilos: confirmed 539499, fix is on the way.
<danilos> jtv-afk, it'd be better if you said "confirmed it's a fluke, fixed it in the wiki page" :)
<jtv-afk> danilos: better in what way, given that it's not the truth?
<danilos> jtv-afk, better if it was the truth ;)
<jtv-afk> wel duh!
<wgrant> jtv-afk: There is of course the other matter of lp-buildd being unhappy about being aborted during unpack.
<jtv-afk> wgrant: who wouldn't be?
<wgrant> Heh.
<jtv-afk> It's *just* *not* *fair*.  I tested against this exact problem, and the only way it could have failed to fail is if the job and the buildqueue happened to get the same id.
<wgrant> I wondered about how the tests managed to pass.
<jtv-afk> Yup.  Same id.  Pure, dumb coincidence.
<jtv-afk> I wonder how I'm going to explain this to my reviewer.
<jtv-afk> but first, afk.
<jml> noodles775, btw, the doc I was showing you is http://www.muthukadan.net/docs/zca.html
<jml> noodles775, it's an excellent guide to zope component architecture, imho
<jml> the buildbot is broken, is anyone working on it?
<jml> bigjools, http://www.kieranholland.com/code/documentation/nevow-stan/#the-tags-module
<noodles775> jml: I'm disabling the test that is (apparently) causing the issue.
<gmb> noodles775: Did you disable that failing test? db_lp is still in [testfix]
<gmb> I mean, I know I can force a build so that I can land a branch, but that seems to be the wrong thing to do.
<allenap> stub: Hi. transaction.get()._resources seems to still contain stuff even when isolation is set to auto-commit. Checking with the psycopg connection seems to give me valid information. Do you object if I go back to checking with the raw connection?
<stub> In autocommit mode, there is no point doing the checks at all.
<stub> I have no objections to reverting to checking the raw connection. You probably need to query the IZStorm utility for all stores and check each of them.
<allenap> stub: Okay, cool. It's useful at the moment, during development, to make sure that tests are valid (i.e. using auto-commit). I also want to prevent regression in the future.
<rockstar> matsubara-lunch, Ursinha, are we having a production meeting this week?
<Ursinha> rockstar: yes sir
<Ursinha> rockstar: why?
<rockstar> Ursinha, because I'm wondering where everyone is.
<rockstar> Oh crap, nevermind.  I forgot that the rest of didn't jump on the DST bandwagon yet.
<Ursinha> rockstar: :)
<Ursinha> matsubara, stub, check, gary_poster, rockstar, bigjools, sinzui, allenap: hey! what about a production meeting now? #launchpad-meeting @ Freenode
<rockstar> Hi
<gary_poster> Ursinha: :-) matsubara standing in for me today
<Ursinha> thanks gary_poster :)
<bigjools> Ursinha: soyuz is sprinting, can you ping me if you need anything
<Ursinha> sure bigjools, thanks
<bigjools> thanks Ursinha
<noodles775> gmb: As you've probably figured by now, I disabled the test with the testfix on devel only.
<noodles775> (missed your message earlier).
<kfogel> sinzui, in lib/canonical/launchpad/emailtemplates/email-processing-error.txt there is a URL ("https://help.launchpad.net/EmailInterface") that is to a non-existent page.  I can just fix it to include "Bugs", as in "https://help.launchpad.net/Bugs/EmailInterface", but the text around it implies that it's about more than bugs.  Are there any other email interfaces to Launchpad?
<kfogel> deryck: (if you know the answer to above, feel free to chime in)
<sinzui> kfogel: There is a page about review email command
<kfogel> sinzui: thanks
<kfogel> https://help.launchpad.net/Code/Review/#Email%20interface
<kfogel> frustrating that search (https://help.launchpad.net/HelpOnActions?action=fullsearch&context=180&value=email+interface) doesn't find that page!
<sinzui> I only search google to find our content. moin suck
<sinzui> kfogel: https://help.launchpad.net/Code/Review
<kfogel> sinzui: thanks.  see above; I found it
<james_w> is https://code.edge.launchpad.net/~james-w/launchpad/code-imports-use-ibranchtarget/+merge/21651 too big for a single branch?
 * james_w thinks of a better way to do it anyway
<jml> james_w, it's probably not too big for a branch like that.
<jml> james_w, you might want to arrange something with the reviewer though.
<james_w> I dropped the makeAnyCodeImport as makeCodeImport means the same thing essentially
<jml> james_w, ok.
<james_w> if we want the distinction then I can do it as a followup branch
<jml> james_w, yeah.
<james_w> ok, less than half the size, much better
<mwhudson> jml: hi
<jml> mwhudson, hi
<kfogel> aaaargh.  Is there no way in the help wiki to do a relative link to an anchor on a different page?  E.g., "[[Code/Review/#Email interface|foo]]" ?  It will come out as "https://help.launchpad.net/Code/Review/#Email+interface", which will not work.  But if you put in %20 for the space, it will escape the % and you'll get %2520.
<jml> kfogel, don't know, I'm afraid. #moin will probably be able to help though.
<kfogel> jml: I'm just adding a manual anchor "<<Anchor(EmailInterface)>>" and linking to that.  Harrumph.
<mwhudson> jml: you pang
<jml> mwhudson, I did?
<jml> mwhudson, hmm. let me consult my oracle.
<mwhudson> jml: about 8 hours ago, it has to be said
<jml> mwhudson, oh yes. in how many places have we implemented two-stage kill?
<jml> mwhudson, because I did a thing.
<jml> mwhudson, https://code.edge.launchpad.net/~jml/launchpad/sftp-poppy/+merge/21627
<jml> mwhudson, guess why the librarian was always taking 5s to shut down?
<james_w> jml: I don't think tachandler can import from lp
<jml> james_w, why not?
<james_w> I'm just finding the bug
<james_w> it's used on machines that don't have the full lp source tree available
<jml> james_w, !!!
<mwhudson> jml: um, did it send a signal then wait 5 seconds to see if it had died yet?
<mwhudson> there should be a comment in tachandler saying this
<jml> mwhudson, it sent a signal, didn't allow the zombie to be reaped, then kept polling every .1s for 5s to see if the zombie had gone yet (which it won't have)
 * mwhudson cries
<jml> there should be a comment in lp.services.osutils and maybe a unit test
<jml> or we could just fix whatever bug it is that forces us to duplicate code this way.
<mwhudson> the issue is that tachandler is used on the buildds
<jml> I mean, exactly how many more things do we need other than the vast size of our code base to block us from having a well organized code base
<jml> mwhudson, so tachandler should be in a separate tree then, I guess.
<mwhudson> there is a difference between what we want and what we get :/
<jml> mwhudson, indeed.
<james_w> jml:  bzr diff -r 10137..10137.2.2
<mwhudson> jml: as a hack to get this landed, you could define the functions in tachandler, but still put them in lp.services.osutils.__all__
<jml> mwhudson, yeah, that's what I was thinking
<jml> mwhudson, although, it's running through ec2 test right now
<jml> is there a test that will block this landing?
<mwhudson> unlikely
<mwhudson> i
<jml> :( :( :(
<mwhudson> i'm not really sure how you could write one, i guess you could execute a script that goes "import _pythonpath; from daeamons import tachandler; import sys; assert 'lp' not in sys.modules"
<jml> well, it's not even clear to me what the constraints are for the buildd
<jml> or what bits of launchpad it actually uses.
<mwhudson> jml: i believe you are physically close to people you can beat into telling you
<jml> mwhudson, yes. I might have to do that.
<jml> mwhudson, anyway, aside from that
<mwhudson> i think the only think it uses outside canonical/launchpad/build is tachandler though
<jml> mwhudson, how many two-stage kill implementations do we have in Launchpad?
<mwhudson> s/think/thing/
<james_w> I guess at one point it may have been desirable to stop people catting the LP tree while their package was building
<mwhudson> james_w: also, launchpad doesn't install so well on HPPA
<mwhudson> jml: um, at least two i guess>
<mwhudson> ?
<jml> mwhudson, where's the other one?
<mwhudson> jml: twistedsupport
<jml> oh of course
 * jml looks
<jml> mwhudson, btw, did you see the dependency graph?
<mwhudson> jml: only enough to notice that eog is an awful svg viewer
<jml> mwhudson, use dotviewer from pypy
<mwhudson> jml: if you insist
<jml> mwhudson, pypy is a python interpreter written in python, it's pretty cool
<jml> mwhudson, :P
<mwhudson> heh heh
<jml> mwhudson, Although see their current topic, "PyPy is a actually a visualization project, we just build interpreters to have interesting data to visualize"
<mwhudson> :)
<mwhudson> #pypy has a history of nicely surreal topics
<mwhudson> jml: it is entertaining how lp.testing is in the centre
<jml> mwhudson, yeah. also canonical.launchpad needs to die.
<mwhudson> well yes
<jml> man this buildd thing has made me angry
<kfogel> intellectronica: ayt?
<mwhudson> thumper: hello?
<james_w> jml: perchance still around?
<thumper> morning
<james_w> morning thumper
<thumper> hi james_w
<james_w> thumper: could you perhaps provide a little SQL help?
<thumper> sure
<james_w> CodeImportSet.composeQueryString() assumes a Product, but that will no longer always be true in my branch
<james_w> I need to rewrite the query to allow for using SourcePackage as well
<thumper> ok
<thumper> james_w: I'm just looking at your code import target branch
<thumper> hmm.. seems to have gone
<james_w> great
<james_w> I keep screwing up the initial settings and forgetting things like prerequisite branches
<thumper> :)
<thumper> james_w: can you paste me a link?
<james_w> https://code.edge.launchpad.net/~james-w/launchpad/code-imports-use-ibranchtarget
<james_w> https://code.edge.launchpad.net/~james-w/launchpad/code-imports-use-ibranchtarget/+merge/21651
<james_w> the intention of that one is just to refactor, not allow any new behaviour
<james_w> and to make the tests more explicit about if they require the code import to be against a product
<thumper> james_w: reviewed
<james_w> thanks
<mwhudson> flacoste: so buildbot farted again
<mwhudson> flacoste: did you read my mail yet? (i only sent it a minute or so ago)
<thumper> james_w: composeQueryString needs to die
<james_w> even better
<flacoste> mwhudson: switching to email...
<mwhudson> thumper, james_w: it's used for the search box on https://code.edge.launchpad.net/+code-import-list
<mwhudson> but maybe that page needs to die now
<mwhudson> (now that (a) we disable failing imports (b) +code-imports is more useful)
 * thumper looks at the page
<thumper> my vote is kill the page
<thumper> +code-imports is where we should add stuff now
<james_w> right, a search box isn't too useful as you could go from search->project->branch->info
 * james_w starts a new branch
<mwhudson> # XXX SteveAlexander 2005-04-11:
<mwhudson> # This is a total mess.  I need to work out what this all means.
<mwhudson> that's just the sort of thing you want to find!
<StevenK> Tests with errors:
<StevenK>    test_import_bzrsvn (lp.codehosting.codeimport.tests.test_workermonitor.TestWorkerMonitorIntegration)
<StevenK> Awwwww! But I didn't change that bit :-(
<mwhudson> StevenK: those tests have a history of intermittently failing :(
<mwhudson> StevenK: what was the error?  was this in ec2 ?
<flacoste> mwhudson: i followed-up by email with more info
<flacoste> mwhudson: i have a solution which looks too good to be true
<StevenK> mwhudson: Looks to be "DirtyReactorAggregateError: Reactor was unclean."
<mwhudson> flacoste: ah, this patch http://paste.ubuntu.com/397470/ makes the two tests that failed in ec2 pass with the mailqueue read-only
<mwhudson> StevenK: !
<mwhudson> StevenK: can you pastebin it?
<flacoste> mwhudson: good! you'll find a simpler ZCML in my reply
<flacoste> mwhudson: i'm fine with you landing this with the simpler ZCML file, we can leave the removal of the possible dead code in sendmail.py for a separate time
<mwhudson> flacoste: ah ok
<flacoste> since it would require extensive QA to make sure that we didn't break all mail-sending scripts
<StevenK> mwhudson: Right, a clag from the log around the error: http://paste.ubuntu.com/397471/
<mwhudson> StevenK: interesting
<mwhudson> StevenK: looks spurious though
<mwhudson> StevenK: can you file a bug with that output in it?
<StevenK> mwhudson: Against which bit?
<mwhudson> StevenK: launchpad-cpde
<mwhudson> flacoste: erm, did you mean include or includeOverrides in your zcml?
<mwhudson> oh, it needs to be files= not file= for the glob to work
<StevenK> mwhudson: Done, bug 541526
<mup> Bug #541526: Test error in test_import_bzrsvn <Launchpad Bazaar Integration:New> <https://launchpad.net/bugs/541526>
<mwhudson> StevenK: thanks
<flacoste> mwhudson: yeah, sorry, this was untested :-)
<mwhudson> flacoste: i'll have a merge proposal for you in about 10 seconds...
<mwhudson> flacoste: https://code.edge.launchpad.net/~mwhudson/launchpad/more-direct-delivery-pls/+merge/21685
<james_w> thumper: what count should be on https://code.edge.launchpad.net/ for imported branches?
<james_w> (see the bottom if you have forgotten it was there :-)
<flacoste> mwhudson: didn't you want to include something about running tests with chmod -w on the mailqueue dir?
<mwhudson> flacoste: hm, maybe, not sure how to arrange that though
<flacoste> mwhudson: how did you do it in your tests?
<mwhudson> flacoste: i hacked it in ec2test's testrunner and then removed the line from the makefile
<flacoste> hmm, right
<flacoste> mwhudson: we could check in the BaseLayer.testTearDown that the directory is empty
<flacoste> mwhudson: or change permission in BaseLayer.testSetUp and restore them in testTearDown
<mwhudson> flacoste: so something like this: http://paste.ubuntu.com/397482/
<flacoste> mwhudson: that's even simpler
<mwhudson> i guess it won't help people who don't use ec2 for their tests
<thumper> james_w: my suggestion would be working code imports
<flacoste> mwhudson: one problem at a time :-)
<mwhudson> flacoste: heh ok
<mwhudson> flacoste: ok, pushing that change, merge proposal diff will update in a sec
<james_w> thumper: just a count of all imports in the REVIEWED state?
<thumper> james_w: yeah
<thumper> what is it now?
<james_w> all "active"
<james_w> so REVIEWED and not for deactivated products/projects
<james_w> and tried at least once
<kfogel> Do we have a standard javascript equivalent of python's string.strip() method?  (strip leading/trailing whitespace, including newlines)?
<james_w> so REVIEWED is a fine approximation in my opinion
<mwhudson> REVIEWED is a decent approximation now we mark imports failing automatically
<kfogel> ah, trim()
<james_w> thumper: https://code.edge.launchpad.net/~james-w/launchpad/code-imports-use-ibranchtarget/+merge/21651 now with less sampledata if you want to re-review
<mwhudson> flacoste: ping? https://code.edge.launchpad.net/~mwhudson/launchpad/more-direct-delivery-pls/+merge/21685
<flacoste> mwhudson: sorry, forgot to approve it!
<flacoste> done
<mwhudson> ta
<james_w> and I'm throwing https://code.edge.launchpad.net/~james-w/launchpad/remove-code-import-list/+merge/21690 at ec2 to look for fallout
<maxb> jml: launchpad-dependencies shouldn't have an ubuntu1 suffix
<mwhudson> that didn't help
<thumper> mwhudson: the split?
<mwhudson> thumper: hi
<mwhudson> that was all a bit off
<mwhudson> odd
<mwhudson> thumper: i reviewed your branch anyway
<thumper> mwhudson: yeah, saw that, thansk
 * thumper goes to fire off three headless tests
#launchpad-dev 2010-03-19
<thumper> james_w: are you wanting me to land ~james-w/launchpad/code-imports-use-ibranchtarget ?
<wgrant> How does the webservice versioning work? Are we really committed to maintaining the full 1.0 API for 5 years?
<thumper> wgrant: ask leonard
<thumper> I'm not sure how it works
<mwhudson> i've been wondering about that
<jml> maxb, hmm. what did I do wrong to make that happen?
<jml> maxb, I was doing what jelmer told me to.
<mwhudson> jml: dch adds -ubuntu to the version number by default
<jml> ahh, I see.
<wgrant> It seems like only a tiny fraction of the API is actually going to be used by anything.
<jml> so what should I do to correct my mistake?
<wgrant> And it's going to be rather prohibitively model-change restrictive.
<wgrant> jml: Have you uploaded it to the PPA yet?
<jml> wgrant, yes, I have.
<wgrant> Ah.
<wgrant> You can't.
<wgrant> Unless you increment the version again pointlessly.
<jml> ok.
<jml> wgrant, but say I had a very good reason for incrementing the version, I could do that and remove the suffix?
<wgrant> jml: Right.
<mwhudson> hopefully the next person to do an upload will remember to clean up the number
<wgrant> Speaking of launchpad-dependencies...
<wgrant> I noticed yesterday that we seem to have an awful lot of external dependencies built into the tree itself.
<jml> well, I've been thinking about adding a suggests or recommends for snakefood so I can land that dependency graph
<wgrant> Things like BeautifulSoup and oauth, which have packages.
<wgrant> And we don't carry any local changes to them.
<wgrant> Does anybody know why they're in the tree, rather than using one of the other three external dep mechanisms?
<wgrant> We also have like 10000 lines of unused JavaScript libraries in contrib/.
<jml> wgrant, I don't.
<jml> wgrant, ... and a bunch of useless .dia & html files in high-level directories
<jml> wgrant, I would look favorably on patches that delete these things.
<wgrant> And a whole lot of obsolete docs, yeah.
<mwhudson> jml, wgrant: https://dev.launchpad.net/LaunchpadPpa?action=diff&rev2=23&rev1=22
<jml> wgrant, I guess the thing is, no one is 100% sure and that's enough to make procrastinating on deleting them very tempting
<wgrant> And a few broken makefiles.
<jml> mwhudson, goodness me there's a wiki page
<mwhudson> wgrant: i think oauth is a bit hacked, maybe?
<mwhudson> or was
<mwhudson> or something
<wgrant>  31 files changed, 9270 deletions(-)
<wgrant> mwhudson: Hm, let me diff again.
<wgrant> But the tests pass with the system one.
<mwhudson> jml: i'm shocked, yes shocked! that you didn't find a wiki page!
<jml> mwhudson, it's a disgrace. I blame those bureaucrats in Brussels.
 * mwhudson adds a link to https://edge.launchpad.net/~launchpad/+archive/ppa
<mwhudson> jml: and cancer causing immigrants
 * mwhudson hits http://www.qwghlm.co.uk/toys/dailymail/ a few times
<jml> Are rising house prices hurting Britain's swans!?
<wgrant> Or will cancer defraid them?
<wgrant> Er, defraud.
<jml> anyway, I'm going to bed. Maybe sometime soon I can land my zope.testing upgrade.
<thumper> I need to finish this damn work so I can read my book!
 * thumper must not get distracted
<wgrant> Can someone please land https://code.edge.launchpad.net/~wgrant/launchpad/show-package-sets-in-queue/+merge/21521 and https://code.edge.launchpad.net/~wgrant/launchpad/export-basic-binary-download-stats/+merge/21544?
<wgrant> Is Launchpad going to burn down if it tries to render a 10000 line diff in an MP?
<mwhudson> wgrant: no, not any more
<wgrant> Ah, good.
<mwhudson> it caps the lines (and bytes) displayed
<wgrant>  31 files changed, 9270 deletions(-)
 * thumper has finished (subject to review) the branch that makes merge proposal emails sent by a job
<wgrant> What's the status of not sending emails for WIP MPs?
<mwhudson> wgrant: dependent on thumper's just finished branch
<wgrant> Excellent.
<thumper> although it is such a wonderful afternoon, I'm losing the will to write the last pipe
<thumper> the last pipe is the bit that actually gets all the emails to send...
<thumper> mwhudson: if your afternoon is like mine, you could take a look at the two reviews on the review channel :)
<mwhudson> yeah ok
 * wgrant has two reviews that are already reviewed but need landing.
<mwhudson> wgrant: does noodles need to ui-stamp https://code.edge.launchpad.net/~wgrant/launchpad/show-package-sets-in-queue/+merge/21521 ?
<wgrant> Hmm, possibly.
<mwhudson> wgrant: also, is direct_inclusion what you really want?
<mwhudson> i admit to not being sure about this area
<wgrant> mwhudson: It's what the archive admins want.
<wgrant> I've talked to the three people who are going to be using that page most.
<mwhudson> ok
<mwhudson> good enough for me
<wgrant> mwhudson: Thanks/
<mwhudson> thumper: i don't think i'm going to get through even the first of your branches, sorry
<thumper> mwhudson: that's ok
<thumper> I'm struggling with warsaw's law myself
<mwhudson> heh heh
<thumper> mwhudson: what are your thoughts about modifying lib/contrib/glock.py?
<thumper> mwhudson: the losas have complained that it doesn't give the lock path when creating the lock
<thumper> mwhudson: I've traced the log output to that file
<mwhudson> oh right
<thumper> oh FFS
<thumper> lib/lp/soyuz/tests/../doc/buildd-slave.txt failed on ec2
<mwhudson> thumper: that file hurts my eyes :(
<mwhudson> i also like the way it's in contrib and has no license information i can see
<thumper> wgrant: does that file have intermittant failures?
<wgrant> thumper: No.
<wgrant> thumper: What was the failure?
<thumper> wgrant: http://pastebin.ubuntu.com/397585/
<thumper> test_min_time_to_next_builder (lp.buildmaster.tests.test_buildqueue.TestMinTimeToNextBuilderMulti) failed in other ec2 test run
<wgrant> That's not one I've seen before.
<mwhudson> glock seems to be lgpl
<thumper> AssertionError: Wrong min time to next available builder (30 > 29)
<wgrant> And it passed for me on devel just an hour or two ago.
<mwhudson> test min time to builder was the one we timezone fixed in a stand up
 * StevenK blinks
<mwhudson> it seems to be cursed
<wgrant> Yeah, that one I recognize.
<StevenK> File "lib/lp/soyuz/tests/../doc/buildd-slave.txt", line 101, in buildd-slave.txt
<StevenK> ...
<StevenK> + ProtocolError: <ProtocolError for localhost:8221/rpc/: -1 >
<wgrant> What What should I do about the pointless embedded code copies I mentioned earlier?
<wgrant> oauth and BeautifulSoup are unmodified copies of old upstream versions.
<mwhudson> well i'm going to go and watch the cricket
<mwhudson> good weekend, all!
<StevenK> Ooh, who's playing?
 * StevenK picks on wgrant 
<wgrant> StevenK: Hm?
<StevenK> Wow, a bug in Launchpad against Soyuz you *aren't* subscribed to.
<wgrant> Ah, just got the email.
 * wgrant isn't a reviewer.
<wgrant> StevenK: Does everybody want that?
<wgrant> It seems like a good idea, except that it removes the remaining tiny bit of security against compromises.
<StevenK> wgrant: I know you're not, but you're in my timezone and you know the code
<thumper> mwhudson: TTFN
<thumper> I hate code that lies
<thumper> however if I want to make it right, I've got a metric shit load to change
<thumper> arse
<wgrant> What's lying?
<thumper> the code if I don't fix it
<thumper> wgrant: we currently have a cronjob called mpcreationjobs.py
<thumper> wgrant: a horrible name, but one I want to change to merge-proposal-email-jobs.py
<wgrant> I'd always assumed that that created MPs, until I noticed that there were no emails coming out until I ran it.
<thumper> wgrant: no that is createmergeproposals.py
<thumper> wgrant: it is a point of much confusion
<thumper> even for me
<wgrant> Heh.
<wgrant> A little WTFing and detective work led me to that conclusion after a while.
<thumper> yeah, well lets aim for less WTFs per minute
<spm> thumper: merge-proposal-email-jobs.py + 1 from losaville, so long as you also modify the name it uses in scriptactivity to be the same. script names that don't match their logged names is a source of pain and frustration.
<thumper> spm: yeah, I also need to change a lot of config stuff
<thumper> spm: also, I've done a drive by fix for adding the lock file to 'creating lockfile' info logging line
<spm> woohoo!
 * thumper EODs
<thumper> can't think straight any more
<wgrant> OOPS-1539ED198
<wgrant> What's the culprit on that?
<StevenK> Timeout in SQL
<wgrant> Well, yes.
<wgrant> No obviously big queries?
<StevenK> No, it's a nice and short one
<wgrant> Hmmm, query count is ~700. That page sucks.
<thumper> wgrant: which page?
<wgrant> thumper: DistroSeries +queue
<wgrant> It does some pretty ugly and elaborate prejoinish stuff, but doesn't make use of it very well.
<wgrant> It would be nice to see the query log, but I have a bit of an idea of what's going on locally.
<wgrant> I wish our sampledata wasn't illegal :(
<adeuring> good morning
<maxb> Hrm. We have all kinds of version skew in launchpad-dependencies across hardy/jaunty/karmic/lucid. people haven't been copying it after upload
 * maxb fixes
<maxb> jml: You didn't put your upload in bzr either :-/
 * maxb imports it
<jml> maxb, are instructions to do that on the wiki page?
<jml> maxb, because I really didn't know what I was doing (as you've probably guessed)
<jml> wgrant, let's delete the sample data!
<persia> Is it possible to create legal sample data?
<jml> persia, I can't begin to explain how little I enjoy legal discussions about code.
<maxb> jml: The dch gotcha was not en-wikied (mwhudson has since docced), but the wiki page does say about the package is in bzr, and mention using bzr mark-uploaded to tag it
<jml> maxb, ok, thanks.
<jml> maxb, I only found out about the wiki page after I tried the upload
<jml> maxb, mwhudson has linked to the wiki page from the ppa description now, so it should be all good.
<jml> tee hee
<jml> every time I say "ppa"
<jml> I hear a noise from bigjools' laptop
<wgrant> Haha.
<mwhudson> heh heh
<bigjools> jml
<bigjools> jml
<bigjools> jml
<wgrant> This reminds me of thumper vs. bigjools in Wellington.
<bigjools> screw you hippy!
<jml> hahaha
<bigjools> I bet it beeps when I say "hippy"
<thumper> ppa
<thumper> heh
<bigjools> joke's on you, it doesn't beep unless I am not looking at the channel :)
<wgrant> bigjools: +queue query counts make me very sad.
<mwhudson> jml: wasn't i talking to you about three hours ago?
<wgrant> It is so incredibly inefficient.
<jml> mwhudson, yes.
<jml> mwhudson, I'm not well rested
<mwhudson> jml: last day of the sprint at least, i guess?
<bigjools> wgrant: it used to be 4000+
<jml> am I ok to land a change that relies on subunit 0.0.4?
<bigjools> I optimised it down to <1000
<jml> mwhudson, yeah :)
<bigjools> it's also f***ing hard to optimise
<stub> Which page is this?
<wgrant> QueueItemsView
 * wgrant finds a particularly horrible example.
<wgrant> https://edge.launchpad.net/ubuntu/hardy/+queue?queue_state=3&queue_text=
<wgrant> Look at that time.
<stub> Got an example that won't timout on me ;)
<mwhudson> 1 â 30 of 128120 results
<mwhudson> that doesn't seem like it's going to work well
<wgrant> That one has ~700 queries in 15-16 seconds.
<stub> nm - it doesn't look suitable as a memcached test
<wgrant> mwhudson: Why not? If it batches properly (which it does), it doesn't need to suck.
<mwhudson> i guess
<stub> Retrieving the last 50 items from 100k ordered results usually requires materializing those results, sorting them and returning the final records.
<wgrant> Well, where is the time spent in OOPS-1539ED198?
<wgrant> In the initial query?
<wgrant> It's a simple one like "SELECT PackageUpload.archive, PackageUpload.changesfile, PackageUpload.date_created, PackageUpload.distroseries, PackageUpload.id, PackageUpload.pocket, PackageUpload.signing_key, PackageUpload.status FROM PackageUpload WHERE packageupload.distroseries = 10 AND packageupload.archive IN (1, 12) AND packageupload.status IN (0) ORDER BY PackageUpload.id DESC LIMIT 8 OFFSET 0"
<wgrant> stub: Is that going to be horrifically bad?
<bigjools> the page is returning a lot of results from a lot of different places.  I'd like to throw away the ORM queries and use storm.execute quite frankly
<stub> It can be very bad. Depends.
<bigjools> it's the most complicated page I know of
<jml> bigjools, why would storm.execute be better than storm.find in this case?
<bigjools> fewer queries
<jml> bigjools, really?
<wgrant> storm.find()s can become arbitrarily complex...
<wgrant> No need to resort to raw SQL.
<bigjools> as I explained to jono in person, it's the property traversals that kill us
<wgrant> Not if we emulate a prejoin.
<bigjools> .execute or .find, whatever
<bigjools> I disliked storm query syntax
<bigjools> err dislike, present tense
<wgrant> Although it appears that Storm can't really prejoin ReferenceSets :(
<stub> wgrant: A zillion LibraryFileAlias retrievals - loading them one at a time doesn't help. Another zillion SourcePackagePublishingHistory requests, one at a time. Death by 1000 cuts.
<bigjools> prejoins in Storm make my eyes bleed
<stub> I wrote a helper for that.
<bigjools> stub: exactly
<wgrant> stub: Where is this helper?
<stub> wgrant: lp/services/database/prejoin.py
<wgrant> stub: Ahh, handy.
<stub> Do we have a wiki page describing the current qa process and wot i should do with these qa- tags?
<stub> nm - found it
<stub> https://dev.launchpad.net/PolicyAndProcess/PreReleaseQAProcess/Experiment
<stub> But I don't understand 'when a bug is fixed in rcmode'
<jml> hmm.
<jml> so, now that I've got subunit in launchpad-developer-dependencies, do I need to create a new ec2 image?
<jml> yes, apparently
<deryck> Morning, all.
<jml> deryck, hi
<jml> henninge, hello!
<henninge> Hi jml! ;)
<jml> henninge, I want to be one of VALID_AMI_OWNERS
<jml> henninge, I see that you are one of those
<jml> henninge, how do I join the club
<henninge> jml: just add your number there
<jml> henninge, which number?
<henninge> jml: AWS account number
<jml> henninge, you mean the one in the form nnnn-nnnn-nnnn?
<jml> henninge, mine starts with a zero
<henninge> jml: yes but without the -
<henninge> let me see
<henninge> jml: leading 0 seems to be fine, Diogo's like that, too
<jml> henninge, well, I'll see if it works for VALID_AMI_OWNERS, which seems to treat these things as numbers
<henninge> oh
<jml> henninge, ~/.ec2/aws_user -- should that have hyphens in it?
<henninge> jml: mine doesn't
<jml> henninge, ok, thanks.-
 * jml tries
<henninge> good luck, jml ;-)
<wgrant> bigjools: So, I've so far cut the query count by 2/3.
<jml> meh. I need to add subunit to the ppa.
<bigjools> wgrant: \o/
<bigjools> is that using that helper?
<bigjools> the big problem was always with having lots of custom uploads on the queue, that wasn't optimised
<wgrant> bigjools: No, that doesn't help for ReferenceSets, it seems.
 * jml thinks
<wgrant> Even if the objects themselves are cached, retrieving a ReferenceSet makes a query to get all the object IDs.
<jml> meh :(
<wgrant> stub: Do you know any way around that?
<stub> No. To discover the results of a database query you need to ask the database.
<wgrant> stub: Well, yes, but there's no reason I can see why one couldn't do that in single big query in a prejoinish sort of thing.
<stub> Sure
<wgrant> But I can't see a way to do that with Storm.
<stub> store.find( (Foo, Bar, Baz, Bing), ....) should do the big query
<stub> And the prejoin helper can be used so you don't need to change callsites to handle the bigger list of results.
<wgrant> That will pre-cache all of the objects. But accessing a ReferenceSet will still make a query.
<stub> I don't follow. Accessing the ReferenceSet makes a query because you need to ask the database for the results at some point.
<stub> store.find(...) doesn't issue a database query. Accessing it does.
<wgrant> I can do a prejoin which will avoid further queries for Reference attribute accesses. I can also formulate a query which will calculate all of the ReferenceSets for all of my objects in one query.
<wgrant> But I cannot use the results of the second in a normal ReferenceSet access.
<wgrant> SQLAlchemy can do eager loading of collections like this, even automatically.
<wgrant> But it appears that Storm cannot, even manually.
<wgrant> This makes me sad, and pages slow.
<stub> I don't see why you 'cannot use the results of the second in a normal ReferenceSet access'.
<stub> Are you trying to use it in a subquery or something?
<wgrant> I have PackageUpload.(sources|builds|customfiles) and a horrible huge hierarchy underneath each.
<wgrant> I have written a query manually to grab them all without making thousands of queries.
<wgrant> But even if I've preloaded all of the objects with that query, PackageUpload.sources[0] will still hit the DB. I want it to not. The way I am doing it now is by using a separate data structure and avoiding the ReferenceSet entirely.
<stub> So you materialized the list or something before calling PackageUpload.sources[0] ?
<wgrant> I have a list equivalent to PackageUpload.sources, from my massive query.
<stub> And is this a result set, or a materialized list?
<wgrant> A materialized list.
<stub> So you might have thrashed the cache.
<stub> If you materialized 100k items, they won't all fit in Storms cache.
<stub> Erm... hang on...
<wgrant> But I haven't told it enough to cache the ReferenceSet.
<stub> I think I'm confusing myself.
<wgrant> I've just made an equivalent query myself.
<stub> You don't cache reference sets. You cache objects.
<wgrant> Right.
<wgrant> But there's no reason I shouldn't be able to cache reference sets.
<stub> You did by materializing it
<wgrant> I didn't materialize the ReferenceSet itself. I couldn't materialize them all in less than a lot of queries, so I created a parallel data structure which I *could* complete in one query.
<wgrant> So Storm is being obstructive here, forcing me to avoid the objects in order to get reasonable performance.
<jml> wgrant, that sounds like an argument for fixing storm.
<wgrant> jml: Yes, that is my point :)
<jml> wgrant, good good. carry on.
<wgrant> I was just wondering if it was already possible.
<wgrant> It appears to not be.
<wgrant> Which seems odd.
<stub> I got confused - didn't click ReferenceSet rather than ResultSet
<wgrant> Ahh.
<stub> So you are building the equivalent of PackageUpload.sources, but have no way of telling Storm that your list should be used instead of PackageUpload.sources
<wgrant> stub: Right.
<stub> I assume 'packageupload.sources = my_list' doesn't work?
 * stub hasn't really used ReferenceSets
<wgrant> stub: Ah, that does seem work (but requires relaxing of security proxies, and is really ugly, and there's no reason Storm can't do that itself).
<stub> First step would be coming up with the syntax.
<stub> Dunno if the SQLAlchemy syntax can be stolen
<wgrant> http://www.sqlalchemy.org/docs/05/ormtutorial.html, search for 'eager'
<wgrant> I'm not sure how Storm-appropriate that would be.
<wgrant> And I'm particularly unsure on how to refer to deeper collections.
<wgrant> Ah, and that workaround won't work with Storm trunk.
<wgrant> Assigning to a ReferenceSet will deliberately crash.
<stub> That looks like what I'd want 'prejoin' in Storm to be able to do, and eager is a better name (still sucky, but wtf does prejoin mean?)
<wgrant> Heh.
<wgrant> It's interesting that Storm has a prejoin implementation -- but only in the SQLObject compat layer.
<bigjools> I miss prejoins in SQLObj :/
<henninge> danilos: I found out what my problem was.
<danilos> henninge, what was it?
<henninge> danilos: the tarfile data was opened with mode "r|*" which is different from "r:*" in that it does not allow random access.
<henninge> danilos: is there a special reason for using "r|*", like performance?
 * henninge was not aware of that difference.
<danilos> henninge, I don't know
<danilos> henninge, note that tarfiles are stream-based, so seeking around them is slow; is there any reason why you wouldn't re-open the file and start over?
<danilos> henninge, when I say "stream-based" I mean: "you have to go through the entire file to read all the file names" and "optimized for saving to and reading off tapes" :)
<henninge> danilos: actually, the files are accessed in the right order but unless I use the iterator, I cannot access them.
<henninge> I mean, I get the error.
<henninge> even with re-opening, I mean.
<danilos> henninge, oh, very, very interesting
<danilos> henninge, sounds like a bug in tarfile module then :)
<henninge> yup
<henninge> danilos: btw, the tarball I am working on is in memory (StringIO)
<danilos> henninge, right, that shouldn't make a big deal, seeks usually work pretty well in memory :)
<henninge> shouldn't random access be just as cheap there as sequential access?
<danilos> henninge, it probably is, but code might not be optimized
<danilos> henninge, it should be easy to check if it makes any difference :)
<danilos> henninge, btw, have you tried doing a seek on StringIO yourself before re-opening?
<henninge> danilos: no, I have not.
<henninge> but it had crossed my mind
<danilos> henninge, anyway, at least confirm it's not seriously affecting performance by timing opening and listing files for a tarball with a big number of files
<danilos> henninge, if it's not outrageously different, go with the simplest solution; if it is, let's find better solutions :)
<henninge> danilos: ok, let me see what I can do.
<danilos> henninge, fwiw, random-access might even be faster for tarfile access if implemented with some heuristics :)
<deryck> adeuring, hi.  I took over Bug #531003 on the board, just to play with queries and follow up with bdmurray.
<mup> Bug #531003: searchTasks with hardware_is_linked_to_bug parameter times out regularly <oops> <ubuntu-qa> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/531003>
<adeuring> deryck: ok
<deryck> adeuring, like you said, though, I'm not sure I see a way to improve this.
<adeuring> deryck: I should probably try to debug my memory. I might have thoufh about these timeouts some time ago, but can't remember any details...
<adeuring> s/thoufh/thought/
<deryck> adeuring, bdmurray has debugged it's just the hardware_is_linked_to_bug option, which just adds the join.  So there's thay M2M table, but it's needed.  And how can you query that any more quickly?  I don't see a way.
<adeuring> deryck: right...
<deryck> and like you say, after caching, it works reasonably fast.
<danilos> deryck, sometimes postgres chooses suboptimal joins as well
<danilos> deryck, s/joins/query plans/
<danilos> deryck, you can influence that with subselects usually
<deryck> he teases and leaves....
<deryck> I don't suppose stub is around at this hour?
 * stub hides behind the couch
<deryck> stub! I see you. :-)
<gary_poster> deryck, I believe that bug 541637 is not a malone issue, but entirely a launchpadlib/webservice issue.  If you agree, I'll change things around one way or another so that launchpadlib and foundations are on the hook, not launchpadlib and malone.  Agree?
<mup> Bug #541637: searchTasks doesn't return all matching tasks?! <api> <launchpadlib :New> <Launchpad Bugs:New> <https://launchpad.net/bugs/541637>
<deryck> gary_poster, yes, thanks!  I was going to ping you today about that one actually.
<gary_poster> cool, deryck, on it
<sinzui> EdwinGrubbs: bac: stand-up in 2 minutes
<barry> is it a known bug that when you add a comment to a bug report in today's launchpad, you do not get a page updated with the comment, but only a weird text that says: [object Object]
<barry> ?
<barry> reloading the page shows my comment
<sinzui> barry: I do not experience that issue. I think you got a timeout on the callback
<barry> sinzui: ok.  i'll keep an eye on it and if it comes up again i'll file a bug
<adeuring> stub: still around?
<sinzui> Am I insane. Do I really see two identical WindmillTestCase classes defined in lp.testing?
<didrocks> is someone available to finish reviewing my branch that has been first reviewed 10 days ago? https://code.edge.launchpad.net/~didrocks/launchpad/expose-sshkeys-bug-357235/+merge/20995
<bigjools> didrocks: you're best asking in #launchpad-reviews
<didrocks> bigjools: doing that now, thanks
<jml> !!!
<jml> http://launchpadlibrarian.net/41290965/buildlog_ubuntu-hardy-i386.subunit_0.0.5-1~ppa1_FAILEDTOBUILD.txt.gz
<jml> my life is strewn with cowpats from the devil's own satanic herd.
<persia> Have you considered going into the home fueling business?
<barry> james_w: ping
<sinzui>  gary-lunch: ping
<gary-lunch> sinzui: oops, not been at lunch for quite awhile now
<sinzui> gary_poster: I am trying to register a view in a test so that I test some yui testunit files, but my registration does not let me access http://launchpad:8085/+yui-unittest
<sinzui> This is what I have tried: https://pastebin.canonical.com/29442/
<gary_poster> looking
<gary_poster> sinzui: everything looks fine to me.  If I were trying to make it work, the first thing I would do is try to be more promiscuous in my registration--that is, instead of (ILaunchpadRoot, IDefaultBrowserLayer) I might go for (Interface, IRequest).  If that failed, the next thing I might do is
<gary_poster> see if my reading of the error is correct--IOW, if I am getting a component lookup error, is it for the object I registered, or is the registered object being found, and some of its code blowing up?
<gary_poster> In this case, maybe LaunchpadView actually does something interesting.
<gary_poster> I'll also doublecheck the interface for registerAdapter to make sure order of arguments is correct, but it looks right, and I suspect you already did that double-checking.
<sinzui> gary_poster: Well I am very relieved at your opinion and suggestion. I can try a general interface and a simpler view. The interfaces used are the same I saw in the debugger then +dotspecgraph registered.
<gary_poster> sinzui: ack, like I said, it didn't look wrong, but that's the kind of thing I'm suspicious of first
<wgrant>         path_expression="string:+binaryhits/${binary_package_release/name}/${binary_package_release/version}/${binary_package_release/build/distroarchseries/architecturetag}/${day}/${country/iso3166code2|string:unknown}"
<wgrant> Hmmm.
 * maxb raises an eyebrow at no one complaining that lp-dev-deps is currently uninstallable on karmic
<wgrant> Anybody around who can look at the traceback for OOPS-1539L2382?
<beuno> wgrant, sure
<beuno>   LocationError: (None, 'filesize')
<beuno> wgrant, http://paste.ubuntu.com/398034/
<wgrant> Huh.
<wgrant> Looks like the librarian GC is being overzealous?
<maxb> Poll for anyone who's listening: How much do you care about using versions that mention launchpad in the launchpad PPA, rather than generic ones like 0.0.5-1~ppa1 ?
 * wgrant likes having the PPA name in there, but doesn't really care.
 * maxb wonders if there's a ~launchpad member who could delete subunit 0.0.5-1~ppa1 (which FTBFS, so no binary packages exist) so it can be replaced with 0.0.5-1~hardy+launchpad1
<beuno> maxb, sure, point me to it
<maxb> https://edge.launchpad.net/~launchpad/+archive/ppa/+delete-packages?field.name_filter=subunit&field.status_filter=published&field.series_filter=
<wgrant> maxb: Ah, you only have launchpad.Append, and that doesn't grant you deletion rights?
<wgrant> That's unfortunate.
<maxb> (URL by construction, since I don't have permission to visit it)
<beuno> maxb, nuked
 * maxb will upload a version that builds shortly
<beuno> I'm off to dinner
 * maxb is more than slightly surprised that we've not needed dh7 in the launchpad ppa before
#launchpad-dev 2010-03-20
<wgrant> maxb: It doesn't depend on -backports?
<cody-somerville> Ugh. Soyuz, how I want to refractor thou.
<wgrant> Which bit in particular?
<cody-somerville> ummm... pretty much all of it.
<wgrant> Heh.
<cody-somerville> First I'd refractor publishing
<jpds> Just make the archive publisher run faster, kthx.
<cody-somerville> The archive publisher isn't really that slow TBH.
<jpds> 20 minutes... unacceptable. ;)
<cody-somerville> Its 5 minutes
<jpds> No.
<wgrant> Not for the primary archive.
<cody-somerville> The primary archive doesn't count.
<wgrant> What do you see as wrong with the current publisher design?
<wgrant> Apart from the code being foul and the names sucking and blah blah blurgggh.
<cody-somerville> Have you looked at a graph of publishing tables?
<wgrant> Heh. Yes.
 * wgrant quickly extends http://people.ubuntu.com/~wgrant/launchpad/buildfarm/current-build-model.png with them and cries.
<cody-somerville> I'm implementing the ability to delete PPAs.
<cody-somerville> The relationship between SourcePackagePublishingHistory and SourcePackageRelease is a mess.
<wgrant> To what level are you implementing it?
<wgrant> I presume the DB structures will stay.
<cody-somerville> Yes, unfortunately.
<cody-somerville> SourcePackageRelease is attached to an archive (ie. upload_archive) when you copy a package to another archive it just creates a new SourcePackagePublishingHistory record that points to the SourcePackageRelease for the source archive.
<wgrant> Yes.
<wgrant> But if you're leaving the actual Archive around, i don't see a problem with this.
<wgrant> upload_archive is only used when saying 'Copied from Some Archive'
<cody-somerville> I suggested that bug bigjools doesn't want to keep it around, we really want to delete the archive.
<wgrant> That's deleting incriminating history, though.
<wgrant> I don't like deleting history, particularly when it contains records of what arbitrary code people have been executing.
<cody-somerville> wgrant, If you copy a copied package then it'll say you copied it from original archive the package was uploaded to, not the archive you actually copied the package from.
<wgrant> cody-somerville: Yes. That is a stupid bug.
<wgrant> It will be fixed along with recording who did the copy and when.
<cody-somerville> Its not a stupid bug, its a critical design flaw. lol.
<wgrant> It's not that much of a problem, is it?
<cody-somerville> (or more like a lazy hack)
<wgrant> Lazy hack that is easily removed, right.
<cody-somerville> wgrant, It is because we have to keep an archive's SourcePackageRelease around if there are SourcePackagePublishingHistory records for other archives that reference it.
<wgrant> cody-somerville: You're actually considering deleting the SourcePackageRelease? Why?
<wgrant> And we already have logic for determining this, in the PPA file expirer.
<wgrant> Please reuse it, or somebody is again going to get very close to inadvertently deleting a few hundred gigabytes of packages that needed to be kept.
<cody-somerville> bigjools said he wants it all deleted. I don't see any reason for keeping it around either myself since it'll have no references to it.
<cody-somerville> wgrant, file?
<wgrant> cody-somerville: The script that expires PPA librarian files has queries to determine whether an SPR is currently in use elsewhere.
<wgrant> Although I guess you really just care about whether it's been used outside that archive.
<wgrant> which makes the query trivial
<cody-somerville> heh, I wouldn't call the query in expire_archive_files trivial.
<wgrant> No, that one isn't.
<wgrant> But your search needs to be more restrictive, so it is.
<cody-somerville> wgrant, Maybe you're more familiar with Storm and can recommend a more efficient storm query: http://pastebin.ubuntu.com/398085/
<maxb> Actually I quite like the feature that if you copy a copied package, it still references the original upload archive
<cody-somerville> wgrant, oops, extra line in there.
<cody-somerville> forget line 3 exists
<wgrant> cody-somerville: So you /are/ actually removing the Archive?
<cody-somerville> wgrant, Thats the current plan, yes.
<wgrant> Ew.
 * maxb seconds that
<wgrant> Apart from the impossibility of setting upload_archive = None, and my revulsion at history deletion, that looks OK.
<maxb> Why is it so important to delete the history?
<cody-somerville> We're patching SourcePackageRelease to drop the NOT NULL constraint.
<wgrant> As long as you also remove everything else associated with the SPR.
<wgrant> (SPRFs, that sort of thing)
<cody-somerville> Yes, we're deleting everything.
<wgrant> This is making it very easy to remove a complex hierarchy of important data and accountability.
<cody-somerville> If the archive is gone, its gone.
<cody-somerville> Sometimes deleting something means really deleting something.
<maxb> And why is this desirable !??!
<cody-somerville> why is it not?
<cody-somerville> We don't want to keep records in the database for a deleted archive.
<cody-somerville> We have the ability to disable archives when we want to do that.
<cody-somerville> For example, copy archives and rebuild archives would be very nice to be able to delete.
<wgrant> Why?
<wgrant> I want to be able to look at the rebuild history.
<wgrant> What benefit does deleting the records provide, besides allowing a single malicious or mistaken click to delete a vast amount of irreplaceable data, and saving a tiny bit of DB space?
<cody-somerville> Do you really care about rebuild history for a rebuild that occurred for a 5 year old release?
<cody-somerville> and the launchpad database is like terabytes so I think the latter is valid, yes.
<cody-somerville> Also, I don't think this feature will be used primarily to delete archives which actually history but instead ones without.
<cody-somerville> for example, if someone wants to rename their account
<cody-somerville> an empty PPA unfortunately blocks that
<wgrant> If they want to rename their account, then we "delete" the PPA.
<wgrant> We hide it and remove it from disk.
<wgrant> We probably even remove the librarian files.
<cody-somerville> manually
<maxb> Incidentally, why don't we just rename the PPA on disk too?
<wgrant> maxb: The indices need regeneration.
<wgrant> cody-somerville: The use of present tense there was perhaps a mistake.
<maxb> ok, delete and republish from scratch then
<wgrant> maxb: Right.
<wgrant> The publisher could just say "oh look, there's no directory there, but there is stuff to publish. Let me take a few more seconds and carefully publish it from scratch"
<maxb> I get that discouraging PPA owners from breaking people's sources.lists is good, but an "are you sure?" would be good enough, I would think
<wgrant> One would think so.
<wgrant> We allow users to break their OpenIDs after such a warning, for example.
<maxb> Heh, icon positioning fail on https://edge.launchpad.net/builders
<wgrant> Yeah, I reported that a couple of days ago.
<wgrant> A somewhat intermediate step in the generalisation refactoring landed.
<wgrant> Bug 540819
<mup> Bug #540819: BuildFarmBuildJob icons on /builders misplaced <trivial> <ui> <Soyuz:Triaged by jtv> <https://launchpad.net/bugs/540819>
<maxb> The PPA publisher seems to take an abnormally long time these days
<wgrant> maxb: Howso?
<maxb> It feels like sometime I'm waiting for multiple 5-minute intervals to happen for a publication
<wgrant> Hmmm.
<wgrant> Unless the publisher oversteps its 5 minute window, that should be impossible.
<maxb> *right* launchpad-developer-dependencies is installable again
<magcius> Where is the vhost config file?
<wgrant> magcius: The same file as the rest of the config.
<magcius> wgrant: which is where?
<magcius> wgrant: I'm looking in /configs, nope, /canonical/config, nope
<wgrant> In the [launchpad] section of your config file.
<wgrant> Which is probably in configs/.
<magcius> http://bazaar.launchpad.net/~launchpad-pqm/launchpad/db-devel/annotate/head%3A/configs/development/launchpad.conf
<magcius> nothing useful there
<wgrant> launchpad-lazr.conf is the important one.
<wgrant> launchpad.conf is deprecated.
<magcius> what is "lazr"
<wgrant> In that filename it refers to lazr.config, the configuration file parser used by Launchpad.
<wgrant> Which is part of the LAZR suite of libraries.
<magcius> In general, what does it stand for?
<wgrant> All developed for use in Canonical's web applications.
<wgrant> It stands for nothing.
<magcius> Was LAZR opened when LP was opened?
<wgrant> Most of it started embedded in various webapps (mostly Launchpad).
<wgrant> it was progressively split out with the first bits released about a year before LP's source.
<magcius> would you mind explaining "traversal"?
<magcius> I looked at doc/navigation.txt, but I didn't understand it
<wgrant> It's how our URLs work.
<magcius> Okay.
<wgrant> /ubuntu/karmic/i386 will traverse to first Ubuntu, then its Karmic release, then its i386 architecture.
<magcius> So it's going down a tree of possible endpoints?
<wgrant> It's taking a path form a URL, applying it to a tree of objects, and ending up with a sequence of objects between the root and the target.
<wgrant> s/form/from/
<magcius> alright
<wgrant> It's not too dissimilar from normal Zope traversal.
<magcius> do things like "+bug" mean anything special in terms of this navigation?
<wgrant> Ugh.
<wgrant> Those are sort of magical and ugly.
<magcius> Wonderful.
<wgrant> Since there's +bugs but also +bug/1234, which doens't really make much sense.
<wgrant> Say you've got /ubuntu/+bug/1234
<mup> Bug #1234: Gina is an unmaintainable mess of command line options, environment variables and shell scripts <Launchpad Foundations:Fix Released by debonzi> <https://launchpad.net/bugs/1234>
<magcius> that URL works? (responding to the bot)
<wgrant> That URL does work, yes.
<wgrant> +bug doesn't represent an actual object.
<wgrant> It purely represents a URL namespace.
<magcius> I've never used Zope. It lost me at the webapp configuration and object database based on pickle.
<wgrant> Well, Launchpad is a Zope application, but it doesn't use ZODB at all (the object database of which you speak).
<magcius> I understand.
<maxb> The thing which really really confuses me about launchpad urls is where special pages like /builders overlap with the namespace of distros/projects
<wgrant> maxb: That's why we have the name blacklist.
<magcius> so does it mean anything to the traversal?
<wgrant> '+whatever' has historically been used to avoid that namespace issue.
<wgrant> But that has been abandoned in the root lately, IIRC because it stuffs up breadcrumbs.
<magcius> i.e. does "+bug" mean "return this special Bug object for the next lookup"
<wgrant> And is a tad ugly.
<magcius> also, I officially hate Loggerhead.
<magcius> 1) the annotate/files distinction for the only purpose of pissing URL editors off
<magcius> 2) the %3A in every goddamn URL
<wgrant> loggerhead isn't really part of LP.
<magcius> 3) no left-margin marker, the sans-serif revision marker, especially painful to identify indentation levels with long stretches of code
<wgrant> magcius: To see how the +bug traversal works, see lp.bugs.browser.bugtask.BugTargetTraversalMixin
<magcius> wgrant: does that class go from ubuntu to +bug or +bug to 1234
<wgrant> '@stepthrough("+something")' lets the method handle paths of the form '+something/foo', and it will take 'foo' as an argument.
<wgrant> magcius: It handles ubuntu to +bug/1234
<wgrant> There is no intermediate class in this case.
<magcius> alright
<magcius> how/where are the DNS entries for bugs.launchpad.dev and such on development machines?
<magcius> (I never understood DNS)
<wgrant> /etc/hosts
<magcius> I've seen in the "Getting Started" guide that you don't need to modify that, and "launchpad.dev" will work
<magcius> Which is what baffled me.
<wgrant> The installation script (rocketfuel-setup) modifies /etc/hosts itself.
<magcius> does it also do that for bugs.launchpad.dev and the rest?
<wgrant> Yes.
<magcius> alright. Does Zope/LP just determine how to dispatch from the request sent, or are there separate locations?
<wgrant> Separate locations?
<magcius> separate servers
<magcius> on separate ports
<wgrant> There is a wildcard Apache vhost which dispatches to a single LP server process.
<wgrant> That checks the Host header.
<magcius> Is that true in the production cluster as well?
<wgrant> On production I believe there are multiple Squids in front of multiple Apaches in front of multiple appservers each running multiple LP processes.
<wgrant> Er, there's haproxy in there somewhere too -- probably between Squid and Apache.
<magcius> Are some LP servers designated to "Bugs" or "Code Hosting" or something?
<wgrant> Or maybe Apache and the appservers.
<maxb> Wasn't Pound mentioned at one point?
<wgrant> maxb: It was replaced with haproxy.
<maxb> ah
<wgrant> magcius: The main webapp is all hosted by one set of machines that do not discriminate.
<wgrant> bazaar.launchpad.net is served by the Codehosting machine, which stores all of the Bazaar branches.
<wgrant> It might also run Loggerhead, but that could be delegated to another machine. I forget.
<wgrant> win 16
<wgrant> Argh.
<maxb> Ah well, that's the first time I've seen you do that :-)
<maxb> We have someone at work who does it so often we've written a plugin for our IRC bot to automatically mock him
<wgrant> Heh
<NCommander> If I'm storing changelogs in librarian, should I give it a more descriptive file name beside "changelog"?
<wgrant> NCommander: I don't think so.
<wgrant> Thankyou for using the librarian, though!
<wgrant> Although maybe NAME_VERSION_changelog could be nice. i don't know.
<NCommander> wgrant: well, I talked it over iwth bigjools once I got a moment to breath
<NCommander> wgrant: I'll submit it as a second branch proposal, and then let BjornT and stub decide what version they want, and kill the one they don't :-)
<NCommander> wgrant: I'm open to suggests on this one :-)
<NCommander> wgrant: also, how do I get stuff OUT of librarian?
 * NCommander isn't so sure on tha tbit
<wgrant> NCommander: Call .read() on the LFA, IIRC.
<wgrant> Merge proposals do it to get the diff.
<wgrant> I can't think of anywhere else in the webapp that does.
<NCommander> wgrant: as long as its possible, I'm happy. Once changelogs land and are migrated (I have to write the migration utility next), I'd like to do the same for copyright files
<NCommander> and get those out of the database
<NCommander>   Ran 5 tests with 0 failures and 0 errors in 0.061 seconds.
<NCommander> woo
<NCommander> dual screens also woo
<NCommander> wgrant: is changelog_id a good name for the column?
<NCommander> or should it be something more decsriptive?
<wgrant> 'changelog' in the DB.
<NCommander> wgrant: ah, ok
 * NCommander notes that greatly reduces the number of edits I have to make then <g>
<wgrant> Why?
<wgrant> Apart from the comments.sql change.
<NCommander> wgrant: I was going to call the column changelog_id
<NCommander> now I don't need to do that
 * NCommander just updates a few comments
<wgrant> Ah.
<NCommander> wgrant: how does ForgienKey() work when updating the model
 * NCommander is a bit confused
<NCommander> wgrant: for instance, the model code to get section in the database is:
<NCommander>     section = ForeignKey(foreignKey='Section', dbName='section')
<wgrant> So, that's deprecated.
<NCommander> I'm not sure how to do one for libraryfilealias
<NCommander> wgrant: oh
<wgrant> But if the class you're using uses it, I guess you should follow.
<NCommander> wgrant: is this a zope thing?
<wgrant> changelog = ForeignKey(foreignKey='LibraryFileAlias', dbName='changelog')
<wgrant> No.
 * NCommander is still kinda iffy on Zope
<wgrant> This is SQLObject.
<wgrant> Which is deprecated by Storm.
<NCommander> wgrant: ah, I see
<NCommander> wgrant: so, for the attribute, how's this for a description:     changelog = Attribute("LibraryFileAlias for the changelog of this SourcePackageRelease.")
<wgrant> I think I'd prefer 'LibraryFileAlias containing debian/changelog'
<wgrant> But I don't really know. There is no consistency.
<NCommander> wgrant: fair enough
 * NCommander runs the soyuz and archiveuploader tests to make sure nothing else has broke
 * NCommander lets the soyuz and archiveuploader test suites run
<NCommander> wgrant: ugh, I'm getting a *ton* of test failures on my branch :-/
<NCommander> wgrant: http://paste.ubuntu.com/398521/ - like that, which seems odd
<wgrant> NCommander: There was an error before that that you broke.
<NCommander> wgrant: no, I'm getting a massive stream of these
<NCommander> like hundres
<NCommander> I don't think I could have broken it that badly
<wgrant> You broke one and it cascaded.
<wgrant> Wait until the run has finished, then rerun the failing tests with -1.
<wgrant> -1 will stop on the first failure in each doctest.
<wgrant> Letting you see what actually caused it all.
<NCommander> wgrant: k
#launchpad-dev 2010-03-21
<wgrant> Hmm.
<wgrant> so, I can publish indices for a PPA the size of the primary archive using NMAF in slightly over a minute.
<wgrant> And it is querying very naively.
<wgrant> So there's no reason the primary archive publisher needs to take so fricking long.
 * wgrant fills the test archive up even more.
<lifeless> wgrant: \o.
<jpds> lifeless: .o/
<lifeless> oh hai
<lifeless> its late for me, must be mucho lato for you
<jpds> Yep.
<lifeless> I'm told postgresql may want to use lmiror by selenamarie. 'may'.
<wgrant> For what?
<lifeless> two things
<wgrant> It doesn't seem applicable to syncing WAL logs files, and I can't think what else it could be for.
<lifeless> they apparrently mirror old packages, and many plugin-like things, to getting into 10's of K of files.
<wgrant> Oh, right.
<lifeless> and secondly yes, WAL files
<jpds> lifeless: I imagine several people will want to, Debian and such.
<lifeless> apparently if you config pathologically you get many many small log files and the mirror process for them to backup etc becomes terrible.
<lifeless> jpds: Debian etc should be fit the profile its designed for.
<lifeless> what interested me is whole new applications I didn't design for finding it close-enough.
<wgrant> Hmm, yeah, I guess for backup-like stuff it could work.
<lifeless> thanks for the vote of confidence ;)
<wgrant> I mean, I didn't see immediately how it would be useful for WAL.
<lifeless> :>
<lifeless> Neither did I
<lifeless> I was like 'oh wow, you've a use for something I wrote? cool!'
 * wgrant beats Storm over the head for being too lazy.
<lifeless> jpds: I think you should go to sleep. So you can relax tomorrow, so that you're relaxed and on the ball checking lmirror Monday.
<wgrant> +1
<jpds> lifeless: Good idea, look forward to it. :)
 * lifeless emotes positive self-interest vibes.
 * lifeless heads to the hotel
<wgrant> Creating a hundred thousand fake packages with associated publications takes a while :(
 * thumper snorts
<magcius> I'm uncertain with a lot of Zope stuff... do you guys like how abstract it is? It seems that a lot of getting Zope working is hooking up components.
<NCommander> How do I start the entire launchpad test suite?
<lifeless>  bin/test.py, I think
<lifeless> \make schema first too
<NCommander> lifeless: with no arguments?
<lifeless> aye aye
<NCommander> lifeless: (I made some database changes, and I finally have enough RAM that I think I can complete the entire test suite before the heat death of the universe I already ran a few sections by hand :-))
<mwhudson> good mornign
<magcius> hey mwhudson
<mwhudson> lib/lp/soyuz/tests/../doc/buildd-slave.txt failed
<mwhudson> because some test expects
<mwhudson> error: (111, 'Connection refused')
<mwhudson> and it got
<mwhudson> error: (104, 'Connection reset by peer')
<mwhudson> gmb, allenap: you guys _have_ seen ParallelLimitedTaskConsumer and related friends, right?
 * mwhudson restarting router
<mwhudson> heh heh
<mwhudson> view-source:https://code.edge.launchpad.net/projects
<mwhudson> seems to make chromium pretty unhappy
#launchpad-dev 2011-03-14
<thumper> can you use with_ on a storm select?
<lifeless> yes
<thumper> yup
<thumper> hmm... I'm wondering how though
<lifeless> well, maybe you mean something different than I do
<lifeless> I suggest starting with my use of it
<thumper> lifeless: I have it working on the general store.find
<thumper> but now I need a subselect for blueprint exclusion
<thumper> I've got the underlying sql tested on staging
<thumper> so I know that the syntax I want is allowed
<thumper> just trying to get it into storm
<lifeless> wgrant: you mean stevenk's patch ?
<wgrant> lifeless: Yes.
<lifeless> thumper: I'm not sure what you're talking about here
<thumper> lifeless: that's ok
<lifeless> thumper: but I wrote something to gary that /might/ be relevant so I'm going to forward it to you
<thumper> ok
<lifeless> thumper: if it doesn't seem relevant, perhaps you can pastebin me some details about what you're trying to glue together
<thumper> lifeless: I'm chatting with jamesh on #storm
<lifeless> now we're getting somehwere https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1899E29#repeatedstatements
<thumper> lifeless: if I have problems, I'll poke you
<lifeless> thumper: I hope the reformulation of the subselect is useful to you
<thumper> lifeless: it is actually
<lifeless> thumper: it will run faster because it doesn't need to look at specification
<thumper> lifeless: yep, got that
<lifeless> cool
<lifeless> wgrant: were you going to tackle NBT this week ? wondering if I should ignore oopses on it
<lifeless> hmm
<lifeless> need to test [r=lifeless][bug=732481] [bug=732496] oops_middleware should call start_response for success, even if there is no body.
<StevenK> Isn't that a fix for the haproxy call?
<lifeless> n
<lifeless> no
<lifeless> its a fix for head
<StevenK> ... which haproxy calls? :-)
<lifeless> not at the moment
<StevenK> wgrant: Er, yes it is.
<wgrant> StevenK: Aren't the DSD scripts not running yet?
<StevenK> wgrant: DSD.new() calls update(), which calls _updateVersionsAndStatus(), which calls _updateBaseVersion()?
<StevenK> Ah, well, that is likely true
<wgrant> lifeless: I will probably get around to deleting it this week.
<StevenK> I was going to QA that on mawson, after I find a good victum
<thumper> anyone know of a place in the code where we do "like" queries with storm?
<wallyworld_> thumper: registry.model.distribution.Distribution.searchSourcePackageCaches
<thumper> OMG, this vocab was so... broken
<wallyworld_> which vocab?
<lifeless> note that we should not do like queries
<lifeless> they are slow
<lifeless> fti or exact matches only
<thumper> lifeless: how do I find what text is indexed for blueprints?
<lifeless> look in database/schema/fti.py
<lifeless> name, title, summary, whiteboard
<thumper> lifeless: how do we do a fti query in storm?
<lifeless> SQL(...) or use the fti helpers
<thumper> lifeless: also, is the fti case insensitive?
<lifeless> I think it is
<thumper> in which case the like queries that were used, were useless
<thumper> as they were covered by the fti anyway
<lifeless> not surprising
<mwhudson> yay lp.blueprints!
<thumper> lifeless: in order to say "id not in (...)" we use Not(Table.Column.is_in(...)) ?
 * thumper tries to remember left joins in storm
<mwhudson> thumper: if you're looking at SpecificationDepCandidatesVocabulary, those LIKEs look completely pointless
<thumper> mwhudson: yes, I've figured that out
<mwhudson> for at least two reasons :)
<mwhudson> wow
<mwhudson> thought they might be so pointless that the query planner can turn them into =
<lifeless> thumper: I've not needed to say 'id not in' using storm syntax
<lifeless> thumper: and tbh, storm syntax vs literal sql is rarely an improvement IMO
<lifeless> thumper: other than the object deserialisation, I don't find writing the expressions using storm to be generally beneficial
<lifeless> thumper: I guess I mean: write it in the simplest way possible
 * thumper doesn't need the left joins
<thumper> d'uh
<thumper> WTF do we have an __iter__ on our IHugeVocabulary?
<thumper> that's just stupid
<thumper> I know that we "technically" need it
<lifeless> we have __iter__ on Distribution
<lifeless> and thats equally bogus
<thumper> but having to provide an implementation is fucked up
<thumper> lifeless: I agree that __iter__ on any model class is bogus
<mwhudson> it helps accidentally OOM the appservers?
<lifeless> thumper: we shouldn't have __iter__ on things that we don't expect actual complete iteration of
<lifeless> thumper: but removing it may be problematic
<lifeless> thumper: __iter__ on model classes is fine, *if* the model is a container
 * mwhudson remembers a bug to do with materializing a list containing all Persons
<thumper> lifeless: the reason we need it there is because IVocabulary defines an __iter__
<lifeless> Distribution isn't obviously a container of any one specific thing
<thumper> I'm pretty sure we don't use it though
<lifeless> thumper: yes, I know - IVocabulary is buggy
 * thumper will raise not implemented
<wgrant> I thought HugeVocabulary existed mostly to avoid having __iter__ :./
<lifeless> wgrant: yes, but we didn't fix zopes assumptions
<lifeless> anyone want to eyeball https://code.launchpad.net/~lifeless/launchpad/bug-711104/+merge/53188 before I land it ?
<wallyworld_> what's the magic command to run yui tests?
 * wallyworld_ goes to look on the wiki
<wallyworld_> ah, it's manual - load it up in a browser
<thumper> :(
 * thumper skips fixing that design problem
<thumper> part one: https://code.launchpad.net/~thumper/launchpad/blueprint-dependencies/+merge/53191
<lifeless> thumper: Thanks
<thumper> np
 * lifeless gambles on lp-land
<StevenK> lifeless: Re facebook, PICTURES!
<lifeless> thumper: so my refactored clause was slightly wrong ?
<lifeless> thumper: ( I don't see why you need specification s there at all
<thumper> lifeless: it is needed in the next bit :)
<thumper> lifeless: https://code.launchpad.net/~thumper/launchpad/blueprint-dependency-vocabulary/+merge/53192
 * thumper afk for shopping and kids ice-skating
<thumper> back later
<lifeless> thumper: still don't see it
<thumper> lifeless: leave a comment, I'll get back to you
 * thumper needs to go now
<lifeless> kk
<huwshimi> lifeless: The last few days I've really noticed a speedup on Launchpad. Did you do something significant or am I only just noticing?
<lifeless> huwshimi: we've brought some more appserver instances online, and moved to an active-passive haproxy config, and we landed a bunch more fixes late last week
<wgrant> The latency on top of generation time and RTT is normally <100ms now, which is nice.
<huwshimi> lifeless: It's awesome, I'm finding I'm waiting for pages now instead of clicking on links and going elsewhere while they load.
<huwshimi> wgrant: That's great. That was like 2.5 seconds before right?
<wgrant> I frequently saw it up to 2s, yeah.
<lifeless> we still have queues
<lifeless> but its a lot better
<lifeless> it will be up to 2 weeks before we get the next stage of fixing this done
<wgrant> lifeless: It's been 5 years.
<wgrant> 2 weekse isn't that bad.
<huwshimi> I feel like we might actually get there with our speed :)
<StevenK> lifeless and wgrant beat you with feeling that.
<lifeless> well
<wgrant> Eeh, 6 months ago I knew it was hopeless.
<wgrant> Now I'm not so sure.
<poolie> here's a slightly interesting thing:
<lifeless> wgrant: we're a great team, as long as we give ourselves the space to do things, we can move mountains
<poolie> google search results contain the url https://code.launchpad.net/bugs/490148
<_mup_> Bug #490148: License details for "Other" are displayed even if not selected <chr> <confusing-ui> <lp-registry> <qa-ok> <Launchpad itself:Fix Released by sinzui> < https://launchpad.net/bugs/490148 >
<poolie> hi wgrant, huwshimi, lifeless
<wgrant> poolie: That's a bit interesting.
<StevenK> poolie: Hi! Are you back from holidays?
<huwshimi> poolie: Hey mate
<poolie> i am back
<poolie> it was good
<poolie> searching for {launchpad licence field} fwiw
<lifeless> poolie: hi
<StevenK> wgrant: Given DSD.base_source_pub() only searches the child for the SPPH, do you think there is value in having it also search the parent?
<wgrant> StevenK: It should search both archives.
<wgrant> It will normally only be in the parent.
<wgrant> Rarely the child.
<StevenK> Right, so I'll change it to search the parent first, and fall back to the child.
<huwshimi> poolie: That's strange. It appears you can replace that subdomain with others and it still works (although I tried with blueprints.lp and it didn't load the subscribers list).
<StevenK> As it is today, I thought it was a little naive.
<poolie> huwshimi, i would guess that we send a "moved temporarily" from that url to the real location
<huwshimi> poolie: I think they must be indexing on the wrong url. For code.lp it redirects to bugs.lp (blueprints.lp didn't)
<poolie> and google therefore caches the orginila
<huwshimi> poolie: Yeah that was my guess
<poolie> strange
<lifeless> huwshimi: most things are published in all domains
<poolie> i wonder where they first found a link to that thing
<lifeless> huwshimi: only a few things aren't (and that results in bug reports)
<poolie> perhaps someone made up that url by hand and google decided it was more important
<huwshimi> lifeless: Which makes me think our subdomains are even more redundant :)
<lifeless> huwshimi: right
<lifeless> huwshimi: as I said, for me at least, its mainly a technical exercise for removing if/when jml wants us to
<lifeless> we'd certainly save on redundant ssl handshakes and js overhead
<huwshimi> lifeless: Is this something we should start putting a LEP together for?
<StevenK> wgrant: Can haz help crafting a query?
<lifeless> huwshimi: only if we want to do it
<wgrant> StevenK: I'm not here, but OK.
<huwshimi> lifeless: So do we need to talk to jml about it then?
<lifeless> huwshimi: I think there are three aspects to it
<StevenK> wgrant: I'd like to find all SPPHs that are in both Ubuntu and Debian that have changelogs set, 'AND spph.status IN (1, 2)' and have the same SPN.
<wgrant> 1) Kill them.
<wgrant> 2) Kill them.
<wgrant> 3) Kill them.
<lifeless> there is a political aspect: in the past mark was very pro this layout
<wgrant> Those are the three aspects.
<wgrant> StevenK: So you can find good things to test with?
<lifeless> there is a technical aspect: what exact actions are needed to /permit/ consolidation
<lifeless> and there is a design aspect: how should things that are currently different in the same relpath on different domains look
<StevenK> wgrant: Right.
<lifeless> lastly, there is the scheduling thing: how important is this : when should we do it
<lifeless> huwshimi: so, last I spoke to jml its not at the top of the roster - and I don't think its a one-man task (but I may be wrong)
<lifeless> huwshimi: I don't think putting a lot of effort into things that aren't schedule-next priority is sensible
<lifeless> if the schedulability of it is related to the size of the task, then investing long enough to figure out the size is worth doing
<huwshimi> lifeless: Is this topic on a list of things that need to happen then?
<lifeless> huwshimi: we practice a LEAN approach of not queuing things up if we can avoid it
<wgrant> StevenK: SELECT spn.name FROM sourcepackagename spn JOIN sourcepackagerelease sprd ON sprd.name = spn.id JOIN sourcepackagepublishinghistory spphd ON spphd.sourcepackagerelease = sprd.id JOIN sourcepackagerelease spru ON spru.name = spn.id JOIN sourcepackagepublishinghistory spphu ON spphu.spr = spru.id WHERE spru.changelog IS NOT NULL AND sprd.changelog IS NOT NULL AND spphu.status IN (1, 2) AND spphd.status IN (1, 2);
<wgrant> s/spphu.spr/spphu.sourcepackagerelease/
<huwshimi> lifeless: OK, as long as we remember to talk about it :)
<wgrant> And you want to actually constrain the archives too.
<StevenK> wgrant: *and* I have to run it through sed? :-(
<lifeless> huwshimi: so no - its not on any list : because we're trying not to have such lists :) - I'm aware of the discussion, and jml /may/ have a list somewhere with it on it
<wgrant> AND spphu.archive = 1 AND spphd.archive = 3
<lifeless> huwshimi: there may well be a bug saying 'this makes things slower'
<lifeless> huwshimi: (and if there isn't such a bug, such a bug would be reasonable to have)
<StevenK> ERROR:  operator does not exist: name = integer
<StevenK> LINE 2: ...           sourcepackagerelease sprd ON sprd.name = spn.id J...
<lifeless> grr testfix
<StevenK> wgrant: ^
<lifeless> StevenK: nameID
<wgrant> StevenK: sourcepackagename, not name.
<lifeless> or yeah
<lifeless> I strongly encourage not using reference columns in queries.
<lifeless> this will help us when we start to detangle the layers
<wgrant> lifeless: This is SQL.
<lifeless> oh
<lifeless> :)
<lifeless> Failure in test /srv/buildbot/slaves/launchpad/lucid-devel/build/lib/canonical/lazr/tests/../doc/pidfile.txt
<lifeless> Traceback (most recent call last):
<lifeless>   File "/usr/lib/python2.6/unittest.py", line 279, in run
<lifeless>     testMethod()
<lifeless>   File "/usr/lib/python2.6/doctest.py", line 2152, in runTest
<lifeless>     raise self.failureException(self.format_failure(new.getvalue()))
<lifeless> AssertionError: Failed doctest test for pidfile.txt
<lifeless>   File "/srv/buildbot/slaves/launchpad/lucid-devel/build/lib/canonical/lazr/tests/../doc/pidfile.txt", line 0
<StevenK> wgrant: Ah, right.
<lifeless> anyone seen that before?
<StevenK> It could have said "This column doesn't exist, you berk."
<lifeless> StevenK: did you have 'name' or 'spr.name' ?
<huwshimi> lifeless: I can't find a bug like that. I can report one, but you would be able to write a much better description of the problem if you want to and have the time.
<lifeless> huwshimi: sure, I can do that
<huwshimi> lifeless: Thanks mate
<StevenK> lifeless: sprd.name, so it should be sprd.sourcepackagename, as wgrant pointed out.
<lifeless> StevenK: was wondering about the context the sql parser had
<StevenK> ... 14,000 rows. I don't believe you, SQL
<lifeless> huwshimi: its covered in https://bugs.launchpad.net/launchpad/+bug/703134
<_mup_> Bug #703134: Launchpads SSL performance is substandard <Launchpad itself:Triaged> < https://launchpad.net/bugs/703134 >
<StevenK> I think we should edit the bug number to be 10 or something :-P
<lifeless> bug 10 ?
<_mup_> Bug #10: It says "displaying matching bugs 1 to 8 of 8", but there is 9 <lp-bugs> <Launchpad itself:Invalid> < https://launchpad.net/bugs/10 >
<lifeless> hah. 'are'
<lifeless> in the isf call on thursday we talked about the ssl cache sharing component - its a prerequisite for LVS, and LVS is up second on (IIRC) spads' queue
<wgrant> How is that a prereq?
<wgrant> Mm, I guess.
<lifeless> wgrant: browsers don't switch ip per-request with round-robin setups
<wgrant> Anyway, /me vanishes.
<lifeless> so its [relatively] rare to do ssl renegotiates
<lifeless> lvs would make it a common event unless we did source ip pinning (but you don't want to do that because of the impact of nat'd conferences)
<lifeless> huwshimi: I like that you've been writing up a bunch of docs
<huwshimi> lifeless: Starting to. It's mostly just stubs and a collection of links to other places... but I have to start somewhere.
<huwshimi> lifeless: I'm trying to add things as I think about them
<lifeless> huwshimi: one thing I'd like to suggest
<lifeless> huwshimi: more things to remember makes development harder
<lifeless> huwshimi: if possible, finding ways to make things automatic would be better than proscribing how things should be
<lifeless> huwshimi: e.g. if all popup forms automatically set their first entry widget as focus, that would be a no brainer win
<huwshimi> lifeless: Yes, that was exactly my thought
<huwshimi> lifeless:  I do want to list the rules for the way things should behave though
<lifeless> huwshimi: it may be nitpicking, but its probably worth rephrasing from 'rule' to 'sane default which solve problem X'
<lifeless> huwshimi: this future proofs discussions: if problem X goes away, the default to solve it is clearly not relevant, whereas rules tend to age less gracefully IME
<huwshimi> lifeless: Nitpick away :)
<huwshimi> lifeless: So I want to have documentation with those guidelines about how to solve or how they are already solved within Launchpad
<lifeless> huwshimi: I basically think we're rule heavy: we /still/ have way to many nonfunctional review rules, and I'm jumping the gun trying to avoid the same problem in a different guise
<lifeless> sigh
<huwshimi> lifeless: That makes sense, I wonder how we can make sure we have a consistent interface if we don't have a bunch of guidelines that should be followed though.
<lifeless> 10/33 POFile:+translate
<huwshimi> lifeless: Or am I misunderstanding something
<lifeless> huwshimi: less code, more reuse
<lifeless> huwshimi: additionally, i don't think following guidelines results in a high quality interface: I find it extremely weird that consistency is axiomatic with ease of use and high quality
<lifeless> huwshimi: ease of use is directly measurable as is quality : clarity, performance, uptake of information are all measurable
<lifeless> huwshimi: time to perform task, time to learn task etc
<lifeless> huwshimi: as a for-instance on less-code,more-reuse: one of the genius things gnome's desktop stack did years ago was to ditch pixel positioning for widgets: instead it uses packing horizontally and vertically: the /default/ for a developer using the toolkit will be to be consistent with most other gnome apps
<huwshimi> lifeless: What I'm trying to solve is that we don't really have enough docs on what we do or why we do them (for UI stuff). For example, I've been reviewing a form that rvba is working on. He would have know exactly how to create the form without a fair bit of back and forward over the last week if there was a doc he could have looked at that said "when creating a form don't use tables, use classes instead of inline
<huwshimi> styles and here's where to look for appropriate styles".
<lifeless> huwshimi: so I think this breaks into a few issues
<lifeless> huwshimi: firstly, we're way to manual in our forms :( - long term issue
<lifeless> huwshimi: secondly, there isn't really a 'right place' to cargo cult from.
<lifeless> by which I mean its massively easier to take a good form from elsewhere and adjust than to start from scratch, but there isn't a 'known good' form that is fully ajaxified, uses all the right styles etc etc
<lifeless> there *are such forms*, but nothing guiding folk to them
<lifeless> I would have expected bigjools to guide rvba to the right place though
<lifeless> huwshimi: using classes seems like standard html-since-2005 to me though, rather than something we need to formally write down :)
<huwshimi> lifeless: Actually this is just a small filter form, of which we have a few, but this one had an extra option which none of the others do, so this was a special case that ended up being quite a bit more complex.
<huwshimi> lifeless: except that we do have a lot of inline styles. So when he wanted to know what to do he looked at our existing code and noticed a lot of inline styles and assumed that was ok
<lifeless> huwshimi: this correlates with what I am saying
<lifeless> huwshimi: its *easier* to copy existing stuff than to follow procedures about what we *should* do
<lifeless> huwshimi: so to get new contributions to be as we want them, its most effective to fix our existing stuff
<lifeless> huwshimi: cheat sheets can still be useful, but I guarantee the first step isn't read-the-cheat-sheet
<huwshimi> lifeless: I totally agree with all that, I just think it is helpful to also have some docs to refer to. Maybe though I need think about having them very general rather than specific cases
<lifeless> huwshimi: lets say you retain 10% of stuff you read once
<lifeless> huwshimi: think about how much stuff you read when you started
<lifeless> huwshimi: and how much of the details you remember now :)
<lifeless> huwshimi: I'm not saying 'do not write docs' : I started saying I'm glad you are writing docs up
<lifeless> huwshimi: I am saying: rules that must be followed become a process burden, so we should demand *real value* from any rules we add.
<lifeless> huwshimi: *and*
<lifeless> huwshimi: making a few /really good/ forms and offering them up as 'copy this' examples (e.g. via the list, in your docs etc) would be a really good way to help your improvements propogate
<huwshimi> lifeless: Alright that sounds good.
<lifeless> huwshimi: I've done something similar with my query scaling tests: I took a single example that was in the code base from a while back, generalised and reused it, and wrote it up on the mailing list - other folk are writing such tests now
<lifeless> huwshimi: and the infrastructure has incrementally improved to make writing them easier.
<lifeless> huwshimi: I simultaneously tried some rule-based improvements, which have had approximately zero impact on the code base: the communication, examples, and continual focus on the one thing have however had a massive impact.
<huwshimi> lifeless: I guess really most people don't refer to docs unless they get stuck with something.
<huwshimi> lifeless: Or they really don't know where to begin
<StevenK> Didn't Julian add a boolean that we can use to check if something returned any rows? I can't recall what it is called.
<lifeless> huwshimi: exactly
<lifeless> StevenK: its __nonzero__ and only on sqlobjecresultset : don't use it.
<lifeless> StevenK: storm result sets have .empty()
<lifeless> StevenK: but note that this is a query: *if* you are going to ask for any rows, just ask, don't check for emptiness
<lifeless> here stubby stubby stubby
<lifeless> StevenK: can you run a query on df for me
<StevenK> Certainly
<lifeless> I'd like an explain analyze of
<lifeless> https://bugs.launchpad.net/launchpad/+bug/734642/comments/3
<_mup_> Bug #734642: POFile:+translate timeouts <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/734642 >
<lifeless> I'm looking for cold cache performance
<StevenK> Can't get much colder than DF
<lifeless> right
<lifeless> it is about 30 times faster than the current hot performance
<lifeless> but if its not hotter cold, the win isn't very important
<lifeless> s/hotter/faster/
<StevenK> lifeless: 1345.664 ms, do you want the output?
<lifeless> please
<lifeless> I wonder if we just need a tonne of ram
<StevenK> Let me apologise in advance, I didn't turn off the pager, so it's going to look like crap, and re-running it won't give the same values
<lifeless> thats fine
<lifeless> pastebomb away
<StevenK> http://pastebin.ubuntu.com/580007/
<lifeless> thanks
<lifeless> so, no worse
<lifeless> but not substantially better
<StevenK> lifeless: I'd welcome your comments on https://code.launchpad.net/~stevenk/launchpad/dsd-base_source_pub-search-parent/+merge/53194 . Mostly interested in your thoughts re lines 27-51 of the diff
<lifeless> StevenK: commented
<lifeless> StevenK: not pubs.empty() would be better than using count() btw
<lifeless> StevenK: only use count() if you need a count and won't be querying all the rows.
<StevenK> lifeless: Thanks, I've taken your suggestion with a little tweak.
* StevenK changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | firefighting: - | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<lifeless> jtv: you might like the new plan in bug 734642
<_mup_> Bug #734642: POFile:+translate timeouts <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/734642 >
<rvba> stub: Hi Stuart. I've just done my first db changing branch and I'd like you to review it when you get a chance.
<rvba> It's been approved by sinzui and the change is really not controversial but since I'm fairly new to the code (joined Canonical 2 weeks ago) please don't hesitate to give me any recommandation.
<rvba> https://code.launchpad.net/~rvb/launchpad/db-add-distro-registrant/+merge/53001
<stub> yup.
<stub> yer - this one looked trivial
<rvba> stub: it is yes.
<jtv> lifeless: Took me a while to decypher what you pasted thereâ¦ do you think it's the WITH that does it?  Or is it mostly that the "potmsgset <> $INT" check moved from the TM part to the POTMsgSet part?  I'm pretty sure we had it in the POTMsgSet part at one point.
<lifeless> jtv: its the with
<jtv> Just acting as an optimization barrier?
<lifeless> jtv: comment 2 is the first iteration of the with, and when I then move the potmsgset I cap the upper estimate too
<stub> rvba: So people decided to keep both owner and registrant, or will owner be dropped in the future?
<jtv> Frankly I don't even see much reason to have the potmsgset <> $INT checkâ¦ because we query the ones with potmsgset = $INT separately anyway.
<lifeless> with alone is 350->690 or whatever, with + move potmsgset is 350..350
<jtv> lifeless: are those numbers estimated costs?
<lifeless> jtv: yeah
<rvba> stub: Owner will stay and be the field still used for perm checking (owner = maintainer for distributions). Registrant is just a historical record of who created the object.
<thumper> https://code.launchpad.net/~thumper/launchpad/blueprint-dependency-vocabulary/+merge/53192 anyone?
<lifeless> jtv: actual execution with the with is ~ 30 times faster than the existing query
<jtv> That's great.  It's too hard to read so I'll just take your word for it.
<lifeless> jtv: :>
<lifeless> jtv: with seems to be a very useful thing, was the main point
<jtv> Soâ¦ any takes on implementing it?
<lifeless> jtv: I figured you'd be interested with your joint rosetta + postgresql interests
<jtv> Of course I am!  But I'm on feature rotation.  :(
<lifeless> jtv: Its about 15 minutes to code up, I'll do it tomorrow morning if noone beats me to it
<jtv> Great, thanks!
<jtv> Can I take it the "with" mainly serves as a way to force the planner to materialize that subquery?
<lifeless> it creates a temporary table
<lifeless> so yes, in this context I think that that is what its doing
<stub> rvba: Is 'registry' actually a good default registrant of the existing Distributions? We might want to be more selective if it gives any genuine power.
<lifeless> the registrant is just to make our db work a bit harder rendering the thing
<lifeless> meep yesterday was busy
<lifeless> non-opstats pages: 7574479
<rvba> stub: AFAIK the whole idea (sinzui is the one driving this) was to separate clearly who registered the object (and this should not give real power) and who the maintainer is. The only use of the registrant field is to display inside the 'Registered by' slot.
<stub> Sounds fine.
<rvba> this might sound silly (and only made to have an extra query) but I think it makes sense because the same thing will have to be done for distroseries.
<stub> Approved. Some things to change listed in the review.
<rvba> here is sinzui's opinion on this (for distroseries):
<rvba> sinzui "Since distros series do not have a direct owner, it seemed okay in the past to reuse the field for registrant. but owner is intended to be mutable. It is not possible BTW to change a series owner. We removed the field from edit forms"
<stub> It sounds silly if we will only ever have Distributions registered by the registry team (which is silliness we have done for years unfortunately). If random users can register distributions, fine.
<lifeless> heh, i just tried to use 'y' to archive this channel :P
<rvba> I'm not sure it's very frequent for random users to register distributions... but AFAIK the whole thing is to cleanup the owner field for distro, distroseries (and also productseries and productrelease) which has been 'reused' in the past to store other things (sometimes the registrant) than what it should store.
<rvba> lifeless: you're right when you say that the current displayed registrant for a distro is the owner. And if we want the changes to be without effect on existing distro we should use the owner instead of ~registry. But I think sinzui's point was that in order to normalise the registrant of objects we should set it to ~registry in the migration script.
<lifeless> rvba: why normalise it?
<lifeless> rvba: surely the existing data is more accurate than ~registry
<rvba> lifeless: since this is a read-only field I guess it's really not a problem to set the registrant to user instead of ~registry.
<rvba> and it might be more informative ....
<rvba> lifeless: I think sinzui's argument of cleaning things up really makes sense. But as you can imagine, being still fairly new to the code, I can't say that I have a personal opinion on this.
<rvba> lifeless: but I really understand your point
<lifeless> rvba: setting it to registry would discard what data we have
<lifeless> I think its wrong.
<rvba> lifeless: right
<lifeless> if we want it to be registry it would be a lot easier to just hardcode that in the templates :)
<rvba> lifeless: it wont be registry for newly created objects
<StevenK> henninge: Hi! Are you good to be OCR today?
<adeuring> good morning
<lifeless> rvba: I know, but if we don't want it to be registry for new ones, we don't need it to be registry for old ones :)
<rvba> lifeless: again, I really understand your point.
<henninge> StevenK: I am.
* henninge changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | firefighting: - | On call reviewer: henninge | https://code.launchpad.net/launchpad-project/+activereviews
<StevenK> henninge: Are you up for reviewing https://code.launchpad.net/~stevenk/launchpad/dsd-base_source_pub-search-parent/+merge/53194 ?
<henninge> StevenK: sure ;)
<lifeless> rvba: where does that leave us?
<rvba> lifeless: I'll just make sure with sinzui (he's really the one driving this) that there is no secret reason why he wanted to set the registrant to ~registry :-)
<rvba> lifeless: but I think you're right
<thumper> lifeless: where's that page that shows the revisions blocking the rollout?
<rvba> lifeless: is that ok with you?
<lifeless> https://devpad.canonical.com/~lpqateam/qa_reports/deployment-stable.html
<lifeless> rvba: sounds great
<thumper> ta
<lifeless> rvba: I suggest optimistically changing it now to get stubs ack on the update script change
<rvba> lifeless: ok
<thumper> lifeless: when is the next rollout?
<rvba> I'm actually doing it right now.
<lifeless> thumper: I have one up on LPS
<lifeless> thumper: but we have no losa until US come on
<henninge> StevenK: Sounds to me like there is a "not" missing somewhere in this comment:
<henninge>  98	+        # Passing in a base version to makeDistroSeriesDifference() creates
<henninge> 99	+        # it in both distroseries, so we need to do it ourselves.
<lifeless> henninge: the test is testing the case where one side is missing the base
<lifeless> henninge: so the statement is correct
<thumper> :(
<lifeless> thumper: I enquired about gsa vg doing it, but they felt uncomfortable
<henninge> hm, then I don't understand it
<thumper> fair enough
<StevenK> henninge: Would you prefer I re-word it?
<henninge> lifeless: wht is "do it" referring to?
<henninge> StevenK: I don't think I understand the comment yet
<StevenK> henninge: "do it" in that context refers to creating the publication.
<lifeless> thumper: what rev do you want live ?
<thumper> 12571
<StevenK> The deployment is for 12582, so that's okay
<StevenK> I want 12569 deployed, so I've also been looking.
<henninge> StevenK: I am sorry, I may be missing some understanding about DSDs here and the implication of that comment.
<henninge> which kind of tells me I might not be understanding what this branch is all about in the first place ...
<StevenK> henninge: Okay, so makeDistroSeriesDifference() can take a 'base' version in the versions dict. That will create a publication in both the parent and the child distroseries. I don't want it to do that, so I need to create the publication myself in only the child.
<StevenK> henninge: The reason for the change is that it is much more likely for the parent distroseries to contain the base publication than for the child too -- hence the change to look in the parent first.
<StevenK> "than for the child to" even
<lifeless> thumper: are both your bp branches ready for review?
<lifeless> thumper: if so I can eyeball your second one for you
<henninge> StevenK: but you are doing that *after* the call to make DistroSeriesDifference, so does that change its behavior?
<StevenK> henninge: Not at all, but it is why I have to call ds_diff.update() after doing so.
<henninge> ah!
<StevenK> Doh, forgot to triage
<henninge> StevenK: so "update" will remove the base version from the parent?
<StevenK> henninge: update tells the DSD "stuff has changed, please look again and update your internals as necessary"
<henninge> I understand that bit.
<StevenK> henninge: I think you're close -- do you want a have a quick chat on Mumble to close the gap?
<henninge> StevenK: that would be helpful, I think
<henninge> let me set that up real quick
<bigjools> wgrant: hello.  Did you see OOPS-1897PPA11 ?
<StevenK> bigjools: It's a public holiday in VIC today
<wgrant> bigjools: As StevenK said, I'm not here today.
<wgrant> But let me look.
<bigjools> those lazy victorians
<StevenK> Haha. Sarah said exactly the same thing.
<bigjools> :)
<wgrant> bigjools: Isn't that just the usual unsigned changes file?
<bigjools> wgrant: hard to tell from that oops
<bigjools> also OOPS-1898PPA5
<wgrant> Missing Files section usually is.
<wgrant> bigjools: That was a manually constructed changes file.
<wgrant> Had only a dsc and a deb.
<bigjools> 1st one is "InvalidEmailAddress: master@ussr.htv4"
<wgrant> I presume that's relevant.
<wgrant> Oh, got them mixed up.
<wgrant> Right, I was looking at 1897PPA5 instead of 1897PPA11
<bigjools> both look like bugs either way, we should respond to the uploader
<bigjools> someone is causing Archive:+delete to time out as well :/
<wgrant> lifeless filed a bug on Archive:+delete earlier.
<bigjools> cool
<wgrant> Both are bugs, yes.
<lifeless> some guy tried to delete packages
<lifeless> that timed out
<rvba> stub: lifeless just pushed the corrections (https://code.launchpad.net/~rvb/launchpad/db-add-distro-registrant/+merge/53001)
<lifeless> so he tried to delete his ppa with 435 active publications
<lifeless> 9 seconds in one update statement
<lifeless> bigjools: I've got a patch up for oops-tools that will get rid of the 'top 10' limit on exceptions
<bigjools> Oo
<lifeless> bigjools: once thats live I plan to get rid of the dedicated soyuz report, because we'll see them easily in the main report.
<bigjools> sounds sensible :)
<bigjools> lifeless: as long as they stick out like very sore thumb, that's ok
<lifeless> bigjools: well, how about we get the patch live, wait a day, and I'll see how it looks to you
<lifeless> bigjools: AFAICT you're the only person wanting a broken out soyuz report now
<lifeless> bigjools: if it looks clear enough in the main report, we can then close off the dedicated report. If its not clear enough, we can revisit in another few weeks.
<bigjools> lifeless: so there's a few OOPSes that should not be oopses and there's bugs filed about that.  When they are fixed, *any* oops after that is genuine and critical.  As long as they are obvious in the report I have no issues.
<bigjools> lifeless: that's a good plan
<LPCIBot> Project windmill build #47: STILL FAILING in 1 hr 15 min: https://hudson.wedontsleep.org/job/windmill/47/
<StevenK> henninge: Thank you for the review. :-)
<henninge_> StevenK: you are welcome ;)
<lifeless> mpt: oh hai
<mpt> lifeless, hi ho
<lifeless> mpt: how did that paperwork go?
<mpt> lifeless, excellent, thank you, I have a shiny new passport
<jml> Good morning Launchpadders
<lifeless> mpt: cool
<lifeless> jml: hai
<lifeless> mpt: how is lp coming along, from your disinterested perspective ;)
 * bigjools waves at jml
<mpt> lifeless, more speed is good. I haven't noticed any other changes recently.
<mpt> Oh, except that heading color changed for some reason.
<lifeless> mpt: are you noticing speed changes?
<stub> rvba: The repush is fine too. I think you are good to send it off to ec2 to land.
<rvba> stub: great, just need a few words with sinzui about this (he's driving this) and I'll land it.
<lifeless> hmm
<lifeless> isd branch mangler probably doesn't do what they want ><
<mpt> lifeless, no, more like I'm no longer noticing slowness. I guess the bane of performance work is that if you do it right, people end up not noticing.
<lifeless> mpt: thats excellent new
<lifeless> s
<lifeless> mpt: theres another step to get to 'snappy'
<mpt> yes
<lifeless> mpt: but eliminating the negative feeling is still a big step forward
<mpt> I still notice that <https://bugs.launchpad.net/ubuntu> takes a while
<jml> mpt: sometimes that bane comes to UX work too
<mpt> oh, sure
<jml> (incidentally, what is the proper verb for 'bane'?)
<mpt> I hope there isn't one
<lifeless> mpt: thats https://bugs.launchpad.net/launchpad/+bug/618406
<_mup_> Bug #618406: Distribution:+bugs-index time outs <lp-bugs> <pg83> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/618406 >
<mpt> Launchpad's default font size is tiny now
<lifeless> mpt: we're going to drop the bugtask.id secondary sort, which makes the query 2000 times cheaper
<mpt> I always have to Ctrl + after arriving
<lifeless> yeah
<lifeless> sinzui has a Plan
<jam> lifeless: is there something I'm supposed to be subscribed to so that I get notification when I can QA a given fix?
<jam> I don't want to be holding up the pipeline, but it often just seems like guesswork to me
<lifeless> jam: the bug
<jam> lifeless: so when it changes to "qa-needstesting" it is supposed to be readi?
<jam> ready
<lifeless> https://bugs.launchpad.net/launchpad/+bug/732481/comments/6
<_mup_> Bug #732481: internal error trying to serve HEAD <qa-untestable> <Launchpad itself:Fix Committed by jameinel> <loggerhead:Invalid> < https://launchpad.net/bugs/732481 >
<lifeless> thats done by a bot that watches what is deployed to qastaging and staging
<lifeless> when it comments, the fix is on qastaging and testable - and needs testing
<lifeless> allenap: speaking of qa
<allenap> lifeless: The js thing? I'm doing that now, as much as I can, and I'll be sending out an email shortly to request some help.
<bigjools> gah, we really need to fix bug change conflicts
 * bigjools identifies a 4-digit bug that will need to be fixed for derived distros
<deryck> Morning, all.
<gmb> allenap: The changes to the way that lp.js is built don't have anything to do with the error that I'm seeing on https://launchpad.net/launchpad, do they?
<gmb> Uncaught TypeError: Cannot read property 'pillar' of undefined
<lifeless> gmb: no
<lifeless> gmb: allenap's changes deployd have not been
<gmb> Ah, right.
<gmb> Bugger, that means danilos and I have to Do Work.
<allenap> What he said :)
<gmb> Thanks anyway.
<lifeless> jml: lp:~tseaver/subunit/py3-hacks may interest you
<jml> lifeless: yeah, it deos.
<rvba> henninge: Hi, I'd appreciate a review of https://code.launchpad.net/~rvb/launchpad/fix-missing-addseries-link/+merge/53219
<rvba> henninge: really trivial refactoring of a few tests.
<henninge> rvba: I am happy to do that
<rvba> henninge: thx
<bigjools> Does anyone have any thought on how useful it is to have a Subscribers portlet on a bug page?
<gmb> allenap: I see a lot more stuff in my JS console today on lp.dev. Is that something to do with your changes?
<allenap> gmb: Can you paste? In what environment?
<allenap> bigjools: You want to ditch it?
<gmb> allenap: Firebug or WebKit. Paste coming up...
<wgrant> bigjools: I think the direct subscriber list is handy... the structural subscribers not so much.
<gmb> allenap: This kind of stuff: http://pastebin.ubuntu.com/580089/
<bigjools> I agree with wgrant
<bigjools> it takes up loads of space and it's fairly useless
<allenap> I'm with wgrant on that, though it might be interesting to know if someone has a structural subscription when adding them as a direct subscriber.
<wgrant> I want to be able to see the strucsub list.
<wgrant> But not always.
<wgrant> There should be a button or something.
<allenap> gmb: Is this in dev?
<gmb> allenap: Yes
<allenap> gmb: That's probably because it uses the debug builds of YUI by default. That's a bit too noisy isn't it? That's readily fixable. If it's a problem I'll file a bug.
<gmb> allenap: That would be great, thanks. Can you tell me how to make it STFU for the sake of the debugging I'm doing now?
<allenap> gmb: No idea :)
<allenap> gmb: Let me see...
<gmb> allenap: I'm happy to prod at stuff until it shuts up, I just thought that you might know. Unless "readily fixable" is Norfolk-speak for "Eh, I dunno."
<allenap> gmb: Can you try the following as a workaround? make clean_js; make JS_BUILD=min jsbuild_lazr; make JS_BUILD= -W jsbuild_lazr jsbuild
<gmb> Sure.
<gmb> allenap: That worked. Thanks.
<allenap> gmb: Woohoo :)
<allenap> gmb: I'll reply to my message to the list to let others know.
<gmb> allenap: Thankee kindly.
<allenap> gmb: Btw, if you s/min/raw/ then it'll unminify lazr.js too.
<gmb> allenap: Cool, ta
<henninge> rvba: Approved but with a few formatting issues that I'd like you to fix, please.
<rvba> henninge: will do, thx.
* benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: henninge, benji | https://code.launchpad.net/launchpad-project/+activereviews
<jtv> benji, are you up for a review?  https://code.launchpad.net/~jtv/launchpad/bug-733245/+merge/53230
<benji> jtv: sure
<jtv> great
<jtv> bigjools: I guess I'll need help from someone privileged to Q/A the creation of DSDs during package publication.
<wgrant> jtv: What sort of performance penalty does that entail?
<jtv> wgrant: whatever it takes to compute base_version, basically, plus change
<wgrant> jtv: Erm.
<wgrant> Can't do that while creating the publication.
<jtv> Oh sorry, right, I create jobs, not DSDs.
<wgrant> Right. How expensive is that?
<jtv> My head was still in a different branch.  :)
<jtv> Shouldn't be very expensiveâ¦ just create a tiny little record holding the distribution, distroseries, and a little bit of json that holds the sourcepackagename id.
<jtv> Oh, and check for duplicates first.
<wgrant> Does it query for existing ones?
<wgrant> Right.
<jtv> Yes
<wgrant> Hopefully that's cheap.
<benji> jtv: I'm done with https://code.launchpad.net/~jtv/launchpad/bug-733245/+merge/53230
<jtv> It's something we can add an index for.
<jtv> thanks benji!
 * jtv reloads
<benji> np
<danilos> anyway knows anything about how LP loads JS and specifically, when/how does LP.links.me and LP.cache gets preloaded? (I see links['me'] is defined in lib/canonical/launchpad/rest/me.py through some event magic, but that doesn't really tell me how that gets assembled into JS)
<deryck> henninge-lunch, adeuring -- is standup time now for you guys?  Or in an hour?
 * deryck suspects DST fail
<danilos> deryck, Europe's got another two weeks of non-DST time, yay :)
<deryck> ah, fun calendaring ahead then
<deryck> :-)
<adeuring> deryck: I assumed the standup starts in 56 minutes
<deryck> adeuring: ok, cool.  Since abentley and I can't have it without you guys, we'll do it then. ;) :)
<deryck> adeuring: we're okay to keep it at this time, until you and henninge-lunch join DST in a couple weeks.
<adeuring> deryck: well, I wouldn't mind to do it right now, but i think henning has lunvch right now
<deryck> adeuring: no worries.  Probably easier to keep it the same for you guys.
<StevenK> I'm on my way to bed, would someone mind forcing db-devel and mailing the list -- it failed to checkout source code.
<danilos> deryck, hi, do you know if there is a special event that is triggered when LP data is populated in JS? (i.e. LP.links.me, LP.cache)
<danilos> deryck, or, how can one ensure that his code runs only after that happens? (asking you because you have touched some JS lately, I understand that this may be inappropriate :)
<abentley> deryck: is there a way to run a launchpad.dev instance using the un-minified javascript?
<deryck> abentley: get latest devel and run `make JS_BUILD=raw clean_js jsbuild`
<deryck> and thank allenap for that :-)
<deryck> abentley: if you want more log output you can do "debug" instead of "raw."  I think it does debug by default right now.
<abentley> deryck: Cool.  I just saw that too.
<deryck> danilos: I don't think there is any event when the cache is setup.  thumper added a bunch of event stuff for in page updates a couple weeks back, but I haven't looked at it yet.  So don't know for sure.
<danilos> deryck, right, so what I have is basically some page setup code that wants has a conditional on whether user is logged in (using LP.clients.me === undefined as the check); got any better suggestions for that?
<deryck> danilos: I would have done it that way, too.
<danilos> deryck, except that it doesn't work because it is undefined all the time, and gets populated later (or so it seems) :(
<danilos> anyway, thanks, won't bother you any longer about this, I guess it's all the same for all of us :)
<deryck> hmmm
<deryck> danilos: ah, right.  It's probably not populated until domready itself.  So you're in a race with it.
<deryck> danilos: if thumper's work didn't add and event for that.  it would be pretty easy to add one to the client code and hook into it.
<danilos> deryck, yeah, so what I want to do is have that code fire something like "lp-event" and than use that, but I don't even know where that code is :)
<deryck> right
<deryck> it's in lib/lp/app/javascript/client.js I think
<deryck> danilos: yeah, that's it:  lib/lp/app/javascript/client.js
<danilos> deryck, yep, found it now, I was looking for clients.me stuff, when it's all about the "cache"
<deryck> right
<deryck> danilos: so it looks like this is setup manually in lib/lp/app/templates/base-layout-macros.pt.  I would think a sniff for it would work.  Did you check it as:   LP.links.me === undefined ?
<danilos> deryck, yes
<deryck> danilos: after domready?
<danilos> deryck, well, it works just fine if you are not inside domready event handler (i.e. after it really loads :)
<deryck> danilos: ok, that was going to be my next question, if doing it after load mattered?  if it works after load, then I'd do that.
<danilos> deryck, right, that's what I want to do but just need to figure how to best do it
<danilos> deryck, so, do you think it would be outrageous to fire something like "lp-ready" even in base-layout-macros.pt when all LP elements are loaded?
<danilos> s/even/event/
<deryck> danilos: I think that would be fine.  The problem you might have, though, is that the links and cache objects are constructed outside of YUI.  So I'm not sure of the timing if you add a YUI block after them.  Seems like it should work, though.
<danilos> deryck, right, I'll try it out and see if it seems stable
<deryck> danilos: ok, cool.  good luck with it!
<danilos> thanks :)
<LPCIBot> Project windmill build #48: STILL FAILING in 1 hr 9 min: https://hudson.wedontsleep.org/job/windmill/48/
<danilos> deryck, actually, domready was not the problem! we forgot to make the event handler for domready a closure, so it was run right after it was passed
<danilos> deryck, I figured that out when I noticed that even with my event I was getting undefined
<deryck> danilos: ah, good catch.  So that makes it simpler for you now, too! :-)
<danilos> deryck, yeah, ta
<deryck> I'm working on Windmill today.  See if I can get it stable enough to re-enable.
<deryck> Just FYI, for anyone watching the Jenkins builds and concerned.
<jcsackett> bac: ping.
<bac> hi jcsackett
<jcsackett> hi, bac. so, about that lp.registry.pillar.
<jcsackett> once upon a time, that was using a separate "activate_collapsible_div" function defined in a file called pillar.js
<jcsackett> but it was later realized that activate_collapsible could just be refactored to allow a more general case.
<jcsackett> that refactor done, i appear to have then forgotten to kill that call in the template, and didn't see an error b/c the functionality i was looking at still worked.
<jcsackett> so, you should be able to totally whack that in what you're looking at and see no ill effect, bac.
<bac> jcsackett: understandable.
<bac> jcsackett: ok.
<bac> i'm landing a branch today so i'll include the whackage
<jcsackett> awesome.
<jcsackett> and sorry for the confusion on this part; i just had to look 40 revisions deep in a really confused branch of mine to figure it out. :-)
<jcsackett> bac: i replied to the email thread so the rest of your team is up to date.
<bac> thanks again
<jcsackett> no worries. my mess in the first place, sorry you stepped in it. :-)
<henninge> adeuring: I am here, btw.
<henninge> ;)
<henninge> adeuring: nu aba
<abentley> adeuring: so this model method you want.  I think it could be a method on TranslationMergeJob that accepted a Packaging and returned the JobStatus of the next pending or running job.  It would return None if there was no matching job.  How does that sound?
<adeuring> abentley: right, that would work. But we start from a package context, so we would also need a package method "list pending translation merge jobs"
<adeuring> well, ok, that coudl also be done in browser code...
<abentley> adeuring: Why do you want a list?
<abentley> adeuring: I meant that it would be a classmethod on TranslationMergeJob.
<adeuring> abentley: I just thought that we basically do a SQL query, and that returns a sequence
<adeuring> and right, a classmethod for TMJob would be fine.
<sinzui> rvba: I replied to your emails. I hope I helped.
<abentley> We would do an sql query, and that would return a sequence, but you don't want a sequence, so why are you asking for a method that returns a sequence?
<rvba> sinzui: I'm reading them right now
<rvba> sinzui: do you want to talk to Rob about the ~registry thing? (the branch is really to land but I really can wait for the final thought on this ;-))
<sinzui> No
<rvba> sinzui: ok then
<rvba> sinzui: (question unrelated to the whole registrant thing): is there a reason why "make lint" does not use format-imports to warn about imports needing to be formatted?
<sinzui> rvba: no reason.
<sinzui> rvba: I certainly could. and do the copyright update too
<sinzui> s/I/It/
<rvba> sinzui: I'll do it then, and thus take this opportunity to lean a little bit more about the build system
<rvba> sinzui: copyright update? you mean automatically warn about the date in the header?
<sinzui> rvba: utilities/update-copyright will update all the dates in the changed files if they need it
<rvba> I like the idea of "make lint" not doing anything by itself but warning, and maybe giving commands to be executed to do the actual stuff ...
<rvba> sinzui: should the format be done automatically or just warn (like make lint already does with the sample data files)?
<bac> benji: can you review https://code.launchpad.net/~bac/launchpad/accordion-style/+merge/53259?
<benji> bac: sure
<bac> benji: great, thanks.  i'm going to have to be gone a bit but can answer questions when i return
<sinzui> rvba: It would be lovely if it could do it automatically. I think that would be surprising though. The the scripts behave differently when there are uncommitted files in the tree. I think lint should warn when when everything is committed.
<benji> k
<danilos> sinzui, hi, fwiw, I think the sharing stuff you CCed jtv and me about is best known by henninge and/or deryck (it seems to be the new stuff they worked on)
<rvba> sinzui: sounds strange to break existing habits though ... why not create a "make format" that would do this, then if new tools are added to the tool set, they could be added here.
<sinzui> danilos: okay, i will do that now. Thank you.
<sinzui> rvba: +1 `make format` that does code cleaning
<rvba> sinzui: ok
<sinzui> rvba: our linter uses pocket-lint which also has code cleanup features like fixing whitespace, line endings, doctests, css etc...
<rvba> sinzui: I saw that you are the driver for the project yes, that's why I was asking you
<jml> sinzui: do you want to have a call today?
<sinzui> I have nothing to bring today. Maybe next week?
<jml> sinzui: sure.
<sinzui> rvba: okay. pull tip. I landed a fix for py 2.7 recently. I have not packaged it. I thought I wold do it over the weekend, but I needed to not hack for a could days to collect my mind.
<rvba> sinzui: you mean in pocket-lint?
<sinzui> rvba: yes. Lp is using an older version.
<rvba> latest revision was on 2011-03-03
<sinzui> rvba: yes
<sinzui> jcsackett: do you have a few minutes to discuss my branch to enable the person merge job: http://pastebin.ubuntu.com/580164/
<jcsackett> sinzui: ten minutes, then sure.
<sinzui> okay thanks
<gary_poster> hey deryck, can you by chance point me to some JS somewhere that I can decipher that manipulates something that is exported as a CollectionField, or in some other way point me to figuring out the preferred way to delete something from a CollectionField in js?
<gary_poster> heh, sorry for the convoluted sentence
<deryck> heh, np. thinking....
<gary_poster> didn't sleep well last night :-/
<gary_poster> Looks like maybe we don't do that now, and we need to add a method on Collection in client.js to do it "right"?
<deryck> gary_poster: was just about to reply with that same statement.  I don't see that we do it anywhere.  We mostly fetch objects directly and mess with them.
<gary_poster> ok deryck, thank you!
<deryck> np!
<jcsackett> sinzui: i am on mumble.
<matsubara> An indirect member of a team that owns a private branch can't access the private branch. Shouldn't the user be allowed? Is there a restriction for indirect members?
<abentley> deryck: I've gotten though the display side of doing the checklist, about to move onto the webservice part.  I sent you an email with some screenshots.  Just wanted to get your take on it.
<deryck> abentley: yes, just about to respond....  looked at the screenshots and they looked great.  Just wanted to peak at the code before I responded.
<abentley> deryck: cool.
<deryck> abentley: yeah, I think this is good.  Any specific questions?
<abentley> deryck: how do I test the controller, especially once I wire it up to do AJAX?
<abentley> deryck: Is there a better way to set/unset the "unseen" class?
<deryck> abentley: ok, so re: testing with AJAX, look at Y.Mock.  http://developer.yahoo.com/yui/3/test/#mockobjects  And lib/lp/soyuz/javascript/tests/lp_dynamic_dom_updater.js is an example.
<deryck> abentley: and as for the unseen stuff, I think that's fine what you have.  I think I would call it toggle_unseen, or something like that, just to be clear what it does.  But it's a fine approach to do it that way.
<deryck> we always use add/removeClass
<abentley> deryck: I wouldn't call it toggle_unseen, because "toggle" to me means that it changes the state to the opposite every time you call it.  This one sets the state.
<deryck> abentley: ok, but it is unsetting unseen, i.e. making the thing seen if needed, yes?
<abentley> deryck: Yes.  The state is the presence or absence of the "unseen" class on the element.
<deryck> abentley: so that you can use one function to turn the class on or off, right?  set_unseen, to me, implies we're only ever making the thing unseen.  Maybe toggle isn't the right word, but I think the name could be clearer.
<deryck> abentley: but this is a minor thing, obviously.
<abentley> deryck: I'd like to make the name clearer, but it's hard to express what it does succinctly.
<deryck> abentley: fair enough
<deryck> set_or_remove_class_unseen_depending_on_state ;)
<deryck> but since it's js, we can call it -- sorcudos
<abentley> deryck: he he.
<abentley> deryck: I could invert the logic and call it set_visibility.
<deryck> indeed.  that would work.
<deryck> so much  nicer than my name.
<abentley> deryck: done.
<abentley> What about mocking the DOM for those Y.one calls?
<abentley> deryck: ^
<deryck> abentley: on call now, sorry
<abentley> deryck: ack.
<deryck> abentley: so mocking the DOM.... do you mean using Y.Mock, or just having some similar HTML in the test html file to interact with?
<abentley> deryck: Oh, I see.  I should be able to just plunk the same code in the test html file, and then interrogate the test file's DOM.
<deryck> abentley: yes, exactly.  That's how we usually do it.
<abentley> deryck: Great.
<abentley> deryck: btw, there's already a function to add/remove arbitrary classes on Y.Node: toggleClass.
<deryck> abentley: ah, nice.  I didn't even know that.  so much there.
<abentley> deryck: By default, it changes the state to the opposite, but you can specify True or False.
<deryck> cool
<lifeless> moin moin
<lifeless> allenap: hey
<lifeless> allenap: is 12584 ok now?
<lifeless> allenap: if not, how long are you proposing we wait to be confident ?
<lifeless> deryck: are you able to qa rev 12586 ?
<deryck> lifeless: yes, going to get it before I EOD today.  starting in about 10 minutes on it.
<lifeless> deryck: :)
<ToyKeeper> I'm not sure if it was mentioned by number, but I'm hoping to resolve bug #734995 soonish for a project this week.
<_mup_> Bug #734995: access denied for group branches (canonical-isd-hackers/oops-tools/isd-oops) <Launchpad itself:New> < https://launchpad.net/bugs/734995 >
<thumper> deryck, danilos: I really wanted to get to add a "lp:user:loggedin" event
<thumper> with the idea that we had a single check point
<thumper> and the rest of the code just used that event
<deryck> yeah, that would be cool
<thumper> then, if we ever got around to it
<thumper> we could have an ajax login
<thumper> and have the page morph
<sinzui> Let's change login to sign in first so that the link matches the words on the SSO pages
 * thumper goes to make a mega-coffee
<deryck> sinzui: I haven't had a chance to review that email you sent.  Long queues today of reviews, questions, etc.  I'll look tonight or early AM tomorrow.
<sinzui> deryck: thanks. rvba is looking for confirmation that my suggestion is sane.
<deryck> ah
<leonardr> sinzui, thanks so much for chasing down those test failures. i'm looking at the code now
<deryck> later on, everyone!
<sinzui> leonardr: I think I should have sent my changes to ec2. I hope I did not break anything in the so-called fix
<leonardr> sinzui: ec2 land?
<leonardr> those test failures don't look related to the bug that was filed. i'll try to reproduce
<sinzui> leonardr: `utilities/ec2 test launchpad=devel` will test and send you the report.
<leonardr> sinzui: the remaining error is a problem in the way the test is written
<leonardr> you are explicitly passing in None for the 'version' argument
<sinzui> \o/ I suck
<leonardr> sinzui: http://pastebin.ubuntu.com/580280/
<leonardr> want me to review the rest of your changes?
<sinzui> leonardr: I should have known to pass the version...I told users on #launchpad to do that last week
<sinzui> leonardr: sure
<leonardr> sinzui: well, you fixed a ton of bugs i never would have been able to fix, so don't feel bad
<sinzui> leonardr: I *think* I fixed an oopse, but I could not find an example. The change I made to the bug nomination view implies if there is ever a validation error on the view, it will oops instead of  showing the user how to fix the data.
<sinzui> I should look again before you land the branch
<lifeless> matsubara-afk: ping (oops-tools)
<leonardr> sinzui: problem #1. it looks like we both added the same lines to security.cfg?
<sinzui> oops
<leonardr> i put mine in a block at the bottom because it seemed really specific to the one task
<leonardr> and you intermixed yours with all the others
<leonardr> i could go either way but we need to pick one
<lifeless> flacoste: 7574479
<lifeless> flacoste: https://devpad.canonical.com/~lpqateam/ppr/lpnet/latest-daily-categories.html
<thumper> mwhudson: ping
<leonardr> sinzui: why did you remove the entirety of bugtask-target-link-titles? is it redundant with another test?
<leonardr> ah, you wrote unit tests instead
<leonardr> ok
<mwhudson> thumper: pong
<thumper> mwhudson: want to review a blueprint branch for me?
<mwhudson> thumper: how complicated is it?
<thumper> pretty simple, https://code.launchpad.net/~thumper/launchpad/blueprint-dependency-vocabulary/+merge/53192
<allenap> lifeless: I think it's okay. I've heard nothing all day, and I've been around various bits of qastaging. I'll mark it qa-ok.
<mwhudson> thumper: yeah alright i'll take a look
<thumper> mwhudson: thanks
<mwhudson> my lp tree is a bit out of date and i want to look at it in meld, so i'll be a few minutes
<thumper> mwhudson: for some reason, no-one likes blueprints
<mwhudson> prod me if i seem to have forgotten :)
<leonardr> sinzui: what does changing the layer from LaunchpadFunctionalLayer to DatabaseFunctionalLayer accomplish? is it just cleanup?
<mwhudson> thumper: i can't possibly imagine why, the code quality is so amazing!
<thumper> mwhudson: what? not even hacking on LP for fun?
<sinzui> leonardr: yes, I had to disamantle the test to discover which conditions were not valid with sane sample data and factory.
<sinzui> leonardr: the tests run faster when they do not setup, clean, and teardown the librarian
<lifeless> allenap: thanks
<sinzui> leonardr: also that doctest I removed was also being run twice because it was implicitly loaded by test_doc, and explicitly loaded in the module I added the replacement test. Everything ran much faster once I test exactly what the view did
<leonardr> yeah, i noticed that duplication
<leonardr> sinzui: why did you remove the "Handling IntegrityError" doctest?
<sinzui> leonardr: doctests do not test error condition. That is the domain of an implementation test...
<sinzui> leonardr: we cannot get to that DB integrity error when the view ensures valid data..We do not permit integrity errors, so there should not have been a test to demonstrate the view was unfinishd
<mwhudson> huh, my launchpad repo just hit 100k revs i think
<mwhudson> the repack is taking a while...
<lifeless> sinzui: we had a doctest checking the view was unfinished? /me facepalms
<sinzui> lifeless: yes, it was quite a surprise. leonardr's branch that validates data before we apply revealed a lot os badness
<leonardr> sinzui: on closer inspection of the diff, i think there are only 2 duplicate entries in security.cfg: binarypackagebuild and binarypackagename for [processmail]
<lifeless> hmm
<lifeless> I wish I could close popup diffs :)
<sinzui> leonardr: just remove my additions. Sorry about the confusion
<leonardr> np
* benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<thumper> I wonder if any one actually uses the +deptree view
<thumper> leonardr: mumble?
<leonardr> sinzui: ok, i think this is ready: lp:~leonardr/launchpad/sinzui-106338
<leonardr> thumper: yes
<thumper> lifeless: can you tell me if anyone ever goes to a blueprint +deptree page?
<leonardr> sinzui: take a look, and i'll try landing again? (it can wait until tomorrow if you want your test to finish first)
<sinzui> leonardr: I am looking now
<leonardr> great
<lifeless> thumper: ppr over a month should be a decent indicator
<lifeless> thumper: https://devpad.canonical.com/~lpqateam/ppr/lpnet/latest-monthly-pageids.html
<sinzui> leonardr: r=me with these trivial changes: http://pastebin.ubuntu.com/580310/
<huwshimi> Morning
<poolie> hi huwshimi
<poolie> hi flacoste?
<flacoste> hi poolie
<flacoste> poolie: call time?
<huwshimi> jml: Good evening
<jml> huwshimi: hi
<jml> huwshimi: I'll just grab some yoghurt & will be right with you.
<huwshimi> jml: Sure
<poolie> flacoste, yes, hi
<flacoste> poolie: i'm on skype
<poolie> hm
<thumper> sinzui: still around?
* lifeless changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | firefighting: - | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews/
<wgrant> :( We are turning into Java.
<lifeless> wgrant: we are?
<wallyworld> wgrant: my irc client decided to disconnect so i missed the lead up to your java comment - why do you say we are turning into java?
<wgrant> lifeless: We now have external projects included in our source tree with Launchpad-specific changes.
<lifeless> wgrant: which project?
<wallyworld> why is that java-esque?
<wgrant> lib/lp/contrib/javascript/yui3-gallery/gallery-accordion/
<wgrant> wallyworld: Java projects are notorious for including all their dependencies in their trees.
<lifeless> move it to lazr-js ?
<lifeless> wgrant: so are c++, C, python, ruby projects IME
<wallyworld> wgrant: not the ones i've worked on :-)
<jelmer> lifeless: btw, did the multiple versions of python packages discussion go anywhere?
<wallyworld> bac: hi brad, are you able to +1 the recent mp you looked at for me now that i have done the requested changes?
<bac> wallyworld: thanks for the reminder.  i'll look now.
<wallyworld> thanks
<lifeless> jelmer: no, I didn't manage to get the thought leaders to step out of their worldview
<jelmer> lifeless: :(
<lifeless> jelmer: feel free to jump in on debian-python
<bac> wallyworld: done.
<wallyworld> bac: thank you!
<lifeless> I've shelved it for now, will probably come back to it, or perhaps we should migrate more massively and just avoid the issue entirely
<wgrant> lifeless: The big problems for Ubuntu are Java projects. And Chromium.
<poolie> hi wgrant, lifeless
<lifeless> hi poolie
<poolie> did i understand you correctly in my last message on bug 716174?
<_mup_> Bug #716174: api error messages should be machine-readable <Launchpad itself:Triaged> < https://launchpad.net/bugs/716174 >
<sinzui> hi huwshimi do you want o mumble today?
<wgrant> Morning poolie.
<lifeless> poolie: oh bah, I was reading mail and just closed it again
<lifeless> poolie: hadn't seen your irc comment
<lifeless> poolie: we may be talking past each other; voice?
<thumper> james_w: ping
<poolie> lifeless, sure!
<poolie> pots or skype or mumble?
<lifeless> skype is easiest
<thumper> mwhudson: can you think of any current blueprints that have linked bugs?
<mwhudson> thumper: not off the top of my head, no
<thumper> hmm... ok
<thumper> mwhudson: did you take a look at the vocab branch?
<mwhudson> thumper: yes, bzr is distracting me
<mwhudson> thumper: i'll have a review in a few minutes
<huwshimi> sinzui: Sorry I completely missed your message
<thumper> mwhudson: ok, thanks
<huwshimi> sinzui: Have you already finished?
<mwhudson> thumper: how can  _is_valid_candidate be called with something that isn't a spec?
<thumper> mwhudson: shitty url traversal?
<thumper> mwhudson: it handled the None case well
<thumper> probably never
<mwhudson> thumper: did you look at how _spec_from_url works?
<mwhudson> (it's horrible, yes, but it will only return a spec)
<thumper> only briefly cause it looked shitty
<thumper> probably paranoia on my part
<mwhudson> ok
<mwhudson> can you remove that please? :)
<thumper> I originally had just the none check
<thumper> sure
<poolie> i'm getting a 'please try again' page talking to lp
<poolie> in fact https://bugs.launchpad.net/bzr/+bugs?field.searchtext=winsshd&orderby=-importance&search=Search&field.status%3Alist=NEW&field.status%3Alist=INCOMPLETE_WITH_RESPONSE&field.status%3Alist=INCOMPLETE_WITHOUT_RESPONSE&field.status%3Alist=CONFIRMED&field.status%3Alist=TRIAGED&field.status%3Alist=INPROGRESS&field.status%3Alist=FIXCOMMITTED&field.assignee=&field.bug_reporter=&field.omit_dupes=on&field.has_patch=&field.has_no_package= is per
<poolie> sistently timing out
<wgrant> poolie: Timing out, or "please try again"ing?
<wgrant> It's working for me.
<wgrant> (although it's slow... 9s)
<wgrant> We are mid-rollout, but it shouldn't happening that much...
<mwhudson> thumper: revewied
<thumper> mwhudson: thanks
<mwhudson> thumper: usual nit-picking :)
<thumper> :)
<poolie> wgrant, both oopsing and giving the proxy error page
<poolie> huwshimi, just a thought on the private bug thing
<poolie> which is that the stackoverflow ribbons are point-in-time notifications, and after you've read them it's reasonable to dismiss them
<poolie> whereas if something's a static property of a bug
<poolie> and potentially coming up on every bug page you look at all day
<lifeless> poolie: back
<poolie> it may get annoying
<poolie> lifeless, let me just finish this post
<poolie> wgrant, timed out again
<poolie> https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1899K1764
<wgrant> Thanks.
<wgrant> Confirmed.
 * wgrant is waiting for it to sync.
<poolie> lifeless, curl -v https://code.staging.launchpad.net/~maria-captains/bzr/diffoptions/+merge/52325
<poolie> cOOPS-1899STAGING65
<poolie> OOPS-1899STAGING65
<wgrant> lifeless: I think your FTI changes have made bug searches significantly slower. See poolie's OOPS (it's finally synced)
<lifeless> poolie: 'X-Lazr-OopsId
<lifeless> wgrant: that would be win, wouldn't it
<lifeless> wgrant: Time: 3684.797 ms (staging)
<wgrant> lifeless: The cost estimate with Bug.fti is 10x the one with BugTask.fti.
<wgrant> I guess it has to load a lot more Bugs.
<lifeless> wgrant: which is freaking odd as it was an || before
<lifeless> wgrant: 2606.15..2606.16
<lifeless> (I'm looking at the count(*) to start with
<wgrant> I am too.
<lifeless> Index Scan using bug_fti on bug  (cost=0.00..2306.35 rows=42 width=12) (actual time=134.970..469.008 rows=2 loops=1)
<lifeless>                Index Cond: ((fti)::tsvector @@ '''winsshd'''::tsquery)
<wgrant> Oh, it's doing that first?
<wgrant> On mawson it does bugtask first, and both are <400ms once the cache is hot.
<lifeless> yes, it does bug_fti first
<wgrant>          ->  Index Scan using bug_pkey on bug  (cost=0.00..54.26 rows=1 width=12) (actual time=0.015..0.015 rows=0 loops=2709)
<wgrant>                Index Cond: (bug.id = public.bugtask.bug)
<wgrant>                Filter: ((bug.duplicateof IS NULL) AND ((bug.fti)::tsvector @@ '''winsshd'''::tsquery) AND ((NOT bug.private) OR (SubPlan 1)))
<lifeless>          ->  Index Scan using bugtask__product__bug__key on bugtask  (cost=0.00..6.24 rows=1 width=16) (actual time=0.033..0.034 rows=1 loops=2)
<lifeless>                Index Cond: ((public.bugtask.product = 1186) AND (public.bugtask.bug = bug.id))
<lifeless>                Filter: ((public.bugtask.status = 10) OR ((public.bugtask.date_incomplete IS NOT NULL) AND (public.bugtask.status = 15)) OR (public.bugtask.status = 15) OR (public.bugtask.status = 20) OR (public.bugtask.status = 21) OR (public.bugtask.status = 22) OR (public.bugtask.status = 25))
<lifeless> wgrant: \d bug
<lifeless> wgrant: \di+ bug_fti
<wgrant> lifeless:     "bug_fti" gist (fti)
<wgrant> No similar index on bugtask_fti, though...
<wgrant>  Schema |  Name   | Type  |  Owner   | Table |  Size  | Description
<wgrant> --------+---------+-------+----------+-------+--------+-------------
<wgrant>  public | bug_fti | index | postgres | bug   | 575 MB |
<lifeless> wgrant: indeed, it was missing on prod a few months back
<lifeless> wgrant: one of the first hints that fti was whack
<lifeless> wgrant: the query doesn't use bugtask_fti now, does it ?
<wgrant> lifeless: I doubt it.
<wgrant> Well, the new query couldn't use it.
<wgrant> The old one could have. I don't know if it did.
#launchpad-dev 2011-03-15
<thumper> AAArrgggghhh!!!!
<thumper> FFS
<thumper> factory.makeBugTarget returns a bugtarget which has a bug with two! targets
<lifeless> conjoined master?
 * thumper shrugs
<thumper> I don't even know what that means
<leonardr> thumper: sinzui and i just worked on that, let me see if i can understand it for you
<lifeless> if you have a bug targeted on the development focus for a pillar, then you have 2 bug tasks
<lifeless> one on the pillar, one on the series
<lifeless> and this is handled specially.
<thumper> lifeless: a bare "makeBugTask" should give the simplest possible, surely
<lifeless> I don't know why we do this.
<lifeless> thumper: if the target is the default series for the pillar, making a bug on that target will implicitly create one on the pillar
<thumper> lifeless: a simple makeBugTask should not target any series
<lifeless> thumper: sure
<wgrant> lifeless: So, are you looking at the bug search issue? It is just about critical.
<thumper> ok, it seems that it has created a bug with two different product targets
<thumper> leonardr: what enlightenment can you offer?
<lifeless> thumper: ok, thats different
<leonardr> lifeless: if you just call makeBugTask() with no arguments, the target is a product, and a product has no need for a prerequisite bugtask, so i'm not sure what's going on
<leonardr> er, thumper -^
<thumper> I know
<thumper> make bug makes a default task
<lifeless> it has to
<thumper> makeBugTask makes a task
<thumper> so when you go makeBugTask with no params
<thumper> you get two tasks
<lifeless> wgrant: so oopses were still nominal yesterday
<thumper> one from the makeBug call
<lifeless> wgrant: are you seeing other data ?
<thumper> and one added in the makeBugTask
 * thumper fixorates
<leonardr> thumper: your fix may conflict with the branch i'm trying to land now, though it should be reconcilable
<thumper> leonardr: what did you do about it?
<leonardr> thumper: i changed something else in makeBugTask
<leonardr> let me see...
<thumper> which branch is it?
<leonardr> lp:~leonardr/launchpad/bug-106338
<wgrant> lifeless: bzr bug searches time out, and when they don't they take >10s.
<wgrant> lifeless: Didn't this change only get deployed like 7 hours ago?
<thumper> hmm...
<leonardr> thumper: we made some other changes to makeBugTask
<lifeless> wgrant: 12582 was the change
<thumper> leonardr: I see that
<thumper> leonardr: I'm going to change my approach in my test to not need it
<leonardr> ok. fwiw, i think your proposed change is complementary to ours
<wgrant> Right, and I see the LPS change for that a little over 7 hours ago.
<wgrant> lifeless: So I don't think we've seen any OOPS reports for that yet.
<lifeless> wgrant: https://bugs.launchpad.net/ubuntu/+bugs?field.searchtext=sshd&orderby=-importance&search=Search&field.status:list=NEW&field.status:list=INCOMPLETE_WITH_RESPONSE&field.status:list=INCOMPLETE_WITHOUT_RESPONSE&field.status:list=CONFIRMED&field.status:list=TRIAGED&field.status:list=INPROGRESS&field.status:list=FIXCOMMITTED&field.assignee=&field.bug_reporter=&field.omit_dupes=on&field.has_patch=&field.has_no_package=
<lifeless> wgrant: what I mean is 'searches on ubuntu seem ok'
<huwshimi> If wanted to add another static class to this: <td tal:attributes="class view/scope_css_class"> would it be <td tal:attributes="class view/scope_css_class text-right"> or <td tal:attributes="class view/scope_css_class" class="text-right">?
<lifeless> The latter
<lifeless> tal is for when you need things evaluated
<lifeless> mmm
<lifeless> it probably won't combine right
<lifeless> you probably need to fix scope_css_class
<StevenK> lifeless: Can you run select count(id) from SourcePackageRelease; on qastaging? I'd like to get a feel for the size of the table
<huwshimi> lifeless: How do I figure out what object that tal statement is using? There are two classes that define scope_css_class.
<huwshimi> lifeless: I guess that's kind of a general question about these templates
<lifeless> huwshimi: bug 735196 may interest you
<huwshimi> specifically the templates in question are lib/lp/answers/templates/questions-index.pt and lib/lp/blueprints/templates/specifications-index.pt
<_mup_> Bug #735196: Text pops out of announcment box <css> <Launchpad itself:Triaged> < https://launchpad.net/bugs/735196 >
<wgrant> :(
<lifeless> T-20
<wgrant> Heh, yes.
<wgrant> It is a painful wait.
<wgrant> But also not today a terribly interesting one.
<wallyworld> wgrant: can you tell me the command to run to tar up an egg. in the past i've just done it manually but i'm sure there's an easier way.
<lifeless> setup.py egg_info ...
<wgrant> wallyworld: It varies.
<lifeless> or setup.py sdist
<wgrant> wallyworld: Often python setup.py egg_info sdist
<lifeless> or bze export
<wgrant> Right.
<wgrant> All of what lifeless says is valid.
<wgrant> Which egg?
<wallyworld> thanks. i'll try one of those. i have proposed a change to lazr-js and want to use it locally while i'm waiting to get it approved
<wgrant> setup.py egg_info -b-somesuffix sdist
<lifeless> wgrant: egg_info does an sdist AFAICT
<wgrant> -b adds a suffix to the version.
<wgrant> Hm.
 * wallyworld goes to impersonate a chicken
 * wgrant deletes code.
<lifeless> holy cow
<lifeless> that with clause I was working on yesterday -rocks-
<lifeless> cold on qastaging:  Total runtime: 151.508 ms
<lifeless> (20 rows)
<lifeless> Time: 193.425 ms
<wgrant> lifeless: POFile:+translate?
<lifeless> yeah
<wgrant> Wasn't the old one several repitions of >1s?
<wgrant> *repetitions
<lifeless> so the old one was 1.1s cold, 300ms hot
<lifeless> this is
<wgrant> Ah.
<lifeless> 190ms hot, 9ms cold
<wgrant> I doubt that, somehow.
<lifeless> stub tested on prob, 70ms cold
<lifeless> 190ms cold 9ms hot. doh.
<wgrant> Heh.
<lifeless> we'll probably find it throws some corner case out and becomes pathological
<lifeless> cats are awesome
<wgrant> Occasionally.
<wgrant> Mine likes to sit next to my computer. This is OK except when the side of the case is off, and tail and CPU fan intersect.
<lifeless> lol
<wgrant> Unhappy cat.
<wgrant> What is yours doing?
<lifeless> right now, curled up in the basement of its climber, snoring gently
<wgrant> Heh.
 * wgrant deletes lots of NullBugTask-related hacks.
<lifeless> wgrant: https://pastebin.canonical.com/44718/ may interest you (re search perf)
<lifeless> dropping the private-bug query -> 1 second
<lifeless> but, 10% of the estimated cost
<wgrant> lifeless: :(
<wgrant> It's still got a really awful plan, though.
<wgrant> mawson goes the other way and is sub-second.
<wgrant>  * 307 Time Outs
<wgrant> So close :(
<lifeless> hard + soft = 1800
<lifeless> 300 down from yesterdays report
<wgrant> And we only had <8 hours of fixedness in that report.
<lifeless> so I think we're good to move ahead
<wgrant> To 12s? Or less?
<lifeless> 12
<wgrant> :(
<lifeless> I'm worried about search
<wgrant> Yeah.
<lifeless> crazy eddie plans
<wgrant> Gnrgh.
<lifeless> jcsackett: hows cve:+index treating you?
<lifeless> Distribution:+bugtarget-portlet-bugfilters-stats will need fixing soon
<lifeless> bugtask:+index is looking pretty happy, as such things go
<wgrant> Bugs' ZCML is crap.
<lifeless> s/[^ ]* //
<wgrant> +editstatus is protected from rendering on a NullBugTask in three or four ways.
<lifeless> win
<wgrant> Why would a task listing ever hit a nullbugtask? :/
<lifeless> well
<lifeless> what was that bug I was looking at yesterday
<lifeless> the one with memcache in it ? it wasn't redirecting
<wgrant> I know why it sometimes doesn't redirect.
<wgrant> That doesn't explain how one would have *ever* shown up in a listing.
<lifeless> oh
<lifeless> thats because the current context has to show
<wgrant> Hm?
<lifeless> in the past anyhow
<wgrant> Our tests are slow.
<StevenK> lifeless: Did you miss my ping?
<lifeless> StevenK: presumably
<lifeless> whats uo ?
<lifeless> ah
<lifeless> yes, i did it
<lifeless> 822218
<StevenK> Ouch, thanks
<thumper> lifeless: your with_ statement seems to have a bug in __contains__
<thumper> lifeless: possibly worth making a test for
<StevenK> lifeless: I think the fastest way to fix the job on loganberry is to run a memcached on it until the job is done. But I think you might want to investigate why we have one memcached per appserver anyway.
<StevenK> wgrant: http://pastebin.ubuntu.com/580404/
<wgrant> StevenK: Looks good.
<StevenK> wgrant: Shouldn't you be the OCR, anyway?
<wgrant> StevenK: A reasonable point.
<wgrant> lp.bugs.tests.test_bugtask_search.TestNoPreloadBugtaskTargets* is SLOW.
<StevenK> How slow?
<lifeless> thumper: what happens?
<wgrant> StevenK: Many minutes.
<lifeless> StevenK: we have a cluster
<thumper> lifeless: the with statement isn't used
<StevenK> wgrant: Kill it!
<lifeless> StevenK: and deterministic routing to them
<thumper> lifeless: and you get:     ProgrammingError: relation "blocked" does not exist
<thumper>     LINE 1: ....id != 14 AND (Specification.id in (select id from blocked))
<lifeless> StevenK: so the fastest way is to make sure the memcaches are configured correctly and have gsa open firewalls if needed.
<lifeless> thumper: thanks
<lifeless> thumper: probably shallow
<thumper> lifeless: probably
<StevenK> lifeless: So the way forward is an RT?
<lifeless> StevenK: in either scenario yes
<StevenK> RT's make me sad.
<StevenK> wgrant: If I'm reading these numbers right, we have processed over 4,500 SPRs with 6 failures.
<wgrant> StevenK: Excellent.
<StevenK> But since memcache doesn't work, we keep looking at the 6 failures.
<wgrant> Right.
<lifeless> heh
<lifeless> doing bzr pull --overwrite lp:~lifeless/storm/with-without-datetime in my lp tree -- bad
<wgrant> Haha
<lifeless> thumper: lp:~lifeless/storm/with-without-datetime contains the fix
<lifeless> thumper: I'll let you make a new egg from the branch, because you'll need that to include it in your branch
<thumper> lifeless: I don't need it
<thumper> just bumped into it
<lifeless> ah
<lifeless> ok
<lifeless> I might bump the egg shortly, thanks for letting me know
<StevenK> wgrant, lifeless: Back of envolope maths: We're looking at ~ 2 months
<wgrant> Really?
<wgrant> How many need to be processed?
<wgrant> That's almost twice my prediction.
<StevenK> My maths was assuming all of them, which I've just realised is dead wong
<lifeless> so, you can make this faster
<lifeless> change garbo so that rather than a 15 minute per loop limit
<wgrant> This runs last, dying only on the hour?
<lifeless> it sets a 50 minute budget for all loops
<lifeless> and per loop allows $remaining_budget as a limit
<StevenK> lifeless: No, let me fix my maths first
<mwhudson> thumper: the ajax love for blueprints is very nice
<lifeless> StevenK: fixing this would be a good idea anyway
<thumper> mwhudson: yay, it is noticed :)
<lifeless> the 15 minute per loop is bong because if 5 loops take max time we'll exceed one hour
 * StevenK looks for the query lifeless wrote
<mwhudson> thumper: you should be coming to UDS, i bet all the ubuntu devs would buy you beer
<StevenK> thumper: I think you should blog about the blueprint changes and link to the screencast
<wgrant> Let's not advertise blueprints.â¦
<StevenK> That may involve jcastro and dholbach flying down to hug you
<thumper> StevenK: I did blog about it
<lifeless> wgrant: its good to tell people about things we're improving
<StevenK> lifeless: select changelog is null, count(*) from sourcepackagerelease group by changelog is null; on qas, plz
<lifeless>  f        | 270845
<lifeless>  t        | 551373
<StevenK> So we are missing 551k changelogs?
<lifeless> on qas
<StevenK> Right
<StevenK> On prod it is likely 547k
<StevenK> Still 48 days, give or take
<lifeless> so, do the change I propose to garbo (and put this job at the end)
<lifeless> its a general win and will give you 3-4 times the throughput
<wgrant> 14:19:52 < lifeless> and per loop allows $remaining_budget as a limit
<wgrant> How is that going to work?
<wgrant> That will allow even the first loop to exhaust the full window.
<StevenK> lifeless: Yes, I was just about to look at that
<wgrant> Preventing the subsequent ones from running at sll.
<StevenK> wgrant: So, garbo-hourly has a budget of 50 minutes. Allow the loops to have a budget of whatever is left of the 50, rather than 15 minutes.
<wgrant> StevenK: Right. So loop 1 has 50 minutes to run it, and does so.
<StevenK> wgrant: Can haz review of https://code.launchpad.net/~stevenk/launchpad/dsd-base_source_pub-archive/+merge/53357
<wgrant> Goodbye loops >=2
<StevenK> lifeless: wgrant has a good point
<wgrant> StevenK: I shouldn't, but since I'm mentored...
<wgrant> lifeless: Could you mentor StevenK's?
<StevenK> wgrant: Well, then I'll ask lifeless for a review, then
<StevenK> wgrant: You don't need to feel obligated to review
<wgrant> StevenK: I am happy to, but I think it's not really appropriate in this case. But this case is pretty trivial, so meh.
<wgrant> Anyone want to review https://code.launchpad.net/~wgrant/launchpad/unuse-nullbugtask/+merge/53363?
<lifeless> wgrant: it would, but only one of them ever uses its full budget atm - the spr changelog one
<lifeless> https://code.launchpad.net/~lifeless/launchpad/bug-734642/+merge/53365
<StevenK> lifeless: That is false, read the logs
<lifeless> StevenK: what else is using its full budget ?
<StevenK> 2011-03-15 02:13:16 INFO    Running OAuthNoncePruner
<StevenK> 2011-03-15 02:28:08 DEBUG   Done. 1 items in 1 iterations, 891.136 seconds, average size 1.000000 (0.00112216283655/s)
<StevenK> That isn't every run, but it does happen.
<lifeless> that looks like a bug
<lifeless> table contention perhaps
<StevenK> It is table contention
<lifeless> not at 15 minutes its not
<StevenK> It blocks on two transactions and sleeps
<StevenK> But I don't want to paste 6 lines to the channel
<StevenK> thumper: https://code.launchpad.net/~wgrant/launchpad/unuse-nullbugtask/+merge/53363?
 * thumper looks
<lifeless> StevenK: the backoff algorithm needs fixing I guess
<thumper> StevenK: done
 * thumper graduates StevenK
<thumper> StevenK: I'll send an email to the list
<LPCIBot> Project windmill build #49: STILL FAILING in 1 hr 7 min: https://hudson.wedontsleep.org/job/windmill/49/
<wgrant> thumper: Thanks. Could you now mentor https://code.launchpad.net/~lifeless/launchpad/bug-734642/+merge/53365?
<StevenK> thumper: !!!!
<wgrant> (congrats StevenK)
<StevenK> thumper: Let me edit ReviewerSchedule
<poolie> yes, congrats
<poolie> wgrant i filed bug 735260 for the problem i mentioned before
<_mup_> Bug #735260: multiple timeouts in bug search <oops> <regression> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/735260 >
<poolie> it seems still broken
<wgrant> poolie: You probably want to convince lifeless that it has regressed.
<wgrant> I am convinced.
<lifeless> yes, the tradeoffs have altered
<lifeless> ubuntu bug search is faster than it was
<lifeless> and we have more of those
<poolie> seriously
<lifeless> yes, seriously
<lifeless> we had 300 oopses from ~ 8 hours of the new search query
<lifeless> (total timeouts)
<poolie> ah, i wasn't questioning your assertion, just expressing frustration at not being able to search
<lifeless> thats down day on day from monday and from the weekend
<poolie> good thing i have email i guess
<lifeless> https://lpstats.canonical.com/graphs/OopsLpnetHourly/
<lifeless> there is certainly a spike
<poolie> i'm glad it's progressing
<lifeless> thumper: https://code.launchpad.net/~lifeless/launchpad/bug-734642/+merge/53365
<lifeless> thumper: oh you have already - thanks
<StevenK> lifeless: Did you want to reping about the LPS feature flags?
<lifeless> poolie: try searching at bugs.launchpad.net/bugs
<lifeless> poolie: its performing fine - 5 seconds to query sshd
<lifeless> StevenK: econtext
<StevenK> lifeless: Removal of the recipe beta feature flag
<poolie> lifeless, https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1900G239
<poolie> https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1900J237
<poolie> is not 'fine'
<lifeless> poolie: thats very odd
<lifeless> because I tried right before commenting
<lifeless> poolie: what query did you use?
<wgrant> Also right after commenting?
<wgrant> Master vs. slave, maybe?
<lifeless> wgrant: possibly
<lifeless> 98 queries/external actions issued in 4.67 seconds
<poolie> i typed 'shortreadverror' in the search field there, both with 'bzr' in the project field and without
<lifeless> 29 queries/external actions issued in 6.53 seconds
<lifeless> bug 413430 bug 402857 bug 673439
<_mup_> Bug #413430: ShortReadvError on index file <data-integrity> <ui> <Bazaar:Confirmed> < https://launchpad.net/bugs/413430 >
<_mup_> Bug #402857: ShortReadvError in pack file: bzr check should check pack file hashes and recover from problems <check> <packs> <Bazaar:Confirmed> < https://launchpad.net/bugs/402857 >
<poolie> just curious, why did you untag it 'oops' and 'regression'?
<lifeless> we use oops and timeout tags as partitions
<poolie> 'oops' means 'non-timeout oops'?
<poolie> ok
<lifeless> search performance is being a problem but we haven't figured out what is up yet
<lifeless> it may well be a coincidence
<lifeless> particularly given that right now the same searches are working for me an dbreaking for you
<poolie> this used to work reliably, and now it doesn't --> regression, is it not?
<lifeless> poolie: it didn't work reliably
<wgrant> For !Ubuntu it did...
<lifeless> wgrant: poolie is searching in ubuntu still - the bug he filed was an ubuntu bug search
<poolie> for me, it's gone from working >95% of the time to working <10% of the time
<poolie> no, it wasn't
<wgrant> Wasn't it in the bzr product?
<poolie> wgrant, yes
<lifeless> https://bugs.launchpad.net/ubuntu/+bugs?field ...
<lifeless> poolie: ^ thats the url in your bug description, which I have not altered
<lifeless> the other two oopses you've reported here I have asked for a sync of, to look at
<poolie> ok
<poolie> it is also failing in plain bzr searches, i'm pretty sure
<wgrant> It is, I saw some earlier.
<poolie> i'm confused, i thought i posted several oopses in the description
<poolie> for instance https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1900J237
<lifeless> you pasted that here, its syncing at the moment
<lifeless> after my suggestion to search on the root context
<lifeless> the J oops is on bzr
<lifeless> 9 second count cold; 2 second count hot
<poolie> <poolie> i'm getting a 'please try again' page talking to lp
<poolie>  in fact https://bugs.launchpad.net/bzr/+bugs?field.searchtext=winsshd&orderby=-importance&search=Search&field.status%3Alist=NEW&field.status%3Alist=INCOMPLETE_WITH_RESPONSE&field.status%3Alist=INCOMPLETE_WITHOUT_RESPONSE&field.status%3Alist=CONFIRMED&field.status%3Alist=TRIAGED&field.status%3Alist=INPROGRESS&field.status%3Alist=FIXCOMMITTED&field.assignee=&field.bug_reporter=&field.omit_dupes=on&field.has_patch=&field.has_no_package= is pe
<poolie> rsistently timing out
<poolie> anyhow
<poolie> does 'regression' mean 'used to work and now doesn't' or is it more subtle?
<lifeless>          ->  Index Scan using bug_fti on bug  (cost=0.00..2306.35 rows=42 width=4) (actual time=43.873..1929.329 rows=7 loops=1)
<lifeless>                Index Cond: ((fti)::tsvector @@ '''shortreadverror'''::tsquery)
<poolie> i don't care what tags it has, i just want to understand what they mean
<lifeless>                Filter: ((duplicateof IS NULL) AND ((NOT private) OR (SubPlan 1)))
<lifeless> poolie: regression isn't strictly defined; I may add it back to the bug once we have a handle on whats up
<lifeless> poolie: regressions and timeouts and oopses all fall in the same priority bucket (and escalated bugs)
<poolie> anyhow, to be clear
<poolie> searching just in bzr was working and now isn't
<poolie> i might have accidentally been in the package context for one search that i pasted into the bug, but it's also failed several times in the project
<lifeless> sure
<lifeless> so, we're looking at it
<poolie> thanks
<lifeless> have been all day in fact
<poolie> i'll let you get on with it
<poolie> just wanted to be sure the bug was accurate
<lifeless> poolie: regression is really about code changes that break things horribly. There is some evidence that this isn't a code change issue per se
<lifeless> *even though* a code change was landed
<wgrant> StevenK: Want to use your new superpowers on https://code.launchpad.net/~wgrant/launchpad/delete-nullbugtask/+merge/53367?
<wgrant> Ugh, forgot prereq.
<wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/delete-nullbugtask/+merge/53369
<StevenK> wgrant: Done.
<StevenK> It's going to hard to get out of the habit of code* everything
<wgrant> Thanks.
<StevenK> wgrant: I fully support your plan to kill NullBugTask, and wish to subscribe to your newsletter.
<wgrant> StevenK: you should also set the MP status, probably.
<StevenK> Also done
<wgrant> You overwrote mine :(
<wgrant> But thanks.
<jtv> StevenK: I need some help with Q/A for generating the DSDJs.
<jtv> I need to turn soyuz.derived_series_jobs.enabled on, which I'm not privileged to do, and then I need to make sure that everything that will create SPPHs still works which I don't know how to do.
<StevenK> Turn it on *where*
<lifeless> poolie: mostly fixed
<jtv> StevenK: on qastaging, I guessâdepends on where we can do whatever generates SPPHs.
<StevenK> gina can't run on qastaging, you can upload and copy, though
<jtv> Does that mean one place will get exercised but another won't?
<StevenK> Obviously.
<StevenK> gina can run on DF, but carefully
<jtv> So if I can get someone to enable the flag on qastaging, I can test uploading to PPAs and copying between PPAs there?
<StevenK> Indeed
<jtv> lifeless: could you enable soyuz.derived_series_jobs.enabled for me on qastaging?
<lifeless> jtv: I don't have the admin bit
<lifeless> jtv: stub does; stub^
<stub> How do I do that? Is this a feature flag or something?
<lifeless> visit qastaging.launchpad.net/+feature-rules
<wgrant> It's a boolean feature flag. Set it default true.
<lifeless> add a line like
<lifeless> soyuz.derived_series_jobs.enabled default 0 on
<lifeless> (flag, scope, sequence, value)
<lifeless> poolie: all sorted now AFAWCT
<wgrant> It's still not great, but it's <2s now.
<poolie> super
<stub> That ff is on now.
<stub> On qastaging
<stub> jtv: ^^^
<jtv> thanks stub
<jtv> StevenK: how do I make dput upload to qastaging?
<StevenK> Ugh, I can never remember.
<wgrant> Erm.
<wgrant> Does asuka run poppy?
<wgrant> I didn't think it did.
<wgrant> You can do recipes and copies there, but not uploads AFAIK.
<StevenK> Doesn't look like it does.
<jtv> So I can't test uploads?
<wgrant> Only on mawson.
<jtv> So I can't.
<wgrant> Not without our assistance.
<jtv> Then could you help out with this?
<wgrant> Yes.
<jtv> Thanks.  We'll need to enable the feature flag first.
<wgrant> No, first we need to update the codebase :(
<wgrant> Will take a few minutes.
<wgrant> jtv: Your change is in db-devel?
<jtv> wgrant: I believe so, but let me check
<jtv> wgrant: it is, as of r10296
<wgrant> Great, thanks.
<jtv> The feature flag is soyuz.derived_series_jobs.enabled
<jtv> (Note underscores; the documentation says whosoever useth dashes in feature-flag names shall suffer eternal, er, suffering
<wgrant> Yes.
<wgrant> I think we all agreed on that.
<wgrant> Still waiting for mawson to update.
<jtv> Meanwhile I'm waiting for my copied package to be published on qastagingâ¦ how long should that take?
<wgrant> Eternity.
<wgrant> The publisher does not run.
<wgrant> Why do you need a publisher?
<jtv> I need to test whether I broke SPPH generation or not.
<wgrant> SPPHs are done in the webapp transaction.
<wgrant> You can check through the API, or even the web UI.
<jtv> How do I do that?
<wgrant> https://qastaging.launchpad.net/api/devel/~PERSON/+archive/PPA?ws.op=getPublishedSources&status=Pending
<wgrant> That should do it.
<jtv> 404
<StevenK> Note the PERSON
<jtv> Ah
<StevenK> jtv: Why did you create a new interfaces file for DSDJs when the two current DistributionJobs use interfaces/distributionjob?
<StevenK> jtv: And it's bugging me, so can I move it?
<jtv> StevenK: you can if you like, but I wanted to avoid the mindless piling-on that has plagued parts of our codebase for so long.
<StevenK> They're tiny interfaces, so meh
<jtv> wgrant: I suppose I need to replace the PPA with something as well?
<wgrant> jtv: Yes.
<jtv> But with what?
<lifeless> only if you want it to work
<wgrant> Whichever PPA you copied it into...
<jtv> I copied into ~jtv/+archive/ppa/â¦ nothing in there strikes me as a PPA name.
<wgrant> The format of the URL might give you a hint :)
<jtv> What, the name is https?
<wgrant> s@~PERSON/+archive/PPA@~jtv/+archive/ppa@
<jtv> Ah
<jtv> So I'm back where I started.
<wgrant> jtv: https://dogfood.launchpad.net/+feature-changelog
<jtv> Yes, that would be it.
<jtv> As for the pending packages, that URL just ended up getting me to the page I was already looking at.  I don't get the impression that the status=Pending is doing much; packages that I thought were published years ago are still in there.
<wgrant> "total_size": 1
<jtv> The word "total" doesn't appear on the page
<wgrant> It's not a page.
<wgrant> It's a JSON document...
<jtv> I think I got redirected at some point.
<jtv> Getting JSON now.
<jtv> Soâ¦ where does the SPPH get created?
<wgrant> Code-wise?
<jtv> Interaction-wise.  What would fail if that went wrong?
<lifeless> thumper: leonard might be the best possible person to look at https://bugs.launchpad.net/launchpad/+bug/735202
<wgrant> Your +copy-packages POST would OOPS, or it wouldn't show up with status=Pending.
<jtv> Ah, ok, so it was fine all this time!  Thanks.
<wgrant> Now, you wanted to test an upload on DF?
<jtv> Sure
<wgrant> FTP to upload.dogfood.launchpad.net
<jtv> OK, I logged in anonymously.
<wgrant> You should be using dput to do that for you.
<jtv> Ah, so reconfigure dput, then dput, then configure back?
<wgrant> Right. Add a stanza to ~/.dput.cf, modelled on the existing ppa one in /etc/dput.cf.
<jtv> Except fqdn = dogfood.launchpad.net?
<wgrant> Right.
<jtv> And then make dput use that stanza somehow.
<wgrant> dput dfppa:jtv/ppa blah_source.changes.
<jtv> And I guess the dfppa is the name of the new stanza?
<jtv> It's uploading, so I guess that's good.
<jtv> â¦and it's uploaded.
<jtv> I'm not seeing it show up on my dogfood PPA page though.
<StevenK> It needs to be processed
<StevenK> Which needs to be run by hand
<wgrant> I'm just clearing out incoming/
<jtv> So any moment now?
<wgrant> Processing.
<wgrant> 2011-03-15 06:31:19 DEBUG   Rejected:
<wgrant> 2011-03-15 06:31:19 DEBUG   Unable to find distroseries: unstable
<jtv> Grrr
<jtv> Uploading again with that changed to "natty."
<jtv> wgrant: could you run it again?
<wgrant> 2011-03-15 06:34:40 DEBUG   Rejected:
<wgrant> 2011-03-15 06:34:40 DEBUG   The signer of this package has no upload rights to this distribution's primary archive.  Did you mean to upload to a PPA?
<wgrant> Your upload path was /ubuntu.
<jtv> How did that get set?
<jtv> I used "dogfood:jtv/ppa"
<wgrant> Sounds like your path in dput.cf is bad.
<wgrant> It should have %(dogfood)s in it
<StevenK> Actually, %(section name)
<wgrant> Right.
<jtv> I don't see any path in there
<jtv> Oh, "incoming"?
<jtv> It currently says "ubuntu/"
<jtv> What should that be?
<wgrant> Copy it from the ppa section, except s/ppa/dogfood/
<jtv> I don't have a ppa section, only an lpdev section.
<wgrant> Even in /etc/dput.cf?
<jtv> There is one there.  ~%(ppa)s/ubuntu
<jtv> So I'll use that then.
<jtv> Grr packag has already been uploaded to dogfoodâ¦ bumping version number
<wgrant> No.
<wgrant> Just dput -f
<jtv> Too late
<jtv> Already uploaded the new one.
<wgrant> jtv: Looks like I need to run security.py
<jtv> ahhh yes
<wgrant> 2011-03-15 06:55:08 DEBUG   Creating PENDING publishing record.
<wgrant> Still going,..
<wgrant> This bit often takes a couple of minutes on mawson; NFI why.
<poolie> https://bugs.launchpad.net/launchpad/+bug/735290 ouch
<_mup_> Bug #735290: changing Project drop down in project group +filebug loses all your work <ajax> <bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/735290 >
<wgrant> jtv: Accepted.
<wgrant> jtv: You can check for a publication now.
<jtv> wgrant: thanks!
<wgrant> The publisher should be running now, too.
<jtv> wgrant: the package's on the PPA listing
<wgrant> Indeed.
<wgrant> Shall I now check for jobs?
<jtv> If that means an SPPH was just successfully created, then that's done.
<jtv> Oh yes please, that too :)
<wgrant> Which table?
<StevenK> DistributionJob
<jtv> DistributionJob
<wgrant> Ah.
<jtv> Look for json metadata consisting of {sourcepackagename: <id>}
<wgrant> Nothing there.
<wgrant> Only some package copy things.
<wgrant> But I guess that's not surprising.
<wgrant> As these are not derived series uploads.
<jtv> Ahh of course
<jtv> So we ought to try that as well.
<StevenK> maverick is a derived series
<StevenK> On DF
<StevenK> So re-upload to that
<jtv> From another distro?
<wgrant> From sid, IIRC.
<StevenK> Right
<jtv> great
<wgrant> I changed that in December.
<wgrant> jtv: I'll give you upload privs. Upload again to /ubuntu
<jtv> Thanks.  Uploading already.
<jtv> (With maverick as the series)
<wgrant> Great.
<wgrant> Done?
<wgrant> Seems to be.
<jtv> Yup
<wgrant> /srv/launchpad.net/ubuntu-queue/incoming/upload-ftp-20110315-071355-000009/~jtv/ppa/ubuntu/libpqxx_3.2-1ppa4.dsc
<wgrant> Wrong path.
<wgrant> I'll move it.
<jtv> Thanks.  I have no idea what was wrong about that, or indeed what might have been right
<wgrant> ~jtv/ppa/ubuntu is your PPA, not the primary archive.
<jtv> Ah, of course, we're not uploading to the ppa any more
<jtv> Or at least, trying not to be
<wgrant> Heh.
<jtv> wgrant: and then I suppose it'd be also worth checking that we can copy a package into maverick.
<wgrant> jtv: Indeed. I'll do that in a sec.
<jtv> Thanks.
<wgrant> :(
<wgrant> JS hacks upon JS hacks.
<jtv> A hex upon JS hacks.
<lifeless> https://bugs.launchpad.net/launchpad/+bug/734751 made me sad
<_mup_> Bug #734751: BranchSet:CollectionResource:#branches timeouts <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/734751 >
<wgrant> lifeless: Oh?
<wgrant> StevenK: Looks like someone fixed Maverick to derive from Lucid again.
<StevenK> Rargh
<wgrant> Actually, it might help if I look at DF.
<StevenK> Haha
<wgrant> Where it does derive from Sid.
<jtv> Websquinting.  The latest meme.
<lifeless> wgrant: incomplete fix :<
<wgrant> lifeless: :(
<wgrant> jtv: So, I can see some bugs in this, but I can't see why it's not working.
<jtv> wgrant: what is it you're seeing?
<wgrant> jtv: Even your upload did not create a DSDJ.
<jtv> wgrant: is it possible to run "make harness" against the dogfood db?
<wgrant> jtv: I am doing that right now.
<wgrant> I wonder if the flag controller isn't set up properly in scripts.
<lifeless> its not setup by scripts
<jkakar> poolie: You rock! :)
<lifeless> scripts need to setup an appropriate controller as part of setting up whatever work context they have
<jtv> lifeless: does that mean that feature-flag checks will fail silently in scripts?
<lifeless> they won't fail
<lifeless> they will find no flags set
<poolie> thanks jkakar, so do you :)
<jkakar> :)
<poolie> i'm just resolving conflicts in and manually testing the other branch
<jkakar> poolie: Thanks.  I probably should have based that branch on the optional-needs-testing one, but I didn't think about it until too late.
<StevenK> wgrant: Over the last 3 runs: 1054, 1061, 1120.
<poolie> np
<poolie> i can refactor the template slightly
<jkakar> Cool.
<LPCIBot> Project windmill build #50: STILL FAILING in 1 hr 6 min: https://hudson.wedontsleep.org/job/windmill/50/
<jtv> lifeless: to me that means that the feature-flag check will fail silently.
<jtv> Which sucks.
<wgrant> StevenK: This is still in 15 minutes?
<StevenK> wgrant: Yes
<wgrant> StevenK: So it's more like 20 days?
<StevenK> I have no idea
<poolie> jtv it seems reasonable to me that the default feature flag controller fails (or doesn't exist) rather than doing nothing
<StevenK> I was going to get a better idea after 3-5 days of it running
<jtv> poolie: yes, I'd expect it to fail in the conventional sense of the word, not just return a value that's basically arbitrary.
<jtv> Soâ¦ how can we fix this and give the scripts in question access to the feature flags?
<poolie> well, i guess, copy across or call the setup code
<poolie> something similar to what's done in features/webapp.py
 * jtv looks
<jtv> wgrant: in any case, for now this bug is qa-untestable.  Thanks for all your helpâI'm afraid we'll have to do it again later.
<jtv> wgrant: by the way, which script(s) would be affected exactly?  gina soyuz_process_upload?
<poolie> jkakar, do you think i can easily write a test for the handling of an 'access denied' error from lp?
<StevenK> If it raises Unauthorized you can test for that.
<wgrant> jtv: process-upload.py, gina.py, copy-package.py, at least.
<jtv> whew
<jtv> thanks
<poolie> StevenK, my question was more whether there is enough mock-type framework in the client to raise Unauthorized at the right point
<lifeless> jtv: you'll want a variant on the controller which supplies the script id as the pageid, or something along those lines
<jtv> lifeless: good point
<lifeless> I think the pageid scope reads from the wsgi stack
<lifeless> so, there is some plumbing changes needed
<jtv> :/
<wgrant> lifeless: Why expose it as a pageid? Why not a separate namespace/
<jtv> Well it needs a pageid anyway, doesn't it?
<wgrant> Why?
<wgrant> script != page
<wgrant> => no pageid
<lifeless> wgrant: could do that, though 'primary key for oops and other stuff' is good enough for me
<lifeless> wgrant: I don't really care about the label; having the pageid scope present in the lookups in a script would be a bug (and make that scope more complex) and vice verca.
<lifeless> pageid wsgi  based scope implementation, I mean.
<jtv> This does not sound like something I ought to be resolving reactively in unrelated feature development.
<lifeless> jtv: well, feature development shouldn't go down unnecessary diversions; this is probably 1/2 day to a day total, which is about break even vs having sysadmins enable/disable cron jobs during early deployment lessons
<lifeless> jtv: so if I was on a feature squad, I'd consider it in scope to incrementally improve infrastructure to do the feature needed
<jtv> It doesn't sound like something I could do in half a day.
<jtv> But we'll see.
<poolie> StevenK, fyi lazr.restfulclient apparently does not raise a separate subclass for unauthorized errors
<wgrant> Argh.
<wgrant> Private bugs make me sad.
<wgrant> They make this NullBugTask removal a little difficult, because NBT is used to conceal the target.
<wgrant> Although it doesn't do a very good job, so maybe we don't care.
<lifeless> wgrant: thats probably the cause of poolies api disclosure bug
<wgrant> lifeless: It is rather related, yes.
<wgrant> I am trying to think of a sensible way to solve it.
<lifeless> raise GoAwayDeniedHaHaHa
<wgrant> lifeless: Do you know the issue?
<wgrant> lifeless: We want to redirect users to the task they requested once they've logged in.
<poolie> i just found another api privacy bug, if that makes you feel any better :)
<wgrant> So we allow traversal to the BugTask.
<lifeless> wgrant: possibly; I suggest you explain it for clarity.
<wgrant> lifeless: This may involve returning a NullBugTask.
<wgrant> Regardless, an unauthenticated user will not hold launchpad.View, so will be redirected to a login page.
<wgrant> Even if the object doesn't *really* exist.
<lifeless> wgrant: why? I mean: we refuse to acknowledge or deny the presence of private content
<lifeless> wgrant: so just a not found error at the original url + a login button should be all we need to send.
<lifeless> wgrant: for /all/ view-denied cases
<wgrant> lifeless: Right, that's what I was hoping you'd agree with.
<adeuring> good morning
<wgrant> Since it's now consistent with what we do with projects.
<wgrant> lifeless: I will redirect all access-denied cases to /bugs/1, which will ask for login and then redirect to the default task.
<_mup_> Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Invalid by compscibuntu-bugs> <EasyPeasy Overview:Invalid by ramvi> <GNOME Screensaver:Won't Fix> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <The Linux OS Project:In Progress> <metacity:In Progress> <OpenOffice:In Progress by lh-maviya> <Tabuntu:Invalid by tinarussell> <Tivion:Invalid by shakaran> <Tv-Player:New> <Ubuntu:I
<wgrant> It means people will lose the task context if they're not logged in, but that only matters for nomination these days.
<wgrant> So, stuff them.
<poolie> https://bugs.launchpad.net/launchpad/+bug/735346
<_mup_> Bug #735346: can't read a collection when one member is private <api> <bugs> <disclosure> <security> <Launchpad itself:New> <lazr.restfulclient:New> < https://launchpad.net/bugs/735346 >
<lifeless> wgrant: *blink*
<poolie> :)
<wgrant> lifeless: Oh?
<lifeless> poolie: thats a simple case by case bug
<poolie> specific to linked_branches?
<wgrant> poolie: That's very interesting.
<poolie> thanks
<wgrant> poolie: lazr.restful checks for launchpad.View on any object it returns rom from a collection.
<wgrant> poolie: So that should work fine.
<wgrant> lifeless: Why are you blinking?
<poolie> lifeless, so i should say "linked_branches" in particular?
<lifeless> yes, I've changed it
<poolie> thanks
<wgrant> poolie: That reveals yet another bug.
<lifeless> wgrant: I thought what we did was issue a 404 *with no redirect*
<lifeless> wgrant: so I'm surprised you're suggesting a redirect
<henninge> rvba: ping ;)
<wgrant> poolie: the __repr__ violates the security proxy and displays the unique_name anyway.
<wgrant> lifeless: Eh, I guess.
<rvba> henninge: yes
<poolie> heh
<wgrant> lifeless: I should probably be consistent, then, and deny traversal through /bugs too.
<henninge> rvba: are you having trouble landing your branch? ;)
<poolie> indeed, i was just going to say that too
<wgrant> Since this seems to be the way we're going now.
<wgrant> It will confuse people. But privacy is confusing.
<lifeless> wgrant: so I think I'm saying:
<lifeless>  - the users browser should show the url they put in
<lifeless>  - if they are logged in its a 404
<lifeless>  - if they are logged out its a 404
<lifeless> [and logging in will let them see it if they have permission]
<rvba> henninge: I was blocked by some failure (Steven called the failure furphy)... but I'm landing it right now and it seems to go fine.
<lifeless> wgrant: unless we introduce 'can know, but can't see' vs 'cannot know about if cannot see' - which traversal permission is about
<henninge> rvba: cool
<lifeless> wgrant: for bugs i don't think we need traversal separate from view
<wgrant> lifeless: What do you want to do about /bugs/1234, where 1234 is an invisible private bug?
<_mup_> Bug #1234: Gina is an unmaintainable mess of command line options, environment variables and shell scripts <lp-foundations> <Launchpad itself:Fix Released by debonzi> < https://launchpad.net/bugs/1234 >
<lifeless> wgrant: 404
<wgrant> lifeless: We don't, no.
<lifeless> wgrant: no redirect
<wgrant> lifeless: Right, OK.
<wgrant> Thanks.
<lifeless> wgrant: until we permit bug deletion, it will permit trivial probing to find private bugs, but it won't disclose their context
<wgrant> lifeless: I imagine we probably have some transaction failures that make that false, but OK.
<henninge> rvba: a branch like that does not have to go through ec2 testing AFAIUI.
<wgrant> (incrementing the sequence but then aborting)
<henninge> rvba: since you only changed test code it should be fine to just make sure those are still passing and then you can do bzr lp-land.
<henninge> rvba: just thought I'd mention that.
<lifeless> wgrant: right
<rvba> henninge: does this mean that I should use special options to land it?
<lifeless> wgrant: its a fault in using monotonic sequences as ids, not in the privacy story
<wgrant> lifeless: Yup.
<henninge> rvba: what I do: Run the tests locally, then do "bzr lp-land" to submit it to pqm.
<henninge> rvba: no need for "ec2 test" or "ec2 land".
<rvba> henninge: ok
<henninge> It's just a lot quicker ...
<rvba> henninge: I did "ec2 land" yesterday (this failed, well the PQM failed, not the tests)  but got advised to use lp-land this morning by Steven
<henninge> rvba: yeah, that's another one.
<henninge> rvb: if the branch already passed ec2 and just bounced off pqm, there is no need to run it through ec2 again.
<rvba> henninge: indeed
<poolie> lifeless, you asked the other day if my kanban script had got any faster because of lp's changes
<henninge> rvba: also, I was surprised to find the merge proposal still in "Needs review" because AFAIK ec2 land and bzr lp-land won't land branches when the mp is not "Approved"
<poolie> it's gone from 2:54 against old lpnet and 2:14 against old edge, down to 1:44
<poolie> which is pretty good
<henninge> rvba: you can set an MP to "Approved" yourself once you received all the necessary reviews.
<poolie> _however_, the data its reading has also changed, and the timing will be data-dependent
<poolie> so it's not a great benchmark
<rvba> henninge: ok ... ec2 land --force can for a land AFAIK
<poolie> but, better than getting slower
<rvba> s/can for/can force/
<henninge> yes ,)
<henninge> ;)
<henninge> but no need for that, that's what I mean.
<rvba> henninge: ok, got it
<StevenK> rvba: I have never used ec2 land --force
<StevenK> So I'd strongly suggest you don't either
<rvba> StevenK: sound pretty much the same to use --force or to Approve your own MP ... but I did it because henninge approved my MP and I had to commit a few cosmetic changes after that
<rvba> StevenK: still, I get your point :-)
<StevenK> rvba: You are allowed to Approve your own MP
<rvba> StevenK: yes, much cleaner solution indeed
<henninge> StevenK, rvba: Yeah, we once argued that setting it to "Approved" is to indicate that all reviews (code, ui, release-critical) have been received.
<henninge> AFAIK tarmac will automatically land branches that are set to Approved.
<henninge> but we are not yet usging that.
<henninge> using, evin
<henninge> even, even
<henninge> ;)
<StevenK> We should be!
<StevenK> And Jenkins!
<StevenK>  /rage, /bitch, /complain
<wgrant> Pagetest stories are bad.
<bigjools> page stories are good, if they are actually stories
<wgrant> bigjools: Not when they span 10 files.
<bigjools> that's not a story, that's a tome
<wgrant> Point.
<StevenK> "Page War & Peace"
<jkakar> poolie: You could use mocker to write such a test, but it might be a bit tricky.
<jtv> lifeless, if you're still here (which you probably shouldn't be :) would it make sense to have scopes like "script:gina"?
<jtv> Or should we encode script names in the flag names where appropriate?
<jtv> For example, we could have a flag "enable" in scope script:gina, and similar for all other scripts if we wanted.
* gmb changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | firefighting: - | On call reviewer: gmb | https://code.launchpad.net/launchpad-project/+activereviews/
<lifeless> jtv: both
<jtv> both what?
<jtv> Oh,
<jtv> yes, I see
<lifeless> example from the web ui
<lifeless> when we want to affect only one / some pages, even for common code, we use the pageid scope
<poolie> jkakar, for now i just interactively tested it
<poolie> mocker could be ok
<jkakar> poolie: Cool.
<lifeless> when we want to probe for something that its only relevant to a particular domain, we use the domain inthe flag name
<lifeless> an example of the former is 'memcache pageid:BugTask:+index default '
<lifeless> an example of the latter is 'recipes.beta default 1 on'
<jtv> lifeless: right ho, get it.  A feature flag for disabling scripts sounds useful, no?  It suddenly looks pretty easy.
<lifeless> jtv: exactly
<jtv> (I wonder why lp.services.features.webapp doesn't export anything in __all__ btw)
<lifeless> __all__ only matters if one is doing import *, or worried that pydoc will show too much
<lifeless> we should probably delete most of our __all__'s now that the import fascist is years out of date
<StevenK> So we should kill the import fascist?
<lifeless> or fix it
<lifeless> its not guarding much in the current layout, because all the layers are mixed up
<jtv> Maybe we should depose the import fascist and have a customs officer instead.
<LPCIBot> Project windmill build #51: STILL FAILING in 1 hr 9 min: https://hudson.wedontsleep.org/job/windmill/51/
<stub> The goal with __all__ is to document public api from internal api.
<stub> lifeless: I know I use it that way - if it is not in __all__, it is only supposed to be used by that package and its tests.
<lifeless> stub: #ubuntu-meeting is looking for you
<stub> We can already disable cronscripts using cronscripts.ini btw.
<stub> If feature flags lived in a high available database, rather than the main db, we could use that for cronscripts.ini.
<lifeless> tomorrow I should get my hbase vs cassandra mail rolling
<lifeless> and I need to look into mongodb
<stub> I suspect zookeeper might be suitable too... can't recall if you found a flaw in that?
<lifeless> need to look more closely at it; its in an awkward spot AIUI
<lifeless> theres some mutterings (long term) about being able to handle london being down
<stub> IIRC zookeeper seemed pretty much designed for our feature flags stuff, but would probably be way overkill if that is all we used it for.
<stub> (same with moving it out of PG - I'd rather have a little cruft like cronscripts.ini and feature flags duplicating work than over engineering a high availability database just for FF)
<lifeless> hmm, why isn't rev 12595 on qas
<lifeless> stub: me too
<deryck> Morning, all.
<lifeless> night all
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project windmill build #52: FIXED in 43 min: https://hudson.wedontsleep.org/job/windmill/52/
<wgrant> gmb: Thanks.
<wgrant> Almost there.
<gmb> np
* leonardr changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | firefighting: - | On call reviewer: gmb,leonardr | https://code.launchpad.net/launchpad-project/+activereviews/
<henninge> gmb, leonardr: Hi! Can I ask a review of either of you, please? It's a UI-related branch that adds a new page.
<leonardr> i think i should defer to gmb on that kind of branch, but i'll take a look if he's not around
<LPCIBot> Project db-devel build #452: FAILURE in 4 hr 40 min: https://hudson.wedontsleep.org/job/db-devel/452/
<henninge> leonardr: that would be nice but it's ok if you turn it down.
<henninge> https://code.launchpad.net/~launchpad/launchpad/translation-sharing-status/+merge/53419
<leonardr> henninge: i'll do what i can and note inthe review if i think it needs more eyes
<leonardr> benji, are you around? i'd like you to look at https://code.launchpad.net/~leonardr/lazr.restful/handle-view-that-provides-no-status/+merge/53342 and tell me if it makes sense from a zope perspective
<benji> leonardr: sure
<benji> leonardr: getting a view is adapting an object and the request to Interface, so instead of getting a view of the execption and then checking that it is an insteance of WebServiceExceptionView you could instead...
<benji> ...define, say, IWebServiceExceptionView and adapt the object and request to IWebServiceExceptionView
<leonardr> benji: i thought about using interfaces. i rejected the idea because existing view classes don't implement any particular interface. but then i made the assumption that they would all be WebServiceExceptionView
<leonardr> so yeah, i could define IWSEV, have it define .status, and make WSEV implement it
<benji> you could register an adapter from (WebServiceExceptionView, request) to IWebServiceExceptionView which simply returns the WebServiceExceptionView instance
<leonardr> benji: is there any advantage of that over having WSEV implement IWSEV?
<benji> leonardr: it depends on how you structure the code, if you adapt from (e, request) to Interface and then check if the result provides IWSEV, then it's esentially the same thing you have now (i.e., not evil, but doing things indirectly)
<benji> but just making WSEV implement IWSEV will not make adapting (e, request) to IWSEV work because the CA doesn't have an adapter registered for that signature
<leonardr> benji: i see. i may change the registrations then. after i do this review i'll see what works
<benji> k
<leonardr> henninge: what exactly is a translation template? is it like a list of strings to be translated?
<henninge> leonardr: exactly
<leonardr> henninge: in template_info, i don't understand why you can assume that self.context has a productseries (line 221 of the diff)
<leonardr> it has an upstream, but is an upstream always the kind of thing that has a productseries?
<henninge> leonardr: this view is only for sourcepackages, so the upstream is always a productseries.
<henninge> leonardr: self.is_configuration_complete includes the check if the productseries exists.
<henninge> leonardr: sorry, was distracted
<leonardr> henninge: np
<henninge> leonardr: actually, I am about to have lunch
<leonardr> henninge: ok, if i have more questions i'll just put them in this channel
<henninge> leonardr: cool, thanks
<leonardr> you have some copy-and-paste comments on line 398 and line 405
<leonardr> and 413
<leonardr> on line 543, the docstring reads like a commit message. you should say what you're actually setting up
<leonardr> you could also refactor the many times you call getViewBrowser, but that's not a big deal
<maxb> Is there any particular reason lazr-js's embedded fulljslint.js is such an old version?
<jml> deryck: your blog forbids me commenting
<deryck> really?
<jml> deryck: so I'll just heckle here
<jml> deryck: "Is the solution to move to Selenium and anything other than Zope test runner? :P"
<deryck> jml: how does it forbid?
<jml> deryck: Forbidden (403)
<jml> CSRF verification failed. Request aborted.
<jml> More information is available with DEBUG=True.
<deryck> ah, crap
<deryck> stupid Django.
<rvba> leonardr: Hi, I just saw you mentioned refactoring tests with getViewBrowser ... it happens that I'm having trouble with a test that uses this
<leonardr> rvba: ok, i don't know anything about this but i'll do my best
<deryck> jml: as to your question, yes I think that is the best solution.  Something of that sort. Either Selenium or Windmill without the Zope wrapping.
<deryck> jml: Though I think Windmill has some general fragility, just because there are fewer people using it and working on it
<leonardr> henninge: line 753, "Translations are enable" -> "are enabled"
<jml> deryck: yeah. that's what I hear.
<rvba> leonardr: it looks to my -newbie- eyes like, when getViewBrowser is used, something might go wrong with the authentication machinery
<rvba> leonardr: but if you're not sure, don't worry, I'm in the process of figuring it out ...
<leonardr> rvba: ok. looking at the code, it looks like getUserBrowser logs in as anonymous and never logs out. are you getting an 'interaction already active' type error?
<rvba> no, the test uses  getUserBrowser with a user, then check a perm ... when I debug the whole thing, I see that the perm is checked against anon
<rvba> leonardr: if you says it might never log out after login in as anon, looks like this might be related
<leonardr> rvba: paste your test to me?
<rvba> leonardr: just sent you an email with the description of the problem
<leonardr> ok
<deryck> jml: comments fixed.  Thanks for the heads up.
<jml> deryck: np.
<leonardr> rvba: i don't know if this will help, but: there are two different levels of authentication in a unit test
<rvba> leonardr: yes?
<deryck> henninge-lunch: can you reply to the email sinzui sent? I suspect your domain expertise can answer this much more quickly than mine.
<leonardr> there's the level at which you're authenticated with the launchpad layer
<leonardr> that's what login(ANONYMOUS) takes care of
<leonardr> that's necessary when you need to access the database or call launchpad utility functions
<henninge> deryck: I can do that.
<leonardr> and then when you get a browser, there's the user whose password gets sent in the Authorization header
<deryck> henninge: many thanks!
<leonardr> if your test checks a permission, that code runs in the launchpad layer
<leonardr> and it will be checked against anonymous
<leonardr> (since getViewBrowser logs in as anonymous and doesn't log out)
<leonardr> if you use the browser to make a request, that code runs as the user whose browser it is
<leonardr> does that make anything you've seen make more sense?
<rvba> leonardr: yes, thanks for the clarification ... not sure this is related to my problem because, like you saw in my revision, I fixed it by changing the owner of the related distroseries
<sinzui> Why do we need getViewBrowser? what does it do that we cannot do using the standard helpers like create_view
<leonardr> sinzui: i dunno. it gives you a browser object, so you can navigate from page to page
<jml> off to cafe for lunch, will work from there.
<sinzui> leonardr: I think navigating from page to page is excellent for user stories. If I want to test a view or a page I think I should with with the actual objects.
<leonardr> sinzui: that sounds fine to me
<leonardr> rvba: as sinzui implies, you might have better luck using create_view instead of getUserBrowser. that would keep all the code in the same level--the launchpad level
<leonardr> something to consider
<leonardr> sinzui: i got a bunch more test failures when i ran our branch through ec2. i haven't had time to look at them yet, but the first one i saw was just am issing import, so here's hoping...
<rvba> leonardr: yes
<deryck> adeuring: ping for standup
<adeuring> whoops, coming...
<sinzui> henninge: I wrongly set the status of your MP to needs fixing. I only had a question
<jcsackett> does profiling still show the query if it's hitting the storm cache instead of the database?
<henninge> sinzui: I'll look at it in a minute
<leonardr> benji: i've got your suggestion working. i'm a little taken aback by the fact that queryAdapter(view, IWebServiceExceptionView) returns None if view is already an IWebServiceExceptionView but there is no no-op adapter registered for it
<leonardr> is there any way around that?
<benji> leonardr: that surprises me; although, I'm also surprised that you need to do that.  Looking at the MP again.
<leonardr> benji: let me push my changes (with the no-op adapter) and maybe you can help me improve it
<benji> sure
<leonardr> pushed
<gmb> leonardr: Can you take a look at https://code.launchpad.net/~gmb/launchpad/mute-button-cleanup-bug-204980/+merge/53401 for me?
<benji> leonardr: instead of view = getMultiAdapter((e, self.request), name="index.html") I expected view = getMultiAdapter((e, self.request), IWSEV); you want a view that provides a particular interface (i.e., it has a status attribute) and there's no need for it to have a particular name
<leonardr> gmb, sure
<gmb> Thanks
<benji> leonardr: wait, I think I'm looking in the wrong place, I'm looking at https://code.launchpad.net/~leonardr/lazr.restful/consistent-error-exposing/+merge/51946 which isn't right
<leonardr> benji: yeah, look at lp:~leonardr/lazr.restful/handle-view-that-provides-no-status
<benji> thanks
<leonardr> sinzui: i'm pretty sure that missing import was the only thing keeping us from finally landing that branch. i've fixed it and am running through ec2 land again
<sinzui> \o/
<leonardr> gmb: describe or give me a link to the bug mail mute button story, so i know what i'm looking at?
<gmb> leonardr: The word "Story" is probably misleading, but to sum it up:
<gmb> As a Launchpad user who receives a lot of bug mail
<gmb> I want to be able to click a "Mute bug mail" button for a given bug
<gmb> So that I no longer receive any email about that bug
<gmb> (This is distinct from unsubscribing, since even if you unsubscribe from a bug it's possible to receive mail about it, for example if you own a project that has no bug supervisor)
<gmb> Muting the bug makes Launchpad drop *any* email it's going to send to you about it.
<leonardr> i see. it's an anti-subscription
<gmb> Yes :)
<leonardr> ok
<leonardr> gmb: in bugtask_index_portlets, it looks like you took a code path for 'is not subscribed' and just changed it to 'is muted'. what happens if you're not subscribed?
<gmb> leonardr: That was because I've added the new CSS class to explictly describe someone as muted. Previously it relied on the subscribed-foo CSS class since muted subscriptions are just another type of subscription.
<gmb> If you're not subscribed you can still click the mute link and it will work perfectly.
<leonardr> ok, so it was code to control the mute button
<gmb> What it does behind the scenes is subscribe you to the bug with a BugNotificationLevel of NOTHING.
<gmb> Right.
<gmb> (Note that gary_poster and I are of the opinion that this way of doing mutes is a bit of a kludge, but I think it's a decent first step).
 * gary_poster nods
<leonardr> gmb: on line 140 and line 143, what is the 'true' vs. 'false' passed in to set_subscription_link_parent_class?
<gmb> function set_subscription_link_parent_class(user_link, subscribed, dupe_subscribed)
<gmb> is the signature.
<leonardr> ok, i'm not sure what it means to have duplicate descriptions. you have a regular one, plus a 'mute' one?
<gmb> leonardr: dupe_subscribed means "Subscribed to a duplicate of this bug"
<benji> leonardr: I checked out the branch and got the appropriate response to queryAdapter(view, IWebServiceExceptionView) when view was an instance of WebServiceExceptionView; I'm working up a small change to your branch to suggest to you.
<gmb> There Can Be Only One (Person, Bug) subscription.
<leonardr> ah, it's a subscription _to_ a duplicate
<gmb> right
<leonardr> ok, you're just setting up the appropriate presentation when subscription is un-muted
<gmb> yes
<leonardr> ok, looks good
<gmb> Cool
<benji> leonardr: how about this: http://pastebin.ubuntu.com/580611/
<leonardr> benji: that runs into the same problem i encountered when i tried something similar, which is that all other web service error view registrations need to provide IWebServiceExceptionView, not Interface
<benji> leonardr: do the other web service error views have the status attribute?
<leonardr> yeah, they all subclass WebServiceExceptionView
<benji> and are they subsclasses of WSEV?
<leonardr> basically i'm afraid that something that used to work like other view registrations no longer works that way
<leonardr> that this will confuse people and possibly cause more failures on the launchpad side
<leonardr> my fears are fuelled by the fact that i can't get all the tests to pass with a system like this
<benji> leonardr: yeah, that is an issue... so in your estimation it is ok to require that the error views be subclasses of WSEV but the existing registration of adapters to the views (i.e., (e, request), Interface, name='index.html) should still work; right?
<leonardr> benji: yeah, i think that's fine. or, we can require that the error views implement IWSEV, since all that matters is that they have a .status
<benji> ok, so if we don't want to change the signature that the adapters have we're stuck with the same adaptation as on the trunk, so that part is a constant
<leonardr> that's the adaptation from WSEV to IWSEV?
<benji> leonardr: I think the code as you have it is the best route.
<benji> leonardr: no, the adaptation to (e, request), Interface, name='index.html to get the view; that part we can't change
<leonardr> ok, let me see how the code is now to remember...
<leonardr> benji: and there's no way to get rid of the no-op adapter?
<leonardr> (without going down one of the paths we've rejected)
<benji> leonardr: you probably can; at least for adaptation like IFoo(obj) where obj already provides IFoo there is no need for anadapter and obj is just returned stright away, I don't recall if the same is done for queryAdapter; worst-case you can put try: IWSEV(view) except: raise e (where e is the original exception)
<leonardr> trying that now
 * jml fixes the conflict
<jml> deryck[lunch]: I guess bug 607438 should be critical these days, since it's an OOPS.
<_mup_> Bug #607438: NoCanonicalUrl: No url for message because message broke the chain. <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/607438 >
<leonardr> benji: take a look at the latest revision
<benji> k
 * jml off home
<bigjools> I am calling "ec2 test" on a db-devel branch but it's getting merged into devel, how can I fix that?
<bigjools> there's a conflict which is not in db-devel yet, or I'd not worry about it
<benji> leonardr: looks good
<leonardr> queryAdapter doesn't work, unfortunately
<benji> that makes some sense though, since there isn't an adapter, but IWSEV(WSEV) first checks to see if the interface is already provided and if not invokes adaptation
<benji> a little philosophysing: in an ideal world we would be adapting (e, request) to IWSEV (without a name) and use that result since that adaptation better captures the intent; that being said, this approach is eminently reasonable given the constraints of the situation
<leonardr> benji, can i get a formal review from you so i can land the branch?
<benji> leonardr: sure; which MP?
<leonardr> https://code.launchpad.net/~leonardr/lazr.restful/handle-view-that-provides-no-status/+merge/53342
<deryck> jml: agreed.  I raised it.
<benji> leonardr: MP approved; I put a couple of notes on there for things you might want to look at
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #453: FIXED in 4 hr 16 min: https://hudson.wedontsleep.org/job/db-devel/453/
<gary_poster> hey sinzui (or deryck), is there a brief history of our IE JS issues I could hear?  AIUI, we generally just turn off JS for IE.  In particular, I'm curious why we are more restrictive than YUI itself; I'm curious if we considered supporting IE >= 7, and just ignoring 6 and below; and I'm curious if we know what our OEM people are using.
<sinzui> gary_poster: We turned of IE because we were too lazy or inexperienced to make our site work with the dominate browsers of the ent
<deryck> gary_poster: what sinzui said ;)
<deryck> gary_poster: and because we don't know how to write js and make common mistakes outside of YUI and found it easier to avoid
<deryck> oem definitely uses IE heavily
<gary_poster> ok, thank you deryck and sinzui
<deryck> well, oem clients or partners or whatever.  *the* oems use IE, not Canonical's division.
<sinzui> gary_poster: Our argument against ie6 is that it represents .05% of traffic and MS does not support it. However, Oor Chinese partners often do use ie6
<sinzui> gary_poster: Canonical/Ubuntu CSS does not support ie6, and I removed out CSS hacks for ie6. I think we are are concerned about ie7-9
<gary_poster> sinzui, I'm less surprised by IE 6 (which is notoriously unpleasant to work with) than IE generally (I've read that 7 and higher are much more acceptable)
<gary_poster> sinzui, I'm +1 on supporting 7-9 and ignoring 6
<deryck> I'm +1 on that as well.
<sinzui> IE 9 likes Lp, but I think several parts are disabled for 9 because our scripts are blocking all ie
<gary_poster> right
<deryck> We need some knowledge sharing so people are comfortable running IE either via wine or VMs.
<gary_poster> yeah
<gary_poster> VMs are easy enough...but then you have to buy Windows
<deryck> right
<sinzui> I do not have Windows. I rely on the good will of volunteers :(
<gary_poster> :-/
<gary_poster> I have a license/vm for XP at least
 * sinzui looks at analytics
<deryck> I keep windows for taxes and gaming, so IE is easy enough for me
<gary_poster> heh
<deryck> i have my priorities clear!
<gary_poster> :-) makes sense to me
<sinzui> gary_poster: deryck 27% of visitors are on Windows, 10% xp 10% 7
<gary_poster> hm
<gary_poster> what about IE versions, sinzui?
 * gary_poster should get the information for our google analytics account
<deryck> but how bad is their experience, too?  There has to be some argue for getting better at IE, just because the experience can be so broken at times.
<gary_poster> deryck, some of the biggest functionality we are building is JS only
<sinzui> gary_poster: deryck: 6% is ie...4% being ie8, 1% ie7, the remaining is 6 and 9
<gary_poster> (I mean, our squad)
<gary_poster> wow
<gary_poster> that's so small
<deryck> yeah, that's incredibly tiny
<gary_poster> and I guess that tells us that 8 is the one to look at right now if we are going to look at all, and not look comprehensively
<deryck> maybe do better going forward, but not worry about past functionality so much?
<gary_poster> (which is such a PITA :-/ )
<gary_poster> I'm alright with that, but if we don't provide noscript alternatives then we may have to set a higher bar
<sinzui> deryck: I do not think ie users can report bugs because the suggestion UI is 100% ajax
<gary_poster> (which again points to "going forward")
<gary_poster> lol
<gary_poster> +1
<deryck> they can't comment on bugs, I know that for sure ;)
<gary_poster> Then we're safe! :-)
<sinzui> deryck: so in this case, we had a show of hands of who wanted support, but ie users did not raise their had :)
<sinzui> hand
<deryck> heh
<gary_poster> lol
<jml> deryck: ta
<sinzui> gary_poster: <rants> We chose YUI because it had good cross browser support, but its support is not as good as jquery.
<gary_poster> sinzui :-( and the majority of the rest of the world uses jquery
<sinzui> correct
<deryck> and we could have picked the best of both -- used jquery for coding, but yui's test framework for testing, for example.
<deryck> but hindsight is always better
 * sinzui recalls some engineers had foresight
<deryck> altough I like YUI 3 very much.  But I'm probably in the minority on this.
<gary_poster> I like it so far too.  But I'd prefer to be with the majority of the world where we can.
<deryck> yeah, agreed.  But if we had built lazr-js with jquery, it would have still be painful.  Sometimes it's not the tool, but what we do with it. ;)
<gary_poster> heh, I believe you :-)
<deryck> so jqeury lovers in LP rejoice that we didn't kill your love of jquery! ;) :)
<sinzui> gary_poster: deryc: the ie issue may not be as bad as the reported bugs. I could not test them during the bugjam to verify two YUI upgrades fixed the issue.
 * deryck always looks at the bright side
<gary_poster> heh
<gary_poster> that would be nice
<deryck> I wondered that, too.  If we could turn it on in various places and see.
<sinzui> We could answer that in a day. I suspect the issues can be fixed by one engineer in less than 8 man days.
<gary_poster> we'd need a critical bug for it to happen anytime soon, I suspect
<leonardr> benji: re "test the previous behavior is gone". that was my intention with test_exception_whose_view_has_no_status
<leonardr> it could probably be renamed
<leonardr> test_non_web_service_exception_is_reraised
<sinzui> gary_poster: deryck, I suspect that many opera bugs may be fixed in the newer versions of Opera. We stopped getting bug reports about it when BjornT left the team
<gary_poster> heh
<gary_poster> I'm afraid I don't care about Opera :-/
<deryck> I think we can't have broken anything serious or BjornT would still file bugs. :-)
<gary_poster> heh, maybe so
<sinzui> gary_poster:  it is 3% of traffic and 90% is 11+
<sinzui> oh wow, that might be the lat zune ever to visit lp
<gary_poster> do you think we should care about Opera's 3% sinzui?  opera is such a choice that I think people can be expected to choose other things.  I'm generally more inclined to care about browser choices that are defaults on popular OSes, and of course browsers that have large percentages irrespective of OS defaults.  Opera is neither.
<sinzui> There are still more iOS browsers than android
<sinzui> gary_poster: I do not think Opera is important at 3%. Opera is always a choice. I think YUI and Opera upgrades allow us to close all opera bugs, but I cannot bring myself to do that without confirmation
<gary_poster> makes sense
<sinzui> gary_poster: We closed a lot of safari bug when Chromium reached 15%. At that point, we unconsciously decided to ensure Lp works with webkit
<deryck> Opera > 9.5 X grade on YUI, which means they assume it works but don't test.  And they close all bugs opened against that browser.
<gary_poster> Chromium meant that we started using webkit regularly too, which may be at least one aspect of the "unconscious" part
<gary_poster> heh, I like that last bit :-)
<gary_poster> ("close all bugs opened against")
<deryck> heh, I thought you might
<sinzui> gary_poster: have you ever looked at who reported and commented on all open and closed Safari bugs? I'll save you the tiime. It is Us. Both Lp and Canonical engineers use Safari.
<gary_poster> huh
<jml> :(
<sinzui> 90% of all safari bugs were report by a Canonical employee, or is the primary user in the comments
<benji> leonardr: (was at lunch) oh, I misunderstood that test, looks good
<lifeless> moin
<lifeless> allenap: ping (qa)
<lifeless> jtv: rev 12599 I'm guessing is also untestable for now ?
 * jcsackett realizes answers is not exported at all.
<henninge> sinzui: I replied to your question on the MP. ;)
<henninge> sinzui: would be great if you could approve the ui now ;)
<sinzui> henninge: r=me
<henninge> sinzui: cool, thanks!
<thumper> morning
<thumper> leonardr: I'm going to miss the standup today as a close friend had a stroke, and I'm going to visit him in hospital
<leonardr> thumper: oh no! i'll run the standup
<thumper> leonardr: just you and wallyworld_ :)
<leonardr> ok, wallyworld, mumble?
<thumper> leonardr: he doesn't arrive for about 40 minutes :)
<thumper> and even then it is 7am for him
<leonardr> ah, ok
<leonardr> wallyworld_ -^ let me know when you get this
<thumper> lifeless: I wish we had a simple way to load all of the objects needed to create canonical urls
<sinzui> thumper: lifeless. I need to change the permissions in for person-merge-job in security.cfg. My branch is based on devel. Will those changes be applied by a no downtime release?
<thumper> sinzui: I think new permissions are added
<thumper> but old ones are not dropped
<thumper> I may be wrong
<sinzui> yeah. I think that too, but I keep hearing disappointing results. I could switch to db-devel and be certain that my changes work
<lifeless> thumper: sinzui: you are correct
<lifeless> it will work
<lifeless> it doesn't work on qastaging yet, but does on prod
<sinzui> ;?
<sinzui> :/
<sinzui> lifeless: Can I ask a losa to do something when the time comes to qa? I can qa on staging at the very least
<lifeless> sinzui: ask them to 'update the qastaging tree on sourcherry and run security.py with --no-revoke'
<sinzui> noted. Thanks!
<wallyworld_> leonardr: hi, will be there in 5-10 minutes. just have to make the kid's lunch
<leonardr> sure
<wallyworld_> leonardr: here now
<leonardr> wallyworld_: ok, 1 sec
<lifeless> matsubara: https://lpstats.canonical.com/graphs/OopsLpnetHourly/20110315/20110316/ is spiky
<lifeless> matsubara: do you know what might be up there?
<leonardr> wallyworld_, i'm on mumble
 * matsubara looks
<wallyworld_> leonardr: can't hear me?
<leonardr> i can hear you
<leonardr> but not vice versa, apparently
<StevenK> lifeless: Sigh. That should happen automatically.
<lifeless> StevenK: it should
<lifeless> StevenK: which of the several things affecting production stability should I defer to get that happening :( ?
<StevenK> lifeless: Ah, well, it is a bug with the update scripts or something more sinister?
<lifeless> StevenK: the losas have a policy that automated scripts can only talk down trust levels, not up
<lifeless> StevenK: and db servers are near the top of the pile
<lifeless> StevenK: so the update script needs to get more complex, part of it moved to sourcherry, become pull rather than push, that sort of thing
<leonardr> sinzui, looks like our branch has landed
<sinzui> \o/
<StevenK> lifeless: That's our issue to fix, or a LOSA issue?
<lifeless> StevenK: losa
<StevenK> Right, and we both know the issue there.
<lifeless> indeed
<leonardr> wallyworld_: `python setup.py sdist` seems to work for me, generating a tarball for lazr-js
<leonardr> what happens when you try that?
 * wallyworld_ looks
<lifeless> StevenK: does bug 734719 count as fix released?
<_mup_> Bug #734719: DSD.base_source_pub assumes the base version is published in the child <derivation> <qa-untestable> <Launchpad itself:Fix Committed by stevenk> < https://launchpad.net/bugs/734719 >
<wallyworld_> leonardr: ok, that worked. not sure what i did yesterday. i assume i need to edit setup.py to change __version__ and commit that before I package?
<lifeless> bugger
<lifeless> one rev in the deploy queue
<StevenK> I was just wondering when we deployed
<lifeless> StevenK: just now
<StevenK> lifeless: Since prod is running r12599, yes, kill it
<leonardr> wallyworld: yes, the __version__ is kept in setup.py for lazr-js
<wallyworld_> leonardr: cool. thanks
<matsubara> lifeless, https://devpad.canonical.com/~lpqateam/oops-temp.html partial report for today and it doesn't have any oops that caught my eye as causing the spike
<lifeless> matsubara: oh, I was unclear
<matsubara> lifeless, btw, you can see there that your fix is working :-)
<lifeless> matsubara: the 50 per hour spike was the bug search issue I referenced in my email
<lifeless> matsubara: the spike I was referring to when I pinged you is how every second how is 0 for everything
<lifeless> so its like 0, something, 0 ,something
<lifeless> Cool, that report is much more useful
<lifeless> the one remaining thing I'd love
<lifeless> and I wasn't sure how to do it easily
<lifeless> would be to make sure that we get one-of-each pageid from the summary, in the detail section
<lifeless> but perhaps thats covered by my change anyway
<lifeless> hmm
<lifeless> https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1900C772
<lifeless> 7.6 seconds of python time
<lifeless> 127 LFA lookups
<bdmurray> Hello, I've noticed a typo in test name and wanted to check on the best way to fix it.
<bdmurray> A whole branch seems like a bit much
<lifeless> bdmurray: we can only land branches
<lifeless> bdmurray: if you're a reviewer you can self review and JFDI
<bdmurray> I guess my hope was somebody would JFDI in the branch they are currently working on
 * thumper is back
<StevenK> bdmurray: Branches are cheap, what's the test name?
<bdmurray> StevenK: its in lib/lp/bugs/model/tests/test_bugtask_status.py - -    def test_privileged_user_can_unset_wont_fix_released(self):
<bdmurray> StevenK: should be unset_fix_released_status
<wgrant> Anyone want to review https://code.launchpad.net/~wgrant/launchpad/hide-inaccessible-bugs/+merge/53529?
<StevenK> bdmurray: Submitted to PQM, thank!
<StevenK> s/!$/s!/
* leonardr changed the topic of #launchpad-dev to: Performance Tuesday | https://dev.launchpad.net/ | firefighting: - | On call reviewer: gmb | https://code.launchpad.net/launchpad-project/+activereviews/
<StevenK> wgrant: Done.
<wgrant> StevenK: Thanks.
<sinzui> wgrant: mumble
<lifeless> flacoste: ppr failed to render I think ?
<lifeless> flacoste: where should I look to see whats up ?
 * thumper sighs
#launchpad-dev 2011-03-16
 * thumper still writing tests
* lifeless changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews/
<lifeless> grr bug heat updates
<lifeless> wgrant: ID OOPS-1901QS5
<thumper> lifeless: you'd like my current tests
<lifeless> thumper: I will?
<thumper> lifeless: they all have self.assertThat(recorder, HasQueryCount(Equals(0)))
<wgrant> with NoStormStatements:
<wgrant> lifeless: How? I changed the target of the original bug to a different project, and it sent me to the bug page as expected...
<wgrant> Using both +editstatus and the inline form of deprecation.
* StevenK changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: StevenK | https://code.launchpad.net/launchpad-project/+activereviews/
<lifeless> wgrant: I think I hit a redeploy
<wgrant> lifeless: How would that change things?
<lifeless> wgrant: transaction committed, appserver killed, refresh - boom
<wgrant> Mmm, I guess.
<lifeless> wgrant: have you seen https://devpad.canonical.com/~lpqateam/ppr/lpnet/2011-03/daily_2011-03-13_2011-03-14/categories.html - lots of requests :)
<wgrant> Anyway, once my branch lands it will just redirect instead.
<wgrant> Of course it conflicted with leonard's...
<lifeless> wgrant: even on post ?
<wgrant> lifeless: HTTP is just that awesome.
<wgrant> We already do it in some circumstances :/
<lifeless> wgrant: sarcasm doth not befit thee
<wgrant> lifeless: That's daily?
<StevenK> lifeless: Is "sarcasm doth not befit thee" or did you make it up?
<wgrant> lifeless: Do we have a graph of 99% time over time?
<StevenK> Sigh. Is it a quote
<StevenK> mwhudson: Hi!
<lifeless> wgrant: thats one day, yes
<lifeless> wgrant: https://lpstats.canonical.com/graphs/PPR/
<wgrant> lifeless: The translations graph is nice.
<wgrant> And should be even nicer tomorrow, I suppose.
<lifeless> wgrant: because of the drop ?
<lifeless> POFile:+translate was at 9.7seconds 99th percentile
<wgrant> lifeless: Right.
<lifeless> hopefully it will be better now ;)
<lifeless> but todays ppr failed to generate
<wgrant> lifeless: +translate seems to mostly complete in under 2 seconds. Nicely done.
<lifeless> wgrant: I think so
<lifeless> pure fluke that I bothered to try a with plan
<lifeless> there may be a fat index impeding regular queries; not sure
<lifeless> I want https://lpstats.canonical.com/graphs/PPR/ to be on a log scale
<wgrant> Indeed, most of our graphs should be.
<lifeless> spm: can tuolumne do that ?
<spm> log scale? not that I'm aware of.
<wgrant> Looks like pycairochart might not support it.
<mwhudson> StevenK: hello?
<StevenK> mwhudson: I was talking to Julian yesterday about DSDs, and he was mentioning that we might want to delete DSDs -- which would remove comments associated with the DSD. Would you prefer the DSD hung around to preserve them?
<wgrant> Under what circumstances would you remove DSDs?
<StevenK> That is what I'm not certain about, but Julian seems to think the DSDJ runner needs to do it.
<mwhudson> StevenK: dsd?
<StevenK> mwhudson: DistroSeriesDifference
<wgrant> I don't think removing them before a series is finished would be a good idea...
<mwhudson> StevenK: you are asking me because i have linaro in my cloak i presume?
<StevenK> mwhudson: And you're in my TZ. :-)
<lifeless> mwhudson: we hear you are slightly technical
<mwhudson> i'm not sure i'm the right person to ask about this though sadly
<mwhudson> StevenK: i think i plead EINSUFFICIENTCONTEXT
<mwhudson> StevenK: maybe get julian to talk to james_w ?
<mwhudson> StevenK: or you can try to explain a bit more context i guess
<StevenK> Or I can in my morning
<mwhudson> like, as wgrant says, "when would you remove them" ?
<wgrant> They need to be kept even between series if the difference persists.
<StevenK> Like I say, I'm unclear on why Julian wants to do that, and I want to get the simple cases working for the DSDJ runner before I bother diving into the harder stuff.
<mwhudson> StevenK: i suggest we acquire some more clarity before discussing further then :)
 * wgrant declares war on Makefile.
<StevenK> mwhudson: Do you know where to find some? :-P
<mwhudson> StevenK: well, find out why julian wants to delete them?
<lifeless> data that is no longer interesting shouldn't be kept around in our primary tables
 * StevenK teaches mwhudson about "humour"
<lifeless> mwhudson: ^ I suspect he's thinking about what is/isn't interesting
<mwhudson> StevenK: heh
<StevenK> mwhudson: Something you might be able to help with -- I'm writing a run() method for a IRunableJob and I'd like to get at the log, so I can add messages to it, is that easy?
<mwhudson> StevenK: isn't there a self.logger or something?
<wgrant> I have never worked out how to get the log from outside a RunableJob, either.
<wgrant> That would be really handy.
<mwhudson> maybe i'm not thinking of the right object
<mwhudson> maybe there just should be :-)
<mwhudson> sigh, a coherent logging strategy would be nice
<StevenK> ... yes
<lifeless> a good logging framework would be too
<StevenK> thumper helped that by binning about 15 loggers from the tree
<poolie> hi all
<wgrant> Hi poolie.
<mwhudson> logging is one of those things that seems easy, but i've never seen it done satisfactorily
<StevenK> It doesn't matter how much it logs, you always want it to log more.
<StevenK> (or less, if it never breaks)
<mwhudson> well
<thumper> StevenK, wgrant: is a SourcePackage object _ever_ created without a distroseries?
<mwhudson> jml added some overenthusiastic logging to the sftp server once
<mwhudson> when the log got to 8 gigs within a day or something, that was decided to be "a bit too much"
<StevenK> Haha
<thumper> StevenK, wgrant: see lp.bugs.model.bug line 1135
<thumper> ish
<lifeless> thumper: SourcePackage should be called DistroSeriesSourcePackage
<thumper> lifeless: I realise that
<lifeless> thumper: there is a DistributionSourcePackage, which shouldn't exist
<lifeless> and a missing table which should exist
<thumper> lifeless: but I've found some weird code in bugs
<thumper> lifeless: it assumes that distroseries may be None in SourcePackage
<thumper> I didn't think it could be
<lifeless> in ISourcePackage, not SourcePackage
<lifeless> one sec
<wgrant> thumper: I think you're right.
<thumper> required = True
<thumper> in which case, I'll simplify the bug code
<wgrant> thumper: But our sample data and/or tests are probably fucked.
<wgrant> Remove it and see what breaks, then fix it.
 * thumper nods
<lifeless> thumper: ok, that looks bogus
<thumper> lifeless: I've simplified the bug code
<thumper> lifeless: as part of my work
<thumper> what I don't understand
<thumper> is the requirements of certain tasks
<thumper> like...
<thumper> what is needed for a source package task to exist?
<thumper> a distroseries task
<thumper> or a distro source package task
<thumper> or either
<thumper> or both
<thumper> there seems to be implicit rules
<wgrant> A SourcePackage task has a DistroSeries and a SourcePackageName.
<thumper> not enforced by the model code
<wgrant> A DistributionSourcePackage task has a Distribution and a SourcePackageName.
<wgrant> See determine_target
<thumper> wgrant: yes... I understand that
<thumper> wgrant: there are implicit expectations of certain tasks though
<wgrant> thumper: Ah, with regard to conjoinment?
<thumper> you can't have a distroseries task without a distro task
<wgrant> Right.
<wgrant> Well, you can, but you're not meant to.
<thumper> wgrant: well, exactly
<wgrant> eg. bug #43150 will make you cry.
<_mup_> Bug #43150: [SRU] maxima frontends fail to connect <wxmaxima (Ubuntu):Fix Released> <gcl (Ubuntu Dapper):Fix Released by wgrant> <maxima (Ubuntu Dapper):Fix Released> < https://launchpad.net/bugs/43150 >
<thumper> the database nor model doesn't enforce it
<lifeless> thumper: \d bugtask
<lifeless> thumper: see the checl constraints
<wgrant> The current series task model only came about in late 2006, early 2007.
<wgrant> So there are old bugs (like 43150) that have invalid things.
<thumper> lifeless: we don't have check constraints of other rows
<lifeless> indeed
<lifeless> I'd like to delete the conjoined concept
<lifeless> I see no benefit in cross-row rules
<lifeless> better to have a more powerful concept of 'target' and fix our seach
<lifeless> *search*
<lifeless> wgrant: wow, the rendering of that bug is --broken--
<lifeless> OTOH, 2.38 seconds
<thumper> I still don't understand the conjoined thing
<lifeless> kinda slow
<wgrant> lifeless: What's broken?
<lifeless> wgrant: no package name shown for the first two tasks
<wgrant> (I use that bug a lot, since it has lots of subscribers, lots of strucsubs, a few dupes, a few tasks, some series tasks, and various bits of activity)
<wgrant> lifeless: Right.
<wgrant> lifeless: There is no DSP task for them.
<wgrant> lifeless: Because they predate the new release management stuff.
<wgrant> Well, "new" meaning "4 years ago"
<wgrant> We need to discuss it with Ubuntu (to convince them to use milestones instead), but I think we can probably fairly sensibly just remove conjoinment.
<wgrant> Is there a function around to mash a string into valid_name?
<thumper> not that I know of
<wgrant> Ah, sanitize_name.
<lifeless> wgrant: well, they don't have the conjoined layout
<lifeless> wgrant: but we could render them nicely regardless
<wgrant> lifeless: They are not meant to exist, so rendering them nicely probably isn't desired.
<wgrant> But I filed a bug on this a long time ago.
<lifeless> wgrant: I think we're either completely agreeing, or talking past each other
<thumper> someone should write a script to add the missing bugtasks
<lifeless> wgrant: my point is that conjoined masters don't express anything that single bugtasks can't
<wgrant> Right.
<lifeless> wgrant: there is a /behaviour/ change needed when the default series is altered
<lifeless> possibly
<lifeless> anyhow - milestones are orthogonal I think
<lifeless> thumper: or we could just simplify things and get rid of the redundant data
<wgrant> lifeless: They're not orthogonal.
<wgrant> lifeless: Do you know how Ubuntu uses series tasks?
<lifeless> for guiding developer effort in backports
<lifeless> not 'backports'
<wgrant> Hah.
<wgrant> No.
<lifeless> I know what I mean
<wgrant> That's one use.
<wgrant> They are used for security updates and SRUs.
<lifeless> yes
<wgrant> But also for indicating release criticality.
<wgrant> Targetting something to the dev series makes it release critical.
<sinzui> thumper: wgrant: I am exhausted, but I can see that this conversation is a sequel to my predicament from last week. thumper the rules and sample data are bad. We may have fixed the factory in the last 12 hours. Have fun storming the castle
<lifeless> wgrant: so, conjoined masters are not needed for that
<wgrant> lifeless: Eeeeh.
<wgrant> lifeless: If there is no DSP task, then the bug won't show up in most listings.
<lifeless> wgrant: that is - and always was - a bug in those listings
<lifeless> our concept of 'bug target' doesn't map directly to obvious queries
<lifeless> if we fix that: e.g. search on distribution=1 or distroseries = distribution.development_focus - a lot of pain just disappears
<wgrant> Right.
<sinzui> make it so
<lifeless> arguably we should search on distribution=1 or distroseries in (distribution.distro_serieses)
<lifeless> that would be a behaviour change
<lifeless> my point here is that we can, without changing behaviour, eliminate conjoined bugtasks
<lifeless> make the bugtask table smaller
<lifeless> and make a bunch of code cleaner
<wgrant> Behaviour is still slightly changed.
<wgrant> But I agree fully.
<thumper> I suppose I should go and get my kids from school
 * thumper wanders off
<lifeless> and that using milestones, or not, is a different discussion, *because* we could do this separately
<lifeless> wow
<lifeless> 252 Time Outs
<lifeless> even *with* that search issue
<wgrant> Nice.
<wgrant> Less than 1500 critical OOPSes.
<lifeless> and a lovely long list of page ids
<lifeless>     2 /    0  https://api.launchpad.net
<wgrant> And full listings.
<wgrant> Yeah.
<lifeless> >< wadl
<wgrant> === Top 50 Exceptions (total of 96 unique items) ===
<wgrant> :(
<wgrant> I guess that is potentially less bounded than the pageids.
<lifeless> nah, oops-tools code breaks my brain
<wgrant> sinzui: Did you get anywhere with the TP corruption?
<wgrant> We should be just below 1000 exceptions tomorrow.
<wgrant> Er, day after.
<sinzui> wgrant: I did not. I wanted to talk to henning. I wondered if he knew something about changes to launchpad translations coordinators
<lifeless> might drop us another second tomorrow
<wgrant> lifeless: Go all the way to 10.
<lifeless> wgrant: no
<wgrant> :(
<sinzui> wgrant: 1. I believe we could fix the data with a stealthy add remove using the UI, but I want to know how this state happened first
<wgrant> sinzui: The SQL breaks my brain.
<StevenK> lifeless: Isn't this our third drop in like a week?
<wgrant> sinzui: I assume there must be a bug there.
<wgrant> StevenK: Only the second.
<wgrant> Hence me wanting to go to 10.
<lifeless> wgrant: a second at a time mitigates the risk of multiple pages right on the boundary and not clear in the ppa
<lifeless> ppr
<lifeless> bah
<wgrant> Meh
<sinzui> oh, I can get jcsackett involved. It might bring back traumatic memories, but I think it will make him stronger
<lifeless> wgrant: this strategy is working, so I take your 'Meh' and raise you 'Meh'
<wgrant> lifeless: Meh.
<lifeless> 147.19s  OOPS-1900K1313  Person:+contactuser <>
<wgrant> Someone's trying to talk to big teams.
<lifeless> wow 973  OOPS-1900M785   Product:+download
<lifeless> wgrant: yes, the 'spam me' button is alive and well :>
<StevenK> K, M, C, Q are all different appservers?
<lifeless> StevenK: yes
<wgrant>    2 ProgrammingError: permission denied for relation revisionauthor
<wgrant> In branchmail.
<wgrant> Huh.
<StevenK> lifeless: Okay, but we don't really care which appserver, right?
<lifeless> StevenK: if you suspect GIL contention
<lifeless> StevenK: then you will want to lookup and see if its a 1/2 thread server, or a 4-thread server
<lifeless> in lp-production-configs
<sinzui> We have had a mechanism for queuing emails to users for 2.5 years and it has exactly one call site, person.merge. I think we can speed up a lot of registry and answers reusing it
<StevenK> I think we can speed up some of soyuz by also reusing it
<lifeless> sinzui: +1 without looking at the code ;)
<wgrant> StevenK: Oh?
<wgrant> StevenK: Only queue accepts send email.
<wgrant> And then only two per package.
<wgrant> It's not great, but it's like 0.1% of the time.
<lifeless> POTemplate:+index 12529 37194.36 8.03
<lifeless> translations will still have a high 99th percentile I think
<lifeless> till that page gets improved
<lifeless> PreviewDiff:EntryResource has a 54 second 99th percentile ><
<lifeless> Distribution:+edit - 33seconds
<wgrant> We need to use <blink> on the timing information if it exceeds the 99%
<wgrant> https://code.launchpad.net/~wgrant/launchpad/bug-456616/+merge/53556, anyone?
<lifeless> wgrant: do you mean by the uid ? and if its over our target 99th percentile?
<wgrant> lifeless: The one by the UID, but anything over the current 99th percentile.
<wgrant> That way it decreases without everything flashing.
<wgrant> With red 72pt bold.
<lifeless> wgrant: if you want to do it, I can describe how I'd do it.
<wgrant> No.
<lifeless> wgrant: I was thinking of a background stylesheet - like demo - with the word 'SLOW' on it.
<wgrant> Hah.
<wgrant> Oh, even better, an animated GIF background!
<lifeless> no
<wgrant> We can take slow pages back to the 90s.
<lifeless> green on black baby
<lifeless> switch the stylesheet
<lifeless> so - seriously.
<lifeless> in my original concept for this, I wanted a pervasive, unintrusive message on problem pages
<wgrant> I was sort of thinking an FF which highlights the time in red if it's exceeded.
<lifeless> the watermark idea was one way to do it
<wgrant> But your suggestion might be nice.
<sinzui> lifeless the background image is really easy. weather configured or using a onload event (life the query count), we can add a class to the body like we do private. huwshimi is landing a replacement for the private background soon too
<lifeless> yes, a new FF which sets the time to alert-if-exceeded would be the implementation I'd choose
<lifeless> sinzui: can you stack background images ?
<sinzui> no
<lifeless> sinzui: so I wouldn't want to break private pages and make them appear public
<lifeless> sinzui: how is that handled on staging etc?
<sinzui> but we also have the maincontent div which allows us to have multiple backgrounds with a little more effort
<lifeless> sinzui: if anyone wants to wire this up, I'd be ecstatic; if they need help or input on getting a value in the main template to trigger this - I can do that in ~ 5 minutes, and its -dead- easy.
<huwshimi> sinzui: lifeless: css3 does actually give us the ability to have multiple background images, but they need to be defined differently.
<sinzui> lifeless: config.is_demo is checking in the base-template and it adds a class to body
<lifeless> like the time in the top right, it will need to be onload based.
<thumper> lifeless: at least this one is solved :-)   2 / 0NullBugTask:+index
<wgrant> thumper: That class no longer exists.
<wgrant> It's unfortunate that it didn't time out much on its last day.
<thumper> wgrant: exactly. Solved.
<wgrant> Heh.
<cody-somerville> wgrant, do PPAs not generate kernel udebs or something?
<wgrant> cody-somerville: AFAIK they should.
<thumper> wallyworld_: ping
<wallyworld_> thumper: hi. hope things are ok with your friend
<thumper> wallyworld_: he is feeling much better
<thumper> wallyworld_: got mumble?
<wallyworld_> yep. give me a sec
<lifeless> poolie: what was the search you were making that timed out? [also, my preference is for new bugs please]
<lifeless> its much easier to dup than to split
<lifeless> poolie: bugsearch, for instance, has about 10/20 different components, all of which can be a problem, so theres no particular reason to think that a timeout in one search is meaningful for a timeout in a different search, unless its hitting the same code path with the same data
<poolie> surely the search is in the oops?
<poolie> for EOFError in /bzr iirc
<lifeless> poolie: oopses take up to 20 minutes to get visibility
<lifeless> 2.45 seconds for me : https://bugs.launchpad.net/bzr/+bugs?field.searchtext=eoferror&orderby=-importance&search=Search&field.status:list=NEW&field.status:list=INCOMPLETE_WITH_RESPONSE&field.status:list=INCOMPLETE_WITHOUT_RESPONSE&field.status:list=CONFIRMED&field.status:list=TRIAGED&field.status:list=INPROGRESS&field.status:list=FIXCOMMITTED&field.assignee=&field.bug_reporter=&field.omit_dupes=on&field.has_patch=&field.has_no_pa
<lifeless> (if that didn't truncate)
<StevenK> Did
<StevenK> &field.has_no_pa
<lifeless> https://bugs.launchpad.net/bzr/+bugs?field.searchtext=eoferror
<lifeless> wgrant: hah, I had just started to look at 607954
<lifeless> wgrant: enjoy :)
<wgrant> lifeless: Just finishing it up now.
<wgrant> The page is actually very lucky.
<wgrant> In most cases now it avoids another three queries per diff.
<wgrant> Because it's already preloaded them accidentally.
<lifeless> hahahahah
<wgrant> Oh, I hate circular imports.
<lifeless> ok, all soyuz errors are clearly visible in the new report
<spm> you should try circular servers some time. server A depends on server B being alive and working; B depends on C; C depends on A. Boom.
<poolie> if it happens persistently i'll file a new bug
<lifeless> poolie: thanks
<poolie> it does not seem as regular as it was yesterday
<lifeless> spm: we do that to keep you on your toes
<poolie> i guess the one off will show up in the oops report?
<lifeless> poolie: we're down 50% on timeouts from yesterday
<lifeless> poolie: yes, it will
<lifeless> poolie: and 75% on timeouts from the week before
<poolie> way to go
 * StevenK looks for the critical bug graph
<mwhudson> StevenK: http://webnumbr.com/.join(launchpad-oops-bugs.all,launchpad-timeout-bugs.all,launchpad-critical-bugs.all) ?
<wgrant> Huh.
<wgrant> I was going to say that that expression could be simplified. I didn't realise it was a real URL.
<lifeless> StevenK: its about to jump
<lifeless> StevenK: as I'm going to file bugs for all the timeouts
<StevenK> Heh
<wgrant> lifeless: Requested the timeout drop yet?
<lifeless> wgrant: *tomorrow*
<lifeless> wgrant: I want a full day without search index fail etc
<wgrant> Ah, right.
<lifeless> wgrant: so when the oops reports come out tomorrow, I'll get spm to drop it, and awaaaay we go
<lifeless> wgrant: I assure you, I'm ask keen as you are to get the cap down low
<lifeless> s/ask/as/
<wgrant> Heh
<lifeless> bugs domain was 50% of our page renders last month
<lifeless> 20M/40M
<mwhudson> lifeless: does that exclude robots?
<lifeless> mwhudson: its apparently  bong
<StevenK> Or API scripts?
<mwhudson> lifeless: :)
<lifeless> https://devpad.canonical.com/~lpqateam/ppr/lpnet/latest-monthly-categories.html
<wgrant> API scripts are excluded.
<lifeless> All Launchpad except operational pages
<lifeless> (?<!\+opstats|\+haproxy)$	39421999
<wgrant> I don't trust that number, though.
<lifeless> API
<lifeless> ^https?://api\.	74901732
<lifeless> Launchpad
<lifeless> .	164821045
<lifeless> so, raw count is 164M
<lifeless> api 75M
<lifeless> but 'non opstats' is only 40M.
<lifeless> wtf
<wgrant> Yeah.;
<wgrant> That's what I'm wondering.
<lifeless> thus - 'bong'
<lifeless> fraaaaaaaanics
<wgrant> Since the daily non-opstats counts are much higher than 39000000/30
<StevenK> And Bugs - search is zero
<StevenK> Which amuses me
<lifeless> thats a whack reges is all
<StevenK> lifeless: Tautology
<lifeless> ah
<lifeless> down the bottom
<lifeless> All launchpad except opstats
<lifeless> (?<!\+opstats)$	118473614
<lifeless> which is santer than the non-haproxy inclusive one ><
<lifeless> -no-freaking-idea
<wgrant> But look at daily categories.html
<StevenK> Registry is bong, too
<wgrant> 6.5m non-opstats-non-haproxy
 * StevenK deletes shipit
<wgrant> Oh. But that might include API.
<wgrant> Yeah.
<wgrant> https://devpad.canonical.com/~lpqateam/ppr/lpnet/2011-03/daily_2011-03-05_2011-03-06/categories.html appears to have 3.2m non-API non-haproxy non-opstats requests.
<lifeless> wgrant: yes
<lifeless> wgrant: according to the daily ppr, most of our growth was scripts
<wgrant> Even assuming a conservative 20 working days a month, that's still 64m page views.
<wgrant> So where is 40m coming from,.
<lifeless> ah, I see what you're cross referencing
<lifeless> yes, its bong
<jtv> lifeless: I'm looking at the feature flag controller.  The handling of per_thread.features looks pretty messy â so I'm wondering if maybe there's some good reason why we're manipulating per_thread.features ad-hoc in so many places?  (If not, I'd like to clean it up)
<wgrant> https://code.launchpad.net/~wgrant/launchpad/bug-607954/+merge/53567, anyone?
<jtv> wgrant: oh, I'm OCR so I might as well
* jtv changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | firefighting: - | On call reviewer: StevenK, jtv | https://code.launchpad.net/launchpad-project/+activereviews/
<lifeless> jtv: the right way would be a participation annotation, but - meh
<jtv> lifeless: what's a participation annotation?
<lifeless> maybe I mean interaciton annotation
<wgrant> You probably want the interaction.
<wgrant> Participation could work too, but I like to pretend that participations don't exist.
<lifeless> jtv: anyhow, it doesn't seem like an overhaul here would help you at all
<jtv> Would improving that be complicated in any way if I replaced all the various reads from and writes to per_thread.features with calls to the existing getter and a new setter?
<lifeless> jtv: and you were concerned about rabbit holes the other day
<lifeless> jtv: I suspect its a noop change
<jtv> I find there's not a lot of cost to cleaning it up, but it saves some wtf in the code that we already have and the code that I need to add.
<jtv> I was just wondering if maybe there's some clever grand scheme behind the current mess.
<lifeless> nope
<jtv> ok, a single getter & single setter it is then.
<StevenK> The London Olympics countdown clock has clapped out after less than a day, the BBC reports.
<StevenK> The precision timepiece was triumphantly unveiled in Trafalgar Square yesterday, but failed to clear the first 24-hour hurdle and is now stuck on 500 days and 7:06:56.
 * StevenK cackles.
<lifeless> jtv: fwiw, assignment is a single setter.
<wallyworld_> StevenK: it's probably running on windows :-)
<StevenK> I thought the Reg's joke of "Well, don't power it with one AAA battery ..." was funnier.
<LPCIBot> Project db-devel build #456: FAILURE in 4 hr 17 min: https://hudson.wedontsleep.org/job/db-devel/456/
<jtv> lifeless: it's not just assignment, it's setattr as well.  Both encode both the per_thread variable and the reference at every instance.  And then a mix of getattr and get_relevant_feature_controller to read it.
<StevenK> lp.services.job.tests.test_runner.TestTwistedJobRunner.test_timeout fails on db-devel on Jenkins again. :-(
<wgrant> Yeah.
<StevenK> jtv: Why createForPublication()? Every single other job uses create()
<jtv> StevenK: this doesn't create a job though
<jtv> it may create any number of jobs
<jtv> So it deserves a different name from the existing create() method.
<jtv> Also, I wanted to specify the relationship to the publication explicitly because it's not as simple as one might infer from just create().
<StevenK> I note you don't actually create the job for distroseries
<StevenK> You create jobs for direct children (which is fine), and the parent (which isn't)
<jtv> Ah damn you're right
<jtv> It should be the children and the distroseries itself.
<StevenK> Right
<jtv> Could you file a bug?
<StevenK> I'll fix it
<jtv> Thanks, and sorry
<StevenK> And then test_createForPackagedPublication_creates_job_for_parent_series fails
<StevenK> Which is broken, anyway
<jtv> Yes, what I got wrong was what the code should do, not just the code itself.
<StevenK> That test is far more broken, with a three-tiered distroseries on distroseries action
<StevenK> But I can use that.
<wgrant> jtv: I have three reviews for you now... or should I throw them at someone else?
<StevenK> wgrant: I'll take one
<wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/bug-390543/+merge/53568 is the most pressing.
<wgrant> Thanks.
<StevenK> wgrant: I can understand the culling of the first 40 lines of checkwatches.txt. Why the rest?
<wgrant> StevenK: I needed to move some of the bits further down, and switch DB users in a couple of places. So I changed the doctest to use the context manager.
<wgrant> More readable and shorter.
<StevenK> wgrant: This is a niggle, but you no longer test checkwatches oops use the 'CW' prefix
<wgrant> StevenK: That was why I hadn't fixed this earlier. But I decided that I don't care enough about testing that to keep that exception around.
<wgrant> Since no other script has a test like that.
<StevenK> Right.
<StevenK> wgrant: You could replace the login() call on 231 with 'with person_logged_in()' if you wished
<wgrant> StevenK: I could.
<StevenK> But I'm not certain how much of the rest of the doctest wants admin rights, so "Meh"
<wgrant> Exactly.
<StevenK> wgrant: r=me
<wgrant> Thanks.
<StevenK> How long I have waited to say that.
<wgrant> StevenK: The others are pretty simple. Could you look at them too?
<StevenK> wgrant: I dunno. What have you got to offer?
<wgrant> https://code.launchpad.net/~wgrant/launchpad/bug-734573/+merge/53572 and https://code.launchpad.net/~wgrant/launchpad/bug-607954/+merge/53567 are the MPs in question.
<wgrant> I am afraid I have no offering beyond the glory of reviewing them.
<StevenK> No beer?
<StevenK> Oh, that's right, you aren't old enough to buy it.
<StevenK> wgrant: Ah, and some trailing whitespace fixed. r=me for the first.
<StevenK> wgrant: https://code.launchpad.net/~wgrant/launchpad/bug-607954/+merge/53567 looks like it is errornously linked to bug 390543
<_mup_> Bug #390543: Checkwatches shouldn't record an OOPS if there isn't an ExternalBugTracker for a given BugTrackerType <lp-bugs> <oops> <Launchpad itself:In Progress by wgrant> < https://launchpad.net/bugs/390543 >
<wgrant> StevenK: Ah, good catch. I was committing to the wrong branch when I was working on 390543, then removed the problematic revs but forgot to unlink.
<wgrant> Fixed.
<StevenK> wgrant: my only concern with the second is line 71. Why are you changing it to a set?
<wgrant> StevenK: It eliminates duplicates, making queries more readable.
<wgrant> Since in most cases the archive and distributions will all be the same.
<wgrant> This will do 'id IN (1,)' instead of 'id IN (1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1)'
<StevenK> wgrant: Could I beg for a comment, since that's quite elegant.
<wgrant> I could do it in the callsite, but this is going to be useful everywhere.
<wgrant> Will do.
<wgrant> StevenK: Thanks.
<StevenK> wgrant: You're welcome. Three critical fixes is awesome.
<wgrant> That's eight for the day. Maybe I'll make it to 50 this cycle.
<StevenK> Crumbs.
<StevenK> Pity it is cheaper to file criticals than it is to fix them.
<wgrant> Heh.
<wgrant> Yeah.
<wgrant> StevenK: sinzui unfortunately did a lot of CSS fixes for 11.03, so he nearly caught up to me :(
<StevenK> Haha
 * StevenK wields his nailgun and nails jtv to the Internet
<lifeless> wgrant: how many did sinzui do?
* StevenK changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | firefighting: - | On call reviewer: jtv | https://code.launchpad.net/launchpad-project/+activereviews/
<wgrant> lifeless: 25 bugs in total :(
<lifeless> wgrant: and you? what time period ?
<lifeless> wgrant: what did I do ?
<wgrant> https://launchpad.net/launchpad/+milestone/11.03
<wgrant> I did 34, you 17.
<lifeless> I should stop reopening, or doing bugtask:=index :> - if I cared about the sheer counts
<wgrant> Heh, BugTask:+index is nearly under control.
<wgrant> It helps that I handled a lot of tiny incident-critical fixes last months.
<lifeless> helps both your stats, and the project :) winwin
<wgrant> Well, it had better be a winwin, because those incidents are damn annoying. :)
<wgrant> lifeless: I can has patch number for bug #715236?
<_mup_> Bug #715236: Validate architecturetag <Launchpad itself:In Progress by wgrant> < https://launchpad.net/bugs/715236 >
<lifeless> wgrant: stub should be around soon, if you want to optimistically grab one - do so from the wiki page
<lifeless> wgrant: what needs changing ?
<wgrant> lifeless: distroarchseries.architecturetag needs valid_name applied.
<lifeless> as a check constraint ?
<wgrant> Yes.
<lifeless> kk
<jtv1> wgrant: sorry, got distracted and forgot about your review.  What still needs reviewing?
<wgrant> jtv1: StevenK did them all. :(
<wgrant> So you are safe for now.
<jtv1> whoops
<jtv1> Terribly sorry about that.
<wgrant> 'tis fine. StevenK was glad to exercise his new superpowers, I am sure.
<jtv1> Ah yes
<lifeless> wgrant: 'Note that adding new users requires manual DB reconfiguration' - security.py might want to detect and error clearly
<poolie> i like the new loggerhead skin
<wgrant> lifeless: Then we have DB upgrades failing instead of scripts.
<wgrant> == bad
<wgrant> And it's also really hard to detect that...
<wgrant> I'm not sure if it's even possible.
<lifeless> wgrant: security.py isn't part of the ddl scripts sequence
<lifeless> wgrant: it runs after
<wgrant> lifeless: True.
<wgrant> But it's still not really feasible to detect.
<lifeless> poolie: on b.l.n?
<poolie> yes
<lifeless> its quite nice
<lifeless> I really want to ditch the b.l.n domain - incorporate loggerhead as routes within c.l.n.
<lifeless> poolie: your timeout was an 11 second count
<lifeless> poolie: something bong there
<lifeless> poolie: given that there are only 4 matching bugs
<wallyworld> poolie: bug 735202
<wallyworld> it gives a 404 as expected when i try it locally
<wallyworld> not sure why
<poolie> http://pad.lv/735202
<wallyworld> it works locally but fails on lp.net
<wallyworld> perhaps someone already fixed it and it's not deployed yet?
<poolie> are you sure you're using the same data situation?
<wallyworld> i try and access a private bug
<wallyworld> it gives 404
<lifeless> wallyworld: using launchpadlib
<wallyworld> when i mark it non-private, it returns the data
<wallyworld> lifeless: i used curl to api.lp.dev like in the bug report
<wgrant> wallyworld: Heh, that changed this morning.
<lifeless> kk
<wgrant> As of a few hours ago they 404.
<wgrant> I should probably take that bug.
<wallyworld> poolie: ^^^^^^^^^^^^^^
<wallyworld> wgrant: i guess that may have been the case
<wgrant> It was an unrelated change.
<poolie> oh great
<wgrant> Well, sort of related.
 * wallyworld looks for another bug to fix
<wgrant> But not raelly.
<lifeless> deleting code good
<lifeless> only another 300KLOC to go
<wgrant> It's more than that.
<lifeless> wgrant: lp ? yes, I don't want to delete lp. Just the fat.
<wgrant> Heh.
<wgrant> ~/launchpad/lp-branches/clean-devel$ find -type f -name '*.py' | xargs wc -l | tail -n 1 605989 total
<wgrant> That's more than I expected :/
<poolie> wallyworld, i filed a couple of similar ones
<wallyworld> poolie: ok
<stub> Urgh.... feels like the beginnings of tonsillitis :-(
<lifeless> stub: :(
<adeuring> good morning
<jtv> hi adeuring
<adeuring> hi jtv!
<jtv> Do we have any reviewers here who are very familiar with feature flags?
<jtv> lifeless: I thought stub can set feature flags?
<stub> Power, yes. Understanding?
<jtv> Na, who cares about understanding
<jtv> Just the thing I needed to be saying at the exact point my boss walked into the room, at the opening of review seasonâ¦
<jtv> stub: I've got a patch up for review that, among other things, lets us disable scripts by feature flag (and the feature would also be suitable for tweaking log levels later).  I thought that would let us reduce losa interrupts, but according to lifeless it doesn't help much in that regard.
 * bigjools waves at jtv
<jtv> so you saw that then :)
 * bigjools averts eyes
<jtv> jml, are you into feature flags in a big way?  I need a reviewer for https://code.launchpad.net/~jtv/launchpad/bug-735319/+merge/53578
<jtv> (Or if not a review, at least a tie-breakerâsee comments :)
<poolie> thanks for tackling that jtv
<poolie> i'll have a look too
<jtv> oh thanks
<jtv> bigjools: I'd be a lot more comfortable with rewriting cron.publish-ftpmaster knowing that it was tested
<bigjools> jtv: chicken/egg
<bigjools> we can do manual testing easily on DF
<jtv> But sounds like the first thing to do is figure out what it does, and write tests for that.
<wgrant> "easily
<wgrant> "
<bigjools> yes, easily
<bigjools> wgrant is so negative!
<jtv> Given how people tend to say "trivial" for things that turn out to be a lot of work, "easy" really worries me as well.
<bigjools> it's all relative jtv :)
<bigjools> jtv: I can do a pre-imp about it with you
<jtv> Oh nice, a pre-imp sprint.  Is next week good for you?
<bigjools> only if you come here :)
<lifeless> jtv: policy for FF changes is losas do
<jtv> oic
<lifeless> jtv: and, as I say its overloaded with the non-db style of killing, and we *must not* attempt a db-style disable during db-deploys.
<jtv> OK, I'll take that bit out then.  Thanks for pointing it out.
<lifeless> thanks
<bigjools> lifeless: what makes bzr do a repack when I pull?
<bigjools> and can I put it off to a time when it's not inconvenient to wait 30 minutes? :)
<poolie> bigjools, if your local repository passes a certain size threshhold
<poolie> urk
<bigjools> ah thanks poolie
<wgrant> bigjools: Mine has done that twice today :(
<bigjools> I might file a bug about this, I think it should prompt/nag
<lifeless> meep - 665  OOPS-1900K1319  Person:+contactuser
<lifeless> bigjools: its an exponential backoff
<lifeless> bigjools: 30 minute ones should be -extremely- rare, but you can make sure it never happens by putting 'bzr pack' into cron
<lifeless> we could/should also permit backgrounding it automatically - packs don't block writers
<lifeless> *packing does not*
<bigjools> 30 mins might be exaggerating, but the last one on my desktop took 15-20, and I am now waiting for dogfood which has slow disks :(
<wgrant> s/disks/everything/
<bigjools> backgrounding/nag/prompt all good :)
<wgrant> stub: Thanks.
 * bigjools filez
<bigjools> Gnome apps really stand out on my desktop, they're the only ones that don't restore their state and throw a million error dialogs.
<wgrant> GNOME session saving used to work great.
<wgrant> I think I last used it in 2002, though..
<bigjools> I know :/
<bigjools> I think I last used gnome in 2002 :)
<wgrant> Hah.
<bigjools> apart from you watching me flinch at that UDS when I tried it
<wgrant> Yes, that was amusing.
 * bigjools installed KDE4.6 yesterday, not a Unity in sight :)
<StevenK> Yes, we know KDE isn't unified.
<bigjools> bwahaha, you've seen the Gnome project? :)
<StevenK> I was hoping you weren't going to point out the irony, but you did ...
<wgrant> Unity has only crashed once this week!
<bigjools> sorry, I wasn't aware Australians did irony
<wgrant> And I actually quite like it now.
<StevenK> bigjools: Isn't that some kind of metal?
<poolie> jtv, ok, done
<bigjools> StevenK: it's what your wife does
<poolie> and so, goodnight
<bigjools> nn poolie
<poolie> if China wants to buy irony we'll sell it
<StevenK> I can see this conversation going nowhere good.
<poolie> oh, Blacktown? :)
<bigjools> West Sydney
<StevenK>  /ragequit
<bigjools> lmao
<poolie> :-P
<poolie> good night gentlemen
<bigjools> lifeless, poolie (if you're around), my bzr pull took 24 minutes :)
<bigjools> I filed a bug anyway
<lifeless> cool
<lifeless> I filed 22 :P
<lifeless> night all
<wgrant> lifeless: :(
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #457: FIXED in 4 hr 19 min: https://hudson.wedontsleep.org/job/db-devel/457/
<wgrant> jtv: Want to review https://code.launchpad.net/~wgrant/launchpad/bug-715236/+merge/53576?
<wgrant> Almost as trivial as they get.
<StevenK> wgrant: r=me
<jml> how did we get 18 new critical bugs over night?
<StevenK> jml: One word: lifeless
<wgrant> lifeless filed a bug for all the timeouts.
<wgrant> StevenK: Thanks.
<jml> do we have new timeouts?
<wgrant> jtv: We now have a full list.
<wgrant> Er, jml
<jml> meh. I've been told that before
<wgrant> jml: The OOPS reports used to only have the top 10, but lifeless fixed that.
<jml> wgrant: ok. that gives some grounds for confidence.
<wgrant> jml: So, we will still have missed some that only occur a couple of times a week, but there probably aren't too many of those.
<wgrant> Hopefully.
<jml> it's been so exciting watching the number of critical bugs go down this last week
<wgrant> I've landed 10 fixes today... so we are only up by about 10.
<jml> \o/
<adeuring> jtv: could you please review a small MP: https://code.launchpad.net/~adeuring/launchpad/translation-branch-sync-return-to-referrer/+merge/53597 ?
<jtv> adeuring: otp
<adeuring> jtv: ?
<jtv> on the phone
<jml> bug 12345
<_mup_> Bug #12345: isdn does not work, fritz avm (pnp?) <isdnutils (Ubuntu):Fix Released by doko> < https://launchpad.net/bugs/12345 >
<bigjools> isdn!
<jtv> adeuring: I'm backâ¦ still need that review?
<adeuring> jtv: yes please
<jml> can I url hack to change the batch size?
<bigjools> jml: yes, lots of people do
<jml> how do I do it?
<bigjools> don't ask me to remember that sort of thing :)
<jml> heh
<jml> well, my immediate need has passed.
<henninge> adeuring: Hi!
<adeuring> hi henninge
<jtv> adeuring: you have been reviewed
<henninge> adeuring: Is that the ReturnToReferrer branch you are having reviewed?
<bigjools> jml: lib/canonical/launchpad/doc/batch_navigation.txt
<henninge> Hi jtv!
* jtv changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | firefighting: - | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews/
<jml> bigjools: ta
<jtv> henninge: have had.
<bigjools> jml: fuck me, documentation :)
<adeuring> henninge: well, jtv said so ;)
<adeuring> and thanks, jtv!
<henninge> adeuring: I put a card for that on the board but I'll leave the satisfaction of moving it to you ... ;-)
<bigjools> basically ?batch=N
<henninge> adeuring: and added bug 735960
<_mup_> Bug #735960: Return to translation sharing details page <Launchpad itself:In Progress by adeuring> < https://launchpad.net/bugs/735960 >
<adeuring> henninge: thanks for doing my bureaucracy :)
<henninge> adeuring: I know how you loath^Wlove it.
<jml> bigjools: thanks.
<jml> looks like it's batch=FOO
<bigjools> that's what I said :)
<jml> bigjools: oops, I missed that.
<jml> the standard library should have a tzinfo object for UTC.
<deryck> Morning, all.
<bigjools> jml: pytz.UTC ?
<jml> bigjools: yeah, I'll use that, but it should be in the stdlib
<bigjools> why do you think that?
<jml> the Python documentation even has an example UTC object
<jml> bigjools: because it would be convenient for many people, and doesn't have the ridiculous politically-imposed maintenance cost of other timezones
<jml> (and because stacks of methods in datetime mention utc explicitly)
<bigjools> fair reasons
<jml> and, also,
<jml> there's an example in the documentation! so they have to maintain the code anyway.
<bigjools> heh
<jml> http://paste.ubuntu.com/581050/ <- crit bug breakdown
<wgrant> launchpad-buildd? Really?
<wgrant> Huh, indeed.
<leonardr> can someone point me to a real-life example of a bugtask with a conjoined master? i'm trying to qa bug 556515
<_mup_> Bug #556515: OOPS when editing conjoined bugtasks via API <dhrb> <lp-bugs> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/556515 >
<wgrant> leonardr: Anything on https://bugs.qastaging.launchpad.net/ubuntu/natty should do.
<leonardr> wgrant: any idea why this happens?
<leonardr> >>> series
<leonardr> <distro_series at https://api.qastaging.launchpad.net/1.0/ubuntu/natty>
<leonardr> >>> len(series.searchTasks())
<leonardr> 0
<wgrant> Hee hee hee.
<wgrant> Try omit_targetted=False, or something like that.
<wgrant> There's a bug.
<StevenK> Just one?
<wgrant> I can't find it.
<wgrant> But I didn't really try very hard.
<wgrant> Bug #320596
<_mup_> Bug #320596: Series.searchTasks() always returns an empty collection <api> <lp-bugs> <qa-ok> <ubuntu-qa> <Launchpad itself:Fix Released by brian-murray> < https://launchpad.net/bugs/320596 >
<wgrant> But it's meant to default to False.
<wgrant> Hm.
<leonardr> ok, now the proble
<leonardr> m is i don't have permission to edit any of these tasks
<wgrant> leonardr: You can set some things...
<wgrant> leonardr: Anyone can set a bug from Triaged to New, for example.
<wgrant> I can do just about anything, though, if you need more testing.
<leonardr> wgrant: i think i just need you to answer my dumb questions about how these bugs work
<leonardr> is this task conjoined? https://api.qastaging.launchpad.net/1.0/ubuntu/natty/+source/nvidia-graphics-drivers/+bug/711409
<_mup_> Bug #711409: [MASTER] -nvidia broken after Jan 31st updates, because it does not yet support xserver 1.10 <i386> <natty> <nvidia> <ubuntu> <xorg> <nvidia-graphics-drivers (Ubuntu):Fix Released> <nvidia-graphics-drivers (Ubuntu Natty):Fix Released> < https://launchpad.net/bugs/711409 >
<leonardr> i was able to edit its status
<leonardr> _mup_ says that's the master
<wgrant> leonardr: That's a conjoined slave.
<wgrant> [MASTER] is in the summary.
<wgrant> It doesn't indicate that it's a conjoined master.
<leonardr> ah
<leonardr> ok, so, i was able to edit a conjoined slave, which afaict shouldn't happen
<wgrant> Er, wait.
<wgrant> That is a *master*.
<wgrant> Confusing terminology is confusing.
<wgrant> The series task is the master, the distribution task is the slave.
<wgrant> https://api.qastaging.launchpad.net/1.0/ubuntu/+source/nvidia-graphics-drivers/+bug/711409 (no natty) is the slave.
<_mup_> Bug #711409: [MASTER] -nvidia broken after Jan 31st updates, because it does not yet support xserver 1.10 <i386> <natty> <nvidia> <ubuntu> <xorg> <nvidia-graphics-drivers (Ubuntu):Fix Released> <nvidia-graphics-drivers (Ubuntu Natty):Fix Released> < https://launchpad.net/bugs/711409 >
<leonardr> i was also able to edit the slave
<leonardr> let's see what happens on the website
<wgrant> Erk.
<wgrant> The slave's details are hidden from the web UI, but you can see them through the API.
<leonardr> yeah, it says 'status tracked in natty'
<leonardr> ok, got it!
* abentley changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | firefighting: - | On call reviewer: abentley | https://code.launchpad.net/launchpad-project/+activereviews/
<deryck> jml: ping
<jml> deryck: pong
<deryck> henninge: ping for standup
<henninge> deryck: oh yeah, coming
<henninge> adeuring: for your search: files are placed in the import queue with a call to addOrUpdateEntry(FromTarball)
<adeuring> henninge: thanks
<henninge> abentley: I guess the job creation is not hidden behind a feature flag?
<henninge> Should it be?
<henninge> Is that even doable?
<abentley> henninge: It's not.  I don't think it should be.
<henninge> abentley: are the jobs being processed now on production?
<abentley> henninge: Not yet, we need a config change.
<henninge> abentley: https://code.launchpad.net/~henninge/launchpad/devel-735954-merge-job-display/+merge/53629
<deryck> abentley: sorry it's taken me so long.  A few conversations developed.  But I have a good head around all the mocking we've done so far....
<deryck> abentley: however, my call with henninge is in 10 minutes.  Shall we chat after that?
<abentley> deryck: no worries.
<abentley> deryck: after henninge's call makes sense.
<deryck> abentley: great, thanks.  Chat soon then.
<vila> jam, jelmer: care to join mumble to test my new setup ?
<vila> meh wrong channel
<deryck> henninge: coming.... sorry.  1-2 more minutes and I'll join you.
 * henninge hurries to get there first ...
<deryck> heh
<jcsackett> deryck: i need some javascript help, and you seem to be the guy to talk to. :-)
<jcsackett> got a few minutes?
<jcsackett> oops, your otp.
<deryck> jcsackett: yeah, call, sorry.
<jcsackett> deryck: no worries, i should have been paying attention. :-P
<deryck> jcsackett: np, really.  so you're right.  I am *the* guy ;)
<deryck> jcsackett: however, you need to take a js-number :-)
<deryck> jcsackett: I have a call with Aaron about js, and then we can chat :-)
<jcsackett> deryck: sounds good. thanks. :-)
<deryck> abentley: shall we chat now?  Mumble?
<abentley> deryck: sure.
<abentley> deryck: http://pastebin.ubuntu.com/581151/
<deryck> abentley: so this works as I thought -- variable names are determined by var keyword.  function names are bound to the closure.  Which is why I was confused the first pass of the discussion. :-)
<deryck> abentley: so a named function outside of a YUI block would be global, but because YUI blocks are closures, named functions are private to the closure.
<abentley> deryck: Is var foo = function() equivalent to function foo() then?
<deryck> abentley: inside a YUI block, basically yes.
<deryck> abentley: it may be a bit slower to execute, but I can't say that for sure.  I would avoid assigning to var names inside yui.
<jcsackett> deryck: i'm unavailable for the next hour, so don't worry about me for a bit if your chat with abentley ends before then. :-)
<abentley> deryck: okay.
<deryck> jcsackett: it ended 5 minutes ago, but I had the above follow up hacking. ^^  I'm done now, though.
<deryck> jcsackett: but I assume we'll chat later?
<jcsackett> deryck: yup. will you be free in about an hour?
<deryck> jcsackett: actually, no.  I have team lead call top of the next hour.  After that?
<jcsackett> deryck: sounds good.
<deryck> jcsackett: ok, chat with you then
<henninge> abentley: I replied to your review and pushed a new version. Thanks a lot. I have to leave now but I hope you can approve this now.
<abentley> henninge: Okay, I'll have a look.
<abentley> henninge: r=me
<henninge> abentley: thanks a lot!
<abentley> deryck[lunch]: could you comment on whether the YUI tests here should be Windmill tests? https://code.launchpad.net/~wallyworld/launchpad/inline-multicheckbox-widget/+merge/52943 (644-827)
<bac> hi sinzui
<sinzui> hi bac
<deryck> abentley: just got off lunch and now tl call.  I'll look closely shortly.
<deryck> abentley: I think that's a good YUI test.
<abentley> deryck: Thanks for checking.
<deryck> abentley: wallyworld had pinged me out of review about mocking, and I don't think we need a custom object there.  I think he can use Y.Mock to simulate the patch request.  Unless I'm missing something about Y.Mock.
<deryck> abentley: but I'll reply to his mail to me about that.  Unless he sees this chat first ;)
<lifeless> hmm
<lifeless> slow librarian performance
<lifeless> flacoste: I'm going to drop to 11 seconds today if the oops report looks good
<lifeless> https://lpstats.canonical.com/graphs/OopsLpnetHourly/20110315/20110316/ suggests it will
<lifeless> flacoste: if this concerns you , speak up now :) - we will have a full days data without index problems etc to work on
<bigjools> lifeless: I am concerned about distrseries:+queue
<lifeless> 3 /   10  DistroSeries:+queue ?
<bigjools> it will become a bigger problem the closer we get to natty release
<bigjools> it's not used much *now* but it will get more use
<lifeless> bigjools: if it starts to spike, raise its timeout
<jcsackett> deryck: you available?
 * bigjools gets the same complaints every 6 months
<lifeless> bigjools: ideally we'd fix it before then
<bigjools> lifeless: that's why I am mentioning it now, it'd be great if you could look :)
<lifeless> I think wgrant has been, I'll see if he's still doing so
<deryck> jcsackett: oh, hello there again. ;)  sure.  mumble?
<jcsackett> deryck: sure.
<lifeless> I have been ignoring my ta stuff this last 5-6 work days and just going out and killing timeouts... I suspect I rather need to buckle up and focus :>
<bigjools> lifeless: also, we'd like your input on doing background page updates via JS for some derived distros work,  rvba will grab you tomorrow (his time) which is later today for you if you're about
<lifeless> sure
<lifeless> you're thinking poll for now ?
<rvba> yes
<bigjools> lifeless: yes, same as the ppa page
<lifeless> seems no worse than we have now
<bigjools> but I wondered if thumper's work had changed any of that
<lifeless> not AFAIK
<bigjools> it would only poll until some state changes
<lifeless> sure
<lifeless> we can tolerate limited amounts of polling
<lifeless> when we get a callback mechanism we'll need to roll it out $everywhere anyhow
<bigjools> the other question is, exactly how much non-js fallback are we worried about these days
<lifeless> + I don't expect derived distros to be huge (like e.g. bugs - 50% of our web pages)
<bigjools> there's only one page that has JS but it has a lot of JS :)
<lifeless> bigjools: so, google have ways to index js only content
<bigjools> I can't remember why we wanted to do non-js fallback anyway
<lifeless> I would talk to the stakeholders for this project
<lifeless> w3m users :>
 * bigjools has a 4 letter word forming
<lifeless> bigjools: anyhow, all sounds fine to me; jml / the derived distro stakeholders are the ones to check about non-js support: its not a technical choice, but a political one :)
<bigjools> yes exactly
<bigjools> unless you count things like not being able to file bugs :)
<lifeless> we fixed that :)
<lifeless> \o/
<lifeless> POFile:+translate is down by ~3 seconds on its 99th percentile
<jml> bigjools: for the bug subscription work, we are deliberately doing js only. (at least, as of last time I spoke about it)
<bigjools> jml: I think that's sensible
<jml> bigjools: we don't have a general policy yet (we will soon), but if you want to proceed assuming that you don't need js fallback, I think that would be fine.
<bigjools> although it does mean I need to learn JS again
<bigjools> cool, thanks for clarifying
<jml> bigjools: hey, deryck linked to a thing about that :)
<deryck> JavaScript for people who know Python?  I recommended that to abentley too, who found it useful.
<bigjools> what's the link?
<bigjools> or should I JFGI :)
<deryck> bigjools: http://pycon.blip.tv/file/4882883/
<deryck> heh, no I didn't mind :-)
<bigjools> ah cool, thanks deryck
<deryck> bigjools: np
<deryck> bigjools: and then read the YUI docs, please :-)
<bigjools> ha :)
<bigjools> oh 30 min video, I was hoping I'd know everything there is to know in 5 :)
<deryck> if only :-)
<bigjools> right, dinner's ready, good night all
<jml> bigjools: perhaps we should arrange another two week training course?
<bigjools> jml: **** *** *** *** *****
<jml> :D
<jml> bigjools: g'night
<bigjools> ciao!
<flacoste> lifeless: like i mentioned to you on Monday, i'm fine with that timeout drop (and thought it already happened, fwiw)
<lifeless> flacoste: we dropped to 12
<lifeless> flacoste: I wanted a clean, good signal before dropping to 11
<jml> 11 is pretty exciting.
<deryck> it goes to 11!
<jml> it feels like not so long ago we were at 30
<lifeless> flacoste: I was sure you'd be ok, I wanted to make sure you were not caught unawares *if* something goes wrong
<flacoste> 11! right!
<flacoste> that was the follow-up reduction you talked about
<flacoste> still good with it
<lifeless> yes
<lifeless> + a lowering of soft timeout to 9
<flacoste> exactly
<lifeless> [if we drop hard to 11]
<deryck> Why not make 10 louder?  And call that 11.
<flacoste> deryck: i didn't know you were making amps in your spare time ;-)
<deryck> heh
<deryck> side business
<jml> lifeless: I'm thinking again about how much I dislike layers and how much I'd like to use a non-Zope test runner.
<lifeless> jml: \o/
<jml> lifeless: I am thinking further of making a TestSuite that runs tests with layers properly
<jml> as in, doing what z.testrunner does but in a more compatible way.
<lifeless> that might be interesting
<jml> it seems easier than moving away from layers
<lifeless> jml: given that we're pretty close on fixtures, it might be better to just move on that
<lifeless> jml: I don't think compatibility is the issue with layers, its hte contract: the reinvoking stuff, for instance
<jml> lifeless: it's the compatibility issue that's tying us to test.py
<lifeless> jml: yes
<lifeless> jml: erm
<lifeless> jml: no, its not
<jml> lifeless: well, you'll have to explain why. also, I don't see a clear path from "improve fixtures" (presumably wrt the graphing stuff) to "run Launchpad tests without z.testing".
<lifeless> jml: there are several different things hanging around here
<lifeless> test.py
<lifeless> z.testing
<lifeless> requiring reinvoking to do ZCA testing
<lifeless> the graph of dependencies is: reinvoking requires zope.testing requires a test.py script
<lifeless> it is trivial to write a TestSuite wrapper than will obey a limited subset of the layers protcol - excluding reinvocation
<jml> lifeless: I don't see why re-invocation would be that hard.
<lifeless> so, (ignoring the bits of zope.testing that are harnesses for other bits of zope we use) to stop using zope.testing we need to fix the behaviour of our layers
<lifeless> jml: let me ask you a different question. What problem are you trying to solve?
<jml> (although I agree it would be better to not need to do it)
<jml> lifeless: I am interested in removing friction from my development process.
<lifeless> jml: The highest priority problem I see on our testing framework list is reinvocation - it costs substantial time and increases peak memory requirements a lot
<jml> ok
<flacoste> lifeless: you are talking about the support when NotImplementerError is raised in a layer teardDown method?
<jml> oops. late for something.
 * jml goes
<lifeless> flacoste: yes
<lifeless> flacoste: we have no good reason for raising that - its just a matter of code to fix the causes
<flacoste> lifeless: agreed, especially since upstream isn't using it much anymore (at least not as part of the default ZCA tests)
<sinzui> did someone delete ~launchpad-chr on late Friday or Saturday?
<lifeless> matsubara did
 * matsubara whistles awaya
<matsubara> sinzui, hopefully that didn't break anything. AFAICT, it was used as an answer contact for  launchpad
<sinzui> I think that is the event that broke team participation. I think merge with ~registry caused confusion in team participation. ~launchpad was a direct member of both teams
<matsubara> sinzui, ah, ok. so it's a known bug but you wanted to know the root cause for it to be triggered?
<sinzui> I still think we can fix immediate issue with a stealthy add/remove member combination. I just need to think which teams are involved
<sinzui> I did not know that merge would corrupt TP. That is a new bug. I am trust trying to see my participation page (bug 733881)
<_mup_> Bug #733881: +participation oops because membership and teamparticipation disagree <oops> <teams> <Launchpad itself:In Progress by sinzui> < https://launchpad.net/bugs/733881 >
<matsubara> sinzui, not sure if it helps, but I have a list of ~launchpad-chr members before I deleted
<sinzui> matsubara: I can see them on staging
<matsubara> :-)
<sinzui> I think I will co-opt my call with jcsackett to discussing our mutual dislike of TeamParticipation and Person.merge
<sinzui> jcsackett: I think you should read https://bugs.launchpad.net/launchpad/+bug/733881 so that we can discuss it in 10 minutes
<_mup_> Bug #733881: +participation oops because membership and teamparticipation disagree <oops> <teams> <Launchpad itself:In Progress by sinzui> < https://launchpad.net/bugs/733881 >
<sinzui> matsubara: you used the delete team link right? Is this the message you saw: https://staging.launchpad.net/~launchpad-chr/+delete
<sinzui> I wonder is delete handled pending members...well merge does not
<matsubara> sinzui, yes, that's the message I saw.
<matsubara> it took a couple of tries because the action timed out but eventually it did delete
<sinzui> thanks. I will use staging then to diagnose what merge did wrong
<jcsackett> sinzui: i already saw that bug on the tl report.
<sinzui> jcsackett: I updated it a few minutes ago with what I see. I want to discuss the hack to make Lp put the data back, and the root cause in delete/merge
<jcsackett> i hate TP.
<jcsackett> i sort of dislike the whole team infrastructure, at this point. :-P
<sinzui> TeamParticipation is easy to hate, but combined with merge rules, I think I have disconcerting feeling of both despair and delight.
<maxb> Chex: Hi - regarding that question you just replied to - the problem is that Launchpad's code import service is broken in respect of SourceForge Mercurial repositories.
<maxb> A third party proxying solution to assist the Launchpad code import service seems a bit of a weird solution, so I'm wondering if you misread the question at all?
<Chex> maxb: ok, so its not just a matter of post issues, then?
<maxb> post? port?
<Chex> s/post/port/ sorry
<maxb> SourceForge have chosen to use a non-standard port.
<maxb> It is of course up to Launchpad to potentially refuse to support it, but it seems like an unlikely decision to make, given how popular SourceForge is
<maxb> Actually, is there any concrete reason why the importds do not have unrestricted outbound TCP access to the internet?
<lifeless> maxb: they did what?
<lifeless> maxb: we don't trust vcs libraries to be bug tree and have no security vulnerabilities
<maxb> who did what? SourceForge? They have chosen to run their Mercurial hosting service on a non-standard TCP port
<Chex> maxb: its just a general policy we have for only allowing what would be typically used, for security
<Chex> lifeless: for reference to what we are talking about: https://answers.launchpad.net/launchpad/+question/147288
<maxb> inbound, totally. outbound, I guess my standards are more heavily pragmatic :-)
<lifeless> Chex: need an rt for it? I think its fine to open it to sf
<Chex> lifeless: I already have one, and the response was for a HTTP Proxy to be setup for this situation.
<lifeless> Chex: thats crazy
<lifeless> Chex: a) it would require significant special casing in the code import infrastructure
<lifeless> Chex: b) it would permit trivial bypassing of our security policy (so we may as well just disable the outbound firewall if we were to do that)
<Chex> lifeless: well, I can see your point on that
<lifeless> Chex: whats the rt ?
<Chex> lifeless: so we already have a HTTP proxy setup, we would just configure this address to point to the remote port, outgoing connection for you
<Chex> lifeless: RT# 44626
<lifeless> Chex: I don't see any benefit in using an http proxy for hg
<lifeless> Chex: its all streaming data
<lifeless> Chex: I've commented (just now) on the question
<lifeless> Chex: I will do so on the rt too
<Chex> lifeless: ok, great, thanks
<leonardr> thumper, wallyworld: it would be great to get a review of https://code.launchpad.net/~leonardr/lazr.restful/entry-introduced-in-version/+merge/53704 sometime today
<leonardr> i'm eod but i'll stick around if one of you wants to take it and ask quESTIONS
<leonardr> or, abentley, if you want to take it
<wallyworld> leonardr: i'll look but thumper will need to +1
<abentley> leonardr: I'm EOD.
* abentley changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | firefighting: - | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews/
<thumper> wallyworld: http://blogs.jetbrains.com/pycharm/2011/03/python-ides-panel-video-from-pycon-2011/?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+Pycharm+%28JetBrains+PyCharm+Blog%29
<thumper> leonardr: mumble?
<leonardr> thumper, 1 sec
<sinzui> wgrant: mumble>
<sinzui> mumble?
<jcsackett> exit
<wgrant> sinzui: Oh, right, half an hour earlier.
<wgrant> Give me a sec.
<sinzui> jcsackett: I think Person.deactivateAllMembers() is the issue, which is not really merge or the remove super team rule. It implements a separate TP cleanup
<sinzui> And it does not handle recursion well
<jcsackett> sinzui: it was at fault once before, but i thought we had cleaned it up.
<jcsackett> this is more of the "we have too many ways to do the same thing."
<sinzui> I recalled we wanted to unify the code, We have not. I think now is the time
<jcsackett> sinzui: yeah, we need one blessed way to handle TeamParticipation.
<sinzui> I can see that it will not know what to do in the case of multiple paths to a team
<jcsackett> sinzui: we don't handle the multiple paths issue well anywhere. at best, we constrain things to where we ignore all but one path.
<sinzui> This gives me hope because the fix looks like deleting code!
<jcsackett> sinzui: did Edwin finish the _cleanParticipation work he was doing before he left?
<jcsackett> he was working on making that the Right Way (TM), and if he did, we want to use that.
<jcsackett> since he knew better than anyone on registry what needed doing there.
<sinzui> TM's _cleanTeamParticipation does a recursive check to build a list of excludes. So I have some faith that wne the method tries to do the right thing, it will work deactivateAllMembers() is simple delete of everything
<wgrant> (the blessed way is probably triggers)
<jcsackett> wgrant: triggers were once a problem for us as well.
<jcsackett> or rather, they complicated diagnosing the problem.
<jcsackett> basically, management of TeamParticipation is a mess. :-P
<mwhudson> neo4j!
 * mwhudson hides
<wgrant> jcsackett: That's why we should do it in one place that catches everything.
<jcsackett> wgrant: i concur. we're on the same page here. i'm -1 on triggers, personally; having things fire off post changes with teams have failed us. i think adopting widespread use of Edwin's work is the way to go.
<jcsackett> but i'm +1 on whatever, as long as we're consistent with it and cover all our bases.
<wgrant> We already have a set of triggers to do exactly this sort of graph work, for packagesets.
<jcsackett> sinzui ^
<jcsackett> that i did not know.
<lifeless> I'm neutral on triggers
<lifeless> we use them
<lifeless> but htey have significant downsides
<lifeless> not the least of which is causing *more* db roundtrips
<poolie> wgrant, was the bug you fixed recently one that can cause
<poolie> 'SimpleViewClass from /srv/launchpad.net/production' object has no attribute 'status'
<poolie> ?
<wgrant> poolie: No, that's a lazr.restful bug.
<wgrant> Which I believe is fixed now.
<wgrant> poolie: When did you last see that/
<poolie> OOPS-1901C853
<poolie> overnight, from a cron job
<wgrant> poolie: I think that was probably fixed by the first deployment this morning.
<wgrant> I've been seeing a few each night too.
<wgrant> It's lazr.restful failing to render a timeout error.
<poolie> so i should just see if it goes away?
<wgrant> Yes.
<poolie> no need to file?
<poolie> ok
<wgrant> Bug 733293
<_mup_> Bug #733293: API fails to render timeout errors <api> <qa-ok> <regression> <Launchpad itself:Fix Released by leonardr> <lazr.restful:Fix Released> < https://launchpad.net/bugs/733293 >
<poolie> i thought it was familiar
<poolie> thanks
<poolie> biab
<wgrant> lifeless: https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1901C2059
<wgrant> SQL time: 766 ms
<wgrant> Non-sql time: 5938 ms
<lifeless> wgrant: yeah, get a profile from qas
<lifeless> I suspect its parsing or something similar
<wgrant> Ah, forgot we could do that easily now.
<lifeless> jml: https://dev.launchpad.net/LEP/ReliableDBDeploys when you have time
<lifeless> flacoste: ^
<lifeless> whee, http://webnumbr.com/.join(launchpad-oops-bugs.all,launchpad-timeout-bugs.all,launchpad-critical-bugs.all) really has quite a jump doesn't it
#launchpad-dev 2011-03-17
<lifeless> do we still use Pygpgme ?
<lifeless> wgrant: https://bugs.launchpad.net/launchpad/+bug/184835
<_mup_> Bug #184835: Add status '9' (which equates to WONTFIX) to the Roundup external bug tracker <bugwatch> <lp-bugs> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/184835 >
<lifeless> wgrant: is that fixed incidentally?
<wgrant> lifeless: No, but it's no longer an OOPS, so it's Low.
<lifeless> wgrant: what about https://bugs.launchpad.net/launchpad/+bug/125068 ?
<_mup_> Bug #125068: Bugzilla bug watch updater crashes on POSTs that return a HTTP error <bugwatch> <lp-bugs> <oops> <story-reliable-bug-syncing> <Launchpad itself:Triaged> < https://launchpad.net/bugs/125068 >
<wgrant> lifeless: Some of those are fixed, others are not.
<lifeless> I wonder if leonardr's work has fixed https://bugs.launchpad.net/launchpad/+bug/106338 ?
<_mup_> Bug #106338: Editing a bug targeted to a release crashes if you directly edit the untargeted task  <lp-bugs> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/106338 >
<wgrant> I need to check waht still shows up.
<lifeless> he says it hasn't fixed it
<lifeless> but I wonder if we're lucky
<wgrant> Indeed.
<wgrant> He tested it.
<wgrant> And said it wasn't fixed.
<lifeless> wgrant: hey - https://bugs.launchpad.net/launchpad-buildd/+bug/497772 - would love to get bm crashes oopsing
<_mup_> Bug #497772: exceptions.AttributeError: 'DebianBuildManager' object has no attribute '_subprocess' <oops> <Launchpad Auto Build System:Triaged> < https://launchpad.net/bugs/497772 >
<wgrant> lifeless: So, roughly, "fuck no"
<wgrant> Generating OOPSes from user-controlled code == bad
<wgrant> It's also really hard, since there is no access to the slaves except through lp-buildd, and that's what's broken here.
<lifeless> wgrant: uhm, thats not what I was saying
<lifeless> wgrant: if the build manager blows up, it should write an oops.
<lifeless> ditto lp-buildd
<wgrant> Where should lp-buildd write an OOPS to?
<lifeless> if the *build* fails, thats not lp-buildd or the build manager blowing up
<lifeless> wgrant: hand it off to the bm
<wgrant> And how do we know a user isn't going to run their own lp-buildd and generate millions of OOPSes?
<lifeless> wgrant: can they talk to the build master?
<wgrant> lifeless: I can easily hijack a buildd and get my own lp-buildd to run there.
<lifeless> wgrant: on the host os?
<wgrant> lifeless: Outside the chroot, but inside the VM.
<lifeless> wgrant: isn't that on a different IP ?
<wgrant> lifeless: lp-buildd runs inside the VM.
<wgrant> We don't talk to the host at all.
<lifeless> *blink*
<wgrant> We don't know that the host exists.
<lifeless> ok
<lifeless> uhm
<wgrant> We just know that there is a command that we can run to reset the VM.
<wgrant> (because we will need to use alternative virt solutions in the future *cough* ARM *cough*, where there is no Linux on the host)
<lifeless> you have a good scary story there
<lifeless> but
<lifeless> if we get an oops from lp-buildd, its time to kill the build
<lifeless> that prevents millions of oopses from one instance
<wgrant> What does killing the build mean?
<wgrant> Killing the build does not kill lp-buildd.
<lifeless> wgrant: AIUI we restart the buildd per build because of precisely the trust issues you allude to
<lifeless> wgrant: so I mean that
<wgrant> lifeless: At the moment we only restart it when the next build starts. But that is itself a security issue, so we should probably fix that.
<wgrant> There seem to be three different ways an externalbugtracker can indicate that the importance is unable to be converted :(
<lifeless> \o/
<wgrant> lifeless: Bug #422960 is cheating.
<_mup_> Bug #422960: appear to be failing to record oops for all +translate HTTP 503 errors <canonical-losa-lp> <lp-foundations> <Launchpad itself:Triaged> < https://launchpad.net/bugs/422960 >
<lifeless> wgrant: I disagree :)
<lifeless> wgrant: it falls into the bucket of a bunch of operational improvements that we should do, IMO
<wgrant> Maybe.
<wgrant> Almost OOPS report time.
<lifeless> yeah
<lifeless> almost 11 second timeout time
<wgrant> Exactly.
<wgrant> What did we start at?
<wgrant> 16?
<lifeless> 20
<lifeless> 17 on edge
<lifeless> step 1 was making both 17
<wgrant> I was going to say 17, but I thought that seemed a bit odd.
<lifeless> since then we've ratched as we fixed things
<lifeless> edge was lower 'to catch slow pages' - but has a different user population, so not very effective at that
<StevenK> And edge still isn't dead. :-(
<lifeless> its in the queue
<wgrant> But at least it no longer has a shorter queue.
<wgrant> lifeless: Your gardening has nearly been successful.
<wgrant> But not quite :(
<StevenK> I'd like to rewrite my Firefox URL suggestions, since most of those tend to be edge URLs
<lifeless> wgrant: thats just the first 75
<lifeless> StevenK: sqlite3
<lifeless> StevenK: update ...
<lifeless> where ...
<StevenK> lifeless: Er, which of the 4,000,000 sqlite databases? :-)
<wgrant> places
<wallyworld> thumper: got a sec on mumble?
<thumper> sure
<StevenK> sqlite> select count(*) from moz_places where url like '%edge%';
<StevenK> 163
<StevenK> :-(
<wgrant> gnarrrrgh
<wgrant> If I "accidentally" break bug searching and manage to globally corrupt the history so the old code is irretrievable, does writing an implementation does not UTTERLY SUCK become critical?
<wgrant> s/implementation/implementation that/
<StevenK> How do you plan to globally corrupt the history? :-)
<wgrant> That's one part of the plan that is not yet entirely defined.
<StevenK> wgrant: To quote Penny Arcade: Stay Good!
<wgrant> lifeless: Is bug #558585 valid?
<_mup_> Bug #558585: librarian outages cause OOPSes on pages showing BranchMergeProposal diffs <code-review> <lp-code> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/558585 >
<wgrant> That's like complaining that Launchpad doesn't work when the DB is down.
<wgrant> Except that mizuho is shit and a SPOF.
<StevenK> Next p-s-c run will hit SPR 98000
<lifeless> wgrant: its arguable
<lifeless> I'm not sure if we should say 'yes, yes it does'
<lifeless> or 'we should degrade gracefully'
<wgrant> StevenK: Isn't the much faster than we thought?
<wgrant> Or are lots of early SPRs missing?
<StevenK> Lots of early SPRs are missing changelogs
<wgrant> I mean missing in general.
<wgrant> I don't know what evil was done before the first import.
<StevenK> Ah. The first SPR id run across was 59510
<wgrant> Right.
<lifeless> wgrant: are you still hacking on https://bugs.launchpad.net/launchpad/+bug/575450?
<_mup_> Bug #575450: Archive:+copy-packages nearly unusable due to timeouts <lp-soyuz> <ppa> <timeout> <Launchpad itself:In Progress by wgrant> < https://launchpad.net/bugs/575450 >
<wgrant> lifeless: The next piece of work in that is bug #733071.
<_mup_> Bug #733071: Archive:+copy-packages slow when potential conflicts detected <timeout> <Launchpad itself:Triaged by wgrant> < https://launchpad.net/bugs/733071 >
<wgrant> But there is also work needed to speed up the UI, but I haven't analysed that yet.
<lifeless> wgrant: there is a stall condition here
<wgrant> Yes.
<lifeless> wgrant: if you split out a bug like this, please assign yourself to all components, or only leaf components
<lifeless> wgrant: you see why ?
<wgrant> Yeah, sorry.
<lifeless> no worries
<lifeless> bigjools pinged me about this morning when discussing the drop to 11 seconds
<lifeless> hes concerned about usability during release
<wgrant> That's +queue.
<wgrant> Not +copy-packages
<StevenK> wgrant: If my one-liner of evilness is right, we've processed 38k SPRs since the 15th.
<wgrant> StevenK: Sounds reasonably plausible.
<StevenK> Pity there is another 500+ k
<lifeless> wgrant: bah, emybrain
<wgrant> Speaking of 11 seconds.
<wgrant> 424 Time Outs
<wgrant> :(
<StevenK> You didn't want 424?
<wgrant> We had 252 yesterday.
<wgrant> So, no.
<StevenK> wgrant: The BUILDMASTER oops concerns me
<StevenK> Since the same thing happened yesterday too
<wgrant> StevenK: That normally just means the source has been superseded.
<wgrant> It often happens.
<wgrant> I thought jelmer had fixed that, but maybe not complete.
<StevenK> I guess EB and ED are also appservers?
<wgrant> EB and ED are edge appservers.
<StevenK> Ahh
<StevenK> They need more fire.
<wgrant> We need to get rid of potassium and gandwana.
<wgrant> Now.
<wgrant> The +copy-packages GET timeouts are all high non-SQL on gandwana and potassium.
<wgrant> lifeless: When are the new appservers happening?
<lifeless> wgrant: probably not till mthaddon is back
<wgrant> :(
<lifeless> elmo was checking with charlies/nafallo about the memory
<wgrant> Is the session queue normally sitting at 0?
<wgrant> ie. would it help to deprioritise gandwana/potassium instances?
<lifeless> cve:+index spiked
<lifeless> jcsackett: has a fix for that
<wgrant> He does.
<lifeless> is it landed?
<wgrant> No.
<lifeless> ><
<wgrant> Kanban says it's in review.
<wgrant> Checking it.
<lifeless> dropping the timeout
<wgrant> Excellent.
<StevenK> Wow, apparently windmill is fixed
<lifeless> rm?
<lifeless> have we done anyting to improve https://launchpad.net/ubuntu/+source/kde4libs/+publishinghistory ?
<jtv> StevenK: did they use the great "rm -rf" debugger?
<wgrant> StevenK: Windmill is not fixed, but wallyworld fixed the broken test.
<wgrant> lifeless: My SPR.copyright fix will have improved that.
<wgrant> It still times out occasionally, though.
<thumper> https://code.launchpad.net/~thumper/launchpad/blueprint-linked-bug-tasks/+merge/53734 anyone?
<thumper> I have three other reviews which are the start of the pipeline
<thumper> but they are trivial
<thumper> and I'm tempted to self review
<thumper> but if someone wants to look at them
<thumper> here they are:
<thumper> https://code.launchpad.net/~thumper/launchpad/bugtask-repr/+merge/53731
<thumper> https://code.launchpad.net/~thumper/launchpad/bugtask-tales-addition/+merge/53732
<thumper> https://code.launchpad.net/~thumper/launchpad/add-publishing-for-factory-distro-sourcepackage-bug-tasks/+merge/53733
<wgrant> In bugtask-tales-addition you seem to be trying to reimplement structuredstring.
<thumper> wgrant: what do you mean?
<wgrant> _make_title escapes stuff.
<thumper> wgrant: I implemented it in the same way that the main link is generated
<thumper> wgrant: I didn't think too hard about the implementation
<wgrant> Yes, and most of our code is buggy :(
<thumper> is that bit buggy?
<wgrant> It's not buggy, but it's not a pattern that we want to perpetuate.
<wgrant> If you are calling cgi.escape, you are probably doing it wrong.
<thumper> where is structuredstring?
<wgrant> canonical.launchpad.webapp.menu, or somewhere stupid like that.
<wgrant> Yeah, canonical.launchpad.webapp.menu.structured is the constructor.
<thumper> hmm...
<StevenK> wgrant: Hm, do you know the diff?
<wgrant> StevenK: Which diff?
<StevenK> wgrant: wallyworld's that fixed the failing windmill test
<wgrant> StevenK: r12593
<thumper> wgrant: can you think of a better place than lp.services.utils for structured?
<StevenK> lp.app.utils
<thumper> StevenK: it isn't really LP specific
<wgrant> I think lp.app is probably more appropriate.
<wgrant> But I'm not really sure.
<lifeless> SpamapS: ping
<lifeless> https://bugs.launchpad.net/launchpad/+bug/627974 - the url I got from you, did you get that from launchpadlib, or were you url hacking ?
<_mup_> Bug #627974: MaloneApplication:CollectionResource is slow/times out <lp-bugs> <pg83> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/627974 >
<thumper> wgrant: lp.app.browser.string ?
<thumper> or lp.app.browser.structured
<wgrant> thumper: string, I think.
<wgrant> Mm.
<wgrant> I don't know.
<thumper> it needs an interface module too
<thumper> I'd not be happy putting it in lp.app.interfaces.launchapd
<thumper> or launchpad even
<thumper> I'd be happy to have the interface and implementation in a single file in lp.services.string
<wgrant> Indeed not.
<thumper> and include escape and structured
<wgrant> But lp.services is not webapp-specific, so lp.services.string is wrong.
<wgrant> lp.services.escaping, maybe.
<lifeless> l.c.l.w.string then
<wgrant> But I'd prefer lp.app somewhere.
<thumper> lifeless: we are trying to remove things from c.l.w
<thumper> wgrant: lp.app with two files?
<lifeless> thumper: I know, but that seems pointless to me, the angst about it is dwarfed by the number production and deployment issues spent fixing moves
<lifeless> I can't bring myself to care about labels
<lifeless> the arguments for moving are that the old structure impeded splitting the tree, but the analysis for splitting the tree was wrong
 * lifeless shrugs
<thumper> perhaps we should just move canonical.launchpad.webapp to lp.app.webapp
<thumper> :)
<wgrant> thumper: I am not sure if bugtask-repr is safe.
<thumper> wgrant: why?
<wgrant> thumper: It potentially exposes the bug ID and target, if you manage to get hold of a BugTask through the webservice.
<thumper> wgrant: the webservice handles launchpad.View
<thumper> wgrant: and a bugtask view permissions filter to the bugs
<thumper> wgrant: so not a valid concern I believe
<wgrant> thumper: The webservice handles launchpad.View for collections, sometimes.
<thumper> having that repr certainly helped my debugging
<wgrant> 8 of todays top 10 OOPSes are fixed. Yay.
<wgrant> The next few are codehosting.
<wgrant> Blue squad sounds like a good one to take those :P
<StevenK> wgrant: I'm having a such a brain fade -- can you remind how to get the latest version of an SPR in a distroseries.
<wgrant> StevenK: That doesn't really make sense. But getCurrentSourceReleases might do what you want.
<wgrant> Depending on what you actually mean.
<wgrant> It is ambiguous in a couple of ways.
<StevenK> wgrant: So, Julian would like to delete the DSD is the derived series and parent series have the same version published. This makes sense to me, but I'm having a brain fade looking for the method to help me.
<wgrant> That's deleting the comment history.
<wgrant> Bad idea to do that so early.
<wgrant> You can possibly justify doing that once a series is done.
<wgrant> But not before.
<StevenK> Why? If something has been synced, the comments are now irrelevant/
<StevenK> s/\//./
<wgrant> StevenK: I want to know why it was synced.
<StevenK> Then you can look up the bug.
<wgrant> There wasn't a bug, because we are no longer in the dark ages.
<wgrant> The sync button has been implemented.
<StevenK> Good point.
<thumper> how do we avoid the ValidPersonCache queries?
<thumper> can we precache it?
<wgrant> thumper: list(getUtility(IPersonSet).getPrecachedPersonsFromIds(ids, need_validity=True))
<thumper> wgrant: I have the people already from a different join
<thumper> although looking at this method, I may change it
<lifeless> thumper: are you filtering on person ?
<thumper> lifeless: I'm looking at two timeouts
<thumper> for spritns
<lifeless> Sprint:+huge-vocabulary?
<thumper> one on +index seems to be getting valid persons
<thumper> and +attendees-csv is just shite
<lifeless> ok
<lifeless> bug #'s ?
<thumper> the attendees-csv could be trivially fixed
<thumper> except for the ircnicknames
<thumper> bug 735996
<_mup_> Bug #735996: Sprint:+index timeouts <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/735996 >
<thumper> there isn't a bug for +attendees-cvs db counts
<thumper> ircnicknames is a SQLMultipleJoin
<thumper> so not sure how to precache the names for all the people :-(
<wgrant> You'll have to make it a cachedproperty returning a list.
<thumper> wgrant: but that doesn't mean we can get them all at once
<wgrant> thumper: It does...
<thumper> how
<wgrant> store.find(IRCNick, IRCNick.personID.is_in(personIDs)
<wgrant> grep around for pre_iter_hook
<thumper> hmm...
<thumper> I'll look around
<lifeless> jtv: perhaps you could comment on https://bugs.launchpad.net/launchpad/+bug/708875
<_mup_> Bug #708875: AssertionError: More than 6 plural forms were submitted <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/708875 >
<jtv> lifeless: should be an unexpectedformdata if it happens from the web UI.
<jtv> Seems a far-out guess that this is coming from the web UI, but I'm pretty sure I made the importer check for this condition way back when.
<wgrant> I think the check is wrong.
<wgrant> I looked at it yesterday and ran away.
<wgrant> There are only forms 0-5
<wgrant> Which is 6.
<wgrant> Not more than 6.
<jtv> The check doesn't even make sense to me.  It's an "else" attached to a "for" loop AFAICS
<wgrant> Right.
<wgrant> That does make sense.
<wgrant> But it's not right here.
<wgrant> (the else block is executed if the loop runs to completion, without a break)
<jtv> Gawd
<wgrant> Pretty much.
<jtv> So yes, that explains the off-by-one
<lifeless> which is the absolute opposite of what *everyone* reads it as
<jtv> The loop breaks when it hits the first plural form that is supported but is not in the form.
<jtv> And when you submit exactly the maximum number of forms, it never does.
<jtv> FFS.
<wgrant> Exactly.
<wgrant> But I thought this must have come up often.
<wgrant> Does it not?
<lifeless> wgrant: still working on bug 714414 ?
<_mup_> Bug #714414: unstack debian sid branches <code-hosting> <stacking> <Launchpad itself:Triaged by wgrant> <Ubuntu Distributed Development:Fix Released by mbp> < https://launchpad.net/bugs/714414 >
<jtv> Probably, yes, but plural forms are relatively rare, and only Arabic uses 6.
<wgrant> jtv: Right, but wouldn't it occur on every plural arabic translation, then?
<wgrant> Or is the last one uncommon?
<jtv> Every complete plural arabic translation made in the UI, yes.
<wgrant> Given the debate around ubuntu-l10n-ar, I guess we don't have many arabic translators.
<wgrant> lifeless: Not really.
<jtv> An upstream translation. or one uploaded by a translation team member, wouldn't have this problem.
<jtv> I'm expounding on the bug.
<lifeless> wgrant: unassign?
<lifeless> is https://bugs.launchpad.net/launchpad-buildd/+bug/719162 deployed?
<_mup_> Bug #719162: [Natty] check-implicit-pointer-functions fails on natty, resulting possibly broken packages <qa-ok> <Launchpad Auto Build System:Fix Committed by julian-edwards> < https://launchpad.net/bugs/719162 >
<wgrant> lamont was looking at that earlier this week. We need an amd64 builder on staging before we can QA it.
<wgrant> So no, it's not yet deployed.
<lifeless> stub: hey
<lifeless> can you see how long the very big query in bug 722787 takes now on prod?
<_mup_> Bug #722787: Product:+filebug timeouts <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/722787 >
<lifeless> I suspect the index repack fixed it
 * stub tries to find a real query
<lifeless> wgrant: and are you doing 729064 ?
<wgrant> lifeless: Yes.
<stub> I can't even cut and paste the query without making terminal explode :-P
<stub> https://lp-oops.canonical.com/oops.py/?oopsid=1875O1575 has bigger problems than just timing out
<lifeless> stub: :P
<lifeless> love our ui
<lifeless> There are currently no open bugs.
<lifeless> https://bugs.launchpad.net/launchpad-project/+bugs?field.importance=Critical&start=225
<stub> lifeless: So the insane query in that oops runs in 3 seconds atm. on one of the slaves.
<lifeless> stub: great,thanks
 * lifeless closes
<lifeless> 224 critical bugs
<lifeless> tada
<lifeless> including 6 5-digit ones
<lifeless> stub: going to cook dinner; catchup after that?
<stub> sure
<poolie> hi
<poolie> re bug 719271, are there multiple threads per scanner process?
<_mup_> Bug #719271: Branch scanner jobs failing in bzrlib <branch-scanner> <bzr> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/719271 >
<lifeless> stub: ping
<stub> lifeless: pong
<lifeless> skype?
<wgrant> https://staging.launchpad.net/successful-updates.txt is odd.
<lifeless> stub: https://dev.launchpad.net/LEP/ReliableDBDeploys
<wgrant> Can we move to Python 3 please?
<StevenK> How do you suggest we move Zope and Storm to Python 3?
<wgrant> Shh
<StevenK> Oh, you don't want problems, you want solutions?
<wgrant> I want non-awful Unicode.
<wgrant> I also want lucid_db_lp to fail sensibly.
<wgrant> Because its current failure is not sensible.
<stub> lifeless: tasque
<lifeless> wgrant: happier with the bug count now ?
<wgrant> lifeless: Mostly.
<wallyworld> wgrant: there's a question about getting increased space for a ppa. who do i assign that to?
<wgrant> wallyworld: A LOSA, or poke a commercial admin
<spm> that doesn't sound right. we haven't been doing space increases for ages. can you guys already do that?
<spm> yourselves as in. ?
<wallyworld> spm: if we can do it, i don't know how :-)
<spm> it's an admin option off the ppa itself. or was.
<wgrant> It needs a commercial admin or a LOSA, and there is approximately one active commercial admin.
<wallyworld> i also have a question about a confirmation code being rejected when someone tried to create an account. assignee?
<wgrant> wallyworld: Ask them to file a ticket at the SSO support site.
<spm> that needs a level of sanity filtering first
<wgrant> Or reassign the question to canonical-identity-provider
<wallyworld> wgrant: thanks. will reassign to cip. what's the sso support site?
<wgrant> https://forms.canonical.com/lp-login-support/
<wallyworld> thanks
<wallyworld> spm: if it is an admin option on the ppa i can't see it. can i ask you to increase the space for the user?
<stub> wgrant: You were correct about Bug #736613
<stub> (The â bug since mup seems to have run out of blow)
<wgrant> Aha
 * wallyworld has soccer training. back later
<bigjools> morning hackers
<adeuring> good morning
<jtv1> hi adeuring
<adeuring> hi jtv1!
<jtv1> StevenK: I realize it's late for you, so you may want to ignore this.  I have minimal tests running for cron.publish-ftpmaster, but basically without any packages.  Next on my wishlist is to make it produce meaningful data so I can test that I don't break it.
<wgrant> You managed to get it to run locally?
<jtv1> Yes
<jtv> In /tmp, not /srv/launchpad.net
<jtv> The test should be able to reproduce it without any system changes.
<jtv> Also, I changed it so it'll run against ubuntutest.  Not sure yet if that's particularly useful.
<wgrant> Knowing the state of the sampledata, it's probably best to run against a completely new distro.
<wgrant> Or you are going to have fun when you start trying to publish stuff.
<jtv> Yes, that's what I was hoping to do.
<jtv> Problem is, I'm lost in the dark.
<lifeless> in a completely temp dir
<jtv> all alone
<jtv> and it's freezing
<wgrant> Don't pollute my terminal with your wide characters again.
<jtv> myâ¦ what?
<wgrant> Oh wait, that was stub.
<jtv> ahânasty stub
<jtv> Is à¸ a wide character?
<jtv> Anyway, wgrant, what would I have to set up?  I'm guessing distro, distroseries, sourcepackagename, sourcepackagerelease, sourcepackagepublishinghistoryâ¦ anything else?
<wgrant> Yes, but not as wide as the â  I was terrorised with earlier.
<jtv> Wait, there's a single character for Â°C?
<stub> I can whine in â if you would rather
<jtv> How do I type those?
<stub> And if you think I'm bitching about the cold, you should hear my maid.
<wgrant> jtv: Distribution, DistroSeries, DistroArchSeries to start. Then you can create some stuff to publish... SPPHs (you may have to add files manually) to test publish-distro, and PackageUploads with PackageUploadCustoms to run through process-accepted to test dist-upgrader signing.
<jtv> Holy cow, it's 18Â°!   Insane!
<stub> You mean 18â
<jtv> wgrant: quite a shopping list alreadyâ¦ thanks
<wgrant> stub: At least there was no exclamation mark that time.
<jtv> stub: yes, but I only know how to type that as Â°C
<jtv> Is there a problem with an exclamation mark after â?
<wgrant> Aaaaaaaa
<jtv> Or as the Danes would say, ÃÃ¥Ã¥Ã¥.
 * jtv is beginning to wonder whether wgrant's terminal is perhaps not entirely prepared for some non-ASCII characters somehow.
<stub> Bug #736613 is the "â" problem
<wgrant> http://people.canonical.com/~wgrant/bad-kerning.png
<jtv> wgrant: thanks, that gives me something to aim for
<bigjools> if you use an irc client developed this century, it would probably work
<lifeless> it works in irssi
<wgrant> This irssi.
<wgrant> It depends on your terminal emulator and font.
<wgrant> +is
<lifeless> ok, irssi with real terminal and font :P
<wgrant> :(
<stub> ê´
<jtv> wgrant: the files I may have to add manually are changelogs for the SPRs?  Or something else?
<wgrant> jtv: SPRFs
<jtv> Isn't that a view?
<wgrant> The view is SPFP
<jtv> Oh
<jtv> How could I have been so stupid?
<wgrant> I know, right.
<bigjools> I knew that would provoke a reaction :)
<jtv> hi bigjools :)
<bigjools> helleau jtv
<sidnei> anyone around for a review?
<poolie> o/
<poolie> (just waving, not volunteering :)
<LPCIBot> Project devel build #547: FAILURE in 4 hr 38 min: https://hudson.wedontsleep.org/job/devel/547/
<wgrant> lp.services.job.tests.test_runner.TestTwistedJobRunner.test_timeout as usual.
<sidnei> oh, hi poolie :)
<gmb> The Kanban board should pop a message when I move a card into the review lane that says "Are you sure, little man?"
<sidnei> gmb, with a picture of clippy?
<gmb> Yeah :)
<lifeless> jml: ping
<jml> lifeless: pong
<lifeless> jml: https://dev.launchpad.net/LEP/ReliableDBDeploys
<jml> lifeless: yeah, I saw your earlier IRC comment about it. It's on the list.
<lifeless> jml: thanks
<lifeless> jml: also, I just replied to bug 655385 - with some ideas about solving a few of our bug challenges all at once
<_mup_> Bug #655385: Allow bug status change from Triaged only for bug supervisor <accesscontrol> <acl> <bugs> <easy> <lp-bugs> <Launchpad itself:Opinion> < https://launchpad.net/bugs/655385 >
<lifeless> jml: I'd be interested in your thoughts on my comment (which was mailed in so possibly not visible yet
<jml> lifeless: ok
<lifeless> its a little adlib, but I felt the pieces going 'click' as I wrote
<lifeless> man, its *really* hard to enter a tag that is a prefix of an official bug tag
<lifeless> jml: its there - https://bugs.launchpad.net/launchpad/+bug/655385/comments/14
<_mup_> Bug #655385: Allow bug status change from Triaged only for bug supervisor <accesscontrol> <acl> <bugs> <easy> <lp-bugs> <Launchpad itself:Opinion> < https://launchpad.net/bugs/655385 >
<lifeless> hmm, night all
<lifeless> bigjools: if you think some of the things I changed should still be critical, please just change them back: I was doing a big sweep as I said, and 100% accuracy is hard to achieve
<bigjools> lifeless: ok, I didn't want to be too presumptuous :)
<lifeless> bigjools: long as you include a reason (so that in 3 months I don't toggle again, blindly :)) it will be fine
<wallyworld> jtv: there's a translation question i have no clue about. can i assign it to you?
<jml> lifeless: !!!
<lifeless> jml: ?
<lifeless> jml: you caught me just as I was about to walk away from the keyboard... whats up?
<jml> lifeless: oh, just that at least one of the bugs that were marked down had been previously marked Critical without comment.
<jml> lifeless: it's no big deal
<lifeless> jml: presumably by me? I do try to comment consistently, though I will admit I rarely bother when the bug is tagged oops or timeout from the start
<jml> lifeless: yeah. it was one or two out of about thirty, and not bugs I care about personally.
<lifeless> jml: if it was the getBranches on sourcepackage one, there was a comment already on the bug saying it no longer oopsed; I was lazy there ;)
<lifeless> http://webnumbr.com/.join(launchpad-oops-bugs.all,launchpad-timeout-bugs.all,launchpad-critical-bugs.all)
<jml> hmm.
<jml> I've got some PPA stats
<jml> questions
<jml> - how many PPAs are there?
<jml> - how many packages are being uploaded to PPAs?
<bigjools> https://launchpad.net/ubuntu/+ppas
<jml> bigjools: awesome :)
<wgrant> There are also a few graphs on lpstats.
<bigjools> yeah w3as just searching for some good ones for jml
<bigjools> like https://lpstats.canonical.com/graphs/NewPPAsWithUploads/
<wgrant> There aren't really any good ones.
<wgrant> That's the only interesting one.
<bigjools> https://lpstats.canonical.com/graphs/PPASourcePackages/
<jml> do those "published sources" and "published binaries" numbers collapse versions?
<bigjools> published implies collapsed
<bigjools> PPAs don't let you publish more than one version
<bigjools> of the same package, I mean
<wgrant> If you mean distinct source versions? No.
<wgrant> If I have the same package in multiple series or PPAs, it will show up a few times in that number.
<jml> I guess what I mean is COUNT(DISTINCT (archive, sourcepackagename))
<bigjools> ah, no idea
<bigjools> anyway I am desperately hungry
<jml> ok, thanks.
<jml> I'm getting a failure in stable running ./bin/test -cvv lp.services.job.tests.test_runner test_timeout
<wgrant> jml: Yes, that test fails on its own.
<wgrant> And sometimes in the full test suite.
<jml> it's consistent, and it's not like either of the failures mentioned on bug #505913
<_mup_> Bug #505913: TestTwistedJobRunner.test_timeout fails intermittently <lp-foundations> <spurious-test-failure> <Launchpad itself:Triaged> < https://launchpad.net/bugs/505913 >
<wgrant> jml: Is it a failure to import _pythonpath?
<jml> wgrant: yeah.
<jml> wgrant: you also get this failure when running the test standalone?
<wgrant> It needs someone who knows that code to look at it.
<wgrant> I spent a couple of hours on it and made very little progress.
<jml> wgrant: I'm keen to have a try today, if I can deal with prior obligations.
<jml> wgrant: just to be crystal clear, you get the _pythonpath import failure when you run the test by itself locally?
<wgrant> jml: Yes.
<wgrant> jml: Working around that in ways that I no longer recall (possibly hacking site.py in the custom PYTHONPATH it uses) gets the same message as the spurious failure, but I was unable to work out how to get the output of the subprocess.
<wgrant> So I made the failure spurious again and ran away.
* jcsackett changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | firefighting: - | On call reviewer: jcsackett | https://code.launchpad.net/launchpad-project/+activereviews/
<wgrant> jml: I initially suggested doing what you considered, but could not think of a reasonable name for the method.
<wgrant> Er.
<wgrant> Swap suggested and considered.
<jml> wgrant: ahh, yeah. I can see how that would be a problem.
 * bigjools waves at jcsackett
<jcsackett> bigjools: publisher config schema review? :-)
<bigjools> jcsackett: yarp :)
<jcsackett> looking now.
<sinzui> jcsackett: will you have time to review https://code.launchpad.net/~sinzui/launchpad/person-merge-job-0/+merge/53706
<wgrant> deryck: I've tried running it 30 times. It doesn't hang :/
<deryck> wgrant: *sigh*
<wgrant> deryck: Yes.
<jcsackett> sinzui: i'll do it next. already had it marked for looking at today.
<deryck> wgrant: I was afraid of that.
<wgrant> deryck: It's also only died a couple of times on Jenkins.
<deryck> wgrant: can you point me at those builds?
<wgrant> And it doesn't die often on ec2 when only running WindmillLayer.
<deryck> wgrant: also, I'm going to try it in a vm with lower resources and see if that helps cause it.
 * deryck is grasping at straws
<wgrant> That's a good plan.
<wgrant> I ran out of straws a week or two ago.
<deryck> I'm spending today on it and if I get no hangs, then will look at other options -- decoupling from Zope test runner or moving to Selenium.
<deryck> abentley: I'm good for a js chat now.
<abentley> deryck: cool.
<wgrant> https://hudson.wedontsleep.org/job/windmill/2/ https://hudson.wedontsleep.org/job/windmill/5/ https://hudson.wedontsleep.org/job/windmill/35/ https://hudson.wedontsleep.org/job/windmill/41/ https://hudson.wedontsleep.org/job/windmill/43/ are isolated spurious test failures. The build I thought was Windmill breakage was in fact the UEC slave going away.
<gmb> leonardr: Is it possible to pass versioned annotations to export_write_operation (for example, if I want to export a write operation in devel but nothing else)?
<wgrant> deryck: So we have had 62 builds without a failure.
<wgrant> Yay.
<wgrant> Is it something about being in a full test run? Low resources? A race? Who knows...
<leonardr> gmb: yes, but you can't do it in export_write_operation, because the definition of a named operation takes place over multiple annotations
<leonardr> you need to use @operation_for_version()
<gmb> Ah.
<gmb> leonardr: So, do I do something like:
<leonardr> to publish an operation in devel and nothing else, at the bottom of the annotation stack you would have @operation_for_version('devel')
<gmb> ... right.
<gmb> :)
<gmb> Cool.
<gmb> THanks.
<leonardr> np
<jcsackett> bigjools: so, this branch is just adding these fields, not refactoring anything to make use of them yet?
<jcsackett> i only see addition, and setting/changing of the fields, not anything using them.
<bigjools> jcsackett: yep, just doing a small focused branch
<jcsackett> bigjools: cool.
<bigjools> because I know that the code change to use this stuff will be pain ridden :)
<jcsackett> bigjools: in the factory, since those are optional fields, is it entirely appropriate that they're always set?
<bigjools> jcsackett: interesting point
<jcsackett> i don't know that it's actually a problem...just raising a possible different take on how it should work.
<bigjools> it would be impossible to set them to None
<bigjools> with that code
<bigjools> I'll change that, thanks for spotting
<jcsackett> bigjools: r=me, with that. :-)
<bigjools> cheers!
<jcsackett> sinzui: looking at your MP now.
<bigjools> Our reviews rock lately
<jcsackett> i think moving to the "just use the review page as the queue" is a big win.
<bigjools> you weren't around in the old days when we used a wiki page ...
 * bigjools shudders
<jcsackett> that sounds like it would be problematic, yes. :-P
<sinzui> problematic was an understatement
<bigjools> and a review team of about 5 people
<sinzui> We had to locate a reviewer and prod each over several days to get approval
<bigjools> r=trivial was great for that :)
<sinzui> except when something broken and kiko looked at the branch and ask how anyone could claim the changes were trivial
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #548: FIXED in 4 hr 15 min: https://hudson.wedontsleep.org/job/devel/548/
<deryck> wgrant: sorry, was on call.  thanks for the build links.  And I'm not sure honestly what it is about the entire run that causes it.
<deryck> if the Jenkins build is just WindmillLayer, then perhaps it could be changed to run the entire suite + windmill.
<jcsackett> sinzui: is there a need for a test on mergeAsync in the failure case?
<jcsackett> i.e. when it returns None instead of a job?
<jcsackett> ah, nevermind. that test exists, just in a different test file.
<sinzui> jcsackett: That is handled by the base runner and tests. the base defines an empty set of addresses.
<jcsackett> sinzui: yes; i saw test_mergeAsync_success on test_person and so nothing corresponding, but it's handled elsewhere. :-)
<bac> hi sinzui -- can you look at this paste as a mid-imp sanity check for the menu problem we discussed yesterday?  https://dev.launchpad.net/JavascriptUnitTesting
<jcsackett> sinzui: r=me.
<sinzui> bac: thank you for updated the docs
<gmb> jcsackett: I have an MP for you to review, if you'd be so kind: https://code.launchpad.net/~gmb/launchpad/make-+subscribe-play-nice-bug-735397/+merge/53839
<bac> sinzui: oops, that was the wrong paste.  :)
<bac> sinzui: i meant for you to look at http://pastebin.ubuntu.com/581613/
<bac> but, yes, i was happy to update those testing docs
<jcsackett> gmb: looking now.
<gmb> Thankee kindly
<sinzui> bac: ugh. I do not think we should be injecting line-height into styles. The anchor is making a bad guess. Only <h3>s have a line-height of 20px. Maybe this is why firefox still shows the icons misaligned in pages
<bac> sinzui: perhaps, but that isn't part of my change.
<sinzui> you cargo-culted it
<bac> sinzui: i'm happy to fix it at the same time, though
<bac> sinzui: i replicated the existing code, assuming it was correct, for the case i was adding
<bac> thanks TAL
<bac> sinzui: the gist of my change to the page template is the conditional addition of "display:none"
<sinzui> bac: why do you use style instead of the invisible-link class. I think I would toggle between two classes <invisible-link|visible-linl>
<bac> sinzui: good idea
<sinzui> we have supported invisible link since 2006 to make link testing easy. I think we can really use it for something else now
<bac> sinzui: otherwise a reasonable way to go?
<sinzui> yes. your implementation is what I would have done
<sinzui> I am worried about spurious line-heights now
<sinzui> damn
<sinzui> sweet. bac. there is only one occurrence. bac. try removing it and look at the page in firefox and safari. I think they will look the same now
<SpamapS> lifeless: pong (no packet loss, but latency is close to interplanetary ;)
<bac> sinzui: thanks.  i'll switch to use invisble-link.  and i'll make the line-height fix
<sinzui> bac: firefox is still showing the sprites offset from where we intended them. I did not check that there was something overriding the line-height in the markup :(
<bac> sinzui: these would be for links such as in the global-actions portlet?
<deryck> henninge: we're all sorted out for feature flag now?
<henninge> deryck: the request is on LPS
<deryck> henninge: ok, thanks.
<sinzui> bac: this template is used by "fmt:link", which is most links
<deryck> henninge: the number in the feature flag has to be incremented above whatever the current highest number is.
<gmb> jcsackett: I need to go and run some errands; I'll respond to your review comments and questions when I get back.
<deryck> henninge: so the 1 is probably something like 20 now?
<jcsackett> gmb: righto.
<deryck> jml: ping
<jml> deryck: hi
<henninge> deryck: hm?
<deryck> henninge: I forgot what that attribute is called, but it has to be current value + 1.
<henninge> deryck: oh, I did not know that. I think it's the priority.
<deryck> yeah, that's it.  priority.
<deryck> henninge: rather, it has to be some value that isn't taken yet. ;)  So everyone does current + 1.
<henninge> deryck: I thought that was only relevant when you have multiple rules for the same flag.
<henninge> it's to solve those ambiguities, I thought
<deryck> henninge: yes, but there's a db constraint it has to be unique across all flags.  the update will fail if not.  it did in the past anyway.
<henninge> deryck: hm, there are multiple rules with "1" and also with "0" in the current rule set.
<henninge> https://launchpad.net/+feature-rules
<deryck> henninge: ah, ok.  So guess I'm wrong then. sorry for the noise.
<bac> sinzui: that line-height styling doesn't seem to have any effect.  i've removed it and bumped it up to very high and see no difference in rendering
<sinzui> bac: in firefox?
<sinzui> Well regardless, we need to remove it. I was hoping that firefox would be fixed
<bac> sinzui: actually, i was only looking at the global-actions portlet, which uses the same styling
<bac> sinzui: it *does* have an effect on those in the involvement portlet
<bac> sinzui: on firefox, in the involvment portlet removing the 'line-height' styling does change the spacing between the lines (yay) but it does not appear to affect the vertical spacing of the sprite relative to the text.
<sinzui> bac: I do not see a style attribute in the involvement portlet links on https://launchpad.net/gdp
<sinzui> oh, no icon
<sinzui> bac: set this aside. I will look into it later
<sinzui> sorry for the diversion
<bac> sinzui: np.  i'm easily diverted
<sinzui> jcsackett: I really like your TestTeamParticipationMesh test. I was able to add two new tests, one using setMembershipData on each member, and the other using deactivateAllMembers. The second test fails as I expect. I will know my refactoring is done when both agree
<jcsackett> sinzui: fantastic.
<jcsackett> gmb: r=me. left a few questions in a comment, but those are strictly for my education, not issues with the branch.
<gmb> jcsackett: Thanks. I'll respond to your questions in the MP now.
<deryck> I may have a Windmill hang now!
<jelmer> deryck: Is that really something to get excited over ? ;-)
<deryck> jelmer: it's the little things that make me happy :-)
<sinzui> jcsackett: ping
<jcsackett> sinzui: pong.
<sinzui> jcsackett: I have a fix for team participation. Do you have a few minutes to mumble about it?
<jcsackett> give me just a few moments to grab a drink, and i will be on mumble.
<deryck> ah.  stuck in a sleep.
<deryck> I told you all sleep was evil.
<bigjools> just ask lifeless
<deryck> oh wow
<deryck> windmill imports time.sleep and uses it inside a while statement
<jml> deryck: that sucks
<deryck> jml: yes, that's a terrible, scary thing to do, IMHO. But turns out it's unrelated.  Just frightening, but not causing the hang.
<jml> deryck: huh
<jml> deryck: very glad you're working on this, btw.
<deryck> jml: looks like were stuck in client.open.  still digging to know for sure. lot of backtrace to get through
<deryck> and thanks!  glad to be working on it myself
<bigjools> night folks
<deryck> crap.  lost my hung process.
<deryck> night bigjools
<lifeless> moin
<jml> g'night
<deryck> Morning, lifeless
<lifeless> hi deryck
<lifeless> wgrant: jelmer: can either of you loook at  https://rt.admin.canonical.com/Ticket/Display.html?id=42954 and answer toms debugging question?
<lifeless> abentley: hey, do you know - do the LP code import slaves need access to the librarian ?
<abentley> lifeless: I don't know.
<jelmer> lifeless: looking...
<lifeless> jelmer: thanks
<lifeless> jml: if you haven't actually gone - did I make any sense in that bug post I pointed you at?
<sinzui> jcsackett: Can you review https://code.launchpad.net/~sinzui/launchpad/deactivate-all-members-fix-0/+merge/53885 today?
<jcsackett> sinzui: sure. i'll take a look in just a bit.
<jcsackett> sinzui: 458 lines? that's much smaller than i was expecting from your earlier concerns. :-)
<sinzui> jcsackett: Deleting lots of code is often small then editing it
<lifeless> jcsackett: hey, how is your cve:+index fix going
<jcsackett> lifeless: made the changes you pointed out and it's out to land.
<jcsackett> ec2 instance died without output on first try, second try seems to be going alright.
<lifeless> jcsackett: cool - third highest oops yesterday
<jcsackett> lifeless: well, let's hope i got everything then. :-)
<lifeless> jcsackett: if you didn't, we can iterate
<jcsackett> sinzui: r=me, and thanks for that branch. i feel that it's a pretty big win.
<sinzui> jcsackett: thank you.
<jcsackett> thumper: you around yet?
<thumper> morning
<thumper> jcsackett: am now, with coffee even
<jcsackett> thumper: cool. i am looking at https://code.launchpad.net/~thumper/launchpad/bugtask-tales-addition/+merge/53732
<thumper> ok
<jcsackett> i think it looks pretty good, but i wonder if now all Formatters will need to define _title_values? it looks like calling it is baked into the base class, but the base class has it as NotImplemented.
<jcsackett> thumper: it might be better for the base form to just return None on _title_values instead, since that's handled gracefully.
<thumper> jcsackett: that seems reasonable
<thumper> can easily fix
<jcsackett> thumper: with that change, r=me.
<thumper> wgrant yesterday also suggested replacing the cgi.escape rubbish with "structured"
<jcsackett> thumper: yeah, that might be better too.
<thumper> jcsackett: that branch is one of a pipeline :)
<jcsackett> thumper: dig. :-)
<thumper> jcsackett: I only want to land the top
<thumper> I just broke it up for review
<thumper> it was getting kinda big
<jcsackett> thumper: dig.
<jcsackett> so, r=me on that little bit. :-)
<thumper> thanks
<lifeless> ugh, checkwatches passwords are in lp-prod-configs :(
<lifeless> abentley: are translation sharing jobs actually running yet ?
<abentley> lifeless: no.
<lifeless> abentley: I'm thinking of marking https://bugs.launchpad.net/launchpad/+bug/735954 qa-untestable then
<_mup_> Bug #735954: Translation merge job display <qa-needstesting> <Launchpad itself:Fix Committed by henninge> < https://launchpad.net/bugs/735954 >
* jcsackett changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | firefighting: - | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews/
<abentley> lifeless: I don't think "untestable" is strictly accurate.
<abentley> lifeless: too-painful-to-test, perhaps :-)
<lifeless> abentley: indeed, and too little risk; I think we should check the page renders on qastaging
<lifeless> abentley: what url should I look at to see the sharing stuff ?
<abentley> lifeless: translations.*/$SOURCEPACKAGE/+sharing-details.
<lifeless> abentley: I get a 404 on https://translations.qastaging.launchpad.net/ubuntu/+source/bzr/+sharing-details
<lifeless> ah, distro series source package
<abentley> lifeless: yes, sorry.
<lifeless> https://translations.qastaging.launchpad.net/ubuntu/natty/+source/bzr/+sharing-details
<lifeless> seems to render ok
<lifeless> and i've clicked around and turned everything I could find on
<lifeless> abentley: that seems sufficient to suggest production won't be broken by the change
<abentley> lifeless: does it show that a job's pending?
<lifeless> abentley: it says No upstream templates have been found yet
<lifeless> I guess that that is impeding it
<abentley> lifeless: doesn't sound right.
<lifeless> abentley: darn, I'll undo my tag change then
<lifeless> abentley: what should we do to be confident this is deployable
<lifeless> [that is, that it won't regress or break anything]
<bac> sinzui: would you have the time and interest to review the branch for the menu links that we discussed this morning?
<abentley> lifeless: it should say something like this: http://pastebin.ubuntu.com/581807/
<abentley> lifeless: the fact that it doesn't suggests that we may not have the correct revno on qastaging.
<lifeless> abentley: 12617
<lifeless> I'll have a poke at unity instead
<lifeless> it should have strings
<abentley> lifeless: I don't know what's going on.  Perhaps the job has not been created.  Perhaps the browser message is being suppressed somewhere.
<lifeless> abentley: I think its because there are no templates for upstream
<abentley> lifeless: Oh, did you actually create a packaging link?
<lifeless> yes, but the upstream hasn't been imported
<abentley> I don't recall making job creation conditional on that.
<abentley> It requires the package to be an Ubuntu package, but I think that's all.
<wallyworld> sinzui: you still there?
<lifeless> abentley: have a look here - https://translations.qastaging.launchpad.net/ubuntu/natty/+source/gtk+2.0/+sharing-details
<lifeless> abentley: tell me what you think
<sinzui> I am
<abentley> lifeless: it looks as though this was an already-existing packaging link.  If you just linked it now, then it should have the message.
<wallyworld> sinzui: i have lost an email you may have sent about dealing with spam. there's a few open questions about removing spam links or accounts sending spam etc. if there a wiki page or email you can flick me which describes sop for that?
<wallyworld> s/an email/any email
<lifeless> abentley: oh, ok.
<sinzui> Not yet. I may write it tomorrow
<lifeless> uhm, will look on needs-packaging
<abentley> lifeless: look now.
<lifeless> abentley: what did you do?
<wallyworld> sinzui: ok. so with accounts sending spam, i assume i should try and see if it's a one off (in case they have been hacked) or if it's an ongoing issue and hence disable that account?
<abentley> lifeless: I deleted the packaging link and then re-created it.
<sinzui> wallyworld: I can find the email I sent. I believe we can see suspended users now so the process is simpler. suspend the user, then send an email to the user's preferred email address asking the him to confirm he has control of his computer, browser, and isp
<lifeless> abentley: ok, cool
<lifeless> so it looks ok to you?
<abentley> lifeless: yes.
<wallyworld> sinzui: will do. thanks.
<lifeless> abentley: thanks for the help
<abentley> lifeless: you're welcome.
<sinzui> wallyworld: I just sent the email I think you remember. It has the text of the messages I send
<wallyworld> sinzui: thanks. much appreciated. i need a better email filing system. there's sooooo much of it :-)
<sinzui> wallyworld: no you do not. I need to document what I have been doing for the last 15 months
<wallyworld> sinzui: wow 15 months of dealing with spam! you poor soul. get it document so we all can help out better :-)
<bac> sinzui: did you see my request ^^?
<sinzui> bac: sorry I did not. I can review it now
<bac> thanks, sinzui!  https://code.launchpad.net/~bac/launchpad/hidden_links/+merge/53911
<lifeless> rockstar: whats jsoops?
<abentley> lifeless: it's what Brendan Eich says when he reviews his life's accomplishments.
<rockstar> lifeless, it's our way of logging js errors.
<lifeless> rockstar: they get passed back to the server?
<rockstar> It's not really defined right now, but we're in the process of doing that.
<rockstar> lifeless, *kinda*  It's still being defined.
<lifeless> rockstar: you might like to add anything constraints wise you come up with to dev.launchpad.net/LEP/OopsDisplay
<rockstar> The very basics of it is "have javascript write a img with the src="/jsoops?<oops-contents>"
<lifeless> rockstar: I'm putting together a redo of the entire oops stack to be more agile and reusable across teams
<mwhudson> rockstar: does this work of Y.on('error') or something?
<rockstar> mwhudson, well, we had a global event called one:error that has some extra handling (like providing feedback to the user).
<mwhudson> ah ok
<wgrant> lifeless: Did you get the code import worker sorted out?
<rockstar> We don't yet have error handling inside YUI just yet, simply because it steps on our current jsoops infrastructure.
<lifeless> wgrant: the librarian usage question? its for the librarian uploader ha rt ticket
<beuno> mwhudson, the idea is that if *anything* fails, even YUI, it's silently reported to our servers
<beuno> I picked that up after talking to one of the gmail devs
<rockstar> beuno, unfortunately, YUI kills our existing jsoops stuff.
<beuno> said that's how they managed to roll out so much crack to so many browsers
<beuno> rockstar, yeah, need to give it some love again
<rockstar> beuno, already on it.
<rockstar> (well, kinda)
 * beuno pretends to not have read that
<lifeless> beuno: lp will want that real badly
<thumper> lifeless: can you mentor wgrant's review https://code.launchpad.net/~thumper/launchpad/add-publishing-for-factory-distro-sourcepackage-bug-tasks/+merge/53733 ?
<lifeless> beuno: I suspect my oops stuff will be highly relevant
 * thumper waves at beuno and rockstar
<thumper> splitters!
 * rockstar waves at thumper
<beuno> heh
<beuno> hi thumper!
<beuno> lifeless, yeah, I only spent a few days on it like 5 months ago, it needs some love and documentation to be able to be used company-wide
<lifeless> beuno: json format ?
<beuno> lifeless, well, it sends it back as a url, so we don't need js to do any mangling (it did fail, so it can't be depended on)
<beuno> once we finish making it play nice with yui, we need something in the backend to parse it
<lifeless> beuno: ok
<leonardr> thumper: https://code.launchpad.net/~leonardr/lazr.restful/forbid-reference-to-entry-not-published-in-this-version/+merge/53918
<leonardr> not 100% happy with the implementation, but see what you think
<thumper> leonardr: ok...
<thumper> leonardr: with the VersionedObject addition, will this mean any locations in LP will need to be fixed?
<thumper> leonardr: or is that wrapped in the entry code?
<sinzui> huwshimi: http://people.canonical.com/~curtis/out-4.ogv demonstrate call-to-action links create by an :after pseudo class
<huwshimi> sinzui: Very nice!
<huwshimi> sinzui: How do you make the arrows?
<huwshimi> sinzui: Are they an ascii character?
<sinzui> huwshimi: http://pastebin.ubuntu.com/581829/
<huwshimi> sinzui: Nice work
<sinzui> huwshimi: I wanted to use something more semantic and implicit <em><a>test..., but the pseudo class has to be on the parent element to hover correctly. So I used an explicit class
<thumper> is LP on python 2.6 or python 2.7 ?
<wgrant> 2.6
<wgrant> Lucid doesn't have 2.7.
<wgrant> :(
<huwshimi> sinzui: I can hear you
<thumper> https://code.launchpad.net/~thumper/launchpad/blueprint-linked-bug-tasks/+merge/53734 needs a mentor review :)
<LPCIBot> Project db-devel build #463: FAILURE in 4 hr 56 min: https://hudson.wedontsleep.org/job/db-devel/463/
<lifeless> wgrant: did you look at the rt I mentioned? - the buildmaster xmlrpc thing
<wgrant> lifeless: The code import worker thing?
<wgrant> lifeless: That I asked you about an hour ago?
<lifeless> wgrant: I referenced two things
<lifeless> 08:00 < lifeless> wgrant: jelmer: can either of you loook at  https://rt.admin.canonical.com/Ticket/Display.html?id=42954 and answer toms debugging question?
<lifeless> and
<lifeless> do the LP code import slaves need access to the librarian ?
<thumper> yes, I think sop
<wgrant> lifeless: 42954 is code imports, not buildmaster.
<thumper> we store the logs there
<wgrant> But yes, I think the import slaves need librarian access.
<lifeless> oh bah, it is
<lifeless> and no, not sorted
 * thumper has to get out of the house
<wgrant> lifeless: What is blocking the HA librarian? Complete identification of all the required firewall holes?
<lifeless> yes
<lifeless> there is an rt for it, don't have the number offhand but you should be able to see it
#launchpad-dev 2011-03-18
<lifeless> I suspect Distribution:+bugtarget-portlet-bugfilters-stats is going to be mondo painful
<wgrant> Yes.
<lifeless> also we need to remove the override on Archive:+index
<wgrant> How low is it now?
<wgrant> The page, I mean.
<wgrant> Not the 16000 override.
<lifeless> 8.26
<wgrant> Nice.
<lifeless> https://devpad.canonical.com/~lpqateam/ppr/lpnet/2011-03/daily_2011-03-15_2011-03-16/pageids.html
<wgrant> Kill it.
<wgrant> Yes, but that takes a couple of minutes to load.
<wgrant> Hmm. Maybe I want to look at Person:+contactuser.
<wgrant> Since its 99% is twice the next worst.
<lifeless> 0.11
<lifeless> mean sql
<wgrant> For +contactuser?
<lifeless> hmm, 99th percentile forsql is mising
<lifeless> wgrant: yes
<wgrant> Right.
<wgrant> It'll be mail.
<lifeless> wgrant: its probably mail sending
<lifeless> wgrant: we would benefit a lot if we had instrumentation on how long sending each email takes
<lifeless> its been hard getting that evaluated by is (for fairly good reasons) - but it still an open question around where the issues live
<lifeless> archive:+index should wait for monday
<thumper> lifeless: can I get you to mentor wallyworld? https://code.launchpad.net/~thumper/launchpad/blueprint-linked-bug-tasks/+merge/53734
<lifeless> thumper: I need to pop up to the shops for lunch stuff
<lifeless> thumper: will review after
<thumper> lifeless: ok, thanks
<huwshimi> Does anyone know of a way to use feature flags in JavaScript?
<lifeless> yes
<lifeless> export the value of the flag in the .pt file
<lifeless> interpret it in your js
<huwshimi> lifeless: That was my plan, but I was having trouble getting some context values. Maybe I'll try again.
<lifeless> its been done before :)
<lifeless> huwshimi: e.g. showing the render time in the top right
<huwshimi> lifeless: heh no wonder it wasn't working. The feature flag wasn't set. The shame.
<lifeless> huwshimi: rotfl
<huwshimi> lifeless: I didn't realise that it didn't keep them between make runs
<lifeless> huwshimi: make schema will zerg them from your dev insance
<lifeless> huwshimi: 'make run' won't reset them
<huwshimi> lifeless: I didn't think I'd done a make schema... but maybe I had
<StevenK> huwshimi: make schema completly resets the db to 'pristine sampledata'
<huwshimi> yeah
<StevenK> Now to wash my eyes out for saying 'pristine sampledata'
 * thumper is mildly confused by a comment in code
<thumper> model/person.py line 1797
<mwhudson> thumper: i guess _getPrecachedPersons isn't in the interface?
<thumper> it doesn't need to be
<poolie> lifeless, i liek your idea about incomplete (bug 737008)
<thumper> the self object isn't security wrapped
<_mup_> Bug #737008: old incomplete bugs are not treated as incomplete <Launchpad itself:Invalid> < https://launchpad.net/bugs/737008 >
<thumper> once inside the object
<poolie> maybe we should make that bug ask for what you said (as Low)
<thumper> you are free of the security proxy on self
<lifeless> poolie: care to do that ?
<poolie> sure
<mwhudson> thumper: sure, but if you did getUtility(IPersonSet) you get a proxied object
<lifeless> thumper: you can use the proxied set and see what happens....
<thumper> mwhudson: yes you do
<thumper> but once you call a method on the object (that is in the interface)
<thumper> the self passed through isn't wrapped
<thumper> otherwise that'd be insane
<mwhudson> sure
 * thumper is 98% sure of this
<lifeless> bbs
<thumper> lifeless: you back?
<thumper> anyone know how to clear the storm cache for tests?
<lifeless> store.reset
<lifeless> store.invalidate
<lifeless> IIRC
<lifeless> hmm
<lifeless> I should have been on leave today
<mwhudson> oh yes, the memorial
<lifeless> I'll talk to flacoste, do a day off next week
<lifeless> thumper: ^ store answers for you
<thumper> lifeless: ta
<thumper> lifeless: do I need both, or just the second?
<lifeless> either
<lifeless> one is deeper than the other
<lifeless> invalidate will cause existing objects to be refreshed from the db on next use
<lifeless> reset will make them unusable and cause errors if you try to carry an object from before the reset forward
<thumper> ok
<thumper> anyone remember off the top of their head what the storm equivalent of the sqlobject SQLRelatedJoin is?
<poolie> lifeless, i wonder if you (or mrevell or someone) should post to the blog to explain about the critical/high/low categories
<poolie> re all the "why is this now Low" comments
<lifeless> hmm
<lifeless> possibly
<lifeless> but there have only been 3 or 4
<lifeless> in several hundred bug updates
<lifeless> (one bug - the wiki one - had a longer discussion that I'm counting as one)
<lifeless> I don't think this is really all that interesting to most users of lp
<poolie> maybe not
<lifeless> I think people are responding because they get notified
<lifeless> and would do so even if they knew that medium is equivalent to low, because the thing they care about is having their change made
<thumper> OMFG
<thumper> this code is soooooo... bad
 * thumper enfixorates
<wgrant> thumper: Which code?
<thumper> sprints
<wgrant> Heh.
<thumper> like the way to add an attendee, it iterated through all the attendees to see if the person matched
<poolie> huwshimi, did you think you fixed the bug where you have to press enter twice to save the tag list?
<poolie> doesn't seem fixed for me on lpnet
<lifeless> one enter selects the tag, the other saves
<lifeless> huwshimi: btw
<poolie> yes, i know
<lifeless> huwshimi: try putting a non-typeahead tag into a bug
<poolie> but that makes no sense if i've already completely entered a tag
<huwshimi> poolie: The fix is in lazr-js which I haven't got around to releasing a new version of yet.
<lifeless> huwshimi: its not done till its done
<poolie> ok thanks
<poolie> just wondered if it was in-progess or failed validation
<wgrant>      1029  OOPS-1902EE937  BugTask:+nominate
<wgrant> 1029 statements on +nominate!?
<lifeless> win
<poolie> sorry just one more thing there, what controls which tags show up in the portlet?
<poolie> oh, deja vu, this is a caching bug, isn't it?
<wgrant> Probably, yes.
<thumper> :(
<wgrant> Sprints are that bad?
<wgrant> Can I burn bug heat?
<lifeless> wgrant: no
<wgrant> :(
<wgrant> Does anybody use it enough that the performance penalty is worth it?
<lifeless> the performance penalty is not intrinsic to having heat
<wgrant> No.
<wgrant> But we need to either optimise or destroy.
<lifeless> optimise
<lifeless> distro are asking for heat improvements
<wgrant> Well, destruction is the ultimate optimisation :)
<wgrant> :(
<thumper> getUtility(IPersonSet).getPrecachedPersonsFromIDs isn't doing what I'd expect
<lifeless> if we don't deliver the (thing) then its not optimisation anymore
<wgrant> thumper: list()
<lifeless> thumper: are you listifying it and passing need_validity=True ?
<wgrant> thumper: You need to evaluate the result.
<thumper> wgrant: yeah got that
<thumper> and I ahve need_validity=True
<lifeless> ok
<lifeless> so what are you seeing
<thumper> but it is still doing the valid query for some of them
<thumper> and if I add more people
<thumper> then it does shit loads more
<thumper> even getting some people
<thumper> I'm wondering if I'm hitting the storm cache limit
<lifeless> where are you doing the preloading ?
<lifeless> there is no cache limit
<thumper> test says different
<wgrant> lifeless: Even in tests?
<lifeless> its much more likely that something is triggering loading and doing a person:fmt before your eager load happens
<thumper> I'm preloading in Sprint.attendances
 * thumper tries something
<lifeless> wgrant: if you test with getUserBrowser, should be same as prod
<lifeless> thumper: you're using the matcher for testing this, right ?
<thumper> yep
<thumper> and the DatabaseFunctionalLayer
<lifeless> BrowsersWithQueryLimit or whatever it was
<wgrant> Hmm.
<wgrant> I wonder.
<lifeless> wgrant: dangerous
<wgrant> Should I turn recalculateBugHeat into something that adds a commit hook?
<wgrant> As a quick fix for all this heat badness.
<lifeless> what do you mean
<wgrant> At the moment any change to a bug calls recalculateBugHeat, which then sets maxheat everywhere and blah.
<wgrant> Some views call addChange several times.
<lifeless> so
<lifeless> uhm
<lifeless> I don't know what you should do
<lifeless> I can share the thoughts I had looking at this previously
<wgrant> Well, I should move heat updating to a job.
<wgrant> But that's a bit hard.
<bac> hi thumper
<lifeless> updating the heat in a context should be exactly two single-row queries always.
<thumper> bac: hi
<bac> thumper: hey could you re-review my branch?  i addressed all of your issues.
<thumper> ok
<lifeless> wgrant: [in the current design/implementation]
<thumper> bac: diff is updating
<bac> thumper: despite what i said in the earlier comment, i've removed the line-height altogether
<lifeless> wgrant: the fact that its now looked like a very low hanging fruit
<lifeless> s/now/not/
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build #464: FIXED in 4 hr 44 min: https://hudson.wedontsleep.org/job/db-devel/464/
<thumper> lifeless: if I reduce the number of people I add to the sprint, I get expected behaviour.  with 30 people, it is fine, with 50 I get some validation queries, with 100 I get people and validation queries
<thumper> lifeless: so I'm guessing it is a caching issue
<lifeless> thumper: I've done some pretty large datasets and not see that, but you might be right
<lifeless> thumper: is it constant regardless of people in launchpad.dev (I use the harness to populate data for that sort of manual test)
<lifeless> wgrant: does that make sense, or have I been to minimal in my description ?
<wgrant> lifeless: It makes sense.
<wgrant> lifeless: I guess it should be reasonably cheap, but it's still going to be somewhat duplicated.
<wgrant> I'll fix the underlying stuff first.
<wgrant> (mostly the fact that DSP's pk isn't (distribution, sourcepackagename), so Storm likes to reretrieve it lots)
<lifeless> wgrant: things like: why do we query 'hottest bug' - if this bug has higher heat, update the cache, if not do nothing unless our heat lowered, and in that case [rare] query highest bug and set if and only if different
<wgrant> lifeless: Right.
<lifeless> wgrant: I don't see any reason to move heat to a job
<lifeless> wgrant: I wouldn't object to it per se, I just don't see anything driving it
<thumper> lifeless: so where is this storm cache defined?
<thumper> rendered locally, a sprint with 100 attendees was 30 queries
<thumper> 200 attendees was 30 queries
<thumper> 400 attendees was 102
<thumper> wondering if I was hitting the limit again
<lifeless> heh, do we even use stormsugar.py
<thumper> I was looking for it
<thumper> but didn't end up using it
<lifeless> thumper: grep for StupidCache
<StevenK> Is that seriously its name?
<thumper> bac: I think you were too aggressive in your code removal
<thumper> bac: with no link it should be a <span> and it is no more
<wgrant> lifeless: Well, for one thing it seems pretty unwise to be locking Ubuntu during every change to an Ubuntu bug.
<lifeless> wgrant: right, but only *one bug* can cause that
<lifeless> wgrant: note that all bug updates will take a row lock preventing deletes on ubuntu
<wgrant> lifeless: We hope.
<lifeless> wgrant: if there are two bugs fighting for peak heat
<lifeless> wgrant: we can deal
<wgrant> Maybe.
<wgrant> StevenK: What are you doing to Jenkins?
<StevenK> Moving it
<StevenK> Well, moving its DNS name to lpci.w.o
<wgrant> Aha.
<lifeless> hmm
<lifeless> does bug expiry look a the project group ?
<lifeless> if so, that would explain why its not working for launchpad
<lifeless> and there is no ui for setting it on project gourp
<wgrant> It doesn't seem to use the project group setting.
<StevenK> wgrant: It should announce URLs as lpci.w.o anyway. hudson.w.o is now a CNAME.
<lifeless> wth defines 'fixed elsewhere'
<lifeless> zomg
<wgrant> lifeless: Where have you seen that?
<lifeless> we support *per distroseries* bug expiry
<wgrant> Er, what?
<wgrant> Do we?
<lifeless> https://lp-oops.canonical.com/oops.py/?oopsid=1857C157#longstatements query 4
<StevenK> That seems like crack.
<StevenK> Pure crack.
<lifeless> its mindblowingly insane
<wgrant> lifeless: Er.
<wgrant> That's not per-distroseries bug expiry.
<wgrant> That's finding bugs in distroseries in distributions that are expirable.
<StevenK> Sigh, lp-oops DIAF
<StevenK> It always misbehaves for me
<wgrant> Authentication drops the query string, yes.
<lifeless> https://bugs.launchpad.net/ubuntu/+bugtarget-portlet-bugfilters-stats - thats where the 'fixed elsewhere' thing shows up
<lifeless> wgrant: thats true, but the query is nutjob noddy anyway
<wgrant> lifeless: Welcome to Launchpad.
<StevenK> wgrant: And https://lp-oops.canonical.com/ is the most unhelpful front page EVER
<lifeless> argh
<lifeless> extra_clauses.append("BugTask.bug IN " "(SELECT DISTINCT bug FROM BugCve)")
<lifeless> -not win-
<huwshimi> StevenK: Ouch, whatever server is running that site must be really hurting if that's running in debug mode
<wgrant> huwshimi: That's devpad.
<lifeless> the fixed elsewhere queries are buggy too - they ignore product series bugs
<huwshimi> wgrant: I'm guessing it doesn't have much ram free.
<StevenK> Meh, it's only 220MiB into swap
<huwshimi> well we probably shouldn't be running it in dev mode anyway
<huwshimi> or rather debug mode
<lifeless> huwshimi: why not?
<huwshimi> lifeless: In debug mode Django holds every query in ram
<lifeless> huwshimi: since start up?
<lifeless> huwshimi: or for a request lifetime?
<huwshimi> lifeless: I think so
<StevenK> The only thing that isn't playing so nice with carob is the PPR
<lifeless> generation ?
<StevenK> Yes
<lifeless> yeah, its started failing again
<huwshimi> lifeless: Sorry I mean, I think it is since start up
<lifeless> huwshimi: right, well that could be cause for concern, yes.
<huwshimi> lifeless: Let me just check the docs
<huwshimi> lifeless: "It is also important to remember that when running with DEBUG turned on, Django will remember every SQL query it executes. This is useful when you are debugging, but on a production server, it will rapidly consume memory." from: http://docs.djangoproject.com/en/dev/ref/settings/#debug
<lifeless> hahahahahahha
<lifeless> my respect for django just increased
<lifeless> so, could you please file a bug on oops-tools noting this?
<StevenK> lifeless: You had some? :-)
<lifeless> StevenK: its gone so far in one direction that things that make it worse, make it better
<StevenK> wgrant: Last SPR processed by p-s-c was for edgy.
<wgrant> StevenK: We are getting somewhere.
<StevenK> Slowly.
<StevenK> The OO.o SPRs are taking 1 minute.
<huwshimi> lifeless: We might have to check our other Django deploys too.
<huwshimi> (I believe we have others)
 * wgrant stabs postgres in the eye.
 * mwhudson has one too....
<mwhudson> (but it hardly uses the database so that might be ok)
<huwshimi> lifeless: https://bugs.launchpad.net/oops-tools/+bug/737327
<_mup_> Bug #737327: django should not be running in debug mode on production, it will eat all the RAM <OOPS Tools:New> < https://launchpad.net/bugs/737327 >
<thumper> https://code.launchpad.net/~thumper/launchpad/sprint-attendees/+merge/53941
<thumper> lifeless: and other review poke
 * thumper goes to get friday takeaways
<lifeless> thumper: reviewing
<lifeless> thumper: I have suggestions fwiw, but nothing mandatory so far
<thumper> ok, I'll look later
 * thumper afk
<jtv> wgrant: do you know where the contents of /srv/launchpad.net/ubuntu-archive/ubuntu-distscopy come from?
<wgrant> jtv: It's maintained by the script.
<wgrant> It keeps two copies of the dists tree around.
<wgrant> So it can do reasonably atomic updates.
<jtv> wgrant: what is "the" script?
<wgrant> "the script" being cron.publish-ftpmaster
<jtv> Ah
<wgrant> It grabs the backed up dists tree ubuntu-distscopy, and moves it to ubuntu/dists.new
<wgrant> It then runs the publisher on that.
<jtv> I see it copying _from_ ubuntu-distscopyâ¦ does it write _to_ ubuntu-distscopy further on?
<wgrant> See cleanup() near the top.
<jtv> So this is a chicken-and-egg situation?
<wgrant> Howso?
<wgrant> # Create backup dists folder, if this is the first time.
<jtv> That copies _from_ ubuntu-distscopy
<wgrant> Then it will move the empty folder to dists.new, swap dists and dists.new bring the new dists.new up to date, then move dists.new into ubuntu-distscopy
<wgrant> s/ bring/, bring/
<jtv> But publish-distro is failing to write a Release file there
<wgrant> Oh?
<wgrant> grumpy?
<jtv> No, a freshly-created distroseries.
<wgrant> Does it have any ComponentSelections?
<jtv> Errr
<wgrant> You need ComponentSelections in all your publishable series.
<jtv> what's a component selection?
<wgrant> Or you'll have no components.
<wgrant> And the publisher will crash.
<jtv> "Components?  But Andy Tanenbaum told me that Linux was monolithic!"
<wgrant> (not optimal, but it's better than lucilleconfig)
<wgrant> Bug #675042
<_mup_> Bug #675042: Release file generation fails for series without components <lp-soyuz> <soyuz-publisher> <Launchpad itself:Triaged> < https://launchpad.net/bugs/675042 >
<wgrant> Does that look like what you're seeing?
<jtv> Yes, except it's not in /var/tmp/archive.
<wgrant> So, try adding a ComponentSelection for main to your new series.
<jtv> Does the test publisher know how to do that for me?
<wgrant> I doubt it.
<wgrant> Not even the LOF does.
<jtv> The Line Of Flight?
<jtv> Launchpad Object Factory.
<jtv> I'm tempted to fix the bug, but in this case that'd probably be hiding incomplete test coverage for cron.publish-ftpmaster, wouldn't it?
<wgrant> It would.
 * jtv sings hi-ho, making componentselections we go
<wgrant> lifeless: Hi.
<lifeless> wgrant: hi
<lifeless> hi ho
<lifeless> hi ho
<lifeless> its off to work we go?
<wgrant> Ha ha.
<wgrant> Could you please explain analyze query 106 from https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1902A2028?
<wgrant> It is a wonderful 9978ms abomination.
<lifeless> \o/
<wgrant> It takes about 30s on mawson, and adding a very sensible index brings it down to 2s.
<lifeless> what bug
<wgrant> Bug #276950, probably.
<_mup_> Bug #276950: DistroSeries:+queue Timeout accepting many packages queue page <lp-soyuz> <queue-page> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/276950 >
<wgrant> Although this is probably only bad on GETs.
<lifeless> still running
<wgrant> Heh
<wgrant> I guess the index was not terribly bad when it was created.
<lifeless> still going
<lifeless> I'm going to go fiddle this portlet a bit more and check back in a bit
<wgrant> Thanks.
<lifeless> ok, on the bug
<lifeless> hot is 600ms
<wgrant> Bah.
<wgrant> Thanks.
<lifeless> cold is 201945.176
<StevenK> Over 2 minutes?
<lifeless> wgrant: so you're adding distinct there
<wgrant> Over three minutes.
<lifeless> to stop redundant rows?
<wgrant> I presume so.
<lifeless> lets see
<lifeless> 53 rows
<lifeless> 720 rows without the distinct
<lifeless> wgrant: so its doing about 10 times the work needed
<lifeless> wgrant: because the non distinct version is still 600ms
<wgrant> lifeless: It must think that a full scan is a better idea. It's probably right.
<lifeless> wgrant: rephrasing it may work
<lifeless> wgrant: its doing index scans
<lifeless> wgrant: an index that permits doing distinct inline may make it a lot better, we can ask stub to play with that later
<jtv> wgrant: I'm adding a makeComponentSelection to the factory.
<wgrant> jtv: Why?
<wgrant> the constructor is fairly trivial.
<wgrant> The only bonus is omitting an import.
<jtv> wgrant: makes it easy to say self.factory.makeComponentSelection(distroseries, "main")
<wgrant> Ah
<jtv> Accepts component, component name, or none.
<jtv> Also, it means that yokels such as yours truly won't have to worry about just how trivial they are to create.  :-)
<jtv> And meanwhile I'm on to the next hurdle: missing Sources files
<wgrant> Even with a component?
<jtv> It may be the thing you ran into with my parallelization branchâit's a-f.
<jtv> WARNING a-f: E: Could not open file /var/tmp/archive/distribution510022/dists.new/distroseries510031/main/source/Sources.bz2.new - open (2: No such file or directory)
<wgrant> Hmmm.
<wgrant> Does that dir exist?
<jtv> (Luckily I rigged this test with a "now tar up the temp directory so I can have a look" finale)
<wgrant> Heh
<jtv> oh wait, this is in /var/tmp
<jtv> It'd be nice to avoid messing with /var/tmp, wouldn't it?
<jtv> Yuck, crud piling up there!
<jtv> wgrant: the dists.new directory isn't present
<wgrant> What's in /var/tmp/archive?
<wgrant> Besides dozens of distribution dirs.
<jtv> Dozens of distribution-like dirs with "-cache," "-misc," "-overrides," "-partner," or "-temp" suffixed to their names.
<wgrant> What evil have to done to cron.publish-ftpmaster? It sounds like it's not moving the backup in.
<wgrant> s/have to/have you/
<jtv> I made it use a temp directory instead of /srv/launchpad.net, I sabotaged the lockfile usage, and I faked dsync-flist and commercial-compat.sh.  That's about it, really.
<wgrant> echo "$(date -R): Moving backup dists into place..."
<wgrant> Does it do that?
<lifeless> wgrant: so its about 300ms in the bpr lookup
<lifeless> and 300-500 (depending on plan) in the bpph filtering
<wgrant> lifeless: Right.
<lifeless> 210814 rows in bpph
<jtv> wgrant: yes it does do "Moving backup dists into placeâ¦"
<wgrant> jtv: Then why is dists.new not there? :/
<jtv> wgrant: well we solved that one by adding the ComponentSelection didn't we?
<wgrant> 16:37:12 < jtv> wgrant: the dists.new directory isn't present
<wgrant> dists.new's existence is not really related to ComponentSelection.
<jtv> wgrant: but how does the (non-)existence of dists.new relate to this problem?
<jtv> To the Sources problem, I mean.
<wgrant> Oh, I guess it probably moved it back away by the time you checked...
<lifeless> wgrant: I'd get stub to try that index
<wgrant> lifeless: Which?
<lifeless> wgrant: the one you said helped on mawson ?
<wgrant> Ah, right.
<wgrant> It still doesn't make it awesome. But I guess we'll see.
<lifeless> whats it on btw ?
<wgrant> bpph(archive, distroseries, status)
<wgrant> It cuts the BPPH search time massively. So probably not much help in the hot case.
<wgrant> Er, distroarchseries, but yeah.
<lifeless> ok, I'm going to stop fiddling
<lifeless> this is another case of overnormalisation
<wgrant> Thanks for trying.
<lifeless> bpr's shouldn't be shared
<wgrant> Oh?
<wgrant> For efficiency they should not.
<wgrant> For every other reason they should be.
<lifeless> or we should do a fact table for this
<lifeless> rather than chained content tables
 * lifeless handwaves
<wgrant> This is the sort of situation in which cross-table indices would work well. The underlying table is immutable, so there is no concern there.
<wgrant> But yes, we do reasonably urgently need to denormalise a bit.
<wgrant> We also need collection preload helpers.
 * lifeless sobs at related_projects in person.py
<lifeless> it aggregates product pillar counts but nothing else
<wgrant> Better than nothing.
<wgrant> Just.
<jtv> Time for the buying of the foodstuffs, ja?
<wgrant> Yay, only two CCW oopses so far today.
<lifeless> wgrant: false positives still, or actual things-we-need-to-act-on ?
<wgrant> lifeless: The latter.
<lifeless> wgrant: excellent
<lifeless> wgrant: well, excellent that we can tell.
<lifeless> what is the issue?
<wgrant> Well, they are sort of actual things. Some of our code is crashing, possibly due to broken remote bug trackers.
<wgrant> But they are real code bugs.
<lifeless> cool
<wgrant> Fragile parsers, basically.
<jtv-eat> hi henninge, hi rvba
<lifeless> I can has review? https://code.launchpad.net/~lifeless/launchpad/bug-711071/+merge/53956
<rvba> lifeless: Hi Robert, I have two MP which include db changes, but I can't get a hold of stub these days ... maybe you could take a look at these (the db part)?
<rvba> https://code.launchpad.net/~rvb/launchpad/db-distroseries-migrate-owner-to-registrant/+merge/53770
<rvba> https://code.launchpad.net/~rvb/launchpad/db-dds-diffpage-form/+merge/53913
<lifeless> rvba: both should be looked at by stub
<lifeless> I will note though that this
<lifeless> 718	- potemplate.distroseries.owner = self.factory.makePerson(
<lifeless> 719	+ makePerson = self.factory.makePerson
<lifeless> 720	+ potemplate.distroseries.distribution.owner = makePerson(
<lifeless> 721	password='test')
<lifeless> is an ugly change
<rvba> indeed
<lifeless> potemplate.distroseries.distribution.owner = \
<rvba> I'm still trying to change that
<lifeless>     self.factory.makePerson(password='test')
<lifeless> would be better
<rvba> the "makePerson = self.factory.makePerson" is only here for not having a really long line
<lifeless> indeed - see the \ instead
<rvba> that was my first move ... but I've been advised the opposite ;-) ... and made the change
<lifeless> who by?
<lifeless> there was a meme that \ is bad, but that is a broken meme
<rvba> Julian, the big guy :-)
<rvba> I don't mess why people that tall :-)
<rvba> s/why/with/
<lifeless> (heck, 80 character limits are bad - we really should make usre of the realities of our development environments)
<lifeless> rvba: so its neither here nor there
<lifeless> rvba: I will only say, that seeing that, I would do a driveby fix if I was in the area
<henninge> rvba: id do this:
<henninge> distribution = potemplate.distroseries.distribution
<henninge> distribution.owner = self.factory.makePerson(...)
<henninge> Hi jtv!
<henninge> s/id/I'd/
<rvba> henninge: this looks like a better option indeed
<rvba> lifeless: any 'final thought' on this?
<lifeless> rvba: what henninge suggests is ok
<lifeless> rvba: however as I say, stub should look at these patches, he should be around AFAIK
<rvba> lifeless: henninge thanks for the drive by advice :-)
<rvba> lifeless: I'll ask him about the db part yes
<henninge> rvba: np
<bigjools> morning all
<jtv> hi bigjools
<lifeless> bigjools: what are the specs of the machine you run lp tests on ? - and how long does it take?
<lifeless> [to run everything, that is]
<bigjools> I have 2 machines
<bigjools> quad core Phenom, with 3 very sad cores doing nothing, takes 270 minutes
<lifeless> thats amd right ?
<lifeless> but 4 core - moderately old?
<bigjools> the "quad core" i5 with SSD takes about 240 IIRC but not run it on there for a while
<wgrant> 270 minutes? Is that with or without Windmill?
<bigjools> w/o
<bigjools> it's about 4.5 hours now
<wgrant> When I got my new machine in December, it was slightly under 3 hours.
<bigjools> yes, we have had a massive regression in time
<wgrant> ec2 is back down around 3.5
<wgrant> Not much worse than it used to be.
<wgrant> Since I killed Windmill.
* adeuring changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: adeuring | https://code.launchpad.net/launchpad-project/+activereviews/
<lifeless> adeuring: https://code.launchpad.net/~lifeless/launchpad/bug-711071/+merge/53956
<adeuring> lifeless: I'll look
<lifeless> thanks
<rvba> adeuring: Hi, I'd appreciate a review of https://code.launchpad.net/~rvb/launchpad/db-dds-diffpage-form/+merge/53913
<adeuring> rvba: sure, let me just finish a review for Robert
<rvba> adeuring: sure
<rvba> thx
<bigjools> wgrant: my last ec2 took 4 hours
<wgrant> bigjools: Mine take between 3:30 and 3:45
<bigjools> lifeless: sorry missed your question, yeah the Phenom is 2 years old now
<lifeless> I'm considering a new desktop
<bigjools> but parallel test runs... could breathe new life into it :)
<lifeless> yeah
<lifeless> will want 2-4GB per core though
<bigjools> lifeless: if money is not a problem I'd get an i7
<lifeless> [because of silly overheads]
<bigjools> or perhaps there's better now
<lifeless> bigjools: I have a gen-1 i7, limited to 6GB memory
<bigjools> they're not slack :)
<lifeless> bigjools: runs like a rocket, but nowhere near enough memory to use all the cores well
<lifeless> not even for compiling drizzle ;P
<bigjools> lifeless: memory contention?
<bigjools> bus ^
<lifeless> footprint
<bigjools> ah
<lifeless> 8 g++ instances >> 6GB of ram, for instance
<adeuring> lifeless: r=me
<lifeless> adeuring: thanks!
<lifeless> should fix 80 timeouts/day
<lifeless> huh, thats oddd
<lifeless> there is no 'set commit message' widget on https://code.launchpad.net/~lifeless/launchpad/bug-711071/+merge/53956
<lifeless> or am I crazy?
<lifeless> there is now. weird
<deryck> Morning, everyone.
<rvba> stub: thanks for your review on my branch (lp:~rvb/launchpad/db-distroseries-migrate-owner-to-registrant) ... a question though:
<rvba> stub: patch-2208-53-0.sql was adding a new 'registrant' column to distribution. This branch renames the 'owner' column into registrant for distroseries. Could you elaborate on where the conflict is?
<stub> rvba: The conflict is in my mind then :)
<stub> I'll change that now
<rvba> stub: all right thx ;-)
<bigjools> lol
<adeuring> rvba: r=me
<allenap> deryck: Javascript tests, lib/lp/registry/javascript/tests/test_milestone_table.html for example, are messed up; the skin appears missing. Applying http://paste.ubuntu.com/582035/ fixes it. I'm wondering if my build changes last week broke this. Do you know much about this?
<rvba> adeuring: thx for the review, just pushed the changes you suggested.
<adeuring> rvba: great, thanks!
<rvba> stub: thank you for your reviews. I just pushed the modifications you requested.
<deryck> allenap: I think the CSS was broken because we changed to fetchCSS: false in the test. I never worked out what CSS it wanted because it was just trying the Yahoo CDN with some massive list of CSS.
<deryck> allenap: so if adding that file link fixes it, awesome!
<deryck> allenap: we had to set fetchCSS false to better play with Windmill as the test runnner.
<allenap> deryck: Cool. As long as it wasn't me :) I might @import it from test.css.
<deryck> allenap: sounds good.
<allenap> Thanks for your help.
<deryck> np.  Thank you for working on this!
<wgrant> jml: Your assignment of that bug is encouraging.
<jml> wgrant: I'm just looking into it right now. No amazing insights.
<wgrant> Oh :(
<wgrant> Well, good luck...
<jml> wgrant: do you have any clue as to why the comment says "run a job that will not time out first, followed by a job that is sure to time out." when the code seems to only run one job?
<stub> rvba: Looks fine
<wgrant> jml: StuckJob is two jobs.
<jml> wgrant: ta.
<wgrant> See iterReady... the first one has an arg of 1, the second 2. Only when the arg is 2 does it sleep.
<jml> yeah. I see now.
<wgrant> It is not really sensible.
<jml> it seems a bit indirect.
<jml> huh. If I'm reading this correctly, TwistedJobRunner only runs one job at a time.
<lifeless> night y'all
<bigjools> lifeless: nn - just replying to your ha uploaders email BTW
<lifeless> bigjools: cool,thanks
<bigjools> nearly done if you want to wait :)
<lifeless> bigjools: take the time to reply fully
<lifeless> bigjools: I'm just lining up the ducks to point losas at after we finish the appserver capacity work, so there isn't a panic - we can knock ideas back and forth for maybe a couple of weeks without impacting schedules
 * lifeless halts()
<bigjools> lifeless: np at all
<jml> :(
<wgrant> jml: No luck?
<wgrant> Or perhaps anti-luck?
<jml> wgrant: some success.
<jml> wgrant: test_shhh manipulates sys.path on import and doesn't clean it up, which is why the _pythonpath thing doesn't bite us on the full suite run.
<wgrant> jml: Ahhh.
<jml> ampoule tries to get the correct sys.path for the "must be available" packages by importing them in the current process and manipulating their path.
<wgrant> Yep.
<wgrant> I got that bit.
<jml> OK.
<wgrant> It's a bit odd.
<jml> went down a bit of a rabbit hole assuming it was something to do with zope.testing respawning.
<wgrant> As did I, but then I deleted some stuff to prevent that and it was still broken.
<wgrant> I didn't think that test_shhh might have been doing path evil.
<jml> I'm going to change test_timeout to temporarily extend sys.path and fix up the test_shhh thing in another branch.
<jml> ... and file a bug about providing a decent mechanism for doing this with python-fixtures.
<gmb> adeuring: Are you accepting reviews today?
* bac changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: adeuring, bac | https://code.launchpad.net/launchpad-project/+activereviews/
<bac> gmb: i am if adeuring isn't
<adeuring> gmb: yes, but i'd like to start my lunch break right now.
<gmb> adeuring: Ok, no worries.
<gmb> So, bac. Hi :).
<gmb> bac: https://code.launchpad.net/~gmb/launchpad/non-js-muting-bug-734732/+merge/53981 if you please
<bac> good morning
<bac> hi mrevell
<mrevell> Hello bac
 * jml sets the test to run 50 times over and takes a brief programmer's break
<abentley> deryck: I'm trying to use jquery, but I can't figure it out.  The queries in lib/lp/code/windmill/tests/test_recipe_index.py succeed on my page, and they shouldn't, because it's a different page.
<abentley> deryck: e.g. self.client.waits.forElement(jquery=u'("div#edit-build_daily a.editicon.sprite.edit")', timeout=5)
<deryck> ok, that's a bit scary.
<deryck> abentley: you don't get an error with that on your page?
<abentley> deryck: No.
<abentley> deryck: it's green.
<abentley> deryck: I'm using a pdb session to test.
<abentley> deryck: meanwhile, $("ul#branch") fails, even though firebug identifies an element that way.
<abentley> Apparently, the $ makes the difference.
<deryck> abentley: can you commit the test you have to your branch so I can pull and run it?
<abentley> deryck: sure, one sec.
<abentley> deryck: pushed to bzr+ssh://bazaar.launchpad.net/~abentley/launchpad/js-translation
<deryck> abentley: ack.
<deryck> oh yippee, another Windmill hang!
<abentley> deryck: run with bin/test -t test_sharing_details windmill --layer=WindmillLayer
<deryck> abentley: let me get a bt on my current Windmill hang, and then I'll kill it and look at yours.
<abentley> deryck: thanks!
<gmb> adeuring, bac: I have a minor JS branch for review, if you'd be so kind: https://code.launchpad.net/~gmb/launchpad/fix-js-errors-bug-737557/+merge/54007
<bac> gmb: i'll do it
<adeuring> gmb: I'll look
<bac> i win!
<adeuring> bac:  ok ;)
<gmb> Thanks :)
<gmb> bac: https://code.launchpad.net/~gmb/launchpad/fix-js-errors-bug-737557/+merge/54007
<gmb> Dur.
<gmb> Sorry, forgot I'd pasted it.
<deryck> abentley: ok, sorry.  got what I need now.  Turning to your branch.
<bac> sinzui: are you saying you want me to put the line-height: 20 back in?
<sinzui> bac: no. I do not want that instruction anywhere in the code base
<bac> sinzui: good -- it's gone
<sinzui> \o/
<sinzui> Sorry about the confusion. I got very distracted yesterday evening. I composed my reply to you, but never sent it
<deryck> abentley: ok, so this has me terrified of the reliability of jquery selectors now.
<abentley> deryck: sorry.
<deryck> abentley: if you try to click jquery=u'("div#edit-build_daily a.editicon.sprite.edit")' it will fail.  But using waits.forElement or asserts.assertNode, it passes.
<deryck> abentley: which clearly makes no sense and is wrong
<leonardr> adeuring or bac, can you review https://code.launchpad.net/~leonardr/lazr.restful/operation-sanity-check/+merge/54014 ?
<abentley> deryck: so avoid using with those?  I've been plugging away using xpath, and it's... beauty-challenged.
<adeuring> leonardr: i'll look
<leonardr> great
<deryck> abentley: I'm not sure what to recommend at this point.  xpath is equally problematic. You often get false positives the same way, from crafting xpath expressions you don't understand.
<leonardr> adeuring: i'll fill in some background in a comment
<leonardr> since the feature is new
<adeuring> leonardr: great, thanks!
<deryck> abentley: or fragile tests from poorly crafted xpath.
<deryck> abentley: and there is no way to hang IDs off this html?  because of the menu stuff.
<abentley> deryck: I wonder if any of our intermittant failures are from waits.forElement(jquery= ?
<deryck> abentley: if it's a failure and not a hang, that's easy to check -- i.e. was it a jquery line?  Hangs don't appear related to this at all.
<LPCIBot> Project devel build #553: FAILURE in 4 hr 58 min: https://lpci.wedontsleep.org/job/devel/553/
<abentley> deryck: Yes, not hangs.  But failing to wait long enough would cause a failure on a later line.
<deryck> right
<deryck> indeed, it's something to look out for.
<leonardr> adeuring: background is added
<deryck> abentley: but we're only using it in recipe tests at this point, I think.
<abentley> deryck: I could use ids, but classes seemed a better approach, because I could consistently use the same value-- class=incomplete, not id=branch-incomplete
<deryck> abentley: I don't see an issue with specific IDs.  Reuse is not always a virtue when dealing with the DOM and js and js testing.
<abentley> Then I can do foo=Y.one('#branch'), foo.one('.incomplete')
<abentley> It just makes it simpler once you've got the checklist node.
<deryck> but you can hang id="branch-incomplete" off the element just for the sake of testing.  sure, it's hacked on, but if the test is less fragile.
<deryck> abentley: I leave it to you, though.  Just warning that xpath is usually a source of trouble at some point.  But maybe this is easy enough xpath.
<abentley> deryck: I would rather not have both class=incomplete and id=branch-incomplete.  That's a DRY violation to me.
<deryck> abentley: sure, but I'm arguing DRY is not that big a deal in HTML. ;) :)
<abentley> deryck: But I hadn't realized the testing implications before, so perhaps it's better to change my approach now.
<deryck> abentley: yeah, HTML is not code. ;)
<abentley> deryck: i'm saying if I add ids, I'll remove the classes.  No need for both.'
<deryck> abentley: but I do agree it's generally better if you can avoid.  less characters, less to download.  but our testing story is weak without the ids.  and you could just use branch-incomplete and not the class.
<deryck> agreed ^^ :-)
<deryck> abentley: ok, so I'll lunch now, if you've got everything you need.
<abentley> deryck: anyhow, I think I have to use xpath with the picker.
<deryck> abentley: maybe so.  this is one area where we've had to use it and can't avoid it in the past.
<abentley> deryck: cool.  Thanks.  Enjoy lunch.
<deryck> ok, cool.  thanks!
<gary_poster> mrevell, hi.  I'm trying to find the accordion prototyping round 2 version 2 slides
<gary_poster> I found https://dev.launchpad.net/LEP/BetterBugSubscriptionsAndNotifications/Testing/AccordionPrototypingRound2
<gary_poster> which has version 2
<mrevell> Ah, glad you got it gary_poster
<gary_poster> but I can't find version 2
<gary_poster> and https://wiki.canonical.com/Launchpad/UserResearch/BugSubsFeb2011/Round2 points to the wrong location (round 1)
<mrevell> Oh, sorry
<mrevell> Let me look
<gary_poster> thank you
<gary_poster> mrevell, found it somehow or other :-)
<mrevell> gary_poster, I've created a page at https://dev.launchpad.net/LEP/BetterBugSubscriptionsAndNotifications/Testing/AccordionPrototypingRound2/Version2 which links to the 5 slides and I've updated the wiki page
<mrevell> on Canonical
<gary_poster> awesome thank you mrevell
<mrevell> no probs :)
<adeuring> leonardr: r=me. Thanks for the new checks and for the explanation!
<sinzui> bac: will you have time to review https://code.launchpad.net/~sinzui/launchpad/tea-and-biscuts/+merge/54041 today?
* adeuring changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: bac | https://code.launchpad.net/launchpad-project/+activereviews/
<abentley> deryck: turns out there's some picker logic in testing.windmill.widgets, so we don't need to use xpath for it.
<deryck> excellent!
<deryck> I forgot about that actually.  Sorry.
<abentley> deryck: np
<abentley> deryck: how would you test for the absence of a class on a node?  Currently, I'm doing self.client.waits.forElement(xpath='//*[@id="branch"]/*[@class="complete sprite yes"]') to ensure the node is not unseen.
<deryck> abentley: that looks fine. Are you worried that by not being explict about the absence of unseen that the test will pass but not be correct?
<abentley> deryck: I'm just wondering if you would avoid using xpath for that, and if so, how.
<deryck> abentley: well, I would have mad the ID "branch-complete" or given it some other unique ID, to avoid the xpath.  As we talked about eariler.
<abentley> deryck: okay, but if you did that and did forElement(id="branch-complete"), how would you know what the class was?
<jcsackett> leonardr: ping
<leonardr> jcsackett, yo
<jcsackett> hi. do you have a few minutes to mumble or skype about webservice stuff?
<deryck> abentley: I wouldn't test that in Windmill.  I would do that in the YUI test.  I wouldn't worry much about DOM in the Windmill test. You really should only care the data was saved correctly.
<leonardr> jcsackett, sure, 1 minuteeeeeee
<jcsackett> leonardr: thanks.
<abentley> deryck: If I don't wait until the dom is updated, how do I know when it's safe to check the model?
<deryck> abentley: do you not add some class when you remove the unseen class, i.e. changing it to "seen"?
<sinzui> deryck: all pickers are using the wrong headings. I do not think the instructions should be a heading at all in fact. That will make the text smaller
<deryck> sinzui: agreed.  Just plain on text.
<abentley> deryck: No, I don't add a "seen" class.  But I could use the other element, where I add "unseen".
<sinzui> deryck: I can start this in a hour or so...I was irked by the very add-member issue this morning when testing
<deryck> abentley: yes, that would work.
<abentley> deryck: Okay, how would that work?
<abentley> deryck: forElement(id="branch-incomplete") still won't wait until the DOM is updated.
<deryck> abentley: I'm getting confused.  :-)  I thought you just said you could wait for the unseen class on the other element.  so waits.forElement(blah blah blah about the other element now).  That won't work?
<deryck> sinzui: thanks!  Did something change recently to cause this?
<abentley> deryck: I've got lots of elements that have the unseen element, so waits.forElement(class="unseen") will not wait until branch-incomplete gains the class.
<leonardr> jcsackett: expose_as_webservice_entry()
<abentley> s/unseen element/unseen class/
<leonardr> exposed()
<sinzui> deryck: sort of. Heading sizes were updated to be like the guidelines. but in this specific case, the instructions exceeded the default width of the picker last December when we fixed security in team membership
<leonardr> expose_as_webservice_collection()
<abentley> deryck: it's fine if the answer is "I would use xpath in that case".
<deryck> abentley: can you craft an xpath expression or jquery selector?  That would be my suggestion, yes.  Use one of those.
<deryck> sinzui: ah, gotcha.
<abentley> deryck: I am already using an xpath expression, so I'll just keep using it, then.
<deryck> abentley: ok.
<abentley> deryck: you said to avoid xpath where possible.  I was just exploring where that's possible.
<deryck> abentley: yeah, maybe it's not possible here.  I'm always worried when we're checking for the visibility of an element.  But maybe there's no other way to know the page changed.
<abentley> deryck: when the page changes, one element becomes visible, the other is hidden, and the href and contents of a link are changed.
<deryck> abentley: so I'd check for the contents change then.
<deryck> abentley: textIn
<deryck> ah, crap, that's an assert
<abentley> deryck: all the uses I see are client.asserts.assertTextIn.  Is there a waits version?
<deryck> abentley: yeah, that's what I just noticed.  Let me look
<deryck> abentley: so that won't work.  Sorry.  You could do waits.forElementProperty to do the classname or href nicer than xpath.
<abentley> deryck: Ah, okay.
<jml> \o/
<jml> OK. I think I've got this failure reproducing consistently.
<jml> Time to make some food and then track the bugger down.
<deryck> Have a nice weekend, everyone.
<bac> hi jcsackett, where do i see the diff overlay you mention?
<bac> jcsackett: nm, i found it
<jcsackett> bac: good thing. i never got the ding i usually get to let me know someone's pinging me. :-)
<bac> nice branch jcsackett, r=ac
<bac> er, r=bac
<jcsackett> thanks, bac.
<jcsackett> sinzui: standup at 5:30 or at 6, given our other members are enjoying the weekend?
<sinzui> We can do it now if you like
<jcsackett> sinzui: right now i am at a coffee shop. i'm figuring out when i need to make the short walk back home. :-)
<sinzui> Ping me when you are ready
<jcsackett> sinzui: righto.
* bac changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | firefighting: - | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews/
<leonardr> wallyworld, do you know where thumper is? i'd like to do the standup and then take a nap
<leonardr> unless... oh, wait, it's saturday for oyu, isn't it?
<sinzui> leonardr: yes it is
<sinzui> take a nap
<leonardr> ok, never mind then. nap time! talk to you on monday/tuesday
<jcsackett> sinzui: i am now in a mumble friendly place. i can do standup whenever you like.
<jml> I think ampoule handle timeouts badly
<sinzui> jcsackett: starting mumble
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #554: FIXED in 5 hr 16 min: https://lpci.wedontsleep.org/job/devel/554/
<LPCIBot> Project windmill build #72: FAILURE in 1 hr 16 min: https://lpci.wedontsleep.org/job/windmill/72/
#launchpad-dev 2011-03-19
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project windmill build #73: FIXED in 1 hr 8 min: https://lpci.wedontsleep.org/job/windmill/73/
<wgrant> It looks like Codehosting is now the proud owner of the top 9 OOPSes.
<lifeless> \o/
<lifeless> erm
<lifeless> :P
<lifeless> grr
<lifeless> 22 /   14  POFile:+translate
<lifeless> need to do -more-
<lifeless> nice
<lifeless> https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1903G688#longstatements
<lifeless> one slow query, nine repeats
<lifeless> and only one run of it was slow
<lifeless> and its 53ms hot
<wgrant> https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1903BZR257061 is really great.
<lifeless> indeed
<cody-somerville> What is ZOP?
<wgrant> Zero OOPS Policy
<lifeless> cody-somerville: https://dev.launchpad.net/PolicyAndProcess/ZeroOOPSPolicy
<wgrant> Huh.
<wgrant> Bug expiry just happened.
<wgrant> Everywhere.
<lifeless> \o/
<lifeless> of course, now we have to check whether we were using incomplete correctly
<wgrant> Yes.
<lifeless> meh
<lifeless> we have enough bugs ;)
<lifeless> ugh
<wgrant> Oh?
<lifeless> lots were marked incomplete and then prioritised etc without changing the status. :(
<lifeless> e.g. https://bugs.launchpad.net/launchpad/+bug/143972
<_mup_> Bug #143972: Package description maintenance <lp-soyuz> <Launchpad itself:Expired> < https://launchpad.net/bugs/143972 >
<wgrant> :(
<lifeless> hmmm, a number of celso ones. grah
<lifeless> yay found one that should have expired
<lifeless> (bug 584442)
<_mup_> Bug #584442: launchpad crashses all the time when I try to submit a bug report <lp-bugs> <Launchpad itself:Expired> <Ubuntu:Invalid> < https://launchpad.net/bugs/584442 >
<lifeless> wgrant: hey, did you see my (exagerated length) thoughts on overhauling status?
<wgrant> lifeless: I did, and was most pleased indeed.
<lifeless> so, it made sense to you?
<wgrant> It seemed mostly sensible. Let me go through it again and tear it to pieces.
<wgrant> lifeless: I cannot really tear it to pieces.
<wgrant> It is basically an extension of what I've thought for a while.
<LPCIBot> Project windmill build #74: FAILURE in 2 min 45 sec: https://lpci.wedontsleep.org/job/windmill/74/
<LPCIBot> Project windmill build #75: STILL FAILING in 0.45 sec: https://lpci.wedontsleep.org/job/windmill/75/
<StevenK> Whoa
<StevenK> 2 minute failure?
<wgrant> StevenK: The usual repository-not-branch issue.
<StevenK> Yes, I've just deleted the salve.
<StevenK> *slave
<StevenK> Build scheduled.
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project windmill build #76: FIXED in 1 hr 8 min: https://lpci.wedontsleep.org/job/windmill/76/
<lifeless> wgrant: hey, coco and germanium are the machines running soyuz (s)ftp, right?
<StevenK> lifeless: Yes
<lifeless> StevenK: thanks
<lifeless> sigh https://bugs.launchpad.net/launchpad/+bug/32528
<_mup_> Bug #32528: sync uploads get Maintainer field overriden in announce mails but shouldn't <email> <lp-soyuz> <soyuz-upload> <Launchpad itself:Expired> < https://launchpad.net/bugs/32528 >
<wgrant> Bah, python2.6 is bad.
<StevenK> Noooo, no more google maps feature
<StevenK> lifeless: Thanks for the ad tip (re: Facebook)
<lifeless> u
<lifeless> np
<lifeless> wgrant: whats the bug # for the process-upload race condition ?
#launchpad-dev 2011-03-20
<lifeless> moin
<lifeless> mwhudson: I wouldn't ping a losa - just ask on the ticket... unless you want to wait till wednesday to remember
<mwhudson> oh right
<lifeless> mwhudson: because it will need tom specifically
<mwhudson> how do i ask on the ticket?  just email [number]@..canonical.com?
<lifeless> or login to rt.admin.c.c with the shared credentials and click on 'reply' on the ticket
<mwhudson> ahh
<mwhudson> so it wasn't you asking a crap question, it was tom :)
<mwhudson> commented on the rt, it's a mystery though (we don't call logger.critical a whole lot)
<lifeless> mwhudson: indeed, I thought I said that in my mail :)
<lifeless> thanks
<lifeless> mwhudson: when triaging, please make sure importance is set
<mwhudson> lifeless: ah yes, loggerhead follows lp rules now
<mwhudson> lifeless: assuming that's the one you meant?  i was just flipping it out of expired
<lifeless> mwhudson: yeah
<lifeless> lots of bugs caught by expiry suddenly catching up with ubuntu and moving onto other things
<lifeless> hmm
<lifeless> wgrant: are you guys on chr this week?
#launchpad-dev 2012-03-12
<StevenK> wgrant: Can you link me that duplicate_of json URL so I can see that the right stuff is still redacted on qas?
<wallyworld_> StevenK: it's a holiday in victoria today afaik
<StevenK> wallyworld_: It is, but he was talking earlier. :-)
<wallyworld_> of course :-)
<wgrant> $ grep duplicate_of.*json /home/wgrant/irclogs/Freenode/#launchpad-dev.log | head -n 1
<wgrant> 15:53 < wgrant> Grab https://bugs.qastaging.launchpad.net/api/devel/bugs/718213/duplicate_of?ws.accept=application/json as anonymous
<_mup_> Bug #718213: Can't access due to content. <Internet Archive - Tech Support:New> < https://launchpad.net/bugs/718213 >
<wgrant> StevenK: ^^
<StevenK> Ah ha!
<StevenK> wgrant: I think 1234 is no longer private ...
<wgrant> That can be easily fixed.
<StevenK> But no redacted in the JSON. :-(
<wgrant> Not if you can see the bug, no.
<StevenK> wget should be anonymous
<wgrant> https://api.qastaging.launchpad.net/devel/bugs/718213/duplicate_of?ws.accept=application/json looks thoroughly redacted to me.
<_mup_> Bug #718213: Can't access due to content. <Internet Archive - Tech Support:New> < https://launchpad.net/bugs/718213 >
 * StevenK suspects caching
<StevenK> wgrant: Okay, if it looks nothing is leaking, I'll mark the bug as qa-ok.
<wgrant> Sounds reasonable.
<StevenK> But my JSON here is still non-redacted. :-(
<StevenK> wallyworld_: You should be fine to put up a NDT in ~ 6 minutes.
<wallyworld_> ok will do
<StevenK> Just wait 6 minutes or so first and confirm the deployment report is green
<wallyworld_> yep, finishing off something first anyway
<thumper> wallyworld_: who is on holiday in Oz today?
<wallyworld_> thumper: supposedly wgrant
<wallyworld_> victoria has a holiday
<wallyworld_> i think that's all?
<thumper> south australia?
<thumper> not perth?
<wallyworld_> maybe, i'd have to check
<thumper> wallyworld_: where are you looking?
<wallyworld_> thumper: vic, act, sa, tas
<wallyworld_> http://www.publicholidaysaustralia.com.au/
<thumper> ta
<wallyworld_> np
<wallyworld_> i ask professor google :-)
<thumper> Eight Hours Day?
<thumper> really?
<thumper> bwahaha
<StevenK> How is that funny?
<wallyworld_> thumper: i can't believe that one either
<thumper> the name is funny
<wallyworld_> stupid if you ask me
 * thumper gives StevenK a sense of humour
<wallyworld_> thumper: has the gtk/qt decision been made? seems gt is the way forward given some of the postings to the mailing list?
<wallyworld_> qt
<thumper> it isn't gtk/qt, but nux/qt
<thumper> and no
<wallyworld_> ok, thanks
<wgrant> thumper: Eight Hours Day is only TAS. VIC isn't that insane :)
<nigelb> wgrant takes holidays?
<mwhudson> not from irc apparently
<nigelb> heh
 * StevenK kicks the qa-tagger
<StevenK> staging is running r11434, but the deployment report only shows up to r11432.
<StevenK> INFO:qa-tagger:Revision 11433 not deployed to LP-staging.
<StevenK> INFO:qa-tagger:Revision 11433 not deployed to db-stable
<StevenK> LIES!
 * StevenK tries to work out why test_sharingservice breaks with his change.
<StevenK> wallyworld_: ^ ?
<wallyworld_> StevenK: what's your change?
<StevenK> wallyworld_: http://pastebin.ubuntu.com/879931/ -- in the branch that creates APs when a product or a distro is.
<wallyworld_> StevenK:  looks at at first read. what's the error?
<StevenK> wallyworld_: The final assert in _test_deletePillarSharee fails with 800 lines of JSON
<StevenK> It's expecting USERDATA, but EMBARGOEDSECURITY is there instead.
<wallyworld_> hmm. not sure. i'll make your change and have a look.
<StevenK> wallyworld_: You'll need to merge in lp:~stevenk/launchpad/product-distribution-accesspolicy first
<wallyworld_> ah, yeah. i'm in the middle of a branch. i'll shelve my changes
<StevenK> wallyworld_: You'll also need to run make schema
<wallyworld_> ok
<wallyworld_> StevenK: just doing the merge and mack schema and test_deleteProductShareeAll I get
<wallyworld_> duplicate key value violates unique constraint "accesspolicy__product__type__key"
<wallyworld_> i'll have to track that down unless you've seen it before?
<wallyworld_> ah never mind, i need to add your changes i think
<StevenK> Right
<wallyworld_> and i see the json mismatch
<StevenK> wallyworld_: Good to know it has stumped you too :-)
<wallyworld_> it's not obvious, look through the code
<wallyworld_> StevenK: i think i know what it is
<wallyworld_> the access_polices and information_types lists were previously aligned
<wallyworld_> in that one was made from the other
<StevenK> Right
<wallyworld_> now, access_polices comes from a db query
<wallyworld_> so this will be wrong
<wallyworld_>         another_person_data = self._makeShareeData(
<wallyworld_>             another, information_types[:1])
<wallyworld_> the test can no longer rely on the first info type being the one to use
<wallyworld_> make sense?
<StevenK> Yup
<StevenK> I wonder if information_types = [ap.type for ap in access_policies] will fix it
<wallyworld_> i may well
<wallyworld_> it
<wallyworld_> since the lists would be "aligned" again
<StevenK> Success!
<wallyworld_> yay
<StevenK> +3/-9. That's my kind of test fix.
<wallyworld_> yeah
 * StevenK lp-lands this branch, finally
 * wallyworld_ unselves
<wallyworld_> unshelves
<StevenK> wallyworld_: Thanks for the help
<wallyworld_> np.
<wallyworld_> StevenK: there's code in sharePillarInformation() which creates access policies if they don't exist. i guess that can be deleted now too
<StevenK> wallyworld_: In SharingService itself? No, that's still good for things like PROPRIETARY
<wallyworld_> makes sense
<wgrant> StevenK, wallyworld_: that should probably go away
<wgrant> It's not yet clear how we'll create PROPRIETARY, but it certainly won't be implicit like that.
<StevenK> Right, I'll drop it
<wallyworld_> the code today only provides proprietary as an option if the project is commercial
<wallyworld_> on the +sharing view i mean
<wallyworld_> in the sharing picker
<wgrant> Even so.
<wallyworld_> sure, i wasn't saying it should be implicit, just reiterating what's there now
<wgrant> We'll see how sinzui's entitlement stuff ends up.
<wgrant> And then work out HTF to do this.
<wallyworld_> yep
<StevenK> So, dump it in this branch, or leave it for now?
<wgrant> Kill, crush, destroy.
<StevenK> Which may well end up creating more test failures.
<wgrant> True, if it deals with proprietary.
<wgrant> So perhaps leave it.
<wgrant> If it causes failures.
<StevenK> I don't know if it will or not
<StevenK> I was willing to push it out and toss it at ec2 and see what happens
<adeuring> good morning
* adeuring changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugtasks: 4*10
<StevenK> adeuring: Did you enjoy your OCR duties over the weekend? :-)
<adeuring> StevenK: ;)
* benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: benji | Firefighting: - | Critical bugtasks: 4*10
<StevenK> Ahh, DST, you heartless bitch.
<czajkowski> and hello there to you too StevenK
<StevenK> Haha
<deryck> adeuring, abentley -- I completely forgot my audio troubles.
<deryck> adeuring, abentley -- I do however have everything else working :)
<abentley> deryck: That's good.
<abentley> deryck: Maybe the conference system?
<deryck> abentley, yeah, let's do that.
<adeuring> deryck: sounds good. I got a somewhat enigmatic error when I tried to upgrade to precise...
<deryck> flacoste, can orange use your conference code for a conference call?
<flacoste> deryck: sure, be my guest
<deryck> flacoste, audio input not working from my upgrade yet.
<deryck> flacoste, and thanks!
<flacoste> deryck: did you report it?
<deryck> abentley, adeuring -- use flacoste's conf number.
<deryck> flacoste, I did not.  But I will definitely.
<adeuring> deryck: ok
<flacoste> deryck: asap is best if you want to have sound in precise :-)
<deryck> flacoste, ok :)  I had other issues with lp and forgot about my audio troubles actually.
<adeuring> abentley: let's use mumble
<abentley> adeuring: okay.
<abentley> adeuring: lp:~abentley/lazr.jobrunner/run-via-celery
<abentley> adeuring: I think we should start using a wiki page to coordinate.  Cool?
<adeuring> abentley: sounds good
<abentley> deryck, adeuring: I've posted my current notes to a wiki page: https://dev.launchpad.net/CeleryJobRunner
<deryck> abentley, nice, thanks
<deryck> adeuring, abentley -- I'm going offline for lunch to run errands here shortly.  But will be back after my lunch.
<deryck> adeuring, abentley -- and I have a working headset now :)
<abentley> deryck: excellent!
<adeuring> abentley: cool
<abentley> adeuring: I forgot to talk about resources with you, but it seems we should stick to 2 worker processes, i.e. one per lane.
<adeuring> abentley: ok
<jcsackett> sinzui: do you know if wallyworld_ had to do anything special to get his services work running on launchpad.dev? it's failing on post, but i don't see anything in any of his emails/docs.
<sinzui> jcsackett, We had discussed a feature flag, but decided it was not needed now
<jcsackett> ok, so i may be encountering a bug.
<jcsackett> i'll investigate--it's blocking creating sample data. i can always use the factory if i don't find a solution soon.
<jcsackett> sinzui ^
<sinzui> :(
<timrc> What needs to happen to subscribe a private team to a code branch? Do I need to be a member of that team?
<sinzui> benji, do you have time to review https://code.launchpad.net/~sinzui/launchpad/entitlement-2/+merge/97074
<benji> sinzui: sure
<sinzui> benji, one moment. I for got the dependent branch
 * sinzui makes diff smaller
<sinzui> benji, https://code.launchpad.net/~sinzui/launchpad/entitlement-2/+merge/97078 will show a smaller diff
<benji> sinzui: ok
<czajkowski> Is a package in a persons  ppa a valid dependency for a package being daily-built by launchpad?
<sinzui> czajkowski, yes, but that relationship is managed via a different mechanism.
<sinzui> czajkowski, The owner of the PPA can specify that the PPS depends on another using +edit-dependencies
<czajkowski> ahh
<czajkowski> sinzui: thank you
<sinzui> czajkowski, recipes just create a source package from a branch, after that, the PPA rules that you are building into come into play...
<sinzui> if two years are using the same recipe to build a branch, one might fail because they are building into different PPAs
<czajkowski> ok think I'll be able to answer the question now
<czajkowski> sinzui: any advice when people keep changing the status of a bug after you've already changed it once and left a comment and they keep reopening it
<sinzui> czajkowski, I threated suspension.
<sinzui> czajkowski, is this bug 945503 that we are talking about
<_mup_> Bug #945503: gcc-4.7 branch imports fails (timeouts) <Linaro GCC:Fix Released> <Launchpad itself:New> < https://launchpad.net/bugs/945503 >
<czajkowski> yes it's rally annoying the way the person keeps changing it after I went to find out the info from you andothers
<czajkowski> :/
<czajkowski> and it's a co-worker as well
<sinzui> czajkowski, doko is an canonical employee. I deleted the Lp from the bug
<czajkowski> sinzui: ahh didnt know I could do that
<czajkowski> thanks
<sinzui> czajkowski, you can delete an affects line for your projects if there will still be another effects line in the table.
<sinzui> I do not think there is a bug in the code in either projects for that bug. The bug is being misused
<czajkowski> nods
<benji> sinzui: I'm done with https://code.launchpad.net/~sinzui/launchpad/entitlement-2/+merge/97078
<sinzui> thank you benji
<benji> my pleasure
<lifeless> flacoste: I'm crook today
<deryck> yay!  I survived the small town equivalent of the DMV!
<sinzui> deryck, Did you notice that all the pictures/frames on the wall were crooked?
<sinzui> deryck, They want to keep you off balance so that you will sign away your soul.
<salgado> benji, hi there.  if you have a couple minutes, I have a trivial branch up for review at https://code.launchpad.net/~salgado/launchpad/do-not-migrate-ubuntu-work-items/+merge/97088 :)
<benji> salgado: sure
<deryck> sinzui, ha! No, I didn't notice that. Just that seemed to be clocks that hadn't been rolled forward for DST were actually clocks that never moved.
<flacoste> lifeless: i have a dr appointment now, but i'll be online later this evening to catch up with bigjools, we could talk then otherwise tomorrow is also fine
<sinzui> deryck, indeed
<benji> salgado: your branch looks good, I had one small suggestion you might like
<salgado> benji, oh, sqlvalues()!  I'd completely forgotten about that. will make the change you suggest, thanks!
<benji> salgado: cool
<salgado> benji, would you mind landing it for me as well? I've just set the commit message
<benji> salgado: I would love to, but my VM for LP dev is currently horribly broken. :\
 * benji moves fixing that up on his todo list.
<salgado> benji, hmm, ok, will see if I find somebody.  thanks for the review, though :)
<benji> my pleasure
<salgado> sinzui, would you mind ec2-landing https://code.launchpad.net/~salgado/launchpad/do-not-migrate-ubuntu-work-items/+merge/97088 for me?
<sinzui> I will do that now
<salgado> thanks sinzui!
<abentley> lifeless: Do you know anything about the amqp "immediate" flag?  I think it's what I want, but it's not behaving like I'd expect.
<sinzui> abentley, Can you advise me about this. I ec2 land does not work for me as of today: https://pastebin.canonical.com/62167/
<sinzui> maybe I can run test and land via PQM manulary
<sinzui> manually
<benji> sinzui: that looks somewhat like a failure that bit me because I wasn't running on Precise
<sinzui> benji, I have been running precise for 3 months
<benji> sinzui: then you deserve not to have this problem ;P
<abentley> sinzui: It looks like the wrong kind of config object is being passed to SMTPConnection(); it now expect a config stack.  I'll look into why that's happening.
<sinzui> abentley, I saw some bzr updates. do I need to get more maybe?
 * sinzui runs update to see
<abentley> sinzui: hmm, maybe.
<abentley> sinzui: potentially a new bzr is now available on remote instances?
<sinzui> bugger I see new bzr and bzrlib, but I cannot install them
 * sinzui forces them to install
<lifeless> flacoste: I've taken sick leave for today, lets catch up when I'm back on deck
<abentley> sinzui: Here is an untested patch that may fix you: https://pastebin.canonical.com/62168/
<abentley> sinzui: (It's for lp-dev-utils)
<lifeless> abentley: what do you want to achieve?
<sinzui> abentley, thank you very much
<lifeless> abentley: (as in why are you using immediate?)
<abentley> lifeless: I want to have two queues, and use one queue as an overflow when the other cannot consume tasks.
<abentley> lifeless: i.e. tasks go to the slow queue unless it's full, in which case they go to the fast queue.
<lifeless> the typical pattern for that in amqp is a topic exchange with different binds
<abentley> lifeless: "full" meaning all worker processes are occupied.
<lifeless> mm
<lifeless> actually, I think you're probably over specifying the behaviour
<lifeless> a key thing with message systems is that you don't - and shouldn't - know whats on the other end
<lifeless> abentley: what problem are you trying to solve?
<abentley> lifeless: I want to implement a fast lane and a slow lane, so that fast tasks complete with low latency and slow tasks complete eventually.
<lifeless> abentley: is the work queue itself stored in the DB still, or in rabbit ?
<abentley> lifeless: the job details are stored in the database, the jobs are dispatched, by id, using rabbit.
<lifeless> abentley: if it is, then I'd suggest just using either a topic exchange, or two exchanges; have everything submitted for fast initially and the timeout mechanism resubmits to the other (topic|exchange)
<lifeless> probably two exchanges makes the most sense here I suspect, as you may need topics to route to consumers on different machines
<abentley> lifeless: At this point, the plan is to use the same machine.
<lifeless> abentley: e.g. one exchange 'jobs-slow' one 'jobs-fast'
<lifeless> abentley: The big picture vision should permit scaling to N machines, even if the first iteration won't, so selecting something with a decent growth path is relevant.
<lifeless> bind all your slow works to jobs-slow, all your fast workers to jobs-fast
<abentley> lifeless: trying fast first means you have to timeout slow jobs every time.  Trying slow first means you only have to timeout slow jobs if you get a slow job while another slow job is running.
<lifeless> abentley: if you know it is slow, queue it to jobs-slow
<abentley> lifeless: I don't know it is slow and I would prefer to operate with zero knowledge.
<lifeless> I think then that this is a case of square peg, round hole. Your description now sounds more like 'I want N homogeneous workers and only M of them are allowed to run past the timeout at any one point in time'
<lifeless> this is quite different to checking whether a slow queue is in use, because you might have your workers disconnected, which in your described model does not mean 'these things are fast'
<abentley> lifeless: That is a reasonable summary.  I'd add that if a task does time out, it is only rescheduled such that it is permitted to run past the timeout next time it runs.
<lifeless> ah, good point
<lifeless> I really need to crawl back into bed and try and shake this illness
<lifeless> uhm
<abentley> lifeless: Sure.  I didn't realize you were sick when I asked.
<lifeless> what will the timeout be?
<lifeless> like
<lifeless> if its 60 seconds
<lifeless> or even 5 minutes
<abentley> lifeless: it looks like 3 minutes.
<lifeless> if we have a reasonable number of workers, and most jobs are fast, then queuing to slow first becomes an optimisation rather than a necessity
<abentley> lifeless: 3 minutes lets 99% of BranchScanJobs and MergeProposalJobs complete without timing out.
<lifeless> so
<lifeless> my suggestion is to queue to fast first, and on timeout queue to slow, and revisit queueing-slow-first if the timeout setting needed to have 99% past-first-time grows unacceptably in the future
<abentley> lifeless: Per Friday's discussion, we want to avoid using more than 2 cores on ackee.
<lifeless> abentley: why is that ?
<lifeless> (I only saw a minute or so of chatter)
<abentley> lifeless: https://pastebin.canonical.com/62113/
<lifeless> ah
<lifeless> so jjo missed that you are not adding new work
<lifeless> you are migrating work
<lifeless> the question then is 'how much work on ackee is from "jobs"' not 'how much spare capacity does it have'
<abentley> lifeless: We are migrating work and adding work.
<lifeless> (and loganberry - we have two boxes servicing our jobs)
<lifeless> abentley: what work is being added ?
<abentley> lifeless: The added work is the slow lane.
<abentley> lifeless: right now, we simply timeout at 5 minutes.
<lifeless> abentley: that is 1% of the total by frequency, and you have an intentional concurrency cap on it
<lifeless> abentley: so even if its huge, we can cap that at one core initially
<lifeless> abentley: also jjo was concisdering ackee to have 4 cores, not the 8 that we effectively get
<lifeless> abentley: so, I think you're completely safe allocating 4 cores to the new runner, migrating across and reevaluating
<lifeless> abentley: a 1% increase in the fast work won't matter from this perspective, and the slow lane you can cap at one worker initially as a safety net
<lifeless> abentley: how does that sound? Does it make requeuing less urgent in your opinion?
<abentley> lifeless: IOW, giving everything a long timeout as we were planning to do with the slow lane?
<abentley> lifeless: Actually, I don't understand.
<abentley> lifeless: Do you mean just having a slow lane?
<lifeless> I mean:
<lifeless> allocate 3 workers to the fast lane, with a 3 minute timeout
<lifeless> 1 worker to the slow lane
<lifeless> queue to the fast lane initially, and on timeout requeue to the slow lane
<abentley> lifeless: How do we implent fast lane/slow lane without requeueing?
<lifeless> once we migrate across the existing jobs, reevaluate and possibly give the slow lane another core, a nd the fast lane another 2
<abentley> lifeless: I think this would be an improvement over status quo.
<lifeless> you were saying that the optimisation of queueing to slow first was needed due to a resource constraint on ackee; I believe the resources are substantially larger, once you consider: twice the number of effective cores and that the only *new* work is (fix cores of slow lanes + 1% of fast lane jobs timing out at 3 minutes)
<lifeless> So I'm suggesting that queuing to slow first is future work we may not need
<abentley> lifeless: they seem to be similar effort, and slow first appears to give better behaviour.
* benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugtasks: 4*10
<lifeless> slow first depends on interrogating the status of the workers
<lifeless> and would still require requeing when slow is saturated
<lifeless> so AFAICT its purely additional development, and will have more corner cases.
<abentley> lifeless: if the immediate flag worked, RabbitMQ would handle determining whether the message could be consumed immediately.
<lifeless> this is an example of 'more corner cases'
<lifeless> the immediate flag should return the message to you if there is no consumer available
<lifeless> ok, I really have to crash; I really think queuing fast first is a better use of development time today - its a simpler system, doesn't have issues when dealing with upgrades and migrations, and can be transparently enhanced to do slow-first in the future if needed
<lifeless> we have way to many frills in LP where we have complexity that doesn't justify itself, this *really* sounds like a case of that to me.
<lifeless> so, I'm going to vamoose; please discuss this further with deryck / abel, or I'll be on deck in a couple days I hope.
<abentley> lifeless: sure.
<cr3> is there a different meaning between methods that start with search, like searchTasks, and those that start with find, like findPerson?
<salgado> cr3, nope, this is just because (AFAIK) there's never been a naming standard for search method names
<abentley> sinzui: did that patch work?
<sinzui> abentley, yes it did
<abentley> sinzui: great.  I'll land a properly-generalized fix tomorrow.
<bigjools> morning
<StevenK> lifeless: O HAI
#launchpad-dev 2012-03-13
<StevenK> wallyworld: Mind reviewing a mostly-mechnical MP? https://code.launchpad.net/~stevenk/launchpad/force-bug-enums-into-line/+merge/97123
<wallyworld> StevenK: sure
<wallyworld> StevenK: have you pushed a subsequent change, mp says diff still updating
<StevenK> I have not, that's it
<wallyworld> and it just disappeared
<wallyworld> so it seems to hang around for a bit
<StevenK> Reload the page, longpoll is dumb
<wallyworld> i have been
<wallyworld> and yet it stilled stayed on the page
<StevenK> Sigh. This is why we can't have nice things.
<wallyworld> no, IE is why we can't have nice things
<wallyworld> StevenK: r=me
 * StevenK lp-lands it, it's very benign
<wallyworld> WCPGW
<StevenK> Haha
<wallyworld> :-)
<StevenK> lifeless: Are you still AFK?
<wgrant> StevenK: He's sick AFAIK.
<StevenK> Well, we both knew that already.
<StevenK> :-P
<wgrant> Hah
<StevenK> But okay, noted.
<lifeless> StevenK: future ref - staff calendar will show you
<lifeless> StevenK: and yes, throat full of razor blades, cold sweats, muscle pain, sinus. ye whole works.
<lifeless> that said,
<lifeless> !ask | StevenK
<wgrant> Also, backscroll.
<StevenK> lifeless: It is not urgent, and for further win, is not work-related.
<StevenK> lifeless: So don't worry about it.
<lifeless> StevenK: well, I'm here for a few minutes, so please do ask
<StevenK> lifeless: You need to be lucid enough to remember your GPG passphrase. :-)
<lifeless> go one
<StevenK> lifeless: http://keyring.debian.org/replacing_keys.html
<StevenK> I'd like to get my 1024D key out, and my 4096R key in.
<lifeless> heh
<lifeless> I should too
<StevenK> lifeless: I can toss you the full fingerprints in privmsg
<lifeless> mail, signed by your old key; isn't that part of the protocol ?
<StevenK> Doesn't have to be
<StevenK> If it still valid, I can choose to sign it with the old key
<lifeless> is your new key in the keyservers?
<StevenK> Yeah
<lifeless> 588A553F, ?
<StevenK> That's it
<lifeless> hmm, I don't have a trust path to it
<lifeless> skype ?
<lifeless> StevenK: ^
<StevenK> Let me get that out
<StevenK> You don't have a trust path to C87FFC2F either?
<lifeless> don't seem to
<lifeless> of course, gpg may have lost local trust settings or whatever
<lifeless> whatever; will take 30 seconds to directly verify
<StevenK> lifeless: I'm Skype-enabled if you're ready.
 * StevenK prods Skype
 * StevenK waits for lifeless to ping timeout from IRC ...
<wallyworld> huwshimi: hello, do you have any exisiting/preferred code to use for a tri-state checkbox? or a preferred implementation technique? eg css or?
<bigjools> tri-state checkbox?
<wallyworld> one which can show three states
<wallyworld> yes, no, maybe
<wallyworld> or similar
<wallyworld> all, nothing, some
<StevenK> Unchecked, checked, or greyed out check. If it is played with, it collapses to checked or unchecked only.
<bigjools> a checkbox with three states... is that one where you need to visit the parallel universe to see the third state?
<wallyworld> no, it's represented typically as a grey tick
<huwshimi> wallyworld: What's the situation?
<bigjools> sorry, juju has made me flippant this afternoon
<wallyworld> huwshimi: for the sharing ui. the user needs to select to share "all", 'Some' or 'Nothing'
<wallyworld> bigjools: fuck off
<wallyworld> huwshimi: 'some' would be grey
<wallyworld> 'all' would be ticked
<huwshimi> wallyworld: Would checkboxes with those labels work?
<wallyworld> huwshimi: tri-state is more compact. the other solution is radio buttons, but yuk
<bigjools> wallyworld: sorry I can't hear you over the sound of my own awesomeness
<wallyworld> huwshimi: radio buttons would take up a bit of space
<wallyworld> and not scale that well
<StevenK> bigjools: Must be pretty quiet, then.
<huwshimi> wallyworld: The standard toolkits don't even SUPPORT tri-state checkboxes
<wallyworld> StevenK: speak up, i can't hear you over the whinging pom
<bigjools> StevenK: good comeback, I must remember that one
<StevenK> wallyworld: :-D
<wallyworld> huwshimi: yeah, there's several css/js based solutions people have used
<huwshimi> wallyworld: I'd really prefer we not use a tri-state
<wallyworld> :-(
<wallyworld> it's a very common paradigm
<StevenK> bigjools: Can't tell if trolling, or ...
<wallyworld> huwshimi: what don't you like?
<wallyworld> huwshimi: if i resort to radio buttons, the sharing picker is sure going to look fugly
<huwshimi> wallyworld: I'd prefer us to be much more explicit, especially as sharing the wrong thing has high ramifications. A tri-state will make the user have to think/guess about what each state might be.
<huwshimi> wallyworld: I'm not convinced it will be ugly. If it is, we may be doing something else worng
<huwshimi> wallyworld: And no, it's not a common paradigm in web applications (it's not even common elsewhere).
<wallyworld> huwshimi: hmmm. ok. i'll rework the ui and see how it looks
<StevenK> The states are ALL, NONE and SOME. If the user changes it, it collapses to ALL or NONE. Sounds pretty clear to me.
<wallyworld> huwshimi: we must look at different apps then
<wallyworld> it's not that common on the web
<StevenK> I remember tri-state from *shudder* my time doing Visual Basic.
<wallyworld> but what about installers as one example
<wallyworld> where you get to choose all packages, non, or some
<wgrant> I don't recall seeing many tristate checkboxes outside horrifyingly awful Microsoft Office options dialogs.
<wallyworld> wgrant: i'm sure i've used a linux installer at some point with them
<wallyworld> maybe suse
<wgrant> Red Hat doesn't exist
<StevenK> If only
<wallyworld> it's a very compact and convenient way to represent all/some/none
<wgrant> And it makes it rather challenging to recover.
<StevenK> Indeed. You can't go back to Some if you click the wrong checkbox
<StevenK> I hate the idea of comboboxes for each too, since that's just disgusting
<wallyworld> no, it's tri-state
<huwshimi> wallyworld: The most common use for a tri-state is to represent the state of other checkboxes on the page.
<wallyworld> as you click, it cycles through the states
<wgrant> wallyworld: I don't think I've ever seen one of those before.
<wgrant> AFAICR they just collapse into one of the two extremes.
<wallyworld> huwshimi: yes, that's true
<wgrant> You can never return to some.
<StevenK> That's what I recall.
<wallyworld> i'm clearly outvoted :-)
<wallyworld> i'll do it as radio buttons
<wgrant> Ewww
<wallyworld> what alternatives do i have? drop down combo?
<wallyworld> i guess i could do that
<StevenK> New widget, but ...
<wgrant> That would be even worse, I think.
<wallyworld> sooo, we don't like tri-state checkeboxes, nor radio buttons, nor drop down combo; there ain't much kleft
<StevenK> |-- ALL / -|- SOME / --| NONE
<StevenK> If you excuse my horrid ASCII-art, that's my idea, but I don't like it either.
<wgrant> I was thinking maybe a lozenge radio button.
<wgrant> Which is slightly less obese than full radio buttons.
<huwshimi> wallyworld: Can you show me the situation somehow?
<wallyworld> maybe
<wallyworld> huwshimi: https://qastaging.launchpad.net/launchpad/+sharing
<StevenK> wgrant: Can haz example? I've never heard of those
<wallyworld> huwshimi: click on the +Share with someone link
<wallyworld> huwshimi: choose a person and then see the 2nd step where you get to chosse the sharing permissions
<wallyworld> huwshimi: there are currently checkboxes but they need to be tri-state
<wgrant> StevenK: The [ Test | One | Two ] on http://arstechnica.com/apple/reviews/2011/07/mac-os-x-10-7.ars/3
<wallyworld> huwshimi: lozenge buttons seem the best suggestion so far i think
<StevenK> wgrant, wallyworld: That was what I meant with my ASCII-art
<wgrant> sinzui will cry, as they have bad history in Launchpad.
<wgrant> But for unrelated uses.
<wgrant> StevenK: Oh, I assumed your thing was more like the horrible GTK3 toggle things.
<wallyworld> StevenK: ah of course. you also printed out naked women on dot matrix tractor printers and stood 100 feet away too didn't you?
<wgrant> With the line that's either on the left or right.
<StevenK> wallyworld: No ... it was the fastest way to get my point across.
<StevenK> Sadly, wgrant did it better, like usual.
<wallyworld> StevenK: just giving you shit
<StevenK> I know. ;-)
<StevenK> wallyworld: Since you're full of it, it has to go somewhere ...
<wallyworld> lol
<wallyworld> huwshimi: you find what i mean ok?
<huwshimi> wallyworld: I did
<wallyworld> huwshimi: i await your verdict :-)
 * StevenK votes for lozenge radio buttons.
 * wallyworld so does wallyworld
<wallyworld> do we have any code in lp for those?
<StevenK> Curious how to do them in YUI
<wallyworld> surely there's a widget someone has donr
<huwshimi> Do we use the lozenge anywhere else in Launchpad?
<wgrant> We used to have lozenge menus, but they were abolished in 3.0.
<StevenK> And sinzui probably danced on their grave.
<wgrant> We all did.
<wgrant> It was a glorious day.
<huwshimi> If we don't use the elsewhere then I don't think now is the right time to be introducing new UI concepts
<huwshimi> wallyworld: What you might have to do is produce a few really quick mockups so we can see the options
<wgrant> Sure, but what else can we use that doesn't look horrible?
<StevenK> But the concept is the correct way to show a tri-state ...
<huwshimi> I'm sure we can do it with any of the options and not make it horrible
<huwshimi> but introducing unusual UI concepts probably isn't the smartest idea right now
<wallyworld> huwshimi: lozenge maybe new for lp but not outside of lp. and lp is very primative so i wouldn't use it's current widget set as a guide to the current best practice
<huwshimi> wallyworld: Right, but no-one has enough time to work on a new UI concept to the point where we get it right
<wallyworld> if not lozenge, then the only choices are radio button and drop down. i can see what they both look like
<StevenK> And they'll both be terrible, I fear
<wallyworld> me too :-(
<huwshimi> wallyworld, wgrant, StevenK: Tell me why all the rest of the options are terrible.
<huwshimi> I want to be convinced
<wallyworld> huwshimi: radio buttons are too verbose and not that well suited to being displayed for each item in a list
<wallyworld> huwshimi: drop downs don't allow the user to easily see all of the choices available without clocking
<StevenK> Yes, does a grid of 21 radio buttons make for good UI?
<huwshimi> wallyworld: How are radio buttons more verbose than the lozenge
<huwshimi> ?
<huwshimi> StevenK: There will be a maximum of nine
<huwshimi> StevenK: Usually six
<StevenK> I doubt it, for large projects.
<wallyworld> huwshimi: they are more compact, radio buttons have the clickable widget and text separate
<huwshimi> wallyworld: We have plenty of space
<StevenK> There are *better* ways to display tri-stated information than an arry of radio buttons.
<StevenK> Just thinking about it makes me want to stab my eyes.
<wallyworld> huwshimi: i think guis are better when they don't misuse space
<huwshimi> StevenK: I'm not convinced that we can do a better job of building them though
<huwshimi> I think I'd rather us do something that we can't go wrong with than introduce a new UI concept that will most likely introduce a support and maintenance cost
<StevenK> I think we have to mock up the three choices
<wgrant> I'm not sure a dropdown is a choice.
<StevenK> Neither.
<wallyworld> i guess it's hard sometimes for people who understand these ui concepts to see how it would introduce a support cost
<wallyworld> it's not like lp is aimed at people who don't know computers
<wallyworld> and if you are using the sharing ui and you can't use a lozenge or whatever then you have no business in the ui
<huwshimi> wallyworld: Sure, but given our track record of UI design, creating a lozenge widget for example has a high risk of failure. It's not just that the user won't be able to figure it out, it's that we won't do a good enough job at creating something that works perfectly.
<wallyworld> oh, the pain, stabbed in the heart
<nigelb> drama queen.
 * nigelb ducks and runs
<huwshimi> wallyworld: We have to consider these things when making decisions. Especially as I'm not sure we'd gain enough to warrant the extra development time (forgetting adding another inconsistent UI concept etc.)
<wallyworld> huwshimi: np. there's a line between "adding inconsistent ui" and "using the right tool for the job" and i guess that's what needs to be looked at
<huwshimi> wallyworld: I think if we do some mockups we can make a decision pretty quickly
<wallyworld> huwshimi: ok. thanks for the input, appreciated
<wallyworld> nigelb: i wouldn't stick my head up after what happened in the cricket :-P
<nigelb> dammit
<nigelb> :P
<wgrant> wallyworld: Oh, nice, a dict marshaller for lazr.restful. How does that affect the WADL?
<wallyworld> wgrant: not sure about what you mean by the effect on the wadl. it generates fine and looks ok but i'm no expert. all the web services tests that invoke that method pass
<wgrant> wallyworld: Right, just wondering how easily we can convince launchpadlib to interpret it well.
<wallyworld>         service.sharePillarInformation(pillar=ws_pillar, sharee=ws_grantee,
<wallyworld>             permissions={InformationType.USERDATA.title: SharingPermission.ALL.title})
<wallyworld> wgrant: you just pass in a dict as expected
<wallyworld> wgrant: the keys are values are marshalled
<wgrant> wallyworld: Ah, right, it's only going the other way that it'll need special code.
<wgrant> Methods that *return* dicts
<wallyworld> wgrant: there's an unmarshall method which is properly implemented (generates json data). will that be used for return types?
<wgrant> wallyworld: The server side has always worked, AFAIK
<wgrant> I wrote a couple of methods which returned dicts
<wallyworld> someone filed a bug related to returning dicts from memory
<wgrant> lazr.restfulclient doesn't know what to do with them, so it just returns them as dicts of strs.
<wallyworld> bug 481090
<_mup_> Bug #481090: Cannot define a method that returns a dict <lazr.restful:Triaged> < https://launchpad.net/bugs/481090 >
<wallyworld> ah, ok, lazr.restfulclient
<lifeless> StevenK: signed and pushed
<StevenK> lifeless: Thanks. I've submitted an RT (two in fact, because I'm a complete blockhead); we shall see what keyring-maint says.
<czajkowski> goood morning
<StevenK> czajkowski: Proof required. :-P
<czajkowski> I have tea and oddles of toast with peanut butter nothign going to upset me :)
<jelmer> g'morning launchpadders
* gmb changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: gmb | Firefighting: - | Critical bugtasks: 4*10
<gmb> Does anyone else see this whilst trying to use the bzr beta PPA?: http://pastebin.ubuntu.com/881655/
<bigjools> gmb: they are not publishing precise packages yet
<gmb> Ah.
<gmb> Bottom.
<bigjools> is precise not beta enough for you? :)
<gmb> I want my bleeding edge to look like it came out of an accident with a bandsaw.
<wgrant> You want ppa:bzr/daily :)
<gmb> :).
<bigjools> christ, my desktop PC has gone 3 weeks with no net connection and now dist-upgrade has taken an hour so far (50 mins of that actually installing)
<gmb> wgrant, You may know the answer to this: Is it wise or unwise to use setup.py from within the Makefile of a debian package?
<czajkowski> joys of precise :)
<StevenK> gmb: If Makefile == debian/rules, then how else are you going to do it?
<maxb> gmb: debian package ... Makefile?   you mean debian/rules?
<maxb> also re bzr ppa, at the moment bzr/beta == bzr/proposed
<maxb> er
<maxb> also re bzr ppa, at the moment bzr/beta == bzr/ppa
<gmb> maxb, StevenK: Well, yes (AFAICT that's calling the Makefile in the top level of the source, which is that I'm referring to).
<StevenK> gmb: Oh, right, your build step calls $(MAKE)
<maxb> urgh, you have a horrid package with both a setup.py and a Makefile
<gmb> maxb, It mixes python and shell stuff.
<gmb> I'm considering doing the python bit manually though
<gmb> Because setup.py just vomits.
<gmb> ... when the package is being built, anyway.
<gmb> Actually, maybe it would be simpler to have a separate package for the python stuff.
<StevenK> Doing the python bit in Make will cause you to put your keyboard cord around your neck and yank hard.
<gmb> StevenK, Ah, I see. Separate package is looking very tempting, then.
<gmb> Screw it, the original package can depend on the new one.
<StevenK> WCPGW
<gmb> Hah.
<gmb> StevenK, maxb: Thanks chaps.
<gmb> At least I know i'm not entirely mad.
<StevenK> But you don't have that far to travel.
<gmb> StevenK, Well, no, I still work on Launchpad. "Madness" is within spitting distance.
<wgrant> gmb: What's the package?
<gmb> wgrant, charm-tools.
 * StevenK reads a conversation on another channel, and reaches for his keyboard so he can wrap the cord around his neck, and then realises it's a wireless keyboard ...
<wgrant> Hm, so -server wrote it?
<gmb> wgrant, Yes, I think so.
<wgrant> Odd that it is so awkward, then.
<gmb> wgrant, Well, what's awkward is that I'm adding Python stuff to it. It worked perfectly until I started pratting about.
<wgrant> Oh
<gmb> Anyway, I'm going to bug clint about it this afternoon :)
<wgrant> That would do it.
<wgrant> Separate package :)
<gmb> Yeah :)
<wgrant> Ooh.
<wgrant> Parallel test suite bugs.
<wgrant> You have something working, then? :)
<gmb> wgrant, "Working" is relative. But yes, we do, now that we've solved a lot of the weirdnesses with lxc and -start-ephemeral.
<wgrant> Heh.
<wgrant> Good good.
<nigelb> StevenK: #firstworldproblems
<gmb> Holy mother of dog and all her wacky nephews. Why are my comments on merge proposals posted with all the spaces stripped out? What's this nonsense?
<fjlacoste> benji: have you looked at wallyworld's lsazr.restful branch yet?
<fjlacoste> i was about to write a review for it
<fjlacoste> but you are a pending reviewers
<benji> fjlacoste: I wasn't aware of it.  I would be happy to look and equally happy for you to do so.
<fjlacoste> benji: i'll take care of it
<benji> I wonder why I wasn't aware of it.  I'll try to figure that out.
<jcsackett> benji: i ifnd when i'm working off the web UI i sometimes miss things b/c i was looking at /launchpad/+activereviews instead of /launchpad-project/+activereviews?
<jcsackett> ... that shouldn't have ended in a question mark.
<benji> jcsackett: ooh, that may be it; I hope firefox auto-complete learns the better one quickly ;)
<jcsackett> benji: i hope it learns it faster than chromium's did for me. :-)
<benji> heh
<fjlacoste> benji: well, i agree that it's weird unless he explicitely asked a review from you
<fjlacoste> because i would have expected that to be assigned to the team
<benji> fjlacoste: he did ask for a review from me (at least that's what it says now), and since the email looks so much like all the non-personal review requests I see I passed over it.  I'm in the process of making messages like that highlighted so they won't slip through again.
<benji> gary_poster: I bet you're already doing this, but it dawned on me that using -vvvvvv with the test runner would show me the actual order in which the tests were run
<benji> gary_poster: you also know that I'm in the wrong channel
<gary_poster> benji yes :-)
<gary_poster> I'm using -vv, which is giving me the order
<gary_poster> I'm switching channels!
<benji> yeah, never remember how many Vs do what, so I just hold down V until I get tired
<sinzui> jcsackett, I reopened bug 933794 and updated it. +audit-sharing still needs a summary of users, policies and level of sharing
<_mup_> Bug #933794: +audit-sharing does not show a block summarising who is shared with <disclosure> <qa-ok> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/933794 >
<jcsackett> sinzui: dig.
<jcsackett> sinzui: thought that might be a new bug; i'm good with just changing the existing one though.
<czajkowski> fjlacoste: any idea why on lp all the footers with Contact Launchpad Support    end up https://answers.launchpad.net/launchpad
<sinzui> czajkowski, I know
<czajkowski> sinzui: oh do tell :)
<czajkowski> dont leave me hanging :)
<sinzui> czajkowski, We want users to ask question in a place we support rather then sending email to a place to be ignored
<czajkowski> sinzui: so where are people finding the feedback mail address
<czajkowski> and how am I getting questions via RT then
<sinzui> czajkowski, The link is not static. anonymous users see a different link because only logged in users can ask  a question..and anonymous users are probably trying to login which is not something we at Lp can help with
<sinzui> czajkowski, anonymous users send those
<czajkowski> hmm ok
<czajkowski> thanks sinzui
<sinzui> czajkowski, many rt emails are in reply to emails generated from Lp, such as commercial inquiries and translation issues
<czajkowski> the commercial folks are using feedback as well
<sinzui> czajkowski, anonymous users see a link to https://help.launchpad.net/Feedback
<czajkowski> just curious as spring cleaning rt
<czajkowski> sinzui: cheers for the ifo
<sinzui> czajkowski, I hope that changes I making these week will reduce the emails. I am replacing the generic license issues message with ones specific to the case. This will land in a few days
<sinzui> The generic email is confusing
<czajkowski> excellent
<salgado> I see we have some BugTasks which have a productseries bug no product, and I also see IBugTask.target can be an IProductSeries.  however, I cannot seem to figure out how to create (in a test) a bugtask with a productseries but no product... the factory method won't do that and if I try to transitionToTarget(series) I get an error:
<salgado> IllegalTarget('Series tasks may only be created by approving nominations.',)
<salgado> heh, now I think I understand... maybe I need to create *and* approve a nomination?
<lifeless> conjoined masters
<salgado> lifeless, you mean this is a conjoined master or I need to figure out what conjoined masters are and create one?
<lifeless> conjoined masters - any series only task gets a context task added if none exists
<lifeless> there should be no series only task bugs around
<lifeless> search for conjoined should give you some context
<salgado> lifeless, oh, ok, that makes sense.  but I don't actually want a bug with series-only tasks... all I want is a series-only task which AFAICT is created via nominations?
<lifeless> salgado: thats my point, there isn't such a thing atm
<lifeless> salgado: can you point to a bug that demonstrates what you want in prod / qas / s ?
<lifeless> bah
<lifeless> iwlwifi has a lot to answer for
<salgado> lifeless, a series-only task?  there seems to be 14177 of them on staging
<lifeless> salgado: how are you determining that ?
<salgado> select count(*) from bugtask where product is null and productseries is not null;
<lifeless> ah terminology
<lifeless> thats a series task, not series only
<lifeless> tasks are always one-of product|productseries|distro|distroseries|distropackage|distroseriespackage
<lifeless> the conjoined master means there is always a matching product task whenever there is a product series task, and likewise for the distro+distroseries and distropackage+distroseriespackage
<lifeless> salgado: you can make a series task by nomination or targeting; if this is for the test suite, makeBugTask(target=someseries) should do it
<salgado> lifeless, right, I understood those tasks with series but no product will have a matching task for the product.  but what I wanted was just to know how to create such a task with a productseries but no product
<lifeless> salgado: via the UI, API or python ?
<salgado> lifeless, I couldn't get makeBug() to do that. it seems to always set the product on the task as well.  I've now managed to do it via makeBugNomination(), though
<salgado> I think I was expecing makeBug() to create a single task bug it actually creates two in that case
<salgado> for product and productseries
<lifeless> salgado: uhm, the db won't let that happen
<lifeless> Check constraints:
<lifeless>     "bugtask_assignment_checks" CHECK (
<lifeless> CASE
<lifeless>     WHEN product IS NOT NULL THEN productseries IS NULL AND distribution IS NULL AND distroseries IS NULL AND sourcepackagename IS NULL
<lifeless> salgado: so yes, it was making two tasks
<lifeless> and you then need to grab the task you want
<salgado> right, yeah, thanks for the help, lifeless
<lifeless> no worries
<lifeless> this is fugly
<lifeless> I want to fix, but one thing at a time
<lifeless> sinzui: did the recent 404 changes drop showing the oopsid ?
<lifeless> sinzui: (It is fine if they did, just seeking confirmation it was deliberate, or whether I need to investigate something being bokred)
<lifeless> sinzui: the breadcrumbs on https://launchpad.net/~ubuntu-security/+archive/ppa/+build/3244524 include links that will 404/403
<sinzui> lifeless,  oops rules should not change
<lifeless> ah indeed
<lifeless> it is a 403 which does not oops
<lifeless> even though it matches the same bad-internal-link rule when I think about it as a human
<lifeless> sinzui: (the ppa is private, the build is unembargoed, but the breadcrumbs still link to the ppa)
<sinzui> lifeless, I had not consider that case. I think the user who can see that page gets Lp.LimitedView thus 200
<sinzui> A 404/403 page does not include breadcrumbs
<lifeless> sinzui: the build page is 200
<sinzui> well my test show that error pages do not have breadcrumbs
<lifeless> sinzui: the build page includes a link to the ppa page
<lifeless> sinzui: the ppa page is 403
<lifeless> sinzui: the link to the ppa page is in the breadcrumbs of the build page
<sinzui> ah. I see.
<sinzui> That is a whole new case. Someone should have written a custom breadcrumb adapter for build that exclude 404/403 situations
<sinzui> lifeless, maybe that build page should not be public since Lp think the parent object is private...
 * sinzui looks for bug
<lifeless> sinzui: the build is public because it is a binary-copied security update
<lifeless> sinzui: https://launchpad.net/ubuntu/+source/postgresql-8.4
<lifeless> This is another example of custom visibility rules deviating from the pattern and making other assumptions unsafe
<lifeless> one way to address it would be to expose the same raw data under each archive its been copied too, or something
<lifeless> I'd need to check code to be totally sure what its doing
<lifeless> I will file a bug about the symptoms
<sinzui> okay
<lifeless> bug 954411
<_mup_> Bug #954411: build breadcrumbs of unembargoed builds in private ppa link to the ppa which is inaccessible <403> <confusing-ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/954411 >
<wallyworld> sinzui: i have to drop my son to school a bit later today since he is going to camp so will miss the standup
<sinzui> wallyworld, thank you
<wallyworld> sinzui: i'll be packing the lazr.restful change, landing the other branches which depend on it, and prtotyping a tri-state ui for the sharing picker
<wallyworld> sinzui: we need a tri-state widget for All/Some/Nothing. tri-state checkbox was rejected on irc. so it's down to radio buttons or a lozenge
<sinzui> Yuck
<wallyworld> sinzui: discuss with  the guys and i'll ctach up with them when i get back
<lifeless> if some means 'do not change', tri-state could work, but if some means some, surely they should be shown?
<wallyworld> lifeless: some does mean some
<wallyworld> so we need something that looks good and is compact when rendered as part of a <li>
<sinzui> wallyworld, I read the scrollback this morning. wgrant is correct hat I do not like the lozenge because Lp users cannot see them. Lp 2.0 required extra support to explain to user the link is already on the page..just click the think that does not look like anything you have see on your operating system before
<wallyworld> sinzui: interesting. i like lozenge widgets do i guess it does come down to personal preference. i guess radio buttons or drop down is what we have to choose from then
<wallyworld> and drop downs were not preferred
<wallyworld> so i'll see what can be done with radio buttons
<sinzui> wallyworld, I agree with compact is more most important and the user must see the options
<sinzui> so I will accept lozenges after a ui test
<wallyworld> sinzui: cool :- ) do we have any existing code for them?
<abentley> wallyworld: We have the order-by losenges in the bug listings.
<wallyworld> abentley: thanks! will look at those
<salgado> gmb, since you're the on-call reviewer today, I thought I'd ask you to ec2-land one branch that was blocked on a bug that was just fixed (bug 953316). would you mind?  (https://code.launchpad.net/~linaro-infrastructure/launchpad/workitems-widget/+merge/94790)
<_mup_> Bug #953316: Change the workitem-migrator job to do so only for Linaro-related blueprints <qa-ok> <Launchpad itself:Fix Committed by salgado> < https://launchpad.net/bugs/953316 >
<salgado> isn't it too late for gmb, though?
<gmb> salgado: Yes, it is :) I'd forgotten to take myself out of the /topic though.
<gmb> Sorry.
* gmb changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugtasks: 4*10
<salgado> it only occurred to me after I wrote that
<gmb> next question is: Why do I have an IRC client open at 9pm?
 * gmb -> exeunt
<salgado> heh
<salgado> abentley, can you ec2-land that for me?
<abentley> salgado: okay.
<salgado> thanks abentley!
<abentley> salgado: I just bugfixed ec2 land, so this will be a good test.
<salgado> oh, I love being a guinea pig ;)
<wgrant> sinzui, lifeless: It is not possible to hide the origin of a build.
<wgrant> Copying a private build to elsewhere discloses the existence of its source archive.
<lifeless> wgrant: thats my understanding, yes
<sinzui> wgrant, StevenK, sorry I am delayed
<wgrant> We forgive you :)
<sinzui> wgrant, maybe half of this bug is fixed: Bug #181401
<_mup_> Bug #181401: push to lp:project or lp:project/series should to set the development focus and/or create the series <lp-code> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/181401 >
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/information_type-model/+merge/97319
<wallyworld> StevenK: wgrant: did you discuss the sharing picker tri-state ui at the stand up?
<StevenK> At length
<wallyworld> verdict? just to prototype something?
<StevenK> Dan likes the idea of the lozonge, Huw does not, the UI design that previous used lozenges was an utter failure. But yes, prototype it up.
<wallyworld> that's the trouble with ui work - everyone seems to have very different opinions
<wgrant> wallyworld: The issue of team members was also raised
<wallyworld> in what respect?
<wgrant> wallyworld: In that there might want to be an option for "None, and none for any of the team members either"
<wallyworld> wgrant: for sure, that was in the prototypes
<wallyworld> but we haven't got to that in the implementation yet
<wgrant> lifeless: Can I set the bugs-fixed-elsewhere hiding flag on prod?
#launchpad-dev 2012-03-14
<StevenK> wallyworld: Want to review https://code.launchpad.net/~stevenk/launchpad/information_type-bugs-garbo/+merge/97335 ?
<wallyworld> StevenK: give me a few minutes
<StevenK> And hopefully wgrant and wallyworld don't hate me, but: https://code.launchpad.net/~stevenk/launchpad/destroy-ie-regression/+merge/97337
<wallyworld> StevenK: bug._setInformationType is already committed?
<StevenK> wallyworld: It's in the pre-req branch. As is the comment in test_bug, but I hadn't pushed so it polluted the diff
<wallyworld> StevenK: r=me
<nigelb> feels very silent today.
<wallyworld> StevenK: with the other one, i think we need to commit to testing any affected pages on qas in IE before qa-ok. or even test before landing
<StevenK> wallyworld: I think I'll just reply to the thread with the MP and see what happens.
<wallyworld> ok :-)
<StevenK> wallyworld: Main reason I linked it to you was not for review, but to hope I wasn't treading on toes.
<wallyworld> StevenK: oh not at all. any of that stuff that I may have checked in would have been due to copy and paste from elsewhere, not deliberately doing it
<wgrant> Needs to be tested before landing.
<StevenK> Oh, WCPGW
<StevenK> :-P
<wgrant> Bah
<wgrant> IE8 doesn't support last-child
<wgrant> IE9 does, though.
<wgrant> Windows XP font rendering is the worst thing in the world, apart from IE6.
<lifeless> wgrant: brainfuk
<StevenK> Haha
<lifeless> wgrant: intercal
<wgrant> Bah
<StevenK> Ruby
 * StevenK ducks
<nigelb> ruby is old. node.js is the new hawtness
<wgrant> "Unknown runtime error"
<wgrant> Thanks, IE8, that's really helpful.
<StevenK> Hahaha
<StevenK> IE8 against what?
<wgrant> Trying to run Launchpad's JS.
<StevenK> Using my branch, or LP in general?
<wgrant> devel + a few unrelated changes
<wgrant> StevenK: How do I use the comboloader these days?
<wgrant> hopefully it will make the JS file small enough that IE's debugger can render it.
<StevenK> wgrant: You need to make sure your Apache is set up -- does 'combo' appear twice in your local-apache-launchpad config?
<wgrant> StevenK: I have the WSGIScriptAlias
<wgrant> And yes.
<StevenK> Does /srv/launchpad.net exist?
<StevenK> s/net/dev/, sorry
<wgrant> Ewwwwww
<wgrant> But yes
 * wgrant must take an axe to this later.
<StevenK> Is /srv/launchpad.dev/convoy a symlink into the correct tree + build/js?
<StevenK> If so, enable the feature flag js.combo_loader.enabled
<StevenK> wgrant: I can't use /var/tmp
<StevenK> You can't follow symlinks you don't own, and if $USER creates the symlink then www-data can not follow it.
<StevenK> We had this argument already.
<StevenK> And I refuse to edit the Apache config on every build
<wgrant> That doesn't make it valid for rocketfuel-setup to shit all over /srv
<wgrant> :)
<StevenK> rf-setup does far worse already
<StevenK> And the evil is in our Makefile, not rf-setup
<wgrant> Hm
<wgrant> Can I disable minimisation in combobuild?
<StevenK> Hmmm
<StevenK> I thought I had
<wgrant> Oh
<wgrant> Someone is trying to set innerHTML of a table.
<wgrant> Bad $someone.
<StevenK> lifeless: You should watch Damien Conway's talk "Temporally Quaquaversal Virtual Nanomachine Programming in Multiple Topologically Connected Quantum-Relativistic Parallel Timespaces... Made Easy"
<StevenK> wgrant: Is http://www.globalnerdy.com/2012/02/16/how-to-tell-html-from-html5-or-microsofts-image-problem/ still true?
<wgrant> wtf
<wgrant> some of our JS is awful
<StevenK> Some?
<wgrant> Well, for example, the "oh, this JS needs to know a URL. let's stick a contentless <a href="someurl" id="something" /> in the page somewhere and hope nobody notices it's there"
<StevenK> Bwahjaha
<lifeless> StevenK: what did keyring-maint say?
<StevenK> lifeless: So it seems keyring-maint is as much a blockhead as me.
<StevenK> The first RT was signed with the new key, and the second RT was signed with the current key.
<StevenK> Gunnar approved the first RT and said it's been done.
<StevenK> He then comment on the second RT with effectively, "DOH! Did I just ... yes, I did."
<lifeless> StevenK: rotfl
 * StevenK stabs ec2.
<StevenK> It failed, and didn't send me a mail
 * StevenK stabs ec2, and then twists the knife.
<wgrant> wut
<wgrant> why on earth are all collapsibles done with a <legend> not necessarily inside a <fieldset>?
<wgrant> When a <legend> can only exist in a <fieldset>..
<wgrant> Also it makes no sense.
<StevenK> wgrant: Did you get the combo-loader working, or you gave up?
<wgrant> Got it working.
<wgrant> Had to hack base-layout.pt to remove minimisation
<wgrant> But it works.
<StevenK> Ah yes, I was going to look at that
<wgrant> It's pretty simple, really.
<wgrant> http://pastebin.ubuntu.com/882822/
<StevenK> "If your membership does expire, we'll send you one more message to let
<StevenK> you know it's happened.
<StevenK> RAHHH
<StevenK> SMASH
<StevenK> wgrant: Ah, but only want raw there if devmode
<wgrant> Sure
<huwshimi> Has anyone here worked with publishing custom events with YUI?
<rick_h_> huwshimi: what do you need?
<huwshimi> rick_h_: I have a widget that I want to publish events from. I have things set up the way I think they should work, but I can't seem to see the events...
<StevenK> rick_h_: WTF, isn't it like 6am?
<rick_h_> StevenK: in CA for pycon sprints
<huwshimi> StevenK: Pycon
<rick_h_> So it's 10:30pm and I'm sprinting
<StevenK> Ah
<huwshimi> rick_h_: Want to see some code?
<rick_h_> huwshimi: sure
<StevenK> rick_h_: Can I borrow you for a sec?
<rick_h_> StevenK: maybe, beware, I'm 3/4 a bottle of wine in :)
<StevenK> rick_h_: wgrant had to add filter: 'raw' for no minified JS from the combo-loader in the pastebin he linked
<rick_h_> right
<StevenK> I'm just wondering what it should be for qas/prod
<rick_h_> I thought we picked that up in our dev settings
<rick_h_> I think qas and prod should both be min
<rick_h_> chrome dev tools can un-minify it if required
<StevenK> Which is filter 'min' ?
<rick_h_> but they should both be running without hte raw
<rick_h_> it defaults to min, just remove the raw setting
<rick_h_> bah, see...qas and prod should both be min, which is the default
<StevenK> Right
<rick_h_> so just leave off the filter all together and it should default sanely
<StevenK> Yeah, I'm trying to figure out how to codify that given our TAL.
<rick_h_> StevenK: remember, you can edit the config after it's setup
<rick_h_> so YUI_Config... all that
<rick_h_> and then "if dev mode, YUI_Config.groups.... tweak the value after the fact
<rick_h_> I guess I should pull up the file
<StevenK> base-layout-macros.py
<StevenK> pt
<wgrant> Intriguing
<StevenK> Damn muscle memory
<StevenK> wgrant: ?
<rick_h_> StevenK: ah right, the whole script block is one script tag, well just add another I guess
<wgrant> IE8 doesn't complain about broken JS on https://launchpad.net/launchpad
<wgrant> But it also doesn't execute much or any of it.
<huwshimi> rick_h_: I'm not sure how useful this mess is to you: http://paste.ubuntu.com/882825/
<wgrant> And it's not disabled, because when I run it locally without removing any anti-IE guards it breaks.
<huwshimi> rick_h_: If you search for xxx on that page it'll show you where the event stuff is
<rick_h_> StevenK: https://pastebin.canonical.com/62277/
<rick_h_> huwshimi: your JS scope on your hover (non custom) event is wrong
<rick_h_> this in there is the ev
<rick_h_> either add the this as the third argument or run a that=this in there
<huwshimi> rick_h_: Oh, perfect!
<StevenK> wgrant: Will <script type="text/javascript" tal:condition="devmode" tal:content="string: YUI.GlobalConfig.groups.lp.filter = 'raw';" /> do what I want?
<huwshimi> rick_h_: I was looking for a much bigger problem :)
<rick_h_> http://paste.ubuntu.com/882833/
<rick_h_> or something like that
<rick_h_> huwshimi: let me know if that helps. I haven't done a ton of custom events and know they can get complicated with event facades and such, but hopefullythat's all it is
<huwshimi> rick_h_: Yup, working. Thanks for your help :)
<rick_h_> huwshimi: np
<wgrant> StevenK: No point using tal:content there
<wgrant> StevenK: Unless TAL's special <script> mangler kills you.
<wgrant> But assume it won't unless proven otherwise.
<StevenK> wgrant: So <script type="text/javascript" tal:condition="devmode">YUI.GlobalConfig.groups.lp.filter = 'raw';</script> with some vertical whitespace sprinkled in would be fine?
<wgrant> StevenK: Probably.
<rick_h_> ok, I'm out, have fun all
<wgrant> Night rick_h_.
<wgrant> lifeless: bugs.statistics_portlet.hide_fixed_elsewhere_count default 0 true?
<wallyworld> wgrant: StevenK: has ec2 land failed for you today or recently with a bzrlib error?
<wgrant> Not for me, but apparently everyone else
<wallyworld> :-(
<wallyworld> workaround?
<bigjools> pqm-submit? :)
<wallyworld> bigjools: i want it to run tests first though
<bigjools> who needs those
<wallyworld> me
<StevenK> wallyworld: I had it fail, you need to update lp-dev-utils
<wallyworld> ah thanks
<StevenK> And my garbo job branch has checkwatches.txt fail? Le sigh
<lifeless> wgrant: yes, but also put it on the blog / lp-users or something ?
<wgrant> lifeless: The premise of the change is that nobody will notice if it disappears.
<lifeless> indeed
<lifeless> so there is a balance to strike :)
<wgrant> That balance is to hide it and watch nobody complain.
 * StevenK is tempted to make a twitter joke
 * StevenK has fun breaking wallyworld's mockup
<wallyworld> StevenK: it's only POC :-)
<wallyworld> what broke?
<StevenK> wallyworld: Click logout, it's funny
<wallyworld> StevenK: ah, that was never meant to work
<StevenK> That much is obvious. :-)
<wallyworld> the point of the mockup for this iteration is the disclosure picker chamges
<wallyworld> anything else that works is not guaranteed
<StevenK> wallyworld: However, hitting the expander for large commercial doesn't show anything and the button that displays isn't wired up
<StevenK> Er
<StevenK> *everything, not anything
<wallyworld> which expander?
<StevenK> The filter one
<wallyworld> StevenK: the filter is not implemented
<wallyworld> it was last revised weeks/months ago
<wallyworld> the thing to focus on for this iteration is the picker :-)
 * wallyworld has to reboot his router. stupid voip port locked up :-(
<huwshimi> wallyworld: You know, flat mockups for the radio buttons, lozenge etc. will do just fine for deciding which direction to go in.
<wallyworld> huwshimi: yeah. but i when we do decide, all the plumbing has to change anyway so i can just reuse what i did there. not much will be wasted
<huwshimi> wallyworld: So you're going to be creating the lozenge etc in javascript as well then?
<wallyworld> huwshimi: not sure. i might hust do a static mockup. but i do think people will want to click and see how they like it etc
<wallyworld> huwshimi: and i thought i'd see the reaction to the radio buttons first
<huwshimi> wallyworld: We need to do flat mockups first. If we need to see interactive ones to be able to decide then we can do those once we've narrowed down the options.
<huwshimi> wallyworld: Doing them at this stage is wasted effort
<wallyworld> i didn't waste much. 80% of the work was in the plumbing which can be resued regardless of what choice we make
<wgrant> Mmmmmm, delicious delicious tag soup.
 * wallyworld is off to soccer
<wgrant> wow
<adeuring> good morning
<czajkowski> aloha
<danhg> morning all
<StevenK> mabac: Hai, your revision of r14942 is up on qastaging and ready for QA, if you can do it soon.
<salgado> StevenK, we're doing it now
<StevenK> salgado: Excellent, thank you.
<mabac> StevenK, yup, shouldn't take much longer
<StevenK> I wasn't going to care about deploying until tomorrow morning my time, so you have about ten hours. :-)
<mabac> StevenK, thank you :)
<czajkowski> mrevell: salgado with bugs that are coming into lp for triaging that are blueprint issues what way do you want me to handle them ?
<salgado> czajkowski, don't know, tbh.  what are the options?
<czajkowski> well two are yours and others are linaro issues. so just wondering will they be considered for development
<czajkowski> or do i traige the all low and add you as the assingee or critical :)
<czajkowski> or opinion :)
<salgado> czajkowski, we just filed 3 or 4 that would be very nice to have fixed, so we'll definitely take care of those.  there are at least another 2 which are wishlists I'd say.  I'll collect them all in an email and state our plans
<jcsackett> and the day begins with a development machine that won't boot from disk errors...
<jcsackett> morning, all. :-)
 * mpt wonders why sinzui changed "private" to "proprietary" in bug 136937
<_mup_> Bug #136937: Cannot file a non-security proprietary bug in Launchpad <bug> <disclosure> <lp-bugs> <privacy> <Launchpad itself:Triaged> < https://launchpad.net/bugs/136937 >
<sinzui> mpt, private could mean embargoed security information, Personal user data information, or proprietary information
<sinzui> mpt proprietary means the information belongs to an organisation any probably will not ever become public
<mpt> sinzui, sometimes. A big counterexample was Launchpad itself when it was proprietary. :-) Then our bug reports were private only when they referred to detailed architecture or source code.
<mpt> but I understand what you mean now, proprietary is the main use case for non-security privacy
<sinzui> mpt yes, and we decided to make them all public when the code became public
<sinzui> There are still proprietary bugs though...we wanted to make all public, but company information is in some comments
<sinzui> mpt, right. Most private bugs in Lp are user-data because apport includes personal user information that requires cleaning before the bug is made visible to Ubuntu's bug squad
* abentley changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: abentley | Firefighting: - | Critical bugtasks: 4*10
<sinzui> abentley, I am pondering job system for projects that send emails or maintainers and possibly updates the project. I need a daily script that create job if one has not already been created within a time frame (week or months). Project reviewers could also use the UI to create jobs (send an email about licensing problems). Does this sound doable?
<sinzui> s/emails or maintainers/emails to maintainers/
<abentley> sinzui: Yes, it sounds very doable.  It's a little like how we create recipe builds.
<sinzui> oh, I will look at that.
<sinzui> abentley, we only autobuild once per day right?
<abentley> sinzui: We used to only autobuild once per day, I'm not sure if that was changed.
<sinzui> My plans might require that jobs in the jobs table not be deleted or actually remain in the job table for up to 6 months. I do not think we delete job.
 * sinzui has to find more plurals and tenses
<abentley> sinzui: That's correct.
<sinzui> fab
<abentley> sinzui: I and adeuring are currently working on https://dev.launchpad.net/CeleryJobRunner so if you need to change code in lp.services.job, please let me/adeuring know.
<sinzui> okay. I doubt I will. I was thinking of a new table+model for project jobs that references the exising job object. (dates and statues)
<abentley> sinzui: Right. I didn't think so either.
<sinzui> still one s short
<sinzui> thanks abentley. I will start sketching this in code
<abentley> sinzui: Once we've migrated to celery, we'll have the option of using celerybeat, rather than cron, for scheduling periodic tasks.
<sinzui> abentley, Oh. I do want that
<abentley> sinzui: The way we do recipe builds is we periodically look for recipes to build, and then schedule those recipes to be built.
<sinzui> Looks like my session is knackered by an update. I am restarting.
<abentley> sinzui: ACK.
<sinzui> oh dear, Starting a window kill the indicators/menus in Unity. starting a terminal is insane as the UI refreshes and the window shrinks by one line each loop until the terminal is just a window frame
<wgrant> Hmm
<wgrant> I've seen a similar sort of thing with qemu for a few weeks.
<wgrant> When run under precise's compiz it has seizures
<wgrant> resizing up and down for 20s or so
<wgrant> Until it contracts to ~1x20px
<sinzui> I think I will not use the indicators for the next day.
<abentley> sinzui: Anyhow, pre-celery, you'd have a cron script that looks at projects and creates jobs, and another that actually runs the jobs.
<sinzui> abentley, indeed that is what I was thinking
<deryck> abentley, hey, sorry running a tad late.  give me 5 minutes to grab a coffee and we can chat.
<abentley> sinzui: post-celery, celerybeat could run a Task that creates Jobs, which would then run in celeryd.
<abentley> deryck: sure.
<sinzui> abentley, how many weeks are we from celerybeat?
<abentley> sinzui: I think at least two.
<abentley> sinzui: This almost sounds like a straight cron script would do the trick, though.  If you had a field on the project to track the last execution.
<sinzui> abentley, I do not want to change project because there can be many reasons a job is in the queue, and there could be several, but I want to avoid duplicate jobs (a job type that I need to create) or jobs that spam users too frequently
<sinzui> I want my process to look at the most recent jobs for a time frame and if the type is not present, create one
<abentley> sinzui: I guess you could have one script that created the jobs and then ran them immediately.
<sinzui> That might be th effective out come. since the task is short
<sinzui> I am separating how the job is created from the job since I already know of two very different ways that the job would need to be created
<deryck> abentley, https://plus.google.com/hangouts/extras/talk.google.com/orange-catchup (when you're ready)
<cr3> what's the difference between login.launchpad.net/$id/+decide and /two_factor_auth?
<stub> cr3: Dunno - login.launchpad.net is the SSO server ISD runs, despite it being on our domain.
<deryck> adeuring, hey, just finished my chat with abentley and we updated the board to better reflect the multiple branches you have going.
<adeuring> deryck: ok, thanks
<deryck> adeuring, also, were you getting something up for review today or tomorrow?  Or you're not sure yet?
<adeuring> deryck: things are nearly ready
<deryck> adeuring, awesome!
<deryck> thanks!
<adeuring> abentley:  https://code.launchpad.net/~adeuring/launchpad/lp-lazr.jobrunner/+merge/97458 (plus  lp:~adeuring/launchpad/lazr.jobrunner-more-tests -- no MP for that branch)
<abentley> adeuring: Cool.   Looking.
<abentley> adeuring: I don't think we can land this with "develop =../lazr.jobrunner".  We need lazr.jobrunner to be an egg in download-cache.
<abentley> adeuring: It looks like lp.services.job.model.job.Job is not derived from lazr.jobrunner.jobrunner.BaseJob.  Should it be?
<adeuring> abentley: It can't hurt, I think, but it's not strictly necessary either. IOW: I don't have any opinion on it ;)
<abentley> adeuring: I think it will be clearer if we do it, so we should, but we can do it in a follow-up.
<adeuring> abentley: right; there are anyway a few things to clean up, like the change to buildout.cfg
<adeuring> (together iwh bureaucracy, like adding lazr.jobrunner to the download cache)
<abentley> adeuring: The purpose of lp.services.job.jobrunner.BaseJobRunner.runJob is to adapt the input to IJob?
<adeuring> abentley: I think so. That was your change, IIRC ;)
<abentley> adeuring: Yes, I believe so.  It almost looked redundant.
<adeuring> abentley: Well, it is a way to "mix" any run() method (i.e., the core of a job) with the "Bureaucratic" stuff, like start(), fail() etc
<abentley> adeuring: It looked redundant with lazr.jobrunner.jobrunner.BaseJobRunner.runJob
<abentley> adeuring: Do you know whether the removeSecurityProxy in job_str is still necessary?
<adeuring> abentley: I did not check...
<abentley> adeuring: I think I did that before I made job_id a part of IJob.
<adeuring> abentley: ok, let's try it. Also, you are right: we can drop BaseJobRunner.runJob(); at least, the tests pass
<abentley> adeuring: How does makeOopsReport access the oops messages?
<adeuring> abentley: makeOopsReport() does net "see" these messages. Adding them is done via a callback in oops_config.
<adeuring> abentley: but makeOopsReport could be used instead of the context manager to add this extra data. (with a small drawbacks.)
<abentley> adeuring: Okay, I see where that's done now.  That's fine.
<abentley> adeuring: By deriving BaseRunnableJob from BaseJob, you'll eliminate a little redundancy: user_error_types, retry_error_types
<adeuring> abentley: right
<abentley> adeuring: You should also specify a version of lazr.jobrunner in versions.cfg
<adeuring> abentley: yes
<deryck> abentley, hey they.  I have one for review when you're available.
<abentley> deryck: ACK.  With you in a few minutes.
<deryck> abentley, no problem.  Thanks
<adeuring> abentley: if we derive BaseRunnableJob from BaseJob, the delegaton mechanism to lp.services.job.model.job.Job.start() etc fails; instead, the "dumb" methods from BaseJob are used. And these are not aware of the DBEnum values of the status property.
<abentley> adeuring: Okay, let's leave it alone for now.
<abentley> adeuring: Okay, so that's everything I see there.  I'm going to mark it "needs fixing" because it needs to specify a proper egg for lazr.jobrunner.
<adeuring> abentley: right
<abentley> adeuring: But don't get me wrong, this is great progress.
<adeuring> thanks
<abentley> deryck: I can look at your review now.
<deryck> abentley, thanks!  https://code.launchpad.net/~deryck/launchpad/buglistings-preload-people-901122/+merge/97041
<abentley> deryck: In getBugTaskPeople, can you just use Bug.owner directly, instead of getting Person.id?  It should be the same value, and it reduces the number of tables involved.
<deryck> abentley, but I need to return the people themselves.
<abentley> deryck: right.  I mean use bug.owner and bug.assignee to generate your person_ids.
<deryck> abentley, ah, right.  Well I could use bugtask.assignee, but I can't use bug.owner without issuing another query to get Bugs. That's why I did the big join.
<abentley> deryck: e.g. http://pastebin.ubuntu.com/883667/
 * deryck looks closely
<deryck> abentley, ah right! I see what you mean now.  yes, I can do that.  thanks
<abentley> deryck: np
<lifeless> deryck: bug.ownerID ?
<deryck> abentley, it's a hold over from the first iteration of this, where I just had a union and selected people, not subselects to get ids.
<deryck> lifeless, that's what I thought abentley was suggesting.  he's talking about simplifying the way I wrote a subselect.
<abentley> deryck: I guess the advantage of your version is that it handles NULL assignee/owner automatically.
<lifeless> deryck: I think you will get much better results using the Person eager load helper.
<deryck> abentley, yeah
<lifeless> deryck: which knows how to get related assets; you know that bugtask is loaded, and bugtask.bug is already loaded.
<deryck> lifeless, where is the person helper?
<lifeless> deryck: so you have no need to do any DB work in this helper at all: just assemble the ids you need and hand off to the person helper.
<deryck> lifeless, I thought doing this would issue more queries:  [task.bug.ownerID for task in bugtasks] ?  Local testing seemed to say so.
<lifeless> deryck: getPrecachedPersonsFromIDs
<lifeless> deryck: if bugtasks is a storm collection, yes it will
<deryck> lifeless, right.  so that's why I made my own query.
<lifeless> thats why you need to listify anything you get from storm before you start wrking with it
<lifeless> slice it first, then materialise and cache
<deryck> ah
<deryck> so I could listify the batch, get the owner ids and hand off to the person helper?
<lifeless> deryck: right, though if it is a batch, not storm, it has a caching layer in it already
<lifeless> deryck: I suggest, grab the ids, hand off, and if you get extra queries, show me the backtrace leading up to the first one and I will help you sort it out
<deryck> lifeless, ok, that's fair.  thanks
<deryck> abentley, so never mind then. :) see ^^.  I'll rework it.
<abentley> deryck: Okay.  Happy hacking!
<deryck> abentley, thanks :)
<deryck> shouldn't be too big a change really.
 * deryck steps away briefly to get food
<benji> abentley: hi, I have a very small MP that I'd like to get on your review docket: https://code.launchpad.net/~benji/launchpad/bug-954319/+merge/97491
<abentley> benji: sure thing.
<benji> cool
<abentley> benji: r=me.
<benji> abentley: thanks
<benji> I am in a state of kanban conflict: the board is now not over the limit (with me moving my branch into landing) but we can't add anything else without going back over
<benji> oops, wrong chan
<benji> deryck: were you ever able to get LP running on a new precise box?  I'm trying to set one up and am apparently having the same problems you wrote to the list about.
<deryck> benji, I did.  I had to install postgresql from oneiric packages. I downloaded them from launchpad individually by hand....
<deryck> benji, and then I'd run make run and install packages individually via apt-get install until it quit erroring.
<lifeless> wgrant: btw, there is demand for a public API for openid->teamsetc
<deryck> benji, I think I've got everything installed now except launchpad-messageque-dependencies.
<lifeless> wgrant: wondering if you want to make it a full fledged thing at the same time
<benji> deryck: I'm glad you got it working.  I guess I'll go the same route.
<deryck> benji, yeah, it was a slow and steady pain, but fixable given time and will power.
<benji> This seems like it should be a priority for us to fix.  I would attempt to do so, but given my knowlege of packaging, it would be a bigger waste of time than I'm already facing.
<wgrant> lifeless: Need to try to push XMLRPC duration variance down a bit first
<wgrant> It's possibly not possible with the current setup, though.
<wgrant> Since we're going to be GILled sometimes no matter what I do.
<wgrant> And really these should complete in tens of milliseconds.
<lifeless> wgrant: I was thinking xmlrpc for sso, API for u1 etc
<lifeless> only sso with its special casing needs to bypass visibility rules
<wgrant> lifeless: Well
<wgrant> lifeless: The others would need to run under service accounts.
<wgrant> Which we don't really do.
<wgrant> And we would need to have extended private team visibility rules.
<wgrant> Or have the service accounts as members of the teams...
<wgrant> ew
<lifeless> the latter is pragmatic
<sinzui> wgrant,  I see I misunderstood Y.Array(my_array). I though it returned a YUI Array, not a native array. I do not see anything I have written in your diff, but I think code I have written for disclosure and teams must be wrong
<wgrant> Test results: bugfilters-ie8 => devel: SUCCESS
<wgrant> Success
<wgrant> sinzui: Oh yes, I initially tried to use it as a constructor too
<wgrant> sinzui: One of the more braindead YUI decisions I've seen...
<wgrant> I think I fixed everything except for some of the derived distros widgets.
<wgrant> I'll recheck.
<wgrant> sinzui: Thanks
<wgrant> wallyworld, sinzui, StevenK, jcsackett: ec2 is fixed if you want to pull lp:lp-dev-utils
<wallyworld> cool thanks :-)
<nigelb> Mornin
<wgrant> Hi nigelb
#launchpad-dev 2012-03-15
<nigelb> hehehe, StevenK++
<nigelb> If you get jono to write a patch for LP, I will buy ou a drink :D
<StevenK> I was tempted to reply, "You can bribe yourself, if you wish ... not sure if it will have the desired effect ..."
<nigelb> heh
<StevenK> nigelb: Like you're old enough to buy alcohol. :-P
<nigelb> StevenK: Hey, I'm not wgrant :P
<nigelb> Wait, what's the age limit in Australia again?
<StevenK> 18
<nigelb> HA. *Well* past.
<StevenK> I thought you were younger than wgrant, TBH.
<lifeless> wait, what?
<StevenK> I'm allowed to be wrong. It happens often enough.
<nigelb> I'm 23 :)
<bigjools> us married people do feel a lot older than we are
<nigelb> heh
<nigelb> bigjools: settled down in australia? :)
<bigjools> yes, thank you
<wallyworld> bigjools: you have it wrong. you're only as you as who you feel
<wallyworld> as young as
<wgrant> win 3
<StevenK> Fail
<bigjools> wallyworld: in that case.... :)
<lifeless> mmm
<lifeless> did something in bug search change recently?   2457 /  120  MaloneApplication:+bugs
<lifeless>    210 /    1  Distribution:+search
<lifeless>      204 /   28  Distribution:EntryResource:searchTasks
<lifeless>      181 /    6  Person:+commentedbugs
<lifeless>      154 /    0  Person:EntryResource:searchTasks
<lifeless> -> is suspicious
<wgrant> lifeless: Huh?
<wgrant> It's been like that for months.
<wgrant> MaloneApplication:+bugs has been bouncing around between 700 and 3000 for months.
<lifeless> hmm
<lifeless> I could swear for a whole week last week it was lower
<lifeless> lots lower
<lifeless> haha! found the culprit. finally.
<lifeless> bug 955657
<_mup_> Bug #955657: checkwatches reports oopses with no page-id <oops-infrastructure> <Launchpad itself:Triaged> < https://launchpad.net/bugs/955657 >
<wallyworld> StevenK: can haz small review? https://code.launchpad.net/~wallyworld/lazr.restful/dict-marshalling-default-key-value-types/+merge/97564
<StevenK> wallyworld: Sorry, looking now.
<wallyworld> np
<wallyworld> i hate doc test failures. especially ones involving webservices
<StevenK> wallyworld: One small comment in return.
<wallyworld> sure
<wallyworld> StevenK: fair point. i'll simplify before landing getMultiAdapter(field.key_type or Field(), request), IFieldMarshaller)
 * StevenK ponders a deployment
<thumper> heh
 * thumper misread that as "StevenK ponders employment"
<StevenK> thumper: Don't you have bugs to write?
<thumper> no, I'm a manager now remember
<thumper> I have minions to write bugs for me
<nigelb> StevenK: "Don't you have bugs to write" *snicker*
<wgrant> wallyworld: How do I run just a single YUI test?
<wallyworld> wgrant: you can't :-(
<wallyworld> it sorta sucks
<wgrant> How do I debug a YUI test? :)
<wgrant> Maybe break on error will be useful
<wallyworld> i debug in firebug
<wallyworld> or my ide
<rick_h> wgrant: debugger;
<rick_h> put that into the test where you want to debug
<rick_h> then use the dev tools or firebug to debug
<wallyworld> aarrrrggghhh - why do we want to add such things to source code
<wgrant> I as hoping to convince the testrunner to give me a traceback rather than just the text of the error.
<wallyworld> there's no need
<rick_h> wgrant: yea, not really atm. I hate it as well
<wgrant> Ah, break on error worked that time.
 * StevenK stabs the team renewal mail
<wgrant> You could just renew your membership :)
<StevenK> s/If your membership does expire, we'll send you one more message to let
<StevenK> you know it's happened.
<StevenK>  /We will keep spamming you until you get annoyed enough to follow the link in this mail./
<nigelb> StevenK: Didn't I complain about this a while back? And you pullec my leg? ;)
<nigelb> *pulled
<StevenK> I get annoyed by the mail everytime a membership is up for renew, and then I click renew and forget about it.
<lifeless> 13:14<
<wgrant> I disagree
<lifeless> ECAT
<wgrant> I guess soon enough we'll have ETODDLER
<wgrant> StevenK: Are you QAing your information_type stuff?
<StevenK> wgrant: Waiting for garbo
<wgrant> Is it running, or are you waiting for it to run?
<wgrant> The latter is not a valid situation.
<StevenK> Why not, it starts in 2 minutes?
<wgrant> Ah
<lifeless> wgrant: I was thinking bug 132300
<_mup_> Bug #132300: mail for new bugs inconsistent with other bug mail (ignores mail-on-my-actions disabled, missing footers, duplicate mail vs structural subscriptions) <email> <lp-bugs> <notifications> <Launchpad itself:Triaged> < https://launchpad.net/bugs/132300 >
<wgrant> lifeless: Bug #66760-4
<_mup_> Bug #66760: palette indexes in eft-theme.c should not be prefixed with 0x <usplash (Ubuntu):Invalid> < https://launchpad.net/bugs/66760 >
<wgrant> lifeless: Bug #66760-4
<wgrant> lifeless: Bug #667604
<_mup_> Bug #66760: palette indexes in eft-theme.c should not be prefixed with 0x <usplash (Ubuntu):Invalid> < https://launchpad.net/bugs/66760 >
<_mup_> Bug #667604: new bug reports with attachments added generate blank email instead of including links to attachments <lp-bugs> <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/667604 >
<wgrant> Ther we go
<lifeless> ah
<lifeless> wgrant: bug 457489
<_mup_> Bug #457489: Bug importer ignores the product.private_bugs flag <bugs> <easy> <lp-bugs> <privacy> <Launchpad itself:Triaged> < https://launchpad.net/bugs/457489 >
<wgrant> _mup_: bug 457489 tags +disclosure
<_mup_> Bug #457489: Bug importer ignores the product.private_bugs flag <bugs> <easy> <lp-bugs> <privacy> <Launchpad itself:Triaged> < https://launchpad.net/bugs/457489 >
<wgrant> If only that worked.
<StevenK> Haha
<StevenK> wgrant: Fixored
<wgrant> Actually, since it wouldn't be a regression, not disclosure :)
<wgrant> Heh
<wgrant> Will see what the Curtis thinks.
<StevenK> I've tagged it, it can stay that way until tomorrow morning.
<czajkowski> I do love the tag lifeless added easy
<wgrant> It is easy.
<lifeless> \o/ heat 498
<lifeless> bug 5977
<_mup_> Bug #5977: Person's related bugs pages do not show closed bug reports <affectsmetoo> <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/5977 >
<poolie> is it a known bug that branch link suggestions seem to be silently timing out?
<lifeless> it may be
<wgrant> Branch search is terribly slow
<wgrant> Because lol what is context
<wgrant> So it's doing string matching on several fields across the entire table.
<wgrant> It's pretty cool, innovative stuff.
<wgrant> Unindexed too.
<lifeless> it may be bug
<lifeless> 742916
<lifeless> bug 742916
<_mup_> Bug #742916: BranchMergeProposal:+index timeouts - slow query plan <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/742916 >
<StevenK> wgrant: For 1979?
<wgrant> Very similar.
<poolie> ok
<poolie> there's something else really strange
<poolie> bug 865977, when cinerama looks at it, has a linked branch
<_mup_> Bug #865977: UserPreferences does not work <Launchpad Development Wiki Moin theme:Triaged by stephaneeee> < https://launchpad.net/bugs/865977 >
<poolie> when i do, it does not
<lifeless> you can't see the branch
<poolie> ah
<wgrant> Yay for branch visibility policies
<poolie> oh course
<wgrant> Luckily they are scheduled for demolition.
<poolie> wbn if the link to the branch had a privacy icon
<lifeless> if it doesn't, it should
<lifeless> please file a bug if appropriate
<lifeless> stub: yo
<stub> yo
<lifeless> weekly thingy time?
<stub> sure
 * stub finishes some typing
<stub> lifeless: oh - my skype capable box is currently upgrading to Precise
<lifeless> stub: nifty
<lifeless> stub: be sure to install pavucontrol so that you can get your mic working again
<stub> ahh... should be right. Still have 5 hours of downloading packages
<poolie> ok, https://bugs.launchpad.net/launchpad/+bug/955745
<_mup_> Bug #955745: link to private branch doesn't indicate it's private <bug-branch-links> <confusing-ui> <privacy> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/955745 >
<adeuring> good morning
<huwshimi> rvba: Do you have some time to talk about testing the YUI custom events and the js packing stuff at some stage?
<rvba> huwshimi: sure.
<jelmer> 'morning launchpadders
<StevenK> mabac: So you sent an e-mail, but you didn't mark your bugs as qa-ok? I'm confused.
<mabac> StevenK, we where waiting for your opinion (or someone in your team I guess). From our point of view none of them are blockers and I've fixed the most annoying bug.
<StevenK> I thought we had guidelines for this sort of thing.
<StevenK> mabac: It doesn't have to be perfect (but that would be nice), it just has to not blow up or cause regressions.
<StevenK> If it works, even better
<StevenK> If it looks ugly, deploy it, and you can iterate.
<mabac> StevenK, I'll read up on the docs... then I'd like to get this in https://code.launchpad.net/~linaro-infrastructure/launchpad/fix-work-item-brackets/+merge/97584 and we're pretty close to beautiful ;)
<StevenK> mabac: It's 9:30pm, if that needs a review, you're out of luck with me, if it's approved I'll toss it at ec2.
<mabac> StevenK, ok thanks. I'll get someone to review it.
<StevenK> mabac: In the meantime, I'd suggest marking your two bugs as qa-ok so we can deploy.
<StevenK> Blocking the deployment pipeline == bad.
<mabac> StevenK, oh I didn't realise we where blocking you. I'll have to learn more about what happens behind the scenes. will fix
<StevenK> mabac: http://lpqateam.canonical.com/qa-reports/deployment-stable.html
<mabac> StevenK,  thank you
<mabac> StevenK, I've changed the tags on our bugs
<mabac> so, anyone feel like picking up a one-line review? :) https://code.launchpad.net/~linaro-infrastructure/launchpad/fix-work-item-brackets/+merge/97584
<StevenK> mabac: https://dev.launchpad.net/QA/QAProcessContinuousRollouts
<mabac> StevenK, thank you for the links.
<stub> mabac: r=stub
<wgrant> https://lpstats.canonical.com/graphs/PPR/ is slightly concerning.
<wgrant> xmlrpc-private is slowly getting faster.
<wgrant> Without us doing anything...
<wgrant> But it's like 60% faster than it was after my last optimisation a few weeks ago.
<mabac> stub, thank you!
<stub> wgrant: What systems use that API again?
<wgrant> stub: The dominant view has been MailingListAPIView for quite some time.
<stub> Might have sped things up by accident, eg. indexes changed to fix a different problem
<wgrant> It seems to now be 4x faster than it was.
<wgrant> Right, but I'm pretty sure we haven't changed these indices...
<stub> Mailman archives where having trouble. Side effect of that with a changed usage pattern or something?
<wgrant> Ah, now that is interesting.
<wgrant> There's 4x as many mailman requests as there were a month ago.
<wgrant> But it's not just additional requests.
<wgrant> Either we've lost 80% of the old ones, or they're all <0.6s now..
<wgrant> So the extra volume is probably just because everything's fast now.
<mabac> StevenK, so when is the next window for deploying, now that I'm not blocking the queue anymore? :)
<wgrant> mabac: I've just requested a deployment.
<wgrant> Will hopefully be out in the next couple of hours.
<mabac> wgrant, oh very nice. thanks!
<deryck> Morning, everyone.
<abentley> deryck: morning.
<wgrant> sinzui: amazing
<wgrant> sinzui: You approved that 30 seconds after it passed ec2test
<sinzui> fate
<sinzui> but I wrote cide instead of code. Maybe my fingers were combining code and suicide
<wgrant> sinzui: Ah, so that's why lp-land is complaining. Can you fix, or shall I pqm-submit?
<sinzui> I will fix it
<wgrant> Thanks.
<sinzui> try it again
<wgrant> sinzui: Much better, thanks.
<czajkowski> sinzui: morning
<czajkowski> wgrant: how are you still awake!
<wgrant> It's only 00:25
<czajkowski> loonatic :)
<wgrant> Many would agree :)
<czajkowski> sinzui: I can across a project today on licience review where it is named Bazar translating it it's bazaar. yay or nay ?
<czajkowski> wgrant: indeed :) am feeling a little tired from being up so early and a swim
<sinzui> czajkowski, you are judging them. They are creating a b2b site
<czajkowski> sinzui: not juding just asking for clarification.
<sinzui> Since the branch is empty I expect that the project will fail to thrive, but I think we should let them prove me wrong
<czajkowski> oki
<czajkowski> thank you
<czajkowski> thats all I was asking wondering if there was any issue wth a name similar to bazaar
<sinzui> They use bazar to mean marketplace, which is a good name for a b2b site
* jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jcsackett | Firefighting: - | Critical bugtasks: 4*10
<sinzui> danhg, do you have time to review my change that splits the generic licensing email into three emails that are specific to the licensing cases: https://code.launchpad.net/~sinzui/launchpad/entitlement-3/+merge/97300
<danhg> Hey sinzui, I can look at this Monday - I have a full day planned in Millbank tomorrow, and I'm finishing off tests/writing up of tests today before I leave for the airport in 2 hours. Is Monday OK? If not, I may be able to look at it tomorrow when I'm in Millbank.
<sinzui> danhg, we do not permit reviews to block landings. I will land the branch later, I will send you the text three emails and will land that separately.
<danhg> OK, sure
<danhg> I'll talk to Matthew about re-arranging stuff.
<sinzui> danhg, are we switching to British English. The original email was in American English
<sinzui> Lp was officially American English from 2008 to 2010
<danhg> sinzui, can you send me just the text that you'd like me to look at and I'll do a quick read-through/edit now?
<sinzui> okay
<danhg> thank you
<czajkowski> salgado: you about ?
<salgado> czajkowski, yep
<czajkowski> salgado: ello, thebugs you've submitted and marked confirmed and tagged but no status has been marked against them, are they for you to keep an eye on for future work, if so cna I mark them low
<czajkowski> I'd rather have the bug queue I watch  up to date so I know what bugs I am to look after
<salgado> czajkowski, I didn't change their priority because I'm not allowed to
<czajkowski> ok
<salgado> https://bugs.launchpad.net/launchpad/+bug/954970
<salgado> https://bugs.launchpad.net/launchpad/+bug/954975
<salgado> https://bugs.launchpad.net/launchpad/+bug/954980
<_mup_> Bug #954970: Should list valid milestones when user enters an invalid one in a work item <work-item-tracker> <Launchpad itself:Confirmed> < https://launchpad.net/bugs/954970 >
<_mup_> Bug #954980: Incorrect error message when a work item without status is entered <work-item-tracker> <Launchpad itself:Confirmed> < https://launchpad.net/bugs/954980 >
<_mup_> Bug #954975: When there are errors on your work items it's not clear on which line the error is <work-item-tracker> <Launchpad itself:Confirmed> < https://launchpad.net/bugs/954975 >
<salgado> czajkowski, those 3 can be high as we'll get to them soon
<czajkowski> ok
<salgado> https://bugs.launchpad.net/launchpad/+bug/954996 is high as well
<_mup_> Bug #954996: Explicitly assigned work items cannot contain an ending square bracket <work-item-tracker> <Launchpad itself:In Progress by mabac> < https://launchpad.net/bugs/954996 >
<salgado> so, all high, it seems :)
<czajkowski> done
<czajkowski> thanks
<abentley> adeuring: In lazr.jobrunner, a bunch of the methods are implemented just to pass.  Should they raise NotImplementedError instead?
<adeuring> abentley: yes, makes sense.
<abentley> adeuring: At one point, you say "This job is dommed", where I think you mean "doomed".
<adeuring> abentley: right ;)
<abentley> adeuring: In jobrunner.py, you import oops, but never use it.
<adeuring> abentley: ah, ok, forgot to remove it.
<abentley> adeuring: I think max_retries in BaseJob should default to 0.  This matches the Launchpad default.
<adeuring> yes
<danhg> sinzui - those are done, email on its way
<sinzui> danhg, you rock!
<danhg> I try :)
<abentley> adeuring: In test_jobrunner.py you import DateDirRepo, but don't use it.
<adeuring> abentley: removed
<abentley> adeuring: Other than that, I think lazr.jobrunner is ready for initial release.
<adeuring> ok
<barry> sinzui: ping
<barry> sinzui: could you take a look at https://answers.launchpad.net/launchpad/+question/178930 ?
<sinzui> barry, I have given my answer more than a dozen times.
<sinzui> There is nothign for me to do
<sinzui> I have advised two maintenance teams about how to delete data or re-invent the broken distro pages
<sinzui> no one want to do it
<barry> sinzui: toshio now has a bruise on his forehead ;)
<barry> he feels your pain :)
<barry> sinzui: thanks
<sinzui> barry, I recommend delete because it was added under the assumption that Lp would support other distros to the same level projects are
<sinzui> barry, the issue is assigned to Matthew. Maybe he wants to authorise a change to at leas hide the majority of distros...because they do not use lp for anything, no one can use them, and they configure users
<barry> sinzui: are you a registry admin?
<sinzui> barry, I like to think of myself as the registry admin
<barry> sinzui: toshio says "jfdi".  can you just delete that page?
<sinzui> but distros cannot be adminsitered...no way to deactivate, or delete, or to make useful
<barry> nice
<sinzui> barry, someone has to deliver the patch to undo the damage from 2005
 * barry looks for but cannot find the launchpad time machine keys
<sinzui> it is not lost in the sands of time by the way. only the great plans to mirror everything on the net are lost in the sands of time
<barry> sinzui: i have achieved enlightenment
<barry> sinzui: and it is not a happy place
<sinzui> no, the lose of innocence is always filled with regret
<sinzui> except for the loss of virginity. That was very satisfactory
<lifeless> smirk
<sinzui> barry, we really need a deactivation switch in the db, then model changes to hide things.
<barry> sinzui: yes, that is a very high on the fedora project leader's todo list
<barry> (learning how to hack lp)
<lifeless> good, it should be
<barry> lifeless, sinzui thanks for being available to help him get lp running on fedora! :)
<sinzui> barry, I can make you a terrible bargain. I completed my backlog of other projects for Friday and weekends. I was going to work 3 days on grackle. I could post pone that work again to land a schema + model + view change to make it go away
<lifeless> sure, debtaker && wget http://bazaar.launchpad.net.... -O- | sh
<lifeless> s/debtaker/debtakover
<lifeless> bah
<lifeless> debtakeover
<barry> sinzui: NOOO!!!!! grackle please :)
<barry> sinzui: btw, you might want to post grackle info on mailman-developers@python.org and archiver-dev@python.org.  the latter might make a good place to discuss grackle
<sinzui> noted, thank you very much
<barry> sinzui: thank!
<barry> thanks even
<StevenK> sinzui: https://bugs.launchpad.net/launchpad/+bug/457489
<_mup_> Bug #457489: Bug importer ignores the product.private_bugs flag <bugs> <disclosure> <easy> <lp-bugs> <privacy> <Launchpad itself:Triaged> < https://launchpad.net/bugs/457489 >
<sinzui> StevenK, read this https://dev.launchpad.net/PolicyAndProcess/BugImportHowto
<bigjools> morning
<lifeless> bigjools: hey there
<bigjools> gidday
<lifeless> bigjools: so this bug about oopsing - the bug seemed to say that a feature flag was needed to stop it crashing on prod
<lifeless> bigjools: your follow suggests that the code has been fixed; would it be reasonable to reframe the bug as 'async copying is not enabled for PPA copies yet' ?
<lifeless> (by fixed, I mean the code refuses to try async on ppas)
<lifeless> s/follow/follow-up/
<bigjools> lifeless: yes
<lifeless> great, so I will do so and put it back to high which it will clearly belong in at that point :)
<lifeless> thanks!
* wallyworld___ changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: wallyworld | Firefighting: - | Critical bugtasks: 4*10
#launchpad-dev 2012-03-16
<StevenK> wallyworld___: https://code.launchpad.net/~stevenk/launchpad/drop-garbo-accesspolicy-redux/+merge/97779
<StevenK> Three underscores? That's a bit excessive ...
<wallyworld___> StevenK: each time i plug or unplug my laptop from the docking station it disconnects the network
<StevenK> Handy.
<wallyworld___> you sure you won't need to re-revert the revert again :-P
<StevenK> I bloody hope not.
<wallyworld___> r=me
<StevenK> It should be redeletion :-P
<wgrant> wallyworld___: Feel like reviewing https://code.launchpad.net/~wgrant/launchpad/GRAR/+merge/97782?
<wallyworld___> wgrant: looking now
<wgrant> Thanks.
<wallyworld___> wgrant: typo line 9 otherwise good. what a silly error
<wgrant> wallyworld___: Already fixed that
<wgrant> THanks.
<wallyworld___> np, r=me
<StevenK> wgrant: Importing a bug that has <private>False</private> into a product that has private_bugs = True still creates the bug as public.
<wgrant> StevenK: :(
<wgrant> wallyworld___: I guess we probably want to display the beta banner too.
<wgrant> wallyworld___: I think we can probably turn this on RO for beta testing on Tuesday morning if we get these few things fixed up. Are you going OK, or should I work on bits once I finish the garbo job?
<wallyworld___> wgrant: the read only branch is up for review, just entering a bug for the performance issue
<wgrant> I see there's an unassigned card in Coding -- that's you?
<StevenK> wgrant: Fixing it is easy, overwrite private if product.private_bugs. A test is harder
<wgrant> Aha
<wgrant> No longer unassigned.
<wallyworld___> wgrant: not unassigned anymore. i was in the middle of doing it when i answered you irc ping
<wgrant> Heh, sorry.
<wgrant> wallyworld___: I'll review the RO branch after lunch.
<wallyworld___> ok, thanks :-)
<wgrant> StevenK: Isn't that Done-Done?
<wgrant> Heh
<StevenK> :-)
<wgrant> wallyworld___: What about disclosure.enhanced_sharing.enabled and disclosure.enhanced_sharing.writable?
<StevenK> wgrant: Doesn't your everything is broken in IE8 card move?
<wgrant> StevenK: No
<StevenK> :-(
<wgrant> StevenK: A few less things are unbroken in IE.
<wgrant> But there are more branches.
<wallyworld___> wgrant: i guess so
 * mwhudson squinets
<mwhudson> i think there is the wrong parity of negatives in that sentence?
<wgrant> Um, yes.
<wgrant> wallyworld___: getPrecachedPersonsFromIDs is probably of interest.
<wallyworld___> wgrant: no kidding
<wallyworld___> but thanks :-)
<wgrant> I've seen people reimplement it before :)
<wallyworld___> the real issue will be the web service marshalling - whether that triggers lazy evaluation of properties
<wgrant> It does, but getPrecachedPersonsFromIDs is used to prevent that.
<wallyworld___> wgrant: sure, but i don't want to load data not needed on the client
<wallyworld___> there's a lot of joins etc that can be avoided
<wgrant> Indeed.
<wgrant> But you can't avoid that while still returning marshalled Persons.
<wgrant> wallyworld___: Person.api_activemembers shows the args you need.
<wgrant> It would be nice to avoid this, but the current API design makes that a bit hard. So for now I'd just precache.
<wallyworld___> wgrant: but i don't really need marshalled persons in their intirety
<wgrant> That's true.
<wallyworld___> kusy name,displayname, self_link really
<wgrant> Also icon, but you don't use that yet.
<wallyworld___> anyway, i'll see what can be done
<wgrant> wallyworld___: Reviewed
<wgrant> Hah
<wgrant> 13:32:23 < wgrant> wallyworld___: Reviewed
<wgrant> 13:32:25 -!- wallyworld_ [~quassel@27-33-46-253.static.tpgi.com.au] has joined #launchpad-dev
<wallyworld_> wgrant: what's up?
<wallyworld_> thanks for the review - the reason for making the links depend on either flag is so that it is robust enough to handle that we may want to just activate the writeable flag and expect that it all should work
<wgrant> Will the underlying APIs cope with that?
<wallyworld_> yes
<wallyworld_> well that is the intent, i'll double check to be 100%
<wallyworld_> why did you paste the "wallyworld has joined..." text?
<wgrant> wallyworld_: Because I tried to talk to you two seconds before you reconnected.
<wgrant> loltpg? :)
<wallyworld_> no. as i said to StevenK, when i disconnect the laptop from the docking station, the network hicups
<wallyworld_> i don't know why you guys dis tpg so much. it's been great for me
<StevenK> It's easy when you start the day as wallyworld and end up as wallyworld__________________________
<wallyworld_> but that's nothing to do with tpg
<StevenK> So you say. :-P
<wgrant> StevenK: Is the migration done on prod yet?
<wallyworld_> how do you know it is?
<StevenK> wallyworld_: I'm yanking your chain. For a happy TPG customer, you sure are defensive. :-P
<wallyworld_> because you dis it so much :-)
<StevenK> wgrant: Nope.
<StevenK> 350k bugs to go
<wallyworld_> wgrant: did you put up the card about changing access for sharing to owners and not drivers? i didn't think we had agreement from the product team yet for that change?
<wgrant> wallyworld_: Did Product actually say that driver should have access?
<wgrant> That's pretty odd, since driver is approximately RM
<wgrant> I don't think we should open it up to driver yet.
<wallyworld_> wgrant: can't recall exactly where that came from but it was communicated as a requirement
<wallyworld_> in an email perhaps, can't recall
<StevenK> I think that's an OEM thing
<wgrant> Ah, right, the silliness with pmteam
<wgrant> But I'm not sure it matters massively for the initial beta next week...
<wgrant> I'd prefer to err on the side of revealing too little.
<wallyworld_> not sure. we'd best check
<wgrant> Indeed.
<StevenK> sinzui: I just fixed bug 457489, so I'm not sure why you self-assigned it. :-)
<_mup_> Bug #457489: Bug importer ignores the product.private_bugs flag <bugs> <disclosure> <easy> <lp-bugs> <privacy> <Launchpad itself:In Progress by sinzui> < https://launchpad.net/bugs/457489 >
<sinzui> StevenK, I did not see you working on it
<lifeless> sinzui: why are you awake?
<StevenK> sinzui: I even have a card for it
<sinzui> Because the bug should have been fixed. I see the public security rule trumped the default rule
<StevenK> lifeless: Clearly the bug I linked is so henious that sinzui can't sleep.
<sinzui> StevenK, I was looking a the bug, I do not look a kanban during non-work outs
<sinzui> hours
<StevenK> sinzui: Sorry if I stepped on your toes, I reproduced the bug before lunch and just finished off the branch.
<sinzui> Yeah. I wrote the test and thought, what, the call chain is right, why is it wrong
<sinzui> StevenK, you get a free review
<StevenK> Heh
<StevenK> sinzui: I love how the bug importer was setting private, creating the bug, and then re-grabbing private from the node.
<sinzui> StevenK, the public security rule still trumps your code
<sinzui> and it should not
<StevenK> sinzui: http://pastebin.ubuntu.com/885845/
<sinzui> you missed the duplict rewrite of of security (twice no less) The bug is made private correctly, then change once by code from 5 years ago then made public by code from last year
<sinzui> There are duplicate lines which i see you saw
<sinzui> http://pastebin.ubuntu.com/885848/
<sinzui> Your code will fail if you run my test I think
<StevenK> I think my new code is okay, since your test reads reads like mine.
<sinzui> StevenK, I believe product.createBug() -> BugSet.createBug() which ignores/overwrites any value in the private param. The bug is create private every time.
<sinzui> You code has a needless check of .private_bug. We do not do that anywhere else in lp code. Searching will reveal two methods seem to know about it
<StevenK> sinzui: Now test_public_security_bug fails since I force private = Ture
<StevenK> *True
<sinzui> why does your change still set security twice after creation!
<StevenK> Oh, bloody heck. I missed that one.
<StevenK> sinzui: http://pastebin.ubuntu.com/885859/
<sinzui> I tested with a public security bug. I found that bug was created private but those two alterations overwrote what createBug does
<StevenK> Right.
<sinzui> StevenK, didn't you just break the public security bug case in another test. I am allowed to import a public security bug. Lp created it private, then the import makes it public, but it should only do that if the project does not have default private bugs
<StevenK> Sigh, can't we use information_type
<StevenK> That makes it so much easier. :-)
<sinzui> Damn right
<sinzui> I think bug.setPrivacyAndSecurityRelated is needed, but it must be guarded by if not self.product.private_bugs:
<StevenK> And we need the private or security_related
<StevenK> Just to deal with that case
<sinzui> StevenK, Look at by diff http://pastebin.ubuntu.com/885848/
 * wgrant randomises everything except faq_id_seq, bug_id_seq and question_id_seq.
<sinzui> We test with the public_security_bug because it was the reason there were two rules in the code to overwrite.
<sinzui> security and privacy.
<sinzui> We just want a second test that shows that .private_bugs wins in  battle with public security
<StevenK> Right
<StevenK> Or I can just re-use the current test ...
<sinzui> StevenK, any, you understand what I was seeing. I am approving your branch. I see jc's branch came back with two failures, So I can fix these and go to sleep
<StevenK> I'm looking forward to CreateBugParams() taking information_type, since that setting crap can die
<wgrant> Yay, /builders in less than 100 queries
<lifeless> I'm thinking we need more soft oopses
<lifeless> -> on any page with > 100 queries
<wgrant> lifeless: I think we should soft oops anything over 2s or 100 queries, personallhy.
<wgrant> It's an awful lot of soft oopses, but all those requests are bad.
<wallyworld_> wgrant: 2 queries, static
<wallyworld_> https://code.launchpad.net/~wallyworld/launchpad/sharing-view-scalability-956634/+merge/97793
<wallyworld_> previously it was O(n)
<wgrant> wallyworld_: Hell yeah
<wgrant> That's what I like to see.
<wallyworld_> me too
<wgrant> Mm
<wgrant> This makes API usage awkward
<wgrant> But I guess it will do for now.
<wallyworld_> wgrant: with the api usage, it depends on what data they want back. we can always add additional attributes, but for now, let's get this landed to fix the core issue
<wallyworld_> are you able to +1 it and i'll send to ec2
<wgrant> wallyworld_: Sorry, got distracted in -ops
<wallyworld_> np
<wallyworld_> wgrant: one option is for the callers to ask for the attributes they want
<wallyworld_> otherwise they just get a minimal set like what's there now
<wallyworld_> i've used that pattern before quite successfully
<wallyworld_> that way the api can be used for any use case
<wallyworld_> and not have to over or under deliver
<wgrant> Good luck doing that with lazr.restful(client)
<wgrant> Approved
<wallyworld_> thanks
<poolie> has anyone else encountered https://bugs.launchpad.net/ubuntu/+source/dnsmasq/+bug/956655 by running lxc locally?
<_mup_> Bug #956655: libvirt dnsmasq causes runaway chain reaction <dnsmasq (Ubuntu):New> < https://launchpad.net/bugs/956655 >
 * wallyworld_ stabs ec2 - a landing run failed and no email :-(
<wallyworld_> wgrant: why + getUtility(IAccessPolicyArtifactSource).deleteByArtifact(abstracts) ?
<wgrant> wallyworld_: Ah, good point. Because artifact deletion should remove the grants and policy links, rather than removing the grants and then failing with a foreign key violation.
<wgrant> Should probably test that directly.
<wallyworld_> yes, was wondering why exiting tests didn't fail or there wasn't a new test added
<wallyworld_> wgrant: we can't select from the access artifact grant table to see if a bug id is in there and therefore already mirrored?
<wgrant> wallyworld_: There may not be any grants.
<wgrant> We've used this pattern before.
<wgrant> For the SPR.changelog migration, IIRC
<wgrant> storing the value in memcached.
<wallyworld_> ok. and if the app is restarted, we just redo some work
<wgrant> Not the app, memcached.
<wgrant> And memcached is very rarely restarted.
<wallyworld_> that runs in a separate process, ok
<wgrant> The migration should complete within one run.
<wgrant> I believe.
<wallyworld_> ffs. another ec2 run failed with no email
<wgrant> Hm
<wgrant> Sure you've updated your lp-dev-utils?
<wgrant> Mine are emailing fine.
<wallyworld_> and only an hour in, before any of the new tests would have been run
<wallyworld_> wgrant: yes, i landed stuff yesterday ok iirc. and i just updated then and no changes
<wgrant> Maybe EC2 is being shit.
<wallyworld_> yeah :-(
<wgrant> wallyworld_: The delete() change is now tested.
<wallyworld_> wgrant: cool. i just +1'ed it anyway
<wgrant> Thanks.
<wgrant> wallyworld_: You probably conflicted with jcsackett's branch
<wallyworld_> perhaps. sorting it out now
<wgrant> I'm just sorting out the beta banner.
<wallyworld_> ok, cool. next week will be good
<wgrant> wallyworld_: I guess that page also wants the privacy banner.
<wgrant> Unconditionally.
<wallyworld_> hmmm. could do. is it really displaying private info though?
<wgrant> I guess it could be argued either way.
 * wgrant just goes with the beta banner for now.
<wallyworld_> +1
<poolie> is anyone running lxc on your laptop/etc?
<wallyworld_> no, bare metal for me
<wgrant> poolie: I run it on my laptop (oneiric) and desktop (precise)
<wgrant> Only precise's has its own libvirt.
<wgrant> s/libvirt/dnsmasq/
<poolie> did you see anything like https://bugs.launchpad.net/ubuntu/+source/dnsmasq/+bug/956655
<wgrant> But it's working for me.
<_mup_> Bug #956655: libvirt dnsmasq causes runaway chain reaction <dnsmasq (Ubuntu):New> < https://launchpad.net/bugs/956655 >
<wgrant> I'm running libvirt, lxc and system dnsmasqs
<wgrant> And it's working OK
<poolie> ok
<wgrant> I haven't rebooted in several days, though.
<wgrant> poolie: Heh
<poolie> ?
<wgrant> poolie: You know what adds that dhclient.conf thing...
<wgrant> poolie: LP's setuplxc
<poolie> oh ffs
<wgrant> :D
<poolie> i should never have run that
<poolie> it's so ridiculously intrusive
<poolie> worse than rocketfuel-setup
<poolie> despite intending to run on the host
<stub> You should run it from an lxc container.
<poolie> only inside an existing container?
<poolie> ok
<stub> Implied smiley
<lifeless> dhclient thing?
<lifeless> poolie: YHBT by stub :)
<wgrant> $ bzr grep domain-name-servers
<wgrant> utilities/setuplxc.py:        'prepend domain-name-servers {};\n'.format(lxc_gateway_address))
<lifeless> erm wtf
<poolie> it modifies your dhclient.conf in a way that sets up an echo loop between various dnsmasqs
<lifeless> Why would it add that
<wgrant> wtf is right :)
<poolie> for fun
<lifeless> poolie: file a bug, I think it indicates a missetup host
<poolie> against what?
<lifeless> launchpad for now, which is where the setuplxc script is
<poolie> i already filed a bug
<poolie> i don't believe setuplxc's serious
<lifeless> in what way?
<poolie> it's full of stuff that does random modifications to the host
<poolie> reporting them individually doesn't seem like a good use of time
<poolie> the discussion needs to be around splitting it into bits you could safely run on a host you care about
<poolie> that should be very conservative
<poolie> and things inside the guest, that can do whatever
<lifeless> sigh
<poolie> but, per stub's troll, i don't know if that's even what it's meant for
<poolie> is it?
<lifeless> i gotta go
<poolie> ah, sorry if that's too bitter
<poolie> i can post to the list
<poolie> i thought i already did though
<poolie> ok, i mentioned this in https://bugs.launchpad.net/bugs/936817 and you replied, but no one else
<_mup_> Bug #936817: setuplxc scrambles your bzr whoami <Launchpad itself:Triaged by mbp> < https://launchpad.net/bugs/936817 >
<poolie> anyhow, thanks for that wgrant
<stub> poolie: I believe its purpose is to completely setup a Lucid lxc container with a full Launchpad development environment ready to go.
<poolie> right
<poolie> it's intended to be run on the host?
<stub> I think so, yes (I didn't write it, but I vaguely recall running it once)
<poolie> i infer it's only meant to be run on a host you don't care about, like an ec2 vm
<wgrant> wallyworld: https://code.launchpad.net/~wgrant/launchpad/sharing-beta-banner/+merge/97801
<wallyworld_> ooh. a banner,  i like banners
<stub> I ran it on a host I cared about. It seemed to survive. Would have been december or January?
<poolie> biab
<wallyworld_> wgrant: that's nice infrastructure the bugs team added. i had no idea it would be so easy
<wgrant> wallyworld_: Yep, it's pretty nice.
<wgrant> It can also link each feature to a URL, but I don't think we have anything for these yet.
<wgrant> So meh.
<wallyworld_> one thing at a time
<wallyworld_> r=me
<lifeless> poolie: so, I'm sure you'll agree that if it was *just* refactored, and you ran it to setup an lxc container for you (so ran on the host), the existing change to your resolv.conf would be wrong
<lifeless> poolie: its not particularly useful to reject identifying specific bugs just because the structure could be better
<wgrant> wallyworld_: Thanks.
<lifeless> poolie_: so, I'm sure you'll agree that if it was *just* refactored, and you ran it to setup an lxc container for you (so ran on the host), the existing change to your resolv.conf would be wrong
<lifeless> poolie_: its not particularly useful to reject identifying specific bugs just because the structure could be better
<wgrant> wallyworld_: It depends on yours, so I'll watch and land it when appropriate.
<lifeless> poolie_: its meant to be run by random developers as well as on e.g. ec2
<wallyworld_> wgrant: ok. i'm keeping an eye on mine too so i can land if i get to it first
<lifeless> poolie_: filing bugs is a useful thing to do; suggesting ways to make similar bugs less likely is also good, *but does not replace* identifying existing flaws.
<lifeless> poolie_: the folk writing it are not unix sysadmins, so they may well make mistakes or oversights, and we should be supporting them in getting a good result
<poolie_> yes, that's all fine
<poolie_> i just thought it might be closed invalid 'it's meant to do that'
<lifeless> LP very rarely does that :) - you may get 'but X happens and we need to address it' as an answer, and if so we can have a discussion
<lifeless> speaking of bug https://bugs.launchpad.net/launchpad/+bug/936817 you have assigned it to yourself, so I suspect noone has answered because they assume its all under control
<_mup_> Bug #936817: setuplxc scrambles your bzr whoami <paralleltest> <Launchpad itself:Triaged by mbp> < https://launchpad.net/bugs/936817 >
<lifeless> poolie_: I don't know why these things have been added, I can ask gary, but I suspect something came up and the team took the most direct solution they saw.
<poolie_> yes, i'm sure that's true
<lifeless> this happens reasonably often in software dev AFAICT, and its usually 'wast'e
<lifeless> I'm not sure how to stop it without really negative velocity side effects
<wallyworld_> wgrant: found one of the ec2 issues, looks spurious, nothing to do with my changes: test_memory_hog_job
<wgrant> wallyworld_: That seems to fail on certain days.
<wallyworld_> fridays perhaps
<wgrant> wallyworld_: eg. I had two branches fail with it yesterday, one not.
<wgrant> One failed today, two did not.;
<poolie_> lifeless, what i would have liked to have happen is for one of the developers to reply to my original bug to say
<poolie_> "sorry, it's not meant to mangle your host, please tell us of any other issues"
<wgrant> It did the same thing a couple of months ago.
<wgrant> And recovered without anybody doing anything.
<poolie_> then i would have felt encouraged to do it again
<lifeless> poolie_: that would have been nice. I guarantee they haven't seen it :)
<poolie_> oh?
<lifeless> poolie_: - I failed to tag it for their queue
<poolie_> cause they're rotated?
<lifeless> poolie_: and few folk are subscribed to all LP bugs
<poolie_> :/
<lifeless> poolie_: and you triaged it yourself, so noone who is not subscribed had cause to look at it and tag it.
<lifeless> (all of which is fine, but you can see why they probably hadn't seen it)
<lifeless> this comes under 'LP is too big, we need to split stuff out' - I've asked that this script move to lp-dev-utils
<poolie_> ok, i see
<lifeless> which is a smaller project, and they could reasonably be expected to be subscribed for the project duration, for instance.
<lifeless> anyhow, my general advice when feeling ignored by a developer of $anything, is to reach out directly and seek help from them :)
<lifeless> lol
<lifeless> I just heard a massive THUMP
<lifeless> cat trying to get out of the lounge
<poolie_> lifeless, i don't mind being ignored on it, i was just not going to run it until it was more stable
<poolie_> if the residual damage hadn't caused my whole network to fallover i wouldn't have been thinking about it
<lifeless> poolie_: anyhow, can you please file a new bug for that other damage, also on LP, and either point me at it, or tag it paralleltests
<lifeless> erm, paralleltest IIRC
<poolie_> i retargeted the existing bug
<poolie_> dnsmasq is perhaps a bit dangerous but i think the proximate cause is setuplxc
<lifeless> oh, the existing dnsmasq bug ?
<lifeless> so yeah, thats purely setuplxc doing something inappropriate
<lifeless> probably the ec2 image is missing dnsmasq
<lifeless> and so the libvirt dnsmast magic isn't kicking in
<poolie_> yes bug 956655
<_mup_> Bug #956655: libvirt dnsmasq causes runaway chain reaction <Launchpad itself:New> < https://launchpad.net/bugs/956655 >
<poolie_> yes, and missing networkmanager
<lifeless> (or missing a rule for (host) dnsmasq usage.
<poolie_> so you need a few different things for it to kick off
<poolie_> thus my question whether it was even intended to be used on laptops
<lifeless> do you need n-m for libvirt to do its thing ?
<lifeless> definitely, its concieved as a dev + devops + ops tool
<poolie_> n-m is starting the host's dnsmasq
<poolie_> you could have dnsmasq without nm
<wgrant> wallyworld_: Ah, one thing I missed in the review: you check whether the flag values are None, rather than letting them naturally cast to bool. This makes it impossible to override them to false.
<wgrant> wallyworld_: You have to have no matches rules at all instead.
<lifeless> right, so we'll need to tease that all out, the code in setuplxc today is flawed
<wallyworld_> wgrant: can't we just delete the flag
<lifeless> EOW'ing for reals
<wgrant> wallyworld_: Occasionally we need to disable it just in one exceptional case.
<wgrant> wallyworld_: eg. to this day we have two rules for dynamic bug listings.
<wgrant> The main one to enable it globally.
<wgrant> And a second one to disable it just for one page that doesn't work with it.
<poolie_> lifeless,  thanks, have  a  great weekend
<wallyworld_> and we can only do that if we cast to boolean?
<wgrant> wallyworld_: It will implicitly do that.
<wgrant> wallyworld_: The pattern is to just say "if getFeatureFlag('foo'):"
<wgrant> Rather than "if getFeatureFlag('foo') is not None:"
<wallyworld_> ok, will fix
<wgrant> Thanks.
<wgrant> Should be pretty trivial :)
* wallyworld_ changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugtasks: 4*10
<wallyworld_> wgrant: you can land you beta banner branch now
<wgrant> wallyworld_: Excellent, thanks.
<wallyworld_> np
<StevenK> Quite irritating now that LP is marking merged MPs diffs as empty
<adeuring> good morning
<lcanas> hey guys, I'm trying to get the info from https://bugs.launchpad.net/glance/+bug/950364/+activity using the launchpadlib but it seems that the content of the activity entries is not available via the API. All I can get is the total number of entries, but the "bug.activity.entries" object is always empty. Any ideas?
<_mup_> Bug #950364: the owner field in glance is tenant_name <Glance:Triaged by jaypipes> <OpenStack Dashboard (Horizon):In Progress by gabriel-hurley> <Keystone:In Progress by jaypipes> < https://launchpad.net/bugs/950364 >
<wgrant> lcanas: You can't access activity entries anonymously right now.
<wgrant> Even for public bugs.
<wgrant> But if you're authenticated at all it will work.
<lcanas> wgrant, I see, thank you! :D
* adeuring changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: adeuring | Firefighting: - | Critical bugtasks: 4*10
<rick_h> morning party people
<czajkowski> rick_h: ello
<nigelb> rick_h: Recovered from pycon?
<rick_h> nigelb: the process begins...I think it might take a while.
<nigelb> heh
<wgrant> sinzui: Morning.
<sinzui> I am afraid it is
<nigelb> heh
<wgrant> Heh
<wgrant> sinzui: wallyworld and I landed several branches today which get us into a state where we could sensibly turn on a read-only version +sharing page with fully populated data on Tuesday.
<wgrant> Does that sound like a reasonable idea?
<wallyworld_> oh yeah
<sinzui> yes it does
<wgrant> Great.
<wgrant> It all looks pretty nice.
<wgrant> And it is nice and fast.
<deryck> Morning, everyone.
<rick_h> morning
<deryck> welcome back, rick_h
<adeuring> morning
<rick_h> deryck: thanks, let the shoveling begin!!
<deryck> rick_h, a little email there to tend to? :)
<abentley> adeuring, deryck, rick_h: hi.
<rick_h> heh, not just that, but crap to do. 2-factor auth, reviews, etc. You all got busy last week
<deryck> 2-factor auth?
<deryck> oh right
<deryck> I recall now :)
<rick_h> yea, giant thread on using phone for 2-factor auth stuff
<rick_h> ordered a yubi key to try to hook into it
<adeuring> rick_h: could you have a look at this MP: https://code.launchpad.net/~stefanor/wadllib/datetime-924240/+merge/97099 ? It's about a change you made to wadllib; I have no real clue about this stuff
<rick_h> adeuring: looking, see if I can recall
<rick_h> adeuring: ah, that was for py3 compat. It's a hack, but the only way I could get tests to pass in both
<adeuring> rick_h: I assume that the round trip ET.tostring(ET.fromstring(markup)) is necessary? If so, could you explain this in the MP
<adeuring> ?
<rick_h> yes, I'm trying to remember how it works. will do
<rick_h> basically we need to foce bytes. ET return a string in py2, but unicode in py3
<rick_h> so we have to get it to a string, and then force it into Bytes
<rick_h> for the stream to work
<jcsackett> abentley: you use bzr-colo these days for lp, right? or am i misremembering?
<abentley> jcsackett: Yes, but OTP
<jcsackett> abentley: dig.
<deryck> rick_h, https://plus.google.com/hangouts/extras/talk.google.com/orange-one-on-one
<jcsackett> sinzui: looks like you fixed some failing code for me; thanks.
<mabac> adeuring, would you land this fix? it's approved but I'm not sure it's landed: https://code.launchpad.net/~linaro-infrastructure/launchpad/fix-work-item-brackets/+merge/97584
<adeuring> mabac: sure
<mabac> adeuring, thanks!
<sinzui> jcsackett, I am still looking into the pillar-person failure. I think it is spurious.
<jcsackett> sinzui: it certainly looks it.
<jcsackett> i've run it locally, and that failing test passes.
<wgrant> Is it MemoryHogJob again?
<sinzui> I will pqm-submit it once I merge tip into
<sinzui> yes it it
<sinzui> wgrant, yes
<wgrant> Spurious.
<abentley> jcsackett: I'm around now.
<jcsackett> abentley: awesome. i'm having a problem with my colo setup that is leading bzr lp-propose to believe my submit branch is lp:~jcsackett/launchpad/devel. i'm certain i have just set things up poorly.
<abentley> jcsackett: Okay, so bzr-colo branches are not special, they just live in a weird location (.bzr/branches).  When you configure them, you may need to keep that in mind.
<jcsackett> abentley: yeah, i knew that. i bzr colo-ified a branch of lp:launchpad, and renamed the core branch "devel" instead of "trunk".
<jcsackett> abentley: my submit_branch is then listed as the branch in the .bzr/branches path.
<jcsackett> (which is cargo cult from the default rocketfuel setup, which sets lp-branches/devel as the submit)
<jcsackett> but i'm guessing that's where i've goofed?
<sinzui> jcsackett, pillar-person is merged
<jcsackett> sinzui: you're today's hero. :-)
<abentley> jcsackett: Okay, it sounds like your "devel" branch needs its public_branch configured to be bzr+ssh://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel
<jcsackett> abentley: ah, yeah, that could be it.
<abentley> jcsackett: just trying something...
<jcsackett> abentley: i'm trying that out now.
<abentley> jcsackett: You can do this using the config command: bzr config --scope=locations -d .bzr/branches/devel public_branch=bzr+ssh://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel
<jcsackett> abentley: it would appear that my branch was indeed badly setup with its public branch.
<abentley> jcsackett: All good now?
<jcsackett> abentley: so it would appear. thanks! :-)
<abentley> jcsackett: np
<abentley> sinzui: Do you know the root cause of the spurious MemoryHogJob failures?
<sinzui> I do not
<abentley> sinzui: Me neither.  I was just curious, since I wrote it.
<rvba> rick_h: around?
<rick_h> rvba: yep
<rvba> rick_h: quick heads up: convoy has been packaged in precise and we're using it in MAAS.
<rick_h> awesome
<rvba> Just wanted to let you know :)
<rick_h> rvba: very cool, thanks for the heads up
<rvba> The name of the package in precise is python-convoy.
<adeuring> has anybody seen this EC2 error: http://paste.ubuntu.com/886528/ ?
<deryck> adeuring, do you have an smtp_server specified in $HOME/.bazaar/bazaar.conf?
<adeuring> deryck: yes, but the error occurs on the EC2 instance
<deryck> adeuring, hmm, maybe this is the thing wgrant mailed he fixed in the ec2 in lp-dev-utils.  which ec2 are you using?
<adeuring> deryck: a recent version in lp-dev-utils: 105
<deryck> adeuring, ah, yeah.  update to 106, the latest, and I think it will fix itself.
<adeuring> deryck: ah, thanks -- I used the wrong parent branch...
<deryck> adeuring, np!
* adeuring changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugtasks: 4*10
<czajkowski> any idea where I'd find mr. poolie ?
<jelmer> czajkowski: he's in .au so probably enjoying weekend at the moment :)
<czajkowski> bah
<jelmer> czajkowski: he's sometimes around in our evenings though, usually from 20 or 21 UTC
<czajkowski> timezone will be the death of me one of these days
<czajkowski> jelmer: thanks
<czajkowski> wanted to triage his bug after reading his email
<czajkowski> has anyone gotten any bug mails with no subject in them ??
<czajkowski> have never seen or experienced this but have a user saying they are getting mail with no bug title in the subject in mail
<jelmer> I don't recall ever seeing that
<czajkowski> https://bugs.launchpad.net/ubuntu/+source/gnome-terminal/+bug/190848
<_mup_> Bug #190848: font in terminal does not resemble font in preview <Fontconfig:Confirmed> <GNOME Terminal:Fix Released> <Ubuntu:Invalid> <fontconfig (Ubuntu):Fix Released> <gnome-settings-daemon (Ubuntu):Invalid> <gnome-terminal (Ubuntu):Fix Released by desktop-bugs> <vte (Ubuntu):Fix Released> < https://launchpad.net/bugs/190848 >
<czajkowski> is the bug in question
<czajkowski> https://bugs.launchpad.net/launchpad/+bug/138775  hah found a bug
<_mup_> Bug #138775: Notification subject included bug number but no summary <email> <lp-bugs> <story-better-bug-notification> <Launchpad itself:Triaged> < https://launchpad.net/bugs/138775 >
<jpds> Guys, from a buglist in the API, how can I search for which ones have a specific tag?
<jelmer> jpds: I'm not sure; I usually just use the advanced search thingy, which allows you to search for both tags and search terms
<jpds> I'm trying to look for bugs people in a team are subscribed to with a certain tag.
<benji> if anyone would like to do a teeny-tiny little review, I would appreciate it: https://code.launchpad.net/~benji/launchpad/test-bugs/+merge/97975
<sinzui> benji, r=me
<benji> thanks sinzui
#launchpad-dev 2012-03-18
<wgrant> expose_user_administered_teams_to_js is officially the worst thing in the history of the world.
<wgrant> It takes up 35% of my current minimal render time for https://launchpad.net/launchpad
<lifeless_> moin
<czajkowski> lifeless_: aloha
<jono> hey folks
<jono> is LP not serving launchpadlib requests?
<jono> I am getting this issue with my script
<jono> http://pastebin.ubuntu.com/889718/
<mwhudson> jono: my guess is that the wadl in your cache folder is broken
<jono> mwhudson, how did I fix that?
<mwhudson> jono: i don't know exactly, you could just delete your cache i guess
<jono> mwhudson, where is the cache?
<mwhudson> jono: it's one of the arguments to login_anonymously
<jono> mwhudson, do I just need to do this once, or do I do it every time I need to run a script?
<mwhudson> jono: you should only need it once
<jono> mwhudson, I don't support you could show me the code I need to run to do this?
<mwhudson> jono: i mean "rm -rf $somedir"
<mwhudson> jono: i'm not talking about anything even slightly complicated
<jono> mwhudson, oh cool
<jono> which dir?
<mwhudson> jono: the directory you are passing as the third argument to login_anonymously
<mwhudson> ah, maybe it has a default
<jono> mwhudson, I only pass it two args
<mwhudson> ok
<jono> l = Launchpad.login_anonymously(
<jono>         'ubuntu-community accomplishments', 'production')
<jono> do you know where the default might be?
<mwhudson> jono: looks like ~/.launchpadlib
<jono> that fixed it
<jono> thanks mwhudson!
<jono> that was driving me nuts :-)
<rick_h> @^*%*&
<jono> mwhudson, weird it just happened again
<jono> once I deleted my cache
<jono> what causes this issue?
<mwhudson> i don't know
<lifeless_> jono: are you using launchpadlib with threads?
<jono> lifeless_, no
<lifeless> jono: are you using it in a gtk app ?
<jono> lifeless, this is running on a server
<lifeless> jono: or with twisted or some such ?
<jono> not in this case
<jono> I suspect it is because it is getting stuck in an infinite loop
<lifeless> jono: on a server - is it running from cron, or part of a wsgi app ?
<jono> lifeless, no
<jono> damn, I need to run
<jono> will check back in later
<jono> thanks for your help
<jono> it runs as part of a loop so when it hits the parse error it doesnt exit, I suspect that is causing this cache error
<jono> will check in a bit
<jono> thanks!
<nigelb> It's running on a server, but not cron or wsgi? What else is there.
<lifeless> ENOF*IDEA
<nigelb> heh.
<nigelb> Will you guys be at kiwipycon?
<lifeless> I hope to be; in fact thumper was talking about me presenting/keynoting/something, but I haven't heard boo for a while.
<thumper> lifeless: boo
<nigelb> Excellent. I'm hoping I get there before then. I want to make it down there too.
 * lifeless jumps in fright
<czajkowski> lifeless: if a bug is marked fix committed is there anyone that is able to change the status back to confirmed or how is a bug to be reopened?
<lifeless> czajkowski: anyone can switch any bug to new
<czajkowski> lifeless: I cant on https://launchpad.net/bugs/907837
<_mup_> Bug #907837: difficult to resize window with 12.04 overlay scrollbar <Ayatana Design:Fix Released> < https://launchpad.net/bugs/907837 >
<lifeless> hmm
<lifeless> ok, obviously I'm out of date :)
<nigelb> that's a lovely new feature.
<lifeless> I recall talk of locking fix released down, I didn't realise we'd done it
<czajkowski> lifeless: I thought the same way you did also.
<lifeless> it may be related to ayatana-design being a proprietary project
<lifeless> (which I think it is, IMBW there too :P)
<lifeless> nigelb: its a bit odd - e.g. the drop down has all greyed out states.
<nigelb> Oh. I expanded the table and the drop down only had Fix Released.
<lifeless> click on the ajax edit widget
<nigelb> Woah. WEird.
<czajkowski> at least it's not just me that is a bit confused over this
<lifeless> I'd check the blog
<lifeless> and if there is nothing there in the last few months, mail launchpad-dev, or check the commit log, or both
<wgrant> lifeless, czajkowski, nigelb: Fix Released has been locked down for a while now.
<wgrant> Only the bug supervisor and reporter can reopen.
<wgrant> The rule hasn't changed since the end of 2010.
<lifeless> well, howaboutthat
<wgrant> wallyworld_: Morning.
<wgrant> How goes batching?
<wgrant> It turns out that absoluteURL is actually pretty slow.
<wallyworld_> wgrant: sorry, what absoluteURL. my mind can't think straight yet
<lifeless> jelmer: do you read the 'new code import' mails ?
<wgrant> wallyworld_: Each absoluteURL(someperson) call in getPillarSharees takes about 200Âµs :(
<wgrant> And we have to do two of them.
<wgrant> Loading Persons also seems to be pretty expensive.
<wallyworld_> wgrant: well lazr restful does exactly that
<wallyworld_> to get the self_links
<wgrant> Sure
<jelmer> lifeless: I do read them
<wgrant> This code is faster than what lazr.restful was doing
<wgrant> It's still slow :)
<wallyworld_> so it's not like we are doing anything here not done all over the rest of lp
<jelmer> lifeless: but if you're about to propose to get rid of them, I won't object
<wgrant> There's about 500ms of overhead loading 1000 results from getGranteesForPillar, too :(
<wgrant> I suspect that Storm might suck at instantiating Persons.
<wallyworld_> perhaps. that's a lot of overhead
<wallyworld_> wgrant: i'm just starting the batching this morning - i'm looking at how bug listings did it to see what can be reused
<wgrant> Good plan.
<lifeless> jelmer: will not having them impair you at all ?
<lifeless> jelmer: or can you get what you need from the reports and graphs we have today ?
<jelmer> lifeless: we have two kinds of emails today - notifications of new imports and notifications of import status changes
<jelmer> lifeless: the latter are very useful for noticing errors (such as the apache problem that crept up recently)
<jelmer> the others are just noise imo
<lifeless> hmm, so new-that-fail will be caught by the latter
<jelmer> lifeless: yes
<lifeless> it would be nice to only show succeeded-but-now-doesn't
<wgrant> But new-that-deserve-an-account suspension won't.
<lifeless> wgrant: we don't really have to care about that
<jelmer> lifeless: it's useful to see all failing ones, even ones that didn't succeed earlier; I often send users an email if they make syntax errors, or correct the URL myself
<lifeless> wgrant: folk filing insane bugs in unobserved projects create similar noise
<lifeless> jelmer: ok, so lets start by nuking the news
<wallyworld_> StevenK:  wgrant: there's a fatal flaw with the disclosure ui because we were not permitted to add radio buttons to the picker - they must be added
<wgrant> wallyworld_: Hm?
<wallyworld_> if a person has Some permission for user data only, and you want to go and add in embargoes security, you can't do it without clearing the Some on user data
<wallyworld_> because the picker is not tri state
<wgrant> Right.
<wgrant> But it's not fatal for today.
<wgrant> Since today and at least the next couple of weeks are RO.
<wallyworld_> no, but it must be fixed
<wallyworld_> and i have the code ready to go
<wallyworld_> since i did it for the mockup
<wallyworld_> i'll do it tomorrow
 * StevenK changes the qa-tagger.
<StevenK> "Last revision deployed to QAStaging is 14969. There are 21 revision waiting to be deployed, and no revisions blocked by QA."
<StevenK> wallyworld_, wgrant: How do you feel if I put up a NDT?
<wallyworld_> fine with me
<wallyworld_> all of my changes are invisible on prod anyway
<wgrant> StevenK: Too late
<StevenK> Typical
#launchpad-dev 2013-03-11
<czajkowski> jelmer: you should join #ubuntu-uk
<jelmer> czajkowski: good idea :)
<lifeless> cr3: o/
#launchpad-dev 2013-03-12
<wgrant> StevenK: Does the PCJ count thing look reasonable?
<StevenK> wgrant: Look reasonable in terms of layout, or query count?
<wgrant> StevenK: layout
<StevenK> wgrant: http://wedontsleep.org/~steven/ff.jpg
<wgrant> StevenK: Looks good
<cjwatson> https://code.launchpad.net/~cjwatson/launchpad/testfix-delete-unmodifiable-suite/+merge/152807
<lifeless> flacoste: o/
<StevenK> cjwatson: r=me
<cjwatson> Thanks
<StevenK> wgrant: Can I have a +1 then?
<wgrant> StevenK: That's a reasonable point
<wgrant> (done)
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/person-deactivate-job/+merge/152809
<wgrant> StevenK: Might it make sense to set the account status immediately?
<wgrant> So they're kicked out immediately but the references are purged over the next few minutes
<StevenK> wgrant: I was wondering about that
<StevenK> wgrant: In the view, or IPerson.deactivate() ?
<StevenK> wgrant: Or both?
<wgrant> StevenK: I think both, I guess
<wgrant> Set the status, kill off the preferredemail (and possibly other emails) and anything else reasonably sane, within the request
<StevenK> Oh, so split the work up
<StevenK> Yeah
<wgrant> StevenK: So perhaps kill off deactivate()
<wgrant> StevenK: Split it into two methods
<wgrant> One which does the easy stuff
<wgrant> Have both the view and the deactivation job call the easy one
<wgrant> Just in case, and so that a deactivation job is sufficient if we ever end up needing to deactivate without going through the view
<wgrant> The view just takes a bit of a shortcut to make it appear more instantaneous.
<StevenK> wgrant: http://pastebin.ubuntu.com/5606844/
<wgrant> That's no method name.
<StevenK> I couldn't think of a better one
<StevenK> earlyDeactivate made me sob
<wgrant> It's not a bad method name, it's not a method name at all :)
<wgrant> But pre or early or something might be an improvement
<StevenK> wgrant: IPerson.preDeactivate(), just checking tests
<StevenK> Hmmmm
<StevenK>     bug_task.transitionToAssignee(None)
<StevenK> Unauthorized: (<BugTask for bug 16 on <Product at 0xf49e8d0>>, 'transitionToAssignee', 'launchpad.Edit')
<_mup_> Bug #16: "Swedish" and "Swedish (Sweden)" should be the same language <lp-translations> <Launchpad itself:Fix Released by daf> < https://launchpad.net/bugs/16 >
<wgrant> StevenK: The job should run in PermissiveSecurityPolicy
<wgrant> So that can't happen
<StevenK> wgrant: How do I set that?
<wgrant> It does automatically
<wgrant> Your test may not be doing that, so you might have to log in as an admin
<wgrant> (or use ZopelessLayer)
<StevenK> wgrant: UserCannotEditBugTaskAssignee: Regular users can assign and unassign only themselves and their teams. Only project owners, bug supervisors, drivers and release managers can assign others.
<wgrant> Which test is this, and what did you change?
<StevenK> wgrant: http://pastebin.ubuntu.com/5606881/
<StevenK> TestPersonDeactivateJob.test_deactivate
<wgrant> Oh
<wgrant> So a new test :)
<StevenK> Yeah
<wgrant> What's the traceback now?
<StevenK> I'll be extending the test to add things like branchsubs and things to make sure DB perms are right
<StevenK> wgrant: http://pastebin.ubuntu.com/5606888/
<wgrant> StevenK: Recall that the deactivation job will now probably run with no interaction, where previously it ran as the user being deactivated.
<StevenK> Indeed
<StevenK> wgrant: Should I grab the janitor and log in as that, then?
<wgrant> Possibly
<wgrant> But check what transitionToAssignee checks
<wgrant> You may be able to simply pass in the janitor
<wgrant> I forget.
<StevenK> It grabs the user from the bag
<wgrant> It doesn't have an override?
<StevenK> It does not
<wgrant> See if there is precedent for impersonating jobs, eg. merges
<StevenK> Merges are done via direct DB access
<StevenK> Maybe we should just set the bag user to the person
<wgrant> Possibly, or the entire interaction for the durating
<wgrant> duration
<StevenK> wgrant: I can't really use person_logged_in in the job's run(), can I?
<wgrant> I don't believe so
<wgrant> See, this is why model stuff should never use launchbag
<StevenK> Heh
<wgrant> Still
<wgrant> StevenK: Perhaps there are few enough methods that adding override arguments makes sense
<StevenK> Making use of person_logged_in in job.run() has the test pass, but it makes me sob
<wgrant> And it makes me murderous :)
<StevenK> wgrant: I think that's everything
<StevenK> wgrant: That MP is updated.
<wgrant> StevenK: Thanks
<wgrant> Will look in a sec
 * StevenK stabs the branch scanner
<StevenK> wgrant: I think it's fair to say that sec is up.
<wgrant> StevenK: Sigh, longpoll, sorry
<wgrant> Fixed now
<StevenK> % bzr grep 'All rights reserved' lib/lp | grep -E '\.(py|js):' | wc -l
<StevenK> 98
<StevenK> wgrant: ^
<StevenK> I think I might fix that up tomorrow morning
 * StevenK grumbles at the loss of buildbot bingo
<butlern> i'm trying to get a source package built in my launchpad ppa, the source package requires the source of another package to be installed but it seems there are no deb-src repos available in the chrooted build env, what's the best approach to getting apt-get source <package> to work? or is that not supported?
<maxb> butlern: It's not really supported - it's not a normal thing ever done for official packages
<butlern> maxb: gotcha, i guess i'll bump the varnish maintainers to add a -dev package for header files required by vmod compilation, thanks
<maxb> butlern: That's certainly the normal way to get access to header files.
#launchpad-dev 2013-03-13
<wgrant> StevenK: https://oops.canonical.com/oops.py/?oopsid=OOPS-95fa6dd923c7434f893c96dbc692e5da
<StevenK> It that going to make me very sad?
<wgrant> Possibly :)
<StevenK> wgrant: I was pondering what to grab next, since that checkwatches trace doesn't help much.
<StevenK> wgrant: Reproduced on dev
<wgrant> Great
<wgrant> Should be an easy fix :)
 * wgrant is stabbing rosetta
<StevenK> Which part?
<wgrant> +translate timeouts
<wgrant> At least one variety of them
<StevenK> Braver man than me
<StevenK> I'm just concerned that Rosetta is a hydra
<StevenK> Cut one head off and 30 grow back
<wgrant> I know almost enough about it now to approach it soonish
<wgrant> Where by "approach" I probably mean "spend two days trying to learn how it works"
<StevenK> wgrant: "Badly" and "slowly"
<wgrant> True
<wgrant> But I think a few extra cols on TranslationMessage (and backfilling its 70M rows) should fix a few things
<StevenK> Ah, denorming or some other plan?
<wgrant> Yeah
<wgrant> Mostly from potmsgset to translationmessage
 * StevenK stabs the branch scanner
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/productseries-preload-for-merges/+merge/153052
<wgrant> StevenK: Why'd you replace the _known_viewers set with a list?
<StevenK> Why does it have to be a set, everywhere else just sets it to a list
<wgrant> Nothing uses .add?
<wgrant> eg. bugs certainly used to
 * StevenK reverts that bit
<wgrant> One day we will be on 2.7 and there'll be no excuse to use lists.
<StevenK> wgrant: Is that your only objection?
<StevenK> Not quite a revert, I've removed a spurious set of brackets
<wgrant> StevenK: I'm wondering if you're preloading too late or something, since it apparently ends up accessing preloaded attributes before they're preloaded.
<StevenK> wgrant: Such as?
<wgrant> StevenK: That would only crash if the attribute it was attempting to precache had already been accessed *before* the preloading.
<wgrant> Which is a potential flaw in the preloading, depending on how prevalent it is.
<StevenK> wgrant: It would have been no such attribute if it hadn't been?
<wgrant> StevenK: Right
<StevenK> wgrant: Maybe DecoratedBranch's cleverness is biting us
<StevenK> It has a cachedproperty for the same thing
#launchpad-dev 2013-03-14
<wgrant> StevenK: https://code.launchpad.net/~ubiquity-slideshow/ubiquity-slideshow-ubuntu/html/+merges
<StevenK> Bleh
<StevenK> wgrant: But proposals is there :-(
<StevenK> wgrant: So, I reproduced that with a DecoratedBranch change. Which also broke a large number of other tests. Reverting to what's on the MP makes the tests happy again.
<wgrant> StevenK: Hm, so you can't reproduce the actual problem?
<StevenK> wgrant: Which OOPS, the first, second or both?
<wgrant> StevenK: Today's
<StevenK> wgrant: I haven't tried to reproduce with a clean devel, but I suspect the changes in the MP I have up will sort it out too
<wgrant> Right, quite possibly
<wgrant> StevenK: Did you work out the possible late preloading thing?
<StevenK> No
<StevenK> I was defeated by DecoratedBranch
#launchpad-dev 2013-03-15
<StevenK> wgrant: So, are you nervous enough about my change in https://code.launchpad.net/~stevenk/launchpad/productseries-preload-for-merges/+merge/153052 ? I spent most of yesterday fighting with strange LocationErrors
<wgrant> StevenK: Mmm
<wgrant> It's safe
<wgrant> Was just slightly concerned about the possibly late preloading
<wgrant> But this can't make it worse
<wgrant> r=me
<StevenK> wgrant: Wasn't Curtis talking about patching zconfig for bug 481512?
<_mup_> Bug #481512: race condition when rotating logs <canonical-losa-lp> <lp-foundations> <qa-untestable> <webapp-infrastructure> <Launchpad itself:Triaged> <ZConfig:Fix Released by fdrake> < https://launchpad.net/bugs/481512 >
<wgrant> StevenK: Yes, he was talking to upstream about it
<wgrant> Might have eventually given up
<StevenK> I can recall him mentioning just patching it into ours
<wgrant> Yup
<StevenK> But I don't recall the condition
<wgrant> That's what you do once you give up talking to upstream
<StevenK> wgrant: So, back to our argument about Archive:+delete-packages, you mentioned a change to the query of doom, but I can't recall it
<wgrant> StevenK: A possible optimisation is to find sources where scheduleddeletiondate is null
<wgrant> Since that should include any that have published binaries
<StevenK> wgrant: I was pondering a new column in SPPH that links to published BPPHs, but that may result in you murdering me.
<wgrant> StevenK: How would that be a) stored, and b) kept up to date?
<StevenK> wgrant: a) as int[], b) probably via newPublication, c) I haven't considered how overrides fit in
<wgrant> It's just about impossible to keep that consistent without very big locks
<wgrant> And I like to keep my publishing tables consistent
<StevenK> wgrant: http://pastebin.ubuntu.com/5615641/
<wgrant> StevenK: What's the query?
<StevenK> wgrant: http://pastebin.ubuntu.com/5615648/
<StevenK> My index on DF didn't work :-(
<StevenK>                            ->  Seq Scan on sourcepackagepublishinghistory  (cost=0.00..67337.11 rows=49673 width=135) (actual time=16.419..2005.405 rows=2893 loops=1)
<StevenK>                                  Filter: ((scheduleddeletiondate IS NULL) AND (archive = 32))
<StevenK> CREATE INDEX temp_spph ON sourcepackagepublishinghistory(archive, scheduleddeletiondate); being the index I created
<wgrant> It probably decides that the uncondemned pubs are dense enough that it's not worth it
<wgrant> Though with those counts it seems unlikely...
<StevenK> I wonder if an (scheduleddeletiondate, archive) index will work
<wgrant> won't make a difference
<wgrant> archive WHERE scheduleddeletiondate IS NULL might
<StevenK>                            ->  Seq Scan on sourcepackagepublishinghistory  (cost=0.00..67337.11 rows=49673 width=135) (actual time=16.185..1321.371 rows=2893 loops=1)
<StevenK>                                  Filter: ((scheduleddeletiondate IS NULL) AND (archive = 32))
<wgrant> What's archive 32?
<StevenK> ~ubuntu-langpack/+archive/ppa
<wgrant> Only 126k total pubs
<StevenK> Yes, it's the pathological case
<wgrant> Anyway, the idea of the scheduleddeletiondate bit was to hopefully *replace* the binary check, not augment it
<StevenK> Oh
<StevenK> (SourcePackagePublishingHistory.scheduleddeletiondate IS NULL OR SourcePackagePublishingHistory.status IN (1, 2)) ?
<wgrant> You can probably drop the latter
<wgrant> Since a published or pending pub won't be condemned
<StevenK> wgrant: Sorry, I'm having trouble visualising the query with the published binaries clause killed
<wgrant> StevenK: Just drop the whole AND (EXISTS ... OR status IN (1, 2)) bit
<wgrant> scheduleddeletiondate handles both of those, if things are working correctly
<StevenK> Time: 414.243 ms
<StevenK> I can paste the EXPLAIN ANALYZE for that if you wish
<wgrant> Not particularly. If it's using the index there's probably not much improvement to be made
<StevenK> It's using                                 Recheck Cond: (archive = 32)
<StevenK>                                  Filter: (scheduleddeletiondate IS NULL)
<StevenK>                                  ->  Bitmap Index Scan on securesourcepackagepublishinghistory__archive__status__idx  (cost=0.00..2395.95 rows=128988 width=0) (actual time=32.526..32.526 rows=126534 loops=1)
<StevenK>                                        Index Cond: (archive = 32)
<wgrant> Blah
<wgrant> That doesn't make much sense
<wgrant> It's using an index, so it has no reason not to use the complete one
<StevenK> Neither of the indexes I tried didn't work
<StevenK> wgrant: Do we care for a full index?
<StevenK> postgres seems hell bend on not using one
<wgrant> I'm not sure why it won't use it, but it should be OK
<StevenK> wgrant: http://pastebin.ubuntu.com/5615680/
<StevenK> I'm not sure I care enough to Storm-ify it, but should be much easier now
<wgrant> Very easy
<StevenK> wgrant: Means the prejoin crap dies too
<wgrant> Yes, just use load_related
<StevenK> wgrant: It uses SPR.version for ordering
<wgrant> Yes, but that's separate from the prejoin
<wgrant> You could use DRS to avoid the load_related
<wgrant> Given that you have to select SPN and SPR anyway
<lifeless> stub: o/
<stub> lifeless: yo!
<StevenK> wgrant: Hmm, I think I'm bashing into sampledata stupidity
#launchpad-dev 2014-03-11
<bigjools> folks, is longpoll still enabled only in the lp team?
<wgrant> bigjools: It is.
<wgrant> It's on the list of things to fix as part of improving MPs
<bigjools> wgrant: what do you think about making it a separate open team so you can opt-in?
<bigjools> what do you need to fix out of interest?
<wgrant> It's a bit more practical to fix properly now that browsers suck a bit less
<wgrant> bigjools: Browser connection limits.
<wgrant> Open a few MPs and all your LP requests will hang.
<bigjools> yeah, hence opt-in :)
<wgrant> bigjools: https://launchpad.net/~launchpad-longpoll-testers will work once we have webop coverage
<bigjools> rock n roll
<bigjools> you could put the old team in it by default
<bigjools> but nice description btw
<wgrant> adeuring: Could you please transfer https://launchpad.net/~hwdb-team/+reassign to ~launchpad-leader?
<adeuring> wgrant: sure
<wgrant> adeuring: Thanks
<wgrant> bac: Should https://launchpad.net/~commercial-approvers still exist?
<bac> wgrant: probably not
<bac> whack it
<wgrant> bac: Done. Thanks.
#launchpad-dev 2015-03-09
<blr> wgrant: afaict, the GitHostingClient is only called by the XmlRpcApi currently?
<wgrant> blr: Yep, you can currently only create a repository by trying to push it.
<wgrant> turnip makes the translatePath XML-RPC call. LP notices that it doesn't exist and creates it, calling back into turnip (while being called from turnip) to create the repository on disk before returning its path.
<blr> wgrant: am I right in thinking GitRepository.destroySelf() only needs to call turnip from the api client?
<blr> wgrant: are we preserving the model in the db, but setting a deleted flag?
<wgrant> blr: At the moment we'll delete the LP database row too.
<wgrant> blr: There's no reason to do complicated deferred deletion just yet.
<wgrant> (LP has a history of not actually deleting things, which causes no end of trouble. We should design for proper deletion from the start here.)
<blr> wgrant: ok, thanks
<blr> wgrant: class StupidCache()
<blr> also enjoyed the lolcat interfaces.
<wgrant> Yup, StupidCache is the cache that remembers everything it's ever seen.
<blr> cjwatson: thanks for the comments re UTF-8 filtering, agree that it makes sense to handle it there.
<blr> cjwatson: the merge conflict was from providing a dependant branch on the MP, it doesn't exist in my local branch - how can I avoid that in the future?
<blr> cjwatson: also noted the style for decorators in LP appears to be func_name_decorator, will amend that as well.
<blr> cjwatson: wgrant: meeting today or tomorrow? (tomorrow in calendar, but we seem to have been shifting back a day recently)
<cjwatson> blr: you know about the prerequisite branch facility in merge proposals?
<cjwatson> Oh, wait, you are using that.  So then I don't understand why that requires you to have conflicts
<cjwatson> blr: I doubt it has much to do with the dependent branch; it probably just means you need to merge lp:turnip into your branch, resolve conflicts, and push
<blr> cjwatson: ok, that makes sense.
<wgrant> Morning.
<wgrant> cjwatson: Pre-reqs can in some circumstances confuse bzr merge into generating more conflicts, but I've only seen it rarelyt.
<cjwatson> wgrant: Not on the underlay branch, surely?
<wgrant> Ah, no.
<wgrant> Misread that.
<blr> cjwatson: would json.dumps(foo, encoding='utf8') be better than ensure_ascii=False?
<cjwatson> encoding="utf-8" is the default
<blr> ah so it is
<wgrant> Anyone encoding JSON as anything else should be shot :)
<cjwatson> The default escaping is just so that it works conveniently over file objects open in binary mode; I would *hope* cornice can cope with unicode, just haven't checked
<wgrant> Hum
<blr> cjwatson: yes it appears to
<wgrant> encoding='utf8' isn't the default, is it?
<wgrant> ensure_ascii=False leaves it as a unicode.
<wgrant> (but cornice should know to encode that as UTF-8)
<cjwatson> s'what https://docs.python.org/2/library/json says
<cjwatson> probably encoding only takes effect if ensure_ascii=True
<wgrant>     If ``ensure_ascii`` is false, all non-ASCII characters are not escaped, and
<wgrant>     the return value may be a ``unicode`` instance. See ``dump`` for details.
<wgrant> If ensure_ascii is true, they're escaped.
<wgrant> Not encoded.
<wgrant> I wonder if encoding= is only useful for charsets that don't match ASCII.
<wgrant> I think encoding is meaningless with ensure_ascii=False, because it doesn't encode at all.
<cjwatson> from a quick look at the code, I think it's actually used for decoding str objects you pass *in*
<wgrant> Ah, that might make sense.
<wgrant> It might also be relevant if eg. you use ensure_ascii=True and want EBCDIC output.
<cjwatson> and if it's not 'utf-8' then it interposes a decoder
<wgrant> No, that still doesn't work.
<cjwatson> so encoding='utf8' is not actually quite the same as encoding='utf-8' here, potentially ... .encode and .decode have always hurt my brain
<cjwatson> I'd leave encoding= well alone :)
<cjwatson> I think if you want differently-encoded output you have to set the encoding on the file object and use json.dump(ensure_ascii=False), or use json.dumps(ensure_ascii=False).encode()
<blr> tests pass with ensure_ascii=False at any rate
<wgrant> Yep, ensure_ascii=False is usually the write way to go for web stuff, since HTTP can handle the charset itself.
<cjwatson> That's probably only meaningful if you've specifically added tests with non-ASCII data.
<cjwatson> At least in Python 2 where you get the "helpful" default of unicode->str working magically until something outside ASCII appears.
<blr> so presumably I need a test with some latin1 data, or something of the sort
<cjwatson> Right, I'd add one with ISO-8859-1 or something that's not valid UTF-8, and one with valid UTF-8.
<cjwatson> And make sure that neither crashes, but that the former ref is silently omitted (so maybe have another valid UTF-8 ref in there as well or something)
<blr> cjwatson: with the ref collection I can silently omit refs unable to encode, but I think we'll also want a validator on get() which returns a 400 perhaps?
<wgrant> Are Pyramid's paths bytestrings or Unicode strings?
<wgrant> LP's are pretty much treated as Unicode, so the ref wouldn't make it through the traversal logic.
<wgrant> turnip-api should probably 404 if it fails to decode a path as UTF-8.
<blr> wgrant: they're unicode
<cjwatson> I think I would prefer 400 ("could not be understood by the server due to malformed syntax"), but either works.
<cjwatson> Hopefully Pyramid already handles it ...
<wgrant> Some frameworks do weird things, so we should probably check that.
<blr> cjwatson: a test should confirm either way
<blr> it would be nice not to need an explicit validator there.
<cjwatson> http://docs.pylonsproject.org/projects/pyramid/en/1.5-branch/narr/views.html#handling-form-submissions-in-view-callables-unicode-and-character-set-issues has some stuff about this
<cjwatson> But certainly we should check
<cjwatson> Hm, that link is about form submissions rather than things like the path, though
<blr> probably still applicable
<cjwatson> maaaaybe
<blr> heh
<cjwatson> http://docs.pylonsproject.org/projects/pyramid/en/1.5-branch/narr/urldispatch.html "Note that values representing matched path segments will be url-unquoted and decoded from UTF-8 into Unicode within the matchdict."
<wgrant> Yay, so it's sane.
<wgrant> But I wonder what it does on failure.
<blr> yes self.request.matchdict['ref'] is unicode
<cjwatson> Yeah, can't immediately find that in the docs
<wgrant> It's pretty modern and Zopey, so it's unlikely to be completely moronic.
<wgrant> But it's not unheardof for frameworks to return a str if things don't decode properly...
<cjwatson> Looks like it raises URLDecodeError, and you can register an exception view for that if you want custom behaviour.  Don't know what the default is.
<wgrant> Right, it probably does something sensible then.
<wgrant> cjwatson: So meeting in 24 hours?
<cjwatson> And a bit, yeah.
#launchpad-dev 2015-03-10
<cjwatson> Hm, the oops linked from https://bugs.launchpad.net/launchpad/+bug/1429358 isn't making sense to me; that stuff *is* preloaded.  I wonder if we're running up against the Storm cache size?
<mup> Bug #1429358: Bugs with large numbers of linked branches time out <bug-branch-links> <oops> <timeout> <Launchpad itself:Triaged> <https://launchpad.net/bugs/1429358>
<wgrant> cjwatson: That's not implausible. And even if we weren't we'd hit serious CPU time rendering all those branches.
<wgrant> cjwatson: btw, that's possibly a dupe of bug #724025
<mup> Bug #724025: BugTask:+index timeout due to high cpu time rendering many bug tasks in bug 230350 <qa-ok> <timeout> <Launchpad itself:Triaged> <https://launchpad.net/bugs/724025>
<wgrant> May be worth closing 724025 in favour of it.
<wgrant> cjwatson: I think all we can reasonably do it truncate the list of branches.
<wgrant> It's not as if a page that listed 700 branches would be particularly usable anyway.
<cjwatson> is it in fact that many?
<cjwatson> launchpad_dogfood=# select count(*) from bugbranch where bug=230350;
<cjwatson>  count
<cjwatson> -------
<cjwatson>     84
<cjwatson> (1 row)
<wgrant> Oh, now that's a bit odd...
<wgrant> cjwatson: Can you test with a larger cache size on DF?
<wgrant> Ah ahhahahahujfewuifweuohjiwfeuihjo
<wgrant> cjwatson: It's the stacking security checker.
<wgrant> With the pathological stacking behaviour of branch-distro.py, which creates a chain a dozen branches long for these.
<wgrant> cjwatson: Feel like writing a recursive CTE?
<cjwatson> Yeah, bumping the storm cache size to 100000 on DF doesn't appear to change the query count.
<cjwatson> Possibly I should finish my ref scanner first :-)
<wgrant> The tracebacks confirm my diagnosis:
<wgrant>   File "/srv/launchpad.net/production/launchpad-rev-17374/lib/lp/code/model/branch.py", line 1408, in visibleByUser
<wgrant>     user, checked_branches)
<wgrant>   File "/srv/launchpad.net/production/launchpad-rev-17374/lib/lp/code/model/branch.py", line 1408, in visibleByUser
<wgrant>     user, checked_branches)
<cjwatson> I've been experimenting with a rhythm of misc maintenance in the morning, git development in the afternoon
<wgrant>   File "/srv/launchpad.net/production/launchpad-rev-17374/lib/lp/code/model/branch.py", line 1408, in visibleByUser
<wgrant>     user, checked_branches)
<wgrant> etc
<wgrant> Oh yeah, it's your afternoon. I should probably sleep.
<cjwatson> But don't want to get distracted by inflating morning tasks
<wgrant> Indeed.
<cjwatson> See you at the meeting then.  I should be on time - we had to cancel our guest-for-dinner due to Kirsten being ill
<wgrant> Ugh, that's unfortunate. See you then.
<wgrant> cjwatson: btw, https://code.launchpad.net/~wgrant/launchpad/sca-api/+merge/252420 fixes one of the OLS issues, if you could look at that today.
<blr> cjwatson: when running the tests, hosting_path appears to be an integer?
<blr> feel like I'm missing something fundamental that may require more coffee.
<cjwatson> blr: It's a stringified integer
<cjwatson> We'll need something more sophisticated at some point, this was a "make it work" measure, we just stringify the id to get the hosting path
<cjwatson>     def getInternalPath(self):
<cjwatson>         """See `IGitRepository`."""
<cjwatson>         # This may need to change later to improve support for sharding.
<cjwatson>         return str(self.id)
<cjwatson> (comment is an understatement)
<cjwatson> wgrant: argh I missed your comment from earlier
 * cjwatson looks
<blr> cjwatson: I see, thanks
<cjwatson> wgrant: r=me, just minor things that looked a bit odd
<wgrant> cjwatson: Thanks.
#launchpad-dev 2015-03-11
<cjwatson> wgrant: Ugh, doing a proper whole-request timeout with requests manages to be even more of a nightmare than it is with urllib2.
<wgrant> cjwatson: ... srsly?
<cjwatson> The sock.shutdown dance is still needed, AFAICS, and it's incredibly hard to pick through the layers to get at the actual socket.
<wgrant> How is this not something that everyone wants :(
<cjwatson> The response object isn't initialised until after the first recv, so a naÃ¯ve attempt hangs in lp.services.tests.test_timeout.TestTimeout.test_urlfetch_timeout_after_listen.
<cjwatson> See https://github.com/kennethreitz/requests/issues/239#issuecomment-22233420 and cry
<cjwatson> I'm seriously tempted to switch it over to using a subprocess rather than a thread so that we can just abruptly kill the thing.
<wgrant> Heh :/
<wgrant> requests is meant to be the sane thing to use :(
<wgrant> I began to doubt that while trying to work out how to, you know, close its connections.
<wgrant> And failing.
<wgrant> Oh
<wgrant> That very issue, in fact :/
<cjwatson> The only other decent idea I have so far is to pick through the urllib3 connection pool and forcibly close everything in it.
<wgrant> I think I tried that but gave up.
<cjwatson> But I think we have to fix this before using GitHostingClient much more ...
<wgrant> That may have just been because I was immensely frustrated with the ongoing production incident, rather than because it was fundamentally too difficult.
<cjwatson> Maybe if I subclass stuff all the way down.
<cjwatson> Subclass HTTPConnectionPool etc. so that I can customise its actual pool and add code to clean it up properly; subclass PoolManager to use my modified HTTPConnectionPool; subclass requests.adapters.HTTPAdapter to use my modified PoolManager
<cjwatson> It might actually be possible with only the advertised API.
<cjwatson> But seriously.
<wgrant> Yup.
<wgrant> At least we don't have an ongoing production outage because of it this time!
<cjwatson> Was that with swift?
<cjwatson> Oh god I'm going to have to subclass Queue.LifoQueue somehow.
<wgrant> Right, the librarian EMFILE incident with Swift connections hanging around.
<cjwatson> OMG I think this foul contraption might work.
<wgrant> I don't believe you.
<cjwatson> wgrant: https://code.launchpad.net/~cjwatson/launchpad/timeout-with-requests/+merge/252570 - I think it's still a net improvement to be able to use other parts of requests, but ugh
<cjwatson> Reassuring that the tests caught my early insufficient attempts, at least.
#launchpad-dev 2015-03-12
<wgrant> cjwatson: O
<wgrant> Bah
<wgrant> cjwatson: https://code.launchpad.net/~wgrant/turnip/no-buildout/+merge/252692
<wgrant> I've dropped buildout and hopefully made the non-buildout packaging a bit easier to use.
<wgrant> setup_requires is approximately the worst thing in the world (it'll make setuptools talk to PyPI behind pip's back, even if you've disabled PyPI in pip), and the only way to work around that seems to be a ~/.pydistutils.cfg snippet.
<wgrant> (we have a couple of our OpenStacky deps in LP patched to remove their setup_requires=['pbr'] for that reason)
<wgrant> I swore not to get distracted by juju today.
<wgrant> And instead got distracted by Python packaging.
<wgrant> And didn't get anything actually finished.
<blr> wgrant: hah, I'm not sure which of the two is more painful :)
<cjwatson> wgrant: Have you tested that this works properly when doing "make deploy" in the charm from vivid?  I'd been somewhat careful about not adding pygit2 to requirements.txt to avoid problems there.
<blr> cjwatson: should be ok, libgit2-21 is in vivid?
<cjwatson> Well, consider if vivid upgrades to a newer version; my point isn't so much what vivid happens to have today, it's being independent of the deployment system
<cjwatson> If w has libgit2-27, I don't want to end up in a dilemma about what version of pygit2 we can use because some of us are on vivid and some on w, say
<blr> cjwatson: perhaps we should just be following trunk
<cjwatson> That isn't my point
<blr> is there a workflow for automating packaging from github?
<cjwatson> My point is to make sure that things work regardless of what libgit2, or none, is on the system you're deploying from
<cjwatson> The strategy I had in place was to install python-pygit2 from the LP PPA in the unit, and virtualenv --system-site-packages to make use of it, so that we can control that precisely without having to deal with pip
<cjwatson> I want to check that adding it to requirements.txt doesn't disturb that
<cjwatson> (It may not, because wgrant also added version bounds that I don't think I tried)
<blr> always a little reluctant to use --system-site-packages, but that does sound sensible.
<cjwatson> I think it's less of a problem with immutable or mostly immutable units
<cjwatson> Then again I come from a distro background, so ...
<blr> no, I think that's fair.
<cjwatson> Anyhow, if wgrant has finished up then I'll try the charm in a bit
<blr> just as buildout was starting to grow on me... :)
<blr> assuming there isn't anything too braindead in commit/log api branch, I'll spend tomorrow on deployment/mojo
<blr> right, bedtime. night cjwatson
<cjwatson> Night!
<wgrant> cjwatson: Yeah, the charm was the bit I wasn't completely sure about.
<wgrant> I suspect that if we want to use requirements.txt with a Debian pygit2 we may end up in trouble, unless we match the dep versions exactly (which can be seen in my diff).
<wgrant> I'm not sure which is more ugly.
<wgrant> system-site-packages, or being tied to a particular libgit2
<wgrant> Though in practice we're tied to a particular libgit2 anyway, since both it and pygit2 break API every release...
<cjwatson> Right, well, the basic problem is that we can't install libgit2 from pip, so that's got to come from the system, and thus we have to go to special care to match it anyway
<cjwatson> Let's see
<cjwatson> wgrant: Nope
<cjwatson> wgrant: http://paste.ubuntu.com/10584585/
<cjwatson> I deliberately don't have libgit2-dev installed here to make sure we don't have hidden dependencies on it
<cjwatson> wgrant: Try pulling cffi, pycparser, pygit2 back out of requirements.txt?
<cjwatson> I see https://twistedmatrix.com/trac/ticket/7795 is making progress, too ...
 * cjwatson tries that
<cjwatson> wgrant: http://paste.ubuntu.com/10584612/ makes the charm work again.
<cjwatson> wgrant: Perhaps a compromise would be moving that to pygit2-requirements.txt, so that you can still use pip for that for local development if you want to deal with the libgit2 issues?
<cjwatson> blr: I think you could drop ['commit_time'] from format_commit in your commit-api branch.  It exactly duplicates ['committer']['time'], and even though pygit2 has both there's no reason for us to.
<wgrant> cjwatson: Sorry, distracted by real life. Let me see.
<wgrant> Splitting those out into a separate requirements file might be sensible, indeed.
<wgrant> cjwatson: The charm didn't whine about anything else?
<wgrant> I'll test it properly before I merge, but need to set up a less tainted Juju VM.
<wgrant> cjwatson: You're not unhappy with a --system-site-packages virtualenv for dev locally?
<cjwatson> wgrant: No other whines.  I'm fine with that kind of virtualenv for dev locally; the way I do it is to use a trusty lxc container that has the LP PPA enabled.
<cjwatson> wgrant: Does https://code.launchpad.net/~cjwatson/launchpad/db-git-more/+merge/252672 look vaguely plausible?
<cjwatson> Or did you want to see the matching code branch first?
<wgrant> cjwatson: I wouldn't mind seeing the code.
<cjwatson> Right, let me finish off the remaining possible tests today.
<cjwatson> Oh, URL scheme for exported refs, that was the other thing I had to work out.
<cjwatson> I think I'll just drop the exports and deal with that in a later branch.
<wgrant> cjwatson: Yeah, variable-length paths ftl.
<wgrant> cjwatson: Oh wow, just notced the Twisted ticket open in my browser. That is good news.
<wgrant> Huh
<wgrant> The turnip charm doesn't actually explicitly install git
<wgrant> It was always there anyway when deploying from utopic and vivid to trusty, but not trusty to trusty...
<cjwatson> wgrant: We'll have to work out some kind of URL scheme for refs, otherwise we can't usefully show summaries of branches.
<cjwatson> wgrant: Strawman: filter to refs/heads/ only, and URL-encode the rest.
<cjwatson> (We may need to show tags at some point, but that should be separate anyway.)
<blr> cjwatson: thanks colin, yes commit_time is redundant, you're right.
<blr> cjwatson: was there anything else that obviously needed addressing?
<cjwatson> blr: I didn't quite get as far as actually running anything against it today, so I was just going on inspection.  Didn't see anything else.
<blr> cjwatson: ok, thanks.
<cjwatson> Hopefully tomorrow, since I'm about to need to fill in the committer information in order to be able to progress with UI.
<cjwatson> But my git-ref-scanner branch was big enough, and I wanted to start with something that only required today's lp:turnip...
<wgrant> cjwatson: Well, traversing refs properly in a Zopey world isn't actually a problem.
<wgrant> cjwatson: You can just have a traverser that consumes segments until it matches or can't match a ref. You don't even need a marker to indicate that traversal should stop, because a ref cannot be a prefix of another, as they're represented (ignoring packed-refs) as a directory tree.
<wgrant> blr: Is your commit-api branch ready for review?
<cjwatson> wgrant: oh, I thought you were vetoing variable-segment-length paths on principle
<wgrant> cjwatson: Oh, no, that would be ugly and there's no need.
<wgrant> If we were using stupid regex-based path matches we'd be in trouble, but luckily Zope is sane in respects like this.
<cjwatson> github seems to strip off refs/heads/
<wgrant> cjwatson: I probed around to work out their rules at some point, let me see if I can find that piece of paper.
<cjwatson> but they do permit multiple segments after that
<wgrant> You can specify the full path, but they try stripping off refs/heads and refs/tags in that order.
<wgrant> Which makes sense.
<blr> wgrant: it is yep
<wgrant> blr: Hm, I see that pygit2 decodes the message for us. Do you know how it does that?
<wgrant> It's probably fine, but best to check.
<blr> wgrant: didn't see anything explicit in the documentation, will have to look at the source.
<wgrant> blr: libgit2 uses the encoding field in the commit message, but I wonder what it does on invalid UTF-8/
<blr> wgrant: perhaps need another latin-1 test for the commit body?
<wgrant> blr: Indeed. Git commit messages are text, and they have an encoding field, so latin-1 can be coped with. But we should ensure we don't crash on cases where the message doesn't decode, as that's technically possible.
<wgrant> Hm
<wgrant> I'm making a whole lot of suggestions here that make a lot of sense if 0.22 changed the API..
<wgrant> Hm, no, these properties seem to exist in 0.21
<blr> tangential, but looking at the changelog, it might make sense to upgrade to 0.22 before working on clone.
<wgrant> Indeed, it makes sense to track upstream as close as reasonably possible.
#launchpad-dev 2015-03-13
<blr> wgrant: thanks, some good things to consider there.
<wgrant> blr: Have you looked at mojo again?
<blr> yep looking at mojo/deployment today.
<blr> will come back to the commit-api branch later, would like to continue with this today I think.
<wgrant> blr: Would be good to get commit-api landed today, since Colin's up to there now.
<blr> wgrant: sure, that's fine, I'll come back to deployment on monday.
<wgrant> blr: We all know how good deployment/packaging/etc. are at eating an inordinate amount of time :/
<blr> oh yes -_-
<blr> particularly when the tool is... temperamental.
<wgrant> Quite.
<blr> wgrant: regarding the assertion in format_commit(), the only caller does currently check for GIT_OBJ_COMMIT, but I suppose we might re-use this elsewhere
<blr> perhaps I should move the exception in get_commit to format_commit?
<blr> would save repetition in get_log
<wgrant> blr: That's not unreasonable, as long as the request returns 400 or something like that.
<blr> wgrant: if I pass an iterator to the view, rather than the commit collection, I'm not certain about having the limit logic in the view.
<blr> feels like it should be in get_log
<blr> would prefer the view not to touch pygit2 if possible.
<blr> I realise it is already leaking with the exceptions, but in general would prefer to minimise it.
<wgrant> blr: I think this makes things uglier, but I'm happy to leave it as and see how things grow naturally.
<blr> wgrant: some weird behaviour with pygit2 unicode Signatures
<blr> wgrant: sig = pygit2.Signature(u'ÐÐ»Ð°Ð´Ð¸Ð¼Ð¸Ñ ÐÐ»Ð°Ð´Ð¸Ð¼Ð¸ÑÐ¾Ð²Ð¸Ñ ÐÐ°Ð±Ð¾ÐºÐ¾Ð²'.encode('utf-8'), u'ÐÐ°Ð±Ð¾ÐºÐ¾@zembl
<blr> a.ru'.encode('utf-8'))
<blr> ugh
<blr> please reassemble, and call sig.name
<wgrant> Is pygit2.Signature meant to be given bytestrings?
<wgrant> It works if you give it encoding='utf-8'
<blr> that's not in the docs O.o
<wgrant> I'd check the code, but I suspect you're meant to give it one of a unicode, an ASCII str, or a str plus an encoding.
<wgrant> I didn't look at the docs, I just tried it :P
<blr> relying on documentation is clearly a major character flaw of mine. I'll work on it.
<blr> wgrant: huh, I can't reproduce that, passing encoding='utf-8' to the constructor throw a UnicodeEncodeError
<wgrant> blr: http://paste.ubuntu.com/10588625/
<blr> sig = pygit2.Signature(u'ÐÐ»Ð°Ð´Ð¸Ð¼Ð¸Ñ ÐÐ»Ð°Ð´Ð¸Ð¼Ð¸ÑÐ¾Ð²Ð¸Ñ ÐÐ°Ð±Ð¾ÐºÐ¾Ð²'.encode('utf-8'), u'ÐÐ°Ð±Ð¾ÐºÐ¾@zembl
 * blr sighs
<blr> wgrant: that does work, sorry.
<blr> why would ascii be the default?
<wgrant> blr: ASCII is the only correct default for interpreting a bytestring without any other information.
<wgrant> (but, for our purposes, we can sensibly say that if there's no declared encoding we default to UTF-8 with errors='replace')
<wgrant> Because this API is allowed to be lossy, and anyone who doesn't use UTF-8 is a fool.
<blr> wgrant: do commit objects with non utf-8 data need to be filtered, akin to refs?
<wgrant> blr: No, in that case we can just do errors='replace'.
<wgrant> We can't do that with refs, since they're identifiers and would be ambiguous.
<wgrant> For metadata it's fine to be lossy.
<blr> ok, that makes sense.
<lifeless> you could do unicode_escape
<lifeless> for errors
<lifeless> IIRC
<blr> wgrant: afaict, pygit2 is already lossy, creating a commit with a latin1 bytestring returns a utf-8 'replaced' string
<blr> or more specifically, I think create_blob() is doing that
<wgrant> lifeless: unicode_escape is the opposite direction.
<wgrant> blr: create blob should take a bytestring, though, since its content isn't text
<blr> hmm nope
<blr> yep, just looking at the code now trying to work out where this is happening
<wgrant> the API is a bit odd, vut it seems to be sensible in terms of str vs unicode
<blr> wgrant: yes it appears to be handling non utf-8 sanely. There's a test now at any rate.
<blr> wgrant: there now if you have a moment to re-review.
<blr> just looking at cjwatson's nonexistent-repo branch, I guess that was a bad assumption that it would throw a GitError.
<blr> or rather, the bad thing is making an untested assumption. :/
<wgrant> Right.
<wgrant> KeyError is a pretty odd error, though!
<blr> thanks wgrant
<blr> hope that will free up cjwatson for his next bit of work. Will return to stackystack.
 * wgrant is making Answers a bit easier to manage from a spam perspective.
<wgrant> But mojo might be a nice distraction if you continue to run into walls.
<blr> would like to get to some of those other LP tasks next week as well, perhaps the keybindings for chunk navigation
<wgrant> Indeed.
<wgrant> That should be even nicer than Ctrl+Enter.
<blr> wgrant: missed your inline comment, damnit. limit is a string there, the view should cast it, rather than doing it there though.
<wgrant> Yup, another usability issue with inline comments :)
<StevenK> Needs more VWS? Or <blink> tags?
<blr> StevenK: bring back <marquee> imo
<StevenK> blr: Doable with JS, and oh look, inline comments already uses lots of it
<blr> ok, going to head off. have a good weekend wgrant
<wgrant> blr: Night. Thanks for sorting out commit-api.
<blr> and thanks for your help with it! ttyl
<cjwatson> blr: I marked the commit/log API tasks done in asana - hopefully that's right
#launchpad-dev 2015-03-15
<wgrant> blr: Morning.
<blr> morning wgrant, good weekend?
<wgrant> blr: Not too bad, indeed. You?
<blr> good, had our monthly bluegrass jam on which is always enjoyable.
<wgrant> Excellent.
<wgrant> blr: Mojo today?
<blr> wgrant: yep!
<blr> hope to have it beaten into submission by EOD
<wgrant> blr: Great, do keep me updated.
<wgrant> blr, cjwatson: I've just moved the post-phase 1 tasks into a separate Asana project, to see if the project works better as a burndown list now.
<blr> wgrant: that seems sensible - the chart will certainly be more meaningful now.
<lifeless> asana?
<wgrant> lifeless: Task tracking system thing. https://asana.com/product
<wgrant> We inherited it from CI, but it's actually not that bad.
<lifeless> vs LP ?
<wgrant> LP's not designed for that, except blueprint workitems, and blueprint workitems are evil.
<lifeless> ah that granularity
#launchpad-dev 2016-03-15
<cjwatson> wgrant: So, I spent most of today rethinking my plans for by-hash and sketching an implementation.
<cjwatson> As you may remember, my initial plan was to do a quick-and-dirty thing with links in the filesystem, but keeping track of superseded dates exactly correctly was getting tricky, and I eventually decided that if I was going to do dodgy database-in-filesystem things then maybe I should just use that database I hear we have.
<cjwatson> It then occurred to me that with a very small generalisation the same table could also be useful to fill in the remaining bits that the DB doesn't yet know about in preparation for diskless archives.
<cjwatson> So my plan is to have ArchiveFile (archive NOT NULL, distro_series, pocket, path NOT NULL, libraryfile NOT NULL, scheduled_deletion_date).  Files that are indexed by a Release file get distro_series and pocket set; when they cease being the current version they get scheduled_deletion_date set to now + a stay of execution; by-hash is synced with the contents of this table for the relevant (archive, distro_series, pocket).
<cjwatson> In possibly the nearish future, we can also overhaul all the other stuff that directly writes misc files under the archive root to also stuff it into the librarian and add ArchiveFile rows (with distro_series and pocket NULL, since in those cases the files aren't indexed and don't need by-hash tracking).
<cjwatson> Then the main remaining bit of diskless archives would be a caching proxy that proxies pool to obvious stuff, dists/**/by-hash to (ArchiveFile, LFA, LFC) lookups, everything else to ArchiveFile lookups.
<cjwatson> Do you see any issues with this plan?
<cjwatson> The part of it required for by-hash is fairly straightforward, but I find it appealing that it's a stepping stone to better PPA publishing rather than a bodge that we'll have to clean up later.
<wgrant> cjwatson: That's where I was hoping this would lead.
<wgrant> But it requires that we be somewhat more thoughtful about the schema.
<wgrant> Custom uploads, for example, should be possible using this means.
<cjwatson> By all means.  I think they are with this; what have I missed?
<cjwatson> ("path" here is a relative path from the archive root.)
<wgrant> cjwatson: They certainly fit into that schema, but there are probably other things that are useful.
<wgrant> If only a text column that describes who owns the file.
<cjwatson> Is there a meaningful notion of ownership here other than the archive owner?
<cjwatson> Or am I being too literal?
<wgrant> For example, something that gardens old d-i uploads could find all the weird files whose owner is custom:debian-installer:$VERSION, then work out the versions that should be alive and prune the rest.
<wgrant> owner in terms of component that manages them.
<cjwatson> Ah, I see.
<wgrant> Rather than the weird path-based stuff we have now.
<wgrant> Which is, err, error-prone.
<wgrant> I avoided mentioning the obvious diskless archive relevance to not tempt you down a rabbit hole unless you were already rather tempted, but here we are :P
<cjwatson> I suppose then the "owner" of an AF that represents something indexed in a Release file might be the suite name.  (No particularly obvious version; we just need to keep everything superseded less than stay-of-execution ago, and anything that's active.)
<cjwatson> We don't especially have to have the DS FK.
<wgrant> Indeed
<wgrant> I have long eanted to remove DDes from the publisher
<wgrant> DSes
<cjwatson> In my case this was not so much rabbit hole as "oh my god I am fed up juggling links".
<cjwatson> But it looks as though it requires at least a bit of checking what the path through the hole looks like.
<wgrant> Indeed
<wgrant> this is, however, a tractable rabbithole
<wgrant> unlike caveats :)
<cjwatson> Er quite
<cjwatson> I do plan to get back to that, but by-hash has a harder time limit and is less soul-destroying
<wgrant> in better news, i almost have HTTP mirroring working
<wgrant> bit more complicated in this architecture than the other variants
<cjwatson> Does that imply you have it working in the simpler protocols?
<wgrant> working enough to proceed
<cjwatson> Nice, I didn't realise you were that far along.
<cjwatson> Which of the two paths?
<wgrant> basically a new turnip-fetch service alongside receive-pack and upload-pack. You send the ferch command in your request to turnip, then plug the connection into the remote upload-pack however you desire
<wgrant> no egress from turnip required, and yoy can handle all the nasty auth bits on your vridging client thing
<cjwatson> Perfect, assuming the bridging client is a roughly known quantity.
<cjwatson> OK, so I think the above requires me to go through custom uploads and sketch out how they could work against AF with distro_series, pocket -> some-better-name-than-owner.  Other than that I think the changes would not need to be extensive.  The only other oddities are Release etc. themselves (trivial), project/ (ditto), and indices/ (maybe want to be loosely associated with a suite so we can ditch old indices more easily, and we ...
<cjwatson> ... should probably start handling them in the publisher rather than in u-a-p, but otherwise do not present difficulties).
<wgrant> the bridging client just needs to establish the connection to both ends and cross the streams
<cjwatson> My sketch impl needs a couple of small changes for that schema adjustment but not much.  The serious work I still need to do is making it not be a eye-bleeding megamethod.
<cjwatson> *an
<wgrant> which in all cases except HTTP (and SSH due to auth) is trivial
<wgrant> HTTP is complicated by stateless-rpc being undocumented
<wgrant> but luckily there is code and tcpdump
<wgrant> cjwatson: indices/ is handled by a thing on snakefruit replacing them over the API.
<wgrant> Pruning old series is simple because the not-owner field includes series details.
<cjwatson> The publisher already knows about having them in overrideroot and u-a-p just copies them; I think it makes more sense to do that directly in the publisher, although it's true that it could be done either way.
<wgrant> Ah, true.
<cjwatson> wgrant: Speaking of publisher hacks, could you have a look at https://code.launchpad.net/~cjwatson/launchpad/dep11-mtime/+merge/288757 ?
<wgrant> cjwatson: Yep, on my list. Was off yesterday, and dealing with gpg/git this morning.
 * cjwatson nods
#launchpad-dev 2017-03-13
<stub> When an SSO account is created, do we also always get a placeholder LP account with a linked emailaddress?
<wgrant> stub: No. If an SSO account sets a username, we get a PLACEHOLDER Person and Account with OpenIDIdentifier but no EmailAddress.
<wgrant> That is the only situation in which a PLACEHOLDER Account can exist.
<stub> ta.
#launchpad-dev 2017-03-15
<stub> The SSO side of bot creation has landed, and we need a review of https://code.launchpad.net/~stub/launchpad/botbake/+merge/319445 for the LP side to catch up.
#launchpad-dev 2017-03-19
<NikoKrause> Hello?
<NikoKrause> It's me
<NikoKrause> I was wondering
#launchpad-dev 2018-03-12
<cjwatson> lifeless: hey - while working on https://code.launchpad.net/~cjwatson/python-timeline/py3/+merge/341301, I noticed that the trunk branch on LP is behind the version on PyPI.  Is it possible that you have a local branch with https://code.launchpad.net/~lifeless/python-timeline/bug-891989/+merge/127951 merged that you forgot to push?
#launchpad-dev 2018-03-13
<lifeless> cjwatson: thats probably three laptops ago
<lifeless> cjwatson: so  Â¯\_(ã)_/Â¯
<lifeless> cjwatson: I'd drop pypi over a repo, diff and commit the differences and shake your fist at me
#launchpad-dev 2018-03-14
<cjwatson> lifeless: I'm always good with shaking fists at you.  Thanks, will do (though not before this dental appointment ...)
#launchpad-dev 2018-03-16
<cjwatson> Huh, nobody appears to have noticed that the descriptions of target_git_{repository,path} and prerequisite_git_{repository,path} on GitRef:+register-merge are identical
<cjwatson> (including me until just now)
#launchpad-dev 2020-03-09
<tomwardill> https://code.launchpad.net/~twom/launchpad/+git/launchpad/+merge/380416 allow 39A to pass it's self signed cert to requests to the test open id server, based on the instance/configuration name.
<tomwardill> and with that, 39A can stand up a working (to the point of login) LP instance.
<SpecialK|Canon> tomwardill: nice :)
<tomwardill> turnip next
<tomwardill> but first, buildd + LXD VM documentation
<SpecialK|Canon> <3 <3 <3 documentation <3 <3 <3
<SpecialK|Canon> (Hello, I have Feels)
<tomwardill> it's mostly 'how much of my existing configuration can I remove because the LXD team have made it better'
<SpecialK|Canon> heartheart deleting code etc.
<leni> Hi, do you have an idea of when would the getAllRepositories be available ? For now I am using getRepositories iterating each git projects.
<SpecialK|Canon> leni: I'd need to talk more with the team here (most of whom have just spent their weekend travelling back from FRA and thus aren't around today) - we'd like to get this to you soon, but also we have a bunch of pre-existing commitments we need to keep in mind
<leni> Ok, I'll continue on my solution then. When it comes out it should be quite easy to substitute
<tomwardill> https://gist.github.com/tomwardill/0d1c1796770614de28ccc274c8bca355 proposal for adding a new page for Developing a Buildd using LXDVM support
<tomwardill> pasting here before I go fight with the wiki to add it
<pappacena> LGTM :-)
<tomwardill> excellent :)
<pappacena> Thanks for this, tomwardill!
<tomwardill> I was just about to ask you to look at it as someone who is new to all of that bit
<tomwardill> I think I'll add it as a new page and then rewrite a bit of the existing local soyuz dev one to point there
<tomwardill> as it's more about 'how to work on buildd' than 'how to run soyuz'
<pappacena> I think I've done this with kvm a couple of months ago, and I think this tutorial is more clear about the steps than wha I remember from the other one... good improvement
<tomwardill> oh yeah, sorry, forgot you'd done some already
<tomwardill> the LXD VM support is lovely, just wish I could work out the uid mapping so we could have users other than 'ubuntu'
<pappacena> I did it more out of curiosity... I don't even know if that machine actually builds something... :-P
<tomwardill> heh
<tomwardill> tbh, that's the state mine is in most of the time anyway ;)
<pappacena> hehehe
<pappacena> LXD VM is amazing! The CLI seems quite similar to create containers
<tomwardill> yeah, it's almost exactly identical (deliberately)
<tomwardill> it's a lot simpler now than when I tried it before christmas too
<lifeless> https://www.citusdata.com/product looks interesting
#launchpad-dev 2020-03-10
<stub> Yeah, Citus really took off when they stopped being a fork and became an extension. I think they did a lot of the work in core to enable that to happen. They are now owned by Microsoft.
<cjwatson> Could anyone review https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/380391 for me, please?  This is the thing Software Heritage needs.
<cjwatson> And I have a few smallish py3 branches too: https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/380137 https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/380385 https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/380386 https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/380388
<pappacena> I'll take a look on your reviews now, cjwatson
<cjwatson> Thanks
<cjwatson> pappacena: Did you notice my feedback on https://code.launchpad.net/~pappacena/launchpad/+git/launchpad/+merge/380252 BTW?  I didn't leave it as a formal vote so far, just a comment
<cjwatson> pappacena: And I think several of your turnip MPs should be landable ...
<pappacena> Right, I saw it, made the change on that branch and forgot to ping you back. I think both this MP and the one for copied_from_archive one (https://code.launchpad.net/~pappacena/launchpad/+git/launchpad/+merge/380382) can be reviewed already
<pappacena> Ah, did we have a deploy for production of the Turnip dependencies branch?
<cjwatson> Oh, so you did, OK
<cjwatson> pappacena: you don't need a deployment for that
<cjwatson> they just need to land on turnip-dependencies master first
<cjwatson> you need (at least) a deployment before *removing* anything
<pappacena> Sure, makes sense. I'll top-approve the MPs changing dependencies, then. Thanks!
<cjwatson> pappacena: Do you have a matching DB patch MP somewhere for BPPH.creator and xPPH.copied_from_archive?
<pappacena> Yes. Let me find it
<cjwatson> Failed to find it in /launchpad-project/+activereviews but I might have missed it
<cjwatson> I'm almost at EOD but these are top of my queue now
<pappacena> https://code.launchpad.net/~pappacena/launchpad/+git/launchpad/+merge/380254
<cjwatson> Thanks.  Ah, I guess you had it as WIP
<pappacena> Yes, I think so.
<pappacena> My bad
<cjwatson> No worries
#launchpad-dev 2020-03-11
<ilasc> am I the only not able to run Turnip API layer? Getting this from a fresh Master clone:
<ilasc> pkg_resources.ContextualVersionConflict: (pycparser 2.19 (/home/ilasc/turnip/env/lib/python2.7/site-packages), Requirement.parse('pycparser<2.18'), set(['pygit2']))
<cjwatson> ilasc: What container setup steps have you done?
<cjwatson> Also Thiago just landed some dependency changes which might be relevant
<cjwatson> Worth checking that your dependencies clone is up to date
<ilasc> cjwatson: went with fresh container and followed the Turnip Readme file for a completely isolated env.
<ilasc> will do, thanks!
<cjwatson> I remember this area being delicate
<cjwatson> Hmm
<cjwatson> I wonder how pappacena managed to land this
<cjwatson> requirements.txt has pycparser==2.19 and pygit2==0.27.4.  But pygit2 0.27.4 depends on pycparser<2.18
<cjwatson> Which is indeed what pip is telling you
<cjwatson> I don't understand how Jenkins passed given that
<cjwatson> pappacena: ^- could you have a look at this when you get in?
<ilasc> agreed - thank you Colin!
<SpecialK|Canon> tests pass on master
<cjwatson> pappacena: pygit2 upgrades are delicate and require careful coordination because of the libgit2 dependency; it may be simplest to back off a little to pycparser 2.17 if that's not too problematic for other reasons
<cjwatson> perhaps whether it passes depends on what version of pip you're starting with
<cjwatson> I wonder if ilasc has a different version of pip on $PATH or something
<ilasc> I'm trying to follow your recommendation and eliminate all the "on the host" setup items. Started by get my setup as similar to Tom's as possible - completely isolated LXD (as I should've done from the begging) and VSCode remotely attached for debug to each LXD
<ilasc> SpecialK|Canon: tests pass in my container too
<ilasc> it's the "make run-api" that fails
<cjwatson> Ah right, so tests pass but then something later complains, I see
<ilasc> yes, sorry! wasn't clear enough at first
<cjwatson> So I guess pserve calls the usual entry point machinery in pkg_resources and that notices the unsatisfied dependencies
<cjwatson> Wouldn't hurt to spend a little time writing a failing test for this while we have the convenient opportunity to do so
<cjwatson> And then we can make sure it passes once pappacena fixes up the dependencies
<cjwatson> (And yes, I see the same "make run-api" failure)
<cjwatson> ilasc: You can run "env/bin/pip install pycparser==2.10" (in the container of course) to temporarily fix up your environment so that "make run-api" works
<cjwatson> Perhaps we should have a unit test that runs "env/bin/pip check"
<cjwatson> Or even just run that in the Makefile after installing dependencies
<cjwatson> Probably the latter makes more sense
<cjwatson> Could also possibly just land an MP that backs out to pycparser==2.10 in requirements.txt.  But pappacena should be aware in case that disrupts their py3 work
<ilasc> cjwatson: noted and thank you, 1:1 with Kristian, reading now, agreed and I will add the Unit Test after lunch
<cjwatson> Possibly build system change rather than unit test, but sure
<pappacena> Hey, folks. I'll take a look on this problem new. I'll probably downgrade this dep. AFAIR, it was probably just raising some DeprecationWarning on py3. Shouldn't be a big problem to downgrade it. I'll add the check on Makefile too. Sorry about that, and thanks for the help, ilasc!
<tomwardill> https://code.launchpad.net/~twom/launchpad/+git/launchpad/+merge/380416 is updated with a shortcut as I couldn't find a bettter way that was reliable
<cjwatson> tomwardill: Yep, I think that'll do
<tomwardill> ta, landing
<ilasc> pappacena: I'm just back from lunch, great timing :) I'll leave you to it in that case - working on the Unit Tests for my "show MP URL" in Turnip, I can stay away from "run make-api" for now
<pappacena> Thanks, ilasc. It shouldn't take long. I'll open the MP in a couple of minutes.
<ilasc> pappacena: excellent, thank you!
<pappacena> It should be good to be reviewed now:
<pappacena> Deps MP: https://code.launchpad.net/~pappacena/turnip/+git/turnip-dependencies/+merge/380545
<pappacena> Code MP: https://code.launchpad.net/~pappacena/turnip/+git/turnip/+merge/380546
<cjwatson> pappacena: r=me on both, thanks
<pappacena> Thanks, everyone.
<ilasc> pappacena: +1
<cjwatson> pappacena: also I've just updated https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/380391 per your comments
<pappacena> Awesome. I'll take a look!
<cjwatson> Cheers
<cjwatson> pappacena: Great, thanks.  You have a bunch of review comments in exchange ;-)
<cjwatson> (Not sure to what extent you'll consider this a fair exchange)
<SpecialK|Canon> cjwatson: Thank you for handling Q689260 (mirror 307)
<cjwatson> NP, I had some useful mental state about that
<pappacena> cjwatson, well... it's a change. And feedback is always good hahaha
<pappacena> *"an exchange", I mean
<tomwardill> Did the thing: https://dev.launchpad.net/Soyuz/HowToDevelopWithBuildd
<tomwardill> Linked under 'Set up for development' on https://dev.launchpad.net/Soyuz/HowToUseSoyuzLocally
<cjwatson> Great, thanks
<tomwardill> I'll try and keep on top of lxd changes there, when we can make it simpler
<SpecialK|Canon> <3
<tomwardill> the drive mounting not doing uid mapping is a bit awkward atm
<tomwardill> cjwatson: can you point me at your oci registry branch?
<tomwardill> I've lost it somewhere
<cjwatson> tomwardill: https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+ref/db-oci-push-rule I think?
<tomwardill> that's the one
<tomwardill> I was searching for registry
#launchpad-dev 2020-03-12
<cjwatson> tomwardill: Could you have a look over https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/380350 and https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/380351 for OCI webhooks?
<tomwardill> oh, I meant to do that yesterday, had the tabs open and everything
<tomwardill> on it
<cjwatson> ilasc: Could you have a look at https://code.launchpad.net/~cjwatson/lp-signing/+git/lp-signing/+merge/380014 ?  It's a prerequisite for the Juju charm
<cjwatson> Also, yesterday I found https://forge.softwareheritage.org/T1734 which relates to my recent GitRepositorySet.getRepositories changes, and updated it with a slew of details
<ilasc> cjwatson: +1
<ilasc> have mp 380391 open for study on the to do list for today, above detail is great for that! thanks for sharing Colin!
<wgrant> Ah, excellent, that hopefully solves much of the problem.
<cjwatson> wgrant: Which does?
<wgrant> That softwareheritage.org ticket, and your reply to it.
<cjwatson> Ah yes
<cjwatson> There's a SH bod on #debian-uk who said that the people working on this are students who just kind of ... show up.  Must be a nice problem to have :)
<tomwardill> can we borrow some :)
<ilasc> :)
<wgrant> That does make some degree of sense.
<wgrant> And would indeed be quite handy.
<wgrant> Though I guess a bunch of us were just students who showed up at one point.
<cjwatson> Hmm I think I can possibly reproduce the turnip-pack-virt memory leak locally
<cjwatson> Doesn't seem to reproduce when looking up nonexistent repos, but if I clone the same existent repo 100x in a loop then I see virtserver's VIRT growing by a MB or so in top.  (sshserver gets bigger too, but I think that's keepalive connections and goes away again)
<cjwatson> Attacking it with meliae now to see if I'm imagining it
<tomwardill> ... I've made the wifi slightly faster in my study by replacing the cable
<tomwardill> one of us is having a productive morning. I suspect it's not me.
<cjwatson> you're optimising your environment! :)
<cjwatson> suspicious growth in function objects
<cjwatson> all in the guts of inlineCallbacks I think
<SpecialK|Canon> Investing in your infrastructure is generally worthwhile!
<cjwatson> I don't think PackVirtServerProtocol.requestReceived ever returns in the successful case
<cjwatson> I stuck a log entry at the end that I'm not seeing
<cjwatson> Oh.  Well nothing ever calls .callback() on that particular deferred, only .errback().  That would probably do it
<cjwatson> (see PackClientFactory)
<cjwatson> is it a one-liner
<SpecialK|Canon> hah
<cjwatson> Need to figure out how to test it but I'm pretty sure it's just https://paste.ubuntu.com/p/vRJsZtMhcW/
<cjwatson> meliae++, made this a great deal easier to track down
<tomwardill> cjwatson: is it worth adding the date_* columns to the OCI credentials and push rules?
<cjwatson> tomwardill: I don't think so really.  They're not going to be displayed on their own pages with the "Registered by <foo> on <bar>" boilerplate or anything; they're more like settings.
<tomwardill> cjwatson: the credentials field, what do I need to do to make it an encrypted field?
<cjwatson> tomwardill: There isn't yet any prior art for making an entire column be encrypted (as opposed to one value in a larger JSON column).  But you should be able to work it out, more or less, by grepping for EncryptedContainer
<tomwardill> ah, that's the magic term i was missing
 * tomwardill tries
<cjwatson> tomwardill: You'll need an IEncryptedContainer implementer (based on NaClEncryptedContainerBase) registered with some appropriate name and that plugs in the correct configured key pair; and you'll need a property on the model that does getUtility(IEncryptedContainer, whatevernameyoupick).{encrypt,decrypt}
<cjwatson> tomwardill: Or if you wanted to try slightly harder you could write a custom Storm variable that does this
<cjwatson> Shouldn't be very difficult, we just haven't needed it so far
<tomwardill> the latter feels more resuable
<cjwatson> Well, variable and property, since they usually come as a paid
<cjwatson> *pair
<cjwatson> e.g. have a look at the implementation of JSON and JSONVariable
<cjwatson> And yes, I think that would be more correct
<cjwatson> tomwardill: I think you can go ahead with https://code.launchpad.net/~twom/launchpad/+git/launchpad/+merge/380200 now - the linked code change is on prod
<tomwardill> ah, cool
<tomwardill> landing
<tomwardill> cjwatson: slightly trivial question, would the storm variable live in storm, or launchpad?
<cjwatson> tomwardill: Launchpad
<wgrant> I don't much like the Storm variable approach. It would make accidental leaks easier, and leave the object unloadable on most services
<tomwardill> wgrant: I don't follow either objection there
<wgrant> Accidentally printing the object's fields would leak the secret, and examining the object's fields on most LP hosts, that don't have the wrapping key, would crash.
<tomwardill> aha, right, that makes sense
<cjwatson> I suppose
 * tomwardill closes storm checkout
 * cjwatson finally persuades turnip tests to pass with only a small hack
<tomwardill> excellent, DB patch landed with only minor buildbot poking
<SpecialK|Canon> Nice
<cjwatson> Could I have a review of https://code.launchpad.net/~cjwatson/turnip/+git/turnip/+merge/380646 please?  As far as I can tell this fixes the turnip-pack-virt memory leak on production that's been taking down the service every couple of days
<SpecialK|Canon> Excellent!
<tomwardill> looking
<tomwardill> test_git.py is inconsistent as to assert(expected, received) or assert(received, expected)
<cjwatson> Is it?
<cjwatson> Oh, I got this backwards, silly me
<tomwardill> test_git_receive_pack_calls_spawnProcess appears to be reversed from test_git_upload_pack_calls_spawnProcess
<tomwardill> so there's both in there already
<cjwatson> receive is using assertThat not assertEquals
<cjwatson> and assertThat requires the matcher as second arg
<tomwardill> oh, so it is
<tomwardill> sorry
<cjwatson> so it was just my new code that was wrong, and I've fixed it now
<tomwardill> cool
<tomwardill> cjwatson: what fails in the test without the callback change?
<cjwatson> it hangs
<cjwatson> since yield requestReceived never returned
<tomwardill> ah... that'd do it :)
<cjwatson> *returns
<cjwatson> (though doesn't hang indefinitely since AsynchronousDeferredRunTest has a timeout)
<tomwardill> have a +1
<cjwatson> thanks
#launchpad-dev 2020-03-13
<cjwatson> You get impressively confusing errors if you accidentally declare a field as allow_none=False, default=None
<cjwatson> I mean, "NoneError: None isn't acceptable as a value" which is fair enough, but had to disable C extensions to figure out which column was involved ...
<cjwatson> Could anyone review https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/380593 and https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/380594 please?
<cjwatson> And maybe https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/380616 to remove some old bzr stuff we no longer need to carry around
<cjwatson> wgrant: Is https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/380059 possibly something you need to look at?  Removes the XPI importer, involved some moderately involved test rearrangements as a preliminary step
<cjwatson> Also also, https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/380475 is a simple bug-fix
<wgrant> cjwatson: I don't necessarily think so, unless you do
<cjwatson> OK, will see if I can find a vic^Wvolunteer
<cjwatson> Translations is in a slightly weird position where changes aren't generally DB or security or anything like that but it's also the main bit that not even I know
<cjwatson> https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/380594 (thanks ilasc for the review) was sort of amusing; I don't think I've ever seen pre-2.2 compat in the wild before
<ilasc> :)
<cjwatson> Trying to remember whether I started learning Python at 2.3 or 2.4.  I think it must have been 2.3 - I vaguely recall having to write compatibility code for set not being in the stdlib yet
<cjwatson> Ah yes, https://launchpad.net/ubuntu/+source/python-defaults/+publishinghistory?batch=75&direction=backwards&start=150 says warty had 2.3
<cjwatson> (The dates are kind of lies there - even if you expand the bottom row, it only says 2005-12-20, which I presume was when warty was first imported into Launchpad; the first few Ubuntu releases ran on Debian's archive management code instead)
<StevenK> Oh wow, I'd forgotten about that
<StevenK> (First Python version for me was 1.5)
<cjwatson> Ah yeah, you were writing linda back then weren't you?
<StevenK> Yup
<StevenK> We didn't even have list comprehensions!
<cjwatson> uphill, both ways, in the snow
<StevenK> You, off my lawn :-P
<cjwatson> Could I have a review of https://code.launchpad.net/~cjwatson/launchpad-mojo-specs/+git/launchpad-mojo-specs/+merge/380678 to fix a CI failure please?
<cjwatson> For the YAML-lovers among you
<ilasc> :-)
<ilasc> on it
<cjwatson> pappacena: I think you possibly forgot to push your allocation to lp:~launchpad/+junk/dbpatches ?
<cjwatson> (for https://code.launchpad.net/~pappacena/launchpad/+git/launchpad/+merge/380254)
<pappacena> Ah, sadly, yes. I forgot. I am out for lunch now. I will do it as soon as I get back
<cjwatson> np, just wanted to make sure it wasn't forgotten
<pappacena> Commited the allocation, cjwatson
<pappacena> (Or do we usually make MP for this too?)
<cjwatson> No, just commit
<cjwatson> Upgrading dogfood for your proxy change now.  I might just about have time to help with a bit of QA there ...
<pappacena> Ok! Thanks!
<cjwatson> pappacena: https://dogfood.launchpadlibrarian.net/412902206/mirrors.edge.kernel.org-archive-probe-logfile.txt looks good; lots of 404s but it's a partial mirror (and dogfood's DB is behind production, which also confounds things a bit)
<cjwatson> pappacena: https://dogfood.launchpadlibrarian.net/412902207/mirror.cse.unsw.edu.au-release-probe-logfile.txt looks odd.  Why is it apparently trying to talk to port 80?
<pappacena> uhm... really strange
<pappacena> let me see if I can find something on the code. I don't remember seeing any hardcoded port anywhere (the tests we have even open random ports for testing...)
<cjwatson> Maybe something to do with _parse defaulting to 80?
<cjwatson> I haven't worked out the relevant code paths though, will leave it to you
<pappacena> " _parse defaulting to 80?" it could be... I'm checking
<pappacena> Right, so for HTTPS we don't use _parse when doing the request (since treq receives the unparsed URL), but to show the error message we use parse. That's why it's showing port 80. I'll submit a MP to properly set the default port on _parse to 443 if scheme is https.
<pappacena> And it failed because this server doesn't seem to support TLS v1.1; only 1.2 and 1.3
<cjwatson> Aha
<cjwatson> Is that due to old OpenSSL?
<cjwatson> (I mean, the fact that we're demanding 1.1)
<cjwatson> Or is it more in our control?  We don't want to be discouraging 1.2+
<pappacena> No specific reason for using 1.1... I neede to explicitly set the version, and I assumed the protocol would be backward compatible, but it seems that I'm wrong...
<cjwatson> Oh, I'd missed that you set it explicitly
<cjwatson> I think we should at least say 1.2, yes
<pappacena> Yep... I set it in a kind of hacky way, because Twisted has a kind of strange interface for the HTTPS policy. It's missing a method in the default "browser-like" policy, apparently...
<pappacena> `contextFactory.getContext = lambda: Context(TLSv1_1_METHOD)`
<cjwatson> Yeah, I'm not totally familiar with that stuff
<pappacena> Me neither... I was fixing one exception at a time, until it eventually worked fine... :-(
<cjwatson> :-)
<pappacena> I'll submit a MP upgrading this context to version 1.2 and "calculating" the port on _parse depending on the scheme in half an hour. I need to pick my daughter at school now.
<cjwatson> Thanks.  No great rush, I'm about to finish for the day
<cjwatson> This looks like a good improvement so far
<pappacena> Well, at least is mostly working (and what is not working seems to be easily fixable) :-)
<cjwatson> Yep
<cjwatson> Haven't had time for the change-override-log re-reviews today as too much production debugging, but I have them open
<pappacena> No problem. I still need to fix the copied_from_archive with the previous feedbacks.
 * cjwatson EODs, I'm not going to make significant progress on Built-Using at this point
<cjwatson> Have a good weekend
<SpecialK|Canon> Take care!
<pappacena> Have a good weekend!
