#launchpad-dev 2009-01-22
<ChEx> hello all.
#launchpad-dev 2010-01-25
<jml> good morning launchpadders
<noodles775> G'day :)
<wgrant> Morning.
<wgrant> jml: Back on the wrong side of the world?
<jml> wgrant, indeed
<noodles775> Hey wgrant !
<jml> where it is deliciously toasty warm
<jml> rather than bloody hot.
<noodles775> yeah, I had +50C difference from getting on the plane in syd and off in Berlin...
<wgrant> Sydney was revoltingly humid when I passed through it today.
<wgrant> But not too hot, fortunately.
<jml> noodles775, not quite a 40C difference for me
<wgrant> jml: Your branch isn't going to land for 10.01, I guess?
<jml> no. a test failure in db-devel sank it, I'm afraid.
 * thumper waves
 * al-maisan waves back at thumper 
<jml> thumper, hi :)
<jml> bigjools, good morning.
<bigjools> eyup jml
<bigjools> jml: bummer about your branch but it's not the end of the world
<jml> bigjools, yeah. I know.
<jml> just frustrating.
<bigjools> yeah the  TZ was against us
<bigjools> db-devel was buggered most of Friday here >:(
<adeuring> good morning
<al-maisan> moin adeuring
<adeuring> hi al-maisan!
<al-maisan> adeuring: how are things? Did you go skiing after all?
<adeuring> al-maisan: yeah, I went for a day to Winterberg ten days ago or so. But last week I was sprinting, so no chance fos a trip to Sauerland...
<al-maisan> ah, I see .. not much snow in AL I guess?
<adeuring> al-maisan: naaa, no snow. just the opposite, it was quite warm :)
<al-maisan> :)
<wgrant> bigjools: How is buildd-manager on dogfood looking?
<bigjools> wgrant: fine as of Friday
<bigjools> but I will re-test today anyway
<wgrant> There is still that odd logging problem, but production doesn't log DEBUG anyway.
<al-maisan> wgrant: is there a bug that describes that problem?
<bigjools> yeah I forgot which problem
<al-maisan> we have so few problems :P
<wgrant> Since the refactoring during the sprint, the loggers retrieved by code called within BulderGroup (not in BuilderGroup itself, but other stuff that it calls) do not appear to actually log anyway.
<wgrant> s/anyway/anywhere/
<wgrant> I suspect that there is some Twisted logging evil going on somewhere, but I've not looked at it in depth.
<bigjools> wgrant: there's 2 possibilities. 1, blame twisted, 2, the logging only exists for some crack tests
<bigjools> (which check debug output)
<wgrant> The output is useful for debugging, though.
<wgrant> (as we saw at the end of the sprint, where I had to put print statements everywhere because the logging wasn't working)
<bigjools> yes
<bigjools> wgrant: in another of my twisted programs: from twisted.python import log; log.startLogging(sys.stdout)
<intellectronica> are people experiencing problems with the QA plan generator? ours seems to be missing many entries
<deryck> Morning
<jml> deryck, good morning
<adiroiban> danilos: hi. regarding bug 127171. It is OK if Ubuntu Translation Coordinators have launchpad.translationsAdmin right for IDistribution ?
<mup> Bug #127171: Rosetta experts not allowed to "Change translators" <Launchpad Translations:In Progress by adiroiban> <https://launchpad.net/bugs/127171>
<adiroiban> this change was done when we set the launchpad.translationsAdmin rights for POTemplates
<henninge> adiroiban: not sure
<adiroiban> but not sure if we should keep it :)
<henninge> oh, you meant danilos ;) He'd know better.
<henninge> adiroiban: but i have not yet seen him today
<adiroiban> the change was done together with IDistroSeries and IPOTemplate
<adiroiban> ok. will wait for Danilo and see what can be done
<adiroiban> I talked about this bug with jtv ( Hi!), but it looks like Danilo did not agree with our aproach :)
<jtv> and hi adiroiban :)
<adiroiban> i was asking Danilo about bug 127171. Will wait for him to come online...
<mup> Bug #127171: Rosetta experts not allowed to "Change translators" <Launchpad Translations:In Progress by adiroiban> <https://launchpad.net/bugs/127171>
<danilos> adiroiban, henninge: sorry, long phone call :)
<danilos> adiroiban, so, why would you want that privilege for UTC on Ubuntu? do you imagine changing a translation group often (or ever)?
<adiroiban> danilos: nope. but this is the current code
<danilos> adiroiban, right, so it's about simpler implementation, right?
<adiroiban> so I was asking if its ok the remove UTC from IDistribution
<danilos> adiroiban, that might be a good reason to let UTCs do it :)
<adiroiban> and leave only for IDistroSeries and IPOtemplate
<danilos> adiroiban, where exactly do UTCs use their privileges on IDistribution otherwise?
<adiroiban> :) not sure
<adiroiban> the change, was one of my first branches
<adiroiban> when I didn't know to much about LP
<adiroiban> right now
<danilos> adiroiban, yeah, I can't think of anything myself right now
<adiroiban> UTC can change lang-pack admins
<adiroiban> but not sure if it vital
<danilos> adiroiban, there is one thing that they should have, and that is setting translation focus
<adiroiban> as lank-pack admins are per distribution
<adiroiban> not sure if UTC can do that now
<danilos> adiroiban, right, I think that makes sense and there is a future need for having them administer more things in a distro (like setting focus; they can't do that yet)
<adiroiban> then we should keep the permission?
<adiroiban> I'm still learning about Zope, and I am not sure what are the best practices for implementing fine grain permission policies
<danilos> adiroiban, so, with all that in mind, I'd say let's just let UTCs change the group as well, just so we don't have to introduce another permission for them
<adiroiban> ok
<jml> heh heh
<jml> no one knows what the best practices are for implementing fine-grained permission policies in Zope.
<adiroiban> :)
<adiroiban> jtv: regarding bug 121520 , don't know if we can have it for this release
<mup> Bug #121520: Language pages show merged accounts <qa-bad> <trivial> <ui> <Launchpad Translations:Fix Committed by adiroiban> <https://launchpad.net/bugs/121520>
<jtv> adiroiban: ah, OK... I should've asked you first.  Can I assign it to 02?
<adiroiban> I am not sure why it is not working on edge
<lifeless> thumper: jml: you might find bug 388325 interesting to fix :)
<mup> Bug #388325: branches linked in bugs are hard to use from bzr <bug-branch-links> <ui> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/388325>
<adiroiban> I have a testcase where I'm creating a merge account
<jml> lifeless, I don't fix Launchpad bugs.
<adiroiban> and everhing is ok
<jml> lifeless, but thanks :)
<lifeless> jml: :)
<lifeless> jml: no but you get to provoke things
<jtv> adiroiban: both ways around?
<adiroiban> jtv: so I will need help for this bug
<adiroiban> jtv: both ways?
<jml> lifeless, my provoking powers are exactly the same as your own.
<lifeless> its unclear to me whether it should be a malone or a lp-code bug
<jtv> adiroiban: I don't know the first thing about account merging but istm that it's asymmetric
<adiroiban> jtv: well, I asked gmb, he reviewed the branch
<adiroiban> jtv: I have merged_account.merge = realaccount
<jtv> adiroiban: oic... there's no asymmetry between the accounts you're displaying then.
<jtv> btw I say "account" but in this case I guess you want "person"
 * jml afk
<adiroiban> jtv: sorry, person
<jtv> I did it too :)
<jtv> adiroiban: it's not a doubly-merged account or anything?
<adiroiban> jtv: don't know what is a doubly-merged account :)
<jtv> adiroiban: e.g. foo-merged is merged with foo-old and foo-old is merged with foo, and you get to see both foo-merged and foo?
<adiroiban> no
<adiroiban> as before the commit there was not foo-old
<adiroiban> just foo
<adiroiban> and foo-merged
<jtv> adiroiban: ah, and you're simply not displaying merged accounts at all...
<adiroiban> yes
<adiroiban> the code was working on my local system ,
<jtv> adiroiban: I'm looking at the diff for your branch, but afaict it simply doesn't touch the template for the page that the bug links to as an example.
<adiroiban> I will have ask somebody to look again over the testcase
<jtv> adiroiban: the diff fixes pofile-details.pt and pofile-translate-contributors.pt, but the bug links to a *language* pas an example.
<adiroiban> jtv: isn't that lib/lp/translations/templates/pofile-translate-contributors.pt
<adiroiban> but the language is including the `portlet`
<jtv> adiroiban: nope... I wouldn't have noticed except the fix in the diff inserts commas between names, which the example page for the bug doesn't do
<jtv> so I think the same fix is needed in language-portlet-top-contributors.pt
<adiroiban> thanks! :)
<jtv> (BTW odd that the text says "top 20" even when there are fewer than 20 people)
<jtv> adiroiban: the devil is in the details.  :)
<jtv> adiroiban: what I would do is Q/A the fix for the other pages as best you can, and file a new bug for the remaining one.
<jtv> Which also means we can keep this one targeted for 10.01.
<adiroiban> well... now that those pages got me into trouble, and that I know a little bit more about Zope, I would no hurry and try to see if I can refactor that code into a macro
<adiroiban> and next time just change it from one place
<jtv> adiroiban: if you can improve it, great.  But you've already landed the incomplete fix, right?
<adiroiban> yes
<adiroiban> the current branch is landed
<jtv> And as far as we know, it didn't actually break anything, so we can treat this as an incomplete fix and have a separate bug for "clean up this fix and add one more template."
<jtv> Add one more template to the places you fix, that is>
<adiroiban> ok
<adiroiban> filling the new bug
<jtv> adiroiban: the good news is that you probably did fix the bug in a lot of places.  :)
<adiroiban> yes :) ... but failed to fix the link from the report
<jtv> adiroiban: btw, I'm fixing up the milestones for bugs that are assigned to you but not targeted to milestones...  did your fix for bug 135008 roll out before the new year?  The commit date is the Sunday before release.
<jtv> or seems to be
<mup> Bug #135008: "Languages in Launchpad" page doesn't auto-focus search field <qa-ok> <trivial> <ui> <Launchpad Translations:Fix Committed by adiroiban> <https://launchpad.net/bugs/135008>
<adiroiban> jtv: not sure about how, if and when my branches were landed on production
<jtv> adiroiban: nm... it looks like it may well have landed earlier
<adiroiban> yep
<adiroiban> maybe it went togheter with Henning changes for the langauges page
<mars> beuno, around?
<beuno> mars, hey
<mars> hey beuno
<maxb> I've noticed that lp:launchpadlib is lacking in tags - I've pushed lp:~maxb/launchpadlib/with-tags with added tags based on diffing the released tarballs. - It's not a merge per se, but should I file a MP nonetheless?
<jml> maxb, I guess. Maybe it's better to just ask leonardr? :)
<sinzui> Ursinha: ping
<Ursinha> sinzui: hi
<sinzui> Ursinha: https://dev.launchpad.net/RegistryTeam/RegistryTestPlans/10.01 was not updated with my last branch that landed in db-devel.
<Ursinha> sinzui: which revision is that?
<sinzui> Ursinha: r8923
<Ursinha> sinzui: it's there
 * sinzui blinks
<sinzui> Ursinha: thanks. I will drink more coffee.
<jml> kfogel, btw, I downloaded Sita Sings the Blues, and have distributed pirate copies to my friends.
<kfogel> jml: alas, there is no piracy with Sita Sings the Blues.  I hope you all enjoy it despite this! :-)
<jml> kfogel, I tried watching it last night, but I fell asleep (it was really good, but it had been about fifty hours since I last slept in anything better than an economy seat)
<jml> kfogel, oh no, you don't understand.
<kfogel> jml: heh, understood.  I won't tell nina.
<jml> kfogel, I put on a funny hat and eye patch and said "Yarrr", then I copied it legally.
<kfogel> .oO ("understood" referred to something other than that which jml said I didn't understand)
<kfogel> Ah!
<kfogel> beautiful
<kfogel> that I WILL tell Nina :-)
<leonardr> maxb: i'm not sure what you mean by tags in this context
<Ursinha> OOPS-1486EA603
<maxb> leonardr: "bzr tags"
<Ursinha> hm
<leonardr> maxb: i've actually never used bzr tags. it looks like you use them to mark released versions? is that what your branch does?
<leonardr> (i'm just asking from curiosity; you should definitely file an mp)
<maxb> Yes. I'm surprised, because this is the first project I've come across which doesn't tag versions
<maxb> Without them, how do you answer questions like "Is this revision included in that release tarball?" ?
<leonardr> maxb: i've been using bzr log. and no one has corrected me since i'm the only person who's had to answer those questions
<leonardr> maxb: tags are a better solution
<maxb> It wasn't obvious to me browsing the log messages exactly where the releases were
<leonardr> maxb: well, you are the second person :)
<leonardr> it's not a scalable solution
<leonardr> maxb: give me the mp and i'll review it. i hope it hasn't caused you too much trouble
<maxb> No, it's fine. I'll do a MP
<mars> sinzui, ping
<sinzui> Hi mars
<mars> hi sinzui.  Just reading your mail reply, and wondering what the goal of CSS bug filing procedures is?  Is it to "get them fixed", or to "get them triaged"?
<sinzui> Mars: Get them triaged. The triaged state tells us roughly how it will be fixed an release
<mars> sinzui, and if the latter, then can you safetly assume that the filing engineer already knows everything about the CSS bug needed to set that priority?  Or is there a risk of them mis-prioritizing, and a more formal filing would prevent that?
<mars> sinzui, I guess I'm missing the original problem that this idea is meant to solve.  Were there a whole bunch of 'New/Undecided' CSS bugs building up that we wanted to triage?
<sinzui> mars:an engineer knows if there is datalose it an impossible impediment to completing a task we say can be done...CRITICAL, assign an engineer and create a CP.
<sinzui> Or the problem is an OOPS, a impediment that many users cannot work around, or is a series goal...High assign it to a milestone, find an engineer
<sinzui> Otherwise it is low, engineers will fix it when there is a oportunity
<sinzui> mars: the problem is that we had High bugs without anyone assigned to fix or a date to see them fixed
<mars> sinzui, ok, so is the goal is fixing said bugs?  Or re-prioritizing them, so they are triaged correctly?
<sinzui> Triage always involves repriorisation of bugs as we discover new information, the code change, and our goals change. Items we thought were High were not high after we ignored the issue for many month
<sinzui> mars: consider that the emergence of Chrome required us re-triage and fix many webkit bugs because it doubled the number of affected users in a few months
<mars> sinzui, ok, still not understanding how this CSS bug filing procedure relates to that.  I thought it was just a way to keep new CSS bugs from slipping through.
<sinzui> Most the recent css bugs we fixed were for webkit
<sinzui> I fixed 4 this release
<mars> ok
<sinzui> bac and deryck were discussing one that they did not know was CSS
<sinzui> mars: If someone cannot do the triage, then he should not, but I think launchpad engineers can determine severity and priority for most bugs in the span of a minute. If he can take time to report a bug, surely he has thought about the impact on users and our own commitments
<Ursinha> intellectronica, deryck, is it known that bug index pages are timing out pretty much consistently?
<deryck> Ursinha, yes.
<intellectronica> Ursinha: oh yes it is. i've got a fix in review
<deryck> what he said :-)
<Ursinha> intellectronica: oh, awesome then :)
<dobey> hi guys
<dobey> i'm trying to run tarmac for one of my projects, and launchpad keeps giving back 502 or 503 errors (usually 502)
<intellectronica> dobey: rockstar might be a good person to ask
<dobey> intellectronica: it looks like the failure is from the call of branch_merge_proposal.createComment()
<dobey> intellectronica: did that API change in some way that's not documented properly?
* gary_poster changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.01 | PQM is closed | gary_poster is release manager | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes | Channel logs: http://irclogs.ubuntu.com/ | Edge bugs searching is timing out, use lpnet
<kfogel> adeuring: hey, thanks for fixing that ec2 failure in 506018.  Shall I just merge your change into my branch?
 * jml is gone.
<mwhudson> thumper: good morning
<mwhudson> thumper: going to the drs this morning, will miss stand up
<thumper> morning people
 * thumper goes hunting for his headset
<cody-somerville> Whats the URL to the buildd farm these days?
<mwhudson> cody-somerville: you mean https://edge.launchpad.net/+builds ?
 * cody-somerville nods.
<kfogel> gary_poster: project groups do not implement IHasBugs, right?  I'm looking at lib/lp/registry/model/project.py:"class Project" and while I see some methods that say "See `IHasBugs`", the class as a whole does not seem to inherit from HasBugs.  (Am I even asking the question in the right way?)
<gary_poster> kfogel, they would implement the interface, which is a different spelling than inheriting from a class.   I don't know the specific answer, but gimme a minute
<kfogel> gary_poster: *nod*
<kfogel> gary_poster: just for context:  What this is really about is issue #506018, the new "+patches" view, which will be accessible from Product, ProductSeries, DistroSeries, DistributionSourcePackage, Distribution, SourcePackage, and eventually Person/Team.  What I'm trying to figure out is, will we have to do any extra coding to get it working for Project Group too?
<mup> Bug #506018: Need a "+patches" view: report lists patches attached to bugs. <story-patch-report> <Launchpad Bugs:In Progress by kfogel> <https://launchpad.net/bugs/506018>
<gary_poster> kfogel: I'm a bit too booked for context :-) but I'll try to answer your question, and show you how I did.
<kfogel> gary_poster: that helps, if you have time.  thanks
<gary_poster> kfogel: yes, it implements the interface http://paste.ubuntu.com/362878/
<kfogel> gary_poster: looking
<kfogel> gary_poster: hey, that was a huge clue bat!  Thank you.
<gary_poster> np :-)
<mwhudson> hmm
<mwhudson> i still seem to be connected to irc
<mwhudson> thumper: can you see this?
<bdmurray> When running make run I encounter a Couldn't find a distribution for 'zc.buildout==1.50.dev-gary-108342' message
<gary_poster> bdmurray: update your download-cache
<bdmurray> gary_poster: a bzr update in download-cache?
<gary_poster> bdmurray: yeah.  rocketfuel-get should do this for you.  If you don't use this you are using the "advanced" path, at least for now.  In any case, if your download-cache is a checkout (the way rocketfuel-get does it), use bzr up; if it is a branch, use bzr pull.
<bdmurray> okay, I'm on 112 now without a change in behavior
<kfogel> hunh.  is it expected that loading a branch like https://code.edge.launchpad.net/~kfogel/launchpad/506018-patch-report will fail right now, with an error:
<kfogel> Please try again
<kfogel> Sorry, there was a problem connecting to the Launchpad server.
<kfogel> Try reloading this page in a minute or two. If the problem persists, let us know in the #launchpad IRC channel on Freenode.
<kfogel> Thanks for your patience.
<mwhudson> kfogel: is it working now?
<mwhudson> there was a blip
<kfogel> mwhudson: working now
<mwhudson> good
<thumper> mwhudson: I could see that
<mwhudson> thumper: yeah, everything seems to be working again now
 * thumper starts digging through the email mountain
<thumper> good
<mwhudson> thumper: i was worried that my isp had disconnected me a few days early...
<thumper> heh
<bdmurray> gary_poster: so I'm still receiving that error
<gary_poster> bdmurray: hm.  What I described is definitely the core problem, so we just need to make sure your download-cache is up to date.  In the launchpad branch, cd to the download-cache (bzr gets confused otherwise, I have found).  Then...do a bzr revno for me.  I get 112.
<bdmurray> gary_poster: yes, that's what I have
<gary_poster> bdmurray:  uh.
<gary_poster> bdmurray: so, cd out of there, run bin/buildout, and put a pastebin up somewhere for me.
<bdmurray> gary_poster: make build is failing as see at http://pastebin.osuosl.org/31259
<gary_poster> bdmurray: yours is an isolated problem, so I'm trying to figure out what it could be.  The only thing this has ever indicated in the past is an out-of-date download-cache.  I'm experimenting.
#launchpad-dev 2010-01-26
<gary_poster> bdmurray: does bzr info for your download-cache indicate (in the Location) that it is "checkout of branch: bzr+ssh://bazaar.launchpad.net/~launchpad/lp-source-dependencies/trunk/" ?
<bdmurray> gary_poster: yes and thanks for looking into this
<gary_poster> ok, bdmurray, does the distribution appear to be in the download-cache when you do ``ls download-cache/dist/zc.buildout-1.5.0dev-gary-r10*``?  You should see ``download-cache/dist/zc.buildout-1.5.0dev-gary-r108342.tar.gz`` as one of the entries
<bdmurray> gary_poster: yes, that is there
<gary_poster> ok, trying next step
<gary_poster> bdmurray: argh, I started with a completely fresh eggs directory, and a ``make build``, and it worked just fine for me.  ok, here's an interesting idea.  what if you do a ``ls eggs/zc.buildout-1.5.0dev_gary_r10*``?  Does that include eggs/zc.buildout-1.5.0dev_gary_r108342-py2.5.egg ?
<bdmurray> gary_poster: no, I only have 105072 there
<gary_poster> darn, there goes that theory
<gary_poster> bdmurray: ok, flailing even more obviously now.  Does ``ls -hl download-cache/dist/zc.buildout-1.5.0dev-gary-r108342.tar.gz`` show 274K?
<bdmurray> gary_poster: yes it does
<gary_poster> argh
<gary_poster> bdmurray, how did you build this tree?  I can try to dupe.
<bdmurray> gary_poster: I believe I just ran rocketfuel-setup
<gary_poster> bdmurray, what happens if you run rocketfuel-get, out of random curiosity?
<gary_poster> Same thing?
<bdmurray> gary_poster: a make build after rocketfuel-get fails the same way
<gary_poster> bdmurray: rocketfuel-get does a make also though...
<gary_poster> bdmurray: try a ``make clean`` and then a ``make build``
<bdmurray> gary_poster: still the same
 * gary_poster curses
<gary_poster> bdmurray, the only other thing that came to mind--this is the last ditch effort before I try to do rocketfuel-setup tomorrow myself--is to make sure that the user that you are running as can read those files.  I noticed that the home directory in your pastebin was ubuntu.  Is everything owned by ubuntu?
<gary_poster> (that is, the ubuntu user)
<gary_poster> (and you are running as the "ubuntu" user)
<bdmurray> gary_poster: yes, it is a vm and that's the only user
<gary_poster> bdmurray: ok.  what kind of vm is it, and how gargantuan is it?  If I could rustle up some place for you to upload it to (which may not be possible), would that be reasonable to upload for your bandwidth?  (It wouldn't for me, for instance; my download is good but not my upload).
<gary_poster> Failing this, I'll try to dupe the problem myself in a vm.  I'm the release manager for this release and so I'm pretty busy, and really busy tomorrow, so I'm might not get to it till Wednesday or so.
<gary_poster> bdmurray: everything was fine, and then it stopped working recently, right?
<bdmurray> gary_poster: no, this is actually a brand new setup
<gary_poster> bdmurray: oh!  ok.  Did you set up the vm yourself, or is this something we provided somehow? (There's been talk of doing this)
<gary_poster> bdmurray: and karmic, right?
<bdmurray> gary_poster: I set it up myself.  Karmic - yes.
<gary_poster> bdmurray: ok cool.  I'll try to get one set up and started tonight if I can, and get back to you if I figure anything out.  Feel free to ping me tomorrow, but forgive me if I don't have it ready.
<bdmurray> gary_poster: okay, thanks! its not a huge hurry
<gary_poster> bdmurray ok cool.  I know you have done some nice contribs lately, so don't want something silly like this to be in your (or anyone elses) way
<bdmurray> gary_poster: thanks
<bdmurray> gary_poster: to be clear eggs/zc.buildout...108342 does not exist only 1050702 does
<rockstar> thumper, ping
<thumper> rockstar: pong
<rockstar> thumper, skype?
<thumper> ok
<rockstar> thumper, lp:~rockstar/launchpad/upgrade-branch-rc
<thumper> rockstar: you realise that you deleted 4 qa items and only added 2?
<rockstar> thumper, yes, because the 4 were actually only two.
<thumper> rockstar: actually 5 and 2
<thumper> ok
<rockstar> They were the same issue.
<thumper> rockstar: is the bad rcfixed yet?
<rockstar> thumper, yes.
<rockstar> It's in db-devel now, should be in stable next year some time.
<thumper> ok
<mwhudson> hooray (?) for 3g
<mwhudson> thumper: did we have much more to talk about
<mwhudson> ?
<thumper> mwhudson: hmm.. I think we covered most of it
<thumper> mwhudson: I'll file some bugs, update the wiki
<thumper> mwhudson: and we can take it from there
<mwhudson> thumper: cool
<mwhudson> thumper: i'll start working on the "no mirrorRequest for noop pull" case
<thumper>  ok
<mwhudson> rtt min/avg/max/mdev = 1171.359/9666.795/18292.795/6516.203 ms, pipe 17
<mwhudson> wow, this is slow :(
<mwhudson> although i wonder if the thunder is affecting my connection....
<wgrant> mwhudson: Will your new place have an Internet connection immediately?
<mwhudson> wgrant: yes, i guess that's the upside to this screw up
 * thumper goes to make dinner
<rockstar> Ah good.  My rc candidate has passed all its tests.
<rockstar> Er, I guess it's been elected, so it's not a candidate anymore.
<al-maisan> Good morning!
<Ryan1> Any chance of having the Assignee column removed on the answers main list? No one uses it (in Ubuntu at least) and it's a waste of space.
<jtv> hi henninge
<henninge> hey jtv !
<jtv> henninge: did you see the oopses in pofile-js-footer?
<henninge> noe
<jtv> autofocus_html_id breaks in some vague way :(
<henninge> p
<jtv> Got some edge oopses, e.g. OOPS-1486EC432
<jtv> henninge: ahhh, I think this happens when the current batch is empty.
<jtv> browser/pofile.py, _initializeTranslationMessageViews
<henninge> jtv: sound reasonable
<jtv> henninge: just lost a bunch of work on the bug report because I pressed <alt> but not <ctrl> while trying to switch workspaces...  shorter version is now at bug 512698; I'm reconstructing the rest of the details in a comment.
<mup> Bug #512698: Oops while rendering +translate page for empty POFile <oops> <Launchpad Translations:New> <https://launchpad.net/bugs/512698>
<jtv> hi danilos
<danilos> hi jtv
<bigjools> wgrant: hi
<adiroiban> jtv: this is related to bug 359180
<mup> Bug #359180: Missing keyboard shortcut for navigation <qa-bad> <trivial> <ui> <Launchpad Translations:Fix Committed by adiroiban> <https://launchpad.net/bugs/359180>
<jtv> adiroiban: hi! what is related to that?
<adiroiban> bug 512698
<mup> Bug #512698: Oops while rendering +translate page for empty POFile <oops> <Launchpad Translations:New> <https://launchpad.net/bugs/512698>
<mrevell> Morning
<jtv> hi mrevell!
<mrevell> hey jtv
<jtv> adiroiban: oh, it's related to the QA failure?
<jtv> adiroiban: want me to put up a fix?
<jtv> (do have a standup now)
<adiroiban> jtv: I have created a new branch for bug 359180, it was approved, but not landed
<mup> Bug #359180: Missing keyboard shortcut for navigation <qa-bad> <trivial> <ui> <Launchpad Translations:Fix Committed by adiroiban> <https://launchpad.net/bugs/359180>
<jtv> adiroiban: then we'd still want a quick fix for this oops... hang on, call first.
<adiroiban> jtv: this can be a quick fix http://paste.ubuntu.com/363127/
<wgrant> bigjools: Hi.
<jtv> adiroiban: the "autofocus_html_id view/autofocus_html_id | string:;" part?
<bigjools> wgrant: I spoke to Joneau and we're happy to remove SPRBU on the proviso that any API is designed to not preclude its addition later
<jtv> JÃ¸Ã±eau?
<wgrant> bigjools: Ah, excellent.
 * bigjools can't find the compse key
<bigjools> compose, either
<wgrant> Any API will only be used in a couple of places, so it shouldn't be too terible.
<adiroiban> jtv: yes
<jtv> adiroiban: that catches an unhandled exception!?
<adiroiban> jtv: nope. it should deal with LocationNotFound
<jtv> adiroiban: why does it produce that exception type anyway?  With these deep-zope tracebacks I usually end up just getting one or two keywords and ignoring the rest.
<jtv> What comes out of the code on our end should be an AttributeError, right?
<danilos> jtv, adiroiban: hi, can you guys please comment on https://bugs.edge.launchpad.net/rosetta/+bug/506714
<jtv> danilos: will do
<danilos> jtv, thanks
<adiroiban> jtv: I don't know why the view is not visible from the template
<jtv> adiroiban: I thought it was that the template can access the view, but an attribute of the view then fails with an AttributeError.
<jtv> Ah!  Maybe things get confused because it's a @property that fails with that error.  Maybe somewhere along the line it's assumed that if you evaluate just foo.bar, a property, and get an AttributeError then it must be because foo.bar does not exist.
<jtv> But in this case, foo.bar is really a method invocation that happens to fail with an AttributeError.
<jml> ok back
<adiroiban> jtv: I have commented on the bug and added a possible fix
<jtv> adiroiban: thanks... meanwhile I'm having a look at the one danilo pointed out
<adiroiban> jtv: should I push the branch for review as a RC ?
<adiroiban> don't know what is the process
<jtv> adiroiban: the first thing is to get a review... I can review it, if you like.  There doesn't seem to be anyone on call.
<adiroiban> jtv: should I get a review for this branch https://code.edge.launchpad.net/~adiroiban/launchpad/bug-359180-take-2/+merge/17785 ?
<adiroiban> or I should create a new one?
<jtv> adiroiban: separate branch please... this is a separate, if related, problem.
<jtv> and if we do want to get a fix for the actual oopses rolled out outside of the normal release schedule, we want a minimal change.
<adiroiban> but the fill fill depend on this branch
<adiroiban> and it is not commited
<adiroiban> but the fix will depend on this branch
<jtv> adiroiban: what other problems are there with the branch that was landed, besides these oopses?
<adiroiban> a javascript keybinding overlap with Compiz
<adiroiban> jtv: I will create a new branch to only fix this issue
<jtv> adiroiban: how much trouble can that one cause?
<adiroiban> creating a new branch ?
<adiroiban> I don't understand the question.
<jtv> adiroiban: what problems does the keybinding overlap cause?
<adiroiban> jtv: minor... if you have compiz, the keybindings will be used by compiz and not send to the Webbrowser
<jtv> adiroiban: so not a big problem, but we're basically telling people to press keys they shouldn't be pressing...  How about we request that your original fix be backed out?
<jtv> You've got a better fix waiting for next release, right?
<adiroiban> yep
<jtv> So then I suggest we fix all these problems by backing out the broken fix.
<adiroiban> fine by me
<jtv> OK, I'll request it.  Thanks!
<deryck> Morning, everyone.
<jml> deryck, good morning.
<bigjools> morning deryck
 * jml is off to run some errands.
<mdd> hey guys, sf.net recently blocked access for some countries (http://sourceforge.net/blog/). is this something that will happen on launchpad, too?
<maxb> traceroute suggests that the Launchpad datacentre is in London, so US law seems unlikely to intrude
<lifeless> maxb: it is
<lifeless> mdd: I'm not aware of any proposal to do that, however we do have a US arm so its not impossible that we might need to do address it
<mdd> lifeless: okey, thanks!
<leonardr> james_w, are you around to talk about your three outstanding lazr.restfulclient branches?
<al-maisan> leonardr: I believe james_w is on vacation this week.
<leonardr> al-maisan: well, that takes some of the time pressure off of me
<maxb> Is there anyone else who would know about UDD branches?
<al-maisan> leonardr: aha :) IIRC there's a platform sprint on next week, so probably even less pressure on you :)
<maxb> I have been wondering why the branches for debian/+source/subversion have been deleted from LP
<al-maisan> maxb: it depends, jml or thumper or mwhudson, lifeless as well probably.
<dobey> leonardr: i just pasted the stack trace on #512552
<mup> Bug #512552: Large POST fails with createComment on merge proposal <Launchpad Bazaar Integration:Incomplete by leonardr> <https://launchpad.net/bugs/512552>
<leonardr> mup, ok
<leonardr> er, dobey, ok
<dobey> heh
<leonardr> dobey, responded
<dobey> ok, i'll try that
<jml> the build is broken
<jml> maxb, hi
<maxb> hello
<jml> maxb, I don't know why the branches for debian/+source/subversion have been deleted from LP.
<maxb> Do you think I should wait for james_w to be back to find out more?
<jml> maxb, sadly, yes.
<jml> hmm.
<jml> actually.
<jml> maxb, I think he sent an email to the UDD list about how he's got a web page that logs the stuff that the importer does
<maxb> Indeed. But I don't think the importer can automatically delete things
<jml> maxb, hmm.
<jml> maxb, are you sure?
<wgrant> The API to do that was only added very very recently.
<wgrant> So it seems unlikely.
<wgrant> (10 days ago, in fact)
<EdwinGrubbs> mars: ping
<mars> hi EdwinGrubbs
<EdwinGrubbs> mars: did you get a chance to look at my email on the rhinos list concerning sprites?
<mars> EdwinGrubbs, yes.  I'm not sure how to answer in that case though.  I defer to Martin and Matthew.
<EdwinGrubbs> mars: did you learn how sidnei is consolidating the sprite images with a script?
<mars> EdwinGrubbs, yes.  I'm not sure if the tool they use is configurable to take whitespace into account.
<mars> EdwinGrubbs, I'll check
<EdwinGrubbs> mars: I assume we have the source code for it. Is that not correct?
<mars> EdwinGrubbs, nope.  They use spriteme.org right now.  If they switch to SmartSprites (http://csssprites.org), then it is configurable, sort of.
<mars> EdwinGrubbs, looking at the SmartSprites docs, doing so would still be a hack
<mars> EdwinGrubbs, is there no way to restructure the HTML to do this?  Because all of the sprite tools seem to assume there is.
<EdwinGrubbs> mars: there are several ways to restructure the HTML, but they all assume that the element using a sprite as the background is only one line. In the most extreme solution, we put a sprite in an element by itself, and it becomes equivalent to an <img> tag.
<EdwinGrubbs> mars: As described in the email, the most difficult layout to handle is where the first line is indented after the sprite, but the second line is not indented, so it appears under the sprite. If we didn't care about that situation at all, we would still have a fairly large amount of work converting all the long <a> tags with sprites to be <a><span class=sprite></a> so that wrapping wouldn't expose the next sprite.
<mars> EdwinGrubbs, found something that may be of interest here: http://archivist.incutio.com/viewlist/css-discuss/104801
<mars> EdwinGrubbs, difficult to get enough context though: http://archivist.incutio.com/viewlist/css-discuss/104799
<mars> EdwinGrubbs, it seems like you have two problems: 1) You need inline elements to obey a hanging indent; And 2)  You need the sprites to not have the bottoms blown out of them
<EdwinGrubbs> mars: I don't see how that mailing list thread applies. It uses vertical-align:botton to make sure that each row of <li> elements line up at the bottom of the element where the image is as opposed to the top of the <li>.
<EdwinGrubbs> mars: your summary of the problems is accurate.
<EdwinGrubbs> mars: look at the answer to question #2 at http://csssprites.org/#faq
<mars> EdwinGrubbs, yes, so every sprite definition using the tool would need the margin directive
<EdwinGrubbs> mars: my point is that we are not the only people encountering this problem.
<mars> EdwinGrubbs, yep.  Looking at the Yahoo and AOL sprites, they space them with a line height of 2-3x
<mars> EdwinGrubbs, anticipating that if menu items exceed that, then you have a problem
<mars> EdwinGrubbs, so one way to fix this would be to regenerate the sprites with each sprite starting at 3x line height
<jml> is staging deliberately down?
<mars> gary_poster, ^ ?
<EdwinGrubbs> mars: exactly
<mars> EdwinGrubbs, 2x line height between each
<mars> EdwinGrubbs, I'm tempted to say that going beyond that height for items means that you can stop using sprites.  For example, switch to list item bullet images.
<EdwinGrubbs> mars: why would you stop using sprites?
<mars> EdwinGrubbs, because they aren't meant to be used for <li> elements that span 5+ lines.  They are meant to clean up a whole bunch of short little items, not big things.
<EdwinGrubbs> mars: but is there any real drawback to using them for 5 line elements?
<mars> Sprites are not the One True Way for every image on the page.  They are a way to save you from making 40+ server requests.
<mars> EdwinGrubbs, these padding issues would be one drawback.
<EdwinGrubbs> mars: how is it a drawback for 5 lines if the solution is the same as for 2 lines?
<mars> EdwinGrubbs, well, imagine a page with a menu of options, each option has 8 lines.  You could fit maybe 5-10 bullets on the page?
<mars> EdwinGrubbs, so sprites would save you 10 unique requests, say.  Compare that to a typical Launchpad page, where you have say 35 unique images.
<mars> 35:1 is a big performance gain
<mars> 10:1 or 5:1, not so much
<mars> now, it is a pain to engineer the sprite for the 5:1 case
<EdwinGrubbs> mars: but those bullets will often use the same sprites that already appear on the page. For example, the picker displays person and team sprites, and it can easily be 3-5 lines in an <li>
<mars> EdwinGrubbs, true, but is it worth the time to re-engineer the sprite image so you can make it fit this one case?  So you can save two requests to the server?  (one for person, one for team)
<mars> that's assuming that those two stand-alone images aren't cached, either
<mars> EdwinGrubbs, if we didn't have sprites, we would just use the person and team image bullets in the <li>, no problem
<mars> Using sprites is a performance optimization for initial page loads
<mars> So the cost of not using that optimization in the picker list (2 requests) is trivial compared to the cost of not using it for 90% of the other page icons (35+ requests)
<mars> EdwinGrubbs, just thought of another approach: did you consider switching the picker list over to a <dl>?
<mars> You could use the sprite on the <dt>, indent the <dd> using the CSS text-indent property
<mars> You still get to use the sprites, but don't have to worry about a 5-line list item any more
<mars> EdwinGrubbs, I have to go to lunch, but let me know if the <dl> idea would work.
<EdwinGrubbs> mars: that is not really any different than an <li> with two spans in it. You would still have the issue of a long <dt> wrapping.
<mars> EdwinGrubbs, a long <dt> would not make much sense.  2 lines at most.
<mars> The <dd> can be as long as needed.
<EdwinGrubbs> mars: my point is that the differences between a <dl> and a <ul> is not where the problems lie. We could do everything we want with spans inside an <li>. Using a <dl> might be simpler, but it really is irrelevant to the sprite problem, which is difficult to re-use.
<kfogel> adeuring: ayt?
<adeuring> kfogel: yes?
<sinzui> EdwinGrubbs: I agree. A <dd> and a <li> are block-like and should not be used
<kfogel> adeuring: I'm writing tests for other entities, for bug #506018.  I thought I'd start with SourcePackage, which means my next step is to create a source package in my testdata.  Oh, or find one, duh.  Let me look around.  Anyway, maybe I don't actually have a question yet.  I just wanted to let you know what I'm doing, in case it sounds insane.
<mup> Bug #506018: Need a "+patches" view: report lists patches attached to bugs. <story-patch-report> <Launchpad Bugs:In Progress by kfogel> <https://launchpad.net/bugs/506018>
<adeuring> kfogel: OK ;) I'm AFK for the 10-15 minutes but will stay around for 2 or 3 hours
<kfogel> adeuring: but, do we have any packages in default test data?  I'm not even sure how to search for them.
<kfogel> just got an OOPS with this:
<kfogel> https://launchpad.dev/+search?field.text=package
<adeuring> kfogel: let me look...
<kfogel> adeuring: oh, wait, I think we found them before
<kfogel> https://bugs.launchpad.dev/ubuntu/+source/libstdc++
<kfogel> https://bugs.launchpad.dev/ubuntu/+source/cdrkit
<kfogel> https://bugs.launchpad.dev/ubuntu/+source/alsa-utils
<adeuring> kfogel: yeah, right
<kfogel> etc
<kfogel> and we opened bugtasks for some of those packages (on "Bug C", which has a patch attachment)
<kfogel> ok
<kfogel> so, https://bugs.launchpad.dev/ubuntu/+source/cdrkit/+patches should work!
<kfogel> adeuring: and it does work, heh
<adeuring> kfogel: great!
<kfogel> adeuring: if you have to go afk for 10-15, no worries.  I'm going to write a test for this in lib/lp/bugs/stories/patches-view/patches-view.txt.
<gary_poster> jml (and mars), staging was down for an update about when you asked, yes
<mrevell> nytol
<mars> EdwinGrubbs, fair enough.  My point was that image reuse is not the reason that you use sprites.
<EdwinGrubbs> mars: it just seems strange to avoid an opportunity to use DRY if there isn't a drawback that we aren't already incuring with the 2 line solution.
<kfogel> adeuring: when looking at patches view on a distro series (e.g., https://launchpad.dev/ubuntu/hoary/+patches), we *should* have the package column, right?  (Currently we don't.)
<kfogel> (and I think I know how to fix it, I just want to make sure it is something that needs fixing)
<adeuring> kfogel: yes, I think that would be useful
<kfogel> adeuring: *nod*
<adeuring> kfogel: BTW, we should similary show the project name on the +patches view for project groups
<dobey> leonardr: re: #512552 the OOPS I just got in HTTPError.content id is: OOPS-1487EA832
<mup> Bug #512552: Large POST fails with createComment on merge proposal <Launchpad Bazaar Integration:Incomplete by leonardr> <https://launchpad.net/bugs/512552>
<leonardr> dobey: so it looks like the size of the post caused the database to timeout?
<dobey> leonardr: it would appear so, yeah
<kfogel> adeuring: oh, good idea!
<kfogel> adeuring: hmm, I think we already should show individual targets in project groups; I just added the "IDistroSeries" in this snippet, but the "IProject" was already there: http://paste.ubuntu.com/363372/   ...or are you saying that we should show product instead of target package, or that we should show both?
<adeuring> kfogel: sorry, i should have looked what we already have for project gruops. I menat that we should display the projects in this case.
<kfogel> adeuring: instead of packages?
<kfogel> adeuring: (I'm a little confused by the "target" terminology.  Target seems very generic to me -- could refer to whatever the bug(task) is attached to.  But we seem to always use it to refer to a package, in practice.  Am I missing something?)
<adeuring> kfogel: I think we don't have packages for projects.
<kfogel> adeuring: where "projects" means "project groups"?
<adeuring> kfogel: yes, bugtargets can be a bit confusing ;) And I meant the we should should on the patches view for project groups (IProjects) the individual projects (IProducts) in the traget column
<adeuring> s/should should/should show/
<kfogel> adeuring: gotcha.  note that we don't label any column "target"; we label it by what entity it is.  So right now it's "package" for most views (or it's absent), but for the IProject view, it will say... well, not "Product", because users don't know that word, but "Project" (meaning IProduct).
<adeuring> kfogel: exactly
<kfogel> adeuring: so currently, in lib/lp/bugs/stories/patches-view/patches-view.txt, I want to associate a bugtask (for one of the bugs we made that has a patch attachment) with a series like https://launchpad.dev/ubuntu/hoary.  I'm using lib/lp/bugs/stories/bug-release-management/40-nominate-bug-for-productseries.txt as a guide.  I'm being very, very verbose about what I'm doing, because I badly wish I were sitting next to you :-).
<adeuring> kfogel: looks good. And I think we can set up virtual desks if necessary ;)
<kfogel> (hunh... I guess one nominates a bug for a series directly; it doesn't require a separate bugtask.  The question of where bugtasks do and don't apply can be a bit confusing).
<adeuring> ...virtual common desks...
<kfogel> adeuring: that virtual desks thing might be a good idea.  so far I'm not blocked, but just in case: how does one do it?
<adeuring> kfogel: no idea in practice. But skype calls for example are sometimes auite useful
<adeuring> Or using gobby
<adeuring> kfogel: actually, I think you don't needs to go through the nomination cycle. Don't we have a factory methods for bugtasks that allows us to dircetly create a task for a product series?
<kfogel> adeuring: let me see
<kfogel> adeuring: we wrote our own make_bugtask()  (meta-factory method for make_thing(), if you recall).  make_bugtask() calls factory.makeBugTask()... let me take a look at that.
<adeuring> kfogel: I think there is an odd restriction that you can't directly create a bug for a product series. But if you have an existing bug, you can add a task for a product series.
<adeuring> kfogel: and we used the helper function make_bug in order to set the importance ad the status
<kfogel> adeuring: indeed.  (although I have it working the other way already, so unless there's a preference, I could just stick with this)
<adeuring> kfogel: well, if it works, things are fine.
<mwhudson> good morning
<dobey> leonardr: did that OOPS provide any useful info for you?
<leonardr> dobey: only to the extent that it confirms my belief that it's not my fault
<leonardr> i'll add a comment
<dobey> ok
<jamalta> is the bugs subdomain broken on edge right now?
<thumper> morning
<dobey> jamalta: broken how?
<sinzui> bac: is https://edge.launchpad.net/sachco really proprietary?
<jamalta> dobey: i get an oops every time i try to load a bugs page
<jamalta> not the case anymore, so i guess it fixed itself
<dobey> jamalta: individual bugs work fine. i think it's timing out for any lists of bugs pages, afaict
<jamalta> dobey: oh right, it is
<sinzui> jamalta: You can disable edge when search bugs, or write very targeted bug searches
<jamalta> sinzui: i disabled edge for now
<jamalta> i just wanted to see if it was an issue that affected everyone or just me
<bac> sinzui: nah, i just needed a test project
<mwhudson> biab
<dobey> leonardr: so the web page is timing out talking to the db as well, with similar content size in the comment.
<leonardr> dobey: well, that lets me off the hook
<leonardr> either launchpad needs to be made faster, or the timeout time needs to be made proportional to the incoming payload
 * dobey votes both
<thumper> that oops from the api has 46s of non db time
 * thumper wonders what it is donig
<mwhudson> biab, again
<jml> gary_poster, yeah, thanks. it interrupted my UDW preparataions, so it was a bit surprising
 * jml gone
#launchpad-dev 2010-01-27
* gary_poster changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.01 | 10.01 is releasing Wednesday 27th Jan at 0900 UTC | PQM is closed | gary_poster is release manager, with flacoste for the actual release | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes | Channel logs: http://irclogs.ubuntu.com/
<Ursinha> hello launchpad people
<mwhudson> Ursinha: hello
<Ursinha> mwhudson: how are you in the future?
<mwhudson> Ursinha: suffering terrible internet
<mwhudson> Ursinha: how are you?
<mwhudson> (we're moving house tomorrow and aaaaaaarghdsadasdasda the stress)
<Ursinha> mwhudson: I'm fine, thanks. Internet is working at least - despite of all thunderstorms we're having
<Ursinha> mwhudson: gah
<Ursinha> mwhudson: worst part of moving *ever*
<Ursinha> the internet
<mwhudson> Ursinha: my isp disconnected me early
<mwhudson> so i'm on 3g
<Ursinha> mwhudson: I saw on identi.ca :/ they did the same to me when I moved back a few months ago
<mwhudson> but on the upside, the new house _should_ be connected already
<Ursinha> mwhudson: oh, and does it work well? here in brazil it kinda sucks
<mwhudson> Ursinha: it kinda sucks, it's about dial up speed
<Ursinha> holy cow
<Ursinha> here it's a bit faster, but unstable :/
<Ursinha> mwhudson: hopefully you'll have internet in your new house :)
<mwhudson> internet from the phone itself is usually much quicker, i think the e63 is just a crappy modem
<Ursinha> mwhudson: hm
<Ursinha> I have one e71 that works well
<mwhudson> that has hdpsa wotsit thing though, i think?
<Ursinha> mwhudson: no idea
<mwhudson> i should steal my wife's 5800 and try with that, maybe
<Ursinha> mwhudson: I'd do that :P
 * mwhudson attempts bzr push over this connection...
<mwhudson> thumper: ping
<thumper> hey
<mwhudson> thumper: random pre-imp type questoin
<thumper> shoot
<mwhudson> thumper: do you think we want to display a different icon for "successful import but no changes" case on a code import page?
<thumper> we could, does it matter?
<thumper> I think that is the question
<thumper> I'd ask beuno or noodles
<thumper> is it something you'd care about if you were looking at an import?
<mwhudson> i dunno, i think it might be interesting to see that 4 of the last 5 imports actually imported revisions
<mwhudson> i'll ask beuno/noodles at some point i guess
<thumper> ok
<mwhudson> it's not a hard decision to change (he says optimistically)
<mwhudson> huh, the tests in this area are actually pretty good
<mwhudson> that's a pleasant discovery :-)
 * mwhudson stabs zpts
<thumper> :)
<Ursinha> hey mwhudson, do you mind setting the milestone/marking as Fix Released if appliable, bug 432830 and bug 444738? please :)
<mup> Bug #432830: rocketfuel-setup does not work <build-infrastructure> <Launchpad Foundations:Fix Committed by mwhudson> <https://launchpad.net/bugs/432830>
<mup> Bug #444738: ec2 land doesn't kill instance when update-sourcecode fails <build-infrastructure> <Launchpad Foundations:Fix Committed by mwhudson> <https://launchpad.net/bugs/444738>
<Ursinha> I'm cleaning up old releases
<mwhudson> Ursinha: i'm not sure what the milestone should be for 432830, i think it's already closed
<mwhudson> Ursinha: but otherwise, done, or will be when the bits manage to get there
<Ursinha> thanks mwhudson
<thumper> OMG, this takes longer than I thought
<thumper> mwhudson: https://code.edge.launchpad.net/~thumper/tuolumne-lp-configs/new-branches-per-day/+merge/18115
<mwhudson> thumper: i understand why you're doing the count(*)/28 stuff, but i'm not sure that's obvious in the labels you've given
<thumper> mwhudson: those were the existing ones I think
<mwhudson> oh right
<thumper> well
<mwhudson> it looks basically ok i guess
<thumper> some were
<mwhudson> i'm not familiar with the code :)
<thumper> mwhudson: and all that is just for new branch owners
<thumper> I've almost lost the will to write more
 * thumper looks at code imports
<mwhudson> thumper: i'm going to go and do the last bits of packing now, so see you on monday!
<mwhudson> i'll probably pop in over the next couple of days to check the internet works in my new place at least :-)
<thumper> mwhudson: probably not, I'll be in London
<mwhudson> thumper: bzr+ssh://bazaar.launchpad.net/~mwhudson/launchpad/less-import-requestMirrors-bug-487357 fwiw
<mwhudson> thumper: oh right
<thumper> mwhudson: I'll take a look
<mwhudson> (branch functionally complete, not stylistically ready really)
<thumper> ok
<mwhudson> thumper: i hope all the travel isn't too soul destroying
<thumper> me too
 * thumper EODs
* spm changed the topic of #launchpad-dev to: Launchpad in read-only 09:00 UTC - 11:30 UTC |Launchpad Development Channel | Week 4 of 10.01 | 10.01 is releasing Wednesday 27th Jan at 0900 UTC | PQM is closed | gary_poster is release manager, with flacoste for the actual release | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes | Channel logs: http://irclogs.ubu
<jml> good morning
<jtv> hi jml!
<al-maisan> Good morning jml and jtv :)
<jtv> hi al-maisan!
<jtv> al-maisan: and while we're here, please note I requested a small review from you.  :-)
<jtv> I know you're busy though, and it's not like we're landing it this week...
<al-maisan> jtv: sorry for the delay .. will look at it today.
<jtv> al-maisan: thanks!  It's basically my work keeping up with yours.
<al-maisan> I see
<jtv> My next topic to work on is the Behavior work to get files back to the server.
<al-maisan> jtv: I remember starting on this before the candidate job selection logic flew apart
<al-maisan> but did not get very far unfortunately
<wgrant> It's reasonably well generalised, but you need to implement a lot of stuff yourself since you don't have a *Build
<jtv> al-maisan: right...  anyway it was never fair to make you do it.  :)
<jtv> wgrant: you're stirring up useful memories there.  There was some attribute on the Build object that we were expected to provide.
<al-maisan> jtv: heh, we were all part of the same gang in Wellington :)
<wgrant> jtv: There is code that assumes that your IBuildFarmJob implementation has a 'build' attribute that provides IBuildBase.
<wgrant> You will probably want to override that, since you don't seem to want an IBuildBase.
<jtv> right
<jtv> I think one option we discussed was to have an IBuildBase that wasn't a Build.
<wgrant> We do already -- SourcePackageRecipeBuild
 * jtv notes with joy that IBuildBase is under buildmaster, not soyuz
<noodles775> wgrant,jtv, it's IBuildFarmJobBehavior.updateBuild (or specifically, on the base-class implementation, specifically updateBuild_WAITING) that needs to be overridden right?
<wgrant> noodles775: And an enormous web of stuff under there, yeah.
<wgrant> But updateBuild is the root of it all.
<al-maisan> jtv: buildmaster is Soyuz in disguise .. hehe :)
<wgrant> There is some stuff in Soyuz that needs to be moved out.
<wgrant> In particular buildqueue.
<wgrant> And the recipe stuff needs to move to code.
<jtv> al-maisan: I find that an intensely disturbing thought...
<al-maisan> jtv: sorry :)
<al-maisan> jtv: please do take note of the smiley as well :)
<jtv> al-maisan: your enjoying the pain you inflict on me is singularly failing to lift my spirits.  :-)
<al-maisan> oh well
 * al-maisan shuts up
<adeuring> good morning
<al-maisan> Good morning adeuring
<adeuring> hi al-maisan!
<wgrant> Why the oddly timed rollout?
<wgrant> So people are around for when Soyuz explodes spectacularly?
<jml> wgrant, no one has told me nothing.
<jml> flacoste, bonjour!
<al-maisan> wgrant: so far it's been pretty well behaved on mawson
<flacoste> morning jml
<wgrant> al-maisan: Yeah, I've exercised lots of paths here, and it seems to work.
<al-maisan> yep, let's see how it holds up post-rollout :)
<jml> 14 minutes!
<jml> gror
<al-maisan> ?
<jtv> adiroiban: I have a branch here that backs out the problem branch... https://code.edge.launchpad.net/~jtv/launchpad/bug-512698
<jtv> I'm working to get that landed this week.
<jtv> Well, rolled out.
<jml> wgrant, well, you see, we knew that poolie would be sprinting in Europe
<jml> wgrant, and we time the rollouts to disrupt him.
<wgrant> jml: Ah, of course.
<mrevell> Morning
<noodles775> Morning mrevell
<al-maisan> Good morning mrevell
<adiroiban> jtv: hi. ok. Is there anything I should do?
<jtv> adiroiban: I don't think so...  I'm fairly confident that I got the reversal right, otherwise I'd ask for help figuring out whether I backed out the right branch etc.  But as I said, no real worries there.
<adiroiban> ok
<ChatMangerChien> hi all
<ChatMangerChien> I am currently working with my own copy of launchpad, on my private network, in order to test it
<ChatMangerChien> I would like to know how to configure the mail settings
<ChatMangerChien> because this page does not really hepl
<ChatMangerChien> https://dev.launchpad.net/Mail
<ChatMangerChien> maybe it's outdated
* mthaddon changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.01 | 10.01 is releasing Wednesday 27th Jan at 0900 UTC | PQM is closed | gary_poster is release manager, with flacoste for the actual release | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes | Channel logs: http://irclogs.ubu
<noodles775> jml, wgrant, if you've got a minute, I've just done a diagram of the recipe tables... let me know if you see any oversights etc.: https://dev.launchpad.net/BuildBranchToArchive#Current db-devel schema
 * wgrant was confused for a moment by the 'SPR' as used there. It's ambiguous now :(
<jml> noodles775, it's got rendering bugs for me
<wgrant> Me too.
<jml> noodles775, does sprbuild actually link to sprdata? I thought it linked to spr?
<noodles775> wgrant: yeah, I'll stop using SPR as an acronym.
 * jml won't :P
<wgrant> SPRBuild should link to SPRData, since the SPR's SPRData may change.
<noodles775> The rendering bugs are just in the text right? (not sure why dia does that)
<wgrant> SPRBUpload is dead for now, too.
<noodles775> yeah, I just put SPRBUpload in even though we're not yet using it (so we remember to either do so or delete it)
<jml> noodles775, what do you think about including more of the Soyuz build & job stuff in the diagram?
<wgrant> At least SPRBJob should be there.
<wgrant> The other stuff is fairly general.
<jml> yeah, but it would help provide context
<jml> it doesn't have to include full column listings.
<noodles775> jml: Sure, I can add SPRBJob, but I'm not too keen on adding all the other soyuz-specific stuff.
<jml> noodles775, btw, I haven't said yet, but thanks so much for doing this :)
<wgrant> Rather like the diagram that filled one side of the board?
<jml> noodles775, any particular reason?
<noodles775> np, I needed to do it to sort out in my head the options for the ui work.
<jml> oh right.
<noodles775> jml: well, mainly because I want to get on with the UI work :)
<jml> noodles775, actually, could we have a call about the UI work some time?
<jml> noodles775, fair enough
<noodles775> jml: yeah, that'd be great. Maybe tomorrow? Just so I've got a chance to get my head around all the possibilities etc.
<jml> noodles775, fo shizzle
<noodles775> oh, and jml, wgrant, any ideas about the question I've got there (redundancy of sourcepackagename)
<wgrant> noodles775: I wondered that myself.
<wgrant> It doesn't seem relevant, unless a SPRecipe can move around.
<wgrant> Which seems unlikely.
<noodles775> ok, I'll create a bug. Thanks wgrant.
<ChatMangerChien> is anyone there ?
<ChatMangerChien> Does anyone know how to configure mail system on launchpad?
<jml> what do you mean "mail system"?
<ChatMangerChien> launchpad does not send email to the outside world
<ChatMangerChien> when someone try to register for exemple
<ChatMangerChien> the mail is sent to root@localhost
<ChatMangerChien> i would like to configure that to send mail to the registrating user
<al-maisan> ChatMangerChien: is that a local launchpad installation that you're talking about?
<ChatMangerChien> yep
<al-maisan> ChatMangerChien: is your local MTA (exim/postfix etc.) set up properly>
<al-maisan> ?
<ChatMangerChien> yep, postfix is weel configured
<ChatMangerChien> mail are sent when using command line tool (mail)
<ChatMangerChien> but launchpad sent all mails to root@localhost
<al-maisan> .. and you'd like them to be sent somewhere else?
<ChatMangerChien> Yep, when a user is registrating, he gives his mail in order to receive information about the registration, but all mails are sent to root
<al-maisan> ChatMangerChien: try tweaking one of these
<al-maisan> configs/development/launchpad-lazr.conf
<al-maisan> 246:default_sender_address: root@localhost
<al-maisan> 247:default_recipient_address: root@localhost
<ChatMangerChien> ok, trying
<al-maisan> ChatMangerChien: .. and there's also:
<al-maisan> package-includes/mail-configure-normal.zcml
<al-maisan> 13:        name="stub" from_addr="root@localhost" to_addr="root@localhost"
<ChatMangerChien> ok, but if I change root@localhost to anything@else, mails will  be sent to the anything@else ?
<al-maisan> ChatMangerChien: I guess so, give it a whirl.
<ChatMangerChien> al-maisan: I changed in 'configs/development/launchpad-lazr.conf' : the mail is still sent to root@localhost, now trying with mail-configure-normal.zcml
<al-maisan> ChatMangerChien: did you re-start launchpad after the change?
<ChatMangerChien> al-maisan: oups :), you're right, forgot to restart
<ChatMangerChien> even with restarting, changes in launchpad-lazr.conf do not change mail recipient, now trying with mail-configure-normal.zcml
<ChatMangerChien> al-maisan: changes in the mail-configure-normal.zcml file is working,  now mails are sent to anyone@else
<al-maisan> ChatMangerChien: great! Glad it works for you :)
<ChatMangerChien> al-maisan:  yep but mails are not sent to the recipient I entered in the web page
<ChatMangerChien> ...
<ChatMangerChien> I was looking at the line with: name="stub" from_addr="root@localhost" to_addr="root@localhost", is there any way to change root@localhost in the to_addr in order to take into account the value send on web pages
<al-maisan> hmm.. sorry .. I wouldn't know how that works .. maybe one of the Launchpad registry people?
<ChatMangerChien> ok thank you
<ChatMangerChien> al-maisan: I finally succed in sending mails ! I changed:
<ChatMangerChien> <mail:stubMailer
<ChatMangerChien>         name="stub" from_addr="root@localhost" to_addr="arnaud@mechantchien.com"
<ChatMangerChien>         />
<ChatMangerChien> to:
<ChatMangerChien> <mail:smtpMailer
<ChatMangerChien>            name="sendmail"
<ChatMangerChien>            hostname="localhost"
<ChatMangerChien>            port="25"
<ChatMangerChien>            />
<al-maisan> ChatMangerChien: cool!
<al-maisan> Thanks for the feedback!
<ChatMangerChien> ty for all :)
<sinzui> I am very depressed to be seeing timeouts creating a release on edge. I many releases on staging (with a slower db) without incident
<bac> sinzui: chat?
<sinzui> bac: sorry: I need to prepare a branch to revert releases
<bac> sinzui: np.  we can do it later or tomorrow
<sinzui> Yes, latter will be good
<xnox> Hello all! i'm having trouble staring launchpad instance with "make run"
<xnox> See http://pastebin.ubuntu.com/363969/
<xnox> Also rocketfuel-get compalains that there is no "shipit" and "identification servers" but I hope that's optional / not opensourced yet
<salgado> xnox, you need to start postgresql
<salgado> and yes, it's ok to ignore the shipit warning
<xnox> Ok thank you. I'll fix postresql instalation then
<xnox> I can't manage to write to /var/log but that's because i've messed about with that
<kfogel> adeuring: http://paste.ubuntu.com/363991/
<kfogel> intellectronica: (see above paste, not sure if adeuring is still around)
 * intellectronica looks
<intellectronica> kfogel: first you need to login($user_email)
<intellectronica> where $user_email is prolly foo.bar@canonical.com
<kfogel> intellectronica: let me show you the whole file, hold on
<kfogel> http://paste.ubuntu.com/363995/
<kfogel> OH
<kfogel> intellectronica: you're right
<kfogel> I see the problem now, no worries
<kfogel> I removed that line from the context where it would be (later) scoped into the user login.
<kfogel> I went one level too meta for my own good.
<intellectronica> i know the feeling
<adeuring> kfogel: sorry, I was afk...
<kfogel> adeuring: np, tom got me out of the pickle
<deryck> boo-yah!  I can reproduce the +filebug NotImplementedError.
<sinzui> bac: salgado: I am working on a branch moves the rules to change bugs "fix committed" to "fix release" to a new  method because this is too much work to do in the request.
<sinzui> bac: salgado: I am tempted to export this new method, but that may be stupid because it will also timeout if API requests are not permitted to run longer lengths of time
<sinzui> bac: salgado: do either of you know if API requests timeouts are handled differently
<bac> sinzui: i do not know for certain.
<bac> i'll bet leonardr does
<leonardr> sinzui: i'm pretty sure they're not. see bug 512552
<mup> Bug #512552: Large POST fails with createComment on merge proposal <Launchpad Bazaar Integration:Triaged by leonardr> <Tarmac:New> <https://launchpad.net/bugs/512552>
<sinzui> leonardr: thanks. I will stop my insanity now
<Ursinha> Edwin-lunch: when you return: I have a list of five bugs assigned to you, Fix Committed and targetted to 3.1.12, that I guess can be marked as Fix Released
<Ursinha> Edwin-lunch: bugs 433809, 494540, 495067, 230801, 297833
<Ursinha> eh, the bot doesn't want to play
<Ursinha> bug 433809
<mup> Bug #433809: project series search fails if it contains a slash <package-link> <Launchpad Registry:Fix Committed by edwin-grubbs> <https://launchpad.net/bugs/433809>
<Ursinha> hello mup
<Ursinha> Edwin-lunch: bug 494540, bug 495067, bug 230801, bug 297833
<mup> Bug #494540: code cleanup <Launchpad Registry:Fix Committed by edwin-grubbs> <https://launchpad.net/bugs/494540>
<mup> Bug #495067: move remaining windmill tests into lp/registry/windmill directory <Launchpad Registry:Fix Committed by edwin-grubbs> <https://launchpad.net/bugs/495067>
<mup> Bug #230801: AssertionError triggered renewing team membership <oops> <registry> <Launchpad Registry:Fix Committed by edwin-grubbs> <https://launchpad.net/bugs/230801>
<mup> Bug #297833: Attempting to invite a private team fails with "Constraint not satisfied" <oem-services> <oops> <Launchpad Registry:Fix Committed by edwin-grubbs> <https://launchpad.net/bugs/297833>
<bac> sinzui: got a few minutes for a quick call?
<sinzui> I will in a few minutes I am on a call
<bac> sinzui: i'll have to do it later.
<wgrant> One day of notice on just the blog before a vast data deletion doesn't seem like a very good idea.
<wgrant> (I saw a lot of data being removed from the librarian two weeks ago, so it cannot have been /that/ urgent)
<kangarooo> hello bug https://bugs.edge.launchpad.net/launchpad-registry/+bug/113564/ Launchpad  should be able to resize user uploaded images i added some possibilities
<mup> Bug #113564: Launchpad should be able to resize user uploaded images <feature> <Launchpad Registry:Triaged> <https://launchpad.net/bugs/113564>
<cody-somerville> wgrant, Has it already happened?
<wgrant> cody-somerville: According to the blog post, yeah.
<wgrant> ALthough it doesn't seem to have been done.
<bac> sinzui: free?
<sinzui> yes
<bac> hoorah
<cody-somerville> wgrant, Hopefully when they do get around to doing it they skip P3As.
<wgrant> cody-somerville: I believe they do.
<wgrant> Yeah, private archives and a few public ones are excluded.
<EdwinGrubbs> Ursinha: yes, those bugs are fix commited. I changed there status. Sorry, I didn't get back to you sooner.
<Ursinha> EdwinGrubbs: no problem! thanks for doing that :)
<EdwinGrubbs> rockstar: do you know what would be causing an oops when loading this mp? https://code.edge.launchpad.net/~leonardr/lazr.restful/generate-multiversion-collections/+merge/18153
<rockstar> EdwinGrubbs, I don't get an oops on that page.
<EdwinGrubbs> rockstar: hmmm, that's weird that just started working again. It's been oopsing for a good 30 minutes.
<rockstar> EdwinGrubbs, huh.
<EdwinGrubbs> I guess I'll just wait and see if it happens again.
#launchpad-dev 2010-01-28
<EdwinGrubbs> rockstar: it looks like it is oopsing still. about 50% of the time.
<rockstar> EdwinGrubbs, got it to OOPS, and I have nfi what's going on.  That's really odd.
<rockstar> EdwinGrubbs, looks like a problem with the librarian.
<kfogel> who wants to do an _absolutely trivial_ merge proposal review?
<kfogel> https://code.edge.launchpad.net/~kfogel/launchpad/cc-script-2010-01-28-updates/+merge/18183
<kfogel> barry: ayt?
<kfogel> thumper: (see above -- trivial review invite :-) )
<cody-somerville> kfogel, I reviewed it for you. ;)
<adiroiban> jtv: Hi. Looks like the branch from bug 359180 was release, even thou the tag was qa-bad
<mup> Bug #359180: Missing keyboard shortcut for navigation <qa-bad> <trivial> <ui> <Launchpad Translations:Fix Released by adiroiban> <https://launchpad.net/bugs/359180>
<jtv> adiroiban: ah, I now realize I neglected to update you on this: it was too late to do anything about it in the original rollout, so we've arranged for the fix to go out with a re-roll.
<adiroiban> jtv: np. I just wanted to make sure everything is ok
<jtv> adiroiban: it's a shame we found out about the problems so late, but no disasters.  It does illustrate how far we've come in terms of qualityâa few years back we'd have worse things than this all the time.
<jtv> adiroiban: I think the main culprit in this case is the confusion at PQM close...  it was known well in advance that something needed fixing, but not that PQM would close in that stateâafter the bad branch landed but before the one that fixed it.
<adiroiban> jtv: yes. there is no test for loading a +translate page for a language with no plural forms
<jtv> Well, not for the single-message viewâwhich is where the oopses happened.
 * jtv looks for any pagetests for this case
<jtv> adiroiban: looks like you dug up a hidden body there.  The _browser code_ is well-tested for this case, but we don't have the integration-testing we'd get from a story.
<adiroiban> I have not looked for pagetests... but I assume since we had one it would fail as a simple HTTP get is enought to  triger the OOps
<jtv> adiroiban: a valid assumption in theory, but the problem with improving practices all the time is you're always carrying around code from back when practices weren't as good.  I've filed bug 513625.
<mup> Bug #513625: Pagetest for translations without plural-forms information <Launchpad Translations:New> <https://launchpad.net/bugs/513625>
<adiroiban> jtv: ok. I was planning to write this test togheter with the fix for bug 359180.
<mup> Bug #359180: Missing keyboard shortcut for navigation <qa-bad> <trivial> <ui> <Launchpad Translations:Fix Released by adiroiban> <https://launchpad.net/bugs/359180>
<jtv> adiroiban: <hug>
<adiroiban> new bug assigned to me :)
<noodles775> Morning
<jtv> hi noodles775
<jtv> wgrant: can you point me to the branch that implements SourcePackageRecipeBuild.build?
<jtv> wgrant: it's not the one I looked at, out of the myriad you have lying around.  :)
<noodles775> I obviously missed the start of this conversation, but SourcePackageRecipeBuild *is a* build (inheriting BuildBase), why would it have a build attribute?
<noodles775> (hi jtv :) ).
<jtv> noodles775: ah sorry, I copy-pasted.  I meant SourcePackageRecipeBuild itself, which would be referred to as something the lines of SourcePackageRecipeBuildJob.build.
<noodles775> jtv: and it is? (looking at db-devel, SPRBJob.build exists?
<jtv> noodles775: gah, I was grep'ing devel.  Silly me.  Thanks.
<noodles775> np!
<jtv> henninge: we got an assertion failure, as far as I can make out because someone tried to post translations while we were in read-only.
<jtv> henninge: I'm filing a bug about it now, unless you already had.
<henninge> jtv: hm, how could they have done it?
<henninge> jtv: no, I have not
<jtv> henninge: load the page before RO mode, submit after.
<jtv> Actually I think that's probably a generic problem.
<henninge> Ah, I see.
<henninge> yes
<jtv> Maybe LaunchpadFormView should check for that as part of standard validation.
<jtv> goedemorgen jelmer!
<jelmer> mogguh :-)
<jtv> jelmer: I looked at the Tuesday LCA schedule the other day and found that _all_ the presentations I had really wanted to see in other rooms overlapped with mine!
<jtv> stub, question about read-only mode: do we have any kind of generic check in form validations to deal with submissions during RO spells?
<stub> any post generates a 'you can't do that at the moment' error page. Any page that tries to write to the db generates the same page. Any page requiring anything but read permission generates the same page.
<stub> So you shouldn't be able to get to the form as it should be protected with Edit or similar permission, and if you do, you can't submit it since we use POST for our forms, and if you could, it couldn't make any database changes.
<stub> As far as I can recall anyway :) Rules are in lib/canonical/launchpad/webapp/publisher.py I think.
<adeuring> good morning
<jtv> stub: thanks...  it does make this oops I'm looking at a bit of a mystery though  :)
<jtv> hi adeuring
<adeuring> hi jtv!
<jtv> stub: oh, I see read-only state seems to be signaled outside of the db... in which case it may appear after validation and during processing of the request.
<stub> Which raises an exception, that it rendered as a pretty Try Again Later error page.
<jtv> Which is fine, except it shows up as an oops.
<jtv> I think we shouldn't be failing this assertion just because we went into read-only mode.
<jml> good morning launchpadders
<jtv> good morning jml
<jml> noodles775, I've just looked over the BuildBranchToArchiveUI page -- good stuff! when would you like to talk about it?
<noodles775> jml: now?
<noodles775> jml: or in 5mins?
<jml> noodles775, 10 minutes :)
<noodles775> great.
<stub> Yay segfault trying to activate the Firebug console
<mrevell> Hi
<jml> mrevell, hello.
<mrevell> Hi there jml
<jml> noodles775, what's your skype id?
<jml> noodles775, http://bazaar.launchpad.net/~ubuntu-branches/ubuntu/lucid/firefox-3.5/lucid/files
<jml> noodles775, http://bazaar.launchpad.net/~ubuntu-branches/ubuntu/lucid/python-launchpadlib/lucid/files
<jml> noodles775, http://pastebin.ubuntu.com/364454/
<noodles775> jml: https://edge.launchpad.net/~michael.nelson/+archive/pocketsphinx/+builds?build_text=&build_state=all
<jml> mrevell, hi
<mrevell> hey there jml, same number as yesterday?
<jml> mrevell, yes please
<jtv> wgrant: something I don't get about Build._handleStatus_OK: if any of that long, complex method fails, it never gets to buildqueue_record.builder.cleanSlave() or buildqueue_record.destroySelf().  Doesn't that leave garbage?
<jtv> ...did we just get spammed?
<gmb> jtv: Heh. That seems to be the case, yes.
<jtv> I wonder if we have a way of getting it out of our IRC logs?
<gmb> jtv: No idea. The logs live on canonical servers, though, so I guess IS would be able to help.
 * jtv trundles in that general direction
<jtv> elmo suggests: "consider making launchpad-dev +r or +R"... anyone who understands that as something else than a chmod option?
<jml> yes
<jml> but I need chanop to do anything about it
<jml> oh look
<jml> That might well have worked.
<wgrant> jtv: If it fails, the status will not be set, so it will just be reprocessed.
<deryck> Morning, all.
<jml> deryck, good morning.
<adiroiban> danilos: Hi, can I continue with bug 340664 ? or you think we should wait to fix the +template timeouts ?
<stub> So I'm generating reports from the zope logs for performance analysis. I'm now confused where things should live.
<stub> Should the script live in scripts or utilities (I think scripts, but I've confused myself).
<stub> And that script imports the bulk from a module in lib/lp/scripts/foo.py or lib/lp/scripts/???/foo.py or what?
<jml> stub, I had thought that scripts was for actual scripts executed as part of Launchpad's normal operations
<jml> stub, and that utilities is for stuff that developers & admins use.
<jml> but, tbh, until someone writes an actual guide... meh.
<stub> jml: I think I'm confused because I don't know if this is mainly adhoc or going to be automated ;)
<jml> stub, ahh ok.
<jml> stub, put it in utilities and move it when you automate it?
 * stub shrugs. Its more admin related anyway so it can probably stay in utils
<jml> sounds good to me.
<kfogel> jml: want to do an utterly trivial code review?  https://code.edge.launchpad.net/~kfogel/launchpad/cc-script-2010-01-28-updates/+merge/18183
<kfogel> intellectronica, beuno: https://code.edge.launchpad.net/~kfogel/launchpad/506018-patch-report/+merge/18181  when you have time (I'm not blocked on it landing).
<beuno> kfogel, I have the tab open, I will get to it within the next few hours
<kfogel> beuno: awesome, thanks
<intellectronica> kfogel: cool. i'll review this latest after standup
<kfogel> intellectronica: that'd be great, thank you.  Note it doesn't have your windmill tests yet, because I want to look at them / grok them / maybe extend them first.  So I'll include them in another branch.
<kfogel> jml: please ignore above, cody somerville reviewed it
<intellectronica> kfogel: sure. and 'my windmill tests' are nothing but a stub, as i left them on the last day of the sprint. they still need heaps of work
<jml> kfogel, ok. I missed it anyway
<kfogel> intellectronica: *nod*
<intellectronica> anyone remembers the trick to make storm spit out the sql it's running?
<intellectronica> ah, the answer is in https://dev.launchpad.net/Debugging#Tracing SQL statements through STORM :)
<intellectronica> and now i can't run lp. i get the following error: http://pastebin.ubuntu.com/364635/
<intellectronica> any idea what's going on and how to fix>
<intellectronica> barry: maybe you know? ^^^
 * barry looks
<barry> intellectronica: do a 'make clean' then 'make'
<barry> intellectronica: it looks like your master-qrunner.pid file got corrupted perhaps
<barry> intellectronica: do a 'make clean' then 'make'
<intellectronica> barry: yes, you're right, the pid file is empty!
<intellectronica> the gnomes ate my pid
<barry> tasty!
<sinzui> gary_poster: I can now speculate with some certainty why closing bugs while making a release passed on staging, but not in production
<gary_poster> sinzui: really, I'm definitely curious
<sinzui> gary_poster: I was able to use the feature to create releases for all the launchpad projects yesterday because some developers closed the bugs...reducing the number of bugs that needed updating...
<sinzui> gary_poster: staging's data is more than two weeks old, so it has fewer bugs in the fix committed state.
<gary_poster> sinzui: ah-ha.  So that's another data update issue to some degree, though it's also indication of the fact that QA at a given point in time is never entirely indicative of actual performance at another point in time
<sinzui> agreed
<intellectronica> barry: strangely, after running make clean and make i get: http://pastebin.ubuntu.com/364646/
<intellectronica> what's supposed to start it?
<barry> intellectronica: make run_all
<barry> intellectronica: but that may be misleading output.  look higher up (might need SHHH= ) for some other error
<intellectronica> it looks like it's unhappy about something called qrunner not running
<barry> intellectronica: what it's really saying is that something else went wrong and when killservice tried to kill mailman, it wasn't started.  that is normal output in that case (i.e. trying to shutdown mailman when it hasn't been started)
<intellectronica> oic
<matsubara> Chex, rockstar bigjools danilo sinzui allenap: LP production meeting in 3 min @ #launchpad-meeting
<al-maisan> matsubara: bigjools is sick
<al-maisan> I'll stand in for him
<matsubara> thanks al-maisan
<al-maisan> you are welcome :)
<barry> intellectronica: ping me by nick if you have more information.  i'm minimizing my irc window
<jml> flacoste, hi
<EdwinGrubbs> beuno: ping
<beuno> EdwinGrubbs, hi
<EdwinGrubbs> beuno: nm, I think I answered my own question. I couldn't find individual images matching sprites, but that is probably just due to images being re-used for multiple css classes.
<EdwinGrubbs> beuno: Is there any existing template that generates the css, or do you just add them by hand based on the existing sprite file?
<bac> barry: ping
<kangarooo> https://bugs.launchpad.net/bugs/297239 maybe this can be solved?
<kangarooo> ups sory- its now reported as done
<beuno> EdwinGrubbs, by hand, yes
<beuno> anyone from the bugs team around?
<beuno> gmb, intellectronica, allenap?
<beuno> I have questions  :)
<EdwinGrubbs> beuno: do you know anything about the bzr-favicon or person-tabs css classes. Those are the only two that don't have matching files, and they don't appear to be used anywhere.
<beuno> EdwinGrubbs, they sounds like left over cruft
<EdwinGrubbs> I'll add them to the garage sale
<beuno> no bug heat in sample data?
<beuno> sinzui, I admire how well you handle the blueprints situation
<beuno> laying out the plan like that is really helpful
<beuno> shows the genuine willingness to help out
<wgrant> I agree -- comments like that one are very useful and encouraging.
<sinzui> beuno: I think I gave the same comment to statik
<sinzui> The work is daunting
<beuno> good to hear Canonical has so many great people  ;)
<sinzui> beuno: Does the u1 web team need private blueprints?
<sinzui> or public API?
<beuno> sinzui, I can't say for sure, but the requests came from U1 hackers
<sinzui> beuno: Those two issues (and structural subscriptions) do not have a lot of bug heat, but thoses are the 3 most requested blueprint features. Users ask me on sprints, irc, and private email for them
<beuno> sinzui, get them to me-too them!
<beuno> soon, it will be up there with wikis
<sinzui> No it wont. We need to invent something sexier than wikis. We seem to have missed the google wave
<beuno> wiki wave?
<sinzui> hmm
<jml> blueprints situation?
 * jml is interested
<sinzui> beuno: Whether you are editing in some annoying markup syntax or a gui, wikis are just bug chunks of pages in a tree. I do not like them because they take a lot of maintenance. I have pondered if these pages would be more useful if they were composed of smaller piese that could be reused (edit it section, not this page) and I could tag them to build alternate views hierarchies. Or even construct a page from a section with 
<beuno> sinzui, yes, I've heard your theory around chunks of data, and I think it has a lot of potential
 * ajmitch sees the word blueprints & hides
<beuno> smart man, ajmitch
<ajmitch> well I saw him mention API stuff & blueprints in the same sentence & am interested to know what people really want there
<sinzui> jml: we were discussing blueprint bug 208539. This lead to other pondering
<sinzui> ajmitch: Launchpad's reporting and search of blueprints is very poor, and the notifications are very poor too. Users want API access to build better reports and notifications
<ajmitch> ok, since I have a branch for bug 146389 which is pretty basic & I promised jml that I'd submit it
<jml> \o/
<sinzui> ajmitch: tags would help search, but reports are more complex. Many users want to see the bugs and the branch work in the same report
<jml> sinzui, I'm thinking of proposing something to get us using blueprints more.
<ajmitch> I looked at tags the other day & was worried at how tied to bugs they are
<wgrant> I still do not see a compelling reason for Bugs and Blueprint to be separate.
<ajmitch> making tags available across launchpad will be a little more work, I think
<sinzui> ajmitch: right, I would make bugtags generic first, instead of copying the implementation
<sinzui> wgrant: I still agree with you. I think a single good implementation would provide 90% of what bug or blueprints needs. Their differentiation is small I think.
<ajmitch> the few unique things about blueprints are to do with sprints at the moment
<wgrant> And the reviewer vs drafter vs assignee, and dependencies.
<ajmitch> dependencies are something I'd love to have in bugs
<wgrant> Particularly the latter is needed on bugs as well.
<thumper> bug dependancies would be good
<wgrant> They were promised at UDS Jaunty...
<thumper> haha
<sinzui> ajmitch: yes. Very few sprints are held to work on bugs, but sprints/meetings/conferences are certainly used by teams for other purposes that working on blueprints
<jml> wgrant: by whom?!
<sinzui> blueprints roles and statues are too complex. They prevent them from being widely adopted.
<wgrant> jml: I don't remember. But it was presented as being one of the big things that would most probably be coming in 3.0.
<sinzui> wgrant: that is an extraordinary promise given that blueprints has not had dedicated developers since 2007
<wgrant> sinzui: Bug dependencies -- little to do with Blueprints.
<jml> wgrant: it's always been a controversial feature, I have never heard anyone even close to promising it being delivered.
<sinzui> wgrant: sorry, lost my concentration
<ajmitch> jml: almost as controversial as having version information for bugs on packages as metadata, rather than text?
<sinzui> yet we hack dependencies every 6 months using the tags, blueprint-to-bug-links, and BjornT_'s progress grid
<jml> ajmitch, I've never heard of that one.
<jml> ajmitch, wgrant: there's a bug in malone about dependencies
<intellectronica> beuno: still have a question?
<ajmitch> jml: that's surprising, the bug version information is something that debian has
<wgrant> version information is very useful.
<ajmitch> I'll look up a bug # for it, I'm sure it's in there :)
<wgrant> It's one thing that debbugs has over Launchpad that makes Launchpad look pretty pathetic.
<jml> ajmitch, it's surprising I haven't already heard of every single major feature request for Launchpad :)
<ajmitch> jml: given that complaints that roll in, sure ;)
<ajmitch> bug 424
<ajmitch> so really early :)
<beuno> intellectronica, hey!  yes, how do I see bug heat on launchpad.dev?
<beuno> I bring up trunk and get no flames!
<wgrant> db-devel
<beuno> I wanted to take a crack at making a lighter icon and such
<intellectronica> beuno: you wait until db-devel gets merged back in?
<beuno> aha
<intellectronica> (or branch from db-devel)
<intellectronica> beuno: let me guess, you're creating better icons for us? :)
<beuno> intellectronica, that was my intention, lets see how much past that it goes
<beuno> I also sent an email about the feature as a whole
<intellectronica> beuno: fountains of beer await you
 * beuno stores away his drivers license, won't use it for sure this trip
<wgrant> What happened to the post-3.0 notifications focus?
<wgrant> Is that still happening at some point, but just pushed back by the upstream linkage work?
* mbarnett changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 4 of 10.01 | 10.01 is releasing Wednesday 27th Jan at 0900 UTC | PQM is open | gary_poster is release manager, with flacoste for the actual release | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes | Channel logs: http://irclogs.ubu
#launchpad-dev 2010-01-29
<noodles775> Morning people
<wgrant> Morning noodles775.
<jml> good morning launchpadders
<mrevell> Howdy fellow travellers
<jml> ahoy
<jml> mrevell, hi
<mrevell> howdy jml, let me check I have minutes
<jml> mrevell, ok. I can call you also
<mrevell> jml, that would be superb. I have no minutes allowance remaining.
 * jml going offline for a while. call my mobile if needed.
<deryck> Morning, all
<noodles775> jml: will you have time for a call this afternoon?
<jml> noodles775, sure.
<jml> noodles775, whenever suits you
<jml> except not 1800 UTC
<noodles775> jml: in 10mins?
<jml> noodles775, great.
<intellectronica> matsubara: hi, i'm back, what did you want to ask?
<matsubara> intellectronica, hi
<matsubara> intellectronica, I wanted to ask about https://bugs.launchpad.net/bugs/401724 and https://bugs.launchpad.net/bugs/401923 they're both targeted to .11. should we drop the milestone for those? retarget to another milestone?
<intellectronica> matsubara: i will untarget and unassign from myself. those are no longer scheduled
<matsubara> intellectronica, ok. thanks
<jml> noodles775, in case you missed it on #bzr, bzr-builder gets it from the changelog.
<jml> noodles775, maybe another mistake in our data model. :\
<noodles775> yeah, I saw. Thanks.
<allenap> abentley: Are you free to talk about TwistedJobRunner at 1700 UTC?
<abentley> allenap: Sure.
<allenap> abentley: Thanks.
<noodles775> Night all.
<abentley> allenap: How did you want to chat?
<allenap> abentley: Skype okay?
<abentley> allenap: Sure.
<abentley> I can hear you.
<intellectronica> gary_poster: so andrea-bs has #the problem only manifests in the testsuite. trying to load many pages result in "Could not adapt" errors
<intellectronica> argh
<intellectronica> the same pages load just fine in the browser
<intellectronica> looks, b.t.w like both myself and andrea-bs will have to go soon, so this is not urgent, but the branch in question is lp:~andrea-bs/launchpad/blueprint-comments-part3
<sinzui> bac: ping
<gary_poster> intellectronica: ok.  do you know what test command I can run to quickly get the failure?
<intellectronica> gary_poster: bin/test -vvt stories/blueprints
<gary_poster> ok
<intellectronica> (it's a sequential test suite, unfortunately)
<gary_poster> :-P :-)
<andrea-bs> thank you intellectronica, gary_poster for your help: I have to leave now. Of course, feel free to contact me by mail if you need
<jml> gary_poster, intellectronica: you both know about the '-1' option, right?
<gary_poster> intellectronica, andrea-bs: my other review has a schema change, so I can't get to it ATM.  I'll try to look at it today and get back to you
<jml> it shows only the first failure.
<intellectronica> gary_poster: thanks a bunch
<jml> gary_poster, is PQM really actually open?
<gary_poster> jml: -1 yeah, thank you
<gary_poster> jml: yes
<jml> gary_poster, wuu.
<gary_poster> :-)
<sinzui> bac the presentation is suggested for upstream quality for a source package does not need to change. I realise now that that the project page must be different from the source package page because the package cares more about the series info. Since a project can have many source packages, a simple graphic treatment is not possible.
<jml> time to convert some inventory into VALUE!
<jml> (that probably means spurious conflicts and make-work from changed APIs & intermittently failing tests, but hey)
<mrevell> nytol
<deryck> So this is what a working internet connection feels like.
<jml> deryck, smooth and slightly oily!
<deryck> and a bit the smell of coffee
 * deryck had to find shelter in a coffee shop
<jml> ahh yes, coffee
<jml> g'night all
<jml> gary_poster, while you're around... would it be possible for me to change the subunit dependency to lp:subunit? it's in a format incompatible with the one currently being used
<gary_poster> jml: that's more of a policy question for mthaddon, if I understand you.  (I don't understand context of "it's in a format incompatible with the one currently being used")
<jml> gary_poster, the one we've got is packs, lp:subunit is 2a
<jml> gary_poster, if I change sourcedeps.conf and run the update script I get errors, I think.
<jml> gary_poster, alternatively, tell me how to make an egg out of subunit :)
<gary_poster> jml, per option 2: we don't have to make it an egg, but something that plays with distutils alone.  We prefer source distributions, actually.  If we had a bit of time next week I'd be happy to look at it.
<gary_poster> jml, per option 1:  "if I change sourcedeps.conf and run the update script I get errors, I think."  Even if you delete the old branch?  I'd be surprised
<jml> gary_poster, sure, not if I delete the old branch :)
<jml> gary_poster, but I guess if I have to do that then it's a special rollout thing
<jml> gary_poster, which means "ask tom"
<gary_poster> jml: well, actually it's messier than that ATM
<jml> :( :(
<gary_poster> jml: right now we have sourcecode.conf and config manager duplicating each other
<gary_poster> sourcecode.conf is supposed to become canonical for the losas, hopefully this cycle (I *think* it is just their work from here on out, but Tom and I need to confer(
<gary_poster> jml: the big question right now (for tom and maybe flacoste) is whether we want a non PQM-managed branch.  In an abstract sense, I don't see why it would make a big difference for us, but sometimes people see things differently than I  for some strange reason. :-) And I'd prefer to see a Python distribution of subunit myself, anyway, so that would be my preferred solution, which would make all of this other discussion irrele
<jml> gary_poster, well, subunit isn't strictly-speaking a Python project
<gary_poster> ah right
<jml> gary_poster, it's built with autotools.
<gary_poster> I forgot
<gary_poster> If we are ok with a non-PQM managed branch and we don't want to/can't make a standard Python distro then we could make the sourcecode.conf script a bit more careful
<jml> gary_poster, given that changing the revision of the branch that we use requires passing the test suite, I don't see why there'd be a problem
<gary_poster> jml: right, that's my take
<jml> gary_poster, also, lifeless is the project maintainer. when it's broken it won't be because of a failing test :)
<gary_poster> script more careful: if the branch already exists locally, and the parent is different than what is supposed to be branched, then blow the local one away
<gary_poster> jml: ^^ That would be easy enough to do if necessary
 * gary_poster said, waving his hands
<jml> gary_poster, yeah, I've thought of making that change myself
<jml> gary_poster, but there's not much point if I can't do my thing on production :)
<gary_poster> jml: yup, that's why I said it after I said, talk to flacoste and mthaddon :-)
<gary_poster> I don't think getting it on prod will be a big deal mechanically
<jml> cool.
<jml> gary_poster, thanks for the help.
<gary_poster> np, jml
<gary_poster> sinzui: do you know the bug for the fact that the "close bugs when creating a release" feature implementation had performance problems?  (I know the feature itself is bug 341687)
<sinzui> gary_poster: do you mean the bug that I closed on the 10.01 milestone today?
<gary_poster> sinzui: Maybe.  I mean the bug for the branch that I approved for a reroll or CP, and that Tom instated as a CP today, which backed out your feature.
<sinzui> ââgary_poster: bug 513321
<abentley> wgrant: Have you landed your buildd-generalization branch, or were you expecting me to land those changes as part of my branch?
<gary_poster> sinzui: thank you, yes
<jamalta> is there a list of launchpad dev's and what teams they are in?
<jamalta> bac's article mentions reviewing with someone from a specific team, but there's too many of you to keep track! :)
<wgrant> abentley: I think you should just go ahead and land yours.
<jamalta> i'm only kidding of course, i just don't know who is on what team for the most part
<wgrant> There's little point in landingmine separately.
<abentley> wgrant: Okay.  I got 4 test failures, but 3 looked spurious, so hopefully, it'll just work.
<wgrant> abentley: What were the failures?
<abentley> wgrant: They're hard to retrieve because they were run under VNC.
<wgrant> Oh, I see your email now.
<wgrant> I didn't know that stuff was actually tested.
<Edwin-lunch> mars: ping
<mars> hi EdwinGrubbs
<EdwinGrubbs> mars: I got smartsprites working and generating all the sprite files, but I am wondering why we don't just use PIL. I was able to take a super simple script someone wrote for sprites and recreate most of smartsprites in 100 lines.
<mars> :)
<mars> EdwinGrubbs, ok, why not?  If it is just a simple script, and the configuration is obvious, then I don't see a problem.
<EdwinGrubbs> mars: if you have some time, I'd like to bounce off some of the design/workflow decisions with integrating this fully.
<mars> EdwinGrubbs, I think it is a bit close to my EoD here.  Could we talk on Monday perhaps?  Before the AJAX call maybe?
<EdwinGrubbs> mars: sure
<mars> thanks
#launchpad-dev 2010-01-30
<beuno> jml, how cold is it over there?
<GabydeWilde_> any chance for translation happening?
<GabydeWilde_> At least half of theZeitgeistmovement developers are russian :P
#launchpad-dev 2010-01-31
<dhillon-v10> hi all, I am trying to implement this feature that would allow users to have more than one assignee/approver for one spec. and I know it involves some modification to PublicPersonChoice() method and it could be something similar to the Add Dependancy where I can return a link that a person can add to, but is this the right way ?
<dhillon-v10> jamalta: hi :)
<mwhudson> good morning
<dhillon-v10> mwhudson: hi :) just read your email on the api docs. thanks for that, i was looking for something like that for a while now
<mwhudson> dhillon-v10: cool
<dhillon-v10> mwhudson: i need some help implementing a new feature (a bug) do you have like 5-10 mins.
<mwhudson> dhillon-v10: sure
<dhillon-v10> mwhudson: I am trying to implement this feature that would allow users to have more than one assignee/approver for one spec. and I know
<dhillon-v10>                      it involves some modification to PublicPersonChoice() method and it could be something similar to the Add Dependancy where I can return
<dhillon-v10>                      a link that a person can add to, but is this the right way ?
<dhillon-v10> mwhudson: sorry about that
<dhillon-v10> mwhudson: copy and paste in irssi fails
<mwhudson> dhillon-v10: um, it will require a database change, i think?
<dhillon-v10> mwhudson: oh i see because previously there was one column to record that info. now there are more because of additional people
<mwhudson> dhillon-v10: right, you'll need a linking table
<mwhudson> dhillon-v10: i don't mean to be overly picky, but is there consensus that this is something that should be done?
<mwhudson> (a link to a bug report, an email thread, or something would be good)
<mwhudson> (i don't know much about blueprints)
<dhillon-v10> mwhudson: I don't know, Curtis marked this one as a wishlist, and I thought I might get started on this one, let me get you the link in a sec.
<dhillon-v10> mwhudson: https://bugs.launchpad.net/blueprint/+bug/2040
<mwhudson> 4 figure bug, awesome!
<dhillon-v10> mwhudson: so what do you say, is it even worth doing, because if its a big change in the database then we can  probably not do it or find a workaround
<mwhudson> dhillon-v10: it's not a very complicated change in the database really, so i don't think that's not a reason to do it
<dhillon-v10> mwhudson: alright then, so where do I get started, I have the dev environment setup and have been playing with it for a while now
<mwhudson> dhillon-v10: maybe a ui mockup is the place to start
<dhillon-v10> mwhudson: so first focus on the INewSpecification part so I can have a baseline ready then go into the database and make necessary changes
<mwhudson> dhillon-v10: i think i meant, "focus on what you want the user experience to be"
<mwhudson> i admit i don't use blueprints much
<dhillon-v10> mwhudson: can you please rephrase that "focus on what you want the user experience to be" is it basically saying make like a little screenshot of what the end result could look like
<mwhudson> dhillon-v10: yeah
<mwhudson> dhillon-v10: it can just be a pen and paper drawing scanned, or mutilate the page in firebug or something
<mwhudson> dhillon-v10: don't put too much effort into it at this stage
<mwhudson> dhillon-v10: is the role of the blueprint approver enforced in code?
<dhillon-v10> mwhudson: yeah I understand, a mock-up shouldn't take too much effort, and don't know about approver enforced part, I am still reading the code so sorry about that
<mwhudson> dhillon-v10: if you can catch curtis (aka sinzui) online, he might be a good person to talk to about this
<dhillon-v10> mwhudson: alright and thanks a *lot* for you help, you rock :)
<mwhudson> dhillon-v10: no worries, i hope your first change to launchpad goes well!
<dhillon-v10> sinzui: ping :)
<dhillon-v10> mwhudson: same here :)
<mwhudson> hmm, sinzui is on uk time this week
<dhillon-v10> mwhudson: I guess I'll wait if he comes back or otherwise I can just email him
<mwhudson> yeah
<mwhudson> sinzui: are you actually there?
<drubin> Is there a link to launchpadapi docs? I have found the raw webservice doc doesn't match up 100% ant the little changes make it kinda hard to work out what the python object relations are
<dhillon-v10> drubin: try this out: http://people.canonical.com/~mwh/canonicalapi/index.html it might help you out
<wgrant> dhillon-v10: That's the internal API documentation.
<dhillon-v10> wgrant: oops sorry you are right :)
<drubin> dhillon-v10: ye, I was a little bit overwelmed
<drubin> ye the biggest issue is getting the correct attributes from the python objects. Lets say I want a team members email. I have to do lp.people['teamname'].preferred_email_address.email    where as in the raw API it lists preferred_email_address_link there are a few of those. just wondering what else is different
<wgrant> drubin: foo_link -> foo, foo_collection_link -> foo
<wgrant> I don't think there's much else.
<drubin> wgrant: Is that the only difference?
<drubin> If that is the case I am pretty sure I can cope :)
<drubin> wgrant: Thanks. Pretty much what I was looking for.
<drubin> Is gzip compression turned on by default? If not where would I look to turn it on.
<wgrant> drubin: They're just automatic niceties provided by launchpadlib. foo_link webservice attributes contain a URL, but launchpadlib knows to instead produce a foo attribute containing the object itself.
<drubin> wgrant: Ye I kinda worked that out, after the first 5mins of using the lib. Was more worried about other stuff I might encounter down the line.
<mwhudson> woo, finally down to 0 unread
<mwhudson> "Diff against target: 248083 lines (+22639/-108557) 345 files modified"
<mwhudson> hard to see how that's ever going to be useful
#launchpad-dev 2011-01-24
<Ursinha> hello launchpad
<StevenK> Ursinha: Hi! How was your flight?
<spm> heya Ursinha
<Ursinha> StevenK, was nice, it was delayed 50 minutes but I got home at last :)
<Ursinha> and yours?
<Ursinha> spm, hey :)
<StevenK> Ursinha: Both of mine were okay, apart from the food.
<Ursinha> ah, food
<Ursinha> StevenK, I'm not taking food into account :P
<huwshimi> Ursinha: Well to take it into account, first you actually have to classify it as food.
<Ursinha> hahahahaa
<Ursinha> huwshimi, good point
<StevenK> huwshimi: Haha. How were your flights?
<huwshimi> StevenK: Better than on the way to Dallas. My flight into Sydney was even 30 min early.
<huwshimi> StevenK: How about you?
<StevenK> Transisting via SFO is annoying, since the walk from domestic to international should be measured in kilometres.
<huwshimi> StevenK: Ouch.
<StevenK> However, both flights were on time, fairly smooth. The only let down was food on the SFO-SYD leg.
<jorjoso> hello peopleÂ¡Â¡
<Ursinha> I'm glad you're all home and fine :) I'll try to sleep to minimize jet lag
<Ursinha> :)
<huwshimi> Ursinha-afk: Night
<jorjoso> I would like to  create a background for ubuntu narwhal do you know the deadline ?
<lifeless> no idea; I suggest asking in an Ubuntu channel
<lifeless> perhaps #ubuntu-devel or #ubuntu-desktop
<jorjoso> lifeless: thankyou
<huwshimi> jorjoso: I believe the date is something like the 13th of March. But if you ask in #ubuntu-devel or #ubuntu-desktop someone might be able to tell you for certain
<jorjoso> thanks
<jorjoso> :D
 * wgrant wanders in.
<huwshimi> wgrant: Afternoon
<wgrant> First leg late, second leg bumpy. Yay, I love airlines.
<spm> better bumpy than a short sharp crash.
<wgrant> It wasn't an A380 this time, so I was safe.
<spm> heh
<StevenK> My flight wasn't an A380. I was hopeful.
<wgrant> You didn't check beforehand?
<StevenK> I did -- when I booked it :_)
<wgrant> I think QF12 (LAX->SYD, not long before mine) was an A380.
<StevenK> I suspect all of the SFO flights won't be A380s
<wgrant> huwshimi: Is the plan still to have you on the green squad call?
<huwshimi> wgrant: Erm, possibly. Does Curtis lead that team?
<wgrant> huwshimi: That's the one.
<wgrant> me/curtis/jcsackett
<huwshimi> wgrant: Yeah I think I'm supposed to be a part of those calls, but I haven't heard any details yet.
<wgrant> huwshimi: We've scheduled them for around 9am AEDT, for now.
<huwshimi> wgrant: ah ok
<wgrant> Which is a bit of an improvement from my previous 20:30 call :/
<wgrant> StevenK: When's yours now?
<huwshimi> wgrant: yeah that would have been horrible
<StevenK> wgrant: 8am
<wgrant> StevenK: Ah, not too bad.
<spm> shockingly mine was just adjusted from 7am to 9am. it's horrible I tell you.
<huwshimi> We have a public holiday on Wednesday right?
<wgrant> Yup.
<wgrant> Have you applied for it?
<wgrant> If not... you'd better hope canonicaladmin unbreaks!
<huwshimi> wgrant: No-one told me I needed to apply for it :)
<StevenK> They can be put in afterwards, HR prefers if they're done before.
<wgrant> You need to enter leave on canonicaladmin.com, I believe.
<huwshimi> Ah right
<wgrant> Is is at the moment, however, less than up.
<huwshimi> wgrant: Would be funny to miss out on a public holiday because a website wasn't working.
<poolie> hi wgrant, huwshimi
<wgrant> Evening poolie.
<huwshimi> poolie: Hey there.
 * huwshimi is heading off... to sleep
<mrevell> Good morning
<jml> good morning
<deryck> Morning, all.
<adeuring> good morning
<leonardr> i have a question for anyone interested in launchpad bugs
<leonardr> my web_link code generally gives the results i would expect, but the web_link for a bug watch has a hostname of launchpad.dev, not bugs.launchpad.dev
<leonardr> is that expected/okay?
<leonardr> i see that either url works, i'm just wondering what's correct and if this is a problem i should look into
 * jml doesn't know
<deryck> leonardr, as you say, either should work.  But the bugs domain would be the correct one.
<leonardr> deryck, do you want to help me figure this out?
<deryck> leonardr, not really :-)  I'm sooooo done with bugs. ;)
 * deryck is kidding
<leonardr> deryck: give me a minute to re-learn what i wrote last week, and i'll explain it to you
<deryck> leonardr, sure, that works
<leonardr> the lazr.restful branch is https://code.launchpad.net/~leonardr/lazr.restful/web-link/+merge/46992
 * deryck looks
<deryck> leonardr, is it just for a bug watch that the bugs domain is omitted?  I assume other resources are in the correct domain?
<leonardr> deryck: yes, for a bug you get bugs.launchpad.dev
<deryck> leonardr, and paste me an example url of the watch, please.
<leonardr> deryck: to explain the branch briefly, we look for a registered function capable of converting a web service request into a website request
<leonardr> web_link: u'http://launchpad.dev/bugs/1/+watch/2'
<_mup_> Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Invalid by compscibuntu-bugs> <EasyPeasy Overview:Invalid by ramvi> <GNOME Screensaver:Won't Fix> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <The Linux OS Project:In Progress> <metacity:In Progress> <OpenOffice:In Progress by lh-maviya> <Tabuntu:Invalid by tinarussell> <Tivion:Invalid by shakaran> <Tv-Player:New> <Ubuntu:I
<leonardr> as it happens, launchpad has defined such a function already, for the librarian
<leonardr> as you can see in https://code.launchpad.net/~leonardr/launchpad/web-link/+merge/47258 once it's scanned
<leonardr> we are using absoluteURL to generate the url to the object, passing in the object + the fake website request
<deryck> ok, just looking over a few things here
<deryck> leonardr, so this seems to me it's on the lp side, not lazr.restful that this is going wrong, yes?
<leonardr> deryck: i agree
<deryck> I'm wondering if we just need a "rootsite" attribute in the zcml for bug watches.  i.e. if it's just omission of this, not by design, that we have both urls for watches.
<leonardr> deryck: i think that's the piece i was looking for
<leonardr>         <browser:url
<leonardr>             for="lp.bugs.interfaces.bugwatch.IBugWatch"
<leonardr>             path_expression="string:+watch/${id}"
<leonardr>             attribute_to_parent="bug"/>
<leonardr> lib/lp/bugs/browser/configure.zcml
<leonardr> of course, fixing that might cause all sorts of other test failures
<leonardr> i'll fix the other stuff first and come back to this
<deryck> leonardr, http://pastebin.ubuntu.com/557658/
<deryck> leonardr, and yeah, just see what sorts of failures we get in tests.
<deryck> leonardr, grep turns up on test using the non-bugs url.  xx-bugwatch-comments.txt
<leonardr> all right, thanks for your help
<deryck> np
<leonardr> deryck: same problem with CVE, actually
<deryck> abentley, adeuring, henninge -- stand up in mumble now, if we all remembered and are here. :)
<adeuring> ok
<abentley> deryck, ack
<henninge> deryck: I am here but have to see if mumble likes me
<deryck> henninge, ok, we have an orange room now
<leonardr> ok, here's another problem--not sure who can help, but i'll spell it out
* henninge changed the topic of #launchpad-dev to: Launchpad Development Channel | PQM is open | firefighting: - | Get the code: https:/â/âdev.launchpad.net/âGetting || On call: henninge | reviewing:  | queue: []  | https://code.launchpad.net/launchpad/+activereviews
<leonardr> the HWDB resources have a web_link and they shouldn't
* henninge changed the topic of #launchpad-dev to: Launchpad Development Channel | PQM is open | firefighting: - | Get the code: https:/â/âdev.launchpad.net/âGetting || On call reviewer: henninge | reviewing:  | queue: []  | https://code.launchpad.net/launchpad/+activereviews
<leonardr> if the attempt to navigate to the HWDB web_link raised an exception, i could deal with that easily
<leonardr> but it gives me a url that's not really present
<leonardr> i'm going to leave it alone for now, but the current solutions i have in my head are ugly, so i would like some help
<jml> sinzui: we're down to talk today. Do you still want to?
<leonardr> adeuring, cr3 says you're the one to talk to about this
<sinzui> jml: I do not have anything for today...but we are keeping this slot right?
<jml> sinzui: I'm happy to if you think it will help you.
<adeuring> leonardr: can you explain the the problem a bit? What is the wrong link?
<sinzui> jml: I want to keep it.
<jml> sinzui: cool. consider it kept, but cancelled for today
<leonardr> adeuring: sure.
<jml> sinzui: btw, can you please send Huw details about your team's standup?
<leonardr> if you GET http://api.launchpad.dev/beta/+hwdb, you get back a response including this:
<leonardr> web_link: u'http://launchpad.dev/+hwdb'
<leonardr> which would be accurate if the hwdb were published on the website, but it's not
<adeuring> ah, I see. The problem is that we don't want the HWDB to be publicly avaiable
<leonardr> adeuring: yes, but i'm not sure how to determine that ahead of time
<leonardr> i think the best solution would be to have the traversal code raise an exception unless the request is a web service request?
<adeuring> leonardr: sounds reasonable. The other option would be to raise Unauthorzied for users who are not in the group "hwb-team" or somesuch
<adeuring> erm, hwdb-team
<leonardr> adeuring: i don't understand. would that url work if i were in the right group?
* jml changed the topic of #launchpad-dev to: Launchpad development https:/â/âdev.launchpad.net/â | PQM is open | firefighting: - | On call reviewer: henninge | reviewing:  | queue: [] | https://code.launchpad.net/launchpad/+activereviews
<adeuring> leonardr: at least formally. OTOH, the is no real content to show on the web.
<adeuring> so, raising an exception for any non-webservice requests is probably more reasonable
<leonardr> adeuring: do you know where to put that code? i have no idea except maybe there's a traversal method somewhere in services/hwdb
<leonardr> adeuring: just out of curiosity, what about a link like http://launchpad.dev/+hwdb/+driver/10 ? would that nominally work on the website?
<adeuring> leonardr: no, the HWDB is only exposed via the webservice
<leonardr> adeuring: in that case, we should definitely raise an exception during traversal
<adeuring> leonardr: right. regarding the right place for the exception: I have no real clue. Maybe gary_poster has a suggestion?
 * gary_poster reads back
 * gary_poster wonders how far to read back
<adeuring> gary_poster: ca 10 minutes
<leonardr> gary: to 9:12
<gary_poster> ok thanks
<leonardr> i guess i'm thinking that we should *traverse* the url after it's generated and see if it raises an exception. but there must be a better way
<leonardr> fwiw, my alternative is to add a lazr.restful declaration @no_web_link and apply it to the hwdb resources. but that feels like duplication of information
<gary_poster> leonardr: making sure I understand.  You want to have lazr.restful know a quick way to determine whether a given url (hwdb in this case) will fail.  You don't really want this to happen when someone clicks on the url--that's too late.  You want to never render it.  Traversal is not a natural place for this in the current scheme of things, because we don't traverse urls when we generate them.  Do I understand prop
<leonardr> gary: yes
<leonardr> i was initially surprised that we could generate urls that wouldn't work, but now i understand that the code is different
<gary_poster> OK.  using traversal would be expensive and unpleasant.  Can we look at what is generating URLs instead, and make that complain?
<leonardr> gary: we can sure try
<gary_poster> leonardr: ok.  Would you like me to dig into that code briefly, or do you want to do it and report back?
<gary_poster> If you want me to dig, you'll need to get me started
<leonardr> gary: i'm going to look up one thing and maybe we can go from there
<gary_poster> ok cool
<leonardr> this would be lib/lp/hardwaredb/browser/configure.zcml
<leonardr>     <browser:url
<leonardr>         for="lp.hardwaredb.interfaces.hwdb.IHWDBApplication"
<leonardr>         path_expression="string:+hwdb"
<leonardr>         parent_utility="canonical.launchpad.webapp.interfaces.ILaunchpadRoot"/>
<leonardr> what's the code that runs that? the zope IAbsoluteURL implementation?
<cr3> leonardr: my concern with raising an exception is that the hwdb still gets published in the wadl, I would much prefer to have a way to extend the wadl dynamically for other selfish reasons :)
<leonardr> cr3: you make a good point. ideally the web_link would show up in the wadl definition for other resources but not for the hwdb
<leonardr> that implies we need some way of knowing that hwdb resources don't have web_link, without access to any particular hwdb resource
<leonardr> that argues for @no_web_link
<leonardr> gary, before you go hunting, what do you think of that? -^
<gary_poster> leonardr: it makes sense to me, tbh.  It's not ideal that you have to make the same logical declaration twice, but I think other frameworks also require separate configuration of url generation and url parsing.
<leonardr> all right, i will land my existing lazr.restful branch and then add no_web_link
<gary_poster> cool
<leonardr> i'll just skim the test failures to see if there are any other obvious problems
<tiagoscd> hey
<tiagoscd> i would like to get more informations about ReportingAPI (https://dev.launchpad.net/Translations/Specs/ReportingAPI)
<danilos> tiagoscd, hey-hey, welcome to the right place (fwiw, I might be the right person to help out about that topic, but development discussions usually happen here)
<tiagoscd> David tell me that I can found help here
<tiagoscd> ok danilos
<danilos> tiagoscd, so, the story there is quite complicated, and I am not sure where it's well written down (if anywhere)
<tiagoscd> when I try to access the page https://blueprints.launchpad.net/rosetta/+spec/translations-reporting-api, that are divulgated in dev.launchpad.net, i got the "Page not found" error
<tiagoscd> so, where I can found the currently source code?
<leonardr> ok, the only other problem i see is that the web_link for a wiki_name keeps the api vhost. probably we can fix that at the same time as the bug watch and CVE.
<danilos> tiagoscd, well, it's all in LP tree... however, the problem is that it'd be ugly to expose the existing LP API because it's broken... we have implemented new ITranslateLanguage interface which is much nicer and we'd want to export that instead of IRosettaStats, but one would first have to switch DistroSeriesLanguage and POFile to use it
<danilos> tiagoscd, see also bug 583979 and bug 583934
<danilos> tiagoscd, discarded branch from Adi on https://code.launchpad.net/~adiroiban/launchpad/bug-583934/+merge/26965 might help you get a better understanding about how it all works
<dpm> tiagoscd, just for reference, the link to the blueprint is https://blueprints.launchpad.net/ubuntu/+spec/community-m-launchpad-translations-reporting-api - I've updated the wiki page that had the broken link (I'll leave Danilo take it from there)
<tiagoscd> danilos, i'll take a look,
<tiagoscd> dpm, thanks for the link
<salgado> sinzui, should a product's driver be the default release manager (called driver in the code) of a series?
<sinzui> salgado: no. We do not want to auto-populate that value. Th user who creates the series should be the release manager (consider the experimental series case), or the project owner should set it
<sinzui> salgado: we expect the release manager to be a smaller group of people than project drivers
<lifeless> morning
<sinzui> lifeless: where are you?
<salgado> sinzui, that makes sense.  I asked because people seemed to be expecting that and got surprised when being product driver didn't give them RM rights
<lifeless> at home
<lifeless> its 4am
 * lifeless is feeling the jetlag 
<sinzui> lifeless: :(
<bigjools> good morning all
<sinzui> salgado: so the user registered a series and did not get RM?
<lifeless> sinzui: it could be worse; I'm reasonably alert, just at a whacky time of day
<bac> morning bigjools
<bigjools> Hello Bradders
<lifeless> hi bigjools
<bigjools> hey jetlagged man
<bac> bigjools: how's austin?
<salgado> sinzui, I'm not sure who registered the series (can't seem to find that information), but the user was a driver of the product and was expecting to be able to create milestones
<bigjools> bac: Sunny and not cold.  \o/
<bac> bonus!
<lifeless> allenap: hi; how would we qa Bug 227334 ?
<_mup_> Bug #227334: checkwatches should raise a BugTrackerConnectError when Mantis doesn't set a return path <bugwatch> <lp-bugs> <oops> <qa-needstesting> <story-reliable-bug-syncing> <trivial> <Launchpad itself:Fix Committed by allenap> < https://launchpad.net/bugs/227334 >
<bigjools> indeed
<sinzui> salgado: understood.
<bigjools> lifeless: are you staying up or heading back to bed?
<lifeless> bigjools: staying up
<lifeless> bigjools: I'm terrible at getting back to sleep once awake
<bigjools> lifeless: ah ok, I am going to take a look at https://bugs.launchpad.net/launchpad/+bug/618395 later, I might need to grab you for ideas.
<_mup_> Bug #618395: DistroArchSeries:+index timing out ~ 7% of the time <lp-soyuz> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/618395 >
<bigjools> it's a general search issue
<bigjools> using LIKE on a binarypackagename causes a seq scan :(
<lifeless> orly?!
<lifeless> :P
 * bigjools 's sarcasm alarm goes off
<lifeless> I'd be delighted to help
<lifeless> I'm not going to pay much attention to work until a more reasonable time, but I'll drop my the laptop every 10-15 and check for pings
<bigjools> lifeless: I'll probably ping you in a couple of hours then
<lifeless> cool
<bigjools> thx
<bac> sinzui: you're still my manager according to canonicaladmin.  you mind signing my expense report for dallas?
<sinzui> okay
<bac> thx
<sinzui> bac: I do not see the request.
<bac> sinzui: doing it now.  i wanted to confirm you were ok with it before i entered
<bigjools> sinzui: what bugs are your guys working on?  I am a one man band until Wednesday, where I become 2 and then 3 on Thursday.  We should co-ordinate a bit.
<sinzui> bigjools: http://launchpad.leankitkanban.com/Boards/Show/14028609
<jcsackett> bigjools: i think everyone on sinzui's team (all three of us) are just grabbing whichever critical grabs our eye.
<bigjools> yeah that's what I was going to do :)
<bigjools> makes it important that we set in-progress
 * jcsackett nods. i agree.
<sinzui> bigjools: I will take bug 700724 in an hour
<_mup_> Bug #700724: Subscription policy inherited from parent team member <disclosure> <teams> <Launchpad itself:Triaged> < https://launchpad.net/bugs/700724 >
<bigjools> sinzui: I think maybe we need a maintenance kanban board instead of 2 sqauds' ones.
<sinzui> bigjools: since there are so many bugs in critical, it is not possible to know what jumps the queue. The bug I am taking is actually the most important bug to take next.
<bigjools> sinzui: yeah we just need to whittle them down, it will be random until there's a sensible number of them left.
<sinzui> bigjools: lifeless directed us to prefer bugs in the appearing in the current oops reports. I think we need to assign the bug when we get it ready-to-code so someone else does not duplicate the work
<bac> sinzui: and there is another for ec2 expenses
<bigjools> sinzui: the oops reports include everything - I presume you mean the top timeouts
<bigjools> I agree anyway
<sinzui> Looking at the critical bugs, I think some have not appeared in months or years.
<bigjools> yes, I invalidated some
<tiagoscd> which IDE do you recommend to develop in Launchpad?
<bigjools> most of us use vim/emacs in a terminal
* abentley changed the topic of #launchpad-dev to: Launchpad development https:/â/âdev.launchpad.net/â | PQM is open | firefighting: - | On call reviewer: henninge, abentley | reviewing: -,- | queue: [] | https://code.launchpad.net/launchpad/+activereviews
<lifeless> sinzui: bigjools: any critical bug is a good one to do :); I agree about in-progress mattering. The ready-to-code angle - I tend to flesh out bugs in the description etc such that the knowledge is captured, perhaps that is sufficient?
<bigjools> lifeless: yeah, I think writing the investigation details in the bug is critical
<tiagoscd> hmm, nice
<jcsackett> lifeless, bigjools: critical, and a good indication that someone else might be getting started (without setting in-progress). i know if i see notes from the last day or so from someone, i'm going to ask if they're working on it.
<sinzui> lifeless: Not every engineer can read the conversation and come to the sam conclusion. The engineer needs to be familiar with the area of code
<mars> interesting: http://initd.org/psycopg/articles/2011/01/24/psycopg2-porting-python-3-report/
<mars> <mfoord> another of our dependencies now on python 3
<lifeless> sinzui: thats true
<lifeless> sinzui: otoh I think if it needs that much context, perhaps set it in-progress when you realise that even if the investigation isn't complete
<lifeless> ?
<sinzui> lifeless: I do that a lot.
<lifeless> seems like we're in vigorous agreement ;)
<lifeless> sinzui: a thought on the team membership thing
<lifeless> sinzui: perhaps we need two separate knobs
<lifeless> 'direct membership is restricted/moderated/open' and 'prevent teams which are members (directly or indirectly) from being open'
<lifeless> with a note on the second one that this is required if assets with significant privileges are wanted
<lifeless> sinzui: I've no idea if this is a jetlagged toenail smoking thing, or a good idea.
<sinzui> lifeless: That sounds like feature development
<sinzui> And I think that will be harder to convey to users
<sinzui> Since we are supporting a handful of teams, I think we should look for an expedient solution. Add ing a nob to control permission probably requires reinventing team participation and inTeam()
<lifeless> sinzui: The conveyance to users is the primary thing
<lifeless> sinzui: in terms of size of development, it could go either way
<lifeless> hmmm, and we don't want the cross product anyway
<lifeless> so if we did use a separate knob we'd need to constrain the settings too
<sinzui> I see a nob as extensive work to clean up a lot of cases. Adding an enum and setting a handful of team to use id nice because little code is needed to support the case
<abentley> lifeless, it appears that the landing you rolled back was due to an unrelated intermittent failure.  I plan to land it again.  Cool?
<abentley> abel, translation-fixing2 etc. depend on translation-fixing, but translation-fixing was rolled back, so don't land them until I've re-landed translation-fixing.
<abentley> abel, I expect to re-land translation-fixing today.
<jml> jelmer: which version of bzr-builder did you say you needed on the slaves?
<jelmer> jml: trunk tip and a newer bzr
<jml> jelmer: yikes.
<jml> jelmer: so we need a release first, basically.
<jml> jelmer: this is to get quilt3 support, right?
<lifeless> abentley: cool
<jelmer> jml: it's for two things
<lifeless> abentley: it would be great if you could also file a critical bug about the intermittent failure
<jelmer> jml: For quilt3 support you'd just need bzr-builder trunk
<jelmer> jml: To fix most of the memory issues that the buildds are running into you need bzr-builder trunk and a newer bzr
<jelmer> jml: Do you mean a release of bzr ?
<jml> jelmer: I mean a release of bzr-builder.
 * jelmer wasn't aware bzr-builder had releases :-)
<jml> oh.
<jml> interesting.
<jml> it has releases
<jml> 0.6, for example.
<jelmer> jml: what is necessary in particular, a tarball?
<jml> jelmer: I don't know. I was going to ask lamont once you told me what you needed :)
<jelmer> heh, ok
<jml> jelmer: but perhaps I can cut out the middle-man :)
<abentley> lifeless, filed: https://bugs.launchpad.net/launchpad/+bug/706992
<_mup_> Bug #706992: intermittant test failures: test_404 test_oldurl <Launchpad itself:Triaged> < https://launchpad.net/bugs/706992 >
<lifeless> abentley: thank you
<jml> jelmer: could you please file RTs for what you need and point lamont at them, CCing me?
<bigjools> lifeless: hi
<lifeless> hi
<bigjools> lifeless: https://lp-oops.canonical.com/oops.py/?oopsid=1846A1401
<bigjools> the top long query is the batchnav's count, but it's issuing it three times (one is coming from the view code which I can eliminate)
<bigjools> wtf
<jelmer> jml: ok
<lifeless> mmm lp-oops is on a go-slow for me
<lifeless> am in now
<lifeless> just
<bigjools> yeah it's very slow
<lifeless> ok, so lets see
<lifeless> 2.5 seconds doing an ilike count
<lifeless> 1.9 getting the batch hits
<bigjools> lifeless: repeated 4 times, 3 of which are from batchnav
<lifeless> wth
<bigjools> yes
<bigjools> I've re-created it locally
<lifeless> so, if its coming from batch navigator 4 times
<lifeless> and the hits 3 times
<bigjools> no
<bigjools> one is from the view code
<lifeless> ah
<bigjools> I was going to make the template use batch/total instead of re-calculating it in the view
<lifeless> hah
<lifeless> so I don't know if you noticed
<bigjools> but then I saw the three queries from the nav
<lifeless> this is the query - '%%'
<bigjools> yeah, blank
<lifeless> postgres doesn't optimise that out
<bigjools> \o/
<jml> jelmer: thanks.
<jml> isn't '%%' literal '%'?
<bigjools> BinaryPackageName.name ILIKE '%' || '' || '%'
<bigjools> the query is issued when it calls "return len(results)" at the bottom of _Batch._get_length()
<bigjools> rather than the cached .listlength
<bigjools> lifeless: ^
<bigjools> which means the _Batch object is getting re-instantiated
<lifeless> bigjools: thats a good thing to fix
<lifeless> uhm
<lifeless> perhaps a pdb statement in the constructor and get the backtrace for each instance locally?
<bigjools> I have the backtrace alreayd
<bigjools> LP_DEBUG_SQL_EXTRA ftw
<lifeless> on staging
<lifeless>  54249
<lifeless> (1 row)
<lifeless> Time: 7986.742 ms
<lifeless> vs
<lifeless>  54249
<lifeless> (1 row)
<lifeless> Time: 2156.876 ms
<lifeless> removing the ftq and ilike when the query is for ''
<bigjools> yes that's one thing to do but we can do better
<lifeless> agreed
<lifeless> I was just checking that the optimisation was significant, in case of my memory failing :)
<bigjools> :)
<lifeless> bigjools: so where is it being instantiated?
<bigjools> I am looking right now
<bigjools> somewhere in zope
<bigjools> tell a lie
<lifeless> I'm going to go cook some breakfast
<lifeless> back with you soon
<bigjools> ok
<bigjools> this rabbit hole is very twisty
<jml> g'night all. be around tomorrow.
 * bigjools headdesks
<lifeless> jml: night
<lifeless> bigjools: ?
<bigjools> lifeless: see lib/lp/soyuz/browser/packagesearch.py and tell me what's wrong with the batchnav property
<bigjools> given that the template uses that property 3 times
<lifeless> hmmm
<lifeless> haven't opened it yet
<lifeless> but I imagine not cached
<bigjools> :)
<lifeless> sounds like two easy fixes then
* lifeless changed the topic of #launchpad-dev to: Performance Tuesday! | Launchpad development https:/â/âdev.launchpad.net/â | PQM is open | firefighting: - | On call reviewer: henninge, abentley | reviewing: -,- | queue: [] | https://code.launchpad.net/launchpad/+activereviews
<abentley> lifeless, fsvo "Tuesday".
<lifeless> abentley: the joys of a spherical earth
<lifeless> sinzui: are you familiar with defaultdict ?
<sinzui> lifeless: to initialise a dict with a sensible value to make code easier to type and read?
<lifeless> yes
<sinzui> we do not use in in Lp very often
<sinzui> We should
<sinzui> lifeless: pause a moment if you have a message for me...just broke my keyboard or window manager
 * sinzui hopes a reboot will fix this
 * sinzui hopes someone see this
 * beuno does not see that
<leonardr> henninge or abentley, i'd like a review of https://code.launchpad.net/~leonardr/lazr.restful/publish_web_link/+merge/47280
<henninge> leonardr: looking
* henninge changed the topic of #launchpad-dev to: Performance Tuesday! | Launchpad development https:/â/âdev.launchpad.net/â | PQM is open | firefighting: - | On call reviewer: henninge, abentley | reviewing: leonardr,- | queue: [] | https://code.launchpad.net/launchpad/+activereviews
<lifeless> hmm
<lifeless> flacoste: random idea; do we have a graph of open timeout bugs?
<jcsackett> ooo. that would be nice.
<bigjools> burndown?
<lifeless> yeah
<bigjools> did you look at lpstats?
<lifeless> even without a trend line it would be a useful eyeball thing
<lifeless> I was last week,
<lifeless> IIRC there was a todo to set one up.
<lifeless> nothing with 'timeout' in the graphs
<lifeless> and the only oops ones are the frequency graphs like https://lpstats.canonical.com/graphs/OopsLpnetHourly/
<lifeless> which seems borked
<lifeless> ah, works on monthly
<lifeless> sinzui: hi
<lifeless> sinzui: is your machine happier ?
<lifeless> bigjools: you might find the analysis in bug 706200 interesting
<_mup_> Bug #706200: Archive:EntryResource:getPublishedBinaries timeouts <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/706200 >
<jcsackett> leonardr: hi, have a sec to talk about webservice_error ?
<salgado> lifeless, I'm getting http://paste.ubuntu.com/557770/ when running lp-propose.  have you seen anything like this before?  I've granted full access to bzr, btw
<leonardr> jcsackett, sure
<jcsackett> leonardr: i'm trying to get one exception to work across the webservice. in the past, i just add to add the webservice_error thing to it, and all was well.
<jcsackett> i'm encountering a situation now where in tests it is still returning a 500 error code, rather than the appropriate webservice_error.
<lifeless> NotFound: Object: <canonical.launchpad.systemhomes.WebServiceApplication object at 0x7b377d0>, name: u'1.0'
<lifeless> salgado: ^ thats a little concerning :)
<jcsackett> leonardr: relevant code bits for above: https://pastebin.canonical.com/42220/
<jcsackett> i'm not sure if i'm missing something in my setup that i did in the past, or if there's some edge case thing cropping up here i'm unaware of.
<lifeless> leonardr: ^ perhaps you could eyeball salgado's backtrace. As I read it we've managed to turn off the 1.0 API, but surely thats not correct.
<lifeless> salgado: we get an OOPS
<leonardr> salgado, lifeless: what's the host here? it's not edge, is it?
<salgado> I don't know; just ran bzr lp-propose
<leonardr> the stacktrace says it's production
<leonardr> when i hit api.launchpad.net/1.0 in my browser i get some json
<leonardr> salgado: please add import httplib2\nhttplib2.debuglevel = 1 to the beginning of the script you're running
<leonardr> let's see what request it's making
<bigjools> lifeless: totally not surprised, debversion_sort_key is horrendously slow
<lifeless> bigjools: 2ms per call
<lifeless> bigjools: but the killer is doing it on huge data sets
<bigjools> exactly
<bigjools> 2ms is not quick when you think about it
<lifeless> :)
<leonardr> jcsackett: look at a stacktrace to see where the CannotDeleteBranch is raised
<lifeless> bigjools: bah, bad math. 0.5ms per
<lifeless> still not brilliant
<bigjools> compared to pulling data out of a column, no :)
<salgado> leonardr, https://pastebin.canonical.com/42223/
<salgado> authorizing now
<lifeless> be better to let the thing be ordered directly from an index
<bigjools> now, why is ec2 test telling me I don't have an ssh agent when I certainly do
<lifeless> that way it wouldn't ned to materialise everything
<sinzui> lifeless: Store.flush() did not, and still does not work...
<jcsackett> leonardr: the stacktrace passed back when the 500 is raised in the test shows it's in the destroySelf method in the paste i sent you.
<lifeless> sinzui: what are the symptoms
<jcsackett> leoanrdr: i'm doublechecking again, but pretty sure that's what i saw.
<leonardr> salgado: that's not the request that's failing. maybe there are two scripts running?
<sinzui> lifeless: mailinglistset is not use Store.of(self), it is using getUtility(IStoreSelector).get(MAIN_STORE, SLAVE_FLAVOR)
<salgado> leonardr, this is the one that fails: https://pastebin.canonical.com/42224/
<leonardr> you also shouldn't be getting that request if you've already authorized the app
<sinzui> I think we are not using the store we think we are
<lifeless> sinzui: ah!
<salgado> leonardr, looks like it's failing to get the authorization
<leonardr> salgado: GET /beta/1.0/
<leonardr> that's the problem
<lifeless> sinzui: so I'm ok with doing a commit if we need to, but I don't understand why we'd need to
<salgado> lifeless, ^
<leonardr> probably there's some code somewhere that hardcodes /beta/ or http://api.launchpad.net/beta/
<lifeless> salgado: thats pretty bong :>
<sinzui> lifeless: I want to change the store to self because the ml object uses self. I do not think there is a legitimate readon to do what it dies
<lifeless> salgado: time to file a bug
<lifeless> sinzui: agreed
<flacoste> lifeless: i have an open task for me to set that up
<flacoste> lifeless: as well as one for OOPS and for regression
<lifeless> flacoste: +4000 on doing it
<sinzui> s/dies/does/
<leonardr> jcsackett: can i see the whole stack trace?
<jcsackett> leonardr: https://pastebin.canonical.com/42226/
<jcsackett> so, i see breakreferences in there, but not sure why that would make the webservice_error magic break.
<salgado> lifeless, bug 707075
<_mup_> Bug #707075: lp-propose fails with a 404 error <Bazaar:New> < https://launchpad.net/bugs/707075 >
<lifeless> sinzui: quick UI comment - https://launchpad.net/ubuntu/+source/linux - doesn't say what the big list at the top is
<lifeless> sinzui: is there any reason we wouldn't explain whats there a little?
<sinzui> lifeless: That is a pathological case. That *is* the description of what the source is
<sinzui> lifeless: we are making the description from the binary package decriptions...just like packages.ubuntu.com
<lifeless> perhaps a
<lifeless> 'Package description:' header?
<sinzui> I am not sure what to do heere
<sinzui> We do not do there else where. We give the name then just show the description
<lifeless> ok
<lifeless> so I'll file a bug
<lifeless> I don't know what to do either.
<sinzui> I should not make an exception for this case. We need to think about how to make sane descriptions
<lifeless> yes
<lifeless> its not a description
<lifeless> its a collection of binary package->description tuples
<sinzui> This is too long an not informative
<lifeless> descriptions require prose
<lifeless> sinzui: bug 707081 FYI
<_mup_> Bug #707081: source package homepage descriptions can be confusing <Launchpad itself:Triaged> < https://launchpad.net/bugs/707081 >
<sinzui> yeah. this is not much better: http://packages.ubuntu.com/source/natty/linux
 * sinzui wonders is software center offer something better
<lifeless> sc works on binary packages
<lifeless> one at a time, and only shows applications not plumbing
<sinzui> I see the MailingList object use IMasterStore(<self|class>) in a few methods, but in other Lp examples, those calls are in static or class methods
<lifeless> -> it has an easier domain
<lifeless> bigjools: I'm very glad I bother to file that depwait bug :)
<bigjools> lifeless: why?
<bigjools> I would have seen the reports and checked :)
<lifeless> ;)
<jcsackett> leonardr: any thoughts re: my stacktrace on webservice_error stuff?
<leonardr> jcsackett: i'm looking at it, i'll have to come back after lunch
<jcsackett> leonardr: ah, dig.
<jcsackett> thanks.
<bac> hi sinzui
<sinzui> hi bac
<bac> sinzui: hey, in product-indext.pt there is a LP.use for lp.registry.pillar....which doesn't appear to exist
 * sinzui looks
<bac> sinzui: thanks
<bac> sinzui: i'm trying to add new JS stuff to that template but would like to see it load cleanly before i pollute it
<sinzui> OH!
<sinzui> bac: yes, deryck's recent changes make that BAD code
<bac> i'm unsure what is meant to be collapsible
<sinzui> bac: huw fixed the license widget by updating a similar line
<sinzui> bac: huwshimi changed LPS.use to YUI().use which creates a separate instance. We do not always want to create a new instance though
<sinzui> jcsackett: where did you register lp.registry.pillar
<bac> sinzui: the string 'lp.registry.pillar' exists nowhere else in the code base but that template.
<sinzui> bac: I understand that. I did not write it. I think jcsackett did
<bac> ok
<jcsackett> sinzui, bac: i think you can safely move that. it predates when all the consolidation happened and was cargo culted.
<bac> jcsackett, sinzui: that JS is bad but doesn't seem to be doing any harm.  can it wait to land with the branch i'm working on...which may be a while?
<bac> "it" is the removal of the lp.registry.pillar stuff
<jcsackett> bac, sinzui: when did that JS become bad?
<jcsackett> the collapsible bit that i created (seemingly ages ago) seems to work when i'm looking at it.
<bac> jcsackett: it is bad b/c lp.registry.pillar does not exist.
<bac> jcsackett:  load launchpad.dev/firefox in a browser and look at the error console
<bac> jcsackett: what is supposed to be collapsing?
<jcsackett> bac: there's a collapsible on the configuration links for projects you can set usage enums on.
<jcsackett> so that once you've configured everything, they default to hidden.
 * jcsackett is looking for relevant bug.
<bac> jcsackett: ok, i wasn't logged in so i didn't see that part and had forgotten about it
<bac> jcsackett: but, the collapsible stuff from lp.registry.pillar does not exist and thus has no bearing
<jcsackett> bac: dig. so yes, i remember adding that in the JS, but it got whacked at some point. my $.20 is that it can wait till you land this branch.
<bac> your twenty cents, huh?  had best listen!
 * jcsackett laughs.
<jcsackett> bit of a typo there, yes. :-P
<bac> when dumb stores advertice 0.50ï¿  can you hold them to it?
<jcsackett> probably not. :-p
<jcsackett> if only because they ultimately control the transaction, and tell you go to get bent.
* jcsackett changed the topic of #launchpad-dev to: Performance Tuesday! | Launchpad development https:/â/âdev.launchpad.net/â | PQM is open | firefighting: - | On call reviewer: henninge, abentley | reviewing: leonardr,- | queue: [jcsackett] | https://code.launchpad.net/launchpad/+activereviews
<jcsackett> henninge, abentley (or anyone else) can i get a look at https://code.launchpad.net/~jcsackett/launchpad/i-hate-storm-base-storm-705197/+merge/47269?
<abentley> jcsackett, sure, looking.
<jcsackett> abentley: thanks.
* abentley changed the topic of #launchpad-dev to: Performance Tuesday! | Launchpad development https:/â/âdev.launchpad.net/â | PQM is open | firefighting: - | On call reviewer: abentley | reviewing: jcsackett | queue: [] | https://code.launchpad.net/launchpad/+activereviews
<abentley> jcsackett, does your lint header mean there was no lint?
<jcsackett> abentley: no, it means i apparently didn't save after adding lint. one sec.
<jcsackett> abentley: mp updated.
<abentley> jcsackett, thanks.
<abentley> jcsackett, r=me.
<jcsackett> abentley: thanks.
* abentley changed the topic of #launchpad-dev to: Performance Tuesday! | Launchpad development https:/â/âdev.launchpad.net/â | PQM is open | firefighting: - | On call reviewer: abentley | reviewing: - | queue: [] | https://code.launchpad.net/launchpad/+activereviews
<james_w> the help for lp.net/launchpad's +filebug should probably be updated
<james_w> it talks about other projects
<deryck> james_w, wiki help page?
<deryck> or bug reporting guidelines?
<james_w> the latter
<james_w> "This is the main Launchpad bug queue, if you don't know which component of Launchpad you found the bug, this is the right place to file it."
<deryck> ah, right
<james_w> I guess it's still somewhat true with lazr.* etc.
<deryck> gary_poster, I think you tried to do this ^^ and couldn't.  and I was going to use you as my test that it wouldn't OOPS now. :-)
<deryck> gary_poster, re: updating bug reporting guidelines
<gary_poster> deryck: right! ok, I'll try
<deryck> gary_poster, thanks1
* henninge changed the topic of #launchpad-dev to: Performance Tuesday! | Launchpad development https:/â/âdev.launchpad.net/â | PQM is open | firefighting: - | On call reviewer: henninge, abentley | reviewing: -, - | queue: [] | https://code.launchpad.net/launchpad/+activereviews
<henninge> leonardr: done. Sorry for the delay.
<leonardr> henninge: thanks
<gary_poster> deryck: success.  I didn't receive a confirmation notification of the change--it just redirected me to the main project bug page--but it worked.  So, QA OK as far as that bug is concerned. :-)
<deryck> gary_poster, excellent.  thanks.
<gary_poster> james_w: Now, "This is the Launchpad bug queue. Please include the exact URL of a page where you had the problem..."  Thank you for reminding us
<james_w> np
<bigjools> gary_poster: do you know if I need to do anything special to generate an OOPS when logging an error in a test?
<abentley> henninge, you are still OCR?
<leonardr> jcsackett: is this existing code that stopped working, or new code?
<jcsackett> leonardr: the addition of webservice and the tests are new.
<henninge> abentley: yes, or did we already chaange it?
<leonardr> jcsackett: can you point me to some place in launchpad where you do something similar but it works?
<henninge> abentley: I am west of you this and next week, remember?
<gary_poster> bigjools: the error reporting utility needs to be alerted of the error somehow.  The Zope publisher usually is responsible for this.  If you are outside of the publishing loop, you need to get the utility and call...whatever the method is.  We have a subclass in the tree you can look at; I'll help dig it up if you like.
<abentley> henninge, no, I forgot.
<jcsackett> leonardr: as far as exporting an error by using webservice_error? yeah, one sec.
<henninge> np ;)
<abentley> bac, ping
<bigjools> gary_poster: I thought the logger did all that?
<bac> hi abentley
<bigjools> this is in a script, sorry if that wasn't clear
<gary_poster> bigjools: what logger?  The Python logger?  If so, no.
<abentley> henninge and I are OCR on Monday, and we are now in the same squad.  We'd like to change schedules so that we aren't OCR on the same day.
<gary_poster> I think the error reporting utility is responsible for the log, IIRC
<bac> abentley: ok.  let me look at the schedule
<gary_poster> errors in the log, that is
<bigjools> yeah, the python logger.  I am testing for a new oops after using logger.error() and there isn't one.
<gary_poster> nope
<gary_poster> won't work
<gary_poster> you need to tickle the error reporting utility
<bigjools> where does it like to be tickled?
<bac> abentley: would you like to move to wednesday?
<bigjools> I don't want to make it wee
<abentley> bac, sure.
<gary_poster> lol
<bigjools> or point me at an example :)
<abentley> bac, starting next week?
<bac> abentley: ok, i'll move you as i've got the wiki page open
<bac> abentley: sure
<gary_poster> bigjools: bzr grep IErrorReportingUtility shows you some usages...lib/canonical/launchpad/webapp/errorlog.py has our subclass ... still looking about
<abentley> bac, thanks!
<henninge> bac, abentley: thanks
<gary_poster> bigjools: .raising, looks like
<gary_poster> def raising(self, info, request=None, now=None):
<leonardr> jcsackett: i think there might be a bug such that the normal error view lookup doesn't happen on a DELETE request
<lifeless> gary_poster: how do you feel about including a 'and if you saw an OOPS please include the OOPS number' in those guidelines?
<gary_poster> where info is output of sys.exc_info()
<gary_poster> humina?
<gary_poster> (==nonsense syllable)
<gary_poster> which guidelines?
<lifeless> gary_poster: stub added logging->oops glue
<bigjools> gary_poster: I'm confused - I thought the logger auto-oopsed on level > info
<lifeless> it does
<lifeless> in scripts
<bigjools> but not in tests?
<gary_poster> I was not aware he did it that way
<lifeless> a test per se doesn't know if its a) a script or b) for the webapp
<gary_poster> cool
<bigjools> ok - meh
<gary_poster> bigjools: the webapp still does stuff the way I said, I expect
<bigjools> right, makes sense
<lifeless> gary_poster: see %(asctime)s %(levelname)-7s %(message)s'
<lifeless> ./lib/canonical/launchpad/scripts/logger.py
<gary_poster> lifeless: oh! I got your original question now
<lifeless> bah, the second line :)
<gary_poster> yeah, good idea to ask for an oops
<jcsackett> leonardr: so that prevents the webservice_error stuff from happening? b/c the code path raising the exception is clearly happening.
<gary_poster> I'll add it
<bigjools> lifeless: in that case, do you have any suggestions on how to test this change to not crash but to log and oops?
<lifeless> bigjools: sure, is the code a) script only, b) both, c) webapp
<jcsackett> leonardr: an example of this working is with the JoinNotAllowed error in lp.registry.errors, tested in lp.registry.tests.test_team_webservice.
<lifeless> I've sligtly different suggestions for each
<jcsackett> i can throw together a paste, if you like.
<bigjools> lifeless: there's a unit test for the method that's used in the script
<leonardr> jcsackett: look in lazr.restful src/lazr/restful/_resource.py, ReadWriteResource.__call__
<gary_poster> lifeless: it is already there: "If the bug you found generated an OOPS id, please include it here as well."
<leonardr> that's the code that catches exceptions
<leonardr> that's the code that should be running
<jcsackett> leonardr: looking now.
<lifeless> gary_poster: sweet
<bigjools> lifeless: this is the change so far but the test isn't working http://pastebin.ubuntu.com/557822/
<lifeless> gary_poster: I was going off th econversation here only
<leonardr> i suggest you put a breakpoint in that 'except' clause and see how it's being handled
<gary_poster> understood, np
<lifeless> bigjools: I would be inclined to:
<lifeless>  - put the logging call at the point where we loop
<lifeless>  - change the AssertionError to a custom exception (eg. UnparsableDependencies)
<bigjools> did you not look at the diff? :)(
<lifeless> bigjools: the test then can be direct (doesn't care about oopses), and the loop should be in the script specific code which has a test helper that wires it all up
<bigjools> but go on
<lifeless> bigjools: I am looking at it
<lifeless> bigjools: perhaps I'm confused
<bigjools> I really, really hate this auto-oops logging thing
<lifeless> bigjools: clearly I'm confused
<bigjools> lifeless: I don't really understand what you mean about script-specific code.  This is a very straightforward unit test.  Sure, we can test that it throws a custom exception but that doesn't test that we've logged an oops.  I suspect that I need to Popen the script, which is nasty.
<lifeless> bigjools: you don't
<lifeless> hang a sec and I'll dig up the test infrastructure
<bigjools> ok, thanks
<lifeless> what I mean by script specific code is this:
<lifeless>  - code in the webapp can assume some stuff, like transactions-around-requests, auto-oops creation etc
<lifeless>  - code in scripts cannot assume that stuff - it doesn't have a proper request context, so everything related to that (including the webapp adapter) becomes tricky
<bigjools> ok. this, I know. :)
<lifeless> so code that is shared between them needs to communicate (probably via exceptions) up to a layer which is script only
<lifeless> otherwise we'll log to the log file and *not* generate an OOPS in the webapp
<bigjools> this method is only ever called from a script
<lifeless> I've looked a bit now and it looks to me like retryDepWaiting is script specific
<lifeless> one sec ELOCAL
<bigjools> updateDependencies you mean
 * bigjools also brb, biological emergency
<lifeless> bigjools: no, I mean retryDepWaiting
<lifeless> which calls updateDependencies
<bigjools> ok
<lifeless> however
<lifeless> both appear script only
<lifeless> so - its a huge shrug and whatever looks cleanest to you
<lifeless> anyhow
<lifeless> lets answer your question
<lifeless> rev 11637 added the feature
<lifeless> bigjools: ok, so anything that calls (in process) LaunchpadScript as a whole thing will add an OopsHandler to logging on the fly
<lifeless> bigjools: I can see two options
<lifeless> bigjools: just check for an error level log message
<lifeless> bigjools: or install OopsHandler manually and then check for an oops
<lifeless> bigjools: personally, if I was making the change, I'd change the raised exception and catch it in retryDepWaiting:
<lifeless> try:
<lifeless>     build.updateDependencies()
<lifeless> except <.>:
<lifeless>     continue
<lifeless> bigjools: just looking at it it seems tricky to actually skip it otherwise
<bigjools> yeha
<jcsackett> sinzui: did we decide 5 or 5:30 EST for group mumble?
<sinzui> jcsackett: 5:30 EST
<jcsackett> cool. thanks.
<poolie> hi jc, sinzui
<sinzui> a stealth greeting
<sinzui> hi poolie
<lifeless> sinzui: hey, disabled users 404 right ?
<lifeless> so bug 283830 is perhaps obsolete ?
<_mup_> Bug #283830: Don't allow crawlers to index the top-level user pages on lp subdomains if the user is deactivated <lp-foundations> <Launchpad itself:Triaged> < https://launchpad.net/bugs/283830 >
<poolie> hi lifeless
<lifeless> hi poolie
<sinzui> lifeless: It is still legit
<poolie> lifeless, i tried out a twisted client along the lines of my rfc, and i liked it
 * sinzui saw that some pages are still being indexed
<lifeless> flacoste: should we make official bugtags for all LEP's ?
<lifeless> flacoste: e.g. issuetracker
<flacoste> lifeless: in the sense of making the existing bug tags official?
<lifeless> flacoste: yeah, so they autocomplete.
<lifeless> flacoste: only 'official' bug tags autocomplete
<flacoste> lifeless: i think it's a good idea
<poolie> hi flacoste
<flacoste> hi poolie
<poolie> i think we normally catch up in 90m,; i'm ready whenever you are
<flacoste> poolie: i thought we were skipping this week, as i didn't thought you'd be around
<flacoste> poolie: but i could do a call in 60 minutes
<poolie> just a brief call could be good
<flacoste> poolie: i have to watch over the babies while Amelie picks up Jules in 30 minutes, it's -21C over here, so she's not getting them out
<poolie> wow
<flacoste> and that's after the sun warmed things up
<flacoste> it was -27C this morning
<flacoste> been years since it wasn't so cold
<flacoste> we usually never cross -23
<flacoste> and that's real temperature
<thumper> high of 23C here today
<flacoste> not inflated with wind factor and such bs
<flacoste> thumper: want to trade place?
<thumper> flacoste: nope
<lifeless> flacoste: I need to take Lynne to the hospital today; I think we're all caught up though. So perhaps we skip today?
<poolie> 35C here yesterday when i was walking around
<flacoste> lifeless: yep, that was the plan
<poolie> felt weird after last week
* lifeless changed the topic of #launchpad-dev to: Performance Tuesday! | Launchpad development https:/â/âdev.launchpad.net/â | PQM is open | firefighting: builds are not retrying at the moment | On call reviewer: henninge, abentley | reviewing: -, - | queue: [] | https://code.launchpad.net/launchpad-project/+activereviews
<poolie> are recipes still in beta? https://help.launchpad.net/Packaging/SourceBuilds/GettingStarted says they are
<poolie> s//closed beta
<lifeless> yes
<lifeless> erm
<lifeless> I don't think it was ever /closed/
<lifeless> its a beta and folk can participate
<poolie> but you don't need to be in a special team?
<poolie> then i'll edit it
<lifeless> you do have to be in the special team
<lifeless> thats how you participate
<poolie> ah, but it's an open team
<lifeless> and the main beta team is in it
<thumper> StevenK: https://bugs.launchpad.net/launchpad/+bug/683798
<_mup_> Bug #683798: Recipes for merged team still showing up in +recipes page <404> <lp-code> <recipe> <Launchpad itself:Triaged> < https://launchpad.net/bugs/683798 >
<thumper> StevenK: mumble is being crackful
<thumper> StevenK: https://bugs.launchpad.net/launchpad/+bug/669288
<_mup_> Bug #669288: person merge code needs to handle colliding recipes <lp-code> <recipe> <Launchpad itself:Triaged> < https://launchpad.net/bugs/669288 >
<StevenK> thumper: Oh, awesome
<StevenK> thumper: And yes
<thumper> StevenK: you get to kill two bugs with one branch
<StevenK> Right
<thumper> StevenK: I've left mumble
<thumper> nothing as frustrating as not being able to hear properly
<wgrant> Morning.
<Ursinha> wgrant, morning
<jml> huwshimi: https://bugs.launchpad.net/launchpad/+bug/325982
<_mup_> Bug #325982: Search URL is long and hard to paste <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/325982 >
<wallyworld> morning
<wgrant> Morning wallyworld.
<wgrant> You got home OK?
<wallyworld> wgrant: yeah, to find a 10000 jobs waiting for me to to, like mowing the lawn etc.
<wgrant> Heh.
<wallyworld> wgrant: i missed the shared shuttle cause there was a shooting on the DART and the train was delayed
<wallyworld> there were cops everywhere
<wallyworld> not on mt train but further up the line or so i was told
<wgrant> wallyworld: Ah, you were out somewhere on the DART?
<wallyworld> wgrant: went with stub and jtv to a Fry's to see if there were any Kindles for jtv. rather large but uninspiring store. prices no better than here
<wgrant> Ahh :(
<wallyworld> there were about 6 different Ubuntu books on the shelf though :-)
<wallyworld> we got photos :-)
<sinzui> wgrant: mumble in Launchpad/Green
<wgrant> sinzui: Give me a sec.
<wallyworld> thumper: ping
<thumper> wallyworld: hi
<wallyworld> can you give me a clue as to what broke that required the show related branches rollback?
<thumper> wallyworld: yeah, but I'm starving
<thumper> wasting away to nothing
 * thumper needs food badly
<wallyworld> :-)
<thumper> wallyworld: skype or mumble?
<wallyworld> ping me when you're fat again
<wallyworld> thumper: mumble when you have eaten
<thumper> wallyworld: lets do it now
<wallyworld> thumper: ok
<lifeless> back
<wgrant> maxb: Hi.
#launchpad-dev 2011-01-25
<poolie> hi wallyworld,thumper
<poolie> wgrant
<wallyworld> poolie: g'day
<lifeless> wallyworld: so remember the query parameters thing
<StevenK> wallyworld: O hai. How were your flights?
<wgrant> Morning poolie.
<wallyworld> lifeless: yeah
<lifeless> wallyworld: matsubara says that oops-tools /should/ handle queries with parameters already substituted
<lifeless> wallyworld: so the simplest thing is ok to do.
<wallyworld> lifeless: \o/
<lifeless> if it doesn't handle them, it has code to normalise that we can just fiddle with
<lifeless> wallyworld: I thought that might make you happy ;)
<thumper> hi poolie
<wallyworld> lifeless: very, cause it means i already have the coding done :-)
<wallyworld> i need to finish a couple of other things and then i'll get a mp up
<wallyworld> StevenK: flights were good - had a spare seat next to me :-)
 * StevenK grumbles
<wallyworld> StevenK: your flight was full then?
<StevenK> Utterly
<StevenK> Surprised they weren't people being asked to sit on others laps
<wallyworld> wow. you fly to sydney and then cancerra?
<wallyworld> s/cancerra/canberra
* henninge changed the topic of #launchpad-dev to: Performance Tuesday! | Launchpad development https:/â/âdev.launchpad.net/â | PQM is open | firefighting: builds are not retrying at the moment | On call reviewer: - | reviewing: - | queue: [] | https://code.launchpad.net/launchpad-project/+activereviews
<StevenK> wallyworld: Why would I do that?
<wallyworld> StevenK: i thought you lived in canberra?
<StevenK> wallyworld: Heh. No.
 * wallyworld should change his nic to rambo
<StevenK> Yes!
<rambo> StevenK: don't mess with me
<StevenK> Haha
<lifeless> righto
<lifeless> time to fire up a garbo job I think
<lifeless> I feel the need to code
<lifeless> thumper: do you know if there is a bug open about the 'edit' link for a reviewers prior vote taking you to a different page (vs ajaxing it up)
<thumper> lifeless: not sure
<lifeless> thumper: fyi 707217
<lifeless> bah
<lifeless> thats a bug number
<wgrant> Anyone to review https://code.launchpad.net/~wgrant/launchpad/bug-707189-forgetful-copier/+merge/47341?
<lifeless> I'll do it
<wgrant> Thanks.
<wgrant> It's pretty simple.
<lifeless> uhm
<lifeless> the unmerged revisions list is huge
<wgrant> It is, yes.
<wgrant> I was wondering about that.
<lifeless> I wonder if jtv's branchrevision change caused that?
<wgrant> I presume so.
<lifeless> it didn't look like it could
<wgrant> No, it didn't.
<wgrant> But I don't see what else it could be.
<wgrant> Was going to investigate after getting this through.
<lifeless> wgrant: while I read, care to file a bug, tagged regression
<lifeless> (and thus critical)
<StevenK> lifeless: I'm ready to +1 it, if you want to mentor my rewview
<StevenK> *review
<lifeless> sure
<poolie> http://www.flickr.com/photos/mbp_/5386185146/
<lifeless> wgrant: tweak
<lifeless> StevenK: you need to read a little closer I think :)
<StevenK> Hm
<StevenK> - distroarchseries=distroarchseries)
<StevenK> 26	+ distroarchseries=arch)
<lifeless> thats the actual bugfix
<StevenK> wgrant: Isn't that supposed to be target_archs ?
<lifeless> the rest is sugar to make it clearer, and tests
<wgrant> lifeless: Ugh, sorry, copied that from a test above.
<lifeless> wgrant: I suspected :)
<wgrant> StevenK: 'for arch in target_archs:' precedes that.
<lifeless> wgrant: there may be others, that stood out to me
<wgrant> StevenK: I could rename arch to target_arch, i suppose.
<wgrant> lifeless: I deliberately fixed them all.
<StevenK> wgrant: No, it's fine
<wgrant> But apparently not.
<wgrant> lifeless: https://code.launchpad.net/~jtv/launchpad/forget-branchrevision-id/+merge/46955 <- in lib/lp/code/model/branchmergeproposal.py, the SourceRevision.branch_id == self.target_branch.id condition is new.
<wgrant> not bad timing.
<wgrant> jtv: Morning.
<jtv> hi wgrant!  Evening here.
<wgrant> jtv: Any ideas on why https://code.launchpad.net/~wgrant/launchpad/bug-707189-forgetful-copier/+merge/47341? has lots of merged revisions listed as unmerged?
<wgrant> There is an extra condition that you added in getUnlandedSourceBranchRevisions.
<maxb> wgrant: hi
<wgrant> maxb: How did you notice the lpia issue?
<wgrant> I presume you don't actually have any lpia users.
<maxb> another package unexpectedly went to dependency-wait
<wgrant> Ah, of course.
<maxb> You're right, if it had been a leaf package, I'd never have noticed :-)
<LPCIBot> Project devel build (388): FAILURE in 5 hr 27 min: https://hudson.wedontsleep.org/job/devel/388/
<lifeless> hmm
<lifeless> I wonder if we can get rid of the tree structure of messages for bugs
<lifeless> it seems to add significant overhead
<wgrant> Aha.
<wgrant> The librarian failures have now shown up on Hudson.
<lifeless> aha?
<poolie> lifeless, it would give me a warm glow if you run and like Wrested
<poolie> not necessarily right now
<james_w> hi poolie
<lifeless> poolie: I've put it in my toys to play with box
<james_w> poolie, did you start a new library?
<lifeless> that box is a little full right now though :(
<poolie> hi james_w
<poolie> :)
<lifeless> 'win' https://hudson.wedontsleep.org/job/devel/388/changes
<thumper> damn
<thumper> I'm slightly confused
<thumper> I remember in dallas removing a PPA from a recipe and having it work
<thumper> but now it doesn't
<thumper> and according to the source, it shouldn't have worked before
<lifeless> abentley: (trying to rule stuff out) did you land your translations patch via ec2 land, or direct ?
<thumper> and I don't know why it did
<poolie> james_w, i did start a new library
<poolie> i think it currently imports something from txrestfulclient but not very much
<poolie> i don't mean any disrespect
<poolie> i just wanted to start from zero so i could understand what was happening
<poolie> lplib has kind of a lot of layering between the app and what's on the wire
<poolie> i would be happy to turn this into a patch to txrestfulclient etc
<poolie> but it will probably change the apis a lot
<james_w> right, I've just seen your mail
<james_w> replied
<james_w> (twice, sorry)
<poolie> np, thanks
<poolie> turning them into lower and higher layers could be good
<poolie> i still think getting an object and then separately populating it is likely to cause bugs
<poolie> but imbw
<james_w> agreed
<james_w> that's what I meant by using this as the building blocks, sorry for not being clear
<james_w> that was me trying to keep it familiar to people who had used lplib, but that wasn't the right approach
<poolie> right
<poolie> are there enough users of txrestfulclient at the moment that we need to worry about keeping it's api stable?
<poolie> (that probably is true for launchpadlib for example)
<james_w> nope
<rambo> thumper: if a branch has a sourcepackage, did we want to include the source package's linked_branches in the related branches list?
<thumper> rambo: maybe?
<rambo> and if yes, then do we need to collect the linked_branches for each sourcepackage in product.distrosourcepackages when the branch is a product branch?
<rambo> would be good to have a proper dataset to try it on
<rambo> StevenK: can we have a mumble chat?
<StevenK> rambo: Sure
<rambo> StevenK: see you at the Blue Oyster
<rambo> s/Blue Oyster/Blue Room
<StevenK> rambo: You're tempting me to find a clip of the Blue Oyster Bar music
 * rambo LOL
<lifeless> ahha
<lifeless> 84 /  192  BugTask:+index is back at ze top
<rambo> StevenK: you got cold feet?
<StevenK> rambo: Sorry, you're breaking up very badly
 * rambo :-(
<lifeless> wgrant: OOPS-1850PPA1004
<huwshimi> Does anyone know where the javascript is that handles the advanced search? I've been told it exists, but I can't see it anywhere.
<lifeless> huwshimi: what javascript ?
<lifeless> huwshimi: its a form submission
<lifeless> (which is why the url is terrible; stock html forms don't know about 'optional' fields.)
<huwshimi> lifeless: This is what is confusing me.
<jtv> wgrant: sorry urgent stuff elsewhereâ¦ don't know what's wrong with that MP you pointed out, but could it be that the branch came off devel?  It's proposed for merging into db-devel.
<lifeless> huwshimi: you're talking about e.g. https://bugs.launchpad.net/launchpad-project/+bugs?advanced=1
<huwshimi> lifeless: Yeah
<lifeless> jtv: lp:launchpad is devel these days
<jtv> oh!
<huwshimi> lifeless: so I'm having a look at simplifying those urls and now I see that the suggestion wasn't to modify the javascript but rather to add it :)
<lifeless> huwshimi: :)
<lifeless> man, we have a tonne of boilerplate in the source for that page
<lifeless> we redefine make_picker for each person picker instance on the page :)
<lifeless> :( :( :(
<huwshimi> lifeless: Thanks. There was a reason I couldn't find it beyond my lack of familiarity with the source
<lifeless> yeah, it doesn't exist :>
<wallyworld> thumper: just talked with stevenk, will include link_branches off the context branch's sourcepackage in the related branches list
<wallyworld> s/link_branches/linked_branches
<thumper> wallyworld: ok
<lifeless> ok, i'm going crazy
<lifeless> whats the difference between
<lifeless> IBugComment
<lifeless> and
<lifeless> IBugMessage
<lifeless> other than spelling
<lifeless> ... one is in browser code
<lifeless> <argh>
 * thumper wandesr off
 * thumper needs coffee badly
 * StevenK blinks at his new MP including lots of unmerged revs
<wgrant> StevenK: We believe that it's fallout from jtv's BranchRevision.id changes a couple of days back.
<wgrant> I haven't investigated thoroughly.
<StevenK> Right
<StevenK> That makes a vague sort of sense
<StevenK> thumper: rofl:
<StevenK>         # XXX MichaelHudson 2010-01-13: Write _mergeSourcePackageRecipes!
<StevenK>         #self._mergeSourcePackageRecipes(cur, from_id, to_id))
<thumper> heh
<lifeless> StevenK: so, I think you're changing the wrong part of the system
<lifeless> StevenK: I've suggested a better change in the review.
<lifeless> StevenK: the red flag that went off was 'filtering in python'
<StevenK> You haven't yet, it seems
<StevenK> lifeless: I'm about to head out, but I would like to get the interval request_daily_builds runs at down to 10 minutes, which I think is sufficent to not really worry with more granularity
<lifeless> StevenK: why 10 minutes? whats the overall goal here.
<StevenK> Out of time, sorry
<lifeless> later then
<lifeless> anyhow, my comment applies regardless of the granularity
<LPCIBot> Project devel build (389): STILL FAILING in 5 hr 5 min: https://hudson.wedontsleep.org/job/devel/389/
<LPCIBot> * Launchpad Patch Queue Manager: [r=abentley][ui=none][bug=705197] Replaces use of storm.base.Storm
<LPCIBot> with a new class, Stormbase,
<LPCIBot> which descends from base.Storm but adds SQLObject's __storm methods.
<LPCIBot> * Launchpad Patch Queue Manager: [r=deryck][ui=none][no-qa] Migrate translations to new API
 * huwshimi signs off
<adeuring> good morning
<al-maisan> moin adeuring :)
<adeuring> hi al-maisan!
* gmb changed the topic of #launchpad-dev to: Performance Tuesday! | Launchpad development https:/â/âdev.launchpad.net/â | PQM is open | firefighting: builds are not retrying at the moment | On call reviewer: gmb | reviewing: - | queue: [] | https://code.launchpad.net/launchpad-project/+activereviews
<Ursinha> good morning launchpad
<jml> Ursinha: good morning.
* leonardr changed the topic of #launchpad-dev to: Performance Tuesday! | Launchpad development https:/â/âdev.launchpad.net/â | PQM is open | firefighting: builds are not retrying at the moment | On call reviewer: gmb, leonardr | reviewing: - | queue: [] | https://code.launchpad.net/launchpad-project/+activereviews
<deryck> Morning, all.
 * gmb lunches
<jcsackett> morning, all.
<bac> benji: do you have a prefered day to do OCR?  since i'll be mentoring you we'll both need to commit to that day.  my slot is currently friday.
<deryck> adeuring, ping for standup
<adeuring> oops, thanks
<adeuring> abentley: I'm pushing lp:~adeuring/launchpad/translation-fixing3 to LP righ now
<dpm_> hi jml, quick question: if I want to know in more detail how the LP api docs are generated and published, who should I best talk to?
<leonardr> dpm_: i can help you
<dpm_> leonardr, ah, great.
<dpm_> leonardr, I've got a few questions on that, let me send you an e-mail.
<leonardr> ok
<abentley> adeuring, ack
<jcsackett> where can one lookup what the timeout thresholds are for qastaging/staging?
<jml> jcsackett: good questions
<flacoste> jml: my calendar still show the product stand-up, i thought you changed it yesterday, google brokeness or you reverted that devision?
<jml> flacoste: starting next week
<flacoste> jcsackett: lp:lp-production-configs
<flacoste> jml: ok!
<jcsackett> flacoste: thanks.
<jml> flacoste: you are welcome to join us.
<sinzui> salgado: do you have time to review https://code.launchpad.net/~wallyworld/launchpad/recipe-find-related-branches/+merge/47367
<salgado> sinzui, sure. can it be after my lunch?
<sinzui> I think that will be fine. wallyworld is asleep for the next few hours
<salgado> cool; I'll do it then
<bigjools> jml: hi
<jml> bigjools: hi. on a call atm.
<bigjools> ah ok, will wait around
<bigjools> jml: so when you're off your call I could do with some help fixing an ec2/paramiko problem, it can't find my ssh keys :(
<jml> bigjools: how can I help you
<jml> bigjools: ahh.
<bigjools> heh, snap
<bigjools> I figured you're a knowledgeable chap
<jml> bigjools: can you paste the error?
<bigjools> ec2: ERROR: You must have an ssh agent running with keys installed that will allow the script to access Launchpad and get your branch.
<jml> I know some things
<bigjools> if I run the paramiko code it uses, I can re-create.  get_keys() returns nowt.
<bigjools> I do have an ssh agent running
<jml> bigjools: what keys are in the agent?
<bigjools> oh god, do I need to manually ssh-add them? .... :/
<jml> bigjools: umm... I don't *think* so.
<jml> bigjools: ssh-add -l should list
<bigjools> I just did ssh-add and it works now
<jml> ahh.
<bigjools> :(
<jml> it ought to just get them from... somewhere
<leonardr> gmb, or anyone else who wants an easy branch to review: https://code.launchpad.net/~leonardr/lazr.restful/web-link-wadl/+merge/47404
 * jml feels embarrassed for not knowing
<bigjools> it ought to
<gmb> leonardr: Sure, I'll take easy work.
<bigjools> this is my maiden ec2 run
<jml> bigjools: I can't recall manually adding a key using ssh-add on this laptop
<bigjools> it might be an artefact of using kubuntu
* gmb changed the topic of #launchpad-dev to: Performance Tuesday! | Launchpad development https:/â/âdev.launchpad.net/â | PQM is open | firefighting: builds are not retrying at the moment | On call reviewer: gmb, leonardr | reviewing: leonardr, - | queue: [] | https://code.launchpad.net/launchpad-project/+activereviews
<maxb> Normally you need to ssh-add keys if you're using ssh-agent
<jml> bigjools: maybe. your key is in ~/.ssh/id_[rd]sa(.pub)?, right?
<bigjools> that's what I thought too
 * gmb thinks that we need a better way to maintain state on the review queue than using the topic. There's too much of it to be helpful in this channel.
<maxb> Later Ubuntu's gnome keyring stuff does weird stuff, which I carefully turn off
<bigjools> jml: yes
<jml> maxb: maybe that's it.
<bigjools> you'd think that paramiko would look in ~/.ssh
<jcsackett> gmb: yes.
<jcsackett> i was noting that yesterday.
<gmb> jcsackett: I'll mail the list about it after I've done this review. Maybe we should just use +activereviews and claim reviews as we start them.
<jml> gmb: that seems a sensible approach.
<gmb> Yay, I said something that made sense. The jet lag must be receding.
 * gmb waits for the obvious rejoinder
<bigjools> I am all out of joke juice this morning
<gmb> leonardr: In your diff it looks like there's a missing or extra single quote:
<gmb> 113	+      <param style="plain" name="web_link" path="$[web_link']"
<gmb> (specifically $[web_link'])
<leonardr> gmb, thanks
<gmb> leonardr: r=me.
* gmb changed the topic of #launchpad-dev to: Performance Tuesday! | Launchpad development https:/â/âdev.launchpad.net/â | PQM is open | firefighting: builds are not retrying at the moment | On call reviewer: leonardr | reviewing: - | queue: [] | https://code.launchpad.net/launchpad-project/+activereviews
 * gmb goes off call
<jcsackett> sinzui: do you recall where one finds oops logs for qastaging?
<sinzui> jcsackett: yes
<sinzui> jcsackett: https://devpad.canonical.com/~lpqateam/ contains the reports we are gathering and the historical reports
<jcsackett> sinzui: ah, thanks.
 * jcsackett bookmarks
<adeuring> henninge_: can you give me a hint how to replace the call of updateTranslation() in stories/standalone/xx-translationmessage-translate.txt?
<jcsackett> sinzui: any notion where the raw oops files go? i recall you pointing it out to me once. i need to fetch a particular oops that happened on qastaging while i was qaing.
<jcsackett> i want to make certain my branch is not the cause.
<sinzui> jcsackett: yes, somewhere in var
<sinzui> I can look in a minute
<jcsackett> sinzui: i can dig through var, just needed to narrow it down from /
<jcsackett> sinzui: found what i needed, thanks. :-)
<bryceh> deryck, we're seeing Return of the Living Unknown with bug #707478
<_mup_> Bug #707478: Freedesktop bug watches are (incorrectly) updated with "importance unknown" <Launchpad itself:Confirmed> < https://launchpad.net/bugs/707478 >
<deryck> bryceh, ah, that sucks.
 * deryck looks more closely at the bug
<deryck> bryceh, so if this just appeared today, I would guess something suddenly changed in the upstream tracker, as you noted in the bug.
<deryck> bryceh, did you notice anything upstream that seems different?
<bryceh> deryck, no, however my upstream cgi tool thingee quit working about a month ago
<bryceh> which I suspect was due to upstream changes to their new bug form
<deryck> ah
 * bryceh goes to look
<deryck> bryceh, if you can spare the time to hunt down what's changed upstream, and dump it in the bug, that bug gets seriously easier to fix.
<bryceh> <brian> bryyce: Do you know why all these bug watches are getting their importance set to unknown?
<bryceh> <kees> was just going to ask about that -- it looks like a bug in the gnome bugzilla status parser?
<jml> benji, gary_poster: wikipedia's zope article has this sentence: "BlueBream (earlier called Zope 3) is less widespread but underlies several large sites, including Launchpad"
<jml> I'm not quite sure what to make of it :)
<gary_poster> heh
<gary_poster> until we get rid of zope.app.*, I'd say they have a point, jml :-)
* bigjools changed the topic of #launchpad-dev to: Performance Tuesday! | Launchpad development https:/â/âdev.launchpad.net/â | PQM is open | firefighting: - | On call reviewer: leonardr | reviewing: - | queue: [] | https://code.launchpad.net/launchpad-project/+activereviews
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build (390): FIXED in 4 hr 59 min: https://hudson.wedontsleep.org/job/devel/390/
<bigjools> how freakin awesome is this https://code.launchpad.net/+daily-builds - 250!
<jml> bigjools: that is pretty good :)
<jml> I like that it goes down to 158 when you select "last 30 days". Software is hard.
<bigjools> it's been about 1 year since we started the daily builds project
<jml> yeah.
<bigjools> the hard bit went surprisingly quickly
<bigjools> then the niggles came
<jml> incidentally, we really need to display lists of data better than that.
<leonardr> hey, gary
<leonardr> a while ago i had a lazr problem with tests picking up the system libraries instead of the libraries in the source tree i was testing
<leonardr> you fixed this by changing the [interpreter] recipe to z3c.recipe.scripts
<leonardr> i believe you helped me fix lazr.restful
<leonardr> i'm having the same problem with lazr.restfulclient now, and i can't figure out the solution
<bryceh> deryck, well, dunno if this has to do with the Importance bug, but I sorted out the problem with my upload tool - had to switch from POSTing the data to bz, to doing it via GET
<gary_poster> leonardr: I didn't fix lazr.restful
<dobey> how often is loggerhead updated for bazaar.launchpad.net?
<gary_poster> It is using a recipe that isn't upgraded to the new stuff
<gary_poster> lazr.restfulclient should be easier
<deryck> bryceh, hmm, ok.  likely not causing the importance issue, but could be related to the same change.
<bryceh> deryck, https://bugs.freedesktop.org/page.cgi?id=release-notes.html isn't mentioning anything obvious to have to do with that change
<gary_poster> leonardr: zc.recipe.egg -> z3c.recipe.scripts, and then use the new zc.buildout
<gary_poster> (1.5.2 IIRC)
<deryck> bryceh, I'll triage it, and hopefully someone on one of the maintenance teams can take a look.
<bryceh> deryck, also btw kees' assumption of breakage with gnome bugzilla turned out to be a false positive one we investigated.  So this seems localized to the fdo bz
<leonardr> gary: i think i'm stuck on "use the new zc.buildout". how?
<deryck> bryceh, ok, thanks
<bryceh> deryck, thanks.
<gary_poster> leonardr: 1) versions.cfg 2) new bootstrap.py 3) maybe update Makefile if it seems to refer to a specific version of buildout when it calls bootstrap
<leonardr> ok, trying it
<jml> lifeless: what steps are left for removing our dependence on z.testing layers?
<jml> (it seems a thing I can tug away at in my spare hacking time)
<leonardr> gary: i'm going to paste this stuff in rather than try to try to figure out exactly what's going on
<leonardr> The version, 1.2.3b2, is not consistent with the requirement, 'zc.recipe.egg>=1.3.0'.
<leonardr> ah, maybe i figured it out
<gary_poster> k
<leonardr> gary: i got it to work, but only with allow-picked-versions=true. what do i need to do now?
<gary_poster> leonardr: figure out the versions that are being used, and update them in versions.cfg.  To figure out, ./bin/buildout -vv | grep 'Picked:' .  I usually then sort the output in Python or my editor or something
<gary_poster> (and remove dupes)
<leonardr> gary, thanks a lot
<gary_poster> np leonardr
<jml> jelmer: btw, what was the RT for the bzr/bzr-builder stuff for the buildds?
<leonardr> gary: i can now run buildout but i'm still getting the error. i'm going to try completely form scratch, but any other ideas?
<gary_poster> leonardr: still what error?  you mean, finding some package or other?
 * gary_poster has call in 3
<leonardr> gary: when i run the tests it still uses the local packages
 * leonardr can work on something else, np
<gary_poster> leonardr: put branch up somewhere and tell me what it is and I will look
<leonardr> i think there's a bootstrap.py problem
<gary_poster> ah, may be
<jcsackett> leonardr: i have some  more questions from what i was asking you about yesterday. have some time?
<leonardr> jcsackett, sure
<jcsackett> leonardr: so, i ran with pdb in the section of _resource you recommended, and it's entering there but skipping the "if IClient..." bit.
<jcsackett> digging in, i discovered that webservice_error only works on exceptions raised by the exported method. but i've set the exported method to reraise, and still no dice. thoughts?
<jcsackett> leonardr: if mumble is better for you than irc, let me know. this does get muddled in text form. :-P
<leonardr> gary, whenever you have time, lp:~leonardr/lazr.restfulclient/test-2
<leonardr> ok, jcsackett...
<gary_poster> ack
<jcsackett> leonardr: yeah. lot of info to parse. feel free to force me to give it in smaller chunks. :-)
<leonardr> jcsackett: try calling lazr.restful.error.expose on the exception before re-raising
<leonardr> we seem to have switched to a system where individual exceptions must be marked--i don't remember this code
<jcsackett> leonardr: got it in one. i had been told to avoid expose in the past, and evidently i then forgot about it. thanks for the reminder.
 * jml dinner
<lifeless> jml: graph optmisation for fixtures
<jml> lifeless: which I guess depends on a declarative syntax for fixtures.
<lifeless> jml: I don't think it does
 * jml blinks
<lifeless> I think it depends on an external dependency graph api to be exposed by fixtures
<bigjools> lifeless: morning
<jml> lifeless: hmm. I think I remember you talking about this before and me not quite understanding.
<lifeless> hi bigjools
<jml> lifeless: I will consult my archives.
<jml> lifeless: meanwhile, I solicit your feedback on https://bugs.launchpad.net/launchpad/+bug/707524
<_mup_> Bug #707524: Debugging branch access problems requires running custom scripts against live data <Launchpad itself:Triaged> < https://launchpad.net/bugs/707524 >
<bigjools> lifeless: I've been experimenting with queries/schemas and adding the debversion_sort_key as a column turns a 350ms query into a 200ms one on dogfood.  I'm going to prepare a schema patch in advance of code changes if you still agree it's a good direction?
<lifeless> jml: I will look at that shortly
<lifeless> bigjools: I think that that is a conservative denormalisation we should be able to trivially keep synced which can offer significant performance benefits
<lifeless> bigjools: e.g. 'fine by me'
<bigjools> :)
<lifeless> bigjools: I'm going to send a mail this morning as perf tuesday wrapup talking about the analysis I did on this
<bigjools> great
<bigjools> once we get the new column I can stick an index on it as well
<lifeless> there may be other such cross-table situations where we have missed the actual cost
<lifeless> bigjools: what is binarypackagename for ?
<gary_poster> leonardr: The bootstrap you have in test-2 is  not right.  use the one from launchpadlib
<bigjools> lifeless: in what context?
<lifeless> Launchpads db schema
<bigjools> it, er, stores binary package names :)
<leonardr> gary: i'm pretty sure i did that before i pushed. let me double check
<leonardr> obviously i did something wrong if you're not seeing that though
<gary_poster> on call again :-P
<lifeless> bigjools: why ;) - was it simply 'well thats third normal form', or something deeper ?
<lifeless> I need to go cook breakfast
<lifeless> back shortly
<bigjools> lifeless: it's been like that since well before my time - I assume it was to save db space
<lifeless> bigjools: note that if it was inlined into binary package release we could sort from index
<lifeless> (even without a debversion_sort_key column)
<bigjools> possibly
<lifeless> (we probably would want a debversion_sort_key column still)
<bigjools> yes, it's effectively pre-caching the result of a slow function
<lifeless> bigjools: sinzui: bug 707478 - would one of you like to nab this as a zomg situation ?
<_mup_> Bug #707478: Freedesktop bug watches are (incorrectly) updated with "importance unknown" <bugwatch> <Launchpad itself:Triaged> < https://launchpad.net/bugs/707478 >
<lifeless> deryck: hey
<lifeless> deryck: wondering if you can help me figure out how to qa the front of the queue
<sinzui> I do not think this is ready to code yet. No one on my team could start this until it is clear what has to change.
<bigjools> same here - I am on my own until tomorrow when Gavin is around, he could look then if it can wait
<deryck> lifeless, hey, let me look at the queue and see.
<lifeless> so, on operational issues, I don't know that waiting for RTC makes a lot of sense : there is investigation needed for sure, but thats no different to needing to do db analysis when figuring out teamparticipation corruption, for instance.
<lifeless> flacoste: what do you think?
<thumper> morning
<flacoste> lifeless: i'd agree, but at the same time, i think it makes sense to leave that for allenap to look at tomorrow
<sinzui> jcsackett: you question-state bug might be one test and one guard, but is `./bin/test -vvc -m lp.answers` fails, talk to me
<deryck> lifeless, we usually mark checkwatches bugs qa-untestable.  With some trackers -- maybe mantis is one -- we can run checkwatches from staging.  Sometimes we get odd connection issues with this.  usually to do with ssl, IIRC.
<flacoste> unless someone is ready to start something fresh
<jcsackett> sinzui: dig. thanks.
<flacoste> in which case, they could start, but they'd likely to have to talk to allenap or gmb at this point
<lifeless> deryck: so I might mark this qa-untestable then
<flacoste> lifeless, deryck: we have a long-standing RT to set-up some bug trackers for QA purpose
<flacoste> it sat very high the queue
<SpamapS> "Bug watch updates for Red Hat Bugzilla are disabled.
<flacoste> but we never got to it
<SpamapS> :( how come?
<lifeless> deryck: does that seem ok ?
<lifeless> bigjools: would it be ok with you if we assign this fdo bug to allenap ?
<bigjools> yep, I was going to suggest it
<lifeless> SpamapS: dunno.
<bigjools> lifeless: what; the bug?  I'll do it
<bigjools> what's
<lifeless> https://bugs.launchpad.net/launchpad/+bug/707478
<_mup_> Bug #707478: Freedesktop bug watches are (incorrectly) updated with "importance unknown" <bugwatch> <Launchpad itself:Triaged> < https://launchpad.net/bugs/707478 >
<lifeless> thanks!
<deryck> lifeless, yes, that seems fine with me.
<deryck> flacoste, we never got that RT finished off because it required coordination with LOSAs and they never responded on it before we moved on to other work, so I never pushed it further.
<flacoste> deryck: we meant losa in this case
<lifeless> bigjools: https://bugs.launchpad.net/launchpad/+bug/660433 - I'm inclined to untestable that as well
<_mup_> Bug #660433: lucid support timeframe information not updated for NEW packages in lucid-updates <lp-soyuz> <qa-needstesting> <Launchpad itself:Fix Committed by allenap> < https://launchpad.net/bugs/660433 >
<lifeless> bigjools: but I don't have a deep enough understanding of this area to be sure that that is wise :)
<bigjools> lifeless: that's perfectly testable on DF
<lifeless> bigjools: ah! what is required to test it ?
<bigjools> I usually get pitti to do an upload, run the publisher, then ask him to look at the Packages file
<lifeless> ok, so in principle we could do on qas once we have an archive for it
<bigjools> please don't skip QA on publisher changes :)
<bigjools> yes
<lifeless> bigjools: this is cron.germinate
<bigjools> yeah
<lifeless> does it actually need an upload at all ?
<bigjools> yes, or the publisher has nothing to do
<lifeless> it looks like it will do work regardless
<lifeless> bigjools: would an upload to universe be sufficient ?
<bigjools> it will, but it depends on whether new input is needed
<bigjools> I am not very familiar with the changes
<lifeless> https://code.launchpad.net/~mvo/launchpad/support-timeframe-fix-660433/+merge/38503 - the english at the top is reasonably clear
<lifeless> packages that come into NEW
<lifeless> were not getting support timeframe set correctly
<bigjools> ok it's going to change existing stuff
<lifeless> AFAICT we could just run it
<lifeless> and diff the more-extras.override files
<bigjools> ok
<bigjools> I'll do it
<bigjools> it doesn't make it clear what the expected changes are though
<leonardr> gary, still the same problem: lp:~leonardr/lazr.restfulclient/test-2
<lifeless> bigjools: thanks!
<lifeless> bigjools: its changing from suite=`$LAUNCHPADROOT/scripts/ftpmaster-tools/lp-query-distro.py development`
<lifeless> to
<lifeless> SUITES=`$LAUNCHPADROOT/scripts/ftpmaster-tools/lp-query-distro.py supported`
<lifeless> so the big thing AFAICT is that a) its going to take significantly longer and b) rewrite frozen distros
<bigjools> hmmm
<lifeless> and its doing this because NEW packages in lucid are not getting support durations exported
<lifeless> I am assuming that the ubuntu 'development' series is 'supported' too
<lifeless> otherwise it seems buggy ><
<gary_poster> leonardr: zc.recipe.testrunner needs to be updated too, because it generates bin/test.  I will try it and then confirm if that helps.
<bigjools> lifeless: so re-writing overrides for older series will make no difference
<bigjools> because the next invocation of the publisher will ignore them
<bigjools> we never, ever change stuff on released series, other than -updates etc.  I'm not sure if this is what he intended.
<bigjools> lifeless: I want to wait to speak to him before saying this is ok, because it needs two publisher runs.  Germinate sets up data that the following publisher run uses.
<lifeless> bigjools: does that mean we also need to change the publisher to not ignore changes to the overrides ?
 * lifeless can see significant overheads arriving
<lifeless> bigjools: should we roll this patch out then, if its going to stall ?
<bigjools> lifeless: 1. no, no and thrice no.  2. the patch is not for the nodowntime set so it won't break anything if you mark it untestable on the provisio we look at it before the next rollout
<lifeless> 1. I meant rollback, not deploy ;)
<bigjools> and I meant you don't need to do that
<bigjools> not right now anyway
<lifeless> ok
<lifeless> so, I propose to:
<lifeless>  - mark it untestable
<lifeless>  - file a new bug saying that we have this time bomb
<bigjools> the code will never run anywhere except cocofigpineapplecumquat
<lifeless>  - assign it to you, mail mvo, and cross every available limb
<lifeless> how does that sound?
<bigjools> great
<lifeless> cool
<bigjools> I'll co-ordinate with him to do tests
<bigjools> when's our next rollout date?
<Ursinha> Feb 9, I guess
<gary_poster> leonardr: http://pastebin.ubuntu.com/558269/ then it works for me (bin/test passes)
<bigjools> Ursinha!
<Ursinha> :)
<bigjools> did you like the cat video?
<Ursinha> bigjools, I had a hard time laughing
<lifeless> huh, no 2011 calendar
<lifeless> I was sure edwin made one
<lifeless> bigjools: while you are around, http://launchpad.net/bugs/707189
<_mup_> Bug #707189: Architecture-independent direct copies can sometimes forget to copy some architectures <qa-needstesting> <Launchpad itself:Fix Committed by wgrant> < https://launchpad.net/bugs/707189 >
<bigjools> lifeless: actually looking at the merge again, I think his intent is only to set the timeframe data for new stuff in old releases
<lifeless> bigjools: If I understand correctly I could qa that via the web UI ?
<lifeless> bigjools: yes, thats his intent
<bigjools> lifeless: yes, copy an arch-indep package w/rebuild, and make sure it creates builds for all three PPA archictectures (which means copying a hardy package)
<thumper> flacoste: hey
<bigjools> lifeless: so the patch is fine - but I'll test it with him anyway since I don't want to fuck anything up with the publisher
<thumper> flacoste: I'm digging into more lazrjs widgets today
<lifeless> bigjools: so NEW stuff will honour overrides?
<bigjools> lifeless: yes
<thumper> flacoste: the multiline text editor needs some work, which I'm going to do
<lifeless> great, my knowledge expands :)
<bigjools> lifeless: because it lives in -updates
<thumper> flacoste: I want to get it to the same state as the picker widget, so you just render the widget and it all works
<thumper> flacoste: which isn't the case right now
<bigjools> lifeless: only the release pocket is frozen
<flacoste> thumper: that sounds very good!
<lifeless> bigjools: bug 707630 for your qa pleasure
<_mup_> Bug #707630: qa needed for cron.germinate changes in rev 12250 <Launchpad itself:Triaged by julian-edwards> < https://launchpad.net/bugs/707630 >
<bigjools> ok
<lifeless> bigjools: so from hardy to hardy? or from hardy to lucid would work (https://launchpad.net/bugs/707189)
<_mup_> Bug #707189: Architecture-independent direct copies can sometimes forget to copy some architectures <qa-needstesting> <Launchpad itself:Fix Committed by wgrant> < https://launchpad.net/bugs/707189 >
<bigjools> lifeless: you need a series that has lpia in it - hardy does
<bigjools> (the target series)
<bigjools> lifeless: well, anything up to karmic is ok
<lifeless> so arch indep only generates i386 though?
<cody-somerville> Does launchpad use mod_xsendfile by any chance?
<lifeless> no
<cody-somerville> for files served from the librarian, does the web server read and stream the file or the launchpad librarian web app do that?
<lifeless> the librarian daemon serves on http
<lifeless> the squid front end caches
<lifeless> and the apache front end does ssl
<lifeless> which web server of those three are you asking about ?
<lifeless> oh, and the haproxy does load balancing
<cody-somerville> lifeless, Were the issues with serving files from the librarian caused by timeouts in the librarian daemon?
<lifeless> cody-somerville: what issues?
<cody-somerville> lifeless, the timeouts
<lifeless> cody-somerville: I don't know what you're talking about
<lifeless> bug #?
<cody-somerville> lifeless, I'll look it up, one second.
<lifeless> bigjools: I think I'll wait for wgrant, my understanding here isn't sufficient for me to be confident
<bigjools> lifeless: it's trivial to test
<lifeless> bigjools: I'd expect a source copy of an arch-all package to only make an i386 build
<lifeless> bigjools: I copied langpack-locales from my hardy ppa to https://qastaging.launchpad.net/~bzr-beta-ppa/+archive/obsolete/+packages
<bigjools> lifeless: yes, but it needs publishing entries in all arches
<lifeless> bigjools: that requires a buildd to build it first, right ? :)
<bigjools> sorry I was confusing you earlier by saying builds, I should have said "builds published"
<bigjools> it does
<lifeless> confusion corrected.
<bigjools> I can update dogfood
<lifeless> and we don't have a buildd for qastaging yet
<bigjools> then we can do it there
<lifeless> bigjools: that would be great, I don't think the patch has propogated to staging yet
<lifeless> abentley: your patch has had a good test run with buildbot, FYI
<bigjools> lifeless: so I can easily QA his change but it would be nice to spread some knowledge, I can take you through it if you want
<cody-somerville> lifeless, I can't find the particular bug at the moment but I know you helped fix what ever was wrong :P
<cody-somerville> lifeless, anyhow, was just curious if you guys used it
<abentley> lifeless, I saw that.  Also saw that the bug is looking not buildbot-specific.
<lifeless> abentley: yeah
<lifeless> abentley: which is both good and bad
<lifeless> bigjools: I'd like that
<bigjools> lifeless: ok just bringing up DF with newest code
<lifeless> cody-somerville: are you thinking of the apport incident months ago?
<bigjools> lifeless: do you know off hand if the revision made it to db-devel et?
<bigjools> yet
<cody-somerville> lifeless, no, thinking of the timeouts when accessing files from the private librarian
<lifeless> bigjools: I haven't looked
<lifeless> cody-somerville: that would be apport then
<lifeless> cody-somerville: so the 'timeouts' are also known as 'firewall rules' :)
<lifeless> cody-somerville: or if you go far enough back, we were serving private librarian content via the front end app servers
<lifeless> cody-somerville: so what would happen is this:
<lifeless>  - request for a 100MB (or whatever) file hits zope
<lifeless>  - zope copies it from the librarian
<lifeless>  - its not in the OS page cache so disk IO happens
<lifeless>  - request times out
<lifeless> once its in the OS page cache the copy to the zope server was fast enough that things would work, but that could take multiple requests
<lifeless> what we do know is have the appserver issue a redirect when someone is authorised to access a private file
<lifeless> s/know/now/
<lifeless> cody-somerville: anyhow, no, we don't use sendfile
<lifeless> no particular urge to do so either
<cody-somerville> I just figure apache2 is probably pretty good at reading and streaming files. ;)
<lifeless> cody-somerville: so is twisted
<lifeless> cody-somerville: and twisted can do our token authentication checks directly rather than having a somewhat awkward external helper to write
<lifeless> cody-somerville: as well as the mapping from url to disk (which is db dependant)
<cody-somerville> lifeless, you can do that with xsendfile too
<cody-somerville> lifeless, your python app can do authentication and what ever and then send the x-send-file header to have apache take over and handle the heavy lifting let your app get back to other things
<lifeless> cody-somerville: what we have now works great; and given how average apache2 actually is at serving from disk, I'm really not inclined to fiddle.
<cody-somerville> lifeless, no disagreement.
<leonardr> abentley, do you want me to review https://code.edge.launchpad.net/~abentley/launchpad/translation-fixing4/+merge/47461 or did you deliberately give it to deryck and henninge?
<abentley> leonardr, I want henninge to review it.
<leonardr> ok, just checking
 * leonardr has had a lonely tenure as ocr today
<abentley> leonardr, I know the feeling.  But I want hennninge to review it, because I'm rewriting translations tests and I want him to make sure I'm not changing their meaning.
<leonardr> got it
<thumper> leonardr: how do I get the exported_as attribute out of a exported field from the Interface
<thumper> leonardr: for example if I have ISomeInterface['fieldname']
<leonardr> thumper: you'll probably need to root around in the annotations. why do you need this?
<thumper> leonardr: I'm wanting to pass the interface field through to the lazrjs text editing widgets like we do for the popup
<thumper> leonardr: so we don't have to explicitly specify the exported name, and accept empty values
<thumper> leonardr: ideally I want to have it automatically generate the text to show too
<thumper> leonardr: I'm refactoring the lazrjs text widgets
<thumper> to make them more magic and awesomer
<leonardr> thumper: one complication comes from the fact that a single field may have multiple exported_as attributes in different versions. of course, you'll want the 'devel' name...
<leonardr> thumper: this should get you in the vicinity
<thumper> leonardr: ok, thanks
<leonardr> no, i wasn't done
<leonardr> that wasj ust me thinking, here's the actual helpful part:
<thumper> :)
<lifeless> anyone ever seen '__storm_table__ missing' in a SQLBase child class ?
<lifeless> has me a little flummoxed
<lifeless> ah
<lifeless> figured it out.
<leonardr> thumper: sorry, my head is abuzz with ideas. let's nail down what we want
<leonardr> you have ISomeInterface['fieldname'], which is exported_as('fieldname2')
<lifeless> I was returning BugMessage.bug, had to return BugMessage.bugID
<thumper> leonardr: yes,
<thumper> leonardr: IProduct['programminglang'] in fact
<leonardr> you are writing some python that takes as an argument the Field object ISomeInterface['fieldname']
<leonardr> is that right?
<thumper> yes
<leonardr> ok. one way of doing this would be to find the IEntry adapter corresponding to IProduct
<leonardr> this would be called something like IProductEntry_devel
<leonardr> but i think there's a simpler way to do it
<leonardr> give this a try and see what you get
<leonardr> call .queryTaggedValue(LAZR_WEBSERVICE_EXPORTED) on IProduct['programminglang']
<leonardr> L_W_E is a constant you can import from lazr.restful.declarations
<leonardr> thumper -^
 * thumper tries
<leonardr> you can get IProductEntry_devel by calling EntryAdapterUtility.forSchemaInterface(IProduct, request)
<leonardr> but then you have the opposite problem--how to see which field was originally called 'programminglang'
<leonardr> however, i do see code that does that:
<leonardr>                     tagged_values = field.getTaggedValue(
<leonardr>                         'lazr.restful.exported')
<leonardr>                     original_name = tagged_values['original_name']
<thumper> <lazr.restful.utils.VersionedDict object at 0x3e5f810>
<leonardr> ok, we can do this
<leonardr> what's its items? it should have ['as'], and that should be the value you're looking for
<thumper> I get 'programminglang' from the versioned dict
<thumper> 'as' is a simple string value
<thumper> which is 'programming_lang'
<thumper> which seems right
<thumper> but what about the problem with devel vs other versions?
<leonardr> thumper: it's fine. we are always using the latest version, and the default state of a VersionedDict is for the latest version's values to be on top
<thumper> on top?
<leonardr> a VersionedDict is a stack of dicts
<leonardr> with bleedthrough
<thumper> ok
<thumper> how can I see the stack?
<thumper> just for my interest
 * leonardr checks
<leonardr> thumper: it's available as .stack
<thumper> cook
<thumper> cool
<leonardr> thumper: we have a EntryAdapterUtility to hide this kind of detail (getTaggedValue) when it comes to the generated IEntry interfaces
<leonardr> we should consider adding a similar utility to manage the tags on the source interfaces
<thumper> hmm...
<thumper> yep
<thumper> I'll go with this for a first cut refactoring
<leonardr> sure
<leonardr> matsubara, do you need a review? i'm eod but i can do one if it's quick
<matsubara> leonardr, yes, I'm writing the mp. just a sec.
<lifeless> wgrant: hi?
<matsubara> leonardr, https://code.launchpad.net/~matsubara/oops-tools/fix-graphs/+merge/47470
<matsubara> so basically the script I added is a python script that replaces the bash script in devpad. It uses the new report objects to generate the reports rather than calling the analyse script. (which I'm going to deprecate once I get rid of all of the scripts/cronjobs using it)
<leonardr> matsubara: what kind of dsl is that in the setup.py?
<matsubara> leonardr, dsl?
<leonardr> 'graph_report = oopstools.scripts.graph_report:main',
<leonardr> that's not python
<leonardr> esp. because it's in a string
<leonardr> what is it? i'm just curious--it looks good judging by the context
<matsubara> right, that's how setup.py sets up new scripts in the bin/ directory
<leonardr> is it normal to have two underscores in name__in?
<matsubara> graph_report is how the script will be called and oopstools.scripts.graph_report:main is the entry point
<matsubara> yep, that's django's orm syntax
<matsubara> leonardr, setup.py will turn that into this script https://pastebin.canonical.com/42317/ in bin/
<matsubara> btw, what do you mean by dsl?
<leonardr> domain-specific language
<leonardr> like the makefile language
<matsubara> ah right. makes sense now :-)
<leonardr> r=me
* leonardr changed the topic of #launchpad-dev to: Performance Tuesday! | Launchpad development https:/â/âdev.launchpad.net/â | PQM is open | firefighting: - | On call reviewer: - | reviewing: - | queue: [] | https://code.launchpad.net/launchpad-project/+activereviews
<wgrant> lifeless: Hi.
<wgrant> I'm not here today.
<wgrant> But what's up?
<lifeless> ah
<lifeless> so, I thought you might have triggered the kde build on rubudinium
<wgrant> Which?
<lifeless> its gone now
<lifeless> shrug
<lifeless> enjoy your awol :)
<bigjools> I am offski, see y'all tomorrow
<wgrant> Night bigjools.
#launchpad-dev 2011-01-26
<thumper> ?!?!?!
<thumper> line 765 of client.js
<thumper> WTF
 * thumper heads for a real coffee
<lifeless> === Top 10 Time Out Counts by Page ID ===
<lifeless>     Hard / Soft  Page ID
<lifeless>      129 /  293  BugTask:+index
<lifeless>       61 /  196  Distribution:+bugs
<lifeless>       56 / 4821  Archive:+index
<lifeless>       28 /   44  Branch:+index
<lifeless>       12 /    3  Person:+related-software
<lifeless>       11 /    2  Product:+filebug-show-similar
<lifeless>        9 /    2  Distribution:EntryResource
<lifeless>        8 /   29  MailingListApplication:MailingListAPIView
<lifeless>        8 /    3  Cve:+index
<lifeless>        6 /  156  POFile:+translate
 * thumper has a bad feeling about this
<thumper> lifeless: when was the last rollout?
<wgrant> thumper: 24th
<wgrant> 12248
<wgrant> 12263 should happen once mthaddon wakes up.
<thumper> no
 * thumper slams the breaks on it
<wgrant> Why?
 * thumper raises the red flag
<thumper> wgrant: try editing a bug description in qastaging
<wgrant> Timeout?
<thumper> no
<thumper> oops
<thumper> ValueError: Unicode results must have a text content type.
<thumper> in the zope publisher
<thumper> I'm trying to figure out why
<thumper> I thought it was just local for me
<thumper> due to me messing around
<wgrant> I get a timeout :/
<thumper> but after I removed all my changes
<thumper> it still oopses
<thumper> so I checked qastaging
<wgrant> Sigh.
<wgrant> lp-oops.c.c can only see up to QS1.
<thumper> I've just emailed lp-dev
<wgrant> Have you tried downgrading lazr.restful?
<thumper> not yet
<wgrant> That's a really strange error.
<thumper> yes, it is
<wgrant> I wonder if Windmill would have caught this.
<thumper> probably
<thumper> actually, I think it is something I did
<thumper> last week
 * thumper thinks
<wgrant> The Windmill tests were passing OK before lazr.restful 0.15.3, I believe.
<wgrant> Hmm.
<wgrant> Not lazr.restful.
<thumper> hmm....
<wgrant> Still broken with 0.15.0.
<thumper> wgrant: same error?
<wgrant> Yes.
<thumper> (Pdb) content_type
<thumper> 'application/xhtml+xml'
<thumper> the publisher expects it to start with 'text/'
<wgrant> Hah.
<thumper> I changed something...
<thumper> but
<thumper> it shouldn't have changed
<thumper> this
<lifeless> thumper: when you get a gap, filing a bug to change the exception to show the wrong type would be helpful.
<wgrant> What did you change? The webservice named adapters thing?
<thumper> lifeless: it is in the zope publisher code
<thumper> wgrant: yeah
<lifeless> thumper: yes
<lifeless> thumper: thus filing a bug rather than changing it in the first instance
<thumper> but this is asking for the representation of the description
<thumper> which "should" have the same implementation
<thumper> I wonder if the resulting content type has changed....
<thumper> we should be able to test this...
<lifeless> on qastaging
<lifeless> SELECT heat\n FROM Bug, Bugtask, Product\n WHERE Bugtask.bug = Bug.id\n AND Bugtask.product = Product.id\n AND Product.project IS NOT NULL\n AND Product.project = 82\n ORDER BY Bug.heat DESC LIMIT 1'<br /> Parameters:()<br /> Original error: QueryCanceledError('canceling statement due to statement timeout\n'
<lifeless> thumper: bug editing works fine on qastaging in the web ui
<lifeless> thumper: could it be your client asking for something odd?
<thumper> lifeless: it didn't for me
<lifeless> https://bugs.qastaging.launchpad.net/launchpad-foundations/+bug/1234 - edit it using the ui
<_mup_> Bug #1234: Gina is an unmaintainable mess of command line options, environment variables and shell scripts <lp-foundations> <Launchpad itself:Fix Released by debonzi> < https://launchpad.net/bugs/1234 >
<thumper> OOPS-1852QS15
<wgrant> I used that very bug.
<wgrant> It broke.
<lifeless> wgrant: timeout getting heat for the project gourp
<wgrant> Oh.
<wgrant> So it was.
<lifeless> I've now edited it three times without trouble
<thumper> OOPS-1852QS16
<wgrant> devpad has the OOPS now.
<wgrant> devpad doesn't have QS16 :(
<thumper> that was my oops for trying to edit it
<lifeless> thumper: using the web UI ?
<thumper> yes
<lifeless> thumper: just now, or before ?
<thumper> just now
<lifeless> thumper: this is odd, if you refresh you'll see i've edited it
<thumper> lifeless: which browser
<lifeless> chrome
<wgrant> Chromium 8.blahblah here.
<lifeless> chromium 8.0.552.224 (685999)
<lifeless> I'll try ff
<thumper> lifeless: not the title
<thumper> lifeless: the description
<lifeless> bah
<lifeless> ok
<lifeless> OOPSes for me
<lifeless> you're not crazy
<wgrant> thumper: Which adapters did you change?
<wgrant> I know you changed person.
<wgrant> But did you also change TextField or whatever it is?
<lifeless> wow, 300ms to get hottest heat for a project gorup
<thumper> wgrant: yes, I changed Text
<lifeless> -slow-
<wgrant> Ah, yes.
<thumper> wgrant: but it does the same thing
<wgrant> So it does.
<thumper> wgrant: it appears to  be the content type
<thumper> that is the problem
<thumper> not the rendering of the description
<thumper> I wonder if there was a lazr restful change that played with the content type
<thumper> it isn't something I remembered from last week
<wgrant> Except that it still dies with lazr.restful 0.15.0.
<thumper> really?
<thumper> WTF?
<wgrant> It was the first thing I tried. Let me try again.
<lifeless> bisect time ?
<wgrant> Done already.
<wgrant> It's 12251, as expected.
<wgrant> Going further into the diff now.
<wgrant> But first trying lazr.restful again...
<wgrant> Also, our appserver startup time fucking sucks.
<lifeless> tagging it bad
<wgrant> Death to ZCML, etc.
<lifeless> it does
<lifeless> ok, https://devpad.canonical.com/~lpqateam/qa_reports/deployment-stable.html should notice in a few minutes
<wgrant> thumper: Confirmed bad with lazr.restful 0.15.0.
<wgrant> Something in the r12251 diff is somehow breaking it.
<lifeless> I'll send a rollback for this now
<wgrant> But I cannot see how.
<wgrant> Be careful.
<wgrant> There was a later lazr.restful update.
<lifeless> wgrant: ><
<wgrant> You cannot roll it back safely without going through ec2.
<lifeless> wgrant: right
<lifeless> ok, actually, its past EOD
<lifeless> so I won't do it
<lifeless> as theres no chance I'll  be paying attention later
<thumper> yeah... that isn't easy to revert
<thumper> as there are later revisions tweaking it
<wgrant> thumper: Have you tried bypassing the JS?
 * thumper pokes
<thumper> wgrant: yes
<wgrant> What happens?
<thumper> well... for the entry, it is working fine
<thumper> hmm....
<thumper> maybe not
 * wgrant compares the requests.
<thumper> I remember testing this though :(
<thumper> it was one of the things we tested to make sure we didn't fuck it up
<wgrant> Well, the requests are identical, so it's not JS, so it is debuggable.
 * thumper stops...
<thumper> and thinks
<thumper> the publisher has the correct string
<thumper> I can see that in the debugger
<thumper> but it is trying to return a different content type
<wgrant> But I cannot see how.
<wgrant> thumper: In both cases the Accept says application/xhtml+xml.
<wgrant> Huh.
<wgrant> In the good case it also returns application/xhtml+xml.
<wgrant> So the content type is right.
<thumper> which good case?
<wgrant> 12250, before the problematic revision.
<wgrant> It asks for application/xhtml+xml, and gets it back fine.
<wgrant> The publisher does not choke.
<wgrant> Er.
 * wgrant headdesks.
<wgrant> I just looked at the adapter diff.
<wgrant> Now it is pretty obvious.
<wgrant> You removed the .encode('utf-8')
 * wgrant tries.
<thumper> wgrant: we checked that
<wgrant> It works fine with that readded.
<thumper> :((
<thumper> the lazr.restful code was encoding it
<wgrant> Huh.
<thumper> we thought we were double encoding
<thumper> lazr.restful gets all the results and encodes that
<thumper> argh...
 * thumper thinks...
<thumper> the multiline editor asks for a xhtml+xml representation of a single field of the resource
<thumper> which would go through a different code path
<wgrant> Right.
<thumper> to the one what asks for the entire entry
<wgrant> Ah, indeed.
<thumper> which is what the chooser needs
<wgrant> So the entry gets encoded by lazr.restful.
<wgrant> But the field does not.
<thumper> I think so
<thumper> but the field should
<wgrant> It should.
<wgrant> Let's see what lazr.restful thinks...
 * thumper goes to dig in lazr.restful
<wgrant> Hmm.
<wgrant> Interesting.
<wgrant> It looks like we just need to fix EntryFieldResource._representation to encode if it is a unicode?
<wgrant> I guess that whatever tests it uses an adapter that returns a str.
<wgrant> Although EntryFieldResource has no direct tests.
<thumper> I guess
 * thumper tries
 * wgrant is already waiting for the appserver to start...
<wgrant> Seems to work OK.
<thumper> wgrant: yeah...
<wgrant> Both entry and field representations looks fine now.
<thumper> wgrant: if you see the EntryResource._representation it encodes utf-8
<wgrant> It does.
<thumper> although
<wgrant> Hmm.
<wgrant> Interesting.
<wgrant> There is a difference in the HTML entry repr.
<thumper> I'm wondering if the EntryFieldResource._representation is called from EntryResource...
<wgrant> It doesn't seem like it.
<wgrant> I'll throw some Unicode in and see.
<wgrant> duplicate_of_link in dev is the empty string, but on lpnet it's 'None'.
<wgrant> (this is in the HTML entry representation, as seen in the browser)
<wgrant> I suppose that's the new reference formatter.
<wgrant> It's not being double-encoded.
<lifeless> ok, I'm really EOD, have to cook dinner etc etc etc
<wgrant> So it looks like EntryResource doesn't uses EntryFieldResource.
<lifeless> if you need me, SMS and I'll come a running
<wgrant> https://bugs.launchpad.dev/api/devel/bugs/1 has a nice snowman, and so does the webapp after changing it.
<_mup_> Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Invalid by compscibuntu-bugs> <EasyPeasy Overview:Invalid by ramvi> <GNOME Screensaver:Won't Fix> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <The Linux OS Project:In Progress> <metacity:In Progress> <OpenOffice:In Progress by lh-maviya> <Tabuntu:Invalid by tinarussell> <Tivion:Invalid by shakaran> <Tv-Player:New> <Ubuntu:I
<wgrant> thumper: I suppose we should wait for a Leonard, though :(
 * thumper checks something
<thumper> :(
<thumper> something seems weird
<wgrant> ?
<thumper> hmm... maybe not
<thumper> https://bugs.launchpad.net/api/devel/launchpad/+bug/1234?ws.accept=application/xhtml%2Bxml
<_mup_> Bug #1234: Gina is an unmaintainable mess of command line options, environment variables and shell scripts <lp-foundations> <Launchpad itself:Fix Released by debonzi> < https://launchpad.net/bugs/1234 >
<thumper> I would have thought that would have the description in it
<wgrant> thumper: That's the BugTask.
<wgrant> s@launchpad/+bug@bugs@
<thumper> oh yeah
<thumper> ok
<thumper> that seems to work fine
 * thumper puts up a lazr.restful branch
<thumper> just running the tests
<thumper> omg
<thumper> it is building otu
<wgrant> Heh.
<thumper> wgrant: I don't suppose the tests pass for you on lazr.restful?
<thumper> wgrant: I get 4 errors and 1 failure
<thumper> wgrant: I did get errors in the setup of  RestrictedPython-3.6.0
<wgrant> thumper: Let me run buildout and see.
<wgrant> ugh, I don't have my usual buildout global download cache config here.
<wgrant> This could take a while.
<thumper> I'm getting the same errors with trunk
<wgrant> Sounds fine, then.
<thumper> we could patch the lazr.restful egg
<thumper> or create an egg with the fix
<thumper> land that
<thumper> and wait for confirmation
<thumper> for inclusion in trunk
<thumper> wgrant: what do you think?
<wgrant> I don't know. I'm wary about patching it, although it seems safe.
<wgrant> It is tempting to just wait for leonard to appear and release 0.15.4 with the fix.
<wgrant> That's not too far off.
 * thumper frowns
<thumper> the unmerged revisions on the code review looks crackful
<wgrant> I don't know lazr.restful well enough to make a sensible judgement here.
<wgrant> it is, yes.
<thumper> the diff is right
<thumper> I think someone "fixed" it
<wgrant> I suspect jtv's BranchRevision changes broke it.
<thumper> :(
<wgrant> But I didn't end up having time to investigate.
<wgrant> And I'm not here today.
<thumper> wgrant: heh
<wgrant> But he changed the relevant method, so I think that's probably it.
<thumper> I'll flick this leonards way and get him to release the fix if it is
<wgrant> Thanks.
<wgrant> Hopefully it will be deployable some time tomorrow.
<wgrant> As for the unmerged revisions, will you be able to investigate soonish?
<wgrant> Bug #707808
<_mup_> Bug #707808: Unmerged revisions list does not exclude merged revisions <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/707808 >
<thumper> wgrant: not right now, I'm focused on recipes
<thumper> and EOD for me now
<wgrant> thumper: 'soonish' could include tomorrow.
<wgrant> ie. should I look at it when I wake up tomorrow?
<thumper> wgrant: yes, you
<wgrant> k
<thumper> are on a bug team :)
<wgrant> Fair point.
<thumper> thanks
<StevenK> wgrant: That would be awesome
 * StevenK bursts into flame too cool down
<wgrant> StevenK: What would be/
<wgrant> StevenK: Huh, it's only like 18Â°C down here.. what is it up there?
<StevenK> 32
<wgrant> Ow.
<wgrant> It's just about perfect here.
<wgrant> Although overcast.
<mrevell> Hi
<adeuring> good morning
<danilos> sinzui, hi, it would be nice to get https://bugs.launchpad.net/bugs/707741 fixed ASAP
<_mup_> Bug #707741: Translations export script failing. <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/707741 >
<wgrant> The other blocker (12251) is first on my list for tomorrow.
<wgrant> danilos: I'll also handle 707741 if nobody else has done it.
<danilos> wgrant, right, but this is pretty critical in the sense that if I was translations team, I'd go right away and do it
<wgrant> It's been failing for 48 hours already.
<danilos> wgrant, which makes it even more so
<danilos> wgrant, the fact that we are doing a very bad job watching our services is not something to brag about
<wgrant> Indeed.
<StevenK> thumper: When you start work, https://bugs.launchpad.net/launchpad/+bug/707919 sounds like something we fixed at the Thunderdome, if you want to comment.
<_mup_> Bug #707919: Reassign to another project animation spins forever <Launchpad itself:New> < https://launchpad.net/bugs/707919 >
<dpm> Bug 707741 is a big blocker for translation teams, who have started complaining. I'm not pointing the obvious, just trying to point out that it's a critical regression for the translations community
<_mup_> Bug #707741: Translations export script failing. <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/707741 >
<Ursinha> morning
* danilos changed the topic of #launchpad-dev to: Launchpad development https:/â/âdev.launchpad.net/â | PQM is open | firefighting: - | On call reviewer: - | reviewing: - | queue: [danilo] | https://code.launchpad.net/launchpad-project/+activereviews
<leonardr> thumper or lifeless, i don't suppose you're still around
<leonardr> i'm looking at the qastaging breakage
<deryck> abentley, adeuring, henninge -- ping for call in 2.
<adeuring> yes
<abentley> deryck, pong
<deryck> adeuring, https://launchpad.net/launchpad/+bugs?field.tag=upstream-translations-sharing
<henninge> wow, I just closed two bugs in 10 secondsd
<deryck> henninge, I think abentley already opened a bug about what you were talking about on the standup.  see bug 706005
<_mup_> Bug #706005: solve the case where sharing is enabled with existing upstreams <upstream-translations-sharing> <Launchpad itself:Triaged> < https://launchpad.net/bugs/706005 >
<henninge> deryck: yup, I added a comment to make that clear. Thanks.
<deryck> np.  thank you
<deryck> adeuring, abentley, henninge -- board is up to date with remaining tasks
<adeuring> ok
<abentley> deryck, cool.
<abentley> deryck, also, yikes!
<deryck> it's not that bad :-)
<deryck> flacoste, you might be interested in that too ^^ (Orange board is updated with cards for remaining work on our story)
* danilos changed the topic of #launchpad-dev to: Launchpad development https:/â/âdev.launchpad.net/â | PQM is open | firefighting: - | On call reviewer: - | reviewing: - | queue: [] | https://code.launchpad.net/launchpad-project/+activereviews
<deryck> flacoste, there is a "misc" card added to track that we still need 2-3 cards for some remaining UI bits.
<deryck> but henninge and I will nail that down in our call today
<jcsackett> can someone review https://code.launchpad.net/~jcsackett/launchpad/branch-delete-api-702620/+merge/47449? it's some quick webservice stuff and tests.
<deryck> henninge, ready for call when you are
<henninge> deryck: give me a minute or two to finish this up, please
<deryck> henninge, np.  take your time.
<henninge> deryck: no, almost done
<deryck> ok
<leonardr> jcsackett, i'll review you if you'll review me
<leonardr> https://code.launchpad.net/~leonardr/lazr.restful/encode-xhtml-field/+merge/47536
<jcsackett> leonardr: deal.
<leonardr> jcsackett: for context, see thumper's recent message to launchpad-dev
<benji> salgado: do you have a minute for a UI review?
<jcsackett> leonardr: dig.
<salgado> benji, sure, though can it be after my lunch? I was about to break
<benji> salgado: sure; let me know when you're back and I'll give you the info
<salgado> will do
<jcsackett> leonardr: what is this SNOWMAN stuff?
<leonardr> jcsackett: it's an obvious unicode character
<leonardr> start up python and >>> print u"\N{SNOWMAN}"
<jcsackett> leonardr: huh. did not know that.
<leonardr> i learned the syntax from thumper--it's v. nice because you don't have to write down the escape
<jcsackett> yeah, i dig it.
<leonardr> jcsackett, r=me
<jcsackett> leonardr: the tree <<<< stuff in the MP is just because of your soon to be born merge oddness, right? i can safely just look at the pastebin diff?
<leonardr> jcsackett: yeah
<leonardr> do you know the best way to do this merge, btw?
<jcsackett> leonardr: i have no idea, honestly. sounds like it could be painful.
<jcsackett> leonardr: maybe lay out the situation in #bzr?
<leonardr> jcsackett: reverting isn't too bad, since i've been committing a feature that's not done yet
<leonardr> ok
<jcsackett> leonardr: r=me. i'm requesting follow up from sinzui (who is usually fast), but if you need this ASAP grab anyone else. sorry i'm still only in mentored mode. :-/
<jcsackett> i can also help huntdown a follow up if you need someone quick.
<leonardr> jcsackett: quick would be good, this is blocking
<sinzui> leonardr: I am seeing unicode in doctests. unicode often causes zope doctests to fail. We may have removed the ascii exception last week though
<leonardr> sinzui: how do you suggest i do this?
<sinzui> leonardr: why am I seeing unicode though. What is that? A BOM?
<leonardr> sinzui: it's a snowman
<leonardr> i am junking up the data with a unicode character to make sure it's properly handled
<sinzui> lots of translations and answers tests encode the output to get \x1234
<leonardr> i'll see if i can do that
<sinzui> leonardr: Does this test play in the zope testrunnner?
<leonardr> sinzui: i think so
<leonardr> it sets up layers
<danilos> sinzui, hi, could you please arrange for bug 707741 to be worked on asap: basically translation exports are broken, and that's something our users depend on
<_mup_> Bug #707741: Translations export script failing. <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/707741 >
<sinzui> oh, that might explain why that user did not get his export email
<sinzui> leonardr: answers uses .encode('ascii', 'backslashreplace') to avoid outputting fun characters
<bac> deryck[lunch]: is bin/jssize still needed?
<sinzui> leonardr: you have my approval with the unicode fix. I included the section about unicode vs. zope test runner in my review.
 * jcsackett is no longer happy with his dev environment.
<leonardr> sinzui: having trouble getting the unicode fix to work, will ping you in a bit
<leonardr> sinzui: basically, the result of webservice.get() is a string, and i can't get it into unicode so that i can use .encode('ascii', 'backslashreplace')
<leonardr> whew, got it
<sinzui> leonardr: all okay now?
<leonardr> sinzui: i think so
<salgado> benji, still want a UI review for your branch?
<benji> salgado: absolutely: here's the MP for background on what the UI is for: https://code.launchpad.net/~benji/launchpad/bug-636193/+merge/46350 the two relevant screen shots are the page that links to the new page: http://i.imgur.com/R6jma.png and the new page itself: http://i.imgur.com/ExwBs.png
<sinzui> jcsackett: where did you find the recent oops reports. I need to see staging for the last 6 hours
<jcsackett> sinzui: i don't know how recent they were, but i found the ones i needed in devpad:/srv/launchpad.net-logs/qastaging/asuka/<DATE>/
<jcsackett> so i assume /staging/asuka/<DATE>/ should be what you want.
<lifeless> *yawn*, moin
<deryck> bac, I think so.  Will gave to confirm, though.
<deryck> bac, about to be on TL call, but will look after that.
<salgado> benji, who's the target audience for that page? LOSAs?
<benji> salgado: yes
<benji> salgado: and developers
<salgado> benji, I'm wondering if it'd make sense to show it to one of the LOSAs ans ask them if it has enough information for them to write a new rule?
<benji> salgado: that makes sense; do you have a candidate in mind?
<salgado> let's see if we can get someone to volunteer
<salgado> LOSA ping?
<Chex> salgado: hello there
<lifeless> perhaps it should link to the LEP as wlel ?
<lifeless> which is the docs for this
<salgado> hi Chex!
<salgado> Chex, I'm reviewing benji's branch which better documents feature flags and we'd like to know if you'd be willing to have a look and tell us if the new page works
<salgado> (i.e. does it have all the info you'd need to write a new rule)
<salgado> lifeless, is the LEP going to be kept up to date to match what's implemented?
<lifeless> salgado: the goal is to move the list of actual flags/scopes out of it. but the conceptual and operational docs will stay there IMO
<salgado> oh, ok, that's good
<salgado> benji, can you do that?
<benji> salgado: yep, I can remove the static docs for flags and scopes from the LEP
<salgado> benji, oh, I meant linking to the LEP, but I'm sure lifeless will appreciate you removing the flags and scopes from there
<benji> salgado: oh; I /could/ do that, but LEPs are unmaintained after implementation, and that one in particular already says things that aren't true, so linking to it seems like a bad idea
<salgado> lifeless, ^
<lifeless> benji: what do you propose for docs for this then?
<lifeless> benji: (its not globally true to say LEPs are unmaintained after impl)
<benji> How do we document things like this now?  Perhaps more importantly, for what audience are we documenting?  I /think/ LP developers will be writing new rules and the lib/lp/services/features/__init__.py docstring contains a nice (pre-existing) description of how feature flags works.
<lifeless> benji: developers will be asking for things as they do features; losas will be changing stuff to deal with operational issues; for isntance the timeout is controlled by a feature flag
<lifeless> benji: I think the LEP is fine as docs; sure it needs a cleanup, we can do that.
<lifeless> benji: the docstring isn't operational-docs.
<lifeless> perhaps it should be, but its not, and with a 8 hour overhead to improve it I don't think anyone will when dealing with a operational issue
 * lifeless practices skepticsm
<benji> lifeless: so link to the LEP from the info page?
<lifeless> benji: thats what I was suggesting, yes. If you think its a bad idea don't do it :)
<benji> I have to wonder if the LOSAs have a cache of operational documents somewhere.
<lifeless> they do
<lifeless> but they get stale as well
<benji> Linking to a document that's already in a known-bad state feels wrong.  Not that I can think of anything better, but I'm +0 on silence being the best approach here (i.e., no link at all).
<lifeless> benji: I'll get it cleaned up
<lifeless> benji: what bits are wrong? just the speculation in the early part?
<benji> lifeless: Unfortunately, I don't recall off the top of my head.  I'd have to re-read it to find the inacuacies, but I recall that there were some rather significant difference between the LEP and the current implementation and functionality.
<lifeless> benji: ok, I think I know what you mean
<lifeless> poolie described a phase space.
<lifeless> I previously updated the lep to ensure it does describe what we have, but left the phase space in place.
<lifeless> I'll happily remove it
<lifeless> sinzui: hi
<lifeless> sinzui: do you need help w/ mailman ?
<lifeless> on qastaging
<sinzui> yes. I do not know why QAS mailman cannot talk to QAS launchpad. we get a 403
<benji> Chex: are you familiar with the feature flags functionality? (https://launchpad.net/+feature-rules)  I'm adding a little documentation to help when writing rules so people will know which flags and scopes are available, screen shot at http://i.imgur.com/ExwBs.png
<benji> Chex: we'd like to know if that presentation of the info makes sense to LOSA eyes
<benji> lifeless: ok, so I should add a link to the LEP from the info page?  (and I suppose also add a special note on the LEP that it is intended to be kept up to date with the implementation, and also a note in the implementation that when it is changed the LEP should be updated)
<lifeless> benji: I don't think we need a special note; LEPs that no longer reflect reality are harmful
<lifeless> benji: I thought our standard practice was to keep things up to date
<lifeless> e.g. gary has been updating things
<lifeless> and so have other people
<benji> 11:11 < benji:#launchpad-dev> are LEPs meant to be kept up to date after they're implmented or are they simply historical?
<benji> 11:18 < flacoste:#launchpad-dev> benji: historical
<lifeless> flacoste: ^ lets talk about this ;)
<benji> lifeless: ^^ there seems to be some confusion on that front
<bac> hi deryck, could i talk to you sometime today with some JS questions?
<deryck> bac, sure.  I forgot about your jssize question.  sorry
<bac> np
<bac> deryck: do you have time now?
<deryck> sure.  let me grab headphones
<lifeless> benji: ok, lets talk constraints
<lifeless> benji: all I care about is that the page list the known scopes etc; if we want the page to be 'sufficient to write a rule' then I care that the exposition about the rules, and the whys, and the approval policy be 'present or linked'.
<lifeless> I dunno if that parses
<lifeless> benji: and with that let me get out of your hair and let you do it as you think best ;)
<benji> lifeless: I parsed it -- correctly, I think. :)  My understanding of the original bug (636193) was to "list the known", which seems like a subset of "sufficient to write a rule" which could be a new bug, so I'll claim victory with the current state of the branch plus feedback from Chex when he gets a chance to look at the screenshot.  Thanks for the input.
<lifeless> my pleasure, sorry to drag this offbase.
<lifeless> Bad habit of mine
<benji> lifeless: on a related note, you reviewed the branch and found one flaw that I've since fixed, do you feel comfortable marking the code part accepted now? (https://code.launchpad.net/~benji/launchpad/bug-636193/+merge/46350)
<lifeless> benji: I think the test_unknown_scope
<lifeless> should do self.assertFalse(scopes.lookup('not-a-real-scope'))
 * benji looks
<lifeless> otherwise a legitimate implementation could return 'yipee-a-i-m*f*' for unknown scopes,
<benji> heh
<benji> ok, will do
<lifeless> benji: can I suggest showing the info on the +feature-rules page, rather than a separate page?
<lifeless> benji: seems like less clicking for admins, and the rules for visibility are identical
<lifeless> but perhaps thats tricky to do? If so its not a big deal
<benji> lifeless: that'd be fine with me; I was expecting that they'd open them in multiple tabs/windows (being expert users) and that they would prefer that to scrolling up and down
<benji> I think it'd be easy to do.
<lifeless> benji: lets see what $losa says
<lifeless> mbarnett: yo, I know you're around :P
<benji> sounds good
<lifeless> docstring_dedent seems like it really wants to live somewhere more common
<lifeless> benji: getUserBrowserAsTeamMember looks like its copied from somewhere? perhaps pull up to the base class?
<lifeless> at a guess getUserBrowserAsAdmin too
<lifeless> benji: recommendation: rather than using tearDown, use addCleanup in setUp
<benji> lifeless: putting it in a more general place would be reasonable.  I normally follow the rule to promote something to a wider scope when it's needed in that scope, but that does assume the knowlege that the thing exists, so the alternative of "put general things in a general place" might pervail in a project with so many devs as LP.
<lifeless> benji: or even make a small Fixture to save and restore those things - I see lots of semi-repetitive code
<benji> lifeless: re. addCleanup indeed; I'll do that
<benji> I'll look into making a Fixture.
<lifeless> (bin/pydoc fixtures is a good starting point for that)
<mbarnett> lifeless: looking into another problem, will need a moment.
<lifeless> mbarnett: no panic
<lifeless> benji:
<lifeless> 440	+    def __init__(self, request):
<lifeless> 441	+        self.request = request
<lifeless> is repeated in most (all?) scopes; perhaps move to the base?
<benji> +1
<lifeless> in fact, its on the base, just needs gc :)
<benji> :)
<lifeless> ok thats all; the structure looks fine. I can see a few places where I might have done it different, but none that matter in any significant way
<lifeless> I'm going to mention one of them
<lifeless> which is that I probably would have delegated the 'can handle' check rather than exposing a predicate regex
<lifeless> anyhow, I'll change my vote now because the critical stuff is already done
<benji> lifeless: thanks.  A small note on the exposed regex (sounds like something you'd go see a doctor for), I did it that way because I wanted to use the regex in the info page (which seemed sane only because it is developers and losas who will be reading it).  Perhaps that should have just been non-executable words decribing the structure of the scopes along with code for "can handle" as you suggest.  But, as you say, it's done and the 
<lifeless> your text cut off
<lifeless> at
<lifeless> 'done and the'
<benji> "it's done and the way it is now won't kill any kittens."
<lifeless> right
 * benji applies -q to himself.
<flacoste> lifeless, benji: updating a LEP once something is implemented is a waste of time in my IMHO
<flacoste> design documents are historical
<flacoste> not live documentation
<flacoste> the code is the ultimate source
<benji> flacoste: I agree.  The maintenance overhead is pretty high and, historically speaking, unlikely to keep pace with the code.  I wish I knew how to really do architechural documentation, but I don't.
<sinzui>  deryck, regarding the ajax retarget project bug. does the callback update the new project's milestones and check permission that I can change status and importance?
<lifeless> flacoste: LEPs are not implementaiton docs
<deryck> sinzui, I'm sure it checkes perms.  Not sure about updating milestones.
<lifeless> flacoste: they document constraints and goals
<lifeless> flacoste: keeping those up to date is valuable IMO
<flacoste> lifeless: they also serve as design documents
<flacoste> as more artefacts are added to it
<flacoste> like mock-ups
<flacoste> and other stuff
<lifeless> flacoste: so, we could delete all that stuff easily when the thing is done, and be left with the core
<flacoste> i agree that keeping constratins and goals up to date is valuable
<thumper> leonardr: morning
<leonardr> thumper, hi
<leonardr> https://code.launchpad.net/~leonardr/launchpad/fix-xhtml-encoding/+merge/47551
<leonardr> that's in ec2 now
<thumper> leonardr: exactly the question I was going to ask :)
<lifeless> flacoste: so it sounds like delete-the-redundant-stuff-after-impl and keep live, will make us both happy?
<flacoste> lifeless: it would, i'd be interested in what jml thinks, care to ask the list and him for feedback, we could make that the last step of the approval process?
<lifeless> sure thing
<thumper> leonardr: any idea why I can't run all the lazr.restful tests?
<leonardr> thumper: i think i only said this in email, but your patch doesn't solve the problem for the default field renderer
<leonardr> so it might be causing failures there. can you paste me the failures?
<thumper> leonardr: I get exactly the same errors running on trunk
<thumper> leonardr: http://pastebin.ubuntu.com/558680/
<bigjools> lifeless: check this out and tell me what you think https://code.launchpad.net/~julian-edwards/launchpad/debversion-denormalise/+merge/47578
<leonardr> thumper: yeah, that's something different. let me just try a sanity check
<thumper> leonardr: ok
<lifeless> bigjools: did you intend to drop the bz2 stuff?
<bigjools> lifeless: que?
<lifeless> bigjools: read the diff
<lifeless> +COMMENT ON COLUMN NameBlacklist.admin IS 'The person who can override the blacklisted name.';
<lifeless> etc
<bigjools> lifeless: the diff is crack
<lifeless> MULTIPROC=0j 4
<bigjools> completel crack, wtf
<bigjools> oh shit
<bigjools> wrong merge target
<lifeless> bigjools: ok, so a few thoughts
<leonardr> thumper: the trunk tests pass for me, so it may be a problem with your setup
<lifeless> bigjools: I'd consider leaving the old index in place
<bigjools> lifeless: wait up :)
<thumper> leonardr: probably...
<leonardr> did you by any chance help jcsackett with his problem yesterday? he had a problem pertaining to error views
<lifeless> bigjools: and making the new one live after the deploy
<thumper> leonardr: points to some missing dependencies or something
<thumper> leonardr: no, I didn't talk to jcsackett yesterday
<lifeless> bigjools: that would reduce the time to do the db patch
<bigjools> lifeless: why's that? - I can probably guess why but...
<leonardr> let me think more deeply about the errors
<jcsackett> leonardr: you solved the only issue i was having yesterday.
<leonardr> either FakeRequest is not being considered a request, or the default registration for IClientErrorView is not taking effect
<bigjools> lifeless: on dogfood it was pretty quick - about 5 mins.  Production will be significantly quicker
<bigjools> lifeless: new one https://code.launchpad.net/~julian-edwards/launchpad/debversion-denormalise/+merge/47580
<lifeless> bigjools: 5 minutes is -slow- :)
<bigjools> lifeless: dogfod is S L O W
<bigjools> prod is typically an order of magnitiude quicker
<lifeless> its going to be running python
<abentley> thumper, I can has review? https://code.launchpad.net/~abentley/launchpad/build-mail2/+merge/47563
<bigjools> lifeless: que?
<lifeless> bigjools: debversion_sort_key is plpython isn't it ?
<bigjools> lifeless: no
<bigjools> ummm maybe
<lifeless> by which you mean yes?
<wgrant> Have we considered using postgresql-8.4-debversion?
<lifeless> there are 5million rows in the table
<bigjools> how do I find out
<bigjools> lifeless: like I said, it will not be that slow on production hardware
<leonardr> thumper: not sure what this means, but i can get your first three errors by removing the final stanza from src/lazr/restful/configure.zcml
<bigjools> lets consider only changing it if we go over budget
<thumper> ?!?
<leonardr> thumper: can you check your zope eggs for local changes?
<lifeless> wgrant: I don't think we have, no.
<wgrant> bigjools, lifeless: There is an index on debversion_sort_key(version) already... I don't see the point of denormalising.
<thumper> leonardr: I was running from a fresh branch of lp:lazr.restful
<thumper> leonardr: so it isn't in an egg
<thumper> leonardr: it did that real slow buildout process
<bigjools> wgrant: it's not about the index
<bigjools> it's about the speed of calling that function
<lifeless> bigjools: LANGUAGE plpythonu IMMUTABLE RETURNS NULL ON NULL INPUT AS
<bigjools> which is shitty slow
<leonardr> thumper: incidentally, i remembered how to stop the slow buildout process
<wgrant> To do what?
<lifeless> bigjools: its plpython
<wgrant> It is mostly used for sorting.
<bigjools> lifeless: ok
<thumper> leonardr: I'd love to know
<wgrant> Sorting should be handled by the index.
<lifeless> bigjools: doing an re.compile on *every call*
<lifeless> arggggggggggggggggggggggggggg
<leonardr> thumper: you need a ~/.buildout/default.cfg
<leonardr> mine looks like this
<bigjools> lifeless: nice!
<leonardr> [buildout]
<leonardr> eggs-directory=/home/leonardr/.buildout/eggs
<leonardr> download-cache=/home/leonardr/.buildout/download-cache
<leonardr> except without all that whitespace
<lifeless> bigjools: hand me a spork, I must remove mine eyes
<bigjools> wgrant: it doesn't use it when the sort is across columns on different tables
<lifeless> bigjools: for your pleasure: database/schema/trusted.sql
<leonardr> create those directories and the eggs will be cached
<lifeless> wgrant: see my perf tuesday mail from wednesday
<bigjools> lifeless: huh.... had no idea it was in there!
<lifeless> bigjools: anyhoe, 5M rows, 0.5ms per row, 2500seconds
<bigjools> wgrant: I made the column on DF and it reduced a query from 350ms to 200 by using it instead of the  func
<lifeless> bigjools: thats for binarypackagerelease only
<thumper> leonardr: ah... cool
<bigjools> lifeless: where did you get that stat from?
<wgrant> Ah, I see.
<lifeless> bigjools: the time it takes to add the column into the intermediary table in the OOPS we see
<lifeless> bigjools: 55K rows takes 10 seconds
<bigjools> now, if we had name as a string on the SPR/BPR tables ...
<bigjools> hmmm that sounds dodgy
<lifeless> bigjools: right, we could index name, version, id desc
<bigjools> lifeless: I'm kinda tempted to do that but it's a major change
<lifeless> bigjools: indeed; I want to talk to stub about a cross table function
<lifeless> bigjools: resulting in a per-table index
<lifeless> I dunno if its possible
<bigjools> I'm interested
<lifeless> hes on leave for a bit
<bigjools> jtv is back tomorrow, he'd know
<lifeless> basically I'm wondering if we can put an index on the history table or the packageversion table
<lifeless> index foo on FUNCTION()
<lifeless> erm
<lifeless> FUNCTION(id)
<lifeless> and have the function grab stuff from related tables and return a tuple
<bigjools> you can do that, see binarypackagerelease_version_sort
<wgrant> That would be really useful, but I've never seen it done before.
<bigjools> although not sure how you'd access related tables, there must be a way
<leonardr> thumper: here's my advice for getting to the bottom of this. put a breakpoint in AbsoluteURL.__str__ or getMultiAdapter, so that you get a breakpoint when there's an exception
<leonardr> and then look at the registered adapters
<lifeless> bigjools: right, I know you can build on a function
<lifeless> bigjools: the question is if you can build on a function *that accesses other tables*
<bigjools> lifeless: anyway are you ok with the general direction of that patch?
<thumper> leonardr: ok, I'll look at it later. Got other things that have bubbled to the top of the todo list
<wgrant> sinzui: How is the translations export issue?
<lifeless> anyhow, if you can you'd return (name, debversion_sort_key)
<leonardr> thumper: you can call getGlobalSiteManager().registered_adapters() to get an iterator over the registered adapters
<lifeless> bigjools: 700K souce package releases
<leonardr> see why the appropriate adapter isn't registered
<lifeless> so 6M total calls we need to make
<leonardr> but, i'm pretty sure it's a problem with your setup (although it could easily be a problem with my setup, since i'm the main person who runs the lazr.restful tests. but, i changed computers recently)
<sinzui> wgrant: I have not looked beyond the bug since I have been chasing mailman issues
<bigjools> creating the index is fast after the column is added, on DF it was seconds
<sinzui> wgrant: I did intend to work on it today
<wgrant> sinzui: I'll do it shortly.
<lifeless> bigjools: sure am, the extra column is totally fine given our state of the art
<lifeless> bigjools: let me do a quick test
<jcsackett> sinzui, wgrant: i had machine issues, so i have not looked at it either.
 * bigjools waits
<wgrant> sinzui: Ooh, squad assignment. I like it.
<lifeless> bigjools: the only concern I have is whether to do it all at once or just add the facility, populate & use in a couple of steps
<sinzui> bigjools: wgrant, lifeless, I reported a bug about 18 months ago that the db's debversion is not the same as launchpads. Many pages use Lp because is behaves better. I think we should have one trusted way to sort by version
<bigjools> we should look at installing the PG package
<wgrant> We should. It is C, so should be faster.
<bigjools> exactly
<bigjools> re.compile on every invocation..... aieeee
<wgrant> Yeah, taking that into account, it should be a LOT faster.
<bigjools> and, it's a package, so it, errr, gets bugfixes
<lifeless> sinzui: we'll need two versions
<sinzui> bigjools: blackname list is compiles once ons startup, why isn't debversion
<lifeless> sinzui: but they should be identical, and the current function isn't up to scratch
<bigjools> sinzui: see database/schema/trusted.sql
<lifeless> bigjools: select debversion_sort_key(version) from sourcepackagerelease;
<lifeless> Time: 71177.609 ms
<lifeless> bigjools: so ~ 5 minutes for all
<bigjools> lifeless: so let's install this package and deprecate our plpython
<wgrant> Hmm. How quickly can we get that installed on mawson to test? :)
<lifeless> bigjools: its a new datatype
<bigjools> lifeless: on which system?
<lifeless> bigjools: we need to significantly more than just install it; I agree its a great thing to do though.
<lifeless> bigjools: sourcherry
<bigjools> new datatype?
<sinzui> bigjools: okay. blackname list does multiple compiles to cover the two paths we might take through the method (admin vs approved tesm)
<sinzui> maybe I can help make a faster version
<lifeless> bigjools: the postgreslq-8.4-debversion package adds a custom datatype to postgresql
<bigjools> that which the function returns?
<wgrant> So you don't use a function; you can just use normal operators on the column.
<lifeless> This package
<lifeless>  implements a "debversion" type to represent Debian version numbers
<lifeless>  within the PostgreSQL database.
<bigjools> yeah, we don't give a rats ass about the type really
<thumper> abentley: this is just a refactoring branch?
<lifeless> we'll need to do data & code migration
<lifeless> bigjools: well, we might if it works well
<abentley> thumper, correct.  I'm building the mail changes on top of it now.
<bigjools> lifeless: no we won't - we can just use it to populate a new column that we sort on
<thumper> abentley: ok
<lifeless> bigjools: thats data migratoin and code migration :)
<bigjools> lifeless: it's trivial
<lifeless> sure
<lifeless> just needs to be done
<bigjools> I'll do it then
<lifeless> in a bunch of places, and we need to make sure we don't regress anything
<lifeless> I'm pro doing it
<bigjools> not in a bunch of places actually :)
<lifeless> just refusing to underestimate ;)
<bigjools> just where it's inserted
<bigjools> we never read it directly
<lifeless> we never read BinarhyPackageRelease.version ?
<bigjools> that's a separate column
<bigjools> I am talking about adding a new column that we initially only sort on
<bigjools> then we can take our time migrating from the old version to the new one
<lifeless> I imagined that the point of the custom data type is to have that *as* the version column, eventually.
<bigjools> yes, eventually
<lifeless> bigjools: right, and thats the cost I'm referring to :)
<bigjools> pffff
<lifeless> anyhow, FWIW, +1 from me.
<bigjools> it's still trivial :)
<lifeless> I'm very interested to hear how it behaves on mawson
<bigjools> lifeless: so, the first thing to ask is that does that package give a debversion_sort_key in C instead of plpython
 * bigjools looks
<lifeless> bigjools: AIUI no
<lifeless> bigjools: it gives a new datatype; you might be able to cast into that datatype from TEXT
<bigjools> ah ok, makes sense
<lifeless> it has direct glue such that an index ON column-of-that-datatype results in something sane
<bigjools> right - so it's roughly equivalent of my patch, except it's not text, it's the new type
<lifeless> so you could do on $casted-text to replicate the brittle situation we have today
<lifeless> bigjools: yes
<bigjools> ok this makes me happy - I'll go with this patch for now and then it's super easy to change later
<lifeless> yeah
<bigjools> my eyes are still bleeding from that plpython....
<lifeless> I suspect the re.compile is ~most of the time.
<wgrant> I wonder if it's changed since 2004.
<lifeless> blame
<wgrant> Ah, it was only written in 2006.
<bigjools> wgrant: sanity check: SPRs are created in gina and distroseries.createUploadedSourcePackageRelease only, right?
<wgrant> bigjools: I hope so.
<wgrant> But maybe you should use triggers?
<bigjools> wash your mouth out young man
<wgrant> The test suite and factory and blah blah blah will all create them.
<bigjools> ah factory was one more place
<wgrant> A trigger is easy.
<wgrant> And makes more sense.
<bigjools> it's also shit ugly and hard to change
<lifeless> wgrant: triggers are problematic
<wgrant> So is this whole idea :)
<wgrant> lifeless: Sure. But even in this case?
<bigjools> triggers are a bit like regexes
<bigjools> if you use them to solve a problem, you now have 2 problems
<lifeless> heh
<lifeless> theres a time and a place
<lifeless> I don't particularly care either way here, because we're unlikely to read-back the field *in this iteration*
<wgrant> leonardr: Is your EntryFieldResource fix in ec2?
<bigjools> well, yes we are by the time I'm done :)
<wgrant> Oh.
<wgrant> It's in PQM, even
<lifeless> bigjools: right, later iterations will be a problem
<bigjools> yes
<bigjools> and I don't want to be left in a situation where we find a crappy bug and can't fix it until the next downtime
<lifeless> indeed
<huwshimi> Morning.
<rambo> huwshimi: morning to you too
<StevenK> sinzui: Hi -- I'm fixing person/team merges to delete recipes, but I can't see any tests for this sort of thing -- I am missing something?
<sinzui> StevenK: yes, one moment
<sinzui> StevenK: /lib/lp/registry/tests/test_person.py TestPersonSetMerge is where you want to add your test
<StevenK> I did, it broke :-)
<sinzui> StevenK: the historical test is a a doctest called doc/person-merge.txt, but I weep when I read it
<StevenK> Haha
<sinzui> StevenK: was the break because there were references left in the db?
<wgrant> StevenK: Merges deleting recipes? Shouldn't only delete-merges delete recipes?
<poolie> <poolie> did wontfix go away?
<poolie>  or is there an acl on setting it?
<poolie> it seems odd i can mark 676766 invalid but not wontfix
<leonardr> i need some advice from someone who knows about soyuz
<poolie> hello leonardr!
<bigjools> o/
<james_w> poolie, there's an ACL IIRC
<StevenK> leonardr: O hai
<wgrant> poolie: Only the bug supervisor can set Won't Fix.
<wgrant> Hi leonardr.
<bigjools> hi leonardr :)
<poolie> ok
<leonardr> i did not expect such an outpouring of support
<bigjools> we are passionate people
<poolie> much better than expecting enthusiasm and not getting it :-)
<leonardr> general question: are there any objects in soyuz that are published on the web service but not on the website?
<StevenK> Yes, packagesets
<wgrant> ArchivePermissions
<lifeless> any reason they are not published on the website?
<wgrant> BPRDCs
<lifeless> e.g. time ?
<wgrant> They are published, but they have no views.
<bigjools> because they need a complicated UI design
<StevenK> wgrant: Currently, if the from_person has recipes, they are left untouched, and then can't be gotten at.
<wgrant> Right, it's not obviously waht the UI is.
<bigjools> and it wasn't needed at the time
<wgrant> StevenK: Right. We should probably move them to the target person, unless it's a delete-merge, in which case we could delete them.
<StevenK> Platform are perfectly happy fiddling with packagesets over the API, FWIW.
<bigjools> yeah but a visual rep at some point would be nice
<wgrant> Then there are things like BPPH and SPPH which have no browser index, but do have some fragment views.
<leonardr> the reason i ask is that web service objects are about to get a 'web_link' attribute and i need to know when to suppress that link
<wgrant> Hmm.
<leonardr> so far i have "hwdb stuff", ICountry, IArchivePermission, and the various IPackageSet classes
<wgrant> Is it worth suppressing them?
<leonardr> i was getting a lot of test failures in soyuz and the urls generated didn't look like real launchpad website urls, so i thought i'd ask
<leonardr> i don't think it's worth going on a witch hunt, but given that the existing tests are failing, i'd rather fix the failure by suppressing the link than fix it by changing the test to look for a useless link
<bigjools> maybe we need a property on the object somewhere to say that it has no UI
<wgrant> Are you generating good URLs for things like branches and MPs, that only exist on code.lp.net?
<lifeless> we should fix that
<wgrant> I mean, could you just check to see if there is a +*index view?
<bigjools> or you can examine the zcml to see if a template is defined
<abentley> thumper, changing around with which transactions get committed worries me.
<lifeless> jml would like one domain for everything (except librarian)
<wgrant> lifeless: +1
<leonardr> since this is not launchpad code, i can't really make that kind of deep introspection
<bigjools> urgh
<bigjools> blacklisting is kinda nasty
<wgrant> I'd just create a web_url for everything.
<leonardr> wgrant: code.lp.net works there are a couple problematic cases, where the vhost should be lp.net but it's actually api.lp.net. i intend to look at those separately
<wgrant> If people access it and it 404s, that's their problem.
<lifeless> leonardr: OTOH this code was written to support LP; can you think of anyway (relayering perhaps?) to make this work better
<lifeless> or as wgrant says, just include it
<leonardr> i don't think it's someone else's problem if we publish a link that doesn't work
<lifeless> and note in the docs that this is a probable url, things not published won't be usable, things on whacky domains may not be usable
<bigjools> I agree
<lifeless> I'd hate the overhead of this to dominate api requests :)
<thumper> abentley: feel free to land what you have and it can be refactored later
<leonardr> as i understand it, determining whether there's a web view for a given url requires traversing the url. is that right?
<bigjools> wgrant: have we got anything that generates the same crap as debian_sort_key anywhere in the code?
<bigjools> debversion_sort_key even
<wgrant> bigjools: I hope not.
<lifeless> bigjools: god no
<wgrant> leonardr: Can't you just look up the adapter for the object?
<bigjools> I didn't think so
<wgrant> leonardr: Since you presumably have the object.
<bigjools> so how TF do I populate it :/
<leonardr> wgrant: adapt it to what? i do have the object
<lifeless> bigjools: there are no tests for it
<wgrant> bigjools: Triggers.
<StevenK> lifeless: So, why do you think my MP will involve recipes only being built every 2 days?
<wgrant> leonardr: Views are adapters.
<bigjools> wgrant: blah blah  blah! :)
<wgrant> leonardr: You need a browser request, an object, and a view name.
<lifeless> StevenK: its a boundary problem
 * bigjools weeps
<lifeless> StevenK: 24 hour check frequency + lag to build + 24 hour window
<StevenK> It's date_created!!
<StevenK> When the build is *CREATED*, not when it started or finished
<lifeless> StevenK: ok, so it will still need a little slack
<lifeless> e.g. 10 minutes
<StevenK> lifeless: Okay, so I change it to 23 hours and 50 minutes
<lifeless> StevenK: I think that would address that concern. The one that really worries me is the doing to much work thing.
<lifeless> StevenK: why does it even examine recipes that are too fresh ?
<StevenK> Because it was designed to run once a day
<StevenK> I'll look at changing the call to see if the db can do the lifting for us.
<lifeless> StevenK: I have a suggestion; you may like (or hate). Move it into the garbo as a tunableloop.
<lifeless> that runs every hour
<lifeless> and will deal with overload gracefully etc
<StevenK> I'd prefer it ran more often
<lifeless> we can do that too, quite easily if we want to. Why ?
<leonardr> wgrant: i can create a new marker interface IWebsiteView. i can register a general adapter to WebsiteView such that getMultiAdapter((object, request)), IWebsiteView) is the same as getMultiAdapter((object, request), name='+index')
<leonardr> is that the kind of thing you're thinking of?
<leonardr> the interface would be defined in lazr.restful and the adapter would be defined in launchpad
<wgrant> leonardr: Sort of. Except that rootsites screw everything up.
<leonardr> wgrant: how so? the adaptation for a code.lp.net object won't succeed if the request is a lp.net request?
<wgrant> leonardr: There will be no +index; just a +code-index.
<wgrant> I believe.
<bigjools> lifeless: did you see my suggestion about soyuz on staging?
<lifeless> bigjools: yes thanks; its in my todo queue
<wgrant> bigjools: FWIW, that email looked OK to me.
<bigjools> lifeless: ok, I'll wait around before filing RTs
<wgrant> But I think we will have to regularly move the archives away.
<bigjools> yes
<wgrant> Since running the publisher on a forked archive is going to BOOM.
<bigjools> yes
<leonardr> wgrant: is there any way to determine by looking at an object whether the named adapter is +index, +code-index, or what?
<lifeless> jam: did I answer your question in the perf thread?
<wgrant> leonardr: I believe it's specified through the defaultView directive. But I don't know its mechanics.
<wgrant> Also, layers.
<wgrant> IBranch has +index, but it's on CodeLayer.
<jam> lifeless: I think part of it was "this is interesting,but I'm not sure I understand what you're saying", so I tried to piece it out and compose a mail as I went
<StevenK> Bwaqhaha
<jam> but yes, I think you've covered my interest
<StevenK> s/q//
<wgrant> leonardr: So, good luck with that!
<StevenK> wgrant: The merge code has an XXX from cprov that states "We only allow one PPA per user"
<lifeless> jam: making sql fast is tricky ;)
<wgrant> StevenK: Yeah...
<StevenK> wgrant: I can't see anything in the merge code for 'delete-merge'
<wgrant> StevenK: I don't know where the codepaths diverge.
<wgrant> But one is taken when you press 'Delete team'.
<wgrant> It deletes stuff, and merges into ~registry.
<StevenK> Perhaps sinzui knows
<bigjools> postgresql-8.4-debversion seems to be uninstallable
<wgrant> bigjools: Oh?
<wgrant> Works here.
<bigjools>  postgresql-8.4-debversion : Depends: libapt-pkg-libc6.10-6-4.8
<wgrant> This is a boring old maverick i386 machine, though.
<bigjools> I am amd64 maverick
<lifeless> what does rmadison claim vis-a-vis lucid
<bigjools> "libapt-pkg-libc6.10-6-4.8" exists in the package database, but it is not a
<bigjools> real package and no package provides it.
<bigjools> grrr
<sinzui> StevenK: delete team == merge team into ~registry. it is just a special case where we know the target and we do some extra cleanup before we start the merge
<sinzui> StevenK: that happen in a view. I reported a bug that the merge rules are wrong for delete because ~registry should not be given subscriptions or  assignments,
<bigjools> wgrant: when did you install it?
<wgrant> bigjools: About 5 minutes ago.
<leonardr> wgrant: i'm interpreting what you're saying as that 'finding the ones we're not using and suppressing their links' is not such a bad idea in practice
<bigjools> wgrant: wtf is solving that dependency for you?
<wgrant> leonardr: Not such a bad idea in theory, but difficult in practise.
<wgrant> And probably not terribly useful.
<jcsackett> sinzui: i have had a total computer crash. may not be up and running again come 5:30.
<wgrant> It would be *nice*, but it's hard.
<wgrant> bigjools: Checking...
<bigjools> some weird crap going on here
<leonardr> wgrant: we may be talking about different solutions. i'm not talking about the adapter-based solution
<sinzui> jcsackett: understood
<wgrant> bigjools: Yes.
<leonardr> i'm talking about the one where we add publish_web_link=False to selected export_as_webservice_entry() calls
<leonardr> that already works, and if we overlook a couple it's not a disaster and it's easy to fix
<wgrant> Ahh.
<bigjools> wgrant: libapt-pkg-libc6.10-6-4.8 is needed by the superseded version....
<wgrant> leonardr: That sounds good.
<wgrant> leonardr: As long as missing one doesn't break anything.
<leonardr> the worst that will happen is someone will follow that link, get a 404, and we'll fix it
<wgrant> bigjools: postgresql-8.4-debversion here wants libapt-pkg4.10, which apt provides.
 * bigjools scratches head
<wgrant> bigjools: My natty amd64 box also wants libapt-pkg4.10.
<bigjools> yes that's what LP is telling me
<bigjools> it wants to install 1.0.3-1 instead of 1.0.3-1build1
<wgrant> sinzui: You're running Newnity, right?
<sinzui> I am
<sinzui> It is stable today
<wgrant> sinzui: I upgraded my desktop to xorg-edgers and dropped fglrx yesterday, and it now runs OK.
<wgrant> But the thing that appears when I click on the Ubuntu button in the top right is a bit odd.
<wgrant> It takes up like 1/4 of the screen and doesn't do much.
<wgrant> And is all corrupt.
<wgrant> Is yours any better?
<bigjools> wgrant: WTF - something's corrupted my sources.list to use lucid....!
<sinzui> wgrant: oh good. I gave up and removed that ppa. I will try it again on my other computer. The menu is busted
<sinzui> wgrant: desktop is missing zietgiest goodness I think. That was the origin of my broken desktop 12 days ago
<wgrant> bigjools: Ahaha.
<bigjools> "Need to get 166MB of archives."
<bigjools> sigh
<wgrant> sinzui: I'm on a newish Radeon, so the new Gallium3D drivers are all that work. I presume older and more Intel cards don't need xorg-edgers at all.
<lifeless> well
<lifeless> on natty xorg-edgers is landing in main atm
<lifeless> aka boom
<wgrant> It is, yeah.
<wgrant> I am pretty impressed with r600g, though.
<wgrant> bigjools: How much of your sources.list was Lucidified? All of it?
<bigjools> yes
<wgrant> Huh.
<bigjools> quite
<bigjools> I have NFI what did that to me
<sinzui> wgrant: I do have an older radeon. I was using edgers last year, but gave it up with maverick. Unity dependencies are easier to achieve with compiz. But as I wrote, desktop is missing bits that work on a netbook config
<lifeless> sinzui: did you run the ppa removal tool ?
<lifeless> sinzui: there was/is stuff in edgers that never hit the primary archive and has a higher version number
<lifeless> if you don't run the removal tool, Things Will Break
<sinzui> lifeless: no, I may have used the desktop's source panel.
<lifeless> ppa-purge
<sinzui> lifeless: I will keep that in mind
<wgrant> Is qastaging broken?
<wgrant> Most things seem to time out.
<wgrant> And the DB never seems to reset.
<StevenK> I noticed lots of timeouts while QAing in the airport lounge at Dallas.
<lifeless> usually cold cache effects
<wgrant> I finally got a page to load after about 20 retries.
<wgrant> And the DB is still three months old :/
<StevenK> I'm concerned about that -- I didn't think it would be *that* old
<wgrant> It's from just before Natty was initialised.
<henninge> wgrant: which one? staging or qastaging?
<wgrant> henninge: qastaging
<wgrant> staging seems to be OK.
<sinzui> qastaging is ancient and awkward
<henninge> yes, staging was restored on the 23rd
<sinzui> I have been using staging for testing that needscurrent data
<sinzui> wgrant: mumble?
<wgrant> sinzui: Sure.
<wgrant> Argh, Unity is breaking things.
<wgrant> I can here you.
<wgrant> *hear, argh.
<StevenK> lifeless: So, isn't qastaging's database supposed to be done at the same time as staging's?
<james_w> lifeless, would you explain what you mean by "conflicting UI requirements"?
<james_w> there's no discussion in the bug or references to any
<cjohnston> Whats up with the horribly long wait time for downloading translations?
<cjohnston> Sorry.. Should have asked in #launchpad
<wgrant> cjohnston: I'm fixing that right now.
<wgrant> cjohnston: The exporter is broken at the moment.
<cjohnston> Gotcha
<cjohnston> thanks wgrant.
<cjohnston> Will it take 3 days to catch up?
<wgrant> I don't know enough to say, sorry.
<cjohnston> Not a problem
<wgrant> Hmm. I want a decorator that wraps a context manager.
<bigjools> lifeless (or anyone), to use the debversion type we need to run a .sql file that comes with the package to load it into the database.  Is there a "right" way of doing that or shall I mangle it in with trusted.sql?
<wgrant> Hm, that's a bit sad.
<bigjools> yes :/
<wgrant> I'd hack it into the Makefile, I think.
<bigjools> yeah that's what I meant
<bigjools> it's in schema/upgrade.py as well
<bigjools> I need a dba
<wgrant> Gaaaaah.
<wgrant> Can't do that.
<wgrant> Try running it a second time :(
<bigjools> ok so it'll need to be run manually
<wgrant> postgres appears to not do CREATE OR REPLACE OPERATOR
<bigjools> but the Makefile needs it for .dev
<lifeless> flacoste: ping
<lifeless> james_w: on the one hand claim is *meant* to stpo notifying other folk; on the other hand you want it to *keep* notifying other folk
<lifeless> bigjools: put it in a patch script
<lifeless> those are run once.
<bigjools> lifeless: urgh
<wgrant> Does anybody know if I can somehow curse a store such that attempts to access it will explode?
<wgrant> In particular, I want to be able to test that the librarian client's addFile method doesn't touch the slave store, even in SlaveDatabasePolicy.
<james_w> lifeless, I said an acceptable solution for me was to notify saying "this review has been claimed"
<lifeless> james_w: I missed that
<james_w> I just don't want a dead end in my email client, as I can't distinguish between "unclaimed" and "claimed" without a GET
<lifeless> wgrant: monkey patch
<wgrant> lifeless: I was hoping for something less bad.
<wgrant> But OK.
<wgrant> I guess I'll look around and see what the policy tests do.
<thumper> hmm...
<bigjools> Hermione Granger is less bad, and has the ability to curse stuff
<thumper> leonardr: don't suppose you are still around?
<wgrant> thumper: Is it still broken?
<thumper> wgrant: no, this is something else
<wgrant> Oh, good.
<wgrant> As long as it's not a blocker.
<thumper> https://bugs.launchpad.dev/api/devel/evolution/+bug/12/importance?ws.accept=text/json
<_mup_> Bug #12: "Next 10 messages" changes Display Settings <lp-translations> <Launchpad itself:Fix Released by daf> < https://launchpad.net/bugs/12 >
<thumper> I'd expect "Medium"
<thumper> I get: [{"token": "UNKNOWN", "title": "Unknown"}, {"token": "CRITICAL", "title": "Critical"}, {"token": "HIGH", "title": "High"}, {"token": "MEDIUM", "selected": true, "title": "Medium"}, {"token": "LOW", "title": "Low"}, {"token": "WISHLIST", "title": "Wishlist"}, {"token": "UNDECIDED", "title": "Undecided"}]
<thumper> https://bugs.launchpad.dev/api/devel/evolution/+bug/12?ws.accept=text/json
<_mup_> Bug #12: "Next 10 messages" changes Display Settings <lp-translations> <Launchpad itself:Fix Released by daf> < https://launchpad.net/bugs/12 >
<thumper> gives me a json dict
<thumper> which has 'importance': 'Medium'
<thumper> so I find the other request confusing
<wgrant> thumper: It does at least have "selected": true.
<wgrant> So it's not entiiiiirely useless.
<thumper> not sure what to do about it though
<thumper> wgrant: it does...
<thumper> but my "refactoring" is broken because of this
 * thumper will revert a small change
<thumper> and refactor something else until I've chatted with leonardr about it
<james_w> does it perhaps list them all so you know what the current user can select?
<thumper> james_w: it does, I expect, but the behaviour is not consistent
<thumper> that is the problem
<james_w> ah, consistent with the full representation
<thumper> james_w: I'd expect the json representation of the field to be consistent whether I ask for the entry or the field
<thumper> I can guess why it was written that way
<thumper> we may need to tweak :)
<thumper> yay lazr.restful
 * thumper goes to lunch
<lifeless> sinzui: hi
<lifeless> sinzui: branch deletion api
<sinzui> yes
<lifeless> sinzui: I've seen reference to some prior decision not to delete related assests
<sinzui> yes. we make the user do it and the mode object raise an exception
<lifeless> is there some where I can read up on that? It seems like we're just shuffling things around at the moment
<sinzui> I do not think we ever document things like this in dev.lp.net
<lifeless> sinzui: is there any reason we can't do that over the api ?
<lifeless> sinzui: or just let the api delete succeeed ?
<lifeless> sinzui: my concern is that this makes programmatic management of branches like the package import ones break down
<sinzui> lifeless: I am current with the discussion I think the way to allow users to shoot themselves in the foot is to 1. add a method that does the extra work and 2. provide a delete_dependents=False arg to the delete method. Let the users be expclite
<lifeless> sinzui: AIUI destructors may not accept arguments
<lifeless> leonardr: ^ is this true ?
<sinzui> lifeless:  less capable users are writing scripts that check for releases, milestones, packaging links, and delete them before deleting the master object
<lifeless> sinzui: and merge proposals ?
<sinzui> lifeless, this is also how some projects differ bugs to a new series/release
<lifeless> sinzui: I believe you have you bypass ownership to do this
<sinzui> That does not surprise me. that is what the other deletes do
<sinzui> I can delete a series and your private bugtask goes, you have no control over it
<lifeless> sinzui: which is why I'm saying that manually looking for and removing relations is not a good design or iddiom
<sinzui> The owner of the master object wins, but that is why we let the users know the consequences in the UI
<lifeless> sinzui: we don't do this on series though ... we just delete it as you say.
<wgrant> 'delete'
<lifeless> anyhow, what I want to achieve is the following:
<lifeless>  - I want james_w's script to cleanup ubuntu package imports to work
<lifeless> maxb: around?
<wgrant> sinzui: What is the status of your mailman QA?
<sinzui> lifeless: I do understand. we need to move th pre-delete rules into the model, and let the user choose to use them. What would be nice is a way to automatically deduce dependencies. Deleting the s-m-r-f chain took a lot of landing to fix rare cases like private and conjoined bugs
<sinzui> wgrant: I do not have syncage: https://qastaging.launchpad.net/~test-list-sync
 * sinzui made up a word
<wgrant> :(
<lifeless> sinzui: could you do me a huge favour? file a (low|high) bug about making this work, with your info, and mention its number in the one about delete oopsing.
<sinzui> I I am landing the config fix now
<wgrant> Otherwise we should be unblocked for deployment in a few hours.
<sinzui> lifeless: okay
<lifeless> sinzui: thank you!
<bigjools> sample data I freaking hate you
<wgrant> bigjools: :(
<bigjools> the fact that it gets loaded after all the patches are run is awkward
<bigjools> it means that I have to change it
<wgrant> Ah.
<wgrant> There are instructions for this :)
<bigjools> since I have an evil plan to make version in the model class point at debversion in the db
<leonardr> lifeless: right, it's either destroyed or not destroyed
<poolie> leonardr, i'd love to talk to you about how wadl should be used by wrested some time
<wgrant> Could someone please review https://code.launchpad.net/~wgrant/launchpad/bug-707741-addFile-master/+merge/47606?
<wgrant> It fixes the rosetta-export-queue.py failure.
<StevenK> wgrant: That looks good -- I think you should also check you can get the file out of the librarian, if you feel it's needed.
<wgrant> slony please go away :(
<wgrant> StevenK: Done.
#launchpad-dev 2011-01-27
<wgrant> StevenK: I can has blessing?
<StevenK> thumper: https://code.launchpad.net/~wgrant/launchpad/bug-707741-addFile-master/+merge/47606 please
<StevenK> wgrant: I was getting there :-)
<wgrant> StevenK: Great, thanks.
<wgrant> So many regressions :(
<lifeless> leonardr: is there some reason we can't have a destructor parameter? That doesn't seem to alter the boolean nature
<wgrant> lifeless: Thanks.
<wgrant> lifeless: Bugs needs a kanban UI.
<wgrant> ... well, this is amusing.
<wgrant> getUnlandedSourceBranchRevisions has a reasonably complex query, and it is completely untested.
<wgrant> I expect that sort of thing from Soyuz, not Code :/
<lifeless> poolie: https://pastebin.canonical.com/42398/
<lifeless> -> allergy injection
 * henninge eods
<wgrant> StevenK: Another regression fix for you, if you have a moment: https://code.launchpad.net/~wgrant/launchpad/bug-707808-unlanded-revisions/+merge/47617
<StevenK> lifeless, thumper: https://code.launchpad.net/~wgrant/launchpad/bug-707808-unlanded-revisions/+merge/47617 if you please
<thumper> StevenK: ack
 * StevenK kicks the merging code
<wgrant> thumper: https://code.launchpad.net/~deon-wurley/phpldapadmin/1.2.x <- are Remote branches without URLs legal?
<thumper> wgrant: yes...
<thumper> wgrant: but sad
<wgrant> WTF
<wgrant> But OK.
<thumper> wgrant: not much point really
<Ursinha> wgrant, hi
<wgrant> Ursinha: Hi, 'sup?
<Ursinha> wgrant, so, there are bunches of ApacheLogParserError
<wgrant> Ah, those.
<wgrant> All fixed, but not deployed yet.
<Ursinha> like this: https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1851PPA1051
<wgrant> Since germanium isn't in nodowntime.
<Ursinha> hmm, I see.
<Ursinha> wgrant, thanks :)
<wgrant> I hope we can germanium in nodowntime soon :(
<wgrant> +put
<wgrant> thumper: Thanks.
<thumper> np
<wgrant> All major regressions fixed, yay.
<StevenK> Now deploy them?
<wgrant> Maybe in 8 or so hours.
<StevenK> Longer, devel will hit testfix
<wgrant> Heh.
<wgrant> Oh fuck, you're serious.
<wgrant> Gnrwgwnfjew
<thumper> why
<thumper> ?
<wgrant> That spurious librarian thing.
<thumper> oh FFS
<wgrant> We should kill the current build.
<thumper> yes
<thumper> +1
<wgrant> spm: Can we easily kill a buildbot build?
<spm> yah. which one.
<StevenK> Where easy is er, not
<wgrant> If not, could you do it the hard way? :P
<wgrant> lucid_lp
<spm> where kill == stop.
<StevenK> spm: 556 on lucid_lp
<wgrant> I'm off to lunch now; could someone force a build once it's done?
<StevenK> wgrant: Aye
<StevenK> spm: Tell me when it stops hurting
<spm> so the lsave is completely ded. how do we clean this up so it won't run that revno again?
<StevenK> The revno is fine
<StevenK> The errors are spurious and we don't want to be in testfix
<spm> oh I see. perhaps getting the spurious nature of the errors fixed would be a GoodIdeaâ¢? ;-)
<StevenK> Here, have a Hudson
<StevenK> That should help
<spm> :-)
<StevenK> spm: Do you need to restart the master to get the slave back?
<spm> seems to be showing back to me. should be able to force and magic happen
<StevenK> It's buildbot, it isn't close to magic
<StevenK> spm: Done, thanks
<spm> anti-magic?
<StevenK> wgrant: Build forced
 * thumper stabs the librarian
<thumper> AttributeError: 'NoneType' object has no attribute 'add_section'
<thumper> while trying to set up the librarian in tests
<thumper> anyone got any ideas?
<lifeless> poolie: ok I'm back
<lifeless> thumper: that means the layer which supplies the config to edit hasn't been setup
<thumper> lifeless: which I'm getting why?
<thumper> it has only just started happening
<thumper> after merging dev
<thumper> also
<thumper> bin/kill-test-services is borked
<lifeless> theres a bug on that
<lifeless> with an explanation; fix should be trivial
<poolie> lifeless, i think your mail is fine
<poolie> i replied too
<lifeless> poolie: well, I've just sent my mail
<poolie> interesting new reply from max
<thumper> lifeless: my librarian layer won't setup
<wgrant> thumper: Turn LP_PERSISTENT_TEST_SERVICES off.
<wgrant> It doesn't work any more.
<thumper> ah
<thumper> poo
<lifeless> iz bug
<lifeless> I put work in to make it work
<wgrant> It's been that way for a week or so now.
<lifeless> but its not actually tested
<wgrant> Ah.
<wgrant> I see.
<lifeless> I didn't *intend* to break it
<lifeless> but I couldn't tell that I *had*
<lifeless> also
<lifeless> its useless for parallel testing
<wgrant> I presumed you turned a blind eye to your breakage of it.
<wgrant> It is, yes. But we have no parallel testing yet.
<lifeless> so I'm not very interested in maintaining it
<lifeless> wgrant: we have some, a decent subset of tests work fine in parallel now
<wgrant> Indeed.
<wgrant> It is close.
<wgrant> The main problem now is AppServerLayer.
<lifeless> indeed
<wgrant> Plus some directories that need randomising.
<lifeless> yes
<maxb> lifeless: You binged?
<wgrant> 2.7 is close enough to be done by a maintenance squad, and I think parallel testing is probably almost there too.
<lifeless> maxb: python-subunit 0.0.6 in the lp ppa
<lifeless> maxb: part of fixing pqm to mail on conflicts
<lifeless> maxb: however it looks like the losas can grab from the bzr ppa ok
<lifeless> wgrant: feature work -> feature queue
<lifeless> wgrant: if you're looking for something to do, I see a single bug causing 16K oopses a day.
<lifeless> wgrant: (hint)
<wgrant> Heh, no, I'm not looking for stuff to do :(
<wgrant> That's the codebrowse connection closed one?
<StevenK> lifeless: No fair if it's loggerhead
<lifeless> wgrant: seriously, yes its close, but I guarantee it will have a long tail
<lifeless> StevenK: loggerhead probably has more test coverage the LP proper
<lifeless> wgrant: and the long tail is what makes me want to give it to a feature squad: get it live in buildbot/hudson, working in tarmac, profile it and assess scaling, disk IO utilisation, make recommendations on a new server to run massively parallel tests
<lifeless> wgrant: figure out the damn db leak bug
<wgrant> lifeless: I think that figuring that out is probably best achieved by deleting layers and zope.testrunner.
<lifeless> wgrant: I have handwavy plans for that
<lifeless> wgrant: its also pretty shallow
<lifeless> but, time.
<wgrant> Indeed.
<lifeless> wgrant: focus will get us places!
<StevenK> lifeless: I'd prefer if codebrowse actually looked like the rest of LP
<lifeless> StevenK: so would codebrowse :P
<lifeless> StevenK: it is themeable
<StevenK> But I can recall you telling me that's pointless in Dallas
<lifeless> StevenK: that doesn't sound like me; I think I may have said tha tthat is the least of the issues
<wgrant> StevenK: Everything is pointless in Dallas.
<lifeless> StevenK: which != pointless, but I could well have been confusing the issue
<poolie> i might try just running tip on a big tree
<poolie> and see how it does
<lifeless> huwshimi: hi
<huwshimi> lifeless: Hey there
<lifeless> huwshimi: I dunno if you've seen https://dev.launchpad.net/BugTriage yet ?
<huwshimi> lifeless: Uh, no. Are you referring to the bug I just filed?
<lifeless> huwshimi: yes I am :)
<lifeless> huwshimi: have a read :)
<huwshimi> lifeless: Yeah thanks. Reading it now.
<lifeless> huwshimi: its useful context to fit in with the team; the specific thing that alerted me was creating a medium importance bug (we don't use medium) and then starting work on it (thus ignoring the critical and high bugs)
<lifeless> huwshimi: I think your particular focus means that looking at all critical/high bugs is irrelevant to you
<lifeless> huwshimi: -but- you probably want to be sorting the design related bugs by the impact/importance you think they have
<lifeless> huwshimi: anyhow, no drama; we have a massive learning curve and I primarily wanted to let you know about this part of it :)
<huwshimi> lifeless: Sure thanks for letting me know. I'm about to submit a bunch of bugs so I'm glad you told me now :)
<huwshimi> lifeless: There was some discussion last week about triaging bugs ourselves. Was there a decision made about that? Am I better off not self triaging for now?
<lifeless> huwshimi: self triage is great
<lifeless> follow the guidelines in the wiki page
<lifeless> beyond that, if there are disagreements, english is a wonderful tool for figuring em out ;)
<huwshimi> lifeless: OK thanks for that :)
 * thumper grrs
 * thumper is slowly getting there
<StevenK> lifeless: So I don't think I can do the heavy lifting with SQL. If the recipe has no builds, http://pastebin.ubuntu.com/558848/ returns nothing, and so no daily builds get dispatched
<poolie> how do i, through the api, find all bugs assigned to a particular team?
<poolie> s/team/person
<lifeless> StevenK: looking
<lifeless> StevenK: you probably want either a LEFT OUTER JOIN or a UNION, or a COALESCE
<lifeless> StevenK: probably a left outer join
<StevenK> lifeless: Storm doesn't have a LeftOuterJoin?
<thumper> hmm...
<thumper> how do I emit a tag in a page template of a particular type?
<lifeless> StevenK: it does, LeftJoin IIRC
<lifeless> we use it all over
<thumper> view/tag is something like "h1", or "span"
<thumper> and I want to get <h1 id='${view/id}'> in the pt based on view/tag
 * thumper is open to suggestions 
 * StevenK is open to patches, never having done joins through Storm before
<thumper> I'm thinking I may need an attribute and structure
<lifeless> StevenK: grep for LeftJoin
<lifeless> StevenK: I'm in the middle of a deep n meaningful over loggerhead
<lifeless> its breaking my brain to think SQL at the same time
<StevenK> Mission accomplished
 * StevenK hides
<StevenK> lifeless: If you want to point what I'm doing wrong in http://pastebin.ubuntu.com/558856/ , that would be awesome. I can't find any usage like this in the tree
 * thumper is almost done refactoring widgets
<lifeless> StevenK: wtf
<StevenK> Clearly then, the answer is "everything"
<lifeless> StevenK: the pattern is
<lifeless> (IIRC)    LeftJoin(lefttable, righttable, condition)
<lifeless> e.g.
<lifeless> SourcePackageRecipe LEFT OUTER JOIN SourcePackageRecipeBuild on SourcePackageRecipeBuild.recipe == SourcePackageRecipe.id
<lifeless> ->
<lifeless> LeftJoin(SourcePackageRecipe, SourcePackageRecipeBuild, SourcePackageRecipeBuild.recipe == SourcePackageRecipe.id)
<lifeless> the result of that is a table itself
<lifeless> so to nest you'd do
<lifeless> LeftJoin(LeftJoin(SourcePackageRecipe, SourcePackageRecipeBuild, SourcePackageRecipeBuild.recipe == SourcePackageRecipe.id), PackageBuild, PackageBuild.id ==SourcePackageRecipeBuild.package_build_id)
<lifeless> but you don't need a nested left join
<lifeless> you only need one outer join - at the point you're willing to have a NULL row
<lifeless> so
<lifeless> Join(LeftJoin(SourcePackageRecipe, SourcePackageRecipeBuild, SourcePackageRecipeBuild.recipe == SourcePackageRecipe.id), PackageBuild, PackageBuild.id ==SourcePackageRecipeBuild.package_build_id)
<lifeless> etc
<lifeless> Join(Join(LeftJoin(SourcePackageRecipe, SourcePackageRecipeBuild, SourcePackageRecipeBuild.recipe == SourcePackageRecipe.id), PackageBuild, PackageBuild.id ==SourcePackageRecipeBuild.package_build_id), BuildFarmJob, BuildFarmJob.id == PackageBuild.build_farm_job_id)
<lifeless> is your using object
<lifeless> then in the where clause you put
<lifeless> Or(BuildFarmJob.id == None, BuildFarmJob.date_created > one_day_ago)
<thumper> w00t w00t
<lifeless> StevenK: does that make sense?
<lifeless> StevenK: use LP_DEBUG_SQL to see the sql being emitted
<lifeless> StevenK: and adjust until you have a query you're happy with
<StevenK> Yes, it made sense
<lifeless> ok
<lifeless> did it help ?
<StevenK> lifeless: Not really, it still doesn't return SPRecipes that haven't built
<StevenK> I suspect SourcePackageRecipeBuild.id == None will help, since they are created for builds, which then creates the PackageBuild and BFJ
<lifeless> StevenK: you probably want to bring back the sprb as well
<lifeless> since that will tell you the when. Or perhaps not.
 * thumper EODs
 * huwshimi waves goodbye
<lifeless> StevenK: need more help? I have cycles in ~ 10
<StevenK> lifeless: Sorry, I was picking up Sarah, and I EOD'd 1.6 hours ago
<lifeless> kk
<lifeless> grab me tomorrow
<lifeless> and we can nut it out
<StevenK> lifeless: Was my plan
<adeuring> good morning
<mrevell> Guten morgen
* allenap changed the topic of #launchpad-dev to: Launchpad development https:/â/âdev.launchpad.net/â | PQM is open | firefighting: - | On call reviewer: allenap | https://code.launchpad.net/launchpad-project/+activereviews
<gmb> Does anyone know what in the sweet hell this is about?:
<gmb> An error occurred during a connection to launchpad.dev.
<gmb> SSL received a record that exceeded the maximum permissible length.
<gmb> (Error code: ssl_error_rx_record_too_long)
<wgrant> gmb: This is on a fresh LP install?
<gmb> wgrant: No, it's an old one. But I did have to re-run rocketfuel-setup recently to correct a couple of pebkacs.
<Ursinha> good morning launchpad
 * gmb re-does the apache config dance
<wgrant> gmb: Odd. I normally only see that when it's a fresh install needing an Apache restart, or I've broken the vhost config somehow.
<gmb> wgrant: Yeah. I think I've borked the vhost config in some myserious way (probably because I have a remote access setup and rocketfuel-setup fought with it).
<wgrant> Ah.
<gmb> I'm redoing it now.
<maxb> I find the nicest way to manage a remote access thingy is to let rocketfuel-setup manage the local-launchpad config file, but a2dissite it, copy it, and modify the copy
<gmb> maxb: Very wise. I shall do that henceforth.
<gmb> Hoorah. It works.
<gmb> /me lunches
<deryck> Morning, all.
<Ursinha> morning deryck
<salgado> what's the best thing to use when writing a setup.py these days?  setuptools?  distribute?
<jelmer> salgado: what sort of features do you need from it?
<jelmer> distutils is widely available and seems to work pretty well for basic python modules
<salgado> jelmer, our needs seem to be rather basic, so maybe distutils will be enough indeed
<jml> salgado: I copy an existing setup.py :)
<salgado> jml, that's always a good strategy
<jml> can someone give me a dumb manager's summary of the codebrowse discussion on the mailing list?
* jcsackett changed the topic of #launchpad-dev to: Launchpad development https:/â/âdev.launchpad.net/â | PQM is open | firefighting: - | On call reviewer: allenap, jcsackett | https://code.launchpad.net/launchpad-project/+activereviews
<salgado> benji, your last message to that mp I reviewed yesterday had only some quoted text from my previous message
<benji> salgado: hmm, that's odd.  I just said "sorry about that, thanks for asking Curtis to look at it"
<salgado> benji, looks like the issue I encountered yesterday; already reported a bug for it
<benji> do you have the bug number at hand?
<abentley> jelmer, ping
<abentley> jelmer, unping
<oyv> hi
<oyv> i'm trying to setup launchpad on a virtual machine, but when i try make run i get an importerror (no module named loom.branch)
<oyv> anybody got any hints?
<oyv> http://pastebin.com/JBn59ekF
<abentley> flacoste, now that domain expertise is spread across squads, maybe it would be good to have a list of domain experts, e.g. on the wiki?
<oyv> (i can import the branch in python shells, so it should be installed and everything)
<jcsackett> oyv: are you following the instructions on the wiki?
<oyv> i use isntructions from here: https://dev.launchpad.net/Getting
<oyv> (and the "running" page)
<jcsackett> oyv, ok. what version of ubuntu are you running?
<oyv> Ubuntu 10.04.1 LTS (lucid lynx)
<jcsackett> oyv: i encountered this at one point; i am trying to remember how i fixed it.
<jcsackett> oyv: i assume you are not actually using bzr loom? it shouldn't fail in that case, i'm just defining what's going on.
<oyv> i don't think i'm using it, not really sure what it is ;)
<jcsackett> it's a plugin for bzr, http://doc.bazaar.canonical.com/plugins/en/loom-plugin.html. many lp developers use it, but last i checked it was not a dependency.
<jcsackett> and if it is, it should have been automatically installed when you hit the update step in the get/run instructions.
<jml> jcsackett: it is a dep
<jcsackett> jml: really? did that change? when i started on lp i was told it wasn't, b/c i was hitting this (or a similar) issue.
<oyv> edb@launchpad:~/launchpad/lp-branches/devel$ bzr plugins
<oyv> <snip>
<oyv> loom 2.1.0 Loom is a bzr plugin which adds new commands to manage a loom of patches.
 * jcsackett supposes he could have been deasily misinformed.
<jml> jcsackett: since 2008, I think. it's needed to process looms
<oyv> doesn't that mean it's installed?
<benji> either is fine with me; but more importantly it means I need to go pour my coffee right now
<benji> pfft, wrong chan
<jcsackett> jml: dig. clearly i was misinformed. :-P
<jcsackett> oyv: sorry, i'm not sure what might be going on. it does seem that it is installed as a plugin.
<jcsackett> i could be of more help if i had a functioning machine right now, but i'm rebuilding after hardware failure yesterday, so i can't explore much on my end. :-/
<oyv> ok, thanks anyway :)
<LPCIBot> Project devel build (396): FAILURE in 4 hr 38 min: https://hudson.wedontsleep.org/job/devel/396/
<LPCIBot> Launchpad Patch Queue Manager: [r=lifeless,
<LPCIBot> stevenk][bug=707741] Fix LibrarianClient.addFile to function under
<LPCIBot> SlaveDatabasePolicy.
<maxb> oyv: I don't suppose you get a traceback related to that ImportError?
<oyv> http://pastebin.com/JBn59ekF
<maxb> When you're running "make run", the copy of bzr-loom that is supposed to be being used is the one found via bzrplugins/loom/ in the Launchpad tree
<salgado> benji, it's bug 708258
<_mup_> Bug #708258: Failed to parse merge proposal comment sent via email <code-review> <email> <Launchpad itself:Triaged> < https://launchpad.net/bugs/708258 >
<benji> salgado: thanks
<oyv> hmm
 * maxb sobs a bit as launchpad-database-setup tries to configure pg8.2
<oyv> there's only one folder in the bzrplugin folder, lpserve
<maxb> Your tree is broken then
<oyv> damnit..
<oyv> ;)
<oyv> thanks
<flacoste> abentley: that's what the wiki pages related to the Launchpad in 30 minutes presentation was meant to do
<flacoste> abentley: https://dev.launchpad.net/Foundations/ComponentHelp
<flacoste> i think Tim did something similar
<flacoste> and so did Bugs
<flacoste> not sure if Soyuz and Translations put their stuff in that format
<abentley> flacoste, Ah, didn't think of looking there.
<deryck> ah, adeuring, call time.  Sorry got distracted running tests and answering emails.
<flacoste> abentley: it would probably make sense to collate all the content in one place, or at least make an easy-to-find index page to it
<adeuring> deryck: ok, no problem
<bigjools> goooooood morning
<abentley> flacoste, +1
<abentley> bigjools, can you confirm that we never send emails for successful binary builds?  Because there are some tests of Build.notify with BuildStatus.FULLYBUILT.
<bigjools> abentley: I can confirm that
<bigjools> that test sounds a bit bong
<abentley> bigjools, great.
<deryck> henninge, I need 5 minutes to wrap up some notes, and then we can chat.   if that works for you.
<henninge> deryck: that's fine
<abentley> bigjools, since recipe builds want to notify on success, I'm moving the differentiation into BinaryPackageBuild.notify/SourcePackageRecipeBuild.notify, and fixing the tests.
<bigjools> abentley: you *want* them to notify on success or you're removing that?
<abentley> bigjools, I *want* them to notify on success.
<bigjools> ok
<james_w> really?
<bigjools> sounds odd to be
<bigjools> me
<james_w> will the coalesce?
<james_w> they
<abentley> james_w, yes.  How else do I know that a recipe build has happened?
<bigjools> prepare for wrath if you do this!
<abentley> bigjools, we've been doing this from the start.
<bigjools> interesting
<james_w> because they happen every day? because Launchpad is reliable and does what I ask?
<bigjools> we really need a better email notification story
<abentley> james_w, no, they only happen when the source changes.
<abentley> james_w, my bzr recipe triggers once or twice a month.
<james_w> are you /really/ going to send people 200 emails every day in the extreme case?
<james_w> how is that useful?
<abentley> james_w, it's useful so that people know when there's a new build.
<james_w> but that information isn't very useful at all as you move towards heavy users
<abentley> james_w, if you get it and you don't want it, you can filter it out.  If you don't get it and you want it, you have no option.
<james_w> do you really want to discourage heavy use of the service using email volume
<allenap> gmb: Do you have time to do a sanity check on https://code.launchpad.net/~allenap/launchpad/freedesktop-importance-flip-flop-bug-707478/+merge/47667 please?
<gmb> allenap: Sure.
<james_w> client-side filtering has been deemed to be not sufficient in the bugs case
<abentley> james_w, client-side filtering is better than no mail.
<james_w> are you sure your users would agree with that?
<abentley> james_w, I am sure you can find users to disagree with anything.
<james_w> well, let's all go shopping then
<gmb> allenap: Approved.
<abentley> james_w, I am just fixing a bug where the wrong kind of notification is sent, not increasing the number of notifications.
<allenap> gmb: Thank you :)
<james_w> ok, then you are absolved from any responsibility for the system
<abentley> james_w, so I've been talking with jelmer about this, and we both agree that there's a lot of room for improvement in the build notification story.
<abentley> james_w, jelmer would like to see a notification of successful binary builds, grouped so that you only get one email for multiple builds.
<abentley> james_w, and then we would be able to omit the success emails for recipe builds.
<bigjools> all emails that LP sends should be controllable from a single page
<bigjools> per person, I mean
<abentley> bigjools, I don't know what to think about that.  It would be a verrrry long page.
<bigjools> potentially but not always
<bigjools> it can be ajaxified but you know what I mean - the subject of being sent machine-generated email is very divisive
<abentley> bigjools, generally, asking users to make choices is bad.  The fewer choices, the smoother something feels.  This makes me think that the need for such a page indicates a design problem.
<henninge> allenap: thanks for the review!
<bigjools> abentley: that's the Gnone way, which is completely disagree with
<abentley> bigjools, I'm not disputing the actual need, but I do wonder whether we're not thinking big enough.
<bigjools> s/is/I/
<henninge> adeuring: did you see my review of your branch?
<bigjools> anyway, I'm not bikeshedding over this
<adeuring> henninge: yes, thanks. Actually, I think we should keep the assert() calls in setUp() because it is so esay to cause a mess there, and I am not sure if this would always result in test failures
<henninge> adeuring: well, I think that is true for a lot of places in the code, though.
<henninge> adeuring: but I don't really mind leaving them in there.
<adeuring> henninge: OK; perhaps my concerns are related to my unfamiliarity with translations ;)
<henninge> adeuring: ... I tried not to be so blunt ;-)
<henninge> adeuring: I have done that before, added asserts to make sure I got things right but once it worked, I removed them.
<adeuring> henninge: ;) but nevertheless, if we can screw the setup just by exchanging two factory calls, I think it is worth to check we don't do that...
<henninge> adeuring: that's why I suggested leaving just the one in there.
<henninge> and a comment
<henninge> "# The order of creation is important here to make sure is_current_ubuntu is set to False.
<henninge> self.assertFalse(message.is_current_ubuntu)"
<henninge> adeuring: ^ like that, only use the right variable for 'message'.
<allenap> gmb: Do you remember why sourceforge.net bug watches are disabled?
<gmb> allenap: Because our super-duper sourceforge slurping screenscraping fantastical disaster rather relied on them never, ever changing their templates, and they did.
<gmb> (I might have overstated how good that code actually was)
<allenap> gmb: Oh yes, thanks. I've just had a look at their templates and they're very much simpler now. Might fix that bad boy.
<gmb> allenap: Don't they offer an API now?
<allenap> gmb: Do they? Awesome.
<gmb> ISTR they do. Maybe I saw that on the ForgePlucker mailing list.
<abentley> allenap, jcsackett could you please review https://code.launchpad.net/~abentley/launchpad/build-mail3/+merge/47679 ?  It's long, but only because of indentation fixes in a doctest.
<jcsackett> abentley: sure.
<abentley> jcsackett, thanks.
<allenap> gmb: I can't find an API, but tell me if you happen upon it.
<gmb> allenap: Yeah, I'm not able to find one either, disappointingly. Must have been a happy dream.
<gmb> I need new dreams.
<allenap> Hehe, you do :)
<abentley> deryck, I feel like I lack the domain knowledge to dive into any of the tasks on the Kanban board.  Can you suggest one?
 * deryck is looking
<deryck> abentley, so I'm sure I lack the domain knowledge too :-)  However, the card for bug 696009 seems a  nice next step.....
<_mup_> Bug #696009: Provide ITranslationMessage.shareIfPossible unit tests <tech-debt> <upstream-translations-sharing> <Launchpad itself:Triaged> < https://launchpad.net/bugs/696009 >
<deryck> abentley, it's still in the area of test clean up.  so will broaden domain knowledge as cleanup happens.
<abentley> deryck, okay, I'll tak that.
<abentley> s/tak/take.
<deryck> ok, cool
<henninge> adeuring: am I to expect a new revision on your branch? Otherwise I'll start landing mine (which includes yours and abentley's)
<henninge> actually, I'll make sure to merge the latest first
<adeuring> henninge: well, I'd prefer to leave the tests as they are... so, no new revesion ;)
<henninge> adeuring: ok, thanks. ;)
<jcsackett> abentley: comments on your MP. feel free to answer there or here.
<abentley> jcsackett, fire away.
<jcsackett> abentley: as commented on the MP, i'm wondering about the use of self.builds in the from_build method.
<jcsackett> mainly, in the first hit, can that get big enough to cause performance issues?
<abentley> jcsackett, it is always one or zero hits.
<jcsackett> abentley: so does the cache help? the cache is always gone at the end of the transaction, yes?
<gmb> I broke the build; rolling it back now. Sorry folks.
<abentley> jcsackett, the cache will not help with the cases we have at hand, because we only use it once.
<jcsackett> so putting it in cached_property is forward looking?
<abentley> jcsackett, but I felt that it made sense to cache it since one does not expect an attribute to be expensive.
<gmb> Oh, nice. I specified --rollback for lp-land but it doesn't seem to have tagged the commit message thus.
 * gmb files a bug.
<abentley> jcsackett, you could say putting it in cached_property is forward-looking.
<jcsackett> abentley: so do you see any problem with the use of self.builds being called in there? since no preloading will happen, can hitting self.builds get sufficiently expensive to be a worry?
<jcsackett> i'm not saying it will. i'm not familiar with this part of the codebase, so i'm wondering.
<abentley> jcsackett, I don't know what you're asking.  Are you wanting me to run an EXPLAIN on the query?
<jcsackett> abentley: i'm just double checking performance concerns.
<abentley> jcsackett, what kind of answer are you looking for?
<jcsackett> abentley: builds is a SQLMultipleJoin; i'm wondering if when self.build gets called we may return a huge rowset in some cases.
<abentley> jcsackett, as I said, it's always one or zero results.
<jcsackett> abenltey, ah! i thought you meant from_build was called one or zero times.
<abentley> jcsackett, there is a comment in the code saying it shouldn't be a multiple join.
<jcsackett> abentley: yes, i see now.
<jcsackett> apologies for the confusion.
<abentley> jcsackett, cool
<jcsackett> abentley: r=me. i need follow up, as a mentee :-P. bac, can you follow up on https://code.launchpad.net/~abentley/launchpad/build-mail3/+merge/47679?
* jcsackett changed the topic of #launchpad-dev to: Launchpad development https:/â/âdev.launchpad.net/â | PQM is open | firefighting: - | On call reviewer: allenap, jcsackett* | https://code.launchpad.net/launchpad-project/+activereviews
<abentley> jcsackett, I've added a comment per your suggestion.
<jcsackett> abentley: thank you. :-)
<deryck> hurrah!  Windmill re-enable branch finally made it through.
<lifeless> jml: morning
<lifeless> deryck: cool
<jml> lifeless: hi
<lifeless> jml: would you like a catchup call - we seem to be trading 1-liners this week
<jml> lifeless: Yes, I'd like one. Only have 15mins though.
<lifeless> skype?
<jml> lifeless: sure.
<thumper> flacoste: ping
<bac> hi abentley
<abentley> bac, hi.
<bac> abentley: i'm trying to mentor jcsackett's review.  the diff in the MP is overwhelmed by lint changes.  i've tried to get a diff with just the non-lint changes but failed.  could you easily whip one up and paste it?
<abentley> bac, not sure.  I don't think they were separate commits, or anything.
<abentley> bac, how's this? http://pastebin.ubuntu.com/559153/
<bac> abentley: 228 is much better than 1654!  :)  thanks!
<aroman> hello, all! Is there any way of seeing how many people are using a PPA of mine/a package? Or even just seeing how many people have branched from bzr or downloaded a .tar.gz? Thanks!
<lifeless> aroman: there is an LP API that can get you download statistics for a PPA
<flacoste> hi thumper
<aroman> lifeless: ah, but there's no sort of graphical frontend to see that information?
<lifeless> not yet
<thumper> flacoste: can you talk briefly about your review?
<aroman> lifeless: ah, alright, well i'll check that out then. I assume it's python, right?
<lifeless> flacoste: hi; can I get a small timeslice today ? (but not for ~30)
<flacoste> thumper: sure, mumble?
<thumper> ok
<lifeless> aroman: its a RESTful api, but we have a python library you can use
<flacoste> lifeless: how small?
<lifeless> flacoste: 10 min ?
<aroman> lifeless: excellent, thanks!
<lifeless> flacoste: crossing t's dotting i's on the loggerhead thing
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build (397): FIXED in 4 hr 31 min: https://hudson.wedontsleep.org/job/devel/397/
<LPCIBot> Launchpad Patch Queue Manager: [r=gary][ui=none][bug=704685] BugSubscription.bug_notification_level
<LPCIBot> is now exposed in the devel webservice API.
<gary_poster> yay gmb !
<gary_poster> oh, boo
<gary_poster> that's hudson
<gary_poster> on buildbot, it is being rolled back :-/
<lifeless> oh, why?
<gary_poster> https://lpbuildbot.canonical.com/builders/lucid_lp/builds/561/steps/shell_6/logs/summary
<lifeless> ah scaling test failed
<lifeless> I'd like to make those more reliable and isolated
<lifeless> they are just a little flaky atm
<lifeless> (different standalone vs in a larger run)
<gary_poster> yeah
<lifeless> I think they are still a net win
<lifeless> but it can be frustrating getting them going
<lifeless> allenap: around?
<jtv> bigjools: got a minute for a question about FTPArchiveHandler?
<bigjools> jtv: yup
<jtv> great
<jtv> I was just wonderingâ¦ it uses the LoopTuner all over the place already, but only for gc.
<jtv> Was it originally meant to commit transactions as well?
<bigjools> no, it's writing out files for apt-ftparchive to use
<jtv> Okay, but it's not writing to the DB that I can see and it's a big suspect in the long-transaction crime mystery I'm pursuing.
<jtv> So it'd seem that a few commits would be a relatively harmless way to get around the timeouts.
<jtv> I was thinking perhaps I could force the main body of work on the slave store, and throw in a few commits.
 * bigjools thinks
<bigjools> that code is probably the bit I understand the least in soyuz
<bigjools> it's never gone wrong since cprov left so I never had to look at it :)
<bigjools> jtv: I'm a bit scared of using the slave store in case of replication delays
<jtv> Scared is sensible.  But what's there to replicate?
<bigjools> all the publishing data it relies on, which is fast-changing
<jtv> I see.
<bigjools> jtv: it would be a good start to work out if it really is keeping a transaction open
<bigjools> if it's not writing to the db can we change the $mumble
<jtv> It has to be.  It's not committing anywhere, and it's looping over lots of packages AFAICS.
<jtv> The $mumble?
<bigjools> yeah, the thing
<jtv> Oh, the thing.
<jtv> The database policy?
<bigjools> isloation?
<bigjools> isolation ,even
<jtv> Isolation level?
<bigjools> dunno if that would affect it
<jtv> That would affect reads, not too much to do with writes.
<bigjools> can we connection r/o to the master?
<bigjools> sigh
<bigjools> connect*
<jtv> I don't think we can, no.
<jtv> Something we could try though is run a test with a slave-only policy, and see what fails.
<bigjools> ok
<bigjools> I see why you suggested the slave now
<allenap> lifeless: Hi.
<allenap> lifeless: Is this about the BugMessage crack?
<jtv> bigjools: Well it also helps scale-out and locking, of course.
<lifeless> yes
<lifeless> allenap: though I'm not sure its crack L)
<jtv> bigjools: unless there's something that changes the database and then immediately runs the script, we should get about as consistent a view of the database from a slave as we do from the masterâjust slightly older.
<allenap> I'm reading the code now to try and remind myself of the weirdnesses.
<lifeless> allenap: so
<lifeless> we decorate Message to make it an IndexedMessage but the actual relation is BugMessage which we don't expose at all
<flacoste> thumper: i'm back
<lifeless> thats beside the point though
<jtv> bigjools: "the thing"âhttps://www.ubersoft.net/comic/hd/2011/01/next-time-try-index-cards
<thumper> flacoste: want to continue?
<flacoste> thumper: sure thing
<lifeless> we can still decorate but get the indices from the db once they are cached
<bigjools> jtv: the thing I am scared of is writing out inconsistent indices in the Ubuntu archive.  That would be fairly nasty.
<bigjools> now while I am scared I am not sure how realistic the chances of that are
<jtv> In that case, we could try moving to the slave and _not_ committing.
<jtv> If that works out we'd still get a single huge transaction, but also a guarantee that it takes no write locks.
<allenap> lifeless: Yeah, that sounds sane.
<bigjools> jtv: well publish-distro does do commits
<jtv> Yes, just not in its huge loops.
<bigjools> after a-f runs
<bigjools> it needs to mark stuff as publishe
<bigjools> d
<lifeless> allenap: the goal is to be able to do a range query on BugMessage and then get just the Messages we want
<jtv> bigjools: I guess that happens all the way at the end?
<lifeless> allenap: which will also let us move to ajax population of pages like bug 1
<_mup_> Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Invalid by compscibuntu-bugs> <EasyPeasy Overview:Invalid by ramvi> <GNOME Screensaver:Won't Fix> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <The Linux OS Project:In Progress> <metacity:In Progress> <OpenOffice:In Progress by lh-maviya> <Tabuntu:Invalid by tinarussell> <Tivion:Invalid by shakaran> <Tv-Player:New> <Ubuntu:I
<allenap> lifeless: Judging by Bug._indexed_messages, that should work well.
<lifeless> allenap: indeed; I wrote Bug._indexed_messages in october or so as part of optimising within the prior schema
<allenap> lifeless: Like load-on-scroll?
<allenap> lifeless: Yeah, it looked like one of yours :)
<lifeless> allenap: hah! I'll take that as a complement.
<lifeless> allenap: and yes, load on scroll
<jtv> bigjools: if we did the actual work on the slave but the marking-as-published on the master, we'd risk marking a slightly newer version as published than we actually published.  Is that a problem?
<allenap> Awesome.
<lifeless> allenap: should be easy if we can get one message + adjacent actions pretty cheaply.
<bigjools> jtv: there's 4 steps in the publisher: 1) write files to pool, 2) domination, 3) generate files for a-f and run it, 4) write release files
<bigjools> jtv: big problem ,yes
<allenap> lifeless: Adjacent actions?
<jtv> bigjools: though strictly speaking, I would expect that that problem already exists to about the same extent
<lifeless> allenap: look at a BugTask:+index page
<lifeless> allenap: it shows action action message action message etc interleaved
<lifeless> allenap: it pretends there is one sequence
<bigjools> jtv: actually, the publishing record would only get written by  this process anyway at this point in its lifecycle
<allenap> lifeless: Ah, yes, I wrote a lot of that ;) I was having an association fail on the word "actions".
<allenap> Well, not a lot, some.
<jtv> bigjools: assuming there's no overlapping runs, I guess it'd only be a problem if there were a few last changes while the script was running, and then nothing before the next run (so it'd think it was already published or something).
<allenap> lifeless: Are you going to work on this, or are you softening me up to do it?
<flacoste> lifeless: i'm free whenever you are
<lifeless> allenap: I'm helping on the oops/performance aspect
<lifeless> allenap: future stuff will be feature cards I suspect, unless you were to have a sudden fit of zomg I want to do this
<lifeless> flacoste: voice?
<flacoste> lifeless: sure
<allenap> lifeless: Okay, so you'll get the index into the database, and the get-adjacent-actions and load-on-scroll stuff is up to others?
<flacoste> lifeless: skype me
<lifeless> allenap: yeah, I don't have long enough timeslices to do larger stuff
<allenap> lifeless: That's cool, I just wanted to be sure what's expected of me.
<lifeless> be excellent
<lifeless> thats it
<allenap> Heh :)
<bigjools> jtv: so it marks everything published before a-f runs
<bigjools> there's the long transaction
<jtv> bigjools: it may help my understanding if I see what that entailsâ¦ do you happen to know where that is done?
<bigjools> jtv: look at lib/lp/archivepublisher/publishing.py
 * jtv looks at lib/lp/archivepublisher/publishing.py
<bigjools> and scripts/publish-distro.py which is what calls it
<bigjools> C_doFTPArchive is for Ubuntu, C_writeIndexes is for PPAs
<jtv> Ah, that helps
<jtv> So the marking-as-publishedâ¦ what do I look for?
<bigjools> ArchivePublisherBase.publish()
<bigjools> follow it down to there from distroseries.publish()
<jtv> ah cool
<jtv> But I think the script commits soon after that happens anyway.
<bigjools> the latter being called from A_publish()
<jtv> The problem is in C_doFTPArchive
<jtv> (I think)
<bigjools> yes
<bigjools> each stage is wrapped it try_and_commit()
<bigjools> in*
<jtv> ahh got it: setPublished
<jtv> And we want to publish a state that's no older than that datepublished timestamp, right?
<bigjools> jtv: so my concern is that we only see half of the records that were just set published if C_doFTPArchive is using a different store
<bigjools> or maybe even none
<bigjools> but that's extreme
<jtv> I wonder if we could set "now minus replication lag" as the publication date.
<bac> hi deryck, you still around?
<bigjools> the date is not a concern
<bigjools> it's the inconsistency
<jtv> bigjools: would that change though if we otherwise ran everything on the slave store?
<bigjools> because what can happen is that we write the pool file, miss it in a-f and then write the release file with it in
<bigjools> that would be quite disastrous
<jtv> What's a-f stand for by the way?
<bigjools> Apt FTPArchive
<bigjools> in that case we'd end up with checksums that are incorrect
<jtv> You're saying the whole pool file might be lost?  Or an individual package that's in there?
<bigjools> no
<bigjools> a-f writes the indexes - with replication lag it could miss something
<bigjools> then if that something replicates before we get to stage "D" where it writes the Release file, the Release file will be wrong compared to the index
<bigjools> I think
<jtv> Well if it's that complicated I probably can't afford to mess with it anyway.  :/
<bigjools> the publisher is *the* most critical part of soyuz, if it goes wrong we can do a lot of damage
<bigjools> which is why I am rather conservative in this area :)
<jtv> If we could move the whole thing minus something small over to the slave, we'd have a consistent view (just a slightly older one) and no worries.  But if there's any sort of read-modify interaction it gets riskier.
<bigjools> yeah
<bigjools> we should go through it sometime and record all the interactions
<jtv> bigjools: would it make sense to do a trial run with a slave-only policy and setPublished disabled?
<bigjools> I dont think we have any tests that do everything at once like that, so we'd need to check the archive state manually
<jtv> Meaning we'd probably miss something?
<bigjools> possibly but if you point an apt client at the archive it would soon belly-ache
<bigjools> we should get wgrant to comment on this
<wallyworld_> thumper: StevenK: we having standup?
<wallyworld_> thumper: i'm talking, jsut a sec
<wallyworld_> thumper: you ok, my sound backend bad
<wallyworld_> fixing
<thumper> wallyworld_, StevenK: FYI https://code.launchpad.net/~thumper/launchpad/refactor-lazrjs-text-widgets/+merge/47634
<StevenK> thumper: I can hear you too
<wallyworld_> thumper:  https://code.edge.launchpad.net/~wallyworld/launchpad/recipe-find-related-branches/+merge/47367
<huwshimi> Ugh, so my laptop has decided that when I'm logged in I should only be able to type numbers :(
<StevenK> Yet, you're typing not numbers. :-)
<huwshimi> StevenK: I am using my netbook to whinge about it
<StevenK> Heh
<wallyworld_> thumper: https://pastebin.canonical.com/42451/
<wgrant> bigjools: Hi
<bigjools> morning wgrant
<wgrant> bigjools: This whole discussion makes me cry.
<wgrant> Why do we want to move it to the slave?
<bigjools> wgrant: no doubt
<benji> huwshimi: numlock?
<huwshimi> benji: Thanks, I just figured that out right then.
<wgrant> We *could* do it, and things would remain consistent, but some things would show as published when they weren't.
<StevenK> thumper: I've been trying!
<bigjools> wgrant: jtv has been investigating but he wants to get rid of a long-running transaction
<wgrant> And the latency would be two hours.
<huwshimi> benji: They could have at least labelled it :)
<bigjools> wgrant: yes that's exactly my concern in addition to incorrect indices vs release files
<wgrant> bigjools: We may have extra stuff on disk, but we may also be removing stuff from disk later that is still referenced by the indices.
<wgrant> So no, we cannot sensibly use the slave.
<wgrant> However, yes, we can remove the long-running transactions by making the publisher not take 300 years.
<wgrant> But that is a fair bit of work.
<lifeless> we could add a ro connection to the master
<bigjools> about 300 years
<wgrant> lifeless: Is it only R/W transactions that are the problem?
<lifeless> wgrant: no
<wgrant> That's what I thought.
<lifeless> wgrant: but they are a bigger problem than r/o transactions
<lifeless> r/o transactions prevent index and row gc (mvcc chaff)
<lifeless> r/w transactions cause related row exclusive locks
<wgrant> Right.
<lifeless> but its the actual *changes* in r/w transactions that matter
 * bigjools gets tests working with a version column of type debversion \\o/
<lifeless> length is just a (poor) proxy for predicting problems.
<wgrant> There should be none. But it would be nice to verify that.
<wgrant> bigjools: Yay!
<bigjools> now, to optimise the queries
<StevenK> thumper: So I use Mumble on the old laptop the audio is choppy and I can talk, if I use the new laptop the audio is excellent and I can't talk
<thumper> heh
<bigjools> wgrant: btw, the 2-builds-per-builder happened again today
<wgrant> bigjools: Same builder?
<bigjools> no,  after talking to lamont it turns out that it happens when he kills a long-running stuck build
<wgrant> I noticed 6 or so builders earlier disabled with strange messages.
<wgrant> But they were all happy once I re-enabled them.
<bigjools> because we rely on aborting nonvirt builders
<lifeless> flacoste: poolie: draft sent to you
<bigjools> so I am going to change to code disable the builder it we see it aborting
<wgrant> lifeless: Do you know what's going on with sinzui's mailing list QA?
<wgrant> bigjools: Why?
<bigjools> abort does not work on nonvirts
<wgrant> ABORTING will drop to ABORTED, modulo a slave bug which doesn't properly kill sbuild because of a missing trap.
<bigjools> exactly
<bigjools> so until that's fixed...
<lifeless> wgrant: yes
<lifeless> wgrant: can I brain dump and have you chase?
<wgrant> lifeless: That is what I was hoping for.
<lifeless> wgrant: ok so lp prod configs was broken for qastaging mailman
<lifeless> the qa port is 9097
<wgrant> bigjools: Until that's fixed, buildd-manager will ignore the builder and there is a nagios check to tell lamont to fix it.
<wgrant> Right.
<lifeless> it said 8097
<lifeless> there is a branch to fix that
<wgrant> I believe that got reviewed and landed.
<lifeless> so if it has
<wgrant> It should be good?
<lifeless> then it should be deployed and the mailman log should be calling into the api with the newer code
<wgrant> It doesn't seem to be landed. I might fix that.
<lifeless> remaining steps
<wgrant> I've only run mailman locally a couple of times, so I'm not 100% on how exactly it works, but I guess I'll work it out.
<wgrant> lifeless: qastaging should automatically update configs with its usual update, right?
<lifeless> wgrant: yes
<lifeless> wgrant: so, 1) get the right port out
<lifeless> 2) make sure its querying members (via the logs)
<lifeless> 3) profit
<wgrant> Great, thanks.
<wgrant> If all goes well, we can deploy in a couple of hours :)
<StevenK> lifeless: So, the query Storm is creating is: SELECT SourcePackageRecipe.build_daily, SourcePackageRecipe.daily_build_archive, SourcePackageRecipe.date_created, SourcePackageRecipe.date_last_modified, SourcePackageRecipe.description, SourcePackageRecipe.id, SourcePackageRecipe.is_stale, SourcePackageRecipe.name, SourcePackageRecipe.owner, SourcePackageRecipe.registrant FROM SourcePackageRecipe LEFT JOIN SourcePackageRecipeBuild ON SourcePac
<StevenK> Hm. That was a bit longer than it looked in the terminal
<wgrant> bigjools: We haven't purged non-main indices for existing PPAs yet, have we?
<lifeless> wgrant: new ppas won't have them at all
<wgrant> I know.
<wgrant> It broke a few things :)
<wgrant> But they're all fixed now.
<wgrant> But we were also going to manually remove the old ones.
<flacoste> lifeless: replied
<lifeless> thanks
<lifeless> poolie: hi
<bigjools> wgrant: no
<bigjools> wgrant: go ahead and clean up, then we can re-enable the daily job
<StevenK> lifeless: O hai. I have added a recipe with no builds and dropped the query and it returns 0 rows
<lifeless> StevenK: you need to paste the query without getting cut off :)
<StevenK> lifeless: http://pastebin.ubuntu.com/559270/ ; I've taken what LP_DEBUG_SQL gave me, replaced the %s's and dropped most of SourcePackageRecipe's columns from the SELECT
<jtv> hi wgrant!
<StevenK> If I drop the extra JOINs, it returns a row
<wgrant> Morning jtv.
<jtv> wgrant: bigjools & I were looking at the long transactions in the archive publisher earlier.
<wgrant> jtv: So I saw.
<jtv> You're everywhere.
<wgrant> We cannot sanely move it onto a slave.
<jtv> I was wondering whether we could move essentially the whole process over to a slave without risking inconsistencies.
<wgrant> We need to make it take less insanely long.
<jtv> Ah.
<jtv> That answers that.
<bigjools> jtv: wgrant made some more scary points that I'd not mentioned in addition to my scary ones
<jtv> Oh good
<jtv> that sounds like fun.
<jtv> Making it take less insanely long looks somewhat doable, but I'll need a way to test.
<jtv> It's FTPArchiveHandler.run that takes the bulk of the time, right?
<wgrant> There are two things that take forever: file list generation, and a-f itself.
<wgrant> The latter can be parallelised.
<lifeless> StevenK: I'm fidddling
<bigjools> wgrant: I'd like to see a-f and NMAF in a race
<lifeless> StevenK: trivially doing left joins all over works, but that can get inefficient
<jtv> wgrant: and the former looks like it may be a typical naÃ¯ve-fetch-inside-loop pattern.
<bigjools> I think it would be close
<jtv> It's _almost_ a simple prejoin but there's an interaction with slicing to watch out for.
<wgrant> bigjools: a-f is hundreds of times slower.
<wgrant> Er.
<wgrant> NMAF is.
<StevenK> NMAF is *slower*?
<wgrant> jtv: Possibly. But the queries themselves are bad.
<wgrant> StevenK: Yes, it issues many many many more queries.
<jtv> wgrant: the pattern, or some of the individual ones?  If you've got a list of known troublemakers, that'd help me see.
<wgrant> jtv: I don't recall exactly. But I believe it's fairly efficient query-count-wise, but the queries are not quick.
<wgrant> It's been more than a year since I seriously tried optimising this phase of the publisher.
<wgrant> I got a *long* way, but it was sort of full of hacks.
<wgrant> Down to 3-4 seconds for each index file, on NMAF, for 1.5x primary's size.
<wgrant> cprov also optimised the a-f file list code a lot just before he left.
<jtv> My may suspect based on following the arrows in the code was FTPArchiveHandler.publishFileLists.
<jtv> That contains some loops over all source/binary packages, I believe.
<wgrant> Yes, but that uses the views.
<bigjools> which are evil
 * jtv crosses himself
<wgrant> It should not do any queries.
<jtv> Luckily I made merit by visiting the temple: saw a working Difference Engine 2 yesterday.
 * bigjools looks for garlic and silver
<wgrant> Until right at the end when I do the disabled chekc.
<wgrant> (which is only once per pocket-das)
<jtv> What's a pocket-das?
<wgrant> (PackagePublishingPocket, DistroArchSeries)
<StevenK> hardy-i386-RELEASE
<StevenK> For instance
<wgrant> How many cores does cocoplum have?
<bigjools> StevenK, wgrant: I am putting my debversion stuff on DF, please to not be touching for a bit
<wgrant> bigjools: k
<bigjools> unless you guys are using it in which case I can wait
<StevenK> wgrant: 4
<wgrant> :(
<lifeless> StevenK: SourcePackageRecipeBuild JOIN PackageBuild ON PackageBuild.id = SourcePackageRecipeBuild.package_build JOIN BuildFarmJob ON BuildFarmJob.id = PackageBuild.build_farm_job right join SourcePackageRecipe  ON SourcePackageRecipeBuild.recipe = SourcePackageRecipe.id
<lifeless> StevenK: put that in from 'FROM' ... 'WHERE'
<jtv> wgrant: it looked to me as if, unless we mess with the code's structure which I'm reluctant to do, it should do a bunch of batched prefetch queries in each iteration of its loop-tuner callback.  If it's using views, maybe that's worth eliminating as well.
<wgrant> jtv: Because it's using views, it's already entirely prefetched.
<wgrant> All you can do is optimise the view.
<jtv> Actually, prefetching may be what's slowing it down then.
<wgrant> However, file list generation is only a couple of minutes.
<wgrant> Running a-f takes 15 or so.
<StevenK> lifeless: That looks good. How do I tell Storm that?
<lifeless> RightJoin
<StevenK> Instead of left?
<lifeless> StevenK: works like LeftJoin in terms of function calls
<lifeless> StevenK: yes; same parameters, same translation
<lifeless> StevenK: play with it a little to get the invocation you need
<lifeless> I'm just checking the query plan on staging
<lifeless> 96ms
<lifeless> so fast
<lifeless> StevenK: you'll also want DISTINCT, because you only want one row per sourcepackagerecipe
<lifeless> StevenK: as written, on staging, we can get
<lifeless>  id |    name     |  owner
<lifeless> ----+-------------+---------
<lifeless>  15 | awn-testing | 1382524
<lifeless> (3 rows)
<StevenK> lifeless: I'm still distilling your query into Storm. *Then* I'll worry about distinct
<lifeless> StevenK: I'd do it for you, but known how this works will help you
<jtv> wgrant: drat.  So to speed the code up significantly we'd have to run parallel apt-ftparchive instances on separate file lists?
<wgrant> jtv: Yes. We already have separate file lists, so it's fairly easy.
<StevenK> lifeless: Storm has SELECT ... FROM SourcePackageRecipe; whereas your query has SELECT ... FROM SourcePackageRecipeBuild; is that pertinent?
<jtv> wgrant: and thenâ¦ just concatenate the outputs?
<wgrant> jtv: No. Each file list results in one index.
<wgrant> If you look at the log, you'll see it generates one index for (maverick-release, source, main), another for (maverick-release, i386, universe), etc.
<wgrant> Each of those is big.
<wgrant> And each of those can be done in parallel.
<lifeless> StevenK: yes
<lifeless> StevenK: or probably yes, show me thje FROM..WHERE bit ?
<StevenK> lifeless: http://pastebin.ubuntu.com/559277/
<jtv> wgrant: ah so parallel runs per architecture, plus one for source, and they should all overlap fairly well.
<wgrant> jtv: Yes.
<jtv> The hard part would probably managing the parallel processes.
<wgrant> Yes. And that's not terribly hard.
<wgrant> In the run I'm looking at, running a-f takes 16 minutes. File list generation takes 1.5.
<StevenK> lifeless: I suspect you got distracted?
<lifeless> StevenK: born distracted
<StevenK> Duh
<lifeless> StevenK: ok
<lifeless> so that thing you pasted means
<lifeless> 'select the things joined together where there are 0 or many sourcepackagerecipes'
<lifeless> what you want is
<lifeless> 'select source package recipes where there are 0 or many (things joined together)'
<lifeless> right joins allow NULL on the left hand side
<poolie> lifeless, spm, as a specific next step on bug 701329, can we get haproxy logs?
<_mup_> Bug #701329: error: [Errno 32] Broken pipe <oops> <Launchpad itself:Triaged> <loggerhead:Triaged> < https://launchpad.net/bugs/701329 >
<lifeless> that is, they include 'every row on the right hand side'
<lifeless> poolie: can we not just catch the exception ?
<poolie> or, try to titrate them to a level where they tell us about interesting errors without flooding the system
<lifeless> poolie: what are you trying to determine
<poolie> why haproxy closed the connection
<poolie> and from which ip it came
<lifeless> poolie: can you back it out a step higher
<lifeless> I don't understand why this isn't a fairly direct code change
<poolie> so, to close this particular bug, we can, say
<poolie> just make it not oops when the connection is abruptly closed?
<poolie> perhaps it should log a warning-type error
<poolie> that seems pretty easy
<lifeless> none of our other servers log on this
<lifeless> *or*
<poolie> it makes me wonder if we are papering over a problem
<lifeless> none of our other servers are experiencing it
<poolie> so i'd like to see a bit more data about why the connection is being dropped
<poolie> haproxy is the obvious place to look for that
<poolie> and, i think it is a bit weird to have a production service where logging is entirely disabled
<poolie> (perhaps "it's weird" is not a reason to change it but i think it's reasonable to ask)
<lifeless> I thought you were told that it was a volume problem ?
<lifeless> haproxy being a bit of a firehose
<lifeless> anyhow
<lifeless> if you're working on this bug
<lifeless> thats cool
<poolie> lifeless, i'd guess that zope etc don't traceback on epipe, and i agree that in general loggerhead shouldn't
<lifeless> uhm, what can I do to hepl.
<lifeless> firstly, have you looked at some of the OOPSes?
<poolie> yes
<poolie> i'd like to turn logging >=WARNING
<poolie> and see if that is in fact a firehose
<poolie> if it is, that's a bit alarming, and i'd like to just see what it's logging
<poolie> then we can turn it off again
<poolie> is that ok?
<lifeless> I've no objections in principle, though the losas generally make informed decisions about this sort of thing :)
<poolie> spm, do you mind trying that?
<spm> we start with "no" and negotiate from there.
<poolie> sigh
<spm> poolie: nope, worth a shot. :-)
<poolie> thanks
 * StevenK was waiting for "Depends. Do you have cake?"
<poolie> the haproxy manual says you can set the log leevl
<spm> StevenK: it's poolie. he lives closer than wgrant, and rides a motorbike ie Hells Angel in training. I've learnt to be nice to him.
<lifeless> poolie: so
<lifeless> poolie: looking at  https://lp-oops.canonical.com/oops.py/?oopsid=1836CBA1
<lifeless> poolie: there are several things that make this hard to analyze
<lifeless> poolie: if I was working on it, I would fix those first
<lifeless> poolie: firstly there is no total time recorded
 * StevenK suspects that poolie is shinier, due to lifeless completly context switching and goes afk for ten or so
<lifeless> poolie: nor any timeline (which we could use to track xmlrpc / bzr call overhead if we wanted to)
<poolie> i've learned to front-load anything that may need SA action
<lifeless> StevenK: sorry, I thought I answered you?
<lifeless> StevenK: @10:44
<lifeless> poolie: if we had timing data, we could assess whether this is really a timeout or not
<lifeless> poolie: in fact, we could add a thing to say 'if the time is > some N, convert to a timeout'
<poolie> yup
<lifeless> we probably need to expose an API call to ask for the timeout
<poolie> but if haproxy is saying "request blah blah timeout, killed"
<lifeless> e.g. 'what should my timeout be please?'
<poolie> that seems easier
<lifeless> poolie: I worry that we're going to need a lot of correlation to match up all 15K a day
<lifeless> poolie: gathering data at source seems more useful (to me)
<lifeless> poolie: anyhow, thats just what I'd do if I was working on it
<lifeless> however
<lifeless> note that haproxy probably won't tell us
<lifeless> if this is coming in behind haproxy
<lifeless> spm: how often does nagios query ?
<spm> i think it's up to 3 times every 10 minutes
<poolie> lifeless, i don't want to match up all of them, i just want to know if haproxy thinks its client is dropping the connection, or if it is dropping the connection itself
<poolie> or if it's oblivious
<poolie> it shouldn't be too hard to answer
<lifeless> sure
<spm> but probably easier to look in the logs and count
<lifeless> I am open to many ways to figure out whats up
 * thumper is busy writing real documentation
#launchpad-dev 2011-01-28
<lifeless> spm: so thats about 1/2 what we see
<lifeless> at most
<lifeless> (even allowing for 2 instances)
<wgrant> lifeless: The deloyment report is happy now, except for 12272 which has been rolled back, but is still shown as not being OK.
<wgrant> lifeless: Halp?
<thumper> hmm...
<thumper> IArchive says description is requried
<spm> lifeless: huh, not even that. looking at prod-1 from the 12th of Jan, ~ 5-8 nagios pings are logged on guava per *hour*.
<thumper> Archive.description says it isn't required
<spm> I wonder. we may have multiple redundant checks coming from elsewhere....
<poolie> wa-hey, a kanban!
<lifeless> wgrant: gmb filed bugs about rollback not being set properly
<lifeless> wgrant: you may need to ignore it
<wgrant> lifeless: Do I have your blessing to request a deploy of 12274, despite qa-tagger's protests?
<lifeless> poolie: https://bugs.launchpad.net/launchpad/+bug/708961
<_mup_> Bug #708961: loggerhead oops reports are missing diagnostic data <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/708961 >
<lifeless> wgrant: let me eyeball
<poolie> anyhow, spm, whenever you get a moment (it's not urgent) i would like to get a bucket-ful of the firehose
<lifeless> wgrant: yes
<poolie> then we can turn it off again
<lifeless> wgrant: there is a partially complete request you can extend to save some time
<StevenK> I don't think we can QA r12272.
<lifeless> wgrant: just grab the new bug numbers etc
<spm> poolie: yeah will do
<lifeless> StevenK: its rolledback
<wgrant> lifeless: Yup.
<StevenK> Right
<lifeless> StevenK: read 12274
<StevenK> lifeless: Yes, but the QA tagger thinks we're blocked on 72
<wgrant> lifeless: Just the bugs listed on the deployment report?
<lifeless> StevenK: so, did you see my reply to your query; you were not less shiny, you were answered.
<spm> lifeless: poolie: we do have other checks, but even that only takes it to around 15 an hour max.
<StevenK> lifeless: Sorry, I thought you were going to explain more
<lifeless> wgrant: for rev in revs, copy Bug 123 -> Bug:123 in the deploy request
<_mup_> Bug #123: There's no direct way to see the project info when translating it <lp-translations> <Launchpad itself:Fix Released> < https://launchpad.net/bugs/123 >
<wgrant> Right.
<lifeless> wgrant: where revs starts at the one after the one I'd gotten up to
<lifeless> StevenK: you need to shuffle order of parameters in your python code
<lifeless> StevenK: to control how things are output
<lifeless> StevenK: do you understand what right outer does?
<StevenK> lifeless: If I follow your query, you're selecting from SPRB, which storm then returns
<lifeless> StevenK: no, I'm not
<StevenK> SELECT SourcePackageRecipe.name, SourcePackageRecipe.owner FROM
<StevenK> SourcePackageRecipeBuild
<spm> poolie: actually - I wonder. is this something we could possibly trial on staging at all? I don't believe we have a haproxy there (??). Just thinking we could start there and see if we can dupe, or get sufficient WTF info. and then look into prod?
<lifeless> StevenK: the *entire* thing between FROM and WHERE is what you select from
<lifeless> StevenK: its a composite table
<lifeless> StevenK: store.using(table description) is how you tell storm about that, which you already do
<poolie> if we have an haproxy there that would be a great place to do it
<StevenK> Right.
<lifeless> StevenK: the .find((Things to select), constraints) is where you tell it *what* to pull out of the composite
<StevenK> lifeless: Out of interest, I can't find any other uses of RightJoin
<lifeless> StevenK: doesn't surprise me
<lifeless> it is materializing a view
<lifeless> but it will scale quite a bit I think
<lifeless> we'll have to revisit it when we're in the 5K mark, I'd say
<lifeless> however
<StevenK> Now the query looks good
<StevenK> Well, it's ordered differently to yours
<lifeless> http://pastebin.com/Vf6Manng
<poolie> searching for bugs assigned to me through the api seems to consistently time out
<poolie> should i file or is this probably known?
<poolie> ah, it is
<lifeless> StevenK: so, fiddling suggests left outers on all the joins will be better here
<lifeless> StevenK: but can I ask
<lifeless> StevenK: why do we need all these different tables ?
<lifeless> poolie: which bug ?
<poolie> the oops finder says https://bugs.launchpad.net/launchpad/+bug/638924
<_mup_> Bug #638924: Milestone:+index timeouts with many bugs <lp-registry> <pg83> <qa-ok> <timeout> <Launchpad itself:Fix Released by edwin-grubbs> < https://launchpad.net/bugs/638924 >
<poolie> and it looks similar, but it's not fixed
<lifeless> the oops finder?
<poolie> https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1854EA3
<lifeless> if you mean the lp-oops ui, its useless
<lifeless> (for our needs)
<StevenK> lifeless: Because BuildFarmJob tells us when the build was created, which links from PackageBuild, which links from SourcePackageRecipeBuild
<lifeless> awesome data collection, brilliant rendering. Terrible heuristic about what bug
<lifeless> StevenK: and we always have all three tables, or never all three, right ?
<poolie> ah, is it
<poolie> in this case it's not a bad guess
<lifeless> poolie: I've been ranting since day one :)
<StevenK> lifeless: Right, we need a SPRecipeBuild to have the other two
<poolie> i don't know if this indicates the same bug was not totally fixed, or if it needs a different bug
<poolie> s//fix
<lifeless> poolie: no, its a terrible guess. Its only chance that it seems relevant to you
<lifeless> the page id is Person:EntryResource:searchTasks
<lifeless> go to https://bugs.launchpad.net/launchpad-project/+bugs?field.tag=timeout
<lifeless> and search for searchTasks
<lifeless> poolie:  https://bugs.launchpad.net/launchpad/+bug/669766
<_mup_> Bug #669766: Person:+bugs timeouts <dba> <lp-bugs> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/669766 >
<poolie> seems similar
<lifeless> same query
<lifeless> the unions in there are a red flag to me
<poolie> right, it looks a bit insane
<lifeless> StevenK: so if we have all three, why are they separate tables?
<lifeless> StevenK: its a great way to make sql slow
<lifeless> StevenK: anyhow, on balance, I've tried a bunch of query plans
<lifeless> StevenK: go with LEFT OUTER...LEFT OUTER...LEFT OUTER
<lifeless> that this whole schema needs fixing is a separate problem.
<lifeless> hmm, I'm getting biting. Lunchtime, please excuse me folks
<poolie> wow i don't know what changed but apis are distinctly faster than they used to be
<wgrant> Which APIs?
<poolie> reading bugs
<poolie> actually i'm not really sure if that's true; i think what this process is doing is not comparable
<poolie> still actually several seconds per bug :(
<wgrant> :D
 * wgrant fixes the db-devel merge conflict.
<thumper> I'm finally done with the text widget refactoring
<thumper> \o/
<thumper> the diff is a bit bigger than originally expected
<thumper> good thing that flacoste has already committed to reviewing it
<thumper> otherwise I'm sure it'd get rejected due to size
<wgrant> haha.
<wgrant> What does it change?
<lifeless> everything!
<wgrant> Surely not.
<thumper> https://code.launchpad.net/~thumper/launchpad/refactor-lazrjs-text-widgets/+merge/47634
<thumper> wgrant: look at the documentation in the diff
<thumper> wgrant: it covers most of the interesting stuff
<wgrant> Yay.
<wgrant> Also ouch.
<thumper> wgrant: why ouch?
<StevenK> thumper: That moves an awful lot of code around, but nice work!
<thumper> StevenK: thanks
<wgrant> thumper: The diff size.
<thumper> the simple enum chooser needs some similar treatment
<thumper> but I'll get to that at some stage
<thumper> wgrant: yeah, it did kinda grow on me
<thumper> wgrant: when flacoste looked at it last night, it was only 850 or so
<wgrant> Heh.
<thumper> but I added lots of documentation
<StevenK> wgrant: Did I tell you how large the bpb-current-component diff got?
<wgrant> StevenK: 4.5kish?
<StevenK> wgrant: 4,800
<wgrant> Ow.
<StevenK> Not quite as bad as the soyuz enums branch, but as close as I've gotten
<poolie> hi
<poolie> i'm getting "sorry there was a problem" talking to the api server
<lifeless> 502?
<lifeless> is there an OOPS?
<poolie> yes 502
<poolie> no oops
<lifeless> one off or repeatedly?
<poolie> at least twice
<poolie> very long timout
<wgrant> edge or lpnet?
<wgrant> When?
<poolie> just now; lpnet
<poolie> curl -v  https://api.launchpad.net/devel/ -H "Accept: application/vd.sun.wadl+xml"
<poolie> pretty sure i ran that command successfully the other day
<poolie> ok, this time it worked, after a _long_ delay
<poolie> between when i pasted the curl line and now
<StevenK> Last revision deployed to QAStaging is 12274. There are no revisions waiting in the queue.
<StevenK> \o/
<poolie> i'll look for or file a bug
<poolie> maybe bug 607961
<_mup_> Bug #607961: wadl generation timeout? <lp-foundations> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/607961 >
<poolie> yes, that's it
<lifeless> wgrant: feel like sending a yee-ha mail to the list ? :)
<wgrant> lifeless: Just finishing closing old bugs.
<wgrant> Did someone just put loggerhead into launchpad-project?
<lifeless> yes
<wgrant> :(
<lifeless> why so sad?
<thumper> wow, we are up to date with production
<thumper> awesome
<wgrant> Need to fix my bugmail rules.
<lifeless> thumper: cunning plan working :)
<wgrant> 237 critical bugs!
<wgrant> We are down to what we were at at the start of last week!
<wgrant> Yay...
 * spm files some more
<wgrant> lifeless: Do all these Bugs story-* tags need to be official?
<lifeless> wgrant: how do you mean?
<wgrant> lifeless: launchpadtags portlet
<wgrant> Ugh.
<wgrant> launchpad's tags portlet is rather cluttered.
<lifeless> wgrant: so, it is, but thats all not just official
<wgrant> Partly because there are lots of very long story-* official tags from back when Bugs was separate.
<wgrant> isn't it?
<wgrant> I thought it was now.
<lifeless> poolie: hi
<lifeless> you marked bug 492614 as new, but I'm unclear why
<_mup_> Bug #492614: bzr serve --http cannot find module loggerhead.util <loggerhead:New> < https://launchpad.net/bugs/492614 >
<poolie> wow, benji's thing in https://launchpad.net/+feature-info is really classy
<poolie> lifeless, i marked it new because it seems possible someone could solve it since he gave more data
<poolie> i may be wrong
<wgrant> poolie: Yeah, I saw that while checking what to close.
<wgrant> It is really nice.
<poolie> at the moment i use 'incomplete' as 'should be killed by the expiry-bot if no more data is provided'
<lifeless> poolie: I'm driving NEW to zero
<poolie> so i see!
<poolie> feel free to take a guess at whether it's invalid or confirmed
<poolie> i would suspect something weird in his environment therefore invalid
<lifeless> jelmer has it assigned to him already
<lifeless> hmm
<lifeless> no, another bug has jelmer
<lifeless> ok, 0 untriaged bugs
<lifeless> phew
<thumper> jelmer: are you working on bug 698032?  I was told you had a solution.
<_mup_> Bug #698032: recipe build for removed recipe triggers uploader exception <oops> <recipe> <soyuz-upload> <Launchpad itself:Triaged> < https://launchpad.net/bugs/698032 >
<lifeless> === Top 10 Time Out Counts by Page ID ===
<lifeless>     Hard / Soft  Page ID
<lifeless>     1398 /  424  BugTask:+index
<lifeless>       34 /   83  DistroSeries:+queue
<lifeless>       18 /   44  Branch:+index
<lifeless>       17 /  356  Distribution:+bugtarget-portlet-bugfilters-stats
<lifeless>       16 /  236  POFile:+translate
<lifeless>       12 /   14  NullBugTask:+index
<lifeless>       12 /   13  Cve:+index
<lifeless> going to be fun on monday
<poolie> thumper i think jelmer is on holiday this week
<poolie> how do i create a team with 'canonical' in the name?
<thumper> poolie: ok, np, just wanting a status update
<thumper> poolie: it isn't urgent
<thumper> poolie: ask a losa to rename a team once you've made it
<wgrant> poolie: There's a fix for that on db-devel. But now you have to get LOSAs to SQL it up.
<lifeless> poolie: hi
<lifeless> poolie: if you're going to set the status of new bugs in Launchpad, can you please follow the bug triage guidelines?
<poolie> wgrant, by asking them to make a new one, or to rename one?
<wgrant> poolie: Rename.
<poolie> lifeless, i'm trying to; in what way am i not?
<lifeless> we don't use confirmed
<lifeless> confirmed is a state an end user might set it to
<poolie> i know
<lifeless> but its a noop for our process
<poolie> i'm setting them to that state specifically because i'm not doing your tirage
<poolie> *triage
<lifeless> poolie: please don't then
<poolie> look, the guidelines don't say "don't use confirmed"
<lifeless> the guidelines are for devs
<poolie> but, i'm happy to not use it if you want
<poolie> would you prefer i triage them, or leave them new?
<lifeless> triaging is great
<poolie> ok
<lifeless> that would be ideal
<poolie> np
<lifeless> I'm sorry if something I said there came across negatively
<poolie> it's fine
<lifeless> confirmed is not a very useful status in the bug tracker IMO - its there mainly to support the Ubuntu 'lets get unskilled labour en masse' meme
<poolie> it's just that i was using Confirmed specifically because i didn't want to contradict your triage process
<lifeless> confirmed doesn't contradict it, it just doesn't help it
<poolie> i certainly agree about that
<poolie> no, i was concerned that i would make it say triaged/high when you didn't think it would be high etc
<lifeless> if there is disagreement, it can be altered beyond the first persons assessment
<lifeless> the broad rule of thumb is critical-for-emergencies, high for next-6-months, low for everythingelse
<poolie> great, i shall do that from here on
<lifeless> cool
<wgrant> Is there any point keeping bug #705841 open?
<_mup_> Bug #705841: bzr branch error: Error -3 while decompressing: incorrect data check <Launchpad itself:Confirmed> < https://launchpad.net/bugs/705841 >
<lifeless> wgrant: I've commented and closed it
<poolie> i think there's another bug asking for a better message
<thumper> does anyone know if it is possible to annotate an existing exception class with a webservice_error code?
<thumper> i.e. one that is in a separate module
<thumper> like bzr-builder
<thumper> in particular InstructionParseError
<StevenK> subclass it and feed it webservice_error?
<thumper> StevenK: but the exception is being raised in bzr-builder code
<StevenK> thumper: Intercept it and raise the subclassed one?
<poolie> stupid question i guess, but you can't just stick attributes onto it?
<thumper> poolie: possibly
<thumper> poolie: I was just reading the webservice_error code
<thumper> found it
<thumper>  error_status(400)(InstructionParseError)
<thumper> where error_status is in lazr.restful.declarations
<lifeless> wgrant: can you triage bug 338858?
<_mup_> Bug #338858: Buildds do not follow versioned build-dependencies <sbuild> <Launchpad Auto Build System:Confirmed> < https://launchpad.net/bugs/338858 >
<lifeless> huwshimi: bug 336913 might be nice if you're driving the wiki theme stuff to zero
<_mup_> Bug #336913: Marking text as --(strike through)-- has no effect <Launchpad Development Wiki Moin theme:Triaged> < https://launchpad.net/bugs/336913 >
<lifeless> huwshimi: (I don't know if you are)
<wgrant> lifeless: sbuild makes me cry.
<wgrant> But sure.
<lifeless> wgrant: sadly we own the stack.
<lifeless> noone in ubuntu looks at these projects
<lifeless> not by default
<wgrant> I know.
<wgrant> However, I have an evil plan and branch to make us use system sbuild, which almost pushes the Perl onto Ubuntu :)
<lifeless> bug 309645 needs a more stateful eye too
<_mup_> Bug #309645: sbuild gets confused when the dsc version does not match the changelog one <sbuild> <Launchpad Auto Build System:Confirmed> < https://launchpad.net/bugs/309645 >
<persia> wgrant, Were you able to address lamont's outstanding issues with that branch?
<wgrant> The main issue is that it probably needs a couple of lines added to sbuild.
<wgrant> And I moved onto other things before I worked out how to solve that nicely.
<huwshimi> lifeless: I was doing some work on the help wiki and decided to clear out all the bugs that had a status above low. I haven't really touched the dev wiki yet. Hopefully I'll do the same thing for that at some stage.
<lifeless> huwshimi: no worries
<StevenK> Can I instruct bin/test to match tests with a regex?
<wgrant> It should by default.
<wgrant> -t takes a regex, doesn't it?
 * StevenK adds a $ and tries it
<wgrant>     -t TEST, --test=TEST
<wgrant>                         Specify a test filter as a regular expression.
<poolie> lifeless, i don't think bug 703807 should be incomplete
<_mup_> Bug #703807: "easy_install pyOpenSSL" says "error: Unexpected HTML page found at http://launchpad.net/pyopenssl/main/0.11/+download/pyOpenSSL-0.11.tar.gz" <Launchpad itself:Incomplete> <pyOpenSSL:New> < https://launchpad.net/bugs/703807 >
<poolie> it's an intermittent error but it can be reproduced, and the user has given as much as they reasonably can
<poolie> there are multiple dupes
<lifeless> sure
<poolie> sorry, there is a dupe, and i reproduced it myself
<lifeless> from what I could see theres still no evidence of lp itself having the issue
<poolie> rather than what?
<lifeless> setup.py doing something weird perhaps?
<poolie> i could reproduce it using curl
<wgrant> I can reproduce it right now using curl.
<lifeless> ok, then triaged critical
<poolie> i'll do that
<lifeless> if the headers show crap coming from the lp squids
<StevenK> lifeless: You did mean this when you said left outer join, right: http://pastebin.ubuntu.com/559348/ ?
<wgrant> At the moment, http://launchpadlibrarian.net/58498441/pyOpenSSL-0.11.tar.gz gets the right type from nutmeg, but the wrong from banana.
<lifeless> wgrant: check_permission('launchpad.Edit', branch) isn't really a URI call
<wgrant> lifeless: That's another way to bypass the security check.
<wgrant> You get to see the URL if any of these three conditions are satisfied: the URL is None, you can edit the branch, or the URL is not on a blacklisted domain.
<lifeless> ok
<lifeless> like I said
<lifeless> it was a question
<lifeless> man
<lifeless> another day
<lifeless> no code
<wgrant> :(
<lifeless> https://devpad.canonical.com/~lpqateam/lpnet-oops.html
<lifeless> Time Out Counts by Page ID
<lifeless> Hard	Soft	Page ID
<lifeless> 384	1807	Distribution:EntryResource:searchTasks
<lifeless> thats a little alarming this early into the day
<lifeless> or perhaps its just one script ;)
<lifeless> all steve beattie :)
<lifeless> hmm 12200 still
<wgrant> Is there a better way of getting the webapp's root URL than looking directly in the config?
<StevenK> Sigh. SQL sucks
<lifeless> StevenK: yes thats what I meant
<StevenK> Which doesn't actually work anyway
<lifeless> you forgot the distinct
<StevenK> How does a distinct help me when the query returns zero rows?
<lifeless> oh
<lifeless> hang on
<lifeless> you need SPR left join SPRB
<lifeless> because you want all rows in SPR
<lifeless> works for me
<lifeless> 97 rows
<lifeless> 96.751 ms
<StevenK> On staging?
<lifeless> yes
<StevenK> How out of date is that database?
<lifeless> using 2010-11-26 as the date
<lifeless> your query doesn't work
<lifeless> the fixed one (switch the SPRB and SPR right after the FROM) does
<lifeless> SELECT DISTINCT SourcePackageRecipe.build_daily,
<lifeless>                 SourcePackageRecipe.daily_build_archive,
<lifeless>                 SourcePackageRecipe.date_created,
<lifeless>                 SourcePackageRecipe.is_stale,
<lifeless>                 SourcePackageRecipe.name,
<lifeless>                 SourcePackageRecipe.OWNER, SourcePackageRecipe.registrant
<lifeless> FROM SourcePackageRecipe
<lifeless> LEFT JOIN SourcePackageRecipeBuild ON SourcePackageRecipeBuild.recipe = SourcePackageRecipe.id
<lifeless> LEFT JOIN PackageBuild ON PackageBuild.id = SourcePackageRecipeBuild.package_build
<lifeless> LEFT JOIN BuildFarmJob ON BuildFarmJob.id = PackageBuild.build_farm_job
<lifeless> WHERE SourcePackageRecipe.is_stale = TRUE
<lifeless>   AND SourcePackageRecipe.build_daily = TRUE
<lifeless>   AND (SourcePackageRecipeBuild.id IS NULL
<lifeless>        OR BuildFarmJob.date_created < '2010-11-26 22:47');
<lifeless> btw http://sqlformat.appspot.com/ is your friend :)
<lifeless> back soon, grocery shopping time
<StevenK> Hm. How do I shoehorn DISTINCT into Storm?
<StevenK> grep is not being my friend
<wgrant> .config(distinct=True)
<StevenK> wgrant: Ah! Thanks
<stub> I think there is a nicer spelling of that, but I can never remember it :-(
<LPCIBot> Project devel build (399): FAILURE in 4 hr 51 min: https://hudson.wedontsleep.org/job/devel/399/
<LPCIBot> * Launchpad Patch Queue Manager: [r=sinzui][ui=none][bug=687564] Fixed bug #687564 so that the license
<LPCIBot> selection widget on new project does not overlap other content
<LPCIBot> on the page.
<_mup_> Bug #687564: license widget sections overlaps on projectgroup +newproject and projects/+new <bug-1> <javascript> <lazr-js-upgrade> <lp-registry> <regression> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/687564 >
<LPCIBot> * Launchpad Patch Queue Manager: [r=bac,jcsackett][ui=none][bug=613958,681125] Fix recipe build mail.
<wgrant> allenap: Can bug #656823 be closed? I know it's probably not actually done yet, but the revs are all deployed.
<_mup_> Bug #656823: Subscribing to a search lacks a UI <lp-bugs> <qa-ok> <story-subscribe-to-search> <Launchpad itself:Fix Committed by allenap> < https://launchpad.net/bugs/656823 >
<wgrant> OK, now this is interesting.
<wgrant> Hudson and buildbot both have the same spurious-looking error which I haven't seen recently, and there are no relevant changes :(
<StevenK> The Hudson error looks to have confused subunit, too
<wgrant> Well, in both cases something has gone catastrophically wrong.
<wgrant> With the subprocess hanging and then being killed.
 * StevenK has a closer look at Hudson
<wgrant> But they are on different branches.
<wgrant> It is rather unlikely that they would both show up within an hour of each other.
<StevenK> That looks like a deadlock to me
<wgrant> Oh.
<wgrant> Windmill was reenabled this morning.
<wgrant> I bet that's not unrelated.
<wgrant> Oh look.
<wgrant> It was tearing down AppWindmillLayer.
<wgrant> Let's see if it does the same locally.
<wgrant> I've so far had it crash in 6 other ways, but not that way :(
<lifeless> wgrant: bug 192135
<_mup_> Bug #192135: Bug nickname field needs better validation <lp-bugs> <oops> <Launchpad itself:Invalid> < https://launchpad.net/bugs/192135 >
<lifeless> wgrant: does the API allow bug nicknames to be set?
<jelmer> thumper: hi
<jelmer> thumper: with regard to bug 698032, I would just ignore and delete the build if the recipe has disappeared
<_mup_> Bug #698032: recipe build for removed recipe triggers uploader exception <oops> <recipe> <soyuz-upload> <Launchpad itself:Triaged> < https://launchpad.net/bugs/698032 >
<jelmer> we're doing the same thing for binary builds of source uploads
<wgrant> lifeless: Yes, and I thought the validator was OK there.
<wgrant> But it misses one case :(
<lifeless> wgrant: so perhaps invalid is premature ? :)
<wgrant> lifeless: It wasn't until you just prompted me to test further. :(
<lifeless> wgrant: don't you hate attention to detail ? :)
<wgrant> Indeed.
<lifeless> \o/
<wgrant> WTF
<wgrant> Bug #32464
<_mup_> Bug #32464: guess_bugtask() fails on distribution tasks without a source package <lp-bugs> <oops> <Launchpad itself:Triaged> < https://launchpad.net/bugs/32464 >
<wgrant> The function referenced there has a higher WTF level than any part of Soyuz.
 * jelmer is curious
<wgrant> I always wondered what IDistribution.members was for.
<wgrant> It turns out it's for making wild guesses at which bugtask an email is directed at.
<jelmer> what is it used for, just for telling the user in what way they are subscribed?
<wgrant> No, it's for incoming email.
<wgrant> That function is used to determine the bugtask to edit, if you don't explicitly tell it.
<wgrant> I just presumed it failed if there was more than one.
<wgrant> But no, it guesses.
<wgrant> In the wrong order.
<lifeless> wgrant: members is a PublicPersonChoice ?
<wgrant> lifeless: Sounds right.
<lifeless> wtf
<wgrant> I presumed it was used for nothing at all.
<wgrant> But I didn't know this wonderful piece of guesswork existed.
<wgrant> It appears to be the only callsite.
<lifeless> nuke it ?
<lifeless> email ubuntu-dev, then nuke it
<adeuring> good morning
<allenap> wgrant: Yes, I've marked bug #656823 released.
<_mup_> Bug #656823: Subscribing to a search lacks a UI <lp-bugs> <qa-ok> <story-subscribe-to-search> <Launchpad itself:Fix Released by allenap> < https://launchpad.net/bugs/656823 >
<mrevell> Hi
<wgrant> allenap: Thanks.
<gmb> Does anyone know if there's a canonical example of a good way of handling Zope enum values in JS widgets (i.e. where we pull in the form with ++form++) other than just rewriting the form manually?
<gmb> I ask because Zope widgets that use enums have uppercase values, but our API demands that they be standard-caps (e.g. Lifecycle instead of LIFECYCLE).
<Ursinha> good morning
<gmb> Oh, never mind. What you do is make the vocabulary not suck.
<LPCIBot> Project db-devel build (329): FAILURE in 5 hr 43 min: https://hudson.wedontsleep.org/job/db-devel/329/
<deryck> Morning, all.
<jelmer> jkakar: hi
<jelmer> jkakar: how does the categorization in the HTML kanban boards work?
* bac changed the topic of #launchpad-dev to: Launchpad development https:/â/âdev.launchpad.net/â | PQM is open | firefighting: - | On call reviewer: bac | https://code.launchpad.net/launchpad-project/+activereviews
<bac> adeuring: are you reviewing today?
<jkakar> jelmer: Hi!
<jkakar> jelmer: It's based on bug tags that start with 'story-'.  Any bugs that are part of the same story will be grouped together in a lane.
<jelmer> jkakar: ah, thanks
<jkakar> jelmer: np.
<jkakar> jelmer: There's one other convention that's worth noting.
<jkakar> jelmer: The bug states, 'Queued', 'In progress', 'Needs review', etc. are all based on information from Launchpad.  You don't need to do anything special to have a bug in the right place, with the exception of 'Needs testing'.
<jkakar> jelmer: Er, sorry, with the exception of 'Verified'.  In order to move a bug from the 'Needs testing' column to the 'Verified' column you need to add a 'verified' tag to it.
<jelmer> jkakar: Ah - I was wondering about that. I have a bunch of "Fix Committed" bugs that are currently sitting in Needs Testing
<jkakar> jelmer: The columns are based on the process we use in Landscape where a QA step happens after code review... so it might not make sense for your projects if you do things differently.
<jelmer> jkakar: yeah, because of the way bzr development works we don't really need a separate step for that
<jelmer> jkakar: It's only minor overhead to set that tag though. I'm going to give the HTML kanban board a try and evaluate in a week or two. If it works well enough we can always improve it.
<jkakar> jelmer: Cool, I'm curious to know how you find it.
<jkakar> jelmer: Also, if you don't care about the difference between 'Needs testing' and 'Verified' you could just ignore the tag.  I might be nice to add an option to turn those columns into 'Ready to release' or something.
<gmb> deryck: You might know the answer to this: are Windmill tests automatically run as part of our test suite? I'm guessing so given the most recent build failure, but I wanted to check.
<deryck> gmb: yeah, they are now again.
<gmb> deryck: Cool, thanks.
<deryck> I'm also about to start looking into that failure, unless someone else has a fix worked up.
<gmb> deryck: I was going to look into it in a bit, but if you're going to take a look I'm happy for you to do so ;)
<gmb> Though if you need an extra pair of eyes just ping me.
* benji changed the topic of #launchpad-dev to: Launchpad development https:/â/âdev.launchpad.net/â | PQM is open | firefighting: - | On call reviewer: bac, benji | https://code.launchpad.net/launchpad-project/+activereviews
<adeuring> bac: well, I am also CHR today... I focused more on that task. Also, I did not see any review requests  this morning I dared to take. But I'll add myself to the OCR set.
* adeuring changed the topic of #launchpad-dev to: https:/â/âdev.launchpad.net/â | PQM is open | firefighting: - | On call reviewer: bac, benji, adeuring | https://code.launchpad.net/launchpad-project/+activereviews
<bac> adeuring: ok, i was just curious
<maxb> Hi, is the revno shown at the botton of production pages a stable branch revno?
<maxb> Basically I'm trying to establish whether http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/12201 has reached prod yet
 * maxb reopens bug 680733
<_mup_> Bug #680733: broken recipe build gets stuck at top of "5 latest builds" list on recipe page, forever <lp-code> <qa-ok> <recipe> <Launchpad itself:New for thumper> < https://launchpad.net/bugs/680733 >
<gmb> benji, bac: Howdy. Can you take a look at https://code.launchpad.net/~gmb/launchpad/hooky-hooky-bug-699719/+merge/47803 for me?
<benji> gmb: sure
<gmb> Ta
<sinzui> Hello everyone. Hello, power, hello internet, hello small heater at my feet
<deryck> hello sinzui.  glad to hear you have power again.
<deryck> adeuring, abentley -- standup ping
<adeuring> yes
<abentley> deryck, hi.
<bac> ping mrevell
<bac> sinzui: how was that "hand ground" coffee?
<sinzui> It was good. I used a French press (why is it French when it was invented in New Jersy)
<deryck> maybe someone from Belgium living in New Jersey made it
<bac> sinzui: would you buy a New Jersey Press?
<bac> oh, the cachet
<sinzui> Or the inventor was named Frenchie (As in the case of the French dipped sandwich)
<benji> sinzui: the same reason Chinese fortune cookies were invented in a Japanese restaurant in San Francisco
<bac> benji: gullible white people?
<sinzui> benji: yes. Those were Shito fortunes
<benji> sinzui: they've gotten better since then
<sinzui> Ice cream was invented in China...a nation of lactose intolerant people. That does not make sense
<sinzui> leonardr and I had some interesting fortune cookies in Montreal. I like the misreading of "You will have an affair with one of your students."
<mrevell> hi bac
<vila> abentley: ping
<vila> oh, standup, bad time I guess
<benji> gary_poster: yellow/Subscriptions includes a first cut description of the mute UI; I'd say we shouldn't include the "no events" part in the normal subscribe workflow
 * benji wanders away into the right channel.
<gary_poster> :-)
<abentley> vila, pong
<vila> abentley: I cam across https://dev.launchpad.net/Foundations/NewTaskSystem/Requirements in the context of the package importer,
<vila> abentley: AIUI from a related thread in lp-dev, the best candidate at one point was celery+rabbitt, is this still true ?
<abentley> vila, yes, that was the leading candidate, but the project was put on hold before I could reach a conclusion.
<vila> abentley: ok, no urgency
<vila> abentley: thanks !
<abentley> vila, no problem.
<abentley> vila, you could consider using one of our existing systems such as the Job System or the build farm if you're looking at migrating.
<vila> abentley: yup, that's was my understanding from the thread
<vila> s/'s//
<vila> i.e. tss
<bigjools> morning
<vila> bigjools: ha, that's why I thought you weren't in England ;-D
<bigjools> vila: yeah, but *Holland*? :)
<vila> bigjools: people wake up later there :D
<bigjools> lol - having been jelmer's manager I know that :)
<abentley> losa, bigjools: do normal staging updates include refreshing the archiveuploader?
<bigjools> abentley: what's the archiveuploader?
<abentley> bigjools, lib/lp/archiveuploader/
<bigjools> abentley: yes, but staging can only do binary uploads from the build farm
<bigjools> or recipe builds
<abentley> I'm not sure what you mean by "can only do binary uploads from the build farm".  Do you mean that the code that performs uploads for recipes is on the build farm and therefore not updated?
<abentley> bigjools, ^^
<bigjools> abentley: there's a process-upload.py cron job that processes any build coming from the build farm; there's no mechanism to process external source package uploads yet.
<bigjools> it's not on the build farm, no
<abentley> bigjools, I'm trying to figure out why my email changes haven't taken effect.  They landed in r10158, which is deployed on staging.
<bigjools> is that recipe build failure notifications?
<bigjools> or success in your case
<benji> bac: will you review my review of https://code.launchpad.net/~gmb/launchpad/hooky-hooky-bug-699719/+merge/47803?
<bac> yep
<abentley> bigjools, yes success notifications.  I'm testing failure now.
<bigjools> okidoki
<bigjools> not sure why you're not seeing those then - you're looking in the staging outbox?
<gmb> benji, bac: FTR, there *is* a feature flag: malone.use-advanced-subscriptions (or something of that nature). But I couldn't use it here because this is very, very pre-alpha.
<abentley> bigjools, yes.  The message "[STAGING] [PPA abentley-test] [ubuntu/maverick] wakeonlan 5.5~maverick1	(Accepted)" is formatted as if it were a user upload rather than a build upload.
<gmb> bac, benji: But Y.bugs.bugtask_index.use_advanced_subscriptions will be set to true by that feature flag once this is ready for alpha testing.
<benji> gmb: that's cool; I was thinking of something more pre-alpha-like; I'm not entirely sure feature flags are well suited for that though, because you may just want the functionality on demand.  I'm curious how you enable that functionality now (i.e., in development).
<gmb> benji: I just hack "Y.bugs.bugtask_index.use_advanced_subscriptions = true; " in above the call to Y.bugs.bugtask_index.setup_bugtask_index().
<abentley> bigjools, my failure test is still "uploading", 3 minutes after the build completed: https://code.staging.launchpad.net/~abentley/+archive/test/+buildjob/2185120
<benji> bac: https://code.launchpad.net/~wgrant/launchpad/trivial-soyuz-ui/+merge/47767 looks like it could use some review love; shall I claim it?
<bac> benji: yes, it looks pretty straightforward
<bigjools> abentley: I'm not sure how often the cron job runs to upload them on staging
<bigjools> DEATH TO pagetitles.py
<abentley> bigjools, Oh, completed now.
<bigjools> coolio
<abentley> bigjools, failure behaviour is unchanged too.  I am really strongly suspecting that process-email was not updated.  Do you know where it runs?
<bigjools> abentley: on staging itself
<bigjools> whatever that box is called
<bigjools> maybe there's a separate tree?
<abentley> bigjools, there are a bunch of boxes that make up staging.
<bigjools> ok
<bigjools> you need to find a friendly neighbourhood losa
<allenap> bdmurray: Do you remember doing @operation_removed_in_version stuff for IHasBugs.searchTasks()?
<abentley> losa, I can't figure out which machine runs "process-upload.py --builds" on staging, but whatever it is, I think it hasn't been updated to r10158.  Could you please investigate?
<Chex> abentley: let me look for you
<allenap> deryck, gmb, abel: Do you remember anything about @operation_removed_in_version stuff for IHasBugs.searchTasks()? I think (well, bzr ann says) that bdmurray was the last to touch the API declaration.
<abentley> Chex, ty
<deryck> allenap: yeah, I think he did that to add a couple params that weren't there before.
<allenap> deryck: Yeah, it looks like there's a different set of defaults for the arguments in devel to 1.0 (no mention of beta). I can't see that it needs the operation_removed_in_version() declaration though, and wondered why its there. I haven't tried removing it, so maybe I should see if smokes comes out.
<deryck> allenap: yeah, I would.  I'm sorry, but I don't recall the specifics.  He worked with leonard to get the format this is in now.
<allenap> deryck: Okay, thanks.
<gmb> allenap: What are the params being removed from?
<gmb> removed.
<gmb> Now here's an amusing thing, my IRC client is only sending *some* of what I'm typing.
<allenap> gmb: The devel declaration it appears, though I'm not sure if the declarations group up or down.
 * gmb looks
<gmb> allenap: For the first time in my life, I dislike decorators.
<gmb> I have NFI.
<jml> ooh
<jml> I know things about stuff
<jml> esp decorators
<allenap> gmb: Lol, cheers :)
 * gmb sits back and waits for the wisdom to flow forth from jml
<jml> what's the question?
<allenap> jml: IHasBugs.searchTasks() has a whatimcalling compound declaration.
<allenap> jml: So that devel and 1.0 have different arguments.
<allenap> jml: But there's an @operation_removed_in_version("devel") declaration.
<allenap> jml: However, the operation appears to be available still in devel.
<allenap> jml: And I'm not sure what was meant to replace it.
<allenap> We'll get lynched if we remove searchTasks.
<bdmurray> Is there anyway to find the mp that added that?
<gmb> bdmurray: Maybe.
<abentley> bdmurray, sure.
<gmb> (Ooh, forensics).
<allenap> bdmurray: Hello :) I guess it was yours from the 18th August.
<gmb> Damn, I was going to get all Grissom.
<abentley> bdmurray, see bzr lp-find-proposal
<bdmurray> allenap: hey, I don't recall specifically but if I looked at it I might remember better
<jml> allenap: searchTasks gets re-exported with different params in devel
<jml> allenap: above the "removed" line is an export line
<jml> allenap: the order of decoration goes from bottom to top, and each decorator gets executed
<bdmurray> this is starting to sound familiar ...
<jml> allenap: so the "export_read_operation" just above the "def" is for the 1.0 version
<allenap> jml: Ah, so when we export different declarations according to API version we need to remove it so that the next one can fit... getting vague, lights dimming...
<jml> and the one just above the "operation_removed" is for the devel version
<jml> it's sort of abusing the fact that decorators are actually executed in a particular order
<jml> as in, it's using them imperatively rather than declaratively
<jml> which is not _so_ bad I guess, but enough to make me start wondering if there's a better way.
<allenap> jml: If I removed the operation_removed_in_version it'll either complain that it's already declared, or it'll stomp over the previous declaration?
<allenap> Sorry, tenses all over the place.
<jml> allenap: I don't know. I know how to find out though :)
<allenap> Okay, I'm going to run some tests now :)
<allenap> Smoke comes out.
<allenap> How interesting.
<bdmurray> this was for bug 320596
<_mup_> Bug #320596: Series.searchTasks() always returns an empty collection <api> <lp-bugs> <qa-ok> <ubuntu-qa> <Launchpad itself:Fix Released by brian-murray> < https://launchpad.net/bugs/320596 >
<brendand_> hello launchpadders
<brendand_> can i please stop getting two emails for each bug status change. kthxbye
<allenap> brendand_: Do you have an example bug?
<allenap> bdmurray: Awesome, thanks for digging that up.
<allenap> And thanks jml too.
<brendand_> allenap - any bug
<brendand_> allenap - it's because i'm a member of a mailing list
<allenap> brendand_: Ah, yes :-/
<brendand_> allenap - which gets set the bug mail
<jml> allenap: my pleasure.
<brendand_> allenap - i guess i could address it with a filter. but can't something be done in launchpad?
<allenap> brendand_: I'm sure that something could be done, but my guess is that it's harder than it seems. The new Yellow Squad are currently working on bug subscription improvements. They might have something in mind.
<jml> is there a bug filed about it?
<allenap> gmb: As a yellow squaddie, can you help brendand_?
<gmb> allenap, brendand_: Do either of you know if there's a bug filed about this problem?
<gmb> (If not, I'll file one)
<brendand_> gmb - i don't know
<allenap> gmb: I don't know; my knowledge of bugs in Launchpad is the exact opposite of mpt's.
<gmb> :D
 * gmb goes to file a bug
<brendand_> gmb - it is easily worked around once you know why it's happening, but i'm sure it happens to lot's of people
<mpt> it appears to be bug 187346
<_mup_> Bug #187346: Multiple duplicate bug mails received when both subscribed to an external list which is the contact for a team subscribed to a bug and subscribed to the bug in some other fashion <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/187346 >
<gmb> \0/
<maxb> Does anyone have any recommendations for how to start learning YUI3 ?
<gmb> brendand_: Does bug 187346 accurately summarise your problem?
<_mup_> Bug #187346: Multiple duplicate bug mails received when both subscribed to an external list which is the contact for a team subscribed to a bug and subscribed to the bug in some other fashion <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/187346 >
<brendand_> mpt - not just bug mails
<brendand_> mpt - even more annoyingly it's merge mails too
<allenap> brendand_, gmb: Also bug 350390 is interesting because of its status.
<_mup_> Bug #350390: Duplicate Messages Across System <email> <lp-registry> <mailing-lists> <oem-services> <Launchpad itself:Won't Fix> < https://launchpad.net/bugs/350390 >
<brendand_> mpt - 'more annoyingly' because the message fields are exactly the same, so impossible to filter
<mpt> Any time a team is subscribed to something, it's a symptom of a missing feature in Launchpad
<jml> hah
<abentley> Chex, how's it going?
<Chex> abentley: ok, I found the branch, and its not on the correct revison, chekcing the restore process now
<abentley> Chex, cool, thanks.
<benji> ok, bac: you can review my review of https://code.launchpad.net/~wgrant/launchpad/trivial-soyuz-ui/+merge/47767 at your leisure
<bac> rt
<jcsackett> sinzui: might you have some time to mumble in a bit? i'm hitting something odd in the testcase for that question assertion bug. (which i can finally work on again).
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build (400): FIXED in 5 hr 2 min: https://hudson.wedontsleep.org/job/devel/400/
<LPCIBot> * Launchpad Patch Queue Manager: [r=allenap][ui=none][bug=688479,
<LPCIBot> 708028] Dragged updateTranslation and its evil allies out of their
<LPCIBot> holes and gave them to the sharks for breakfast.
<LPCIBot> * Launchpad Patch Queue Manager: [r=gmb][ui=none][bug=707478] bugs.freedesktop.org (Bugzilla 3.4.6)
<LPCIBot> has decided to take notice of the columnlist parameter we pass over.
<LPCIBot> * Launchpad Patch Queue Manager: [r=lifeless][ui=none][bug=665135] Don't OOPS when attempting to
<LPCIBot> display an empty remote branch URL.
<LPCIBot> * Launchpad Patch Queue Manager: [r=allenap][ui=none][bug=669288,
<LPCIBot> 683798] Delete SourcePackageRecipes when merging.
<sinzui> jcsackett: in a few minutes
<jcsackett> sinzui: thanks.
<bac> benji: re CSS i was talking about the same section
<benji> bac: ok, cool; I still don't understand why that constitues "CSS ... for global styles".  I will continue to seek enlightenment.
<bac> benji: perhaps i wrote badly.  i was trying to say only put stuff in the CSS file if it is reusable
<dpm_> hi deryck. It seems we've got a critical problem with translations, whereby upstream imports are overriding Launchpad transaltions unconditionally. I've been in touch with Danilo and Henning about this, but they are away today, and I'm not sure whether I should talk to your team or to the maintenance squads. May I just forward you the e-mail with the discussion and let you decide?
<deryck> dpm_: sure, that works.
<deryck> dpm_: is it fallout from the recent feature work?  i.e. something has changed and now causing issues?
<benji> bac: ah, that makes more sense; perhaps I've been hit over the head with the inline-styles-will-kill-kittens stick too many times
<bac> :)
<jcsackett> sinzui: have a sec? i don't know that mumble is necessary.
<dpm_> deryck, sorry for the delay, Evolution crashed on me. I've now put you on CC on the thread. Yes, it seems that this has to do with the new upstream imports in Ubuntu that came with the last LP rollout
<jcsackett> sinzui: https://bugs.launchpad.net/launchpad/+bug/697294
<_mup_> Bug #697294: AssertionError editing a question  <oops> <questions> <Launchpad itself:In Progress by jcsackett> < https://launchpad.net/bugs/697294 >
<deryck> dpm_: ok, cool.  I'll look closely at the thread now.
<dpm_> thanks a lot deryck. I can provide more info if it helps, but I'm not familiar with the LP code
<deryck> dpm_: no worries.  I'm new to translations, too. :-)  But I'll see if I can at least get my head around the issue, and work out what we can do next.
<dpm_> deryck, I appreciate that, I know you guys are doing an extra effort getting your way around parts of LP new to you now :-)
<deryck> dpm_: ok, so let me see if I get the main points right....
<dpm_> ok
<deryck> dpm_: 1) we messed up huge here :-) because we're updating translations we shouldn't (i.e. we're not following the precedence rules we always have....
<dpm_> yep
<deryck> dpm_: 2) but the damage is done for now since you disabled imports.....
<dpm_> yes
<deryck> dpm_: 3) and we need to work out a) what happened and b) how quickly we can get this fixed.
<dpm_> exactly, well summed up. At least that's my understanding. Henning or Danilo would perhaps be able to add more context, but that's how I understood it.
<deryck> ok, cool.
<deryck> so the problem is that we really don't have any domain expertise until Monday
<deryck> and if we're not causing any more damage, I don't think we should call people at home now, unless you think it requires that kind of attention.
<dpm_> no, I don't think it does
<deryck> dpm_: ok, so I'll send email to my squad now, and make sure we're clear that this has to be looked into ASAP when everyone is working again Monday.  sound fair?
<dpm_> Right now I just wanted to raise the issue, I assumed that even with domain expertise this is not something that can be fixed quickly
<dpm_> deryck, that sounds good, thanks a lot
<deryck> dpm_: np.  Sorry this got messed up so badly.
<dpm_> no need to apologise, let's look at how to fix it next week :)
<jml> gary_poster: hi
<gary_poster> hey jml
<jml> gary_poster: just letting you know that I have indeed received your email, and that I'll be looking at the LEP again as soon as I may. (Probably my Monday morning)
<gary_poster> jml, awesome, thank you.  I'm hopeful that there won't be any surprises for you given out previous conversations, and am proceeding in that hope, so that timing is fine.
<gary_poster> *our
<jml> gary_poster: cool :)
* adeuring changed the topic of #launchpad-dev to: https:/â/âdev.launchpad.net/â | PQM is open | firefighting: - | On call reviewer: bac, benji | https://code.launchpad.net/launchpad-project/+activereviews
<jml> I wish subunit were one project per language
<jelmer> I sometimes wish that too, but that's only because I'm not doing the release management / packaging :-)
<jml> gah, emacs redraw bugs
<deryck> bdmurray, hey.... I thought you were one of the main ones driving to disable subscriptions for ubuntu.  Am I completely recalling this wrong?
<gmb> Have a good weekend folks.
 * gmb -> exeunt, in pursuit of a Friday evening
 * benji thought exeunt was plural; maybe gmb is a colonial organism.
<bdmurray> deryck: No, you are right and have exposed me!
<deryck> heh
<deryck> bdmurray: ok, at least I'm not crazy then
<bdmurray> deryck: nope not at all.
<bdmurray> deryck: its complicated ;-)
<deryck> bdmurray: invariably everything with Launchpad is, for some reason ;)
<deryck> bdmurray: I won't call you out on list for the two-faced requester that you have become.  but I'll probably tell gary_poster this in secret.
<gary_poster> deryck, bdmurray, lol
<bdmurray> deryck: I think I wrote some of the code disabling it so it wouldn't be hard for somebody to out me
<deryck> heh
<gary_poster> so, bdmurray, you share deryck's social concern, but would really like the functionality for yourself?
<gary_poster> IOW, as a clunky interface example, you would like to say that users *can* subscribe to distributions, but they get warned about it?  Or...you have to be a member of some team?
<bdmurray> gary_poster: yes, that's a fair statement my concern is people doing something they didn't intend and getting hate mail about it.
<gary_poster> got it, bdmurray.  yeah, hate mail is no fun.  OK, I'll add that to the LEP discussion too.  thank you
<bdmurray> It'd be neat if you could see how many bugs would have met your criteria over the past 24 hours or a week
<bdmurray> You could even just allow distro subscriptions via the API and I'd be happy
<gary_poster> how many bugs: that would be neat, I agree.
<gary_poster> distro subscriptions via API: ok
<deryck> I need someone smarter than me about layers to weigh in on my proposed fix for Windmill LayerIsolationError issues.
<deryck> maybe gary_poster or lifeless  (if around)? :-) ^^
<gary_poster> deryck: oh meh.  :-) lifeless' layer knowledge is way more recent than mine, but I guess I'll give it a try.  I know you are EoD soon, but I'd like to wrap something up first.  10 min?
<deryck> gary_poster: totally fine.  I don't EOD for another 2 hours.
<gary_poster> oh ok cool
<gary_poster> will ping soon then
<deryck> gary_poster: ok, cool.  thanks!
<gary_poster> np
<lifeless> gary_poster: deryck: ?
<gary_poster> +1 on lifeless answering your question, deryck ;-)
<deryck> hi lifeless.  So I'm trying to fix the LayerIsolationError problems breaking the build....
<lifeless> \o/
<deryck> lifeless: the simpleness of this fix has me worried I missed something.  https://code.launchpad.net/~deryck/launchpad/kill-windmill-test-pain-ftw/+merge/47852
<deryck> would value input from you
<lifeless> uhoh, we're in trouble then :P
<deryck> heh
<lifeless> looking
<lifeless> hmm
<lifeless> so this assumes that None is the correct value
<lifeless> what is actually making the change?
<lifeless> ah
<deryck> well the BaseLayer throws the error if it isn't None, i.e. we want it to be None when we're done.  Windmill messes with the timeout at various places.  Windmill itself, not our use of it.
<lifeless> so our hardcoded check is for a timeout of None
<deryck> right
<lifeless> so setting it to None is indeed appropriate
<lifeless> grah, someone needs to take that library out and shoot it; changing global state is Not Cool
<deryck> yeah, and just as I was making peace with the monster
<deryck> :-)
<lifeless> an ideal fix would be to find and wrap all those places
<lifeless> as we *may* still run into issues within the layer that uses it
<deryck> I can fix Windmill itself, but I was wondering if this would get the build working again as a temp fix.
<lifeless> but this certainly should address our symptoms
<deryck> shall I XXX this line with a bug describing the badness of Windmill and then land it?
<lifeless> deryck: +1
<deryck> lifeless: cool, thanks!
<lifeless> yes indeedy, and a bug somewhere for us to hammer on later; High pri IMO.
<deryck> bug 709438
<_mup_> Bug #709438: Windmill mucks about with socket default timeout and shouldn't <javascript> <windmill> <Launchpad itself:Triaged> < https://launchpad.net/bugs/709438 >
<deryck> I can fix this next week.
<deryck> I need to get a patch upstream for the 512k bug anyway
<bdmurray> gary_poster: looking again I think bug supervisors can structurally subscribe to distro bugs if a bug supervisor is set
<bdmurray> gary_poster: at least that is the way it was when I wrote it
<gary_poster> bdmurray, oh?  I'm afraid I'm not even sure how bug supervisors fit into it
<bdmurray> gary_poster: its only for distributions with bug supervisors set that structural subscriptions are restricted for and they are restricted to the members of the bug supervisor team
<bdmurray> gary_poster: so there is no need to "re-allow" structural subscriptions on distributions
<gary_poster> ah!  so you can already subscribe to distributions, so if we add filtering for that then that will allow you to use that again?
<bdmurray> not again but yes
<gary_poster> I thought you were not using that now, because it was too much?
<bdmurray> Okay, yes *I* personally have not been subscribed to all ubuntu bugs for a long time now
<gary_poster> :-)
<gary_poster> ok
<gary_poster> I think I got you, then.  thanks for clarifying
<bdmurray> gary_poster: here is the mp https://code.launchpad.net/~brian-murray/launchpad/bug-556489-distro-struct-sub/+merge/31090
<gary_poster> awesome, thanks
<bdmurray> no problem
<SpamapS> why oh why can't I delete my own comments on a bug? :-(
<dobey> SpamapS: buyer's remorse? :)
<SpamapS> wrong window :-/
<SpamapS> at any given time of day I have between 5 and 20 bugs open in chrome
<SpamapS> sometimes I mistake them for eachother..
<SpamapS> I can only imagine how that lady with octuplets must feel :-P
<dobey> heh, i wish i could delete others' comments sometimes. especially when they are mailer autoreplies
<lifeless> so
<lifeless> that would be nice
<lifeless> before we make it cheaper
<lifeless> I think we need stable indices (in progress), as well as some sort of defined rule for who can hide what, and who can unhide what.
<lifeless> (and who can see whats hidden)
<dobey> or a collapsable view that is collapsed by default except the most recent comment, or something
<dobey> busy bugs can be pain to follow when you have to scroll 2000 pages to see the last comment :)
<lifeless> that might be nice too
<lifeless> I don't know if you know, but we don't show all comments by default
<lifeless> we show first + last 80 or something
<dobey> still too much, i think. even with only showing partial comments, because each comment can still be quite large, even if it's partial
<lifeless> sure
<lifeless> the End key is also helpful
<dobey> but no i didn't really know that
<dobey> yes, but browsers are slow.
<lifeless> excellent, I think we squashed Distribution:EntryResource:searchTasks
<dobey> especially on complex sites like launchpad
<lifeless> Monday, time to drop a second off of the timeouts
<lifeless> dobey: Launchpad is slow for many reasons :)
<dobey> indeed
<cr3> lifeless: I noticed a bug mentionned in a thread by poolie about a couple queries taking a long time, one for the count() and the other for the actual result set. have there been discussions to improve retrieving result sets so that they don't necessarily need to also return the count?
<lifeless> yes
<lifeless> I first raised this in July I think
<cr3> lifeless: if these discussions are recorded somewhere, I'd be interested :)
<lifeless> jtv made a good suggestion on a different dimension at the epic, to improve the efficiency of slicing
<lifeless> cr3: the dev mailing list
<cr3> lifeless: will have a look, I had ideas too about retrieving slice+1 items or somesuch, nothing terribly innovative but might provide a good bang for the buck
<lifeless> cr3: so for nontrivial counts we should use query plan estimates - there is code in the tree
<lifeless> cr3: for efficiency rather than offsets we should use limits
<lifeless> e.g. instead of offset 100 limit 76, use id > 1234 LIMIT 76
<cr3> lifeless: hm, that never occured to me but makes total sense!
<deryck> Have a nice weekend everyone.
<wgrant> sinzui: Morning.
<sinzui> hi wgrant
<wgrant> sinzui: The new nightly.sh is on loganberry, which means p-r-f won't run until you land your crontab change.
<sinzui> yes
<sinzui> I struggled with that.
<sinzui> spm merged it about 18 hours ago, but did not push the branch. Chex helped with that
<wgrant> Ahh, I asked him about that, but he didn't say he'd merged it.
<sinzui> There was a lot of confusion when I asked Chex to merge it and we had a zero-length patch
<wgrant> Heh.
<lifeless> hi sinzui
<sinzui> So the next step is to ensure crontabs are live
<sinzui> hi lifeless
<wgrant> sinzui: Apparently that is done before the branch is touched.
<wgrant> That is, the crontabs become live *first*.
<sinzui> ?
<sinzui> So so I wait for hate mail form script-monitor to learn of the change is in production?
<wgrant> sinzui: IIRC spm said that the change is made live first, then put into the branch.
<wgrant> So we should have no hatemail.
<wgrant> Which will be a pleasant change.
<lifeless> laters
<wgrant> sinzui: In https://code.launchpad.net/~wgrant/launchpad/trivial-soyuz-ui/+merge/47767 I hardcode 'https://launchpad.net/'... is there a good way to determine that from the config?
 * sinzui thinks
<wgrant> Hmm.
<wgrant> It is really tempting to kick Windmill out again until deryck can get it to work.
<wgrant> Because we're not leaving testfix until that happens.
<wgrant> Any objections?
<wgrant> It's a one-line change to reblacklist.
<bigjools> jfdi
<sinzui> wgrant: I think we want to use config.vhost.mainsite.hostname for the case I see in the merge
<wgrant> sinzui: OK, I wondered if there was some way apart from digging directly in the config.
<sinzui> wgrant: list a helper function?
<sinzui> s/list/like/
<wgrant> Yeah.
<sinzui> We may have had some in the past, but not now. webapp.vhosts.allvhosts is used in some places. it is a specialised config
<wgrant> sinzui: allvhosts.configs['mainsite'].rooturl looks relevant.
<sinzui> indeed. I think that is what canonical_url uses
<jcsackett> sinzui: are we doing standup today, given it's wgrant's saturday?
<wgrant> I hope not.
<jcsackett> wgrant, i think you're free. :-)
#launchpad-dev 2011-01-29
<lifeless> wgrant: deryck is landing a fix alrady
<wgrant> lifeless: So he said on the list.
<wgrant> I've rolled back my rollback.
<wgrant> But there is no sign of his fix.
<maxb> Does launchpadlib have a stated API compatibility policy?
<wgrant> maxb: launchpadlib or the API?
<maxb> The launchpadlib python api
<maxb> erm, the directly coded, not WADL-driven bits
<wgrant> launchpadlib: no, but we tend to respect it because Ubuntu likes it. the API: yes, but we tend not to respect it apart from exceptional cases.
<wgrant> 1.7 and 1.8 had auth API breaks, but 1.9 restored the 1.6 API.
<maxb> Basically I'm upset that Launchpad.login_with(service_root=xxx) changed its definition between 1.5.4 and 1.5.5, and further upset that API compat for service root constants seems to have been removed in 1.(x>6)
<wgrant> Oh? STAGING_SERVICE_ROOT is still there... but EDGE_SERVICE_ROOT is gone.
<maxb> yes (Admittedly I though STAGING_SERVICE_ROOT had disappeared too, but that was just me misreading)
<wgrant> Boom.
<wgrant> Something managed to break all the virtual builders.
<lifeless> wgrant: https://code.launchpad.net/~deryck/launchpad/kill-windmill-test-pain-ftw/+merge/47852
<wgrant> lifeless: I know. And the fix looks OK.
<wgrant> But it seems to have not made it out of EC2.
<lifeless> wgrant: merge dnow
<wgrant> lifeless: Yeah.
<LPCIBot> Project db-devel build (330): STILL FAILING in 6 hr 46 min: https://hudson.wedontsleep.org/job/db-devel/330/
#launchpad-dev 2011-01-30
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build (331): FIXED in 6 hr 22 min: https://hudson.wedontsleep.org/job/db-devel/331/
<LPCIBot> Launchpad Patch Queue Manager: [rs=buildbot-poller] automatic merge from stable. Revisions: 12284
<LPCIBot> included.
<lifeless> === Top 10 Time Out Counts by Page ID ===
<lifeless>     Hard / Soft  Page ID
<lifeless>      182 /  299  BugTask:+index
<lifeless>       13 /   12  Archive:+copy-packages
<lifeless>       12 /    3  Cve:+index
<lifeless>       10 /   11  Branch:+index
<lifeless>        8 /    0  BinaryPackageBuild:+retry
<lifeless>        7 /   27  DistroSeriesLanguage:+index
<lifeless>        7 /    8  Person:+uploaded-packages
<StevenK> Why is the top 10 only 7?
<wgrant> Indeed, 2-4 are omitted.
<wgrant> Bad lifeless.
<lifeless> that would be freenode fail
<lifeless> their new software throttles far too aggressively
<thumper> Monday morning, and all quiet
<lifeless> heh
<lifeless> can fix that
<lifeless> thumper: btw maxb reopened your stuck builds fix
<thumper> yeah
<thumper> saw that
<lifeless> hmm
<lifeless> in linking series to branch
<lifeless> would it make sense to offer a likely list? e.g. code imports + branches called 'trunk', 'MAIN', 'HEAD' etc
<thumper> lifeless: file a bug :)
<lifeless> ok
<thumper> are the windmill tests disabled?
<lifeless> I'm not sure
<jelmer> g'morning nz
<lifeless> woo
<lifeless> no new bugs
<lifeless> no confirmed bugs
<thumper> hi jelmer
<wallyworld> morning
<thumper> wallyworld: morning
<thumper> wallyworld: thanks for taking the CHR slot
<wallyworld> thumper: np
<lifeless> and woo every undecided importance bug is incomplete.
<mwhudson> ah, this is why i have so much email, perhaps
<mwhudson> not that i get launchpad bug mail any more
<lifeless> james_w: bug 645640
<james_w> yes?
<lifeless> james_w: you commented that its blocked, but not by what
<lifeless> ah bah
<lifeless> wrong bug
<lifeless> hang a sec while I try to find the right one again
<james_w> well, why is that one incomplete?
<lifeless> it was incomplete pending a reply, which you have given
<lifeless> james_w: though I do have another question for you on that bug :)
<lifeless> james_w: bug 395200
<lifeless> poolie: call?
<wgrant> Anyone up for reviewing https://code.launchpad.net/~wgrant/launchpad/bug-710360-gina-wheezy/+merge/47942?
<lifeless> wgrant: ugh
<wgrant> Yes.
<lifeless> wgrant: are you a reviewer yet?
<wgrant> No.
<wgrant> I need a mentor.
<lifeless> ok, I'll mentor you
<wgrant> lifeless: Sure?
<lifeless> yes, I think I can handle it
<wgrant> Thanks!
<wallyworld> thumper: you free yet?
<thumper> wallyworld: yep
<thumper> sorry
<thumper> mumble?
<wallyworld> ok
<lifeless> wgrant: can you let bac know, get yourself on the roster etc etc
<thumper> StevenK: ping
<thumper> StevenK: mumble ping
<StevenK> thumper: Hai
<StevenK> thumper: I can
<StevenK> thumper: You think?
<StevenK> thumper: I need a bit more time to clean up some SQL
<StevenK> thumper: Yes
<StevenK> Same thing
<StevenK> It's all IPerson.merge
<wgrant> Speaking of which, can I object to the delete-recipes-on-merge thing? :)
<StevenK> wgrant: thumper just did
<wgrant> Yay!
<StevenK> thumper: So, I don't think the queue-recipes will take very long
<StevenK> thumper: Then I'll rip team-merge apart
<StevenK> thumper: branch merging is all done with SQL
<StevenK> thumper: def _mergeBranches(
<thumper> wallyworld: https://launchpad.net/oops-tools
<thumper> wallyworld: http://pastebin.ubuntu.com/560402/
<thumper> lifeless, mwhudson: didn't someone organise to have the doctests rendered as ReST and available somewhere on a site?
<mwhudson> thumper: BjornT_ was working on that
<lifeless> thumper: benji presented a plan
<mwhudson> thumper: ages ago though
<lifeless> thumper: in july
<thumper> so not yet then...
<lifeless> not yet
<mwhudson> searching my mail archive turns up http://people.canonical.com/~bjorn/lp-architecture/
<mwhudson> but i don't know if that ever got past proof-of-concept
<lifeless> it was a redo
<lifeless> new docs
<lifeless> new subtree
<lifeless> existing stuff untouched
<mwhudson> ah right
<lifeless> I decided it wasn't worth pursuing in that form
<lifeless> wow RootObject:+daily-builds is sick
<thumper> lifeless: how sick?
<lifeless> https://devpad.canonical.com/~lpqateam/ppr/lpnet/latest-daily-timeout-candidates.html
<lifeless> never renders in < 9 seconds
<thumper> heh
<thumper> 273 queries won't help
<thumper> should aim for < 20
<lifeless> yup
<lifeless> I was looking for pages whose 99% completing fell between 14 and 15 seconds
<lifeless> to see the impact of dropping the timeout by a second
<lifeless> only two
<lifeless> that, and PillarNameSet:EntryResource:search
<lifeless> both < 50 hits a day
<thumper> lifeless: probably a day or so to fix that one
<thumper> StevenK: the morning standups just aren't the same if you can't talk
<thumper> StevenK: plz fix :)
<StevenK> Here I was thinking you'd say they're better if I can't talk
<wgrant> lifeless: You know that armel PPAs are coming RSN, right?
<mwhudson> fsov rsn
<wgrant> I believe that's included in the definition of RSN.
<jelmer> wgrant: I thought we already had armel PPAs?
<mwhudson> jelmer: well sort of, but they're not virtualized
<wgrant> jelmer: We have restricted fake virtualized builders.
<wgrant> That are not virtualized.
<jelmer> right, so we have armel PPAs :-)
<StevenK> lifeless: Does http://pastebin.ubuntu.com/560425/ look sane to you? I don't think Storm is bracketting the two date_created conditions correctly
<lifeless> StevenK: why do you have the two comparisons with date created ?
<StevenK> lifeless: Because if there is a build that's been created within the last 24 hours, we don't want to schedule another
<lifeless> that only requires a <
<lifeless> not < is equivalent to >=
<StevenK> Er, won't that return builds that have been scheduled in the last 24 hours?
<lifeless> StevenK: I think your math is broken :)
<lifeless> StevenK: date created is a point in time; we are interested when the build was scheduled before now-23h50m
<lifeless> StevenK: if we had an interval, we would query on interval > 23h50
<lifeless> StevenK: if we have a position, we query on things before that position
<StevenK> lifeless: Keep in mind a recipe can have more than BuildFarmJob
<StevenK> More than one
<lifeless> ah yes
<lifeless> I really dislike this data model
<StevenK> So I'd like to make sure if there is one BFJ, it was created more than one day ago, and if there are more than one, that they all were created more than one day ago
<StevenK> lifeless: Patches welcome
<StevenK> lifeless: But keep in mind that BFJs are used by more than just recipes
<lifeless> sure
<lifeless> the basic problem is that you want to work on stuff that is 'current' but the schema is an archival schema
<lifeless> better to hav a current schema and an archive schema, if you want archiving.
<wgrant> lifeless: What does current mean?
<lifeless> wgrant: live, active
<wgrant> In this case we are looking for records that might have built a day ago.
<wgrant> Are they archived?
<lifeless> wgrant: in a different model you wouldn't be looking for anything in the archive schema
<lifeless> wgrant: so that question is noddy
<wgrant> Oh?
<wgrant> How would you model it?
<lifeless> StevenK: so, we want SPR with either (no builds) or (no builds in the last 24 hours) right ?
#launchpad-dev 2012-01-23
<lifeless> oh hai
<nigelb> Morning!
<lifeless> o/~
<poolie> hi nigelb
<nigelb> Hey poolie
<nigelb> Back to normal after Budapest? :)
<poolie> slept til 11 :)
<poolie> something about this trip really messed me up
<nigelb> Heh
<lifeless> flacoste: bug 772954
<_mup_> Bug #772954: "Opinion" bug status causes user confusion <opinion-status> <Launchpad itself:Triaged> < https://launchpad.net/bugs/772954 >
<nigelb> Would that bug be marked as opinion?
 * nigelb runs
<poolie> heh
<poolie> lifeless, the plan is to delete it?
<poolie> or that was the plan?
<lifeless> that was jml's intent
<lifeless> I believe it needed a sign off that didn't happen before he moved on
<lifeless> flacoste and I were talking about it with the Ubuntu qa folk in Budapest
<poolie> ..
<poolie> a 'lock' flag still seems attractively simple to me
<poolie> rather than having policy per state
<poolie> i think there were some arguments against it though
<lifeless> thats a rather separate discussion
<lifeless> opinion has no policy associated with it - it is identical to invalid and wontfix.
<poolie> hm, really
<poolie> i thought the point of it was unprivileged users could not move things out of that state?
<lifeless> no
<lifeless> it was purely a different way of saying wontfix / invalid
<poolie> ah ok
<poolie> so bug 772954 could probably be low, unless you still want it high
<_mup_> Bug #772954: "Opinion" bug status causes user confusion <opinion-status> <Launchpad itself:Triaged> < https://launchpad.net/bugs/772954 >
<poolie> it causes some noise edits but it's not that much of a problem
<lifeless> I think it is still high, as it speaks to the usability of LP and is shallow (barring political aspects)
<lifeless> wgrant: bug 767258 - got 2c?
<_mup_> Bug #767258: weird PPA stats before Feb 10 <ppa> <Launchpad itself:Triaged> < https://launchpad.net/bugs/767258 >
<wgrant> lifeless: 'sup?
<lifeless> do you have any idea what the story is?
<wgrant> Nope :/
<wallyworld_> StevenK: your lp meta deps changes mean that convoy is installed automatically when upgrading launchpad-dependencies, tight?
<StevenK> wallyworld_: It should be, yes.
<wallyworld_> StevenK: to test, i uninstalled convoy and reinstalled lp-deps (after apt-get update) and it's didn't install convoy
<wgrant> wallyworld_: launchpad-developer-dependencies
<wgrant> Not launchpad-dependencies
 * wallyworld_ headdesks
<wallyworld_> ok, sorry
<StevenK> So I can cause an oops by browsing to "launchpad.net/~/+archivesubscriptions/foo", but if I attempt the same thing in a test, I get a much different traceback?
<wallyworld_> how different?
<lifeless> 919 high bugs; only have to get 2/3 more
<StevenK> wallyworld_: OOPS-1ff1f2efe2aad173a4eb3eb85bdd679d versus http://pastebin.ubuntu.com/813964/
 * wallyworld_ happy that lp-oops login is fixed
<wallyworld_> StevenK: apart from the test browser setup difference which is explainable, they both enter publish at the same point and  barf on traverseName differently. i have no idea why
<wallyworld_> either
<wallyworld_> perhaps devel trunk has changes not yet in prod
 * wgrant slays the inventor of timezones, and the author(s) of Python datetime handling.
<wallyworld_> wgrant: if it's any consolation, it's also broken in Java :-/
<nigelb> wgrant: Isn't part of python datetime handling written by stub? :D
<wgrant> nigelb: pytz is his, yes.
<wgrant> It's not terrible -- it has to work around the horror that is the standard library's datetime handling.
<nigelb> Heh, yeah.
<wgrant> Hmm
<wgrant> https://github.com/elasticinbox/elasticinbox appeared about two weeks before we started grackle, but after I last looked for similar things :/
<wgrant> Although it doesn't seem to do much yet.
<lifeless> wgrant: hah!
<adeuring> good morning
<Laney> can somebody help me unbrick qastaging? :-)
<Laney> for example https://qastaging.launchpad.net/ubuntu/+source/quilt
<cjwatson> there was activity on the ops channel that suggested StevenK/spm already did?
<Laney> works now
<Laney> didn't when I pasted it ...
<Laney> but regardless, yay, thanks.
<Laney> can someone accept one of the banshee syncs in qastating/oneiric/unapproved? Didn't realise it was frozen.
<Laney> I need the SPPH.
<cjwatson> Laney: done
<cjwatson> hmm, oops, rejected
<cjwatson> I'll accept the other one :)
<Laney> heh
<cjwatson> Laney: hmm.  I think, queue or not, you need to sync it into the Proposed pocket, not Release
<cjwatson> actually no, sorry
<cjwatson> it's in accepted now
<Laney> waiting for it to appear on https://qastaging.launchpad.net/ubuntu/+source/banshee/+publishinghistory
<cjwatson> qastaging's oneiric is only frozen not current
<Laney> yeah
<cjwatson> you won't get an SPPH until process-accepted runs, and I'm not sure that will ever happen on qas
<cjwatson> bigjools: ^ ?
<Laney> maybe I won't bother QAing the non-sponsored case ...
<bigjools> the PCJ has to run again
<Laney> is it all going to happen automatically?
<cjwatson> oh, it doesn't require process-accepted?  curious
<rick_h> morning
<bigjools> p-a is not used for syncs
 * bigjools otp, soryr
<cjwatson> Laney: ah, it was rejected because it's older than oneiric's version on qas
<cjwatson> Laney: https://qastaging.launchpad.net/ubuntu/+source/banshee/+publishinghistory - you tried to sync 2.1.4-1
<Laney> oh, I fail at comprehension
<Laney> cjwatson: for some reason I always get banshee versions out-of-order in my head. Anwyway, synced ghc which should be OK to accept.
<cjwatson> Laney: done
<cjwatson> Laney: https://qastaging.launchpad.net/ubuntu/oneiric/+source/ghc/7.2.1-1
<Laney> cjwatson: great, thanks: http://paste.debian.net/153313/
<cjwatson> cool
<Laney> should I do the non-sponsored case to make sure it is None? I guess so.
<wgrant> cjwatson: I'm not sure that copyPackage likes binaries very much.
<wgrant> It certainly hasn't been tested with them to any significant extent.
<wgrant> Which may be relevant.
<cjwatson> aha
<cjwatson> noted
<Laney> argh, forgot to remove sponsor. If someone could accept the more recent mono-uia sync from https://qastaging.launchpad.net/ubuntu/oneiric/+queue?queue_state=0&queue_text= that'd be great.
<cjwatson> Laney: done
<Laney> bonus
<Laney> qa-ok
<wgrant> rick_h: Can you organise an ndt once you QA your regression fix?
<rick_h> wgrant: sure, is there a doc for that? /me hits the wiki
<rick_h> wgrant: if not I can get abently to walk me through it as I'm sure he's done it before
<StevenK> rick_h: Add a line to the Requested Deployments table on LaunchpadProductionStatus
<StevenK> rick_h: The fields should be self explanatory, and the docs are the paragraph before the table
<rick_h> StevenK: ok thanks, and then ping a webops?
<rick_h> ok
<StevenK> rick_h: Right
<StevenK> rick_h: Are you good WRT the NDT?
<rick_h> StevenK: looking/reading
<rick_h_> not to self, you can't 'undo' in irssi with ctrl-z...that will just kill irssi
<rick_h_> StevenK: I think I'm ok.
<cjwatson> ctrl-z will suspend irssi, not kill it, surely ...
<cjwatson> use 'fg' to get it back
<rick_h> ah, nice
<rick_h> ok cool
<rick_h> StevenK: ping, still around? With the use-convoy I should be able to remove your ppa, uninstall convoy,and get it picked up from the lp deps repo?
* benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: benji | Firefighting: - | Critical bugtasks: 3*10^2
<bigjools> cjwatson: any ideas what might be wrong with this build? python setup fail...   https://launchpadlibrarian.net/90814529/buildlog_ubuntu-precise-i386.maas_0.1-0%2B36%2B4~precise1_FAILEDTOBUILD.txt.gz
<cjwatson> bigjools: looks like a package bug to me - have you tried in an up-to-date sbuild?
<bigjools> cjwatson: we figured it out - unicode issue...
<cjwatson> ok
<bigjools> when I say "we" I meant allenap of course :)
<rick_h> anyone familiar with the db error handling here: b/lp/services/webapp/tests/test_error.py ?
<rick_h> we've got a failing test: (at the end of this very very long test report) https://pastebin.canonical.com/58603/
<rick_h> adeuring: so this is the bit that's doesn't like the feature flag in some of the oops blocks "Expression: <NotExpr u'request/features/js.combo_loader.enabled'>"
<rick_h> it's in the features/flags.py and works when running.
 * adeuring is looking
<rick_h> adeuring: I think it's this part: https://pastebin.canonical.com/58605/
<rick_h> some how the feature flag bit is getting traversed?
<adeuring> rick_h: so this is not the regular context but an error, right?
<rick_h> right, this is an error that comes out of a test checking if the db disconnect handling is working right
<rick_h> so somehow in the process of trying to handle this error, and respond with a noce 503, it blows up on the feature flag bits and instead returns a 500 with the oops dump it looks like
<rick_h>  /noce/nice/
<adeuring> rick_h: I am not very familiar with LP's error handling -- but it could simply be that this error is raised even before the feature machinery is configured.
<rick_h> yea, that's what I'm beginning to think. Ok, so the *fix* would be to add some logic to check for the feature flag exists before checking the value I suppose
<adeuring> rick_h: so, a possible fix would be to check if request/features exists.
<adeuring> rick_h: the test is about a broken database, and we store feature flag values in the database
<rick_h> adeuring: right, I gotcha. Took me a bit to go through all the traceback info and put it together. Thanks, I'll look up the tal to do a pre-check and see if that's enough to bypass that logic in the base-layout-macros.pt
<adeuring> rick_h: yes, you could do something like <tal:condition request/features Â¦ nothing" -- but that is a bit ugly because it can hide other errors
<adeuring> rick_h: something like a property "features_are_ready" would be more clean
<rick_h> ok
<adeuring> rick_h: or even an empty features object that is later replaced by the "real one"
<rick_h> benji: ping, wonder if you can give me a hand.
<benji> rick_h: sure, what's up?
<rick_h> benji: so first I'm trying to fix test failures from wallyworld_'s branch he tried to land. I'd like to get them reviewed on their own without tying you to his full branch
<rick_h> https://code.launchpad.net/~rharding/launchpad/updated_wally_use_convoy
<rick_h> benji: but then once I get past that, I'm trying to figure out how I can get the branch through ec2 land. I'd need to either do a new MP? or can I somehow try to land his branch?
<rick_h> I supposed I'd just have to do a new MP, get a new sign off, and then run the land command from there?
<rick_h>  /supposed/suppose
<rick_h> benji: here's basically would I would do as a MP description for my changes if that helps: https://pastebin.canonical.com/58616/plain/
<benji> rick_h: looking now
<benji> rick_h: I think the best approach would be for you to do an MP for your branch, making his branch a prerequisite so the diffs show just your changes.  Then you can EC2-land your branch.
<benji> given your description of the changes, the review should be quick and easy
<rick_h> ooh, I cna make his a pre-req? ah cool I've not used that.
<rick_h> ok, will do thanks.
<benji> gary_poster: file_append adds a line at the end; that's what "append" means ;P
<benji> pfft!  wrong chan
<benji> I can blame sleep deprivation this time.
<rick_h> benji: ok, here you go when you get time: https://code.launchpad.net/~rharding/launchpad/updated_wally_use_convoy/+merge/89732
<rick_h> benji: I'm in particular wondering if I should do the change for the feature flag issue differently
<benji> rick_h: cool, I'll take a look in a little while
<rick_h> benji: ty
<lifeless> morning
<nigelb> mrevell: Happy Birthday!
<nigelb> (off all the things, skype reminded me its your birthday)
<nigelb> lifeless: Morning!
<lifeless> hiya
<mrevell> nigelb, Oh thanks. Not til the 25th :)
<wallyworld_> sinzui: hi, did you see my email?
<sinzui> wallyworld_, I did, I guess this means you did not get my reply
<sinzui> benji, do you have time to review https://code.launchpad.net/~sinzui/launchpad/export-private-bugs-0/+merge/89784
<benji> sinzui: sure
<sinzui> wallyworld_, to repeat my reply, I like your suggestion, but we track access to bugs and branch by subscription to those objects, and in the future, in the accesspolicyartefact table for reporting in the disclosure pages. I do not think we should introduce a change to how we grant access to things now. We need to talk to wgrant about the time to start switching to the new rules
<rick_h> wallyworld_: ping, you see my MP/email?
<lifeless> flacoste: http://www.akamai.com/
<rick_h> oooh, we getting serious about some cdn fun?
<benji> sinzui: your branch looks good
<wallyworld_> sinzui: what you say is true i did think along the same lines. i was trying to avoid introducing another situation requiring porting to the new methodology, and doing it in the security adaptor seems cleaner. but i'll do it the current way and we can deal with it later
<wallyworld_> rick_h: yes, thanks. a bit of chaos in my house this morning so no reply yet sorry. but looks good. as soon as we get the rt sorted, i'll land
<rick_h> wallyworld_: k, tests are still running. I'm going to be off to get the boy from day care soon, but will check back in after he hits bed in about 3 hours
<wallyworld_> rick_h: given it was only 4 failing tests and you fixed those, i think we will be good :-) i may end up just merging your branch into mine and landing that, depending on timing of it
<rick_h> wallyworld_: yea, all for that. I just wanted to get my changes reviewed because I was a little unsure of my fix for the 503'ing tests
<wallyworld_> will look before I merge :-)
<rick_h> so setup the MP and hit ec2 test hoping I could get a solid answer that it doesn't bork anything else before you came around
<wallyworld_> yep, i hear ya. thanks :-) haven't looked in too much detail  yet
 * wallyworld_ now has to run away - raining hard here, first day of school year, limited parking, lots of books, will be chaos
<abentley> lifeless: When a comment is too large for us to render, I plan to provide a download link instead.  Do you think that link should be the plaintext message body or the raw email text (where applicable)?
<lifeless> abentley: I think either would be fine, but the raw email text would include email addresses which the user may have set to 'hidden' in the DB schema.
<lifeless> abentley: so I would lean towards the plaintext version
<abentley> lifeless: I prefer that, too.
<StevenK> rick_h: Yes.
* benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugtasks: 3*10^2
<cjwatson> benji: thanks for the approval of https://code.launchpad.net/~cjwatson/launchpad/kill-old-switchdbuser/+merge/89459 - do you think you could land it for me?
<benji> cjwatson: sure, let me kick it off real quick
<wgrant> cjwatson: Ah, nice, thanks!
<benji> cjwatson: torpedoes away!  I'll let you know how the tests turn out.
<cjwatson> I should get mail
<StevenK> benji: But so will ec2 ...
<benji> StevenK: I wasn't aware of that.  Cool.
<rick_h> StevenK: did my update to convoy look ok to you? The updated url routing bits?
<StevenK> rick_h: Ah, I was just thinking about that.
<sinzui> http://pastebin.ubuntu.com/814808/
<wallyworld_> rick_h: your mp looks good. will merge into mine and land today since i'm hopeful i can also get bb updated. wish me luck :-)
<rick_h_droid> wallyworld tests all passed!
<rick_h_droid> just got the email
<wallyworld_> rick_h: bloody marvellous!
<StevenK> rick_h: Link me your MP for convoy?>
<StevenK> s/>$//
<rick_h_droid> land this SOB
<wallyworld_> yep
<rick_h_droid> SteveK not handy. check in convoy for the latest bug with the MP against it
<StevenK> rick_h: Did you want a small review, by the by?
<rick_h_droid> SteveK if you've got feedback. waiting for the convoy folks to review/merge
<StevenK> rick_h: https://code.launchpad.net/~stevenk/launchpad/better-error-+archivesubscriptions/+merge/89798 is what I was talking about ... :-)
<rick_h_droid> ah, not at the moment sorry dinner time
<StevenK> rick_h: That convoy MP looks good -- I'd suggest you update the bug, which still has revno as the first directory
<rick_h_droid> ah thanks. updated the merge proposal but not the bug
<StevenK> wallyworld_: Do you want to review https://code.launchpad.net/~stevenk/launchpad/better-error-+archivesubscriptions/+merge/89798 ? wgrant helped, so he can't really review it.
<wallyworld_> StevenK: ok, will look
#launchpad-dev 2012-01-24
<wgrant> lifeless: So, access policy model.
<rick_h> StevenK: wallyworld ok, boy is in bed. You guys need anything from me then or are we all set?
<lifeless> wgrant: shure
<wgrant> lifeless: The queries end up sufficiently complex that the array indices don't seem to help.
<wgrant> lifeless: Adding an extra intermediate table to link to multiple policies might perform OK, but it's really hard to say on DF.
<lifeless> hmm
<wgrant> I have schemas + populations + queries at http://people.canonical.com/~wgrant/launchpad/access-policies
<lifeless> how do they perform
<wgrant> I can't tell.
<wgrant> DF's performance is too variable.
<lifeless> would you like me to run some shit somewhere
<wallyworld> rick_h: just waiting for the buildbot upgrade. thanks for asking. next thing is to get the convoy patch done etc but you have that in hand already
<rick_h> wallyworld: ok cool.
<wallyworld> rick_h: i merged your branch and will land mine as soon as we are ready to go. i'll let you know what transpires today
<rick_h> wallyworld: ok, sounds like a plan
<wgrant> lifeless: I'd like if you could experiment a little. There are two there for which performance on qastaging would be interesting. accesspolicy*.sql set up the schema and populate it with real data. The other two are queries for the corresponding schemas.
<lifeless> are they mutually exclusive ?
<wgrant> Yes.
<lifeless> wgrant: your tests are missing analyzes of some temp tables
<lifeless> and with-arrays has a syntax error
<lifeless> wgrant: some initial results: http://paste.ubuntu.com/814915/
<wgrant> lifeless: The two sets of queries are basically just a dumping ground for things that I tried.
<wgrant> Unlike the schema bits, they're not meant to be executed in one go verbatim.
<wgrant> Hah, one of those is for launchpad, the other for ubuntu.
<lifeless> hmm, I'm surprised at the scope fo rthe CTE working in the second half of the union
<wgrant> Where?
<wgrant> A lot of these queries are probably horrible, as I maimed them in horrible ways to try to get different plans.
<lifeless>   OR EXISTS ( WITH teams AS (SELECT team FROM TeamParticipation WHERE person = 21997)
<lifeless> ..
<lifeless> UNION
<lifeless> ...
<lifeless> teams
<lifeless> the LP with-arrays example
<wgrant> Ah, right.
<lifeless> anyhow, LP is 300ms which is reasonably snappy (for the batch)
<wgrant> Yeah, that's reasonable for LP.
<wgrant> Which is the normal case.
<wgrant> Ubuntu is another story.
<wgrant> I was able to do any LP thing in 500-700ms on DF.
<lifeless> its bringing back 5800 bugs
<wgrant> Hm, I apparently lost the original query at some point.
<lifeless> so the ubuntu case would be 100 times slower
<wgrant> That is the one with the current subscription-based queries.
<wgrant> Right, except it uses a different plan.
<lifeless> also
<lifeless> I tihnk the sort is wrong - see the sort bug I filed today
<lifeless> we do -importance, bugtask.id
<lifeless> these days, at least for some scopes.
<lifeless> 5.1 seconds for ubuntu
<lifeless>          ->  Nested Loop  (cost=0.00..4068090.66 rows=152081 width=69) (actual time=0.206..5084.616 rows=98557 loops=1)
<lifeless> this
<lifeless> ORDER BY BugTask.importance DESC, BugTask.id LIMIT 76
<lifeless> makes it
<lifeless>  Total runtime: 160.709 ms
<wgrant> orly
<lifeless> dunno how well it will scale when you're at offset 5000 (or constraints based that deep in)
<lifeless> http://paste.ubuntu.com/814929/
<wgrant> https://pastebin.canonical.com/58655/ is the current query, for comparison
<lifeless> so, thats fast because 1/2 the bugs are ubuntu bugs, and the index matches the sort, so it goes 'wha-hay, lets just walk the index'
<wgrant> Yep
<wgrant> Like we saw a couple of months abck
<lifeless> count(*) is  Total runtime: 3892.111 ms
<lifeless> but that is largely separate
<wgrant> How I envy every other bugtracker :(
<lifeless> because they don't have to handle our scale?
<wgrant> Right.
<wgrant> I guess they also lack bugtasks.
<lifeless> wgrant: I have a possible search schema
<lifeless> now where did I put it
<wgrant> That's what I was hoping :)
<lifeless> http://paste.ubuntu.com/814938/
<lifeless> wgrant: does that help ?
<wgrant> lifeless: Maybe :)
<StevenK> Hmmmmm. Looks like there is a bug with how nominations check
 * StevenK checks if he can break it, to see if he should notify and redirect
 * StevenK peers at the dev sign-in form.
<StevenK> Oh yes, wgrant ripped out passwords.
<wgrant> Yeah
<wgrant> No passwords for you.
<StevenK> Hmmmm, if wgrant isn't being evil to DF ...
<wgrant> I'm not, no.
<StevenK> wgrant: O hai, Mr OCR -- https://code.launchpad.net/~stevenk/launchpad/notification-already-targetted/+merge/89826
<wgrant> StevenK: Shouldn't those permissions work?
<wgrant> If they're defined at all for those objects, they should surely be transitive.
<StevenK> wgrant: They don't seem to, and we spent a while hammering out userHas*Privileges()
<StevenK> So we should use them ... :-)
<wgrant> Sure, but if there are incorrect launchpad.Driver and launchpad.BugSupervisor for ISourcePackage, we should probably fix that.
<wgrant> It's fine if they don't exist.
<wgrant> But not if they exist but are incorrect.
<StevenK> I think it's being called on a SSP
<wgrant> Er
<wgrant> Doesn't only code use SSP?
<wgrant> s/code/Code/
<StevenK> https://bugs.launchpad.net/ubuntu/maverick/+source/libvirt/+bug/632696/+nominate should work for me, I have bug supervisor superpowahs, right?
<_mup_> Bug #632696: libvirt won't start a VM with serial or console <amd64> <apport-bug> <maverick> <server-mrs> <libvirt (Ubuntu):Invalid> <libvirt (Ubuntu Maverick):Confirmed> < https://launchpad.net/bugs/632696 >
<wgrant> Sure.
<wgrant> But is that because the permissions are incorrect, or because they don't exist?
<wgrant> If they don
<wgrant> don't exist, this fix is correct.
<StevenK> I'm not sure how to check
<wgrant> Look for corresponding adapters.
<wgrant> Or, better still, use bin/harness and try to adapt them :)
<StevenK> You just lost me
<wgrant> Check which security adapter happens when you try to adapt ISP to launchpad.Driver or launchpad.BugSupervisor.
 * StevenK drowns in zope calls
<StevenK> And checkPermission() is utterly incomprehensible. :-(
<wgrant> IAuthorization(someobj, name='launchpad.Something')
<StevenK> I was just coming to that conclusion, due to iter_authorization.
<StevenK> In [5]: IAuthorization(spn, 'launchpad.BugSupervisor')
<StevenK> Out[5]: 'launchpad.BugSupervisor'
<wgrant> name=
<wgrant> The second argh is the default.
<StevenK> Bah
<wgrant> -h
<StevenK> TypeError: 'name' is an invalid keyword argument for this function
<wgrant> May have to use queryAdapter or getAdapter directly.
<StevenK> In [16]: auth = queryAdapter(spn, IAuthorization, 'launchpad.BugSupervisor')
<StevenK> In [17]: print auth
<StevenK> -------> print(auth)
<StevenK> None
<wgrant> spn?
<StevenK> Bleh
<StevenK> factory.makeSourcePackage(), and then the above on that results in the same
<wgrant> That suggests there's no adapter at all.
<wgrant> So your solution may be correct.
<wgrant> But check similar for launchpad.Driver
<StevenK> In [23]: auth = queryAdapter(sp, IAuthorization, 'launchpad.Driver')
<StevenK> In [24]: print(auth)
<StevenK> <lp.security.SeriesDrivers object at 0x80e1dd0>
<StevenK> wgrant: I had secessfully managed to block lib/lp/security.py from my mind, damn you.
<StevenK> There are no adapters for ISP at all
<wgrant> StevenK: Right.
<wgrant> So your description is incorrect, but your code is right.
<StevenK> wgrant: "Work around lack of ISP security adapters by using IBugTask.userHas*Privleges()" ?
<wgrant> StevenK: Something like that.
<adeuring> good morning
<rick_h> morning
<rick_h> ruh roh, did the use-convoy not make it last night?
<StevenK> rick_h: I don't think buildbot hasn't been upgraded
<StevenK> Sigh, grammer fail
<rick_h> heh, ok
<StevenK> Why is the BranchRevision SQL on the branch page taking 8.8 seconds? :-(
<wgrant> StevenK: Probably the scanner being a bit greedy with its locks.
<StevenK> And this is terrible. Thanks the base64 disease in OOPSes, you can't trigger an OOPS on one machine and then eyeball-copy the OOPS to the other.
<wgrant> It solves more problems than it causes :)
<wallyworld> rick_h: hi, still waiting on the rt to be done before we can land
<rick_h> wallyworld: ok cool
<wallyworld> i was hoping to be able to tell you it was done
<rick_h> heh, np
<rick_h> I need to work off that branch to start work on the other issues so I'm ancy. I'll be patient :)
<wallyworld> i want it done too
<wallyworld> i initially needed to update the rt with the required version info so there was a hold up there, but hopefully there's no more roadblocks
* Topic unset by gmb on #launchpad-dev
<gmb> Oh, sod it.
* gmb changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: gmb | Firefighting: - | Critical bugtasks: 3*10^2
<gmb> Although, that said...
 * gmb lunches
<tumbleweed> lifeless: you appear not to be subscribed to bug 583426. Are you ok with the dh_python2 namespace approach for lazr?
<_mup_> Bug #583426: Missing __init__.py <apport-bug> <i386> <lazr.restfulclient (Ubuntu):Confirmed> <lazr.uri (Ubuntu):Confirmed> <protobuf (Ubuntu):New> < https://launchpad.net/bugs/583426 >
<deryck> abentley, adeuring, rick_h -- hey there, guys.  did you guys hold the standup?
<abentley> deryck: yes./
<deryck> abentley, adeuring, rick_h -- shall we do it now, or in 10 at the top of the hour?
<abentley> deryck: I'm ready now
<adeuring> either time is fine for me
<rick_h> yea, fine either way
<deryck> ok, let's do it now then.
<Laney> bigjools: am I right in thinking that spph.creator wasn't there from the start of copyPackage? i.e. early syncs using that method won't have that set
<bigjools> Laney: ummmm I can't remember.  It's NULL for non-syncs for sure.  rvba, can you remember?
 * tumbleweed recalls that being the case
<rvba> Laney: bigjools yes, that's true, creator was introduced afterwards.
<Laney> hmm OK
<Laney> was wondering why we misidentified some early syncs. guess there's no way for us to tell
<tumbleweed> Laney: date range...
<Laney> tell /who/ synced
<Laney> rvba: thanks for the info
<rvba> welcome.
<deryck> abentley, I'm free now.  Sorry it took so long.  mumble or g+ hangout?
<abentley> deryck: mumble
<jcsackett> sinzui: regarding the grackle-lp-views branch. create bug to land, or just land no-qa?
<sinzui> correct
<sinzui> jcsackett, I unassigned myself from grackle-client. I need to announce the terminology/use cases decisions from Budapest, and revise  interactive mockup to hand off to Huw and Ian
<sinzui> jcsackett, I will take the card back tomorrow if all goes well
<jcsackett> sinzui: ok. when you say correct, was that to no-qa? could have applied to either clause of my question. :-P
<sinzui> jcsackett, does it fix a specific bug?
<sinzui> I think not so no-qa
<jcsackett> sinzui: that's where i was leaning, but i like second opinions. :-P
<jml> is there any auth on the LP oops system?
<jml> i.e. what mechanism do you use to prevent random bozos on the Internet submitting oopses?
<sinzui> jcsackett, I might be the wrong person to ever ask the question. I QA most of my no-qa bugs because I am paranoid. I would still visit the new view you created and verify the page title and headings
<jcsackett> sinzui: oh yeah, i'm not thinking fire and forget. just whether it's worth filing a filler-bug.
<jcsackett> i think we agree that it's not, but i totally plan on looking at the branch on qastaging.
<sinzui> jcsackett, I do not think a bug is needed. The view does not provide value to anyone yet
<abentley> jcsackett: Are you handling the case where message body is too long?
<abentley> jcsackett: I'm a bit obsessed with that right now, because that's the root cause of bug 911090
<_mup_> Bug #911090: merge proposal is broken with consistent timeouts <timeout> <Launchpad itself:Triaged by abentley> <Tarmac:New> < https://launchpad.net/bugs/911090 >
<jcsackett> abentley: our plan is to auto-truncate the body when displaying messages, iirc.
<jcsackett> ...auto-truncate is redundant.
<jcsackett> we'll truncate long messages, and allow another means of seeing the whole thing.
<jcsackett> but we're not a stage where any solution to that is set yet.
<abentley> jcsackett: Cool.  Yeah, I'm implementing that for comments now.
<jcsackett> abentley: yeah, i can see that being a pain in comments.
<abentley> jcsackett: We have three classes of message bodies: Short, long and excessive.
<abentley> jcsackett: long and excessive messages get truncated, with a download link.
<abentley> jcsackett: long messages also get a "Read more" link that lets you view the message in the Launchpad UI.
<jcsackett> abentley: that sounds like a good solution. i'll ape it so we don't have two means of dealing with the same problem. :-)
<abentley> jcsackett: Well, don't go overboard.  Since most comments can be adapted to IMessages, maybe we should just move the code to IMessage.
 * jcsackett nods
<jcsackett> abentley: when do you think you're code is likely to start landing?
<abentley> jcsackett: this week.  Tomorrow if I'm lucky.
<jcsackett> we're doing a lot of incremental landings for grackle integration, and nothing is going to get linked to and really wired in until we can integrate grackle properly, so i can probably move relevant bits into IMessage as i go.
<jcsackett> awesome. i'll take a look as i'm going then and if it works take advantage of that code.
<jcsackett> abentley: moving it into IMessage though may best wait to the day we start moving all of our message stuff into this same archive idea, which i think is longer-range than getting grackle in for mailing lists.
<abentley> jcsackett: Part 1 is here, but there's a lot of other stuff: https://code.launchpad.net/~abentley/launchpad/bugcomment-as-icomment/+merge/89497
<jcsackett> abentley: dig. i think i recall you mentioning some of this at the epic.
<abentley> rockstar: I want to turn ICodeReviewComment.message_body into a method.  It used to be one, but you changed it into a property.  Can you comment?
<abentley> rockstar: (I want to do this because properties are always rendered, and Tarmac is making 16MB message bodies.)
<rockstar> abentley, *groan* I can't comment.  It's been too long.
<abentley> rockstar: Okay.
<rockstar> I think that might have been the result of a code review, actually.
<abentley> rockstar: It didn't look like it.  I was the reviewer, and I don't remember either.
<rockstar> Oh, uh, probably ignorance on my part then.
<abentley> rockstar: r7719.1.4 if you're curious.
<rockstar> Oh, it was part of the API!
<rockstar> leonardr told me to do it that way.
<jml> flacoste: fwiw, the thing I talk about http://code.mumak.net/2012/01/undistract-me.html addresses one of your past concerns about LP's test suite
<flacoste> jml: i saw!
<abentley> rockstar: Doh!
<jml> flacoste: cool :) it's working out really well for me
<rockstar> abentley, I think there was something about the API decorators that was causing problems.
<lifeless> rick_h: hey
<lifeless> rick_h: have you looked closely at handlebars?
<lifeless> jml: our oops system is firewalled off
<lifeless> jml: we have a plan for a javascript oops system, which will let random bozos submit oopses
<lifeless> -> automated signal detection will matter, but the fact is random bozos will need access (can't prove a browser was on our website...)
<lifeless> abentley: perhaps we should reject the 16MB messages (and offer attachments instead) ?
<abentley> lifeless: We generally render textual attachments, so I don't think that helps, and I prefer systems that don't have discontinuities.
<lifeless> abentley: we don't render textual attachments in bugs - at all
<lifeless> abentley: diffs are a dicontinuity already :)
<abentley> lifeless: I'm talking about code review comments.
<rick_h> lifeless: I'm using it for my personal app. I'm liking it so far.
<rick_h> lifeless: I've not looked at the source much though, so could be evil, but doubt it if YUI is building it into 3.5
<sinzui> jcsackett, mumble?
<deryck> G'night, all.
<wallyworld> rick_h: use-convoy branch now in ec2 for real :-D
<StevenK> It was for real last time, too. Now it will land if the tests passs.
<StevenK> s/\(ss\)s/\1/
<wallyworld> StevenK: that's what i meant by "for real"
<StevenK> Hah, another vuln in wordpress <= 3.3.1
<rick_h_droid> awesome
<rick_h_droid> stevek I might need a hm. f
<rick_h_droid> bah
<rick_h_droid> a hand with some make/buildout magic to help tests setup
<rick_h_droid> I think we're still going to want a copy of yui and such in build/js
<wallyworld> rick_h: so, you have the convoy patch sorted, and are now looking at the tests? the first thing i'd do is change the yui seed script location in all the html files  so that it doesn't use the symlink in icing
<rick_h_droid> wallyword yea, I had a discussion with deryck on it today. I think the key will be pointing all seed files at the build js directory and create a symlink there to change yui versions as required
<lifeless> rick_h: hey hey
<rick_h_droid> howdy
<lifeless> rick_h: so yeah, I've been looking at it to asses a pybars port
<rick_h> lifeless: ah, nice
<wallyworld> rick_h: what you say above about yui tests matches how i envisaged it would work also
<rick_h> wallyworld: yea, so I'm working on my make/buildbot skills because right now things are built straight into the convoy directory
<rick_h> and we'll need a new command to move the symlink in the build/js directory from one version of YUI to another and such
<rick_h> I did it with a bunch of magic JS code today and deryck suggested I might have overcomplicated the issue a little bit :)
<wallyworld> yeah, kiss is good
<rick_h> it's ok, I got the oreilly gnu make book and I'm starting to try to make it less scary for me
<wallyworld> a goal here for me at least is to totally remove the yui symlink and anu other build cruft from the icing dir
<rick_h> right, I'm making sure nothing but the link to combo.css comes from icing any more
<wallyworld> sounds good
<rick_h> anyway, work for tomorrow. Good luck with the landing
<rick_h> time to work on getting the boy to bed
<wallyworld> thanks :-) will let you know
<lifeless> rick_h: blah, elocal
<lifeless> rick_h: so, it seems to me that handlebars has kindof missed the 'separation of code and presentation' angle of moustache
<lifeless> rick_h: I was wondering if you had any insight into that
<StevenK> wgrant: QA for r14717
<rick_h> lifeless: I find it provides a decent middle ground. You can define new blocks and helpers
<rick_h> but it's still fairly limiting as a template language goes
<rick_h> I mean, I know there's a very anti-code in the templates kind of thing, but I also feel that view belongs in the view (template) and things like concat'ing, prettying dates, etc aren't model duties
<rick_h> lifeless: which part do you find crosses the border?
<StevenK> wallyworld: Can you look at kanban and see if any cards can move? I know the convoy one can't yet, but some others may be able to
<wallyworld> ok
<wallyworld> StevenK: the two cards in landing are part of the use-convoy branch, so there's nothing obvious to move
<StevenK> wallyworld: Right
<lifeless> rick_h: e.g. on here http://handlebarsjs.com/ - the 'list' helper.
<lifeless> rick_h: seems like a terrible reason to have that facility.
<StevenK> wgrant: I have a deployment request prepared for when we're good.
#launchpad-dev 2012-01-25
<rick_h> lifeless: ok, that looks a bit nutty and I can't think of a real use case I'd do that atm
<rick_h> lifeless: but if you go through to the block helpers I think that idea is sound to be able to help with the iteration and such.
<rick_h> that's where I get into I like a tpl tool that lets me build helpers to be used site wide for consistancy
<rick_h> so for instance, I could seee creating a block that's part conditional to help make sure output without any data provides a decent "no results found" consistant bit of html
<lifeless> sure
<lifeless> does moustache allow nested templates?
<lifeless> (because that seems both less general (and more optimisable) and a bit easier to reason about
 * rick_h dbl checks, I didn't hink so but it's on the ticket "wishlist"
<lifeless> anyhow
<lifeless> having looked at it
<lifeless> I think we can easily implement a handlebars compiler
<rick_h> lifeless: looks like it's got "partials" which are kind of like it
<lifeless> obviously the extension bits are going to need some idiomatic translation to python
<lifeless> depending on how efficient we want the compiler etc
<rick_h> lifeless: but yea, the one thing to think about is that when you get to doing a lot of client side rendering you're getting dump data dumps from an api and you're not going to get a ton of logic ability from that end
<lifeless> rick_h: well, I want LP's main tier to also be just an API consumer :)
<lifeless> so I don't see a great deal of tension there
<rick_h> lifeless: ok
<lifeless> rick_h: one of the nasty tricky performance things zope has, for instance
<lifeless> is this model:
<lifeless> outer template, inner template for e.g. 'view of a bug'
<rick_h> lifeless: end of the day I'm not the best person to ask. I'm a fan of SqlAlchemy, Mako, Pyramid, things that give me enough rope to shoot myself, but I can't stand a tool that tops out and I end up doing stupid hacks around the tool to get something done.
<lifeless> the outer template iterating over views of the inner template
<StevenK> rick_h: You shoot yourself with rope? Really?
<lifeless> indeed, totally with you on that
<rick_h> StevenK: heh, cold meds are going to my head finally :)
<StevenK> Haha
<lifeless> rick_h: ubuflu?
<StevenK> rick_h: Ugh, you're sick too?
<rick_h> lifeless: yea, and I do some of that in my app be nesting views in views
<rick_h> not sure, might be toddler disease
<rick_h> he had pink eye over the weekend
<lifeless> rick_h: this tends to perform terribly unless you decide upfront what is needed
<rick_h> and today I'm running on E
<rick_h> lifeless: right, it's much more memory intensive and such
 * StevenK raises one eyebrow
<lifeless> rick_h: which to me, speaks to having a single simple top level template, more or less
<lifeless> rick_h: not memory so much - late evaluation / lazy loading
<lifeless> rick_h: death by 1000 cuts
<rick_h> lifeless: right, but then you're template is going to have to be smarter in order to intellegently handle corner cases/etc
<lifeless> rick_h: something, yes
<lifeless> rick_h: I guess I'm not satisfied with the answers we have yet
<rick_h> lifeless: let's find a use case and go to town :)
<lifeless> rick_h: and I'm trying to figure out if we want to say 'use moustache (and no handlebars features)' or 'use handlebars (and port to python)' or 'lets roll a third thing'
<rick_h> lifeless: yea, and not sure we will. These decisions tend to show if they work/not more in real use cases with real fire going by
 * StevenK spies rick_h as the last requested deployment
<rick_h> StevenK: yea, I did my first NDT request :)
<StevenK> rick_h: You found it was pretty easy?
<rick_h> lifeless: right, I've hit limitations in mustache and especially issues with pystache so I'm pretty much in the "anything but stache" camp atm :)
<rick_h> helpful, aren't I
<rick_h> StevenK: well I screwed up, I thought approval was from webops and I missed getting teh bugs linked/etc
<rick_h> but after a few tries and wgrant holding my hand I got it
<wgrant> :)
<wgrant> lifeless: ha hahhhahahh hah '
<wgrant> fewfw
<wgrant> fwef
<wgrant> So, as it turns out, the date logic is fine.
 * wgrant backs up the oopses and tries it.
<lifeless> rick_h: conceptual or implementation issues?
<rick_h> lifeless: implementation for the most part.
<StevenK> wgrant: Then what's the bug?
<rick_h> it's slow, the pystache implementation is broken compared to JS, and I don't see how we'd port over a lot of the .pt logic over to it (so that part is conceptually)
<lifeless> rick_h: by .pt you mean [me]tal templates
<lifeless> ?
<lifeless> rick_h: the js version or the py version is slow?
<lifeless> rick_h: the py version looks horrendously slow: see _render_sections
<rick_h> lifeless: yea, a lot of things like the macros and such you'd have to look at porting over
<lifeless> right
<lifeless> I think there is substantial work to do
<lifeless> However, what we have now with a mix of:
<lifeless>  - hand rendered
<lifeless>  - tal
<lifeless>  - metal
<lifeless>  - stache
<lifeless>  - chameleon
<lifeless> is pretty much unmaintainable
<rick_h> heh, you won't find me arguing there
<lifeless> so I think for now the best solution is to say 'take stache and run with it *gathering up data where it does not fit well*'
<rick_h> ok, time to go take more meds and hit the pillow. Sorry.
<lifeless> I'm open to the idea of different (tiny) DSL's for different problem domains
<lifeless> ciao, get well
<rick_h> lifeless: understand
<StevenK> lifeless: You have mostly recovered?
<lifeless> StevenK: yeah, still a bit snuffly & cold sweats but nothing like I was
<lifeless> StevenK: lynne is coming down with it now though :(
<StevenK> :-(
<StevenK> I'm still coughing, but due to asmtha, it takes me a while to shake them off
<lifeless> yeah, me too - asmtha sucks
 * wgrant escaped the plague this time.
 * StevenK ships wallyworld to wgrant's house so he can share in the misery.
<wgrant> Although I did get whooping cough in Wellington, so I guess it's fair.
<lifeless> https://dev.launchpad.net/PageTemplateHacking
<wallyworld> i love sharing
<lifeless> this talks about mochikit
<lifeless> :(
<wgrant> lifeless: :D
<StevenK> lifeless: Hey, we got Mochi out of the codebase, it can be up to others to remove it from the docs
 * wallyworld didn't even know there were docs
<lifeless> wgrant: so, not date ranges.
<lifeless> there is also this page https://dev.launchpad.net/Web/TemplateCodeReuse (it doesn't talk about mochikit, but abou ttemplates)
<wgrant> lifeless: The fix is http://paste.ubuntu.com/815998/ :)
 * lifeless headdesks
<wgrant> Myes.
<lifeless> wgrant: +1
<lifeless> sorry
<wgrant> Currently testing on a hardlinked copy of the production oopses.
<wgrant> Seems to be working.
<StevenK> Hardlinks?
<StevenK> You bad man.
<wgrant> With 6GB of free space and 100GB of OOPSes, I had little choice :)
<lifeless> StevenK: get well :)
<james_w> lifeless, did you have an opinion on whether oops should change bson libraries?
<lifeless> nope
<lifeless> (no opinion)
<lifeless> james_w: ^
<james_w> ok
<james_w> it's rather annoying having the two of them
<lifeless> I agree
<lifeless> I think a patch to work with either would be pretty easy to do
<lifeless> and avoid the question entirely
<lifeless> well, it would leave open questions for debugging like 'which one does THIS WEIRD THING' etc
<lifeless> james_w: all I did was grab the first one that I found on pypi
<james_w> yeah, it's understandable
<james_w> unfortunately it's the pymongo one that got packaged
<lifeless> its daft that the mongo guys didn't register the name on pypi
<lifeless> daft and ANNOYING
<wgrant> And very much in the style of the rest of the NoSQL DBs :)
<james_w> well, it's another top level in pymongo which is registered, so it's annoying that pypi lets this happen
<lifeless> james_w: how do you mean ?
<james_w> http://pypi.python.org/pypi/pymongo
<wgrant> That's insane.
<james_w> if you pip install pymongo you get bson too
<lifeless> aiee
<james_w> and nothing complains at any point
<wgrant> :D
<lifeless> so thats not pypi's problem
<lifeless> it /could/ be, but it isn't :)
<james_w> and apparently "pip install pymongo; pip install bson" != "pip install bson; pip install pymongo"
<lifeless> of course it isn't.
<lifeless> the pymongo packaging is doing something nobody expects
<lifeless> (which you know, and I know you know :P)
<lifeless> anyhow
<lifeless> like I say, I have no opinion
<lifeless> whatever works
<wgrant> lifeless: btw, pycassa 1.4 (which I packaged yesterday) incorporates the concurrent schema change workaround that you have in oops-repository.
<lifeless> wgrant: \o/
<wgrant> It was also only released a day or two ago, but it seems to work fine.
<lifeless> I can't remember if I filed that as a bug or whinged in private email or whatever.
<wgrant> I also packaged new python-thrift and cassandra, because having an outdated new stack seems silly.
<lifeless> but its cool they have seen sanity
<wgrant> Not sure what U1DB are doing.
<lifeless> wgrant: have you tossed a delta to elmo ?
<wgrant> But I couldn't see anything even vaguely modern in PPAs.
<wgrant> Not yet, no.
<lifeless> he might appreciate that
<lifeless> he was mumbling about updating
<wgrant> Yeah.
<wgrant> I think I might drop support for <precise or at least <oneiric and try to get rid of some of the jars.
<wgrant> elmo's already only support >= natty.
<wgrant> Because JAVA! :D
<lifeless> I'd check with elmo about that (so that we can share packaging)
<wgrant> Of course.
<wgrant> But everything relevant should be precise in like 4 months.
<wgrant> So I don't imagine too much resistance.
<wgrant> Hmm.
<wgrant> Is our bson library very slow, I wonder.
<wgrant> Because this prune is taking forever.
<wgrant> It's only doing a few per second.
<james_w> anyone looked at https://bugs.launchpad.net/python-oops-tools/+bug/903573 at all?
<_mup_> Bug #903573: TypeError: start_response() takes no keyword arguments <python-oops-tools:Triaged> < https://launchpad.net/bugs/903573 >
<james_w> wgrant, jam says it is slow
<wgrant> :(
<lifeless> james_w: that looks like the broken django stack
<james_w> lifeless, I don't think so
<james_w> the django.py replaces a different method
<lifeless> bah
<lifeless> I meant mod_wsgi
<james_w> yeah, that's what I'm thinking
<james_w> but I think "broken" may just be "old"
<lifeless> well
<lifeless> WSGI has been like this for a -long- time
<lifeless> so what is happening here is that an oops in oops-tools itself is generated and is trying to be shown to the user, but mod_wsgi is barfing
<james_w> yes
<james_w> ah, it's a bug in oops-wsgi by the look of it
<lifeless> oh?
<james_w> I'll send an MP in a minute
<lifeless> cool; could you use english in the meantime?
<james_w> it's pretty simple
<james_w> there are two locations where it calls start_response with exc_info
<james_w> one passes it positionally, the other as a keyword
<james_w> the latter is wrong apparently
<lifeless> bah
<lifeless> thanks for spotting that
<lifeless> '(As with all WSGI callables, the arguments must be supplied positionally, not by keyword.) ' I swear that wasn't in PEP 3333 last time I looked
<james_w> lifeless, have you looked in to how to encourage django to use a response with the oops id on an uncaught exception?
<james_w> if you don't provide a 500 template then you get the oops one, but the oops it points to is an exception complaining that you don't have a 500 template
<lifeless> jamesh: has
<lifeless> oh, for oops-tools itself I know we need to provide a 500 template
<lifeless> there is a patch for django, last I checked it was a bit stalled
<lifeless> https://code.djangoproject.com/ticket/16674
<james_w> oh, I thought that was what the django.py fixed
<lifeless> it works around it
<james_w> http://ec2-107-22-92-8.compute-1.amazonaws.com/pkgme/+oops is what I'm getting currently
<lifeless> https://code.djangoproject.com/attachment/ticket/16674/wsgi-expose-exc-info-v2.patch
<lifeless> thats the actual fix
<lifeless> the django module in oops-wsgi sniffs the exception to ensure you get an oops
<lifeless> but it can't influence django to make it expose the error to the wsgi stack
<jamesh> james_w: you mean to display the OOPS ID on the error page?
<lifeless> so the main oops module just sees a successful reponse
<james_w> jamesh, yeah
<james_w> lifeless, ah, I see
<jamesh> james_w: I haven't been able to work out a good way to solve that with the new infrastructure
<james_w> so if it were properly fixed then oops' response would take over
<james_w> ok, it's not so important to us at this point
<lifeless> james_w: do you get an OOPS emitted to (rabbit|disk)
<james_w> lifeless, yep
<jamesh> the old oops code we have in U1 can preallocate the oops IDs on demand, so we can use that in the error page
<lifeless> jamesh: we could allow that too, if you wanted to use a timeuuid or something, and put the id in the context
<jamesh> one idea I had for the new code might be to set a short lived cookie in the WSGI middleware, and look that up with JS in the error page
<lifeless> jamesh: ooh, nice.
<jamesh> that has the downside that cookies are shared between all tabs in a browser
<james_w> lifeless, for this patch, how much do you care about a test?
<james_w> it's not the easiest thing to test after all
<lifeless> 'meh'
<jamesh> so if you open 20 pages at once and five fail, you might see the same OOPS ID on each
<lifeless> yeah
<lifeless> uhm
<lifeless> wonder if you can inspect the response headers from the page itself
<lifeless> jamesh: shoving id in the context would be fine.
<lifeless> james_w: should be easy to test if you want to; just have an outer start_response(*args, **kwargs) and check kwargs is empty
<james_w> lifeless, ah, good point
<jamesh> I don't think there is a way to inspect response headers unfortunately
<lifeless> so, a django error template that puts content['oops.context']['id'] = str(uuid.uuid1()) => win
<lifeless> jamesh: ^
<wgrant> Hmm
<wgrant> Regression
<wgrant> 5633 AssertionError: Ambiguous view name.
<wgrant>     GET: 5629 Other: 4 Robots: 98  Local: 239
<wgrant>      168 http://feeds.launchpad.net/%7Ebarry/latest-bugs.atom (Person:latest-bugs.atom)
<wgrant>         OOPS-00033c67366d1e7513397953f71c9b29, OOPS-001af265e7d5322fdd12c9b12b4fed8c, OOPS-00ae6fedb93aa14a7fc508d51b9fa6a7, OOPS-010f606986cdc5008d999fabe085e6aa, OOPS-01432c961a9928b26bb0d210d723a53b
<wgrant> Ah
<wgrant> I bet it's the bug listings release
<wgrant> It doesn't work with feeds.
<wgrant> Which are always anonymous, so were never tested.
<wgrant> lifeless: ^^
<lifeless> grah
<lifeless> I guess we need to rollback
<jamesh> lifeless: I guess that would work.  Do we lose anything by having the view assign an ID instead of the publisher doing so?
<wgrant> lifeless: It's been broken for a day, and it's a major thing.
<wgrant> "it" == bug listings
<wgrant> So I think we should keep it on.
<wgrant> It's not as if anybody uses feeds anyway.
<lifeless> well, 5633 polls for them
<lifeless> wgrant: lets start with a bug
<lifeless> jamesh: I think its fine to allow the earliest point that knows an error is on the way to assign an id - the hash based aproach is just one way to get a unique id
<lifeless> jamesh: oops can render errors itself
<lifeless> jamesh: or you could pass a django template into oops
<lifeless> jamesh: or you can assign the id earlier, if the framework really wants to render its own errors
<lifeless> jamesh: the existing facilities are the trivial string template that oops has itself or a callback you can supply to the middleware constructor
<jamesh> lifeless: I think giving frameworks the ability to render their own error pages makes python-oops-wsgi more appealing, since it can be dropped into an existing app
<lifeless> jamesh: I think the first thing I would attempt would be to discard the django error page but wire up the callback to let me use a django template to render the error page
<james_w> so, I can't do this patch as I am yet to get close to having a working buildout
<wgrant> lifeless: Hm
<lifeless> james_w: oh? what happens?
<wgrant> 'bugs.dynamic_bug_listings.enabled pageid:Person:latest-bugs.atom 2 '?
<lifeless> wgrant: might work, we should try
<wgrant> That seems to be the only popular affected page.
<jamesh> lifeless: well, once we've got that django bug report I filed fixed, that should be possible: if start_response() is called with an exc_info, just re-raise it
<lifeless> jamesh: that seems to be slightly different
<jamesh> then present an error page after the exception bubbles back up to the middleware
<lifeless> oh right
<lifeless> so yes, get the middleware's start_response to be called with exc_info
<jamesh> with that in mind, the debug error page Django presents is pretty nice
<lifeless> ah, make_app(template=...)
<jamesh> so when developing an app, I'd prefer to see that than an oops-wsgi error page
<lifeless> jamesh: sure, but not for deployed apps ?
<lifeless> jamesh: so have dev mode eat the exceptions and render itself
<lifeless> jamesh: anyhow, I'm happy to let the app allocate ids
<lifeless> pragmatic
<james_w> lifeless, confusing error messages
<jamesh> lifeless: I'm of two minds there.  The error pages we've got wired up in U1 don't do much , so would be pretty easy to reimplement in the middleware
<lifeless> james_w: pastebin ?
<james_w> apparently when it said I might have misspelled a dependency, it really meant that it couldn't install from the cache because the cache is currently empty
<lifeless> james_w: you're using the LP cache, right ?
<jamesh> on the other hand, if I could just get/create an OOPS ID in the existing error page code, that would be a much smaller change
<james_w> lifeless, nope
<lifeless> james_w: use the LP cache
<jamesh> and I suspect other people trying to integrate python-oops would feel the same
<lifeless> jamesh: these are not exclusive options
<jamesh> lifeless: I understand that.
<lifeless> :)
<lifeless> I think we should make it easy to get going
<jamesh> I'm just thinking out loud about why I might pick one option over the other
<lifeless> with better/more organised stuff being something you can grow into
<lifeless> jamesh: I'm glad you are doing so :) I was just adding commentary
<lifeless> my cat wants to eat my galaxy tab
<lifeless> its a little disconcerting
<james_w> lifeless, https://code.launchpad.net/~james-w/python-oops-wsgi/start_response-exc_info/+merge/90031
<lifeless> jthanks; I have some feedback
<lifeless> its a little more than trivial, or I'd just do it
<StevenK> wgrant: Have you seen bug 921175?
<_mup_> Bug #921175: Bug overview page tag links are not properly escaped, thus made useless <404> <bugtag> <tales> <Launchpad itself:Triaged> < https://launchpad.net/bugs/921175 >
<StevenK> And no, it's not a dupe. :-(
<wgrant> I have indeed.
<wgrant> lifeless: How should I do the case-insensitivity in python-oops-tools?
<wgrant> There's no iin :)
<wgrant> And I don't seem to be able to lower() or upper() it first.
<lifeless> wgrant: this is the db query ?
<wgrant> yes.
<wgrant>             to_delete = set(Oops.objects.filter(
<StevenK> I don't want to perform the same fix in the same way within a few days. :-(
<wgrant>                 date__gte=prune_from, date__lte=prune_until).exclude(
<wgrant>                     oopsid__in=references)[:10000])
<lifeless> wgrant: I'm curious what is mangling the case in the first place; is it just fimble-fingered copy-pastes ?
<wgrant> lifeless: Hmm?
<wgrant> lifeless: Your LP API upper()s them before they're returned.
<lifeless> wgrant: it does? Ok, so I am stupid.
<lifeless> I have -no- idea what I was thinking.
<lifeless> just checking the schema
<wgrant> oops-tools was already upper()ing them in parts.
<StevenK> wgrant: So I guess I write a IBugTag and then a tales adapter?
<lifeless> wgrant: that was eliminated a while ago
<wgrant> StevenK: No. There's no sensible way to do a fmt:url here.
<lifeless> wgrant: it does that on input only, for old-style oopses
<lifeless> wgrant: the db is case sensitive, and indexed case sensitively
<wgrant> lifeless: Ah, right, I remember that now.
<lifeless> wgrant: I'd fix the API TBH
<wgrant> I only didn't fix it because I presumed you had your reasons :)
<wgrant> I'll not land the datedir-repo fix, then.
 * lifeless is apparently a camel toenail smoking crackhead
<wgrant> Just cowboy it and prune to get us out of immediate peril..
<wgrant> And fix the API this afternoon :)
<StevenK> wgrant: There isn't? We can create an object with the context and the tag and we can construct an URL quite easily?
<lifeless> which is another way to say I have -no- idea why I did that. Probably to make some test pass or something
<wgrant> StevenK: FSPOEVO "quite easily"
<StevenK> wgrant: It's the same code as the view uses? canonical_url() + urllib.quote() ?
<wgrant> POE == Probably Over-Engineered
<wgrant> Introducing a new content class and interface for this seems like utter overkill.
<wgrant> Particularly since it's just a special case of a search.
<lifeless> what I don't understand
<lifeless> is how the tags are getting into the template without escaping
<lifeless> are we manually rendering the links?
<wgrant> lifeless: They're (HT|X)ML-escaped.
<wgrant> They're not URL-escaped.
<lifeless> ah
<lifeless> fairy nuff
<wgrant> People don't understand escaping :(
<lifeless> well
<lifeless> doesn't help that two different escape rules are needed
<lifeless> because two different bodies created the dsl's for presentation and linking
<wgrant> Well, our code shouldn't lead us to sin.
<wgrant> We shouldn't have to think about escaping :)
<wgrant> Anyway.
<wgrant> Time to prune 70GB of OOPSes.
<wgrant> forrealz
<StevenK> wgrant: I'm just unhappy that I you're suggesting I propagate my horrible python:tag[x] madness from the last fix.
<wgrant> StevenK: You could do it in the view if you want.
<wgrant> And for this you will probably have to.
<wgrant> Because of mustache.
<StevenK> wgrant: And then use structure and you'll probably kill me
<wgrant> But introducing 100 lines of code where three will do is silly.
<wgrant> No structure.
<StevenK> TBH, I've not looked at the the view or mustache template
<lifeless> moustache renders on the client side
<lifeless> so you'll have to fix the data dict it renders from (or the moustache template itself)
<lifeless> it also renders server side :)
* StevenK changed the topic of #launchpad-dev to:  https://dev.launchpad.net/ | On call reviewer: StevenK | Firefighting: - | Critical bugtasks: 3*10^2
<wgrant> Yay
<wgrant> 360 criticals
<wgrant> We can round up!
* lifeless changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: StevenK | Firefighting: - | Critical bugtasks: 4*10^2
<nigelb> :|
<nigelb> what's wrong with 3*10^2+60
<wgrant> om nom nom oopses
<lifeless> nigelb: future proofing
<StevenK>  /wrists
<StevenK> nigelb: 3.6*10^2, ITYM?
<nigelb> ooh. that wworks too!
<StevenK> nigelb: Haha. Haddin comes out and hits a six for his first shot
<nigelb> StevenK: For a guy who doesn't like cricket, you sure watch lot of it...
<StevenK> Australia is win, incredibly comfortably, so I like it.
<StevenK> s/win/winning/
<nigelb> haha, troll!
<nigelb> StevenK: out! :D
<StevenK> Yes, bah.
<nigelb> I like how that happened just after you trolled :D
 * StevenK sends Ponting a cranky e-mail
 * StevenK tries to work out *HTF* this mustache template is populated.
<mwhudson> australia are practically collapsing now
<nigelb> StevenK jinxed it!
<StevenK> mwhudson: And you love it
<mwhudson> heh well
<mwhudson> this series is not especially interesting any more
<StevenK> mwhudson: So tell me, do you go for NZ as well, or just whoever is playing Australia?
<nigelb> lol
<nigelb> I'm going to save these logs to show back to StevenK when he says he doesn't like cricket the next time
<StevenK> nigelb: I can't just sit and watch a full test match, because I *will* go mad. But I can wonder past and see what the score is every 15-20 or so
<nigelb> I can't watch a full test match ever.
<nigelb> My roommates can watch a full test match and the highlights :/
<nigelb> s/can/will/g
<StevenK> Haha
<StevenK> nigelb: TBH, this isn't so much a test match as a routing
<nigelb> Hah.
<nigelb> we need #ubuntu-cricket :D
<wgrant> StevenK, lifeless: https://code.launchpad.net/~wgrant/launchpad/bug-911520/+merge/90040
<StevenK> wgrant: r=me, but no fair including my complaints in it
<wgrant> Bah, I missed one.
<wgrant> I removed a couple of irrelevant lines.
<wallyworld> StevenK: In Python 2.6, round(0.29999999, 5) = 0.29999    whereas it works in Python 2.7 ie gives 0.3 :-(
<StevenK> Bwaha
<wallyworld> so i'm commenting a line of a doc test so that use-convoy can lp-land
<wallyworld> that was the only ec2 land failure
<StevenK> Don't bother
<wgrant> wallyworld: that's a new test?
<wallyworld> wgrant: nope.  that's why i'm very confused. lib/lp/app/widgets/doc/location-widget.txt
<wallyworld> we didn't touch that file
<wallyworld> apart from trying to get it to pass testing
<wgrant> Don't randomly comment out bits of tests because you don't understand why they're broken :)
<wallyworld> very strange
<wgrant> It will fail on your machine.
<wgrant> But should pass on ec2.
<wallyworld> nope. fails on ec2, passes locally
<wallyworld> and it's a one line, meaningless test
<wallyworld> we need to get this landed
<wallyworld> will fix after
<wgrant> Can you pastebin the failure?
<wgrant> I have a change in my 2.7-fixes-0 branch to fix that very test.
<wgrant> But we need to work out why it's failing for you.
<wallyworld> https://pastebin.canonical.com/58721/
<wallyworld> it works for me
<wallyworld> it doesn't work for ec2
<wallyworld> also works for rick
<wgrant> Confused
<wgrant> That test does not match devel
<wgrant>     >>> widget.center_lng
<wgrant>     0.29...
<wgrant> That's devel.
<wgrant> So you have changed the file, to something that only works in 2.7.
<wallyworld> yes, and it failed the first time because it wanted 0.3
<wgrant> It didn't :)
<wgrant> It only does that on 2.7
<wgrant> Which ec2 is not yet.
<wallyworld> we only changed it because ec2 complained from memory, but not sure now
<wallyworld> but round(0.299999) should always give 0.3 ffs
<wallyworld> regardless of pythin version
<wgrant> round returns a float, not a str
<wallyworld> which should print sensibly you would think
<wgrant> floats are a little more complicated than that :)
<wallyworld> sure, but a rounded value should print out as expected
<wallyworld> i'm not sure what rick is running, 2.6 or 2.7
<wgrant> wallyworld: What does rounding mean for an IEEE754 number?
<wgrant> It's a float, not a decimal.
<wgrant> You'd have to somehow work out the shortest sequence of digits that lossily evaluates to the same thing.
<wgrant> Which I guess must be what 2.7 does.
<wgrant> But that has little to do with rounding.
<wallyworld> guess so. but i'm still surprised given the low percison it is being rounded to that it doesn't work as expected
<wallyworld> we still have more cleanup to do, so we'll fix it as part of that. it's one line of a test for a very peripheral feature
<wgrant> http://bugs.python.org/issue1580
<wallyworld> seems like a right royal mess
 * StevenK grumbles at the RMS
<StevenK> (New name for the RTA)
<wgrant> Non-base-10 floats are, yes.
<wallyworld> StevenK: can you land ~wallyworld/launchpad/use-convoy for me? pqm-submit doesn't work either :-(
<StevenK> wallyworld: What's the error?
<wallyworld> same thing as with lp-land
<wallyworld> missing get function
<StevenK> Didn't the update of bzr-pqm fix it?
<wallyworld> i had thought so, but then i apt updated. and it seems broken again. but not sure. let me check
<StevenK> wallyworld: Are you right for me to land this?
<wallyworld> StevenK: just updated my .bazaar/plugins/pgm checkout and there was one update but it's still broken
<wallyworld> StevenK: yes please
 * StevenK sighs at lp-land and branches it locally
<wallyworld> don't forget to edit the commit message to remove the duplicated tags
<wallyworld> that lp-land puts there
<StevenK> Yes, thanks, I've used it before.
<wallyworld> sorry, didn't mean to tell you to suck eggs
<StevenK> My tree is WADLing
<wallyworld> StevenK: this branch includes a make switch to disable the WADL stuff
<StevenK> Yes, I know that too. I've read the entire MP.
<wallyworld> ok, i'll shutup now
<StevenK> I don't believe that. :-P
<wallyworld> well, stranger things have happened
<StevenK> wallyworld: Merged.
<StevenK> wallyworld: Did you want the PQM message?
<wallyworld> \o/ thanks :-)
<wallyworld> nah
<nigelb> oh, one of those days when I wwant to have a bash.org like site for LP.
<StevenK> r14722 for reference
<wallyworld> StevenK: thanks. just in time for bb too :-)
<StevenK> What? Oh crap
<wallyworld> uh oh
<wallyworld> ?
<StevenK> Oh well, WCPGW
<wallyworld> you mean with buildbot?
<nigelb> StevenK: Famous last words.
<StevenK> wallyworld: I mean if it fails buildbot, it gets reverted
<StevenK> nigelb: Yes, that's why I said them
<wallyworld> let's hope the bb updates worked then i guess
<nigelb> StevenK: Also, "lets see what this button does"
<StevenK> It makes the Indian cricket team lose every mat.... OH.
<nigelb> dammit.
<wallyworld> StevenK:  at least it will fail during make so we'll know soon enough
<StevenK> Damn it, laughing a lot sends me into a coughing fit
 * wallyworld goes to get the kid from school and hopes bb is still green when he gets back
 * StevenK can't work out how this mustache template is populated
<wgrant> StevenK: Find both things that render it and see where they get their data? :)
<StevenK> wgrant: I just found it
<StevenK> -            'tags': [{'url': base_tag_url + tag, 'tag': tag}
<StevenK> +            'tags': [{'url': base_tag_url + urllib.quote(tag), 'tag': tag}
<wgrant> Right.
 * StevenK stabs the messaging indicator more
<wgrant> Ewww
<wgrant> lp.mustache
<wgrant> Is there not a way to do that that isn't a crime against humanity?
<StevenK> I already asked
<StevenK> But yes, it's incredibly disgusting
<wgrant> wallyworld: Some bits use YUI.add, some YUI().add. Why?
<StevenK> Sigh. Python needs Data::Dumper
<StevenK> And no, pprint doesn't count, it's utterly poor in comparsion.
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/link-bug-tags-correctly-redux/+merge/90044
<wgrant> StevenK: Done
<StevenK> wgrant: Thanks
<wallyworld> wgrant: they are not supposed to use different forms. i thought i had found and fixed them all
<wallyworld> the codebase was a bit of a consistency mess
<wgrant> Heh, yes.
<wgrant> It's certainly no worse than it was.
<wgrant> I just hoped it would make it perfect :)
<wallyworld> it will be. i'm pissed that i missed some
<wallyworld> i'll let rick know and he can clean up during his day
<wgrant> That's nice of you :P
<wallyworld> well, we want it done asap :-)
<wallyworld> and he's working on the next steps
<wallyworld> so it won't be much to fix
<wallyworld> if he can't then i will for sure
<wgrant> Yup
<wallyworld> at least buildbot is running the tests, so that's one hurdle overcome
<wallyworld> meaning the bb update worked :-)
<stub> I'm adding Lucid PG 9.1 to the PPA and will be adding PG 9.1 to the database-deps
<wgrant> stub: Slony likes it now?
<wgrant> I'm not sure we can sensibly upgrade until we acquire more CPUs :/
<stub> Good enough to get started - fixes are in, might need to upgrade our slony if we see the 9.1 issues
<stub> Why do we need more CPUs?
<wgrant> stub: Unless we want to do an in-place upgrade on wildcherry, we'd have to switch the master for a day.
<stub> Yes. I'm investigating inplace. Looks fine except for plpython stored procedures, so I need to investigate if the workarounds are sane.
<wgrant> Seems a bit sad to do that.
<wgrant> But I guess.
<stub> (perhaps waiting for PG to fix the plpython pg_upgrade bug is best - discussions ramping up again after the break)
<stub> The work around is creating a symlink. I need to determine if that is going to break future inplace upgrades
<stub> In place isn't sad - looks like minutes, but of course will need to confirm timings on staging once I've got the glitches ironed out
<wgrant> Well, we would presumably miss out on some of the improvements on unrebuilt DBs.
<stub> But before we go upgrading production and staging, we need the test suite passing
<wgrant> Bah.
<wgrant> Where's the fun in that.
<stub> Yes, bloated would remain bloated.
<wgrant> Is there anything much apart from the string escaping thing?
<stub> Some minor work in database/schema so far, nothing unexpected. I want to get ec2 able to run the test suite since I can't be arsed running the full test suite locally.
<stub> lp:~stub/launchpad/postgresql-9.1
<wgrant> Ah, not too bad.
<adeuring> good morning
<lifeless> wgrant: we don't have capacity to switch masters atm
<wgrant> lifeless: That was why I asserted that we couldn't sensibly upgrade until we acquired more CPUs.
<wgrant> I guess we need to get used to in-place upgrades if we're going to drop slony at some point.
<wallyworld> rick_h: you thre yet?
<rick_h> wallyworld: what's up?
<wallyworld> rick_h: THE branch is almost through buildbot :-D
<rick_h> yay
<wallyworld> i missed a couple of places where it used YUI().add instead of YUI.add
<rick_h> that's ok, YUI().add should be safe
<wallyworld> yeah, but it would be cool to clean up
<rick_h> k
<wallyworld> also, i had to disable a test
<wallyworld> you know that location-widget.txt test where round() was used?
<rick_h> ok, which test and I'll go back at it
<rick_h> yea
<wallyworld> well, round() is broken on 2.6
<wallyworld> but it works in 2.7
<rick_h> lol, but of course it is. hmmm, I ran it in 2.6 and it passed?
<wallyworld> that's what i thought. but i tried it in 2.6 and it failed, as did ec2
<rick_h> oh, no guess I'm on 2.7
<rick_h> sorry, don't even know which python version I'm on
<wallyworld> so, i commented that line of the doc test
<rick_h> ok
<wallyworld> and we will be on 2.7 soon
<wallyworld> i fell dirty doing that but we needed to land this thing
<rick_h> rgr
<wallyworld> would be nice to fix in 2.6 if we could do so feasibly
<rick_h> something is up with that test anyway, forever and ever it's just always spit out perfect 3.0 and suddenly it's doing 2.999999 consistantly? strange
<wallyworld> yeah
<rick_h> yea, I can look at what with round is broken and a different way to do it
<wallyworld> the pressure will be off a bit IF this passes qastaging once bb has finished with it :-)
<rick_h> yea, will be nice
<wallyworld> how long do you think we will need to get the final convoy stuff sorted?
<rick_h> it's actually fine
<rick_h> convoy dev reviewed it yesterday and I fixed (added a test) and resubmitted for review
<rick_h> I'd feel comfy using that brach for our own ppa
<wallyworld> so it's just the deployment coordination that needs doing
<rick_h> so we can do that any time, just have to change to their upstream once they merge it in
<wallyworld> i mean getting prod systems all set up
<rick_h> right
<rick_h> I'm sure we can get our hands held through that processes without too much trouble
<wallyworld> we should send an email to dev and ask a few suckers ^H^H^H^H^H^H guinea pigs to try it
<rick_h> yep, I've got a couple of people actually from my ubuntu loco ready
<wallyworld> cool
<rick_h> and jcastro has suckered me into enough stuff I can try to get him as well. He owes me and should be in lots of places of LP I don't go every day
<wallyworld> ok. i think the intent is to turn on the ff on prod for at least ~launchpad
<wallyworld> initially
<StevenK> So we need the MP merged in, a new release -- if upstream actually make a tarball, that would be awesome.
<rick_h> StevenK: yea, I submitted my changes back after their review like 1hr later
<wallyworld> yeah :-)
<rick_h> so hopefully they're looking at it soon
<StevenK> Then we need to work out the deployment strategy -- I think I understand that
<wallyworld> i think we will need about a week for that?
<rick_h> StevenK: but at the end of the day, he only wanted an extra test so no complaints from the changes themselves
<wallyworld> to get everything done
<StevenK> rick_h: Sure, but it was still marked Needs Fixing
<rick_h> is it? I marked it back to needs review yesterday?
<StevenK> rick_h: His vote was Needs Fixing, I mean
<StevenK> What I'm not clear about is how to set the convoy root url in the code
<StevenK> /+combo/r..../?...
<rick_h> StevenK: ah right
<StevenK> rick_h: wallyworld and I are off tomorrow
<rick_h> StevenK: right, so my understanding is that the .wsgi file will set a combo root of /var/tmp/convoy and our application code (LP) will set the root to /+combo/$revno
<StevenK> rick_h: So you have today and tomorrow to beat sidnei with a stick.
<StevenK> rick_h: On qas and prod, it probably won't be /var/tmp/convoy, but sure.
<rick_h> so on disk will be /var/tmp/convoy/rev12345 and /var/tmp/convoy/rev12346
<rick_h> ok, well whatever that dir is, but that's my understanding of how that works out atm
<StevenK> rick_h: The disk stuff I'm fine with -- that's the easy bit.
<StevenK> What I'm not clear about is the *code*.
<rick_h> what part weren't you sure about then?
<rick_h> so the code is in the YUI.GlobalConfig that the base and the combo roots are '/+combo/rev12345'
<rick_h> with a prefix of yui or lp based on the group
<wgrant> rick_h: round() returns a float.
<wgrant> It's not broken.
<wgrant> It's just impossible to represent most decimal fractions as non-decimal floating point formats :)
<rick_h> wgrant: right, that's cool. expected even.
<StevenK> Oh, we have a TAL define for revno
<rick_h> StevenK: right
<wallyworld> i've never had an issue like this in Java for example
<wgrant> wallyworld: You probably don't use doctests in Java :)
<StevenK> rick_h: Right. So we add a combo_url of string:/+combo/r${revno};
<wallyworld> yes, but i print stuff out
<wgrant> It's still == 0.3
<rick_h> StevenK: exactly
 * StevenK is no longer concerned
<rick_h> that's ok, one day I'm going to be at pycon with the guy that invented doctests and he's going to miss the rest of the conference because I've locked him in a closet with a handful of saltines
<wgrant> wallyworld: And whatever method you use to print probably performs rounding.
<StevenK> rick_h: BWAHAHA
<wgrant> wallyworld: Whereas here in the doctest we're just using __str__/__repr__, which need to return something that is equal.
<wgrant> Not just vaguely correct.
<cjwatson> hmm.  last night's generate-contents-files run:
<cjwatson> 2012-01-25 07:05:13 DEBUG   Installing new Contents file for precise/amd64.
<cjwatson> 2012-01-25 07:05:17 DEBUG   Installing new Contents file for precise/armel.
<cjwatson> 2012-01-25 07:05:20 DEBUG   Installing new Contents file for precise/armhf.
<cjwatson> 2012-01-25 07:05:23 DEBUG   Installing new Contents file for precise/i386.
<cjwatson> 2012-01-25 07:05:26 DEBUG   Installing new Contents file for precise/powerpc.
<cjwatson> but the versions in dists/ are dated 2012-01-24.
<wgrant> It probably raced.
<wallyworld> wgrant: sure, but that web page you pasted earlier talks about how the algorithm for strigification needs improving
<wallyworld> StevenK: are we going to upgrade the apache wsgi lib from suggests to depends?
<wgrant> wallyworld: Well, FSVO "needs"
<wallyworld> :-)
<StevenK> rick_h: Is there a flag in your convoy MP to turn on the dir walking stuff?
<StevenK> wallyworld: My jury is still out.
<rick_h> StevenK: no, not a flag. It *just works*
<cjwatson> hmm, yes, 07:05 is just after the start of a publish-ftpmaster run.
<StevenK> rick_h: Fair enough
<rick_h> if the extra bits are in the url it routes, if the url only has a /convoy then it doesn't
<wgrant> 2012-01-25 07:03:05 INFO    Creating lockfile: /var/lock/launchpad-publisher.lock
<wgrant> cjwatson: ^^ from publish-ftpmaster.log
<wgrant> cjwatson: Is the extra arch making g-c-f take too long?
<StevenK> rick_h: Then beat sidnei with a stick, get his +1 and get him to land it.
<wallyworld> StevenK: ok. i'm just asking because it will affect the "how to" instructions
<rick_h> will try
<StevenK> wallyworld: Sure.
 * rick_h goes to look up what TZ he's in
<cjwatson> It takes three hours.  It's probably dumb luck whether it happens to run when publish-ftpmaster is running.
<cjwatson> Now that we're on half-hour publishing I guess the odds have got worse.
<cjwatson> Maybe it should copy into dists.new if it exists.
<wgrant> cjwatson: The publisher deliberately skipped an hour a day for this reason.
<wgrant> Does it not still?
<cjwatson> It doesn't, but in any case it would have to skip rather a lot ...
<gmb> Has anyone seen this before when trying to pull devel? https://pastebin.canonical.com/58731/
<gmb> I can't tell if it's a codehosting problem or a local bzr problem.
<wallyworld> rick_h: buildbot just completed, branch should be on qastaging in around 45 minutes or maybe a bit sooner
<gmb> Oh, it's a "I don't get prompted for my SSH key" problem.
<gmb> Weird.
<StevenK> We need the new convoy, and we need to change the base template before we can think about talking to a webop to configure qas/staging
<wallyworld> yes
<wgrant> gmb: See #-ops
* StevenK changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugtasks: 4*10^2
<StevenK> rick_h: https://code.launchpad.net/~stevenk/launchpad/combo-url/+merge/90093
<rick_h> wallyworld: awesome
<rick_h> StevenK: looks good to me. I'd approve but I'm in mentor mode anyway so someone would have to second it
<StevenK> rick_h: It even works, so you can +1 if you wish
<rick_h> StevenK: ok, made my comment and marked it as needs review ok?
<wallyworld> StevenK: how does it work in dev mode where revno is "unknown"?
<rick_h> hah, wallyworld for the block. Yea, because in dev mode the path is /var/tmp/convoy and the revno will need to be empty/non-existant
<StevenK> wallyworld: In make run, the revno is expanded out, but the current convoy ignores the path
<wallyworld> yes, but when we upgrade it will break, no?
<StevenK> When we upgrade it, combo-rootdir will change
<StevenK> So this can land now, and will not need to change.
<rick_h> StevenK: but if you have make run, and then you make some changes/commit
<rick_h> will the revno get incremented?
<rick_h> I'm not seeing how that'll be easy to manage
<wallyworld> rick_h: in dev mode, revno is always "unknown" IIRC
<StevenK> Then the new rootdir will be populated
<StevenK> wallyworld: It is not
<wallyworld> i could have sworn it displayed as such on lp.dev
<wallyworld> or something similar
<rick_h> StevenK: ok, well we'll need to test/verify that 100% and figure that out to go with that MP change
<StevenK> I did test it
<rick_h> StevenK: with the new convoy branch?
<rick_h> because the current convoy doesn't care about anything in the url, while the new one will end up with missing files if the path isn't right
<StevenK> rick_h: Right, which is why it's okay to land now.
<rick_h> right, I'm just saying that I want to make sure we don't *forget* that there's a future breakage there
<rick_h> and we'll want to think ahead a step and have a plan for getting it to work
<StevenK> After we switch to new convoy, combo-rootdir will change to populate into CONVOYROOT/rev<revno>
<rick_h> StevenK: right, but I'm not sure that'll work out in real development
<StevenK> It sounds okay to me
<StevenK> rick_h: But we can kick that around when the new convoy turns up
<deryck> Morning, all.
<deryck> bigjools, ping
<bigjools> deryck: o/
<mabac> stub, I'm about to propose a patch for the launchpad db schema and would like to ask you about the table permissions. Should I make up a name for a script user and put that in the security.cfg? I notice that there was en email thread about perhaps not having per script table permissions. Did anything happen with that?
<stub> mabac: Every script or process needs to use a unique database user defined in security.cfg. This is unrelated to creating new tables.
<stub> mabac: If you create a new table, the security.py script will complain if nothing has permissions to it so you will need to add something to security.cfg (even if it is just adding an entry to the [public] section stating nobody has access).
<stub> mabac: Assuming you are going to be creating a new table and a new script, you are probably best adding the new database user with permissions in the same branch as your db patch.
<mabac> stub, aha, I was wondering how to add the new user. thanks, I'll dig for that
<mabac> stub, and yes, we'll be adding new tables and new scripts
<stub> mabac: We still have per script table permissions, but feel free to inherit permissions from elsewhere to make life easier. Not the 'write' group though, as that is a real mess.
<stub> mabac: I'm trying to find a middle road between annoyed developers and throwing out all access controls
 * jelmer wonders if it's a coincidence that the lp branch scanner doesn't scan anything beyond revision 99999 in lp:~vcs-import/chromium-browser/trunk
<mabac> stub, :) that's not trivial, I get that.
<mabac> stub, so to inherit permissions, I just make the new user part of a suitable group?
<stub> mabac: If your scripts are related, they each will need a unique user. They all might be clones of each other though, all inheriting their permissions from one place.
<stub> mabac: Yes, exactly.
<stub> mabac: user and group is interchangeable - the terminology dates from when they were separate in PostgreSQL. Now there are just roles so you should be able to have a group inheriting from a user and vice versa.
<mabac> stub, cool, I didn't know that. thanks. I'll give it a go and let you know when I have something for you to review
<stub> mabac: np. let me know if you need a hand, here or email or whatever.
<mabac> stub, I'll certainly let you know
<mabac> stub, thanks!
<mabac> ls
<mabac> oops
* abentley changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: abentley | Firefighting: - | Critical bugtasks: 4*10^2
* kornbluth.freenode.net changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugtasks: 4*10^2
<abentley> sinzui: I would like to stop using the fictional "application/text" content type for downloading the CoC, and instead use "text/plain".  Is this a good idea, or the greatest idea ever?
<sinzui> abentley, I think the issue is that users are exceedingly adept at copy and paste errors. I think the hack is to ensure that the user gets an untampered file.
<abentley> sinzui: The content-disposition field should ensure the users save the file.
<sinzui> abentley, This is only half the problem. These same users still paste a tampered signature into the forms and Lp tells them to go jump in lake of jello
<sinzui> abentley, Then you have the greatest idea ever
<abentley> sinzui: Okay.  I will verify that users are prompted to download in Firefox, Chromium and Opera.
<abentley> sinzui: Prompted to save, rather.
<sinzui> fab
<mabac_> stub, do you think something like this looks acceptable? http://bazaar.launchpad.net/~mabac/launchpad/workitems-schema-changes/revision/11322
<mabac_> salgado, ^^^ I'm not sure what scripts we will have to add, trying to create something to start from
<salgado> I think we'll only need the one to maintain specificationworkitemstats, no?
<stub> mabac_: That is fine.  As these are scripts, you will also need to inherit from the 'scripts' group at a minimum so it has permission to update the script activity table.
<mabac_> salgado, that's what I think too. just my cautious nature, expecting more unexpected things ;)
<mabac_> stub, aha I missed that. thanks.
<mabac_> stub, and for read access from lp code, I can just add SELECTs to the public section, right?
<stub> mabac_: For the appserver, add it to the [launchpad_main] section
<stub> mabac_: It is truely needed to be read by everything, public would be appropriate but I don't think we have any tables in that state apart from featureflag
<mabac_> stub, yes, this is nothing special in that respect.
* abentley changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: abentley | Firefighting: - | Critical bugtasks: 4*10^2
<lifeless> .
<lifeless> james_w: are you using the django ORM ?
<james_w> lifeless, yes
<lifeless> james_w: you might like to do a oops-django package providing django ORM integration for oops.
<james_w> yes
<lifeless> james_w: erm, rather timeline-django is what I menat.
<james_w> in this case we don't really care
<lifeless> (oops-timeline already exists, so timeline-django is the right join)
<james_w> give jml a few more days and he's likely to rip the db out of this project
<james_w> but it would be nice to have anyway
<lifeless> heh
<lifeless> what will you be using for persistence?
<james_w> nothing
<james_w> this project as it stands doesn't need it
<lifeless> I'm intruiged
<james_w> it's a webhooks-based interface to celery (rabbit) bascially
<james_w> in fact we'd probably have been better off with just celery
<lifeless> oh, interesting
<lifeless> you know about txlongpoll right ?
<james_w> so there's persistence in rabbit for the lifetime of the job, and there's persistence in the other service
<lifeless> (long poll interface to rabbit)
<james_w> I've heard of it
<james_w> I'd like to discuss if it's a good fit, but I've got to dash now
<lifeless> ciao
<lifeless> ping me whenever, or drop me a mail
<james_w> ok
<wallyworld__> sinzui: was that a typo in the team lead notes?
<sinzui> wallyworld__, about wgrant?
<wallyworld__> sinzui: the test suite running in *8* seconds?
<sinzui> wallyworld__, not lp's testsuite
<sinzui> wallyworld__, MaaS
<wallyworld__> right, makes more sense now
<sinzui> I countered with GDP's testsuite now runs in half the time, 1 second
<wallyworld__> i was thinking maybe a few 000's were left off
 * wallyworld__ wonders if qastaging is broken - it's been down a while
<wgrant> rm: cannot remove `/var/tmp/convoy/yui2/event/event.js': Permission denied
<wgrant> So, yes.
<wgrant> qastaging is quite broken :)
<wgrant> wallyworld__: ^^
<wgrant> wallyworld__: Also, aren't you not here today?
<wallyworld__> wgrant: neither are you :-)
<wgrant> Yeah, but I had to check that oops-prune finished :)
<wallyworld__> hmmm. not sure of the reason for that error off thr top of my head
<wgrant> Why is it trying to touch /var/tmp?
<wgrant> On prodish?
<wallyworld__> /var/tmp is where convoy looks to load yui from
<wallyworld__> like how we use /var/tmp/bazaar etc
<wgrant> Yes, but not outside dev.
<wgrant> /var/tmp is not used on anything vaguely productionish.
<wallyworld__> let me check something
<wallyworld__> wgrant: so there's a CONVOY_ROOT var that needs to be set up
<wgrant> o_O
<wallyworld__> it's set in the Makefile to /var/tmp/convoy
<wgrant> Is there precedent for that?
<wallyworld__> yes, CODEHOSTING_ROOT
<wallyworld__> also set in the Makefile
<wgrant> That's not precedent.
<wgrant> That's only used in dev targets.
<wgrant> It looks like we now have to set this variable on every machine.
<wgrant> Which is a bit silly.
<wallyworld__> yes, it should be in .conf somewhere
<wgrant> Anyway, seems like we need to revert it, or check if LPCONFIG=development, or something like that.
<wallyworld__> if it is in .conf, can the Makefile get access to that setting?
<wgrant> Or split it into a target that isn't jsbuild.
<wallyworld__> can we get it going without reverting
<wgrant> I don't know. How do we get it going?
<wallyworld__> if it's "just" a setting that can be tweaked
<wgrant> What is the desired end goal?
<wallyworld__> to have the combo loader root set to something that works ie not /var/tmp/convoy if that doesn't work
<wgrant> Isn't the real goal at present probably to have it not built at all?
<wgrant> Since we're some way from using it.
<wgrant> And don't have deployment sorted out yet.
<wallyworld__> yes, but we want to use it locally, so need a way to have it built
<wallyworld__> we could also use a flag
<kirkland> wgrant: so back to https://bugs.launchpad.net/bugs/919241
<_mup_> Bug #919241: Sources published in a private ppa are downloadable by any subscriber <Launchpad itself:Triaged> < https://launchpad.net/bugs/919241 >
<wgrant> Sure, but until it's finalised you can use a separate Makefile target, or an envvar to enable it.
<wallyworld__> yes, that would work
<kirkland> wgrant: what about dropping a 2-line .htaccess rule in the PPA's pool/.htaccess that 403 denies *.gz?
<kirkland> wgrant: ie:
<kirkland> RewriteEngine On
<kirkland> RewriteRule .*\.gz$ - [F,L]
<kirkland> wgrant: this isn't something i need all the time, so I could just ask the LOSA to turn that on when s/he sets up my private PPA
<kirkland> wgrant: and profit!
<wgrant> kirkland: That would break builds.
<wgrant> And they could still be downloaded through the web UI.
<kirkland> wgrant: builds do wgets?
<wgrant> They get the private sources from ppa.launchpad.net, yes.
<kirkland> wgrant: hmm, seems like that could be further handled through another rule that whitelists the build requests
<wgrant> kirkland: But the builders are untrusted :)
<wgrant> Unless we restrict it to the buildd user.
<wgrant> Which could work.
<wgrant> But we'd still have to fix permissions in the app.
<wgrant> Since they were inappropriately loosed just before I joined.
<wgrant> loosened.
<kirkland> wgrant: so this issues is really, really causing major headaches as to how we're trying to use commercial launchpad ppas
<kirkland> wgrant: it seems so simple to me on the outside
<wgrant> Everything seems simple from the outside :)
<kirkland> wgrant: though I appreciate that there's complexity i don't understand
<kirkland> wgrant: from my PoV, just deny public (outside) downloads of .tar.gz, one way or another
<wgrant> Sure, but there are at least two ways that need to be stopped.
<wgrant> And one of them is complex because the buildds use it.
<wgrant> And we have to work out how it works with PPA dependencies etc.
<wallyworld__> wgrant: the icing dir is set to lib/canonical/launchpad/icing. we could use perhaps lib/canonical/launchpad/convoy for now just to get make going ?
<wgrant> wallyworld__: I'd prefer build/convoy or something like that.
<wgrant> You'll need to hack everything qastaging script to set it.
<wgrant> And staging.
<wgrant> And production.
<wgrant> So possibly unwise.
<wallyworld__> if i understand correctly, it's just the make that is failing
<wgrant> Right.
<wgrant> But most things will do a make.
<wallyworld__> so we can cowboy the makefile?
<wgrant> And this affects asuka/gandwana/tellurium
<wallyworld__> to get it working
<wgrant> And every production machine.
<wgrant> We don't want to unbreak qastaging, because then we'll deploy and fuck production :)
<wallyworld__> or i can just lp-land a tweak
<wgrant> I would lp-land a tweak to split it into a separate target for now.
<wgrant> Until you sort out how deployment will work.
<wallyworld__> i'll come up with something just to get going and get you to +1.
<wgrant> Great.
<wgrant> kirkland: So, I'm glad to see thsi being looked at, but it's not as easy as it seems.
<wgrant> kirkland: And there may even be things I haven't thought of.
<wgrant> kirkland: Because PPAs were not designed to do what you want.
<wgrant> They can be made to, and it's very doable, but it's by no means trivial.
#launchpad-dev 2012-01-26
<kirkland> wgrant: ack
<lifeless> kirkland: consider that other buildds can run arbitrary users arbitrary code
<kirkland> lifeless: yeah
<lifeless> now if you say I don't care about that, thats fine
<lifeless> but we can't really offer a secure embargo *and* have such an obvious wart that we'd need to warn folk off
<lifeless> we don't currently have a bi-directional channel for 'I need file X' from the buildd's.
<lifeless> we probably need something like that to deal with arbitrary build-dependencies w/in the PPA (unless you say 'no build dependencies with hidden sources needed')
<lifeless> $blah etc.
<wgrant> lifeless: Huh?
<wgrant> You don't build-depend on sources :)
<lifeless> wgrant: I know. Still sickish.
<wgrant> We know the needed source files before we start the build.
<lifeless> wgrant: I regretted that bit as soon as I wrote it.
<lifeless> well
<lifeless> as soon as I hit ENTER.
<wgrant> Heh
<lifeless> anyhoo
<lifeless> key thing - kirkland - if you want to work on this, by all means. The constraints are: soyuz assumes source available everywhere today; there is no guaranteed sequencing for htaccess changes and publication IIRC (different processes write it, IIRC), buildds are untrustable
<lifeless> personally, I'd still try a TLT approach first and foremost.
<wgrant> Indeed.
<wgrant> It's easily doable now.
<kirkland> lifeless: what's TLT?
<lifeless> TimeLimitedToken
<lifeless> a time limited access pass to one file in the private librarian
<kirkland> lifeless: ah, tokay
<lifeless> click on a bug attachment on a private bug, or a private build log, for instance.
<lifeless> it will work for a day, but then stop working unless you go back to the LP web UI and click on the link there again
<lifeless> which does a redirect dance to issue a token and forward you to the token-including url on the librarian
<wallyworld__> wgrant: https://code.launchpad.net/~wallyworld/launchpad/fix-convoy-make/+merge/90225
<wallyworld__> wgrant: i'll need to ask to to lp-land also since it's broken on my system
<wgrant> wallyworld__: What creates build/js/yui/yui-3.3.0?
<wgrant> buildout?
<wallyworld__> wgrant: yui-dep.py
<wallyworld__> ah, hang on, yui-deps uses that dir
<wallyworld__> i think it's buildout.cfg, yes
<wallyworld__> is where it's defined
<wallyworld__> yes, just refreshed my memory, it is buildout
<wgrant> Great.
* wgrant changed the topic of #launchpad-dev to: JS build broken until r14727 | https://dev.launchpad.net/ | On call reviewer: abentley | Firefighting: - | Critical bugtasks: 4*10^2
<james_w> lifeless, so, thinking about it over dinner, I'm not sure that we would get any benefit from txlongpoll in this bit of the system, but we may well want to use it for updating the browser when the this backend processing job gets all the way back to the user-facing web service to update browsers. Is there a reason that we would want to use it between that user-facing service and the backend jobs that I'm missing?
<james_w> lifeless, if it's not a good time then this can obviously wait
<lifeless> james_w: sorry, was on the other machine
<lifeless> james_w: uhm,I don't know what you're putting together at the moment, so I can't really comment ;)
<james_w> Yeah, I realize it's a bit vague
<james_w> We're basically adding an async job system to developer.u.c that packages tarballs
<lifeless> and you wanted webhooks in there somehow
<lifeless> ?
<james_w> We have webhooks between d.u.c and the service that coordinates the workers
<james_w> pkgme-service is the second system
<lifeless> any reason for webhooks in particular?
<james_w> Not strong reasons
<james_w> In fact it's probably pointless
<lifeless> so for LP I've advocated PSHB as the public event system for a while
<lifeless> with rabbit internally
<james_w> Given that rabbit is used as well
<lifeless> txlongpoll isn't quite PSHB, but a little refactoring and it could be the browser side, with our own internal PSHB hub
<lifeless> james_w: I presume you would like events out of LP too
<james_w> For this? Eventually
<james_w> For other things, definitely
<lifeless> hah, '404 Bugs reported by me '
<james_w> heh
<lifeless> anyhow, broadly speaking:
<lifeless>  - I think one event system internally, and one publically is fine
<lifeless> public/internal being split on *data* protection not teams.
<lifeless> and I'm massively pro using event systems to let different bits be built separately
<lifeless> I presume you've seen the guidelines for things in LP done this way?
<lifeless> they dont' apply directly to you but may still be useful
<james_w> I think so, you mean ServicesRequirements?
<lifeless> yes
<lifeless> .
<lifeless> I hate my ISP. Hates.
<lifeless> They couldn't keep DSL reliable if they were paid to.
<james_w> Yeah, we look over that doc frequently
<lifeless> james_w: I love feedback, if you ever have any :)
<james_w> sure will
<lifeless> this is totally unrelated but you may find it fun - http://piumarta.com/software/maru
<StevenK> lifeless: You *do* pay them to keep your DSL reliable.
<lifeless> StevenK: I know
<stub> What is the easiest way to get a package from Oneiric universe into a PPA for Lucid? Copy without rebuild should be fine, but I think that is only point and drool when the source is a PPA?
<wgrant> stub: You can't copy it directly; the source name conflicts with the 8.4 one.
<wgrant> You'll need to rename it to postgresql-9.1-debversion, like the binary already is.
<stub> I'm not that far. Can't even see a link to download the sourcepackage so am bzr branching atm.
<wgrant> http://launchpad.net/ubuntu/oneiric/+source/postgresql-debversion
<wgrant> Scroll down a little, and there is the link.
<stub> I still don't see it
<stub> oh.. not looking for a .deb am I
<adeuring> good morning
<bigjools> lifeless: yo
<bigjools> I can haz testresources release please?
<rick_h> wgrant: still around? wondering about the combo loader/convoy setup
<StevenK> rick_h: Did you get the convoy MP approved?
<rick_h>  StevenK I did from U1, I'm waiting to hear back from landscape folks
<rick_h> they're not as sure how the heck it works on their end
<StevenK> Well, they don't need to upgrade to it right away ...
<rick_h> StevenK: right, I'm pinging sidnei right now to let him know I talked with landscape and we should be good
<rick_h> so I'm prodding to get the MP merged
<rick_h> they both had to check their apache configs because my change would break things on their end
<rick_h> however, sidnei says they have rewrite rules that negate my change for them so it should be all good
<rick_h> StevenK: he says all sounds good to him, so hopefully sidnei will merge soon now
<rick_h> StevenK: did you follow how it is that not all prod machines will have convoy installled and avail in their setups? I kind of assumed they'd have to because the same apache config/etc was serving the combo files so they appeared to be from the app server?
<StevenK> rick_h: Right, it makes sense.
<rick_h> So then convoy and the combo loader live on one app server?
<wgrant> rick_h: Apache doesn't run on the appservers
<wgrant> Icing is served from the Apache on the frontends, banana and nutmeg.
<wgrant> They run apache+squid+haproxy
<wgrant> They are the only things in production that need to run convoy.
<rick_h> ok, so the app servers need a config to pass the requests up a level then and then they'll cache the combo files?
<wgrant> rick_h: The appservers aren't involved.
<wgrant> rick_h: The frontend Apache on banana+nutmeg will intercept /+combo and send it to the local WSGI app
<rick_h> ok
<wgrant> All the appservers do is generate the +combo URL.
<rick_h> ok, gotcha
<StevenK> wgrant: lifeless' opinion was WSGI was too risky to run on banana/nutmeg
<StevenK> So we'd proxypass to the appservers
<wgrant> That's my opinion too, but I thought webops said otherwise.
<StevenK> webops agreed with lifeless
<rick_h> no, at Budapest we ping'd webobs on it and they said it'd be fine I thought
<wgrant> That's what I thought, yeah.
<rick_h> heh, so we're definitely not doing this on a friday :)
<StevenK> I don't even want to set up qas/staging on a Friday
<StevenK> rick_h: Oh yeah -- do you remember how you wanted +combo at the end of the URL? So say someone generates a branch called 'make-use-of+combo' == game over
<StevenK> Since then the branch URL ends with it
<rick_h> StevenK: well it would have to be /+combo has the branch namne
<rick_h> the / is improtant I believe
<wgrant> What benefit does that provide?
<wgrant> It's important for +login, but that's all.
<rick_h> nothing, it was just the initial thought that urls that end in /combo would be sent to convoy so that things worked kind of like a normal url routing. /prod1/combo /prod2/combo etc
<rick_h> it's something that got changed based on StevenK's advice so we should be good
<nigelb> WCPGW
<StevenK> I'd rather qas wasn't on fire over the weekend.
* jcsackett changed the topic of #launchpad-dev to: JS build broken until r14727 | https://dev.launchpad.net/ | On call reviewer: rick_h*, jcsackett | Firefighting: - | Critical bugtasks: 4*10^2
<james_w> hi, would someone have a few minutes to do a wadllib release for us to put in Debian/Ubuntu?
<james_w> I assume it's just a few minutes to do a release at least
<rick_h> james_w: that's a good question. I suppose that'd fall under my territory and I'm not sure how that's done.
<rick_h> but fortunately I see deryck just joined in here so I can pester him about it :)
<james_w> :-)
<rick_h> deryck: james_w> hi, would someone have a few minutes to do a wadllib release for us to put in Debian/Ubuntu?
<deryck> oh, hmmm, that's not my thing either. :)  Let's look around the room.
<deryck> maybe flacoste or lifeless?
<james_w> https://dev.launchpad.net/HackingLazrLibraries#releases
<james_w> maybe that helps?
<deryck> dang, there's always a wiki page.
<deryck> just once I want to punt. ;)
<rick_h> james_w: yea, just looking at that and checking the branches/etc
<deryck> rick_h, back to you. ;)
<james_w> heh
<james_w> deryck, "accidentally" delete the wiki?
<rick_h> james_w: so we're just looking to create a tarball of the latest lp:wadllib right?
<deryck> ha!
<james_w> rick_h, I believe so
<james_w> the branch I want has landed at least :-)
<rick_h> james_w: excellent, I'll get it checked out and work my way through this, give me a few
<james_w> thanks rick_h, much appreceiated?
<james_w> err, that's not a question
<james_w> it surely is
<rick_h> heh
<rick_h> james_w: in looking over the changes this would be a 1.2.0 -> 1.2.1 change?
<rick_h> oh nvm, add py3 would seem a bigger point
<jcsackett> james_w: rick_h and i have been looking at your MP for python-oops, and rick_h has pointed out that it doesn't look like the code actually requires bson. what was the situation you encountered leading to this change? did something break on you, or is it just in response to other comments in the code about "must be bson serializable"?
<jcsackett> james_w: i see other parts of our oops infrastructure require bson, which is what i believe prompts the comments in lp:python-oops.
<james_w> jcsackett, rick_h, that's a very good point
<james_w> I think I just invented that dependency
<james_w> it's actually python-oops-datedir-repo that doesn't mention it in the README
<james_w> I think I just read the mailing list thread and assumed
<jcsackett> james_w: easy thing to do; it took me a few moments of digging through the related code to figure out what might be going on. :-)
<james_w> I've pushed a new revision that drops bson
<james_w> thanks for catching it
<james_w> it seems download-cache is pruned without regard to other projects that rely on it (e.g. python-oops)?
<rick_h> james_w: got a sec to give me a hand here? I've branched this and run bootstrap, buildout, and running tests are failing
<james_w> rick_h, wadllib?
<rick_h> I'm guessing there's a step I'm missing here?
<rick_h> james_w: yes, do you need to run a python setup.py develop kind of thing first?
<james_w> I don't think so
<james_w> let me buildout a copy and see what I get
<james_w> rick_h, thanks for the thorough review on https://code.launchpad.net/~james-w/python-oops/update-readme-dependencies/+merge/90028 you were correct about iso8601
<abentley> rick_h: could you please review https://code.launchpad.net/~abentley/launchpad/data-download-view/+merge/90269 ?
<rick_h> abentley: sure thing, will be a few.
<abentley> rick_h: thanks.
<rick_h> james_w: awesome, glad those weren't needed.
<rick_h> james_w: with wadllib, the doctests import wadllib, but that would mean there was an egg for it for it to import from right?
<rick_h> since the doctest it in the wadllib directory, I think
<james_w> rick_h, yeah, I would expect buildout to take care of that though
<rick_h> james_w: yea, I think I'll ping benji
<rick_h> benji: got a sec to give me a hand with this wadllib fun?
<benji> rick_h: sure, what's up?
<rick_h> benji: I've been asked to do a release of it and I"m trying to get the env setup so I can run tests and increment versions and all that fun
<rick_h> benji: my understanding is I need to branch, bootstrap.py, and buildout and from there .bin/test should work ?
<rick_h> benji: except the tons of the doctests blow up on me and I'm not sure I've got things setup right
<benji> rick_h: that's the right steps; let me see if I can reproduce the problem
<james_w> I can't even bootstrap
<james_w> it's complaining about the version of distribute in a seemingly contradictory way
<rick_h> james_w: yea, I had to create the download-cache directory to bootstrap
<james_w> ah, I'll try that
<james_w> I was using lp-sourcedeps
<rick_h> james_w: if you do it in a virtualenv like the docs say, it gives a giant warning that you're double dipping the no-site-packages, but I think that's just a warning not a true error
<james_w> rick_h, so my first error is:
<james_w>    ParseError: not well-formed (invalid token): line 1, column 1
<rick_h> james_w: yea, same here
<rick_h> so I guess the first couple do pass so perhaps it's not my setup but actual failing tests
<james_w> wadl_string starts "<?xml version="1.0"?>"
<james_w> I wonder if the Application API has changed
<rick_h> james_w: I'm going to get a py3 env setup since that was the last commit and see if it works there and can start figuring out blaming py versions or what
<james_w> ok
<benji> rick_h: I've narrowed it down to some problem elementtree is having parsing the WADL XML
<rick_h> benji: ok, yea that makes sense
<rick_h> benji: so my setup is ok then, just a matter of real failing tests
<benji> rick_h: yep (although there is at least one thing in the buildout that needs to be fixed, buildout.cfg defines a cache directory and eggs directory, but it shouldn't
<rick_h> benji: right, that fits with my download-cache I had to create I think
<benji> right, it shouldn't do that
<rick_h> sorry, getting a crash course in buildout which was on the plan for today anyway but wheeee
<james_w> ah, _make_unicode
<benji> (if you want a download cache and an egg cache (and you do), they should be configured in .buildout/default.cfg)
<james_w> elementtree hates unicode apparently
<rick_h> bah, setting up with py3 fails as well with issues getting the right distribute it looks like
<james_w> rick_h, I just added the distribute it said it picked to versions.cfg
<rick_h> james_w: yea, just tried that but I get http://paste.mitechie.com/show/516/
<rick_h> I updated the bootstrap.py to be a py3 compatible one from: http://pypi.python.org/pypi/zc.buildout/2.0.0a1
<james_w> did you add "distribute = 0.6.24" to versions.cfg?
<rick_h> it downloads the right distribute egg into /tmp
<rick_h> james_w: yes
<rick_h> hmm, why do some packages have one = and some == ?
<benji> rick_h: it's a huge hack, but this makes the tests pass: http://paste.ubuntu.com/817767/
<benji> rick_h: I need to attend to some other things now, but feel free to contact me if I can be of assistance.
<rick_h> benji: ok, thanks for the help. I'm going to ping allenap since he has the last commit commenting on py3 support
<rick_h> allenap: how does this work please? james_w would love a release but it seems kind of broken atm
<james_w> I wonder about http://paste.ubuntu.com/817769/
<allenap> rick_h: What are we talking about
<allenap> ?
<rick_h> allenap: james_w would love a wadllib release, but when I pull it and try to run tests/prep it the tests fail, there's buildout errors about download-cache, and when trying to make it work in py3 the bootstrap doesn't seem to work and I can't get bootstrap to run once I download the py3 compat version
<rick_h> allenap: since you have the last commit about py3 compatibility I'm hoping you know what's up and how you made it all work
<allenap> rvba: Right, let's have a look.
<rvba> allenap: I suppose you meant to talk to rick_hâ¦ or do you need me for something?
<rick_h> rvba: I think he was too quick on the autocomplete
<allenap> Dammit.
<rvba> allenap: that's ok ;)
<rvba> Hi rick_h btw :)
<rick_h> rvba: howdy? hope FR is well today lol
<allenap> rick_h: Just try: python3 setup.py test
<rick_h> allenap: without any buildout setup?
<allenap> rick_h: Doesn't need it :)
<rick_h> allenap: AttributeError: 'HTTPMessage' object has no attribute 'getheaders'
<allenap> rick_h: Okay, maybe it does. Are you on Precise?
<rick_h> allenap: no, oneiric. I've gotten py3, py3-setuptools and that's it so far
<rick_h> well, installed py3 pip manually as well since that wasn't in a deb yet
<rick_h> http://paste.mitechie.com/show/517/ allenap
<allenap> rick_h: Ah, I am. Okay, buildout doesn't work on Python 3 (afaik). Does python2.7 setup.py test work?
<rick_h> allenap: no, we get the test failures
<rick_h> allenap: which benji *fixed* with http://paste.ubuntu.com/817767/
<rick_h> but guessing that's not going to be nice in py3
<benji> "fixed" in some sense of the word ;)
<rick_h> allenap: and heads up, buildout *kind* of is supposed to work with py3: http://pypi.python.org/pypi/zc.buildout/2.0.0a1 bit alpha == scary and all that
<rick_h> hmm, this error was supposed to be fixed it looks like: https://bitbucket.org/tarek/distribute/issue/206
<allenap> rick_h: From a clean tree, I do: mkdir download-cache && python bootstrap.py && bin/buildout
<allenap> The reason that download-cache is missing is, I think, so that it can be easily symlinked to the Launchpad download-cache.
<allenap> Same with eggs.
<rick_h> allenap: for the python2 side I did that and got the tests failing with unable to parse the xml test data file
<allenap> rick_h: Do you have a pastebin for that?
<rick_h> allenap: http://paste.mitechie.com/show/518/
<rick_h> this is what benji's "cast to str()" fix corrected
<rick_h> allenap: ok, on the py3 side I updated my distribute version and python3 setup.py test passes
<benji> allenap: the download-cache and eggs-directory should be configured locally, not in a project's buildout; configuring them locally will still allow them to be shared with LP
<allenap> rick_h: Can you try with this patch? http://paste.ubuntu.com/817787/
<rick_h> allenap: sure thing, sec
<allenap> benji: Yeah, agreed. I guess this was cargo-culted from Launchpad.
<benji> probably
<rick_h> allenap: same error for me
<deryck> abentley, shall we G+ hangout or mumble for our call?
<allenap> rick_h: Have you removed the str(...) thing?
<abentley> deryck: Your call, but I'm having a bad hair day.
<rick_h> allenap: yea, I never applied it
<deryck> abentley, let's G+ then, it will be even more fun with bad hair. :)
<rick_h> http://paste.mitechie.com/show/519/
<deryck> abentley, should have an invite in bound.
<allenap> rick_h: Can you try with another patch? http://paste.ubuntu.com/817803/
<allenap> rick_h: Sorry about this; I can't replicate the problem :-/
<rick_h> allenap: sure thing
<rick_h> allenap: success, tests pass
<allenap> rick_h: \o/
<rick_h> let me run that on the py3 side and make sure that still runs as well then
<rick_h> grabbing some quick grub/eyeball break. Thanks for helping with this.
<rick_h> I'll let you know how the py3 side goes
<allenap> rick_h: I'm not sure that _make_unicode() in _from_string() is needed, but I guess we need some tests to prove that.
<rick_h> allenap: k
<rick_h> james_w: so summary, how bad do we need the release atm? I've got to catch up on some code review duties today and it looks like we need some tests and tweaking of the buildout setup and docs before I'd really feel comfy doing a release.
<rick_h> allenap: do you think adding py3 support is worthy of jumping 1.2.0 to 2.0?
<allenap> rick_h: A non-technical question. I suddenly feel out of my depth. Why not? (Why? too).
<rick_h> allenap: so just to shake poeple into seeing 2.0 means craps changed from 1.2, 1.3 means small changes, I should check that changelog
<rick_h> allenap: purely opinion and not having worked on this don't feel qualified to make it
<james_w> rick_h, it's not urgent
<james_w> cjwatson, do you still have your py3 wadllib environment around by any chance?
<allenap> rick_h: Yeah, then stick with 1.x I guess. I assume that means it's working with py3?
<james_w> rick_h, which change makes it pass?
<rick_h> james_w: that patch file from allenap http://paste.ubuntu.com/817803/
<rick_h> and for py3 you need to make sure you've got distribute > 0.6.20
<rick_h> james_w: and realize not to use buildout
<rick_h> james_w: finally, I want to file a bug and get the download-cache thing fixed per benji on the py2 side
<rick_h> and update some of the docs in https://dev.launchpad.net/HackingLazrLibraries#releases
<james_w> rick_h, I don't think that actually fixes the bug
<james_w> as in, it makes the tests pass, but only because they are doing something different
<rick_h> james_w: right, which is why I want to hold off on the release and update some tests and look at this more carefully
<rick_h> I'm not sure what wadllib does tbh and so need to catch up
<cjwatson> james_w: probably
<james_w> I think http://paste.ubuntu.com/817769/ may well be better
<cjwatson> (2.0 seems overkill)
<james_w> cjwatson, would you try the patch in http://paste.ubuntu.com/817769/ if you do please?
<rick_h> cjwatson: k, thanks. 1.3 it is
<cjwatson> james_w: that looks wrong; it would prevent Application from being initialised using a Unicode string
<james_w> cjwatson, true, but that currently fails on Python2
<cjwatson> really?  what's the problem we started with here?
<rick_h> cjwatson: http://paste.mitechie.com/show/518/
<james_w> cjwatson, the tests fail under Python 2
<cjwatson> the intent of the code definitely seems to be to accept either bytes or unicode there
<cjwatson> james_w: they pass here
<rick_h> cjwatson: yea, they pass for allenap, appears he's on precise?
<james_w> because it gets bytes, unicodes them, puts them in a StringIO and then reads them
<rick_h> on my oneric it fails and benji and james_w got the same failures
<james_w> oneiric here too
<cjwatson> james_w: which should work fine in Python 2
<cjwatson> its StringIO should tolerate either
<james_w> so an API difference in elementtree or pkg_resources
<benji> precise here
<james_w> hmm
<cjwatson> but it's true, I see the failure in an oneiric chroot
<cjwatson> with Python 2.7 even
<james_w> if you drop the unicode call then it works again, suggesting that elementtree is choking on unicode
<james_w> so if we're on Python 2 we should skip unicoding there, and all should work right?
<james_w> no change for Python2, and the right thing for Python3?
 * cjwatson is analysing ...
<allenap> Doing unicode(a_string) does an implicit decode, and you shouldn't do that to XML bytes unless you know the encoding.
<allenap> Well, you shouldn't do an implicit decode on XML at all.
<allenap> Keep it as bytes, let the parser do the rest.
<cjwatson> Using BytesIO in Python 3 prevents people passing in strings in the natural way, though.
<allenap> (The first pkg_resources change I made might have been a red herring.)
<cjwatson> If you want to do that, you'll have to encode strings.  Seems clunky though.
<allenap> cjwatson: The second patch (http://paste.ubuntu.com/817803/) still allows for strings and streams.
<cjwatson> so minimal reproduction case in oneiric:
<cjwatson> >>> import xml.etree.cElementTree as ET
<cjwatson> >>> from cStringIO import StringIO
<cjwatson> >>> list(ET.iterparse(StringIO(u"<foo/>")))
<cjwatson> Traceback (most recent call last):
<cjwatson>   File "<stdin>", line 1, in <module>
<cjwatson>   File "<string>", line 84, in next
<cjwatson> cElementTree.ParseError: not well-formed (invalid token): line 1, column 1
<cjwatson> allenap: with the justification being that opening XML files in text mode is wrong?
<allenap> cjwatson: Yes.
<allenap> There is ET.fromstring that seems to do the right thing.
<cjwatson> I guess.  But in that case you'll still need to use BytesIO in Python 3.
<cjwatson> And I think it'd be better to call it BytesIO in Python 2 as well (from cStringIO import StringIO as BytesIO) for clarity
<james_w> agreed (BytesIO)
<allenap> Yeah, I agree too.
<james_w> allenap, so the fix should be to use fromstring?
<allenap> james_w: Yeah, I think that makes sense. Punt the problem over to the stdlib :)
<james_w> allenap, are you putting a patch together?
<allenap> james_w: I wasn't. I thought rick_h and benji were doing that?
<rick_h> allenap: I stepped back for a few while you guys debated the best fix. I think I can put something together based on your conversation. It'll probably be tomorrow when I get that together and I'll bug you guys again if I hit trouble
<james_w> great, thanks rick_h
 * rick_h saves this irc log like there's no tomorrow :)
<rick_h> abentley: ping, question on the MP. line 92 removes the content-length header, but I don't see where it gets added? I was expecting DataDownloadView to add it in __call__?
<rick_h> abentley: but I do see you have a test for it checking the length in line 188
<abentley> rick_h: It happens in the Zope machinery.
<rick_h> abentley: ah ok
<rick_h> abentley: so the previous setting was just superfluous?
<rick_h> abentley: bah nvm, I see the comment in there about that point
<allenap> rick_h: Cool :)
<rick_h> james_w: so think I've landed your changes to the python-oops, not sure how long it'll take to show in trunk
<rick_h> james_w: so ping me if you don't seem them in a bit
<james_w> rick_h, great, thanks
<rick_h> james_w: there it goes, looks like it went through yay
<james_w> yay
<abentley> sinzui: do you know anything about DistroSeriesDifferenceComment ?
<sinzui> I do not
<sinzui> abentley, I *think* it is an explanation for a change in packaging
<adeuring> rick_h: fancy a review? https://code.launchpad.net/~adeuring/launchpad/bug-834303/+merge/90304
<rick_h> for you adeuring, any time :)
<sinzui> eg backport upstream fix to lucid-based series for partner
<adeuring> rick_h: thanks!
<abentley> sinzui: Okay, thanks.  I just asked 'cause none of the authors were around.
<deryck> abentley, rick_h, adeuring -- about to have to go offline to head home.  tornadoes are coming and need to relocate for safety.
<rick_h> deryck: ok, head deep
<adeuring> deryck: good luck!
<deryck> but will work online once back there.
<abentley> deryck: ack.
<deryck> it's safe now, but we get a bit paranoid around here when those storms come now. :)
<deryck> so before they hit, I'll get home. :)
<rick_h> adeuring: in line 30 of your MP, do you need the cast to set? will you possibly get multiple of the smae subscriber_id?
<adeuring> rick_h: good catch -- the IDs are unique. I'll remove the set()
<abentley> jcsackett: you're mentoring rick_h, right?
<rick_h> abentley: yea, did I not add him to the reviewers list?
<rick_h> abentley: he was nomming some lunch a few ago
<abentley> rick_h: Oh, I just wanted to make sure a mentor review was going to happen.
<rick_h> abentley: yea, I've added him as a requested review and should happen when get gets back from lunch
<jcsackett> abentley, rick_h: i'm back. had a bit of hiccup, but looking through reviews now.
<jcsackett> sorry for delays. :-P
<abentley> jcsackett: No worries.
<jcsackett> abentley: i have nothing to add to rick_h. that's a really nice change, r=me.
<abentley> jcsackett: You guys are going to have to get tougher with me, or I'm gonna get a swollen ego :-)
<jcsackett> abentley: i promise to tear a branch to pieces some day. just not today. :-)
<abentley> jcsackett: thanks.
 * jcsackett laughs.
<jcsackett> only from lp devs do we thank each other for promises to be mean in MPs.
* jcsackett changed the topic of #launchpad-dev to: JS build broken until r14727 | https://dev.launchpad.net/ | On call reviewer: jcsackett | Firefighting: - | Critical bugtasks: 4*10^2
<thumper> hi people
<thumper> is there someone who understands PPA builds around?
<thumper> or should I wait for wgrant?
<lifeless> !ask
<thumper> hi lifeless
<lifeless> thumper: :) hi. You know the story. Don't ask to ask.
<thumper> if I have two packages being built into a PPA, A and B
<lifeless> recipes?
<thumper> and api/abi breaks in A will cause B to fail
<thumper> B depends on A
<thumper> if I change A
<thumper> will B get built?
<thumper> not recipes, just ppa stuff
<lifeless> ok
<thumper> do we need to manually kick off a build for B?
<lifeless> B won't rebuild just because A changed.
<lifeless> jml: hi
<thumper> ta
<lifeless> What you'd want to do in general if you have an API break is to add breaks: B =< $lastbuild and that will stop folk co-installing incompatible pairs
<lifeless> kirkland: its really not that much work
<lifeless> abentley: the other day, talking about big comments- I knew you were talking about merge proposals. I was suggesting we could reduce special cases by making the bugs and merge proposal comment threads more similar
<abentley> lifeless: I have already done a lot of work to make the bugs and merge proposal comments more similar.
<lifeless> cool
<lifeless> I was just wrapping up that chat, I had an ELOCAL and then you had EOD'd.
<abentley> lifeless: So when this code does go live, code review comments, not just bug comments, will be truncated and have a "Read more" link if they're unusually long.
<abentley> lifeless: And both will have "Download full text" links, if applicable.
<lifeless> cool
<lifeless> thats really nice
<lifeless> do we serve those comments from the librarian ?
<lifeless> If not, should we? [e.g. migrate their full content out of the DB, leave behind just the truncated form].
<abentley> lifeless: We don't serve them from the librarian.
<abentley> lifeless: I wasn't going to, because we need to generate the body text, and possibly obfuscate email addresses.
<lifeless> ah, do we sometimes not obfuscate?
 * lifeless goes to triage more bugs \i/
* jcsackett changed the topic of #launchpad-dev to: JS build broken until r14727 | https://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugtasks: 4*10^2
<james_w> hi lifeless
<jelmer_> uhoh
 * jelmer_ takes cover for the incoming flood of email
<abentley> lifeless: We only obfuscate email addresses for anonymous access.  Logged in users will see email addresses, just as they would in the normal UI.
<wgrant> lifeless: No, ABI breaks are handled in sensible projects by incrementing the SONAME :)
<jml> lifeless: hi
<lifeless> jml: hi! Did you see my testtools release prompt mail.
<lifeless> poolie: do you think bug 567632 is likely to get a timeslice in the next 6 months (from a bzr'r) ?
<_mup_> Bug #567632: loggerhead could be faster if it used statictuple <performance> <loggerhead:Triaged> < https://launchpad.net/bugs/567632 >
<jml> lifeless: I saw it yeah. Haven't read it yet.
<lifeless> jml: do you have time to read it now?
<jml> lifeless: sure.
<lifeless> thanks
<jml> lifeless: my laptop screen has died & I've spent the last couple of days setting up a VM on my mac
<lifeless> jml: ugh
<lifeless> jml: commiserations
<jml> lifeless: and my house has a mouse infestation so I'm staying at friend's place
<jml> lifeless: so not a lot of free time at computer
<lifeless> jml: I could fedex you a kitten.
<jml> lifeless: the landlord probably wouldn't approve
<lifeless> ah yes
<jml> lifeless: perhaps we could adopt the mice as pets and have him evict them
<lifeless> -so- glad to be free of that :)
<wallyworld_> wgrant: https://bugs.launchpad.net/jockey/+bug/293399
<_mup_> Bug #293399: jockey messed up with driver installation status <Jockey:New> <Ubuntu:Invalid> < https://launchpad.net/bugs/293399 >
<poolie> hi jml
<poolie> lifeless, no
<jml> poolie: hello
<jml> I've been reading about R, recently
<jml> it's good
<jml> very glad I bought a book
<poolie> it is interesting
<lifeless> jml: thanks for replying
<jml> lifeless: np
<lifeless> jcsackett: will you be landing the loggerehed bug 276768 branch ?
<_mup_> Bug #276768: Long revision comment does not wrap <launchpad-loggerhead:Invalid> <loggerhead:In Progress by pr0gg3d> < https://launchpad.net/bugs/276768 >
 * jml defers solving VM clock skew until the morrow
<lifeless> sinzui: bug 601392 - is that easy, or even trivial ?
<_mup_> Bug #601392: Loggerhead has poor text-background contrast with Chromium <css> <ui> <wcag> <Launchpad itself:Triaged> <loggerhead:Triaged> < https://launchpad.net/bugs/601392 >
<sinzui> lifeless, trivial someone is landing a loggerhead css change at this minute I think
<lifeless> poolie: oh, t'other thing - your in-page timeline view; how is the post-landing fixup coming along?
 * lifeless hits enter 5 times to make tagsa pply
<poolie> i have not touched it you
<StevenK> lifeless: I think you need to visit your ISP.
<poolie> *yet
<poolie> i still intend to but i haven't got to it
<sinzui> StevenK, I updated the merge bug with what flacoste and I discussed.
<StevenK> sinzui: Excellent, thanks.
<lifeless> StevenK: the tags thing is an LP regression
<lifeless> StevenK: but yes, I should visit my ISP
<lifeless> poolie: well, I trust you will get to it soon!
<wgrant> lifeless: It sometimes works first time, which is odd.
<wgrant> And I think it regressed in the last monthish.
<lifeless> yeah
<james_w> lifeless, if I am creating new python-oops-* projects, do you have a recommendation for who should own them?
<lifeless> james_w: yes, I do
<lifeless> james_w: https://dev.launchpad.net/CreatingNewProjects
<rick_h> StevenK: if you get a sec can you fix your convoy branch for the tests_require? I've gotten permission from rockstar to merge my branch and yours in there once fixed so we'll have an updated dev of convoy
<james_w> lifeless, that seems wrong to me in this case
<StevenK> rick_h: That change is not really needed, now that we use a .deb of convoy, but sure, why not.
<james_w> lifeless, it would have the Launchpad team owning code that they never used or worked on, and which is currently unused in Launchpad with no plans to do so, while the people that do work on it don't have the powers over the project
<lifeless> james_w: well, whats the case
<james_w> firstly python-oops-celery
<lifeless> james_w: and who do you want to be responsible for running the project (bug triage, code review, doing releases)
<james_w> Online Services
<james_w> as the users of this code
<lifeless> I have no objection to something named like other python-oops- projects being run by other folk than the lp team
<james_w> ah, I missed the bit at the bottom
<lifeless> viva la communite
<lifeless> personally, in this case, I would just use lp, as lp is pretty good at low latency reviews etc
<lifeless> I don't see a great need to be precise about things
<lifeless> OTOH if that is uncomfortable for you, then feel free to have it separate
<lifeless> the 'Not part of Launchpad proper?' section doesn't cover you here, because 'A growing number of the projects the Launchpad team work on ' doesn't fit
<rick_h> StevenK: yea, but it's still a good form upstream change and while I'm merging I'd like to clean it up
<james_w> true
* wallyworld_ changed the topic of #launchpad-dev to:  | https://dev.launchpad.net/ | On call reviewer: wallyworld | Firefighting: - | Critical bugtasks: 4*10^2
<lifeless> james_w: so there are two groups interested in the code: online services (need it for their new project), LP (may use celery, and care about it as part of the oops ecosystem)
<lifeless> LP is set up for public project governance with contributions from all over.
<lifeless> Online services isn't, so far anyhow.
<StevenK> rick_h: Is done
<rick_h_droid> ty much
<james_w> I think we're sufficiently set up to handle these projects
<lifeless> james_w: I think you need to decide how you want it setup and just do it ;)
<lifeless> james_w: if you make it easy for LP devs to contribute, if/when needed, so much the better
<james_w> https://launchpad.net/python-oops-celery
<james_w> https://launchpad.net/python-oops-dictconfig
<lifeless> james_w: the rest of that page may be useful inspiration for you
<lifeless> cool
<james_w> there's some reviews there if you feel so inclined
<lifeless> so, this is why the page for lp stuff says what it does : tracking N different sources of things to do is horrible
<lifeless> I'm happy, if you want a review, to be pinged, or even to be in the review team.
<james_w> I'll have no problems getting reviews when the sprint resumes tomorrow, so it's just if you are interested in them
<lifeless> but its very unlikely that I will ever find them todo in my daily gtd routine :)
<james_w> I'll be sure to explicitly request a review from you if I want one
<lifeless> cool
<james_w> I'll package them up tomorrow, then integrate them with our app
<lifeless> sweet
<james_w> it's two lines to turn on oopses for celery now, and similar for wsgi
<james_w> all that's missing is timeline-django, but as I said that's not important for us currently
<lifeless> james_w: dictconfig is interesting
<lifeless> james_w: perhaps each publisher should accept that instead of having a central clearing house
<james_w> inspired by recent logging changes
<james_w> lifeless, that might be a good compromise
<lifeless> your .testr.conf is old-skool
<lifeless> [DEFAULT]
<lifeless> test_command=${PYTHON:-python} -m subunit.run $LISTOPT $IDOPTION testrepository.tests.test_suite
<lifeless> test_id_option=--load-list $IDFILE
<lifeless> test_list_option=--list
<lifeless> the id option and list option will let you use --parallel
<lifeless> and let it detect deleted tests a little more reliably
<james_w> ok
<james_w> it's the same one I've been copying around since testr started I think :-)
<james_w> there should perhaps be a "testr make-config" or something
<lifeless> so yeah, I think if we had some factory style hook
<lifeless> we could accept a dict in the core and have each publisher implement its own dict->instance logic
<lifeless> the constraints are that the core must not know a-priori about the available publishers (at least at the level of not needing python or packaging dependencies on them)
#launchpad-dev 2012-01-27
<wallyworld_> rick_h: i've discovered a fatal flaw in our changes to how we instantiate the YUI instances
<rick_h> wallyworld_: ruh roh /me hides
<wallyworld_> rick_h: the change from using a single YUI instance LPS to creating a new YUI instanc eeach time we do a YUI().use(...)
<wallyworld_> there's code which puts stuff in a well known global namespace
<wallyworld_> and now each YUI().use(...) is sandboxed
<rick_h> wallyworld_: right, which code is this? I've been working on moving that code out
<wallyworld_> so different bits of YUI code cannot see this shared info anymore
<wallyworld_> there's 2 places - one i'm developing now (and is broken after merging trunk), and also the butask deletion code which is on prod
<rick_h> wallyworld_: right, that shared stuff needs to become a YUI module and then each use() that needs it has to require it
<wallyworld_> so the current bug task deletion setup is in form-picker-macros.pt
<wallyworld_> it attaches a function to a 'lp.app.picker.connect' namespace
<rick_h> gah, yea that's my task after all this, getting JS out of the .pt files
<rick_h> wallyworld_: ok, so can that code be moved to lp.app.picker YUI module?
<wallyworld_> the issue here is that we have all this js in pt files and we need a global namespace to be able to glue stuff together
<wallyworld_> no, because it's bespoke code
<wallyworld_> the picker is generic infrastructure
<rick_h> wallyworld_: so that's why I've got the huge glob of JS code in the layout macro file
<rick_h> since that makes it global
<rick_h> hmmm, but I see, it gets attached to the YUI module ugh
<rick_h> it's just so wrong...
<wallyworld_> hmmm. but here, the code is paramaterised according to the config of the field the picker pertains to
<wallyworld_> and that config is available in form-picker-macros
<wallyworld_> and nowhere else easily accessible
<rick_h> sec, let me pull up the code so I can see what it is
<wallyworld_> rick_h: wanna mumble briefly?
<rick_h> wallyworld_: yea, give me a sec to dock and reset the laptop
<wallyworld_> ok. sorry to intrude after EOD
<rick_h> all good, I was just merging/psuhing out convoy changes to trunk anyway
<wallyworld_> rick_h: meet you in Yellow 1-1?
<rick_h> k
<rick_h> http://paste.mitechie.com/show/521/
<lifeless> wallyworld_: hi
<wallyworld_> lifeless: g'day
<lifeless> wallyworld_: I've just replied to your MP about branches; I'm letting you know in case my reply doesn't make sense.
<wallyworld_> ok. let me have a look
<lifeless> general pattern of EAFP vs LBYL
<wallyworld_> your suggestion makes sense.
<lifeless> cool
<lifeless> thanks for looking
<wallyworld_> i was trying to do it in the picker though so that they could simply pick someone else if they misclicked
<wgrant> And this is on +register-merge, right?
<wallyworld_> yes
<wallyworld_> the same pattern as we use for bug tasking assigning
<lifeless> wallyworld_: this would still look like being in the picker, wouldn't it? They click on 'OK', it comes back and says 'nooo'
<wallyworld_> ie if a use is not a contributer
<lifeless> wallyworld_: then they can click to really do it, or click on someone else.
<wallyworld_> there is no ok in the picker, just the link of the person
<lifeless>  bah yes
<wallyworld_> and they use the picker as part of filling out the form
<wallyworld_> ie it's just one field
<lifeless> well, there are two places you need to do this
<lifeless> +register-merge and also when you request an additional review.
<wallyworld_> yes
<lifeless> oh interesting corner case for you
<lifeless> what if someone in the default review team does not have access to one of the involved branches.
<lifeless> e.g. they never click on the picker.
<wallyworld_> so the thinking was not not have them submit the mp with a reviewer who could not see stuff if we could catch it beforehand
<wallyworld_> hmmm.
<wallyworld_> that default reviewer would be auto subscribed atm
<lifeless> orly?
<wallyworld_> not sure if that's the best thing
<wgrant> I doubt it.
<wgrant> We cannot have autosubscription.
<lifeless> wallyworld_: or do you mean with your code changes?
<wallyworld_> no, i mean in my branch
<lifeless> ah
<wallyworld_> so i think we need a check on form submission as well
<wallyworld_> as when using the picker
<wallyworld_> so we catch the issue early if we can
<wallyworld_> but there's a safety net also
<lifeless> so, I would expect the default review team to get the subscription (which is near enough to equivalent), but I do think this needs to be vetted, not automatic.
<wallyworld_> ok, so if there are any visibility issues when the form is submitted, we return and let them confirm
<wallyworld_> as per your suggestion in the mp
<wallyworld_> so we attempt to alert them when using the picker, but also catch things at the back end if required
 * wallyworld_ has to go to airport to pick up someone, back soonish
<wallyworld_> lifeless: thanks for the input, much appreciated as always
<lifeless> de nada
<rick_h> StevenK: so heads up, convoy stuff is landed in their trunk.
<rick_h> StevenK: let me know if you need anything else then to look at setting up our packages with the directory traversal/etc
<StevenK> rick_h: Right, let me look at updating our packages.
<wallyworld_> wgrant: if i put an exported api  method on IBranchSet, how to a call it from js, using an instance of Y.lp.client.Launchpad()? Using get(...)?
<wallyworld_> in python it seems to be done as lp.branches.getByUniqueName(...) etc
<wgrant> wallyworld_: The URL is /branches
<wgrant> However you were calling something on the person URL, use /branches instead.
 * StevenK grumbles at his test convoy recipe
<StevenK> "nest-part packaging lp:~launchpad/convoy/packaging debian" doesn't seem to work :-(
<wgrant> StevenK: Oh?
<wgrant> Also, packaging-only branches are vile.
<StevenK> bzr: ERROR: Merge-into failed because Source tree does not contain debian.
<wgrant> Don't do it.
<StevenK> No?
<wgrant> merge
<StevenK> :-(
<wallyworld_> wgrant: yes, i was using a named_get() and thought i needed a context object
<wgrant> wallyworld_: You should just be able to use a URL. Otherwise construct an object from the URL first.
<StevenK> wgrant: I guess you want a one branch recipe
<wgrant> StevenK: No.
<wgrant> StevenK: Have a branch of trunk, with packaging added.
<wgrant> Create a recipe based on trunk with that merged.
<lifeless> wow
<lifeless> 'The membership status of Allison Randal (allison) in the team Launchpad
<lifeless> Stakeholders (private-canonical-launchpad-stakeholders) was changed by
<lifeless> the user himself from Approved to Deactivated.'
<wgrant> Yep
<wgrant> Longstanding bug about that.
<lifeless> anyone else see the glaring issue there?
<lifeless> wgrant: whats the bug # ?
<lifeless> ah 114753
<lifeless> hmm, https://bugs.launchpad.net/launchpad?field.searchtext=himself does not find it
<lifeless> I suspect our FTI logic didn't include description at that point, or something
<StevenK> lifeless: Ugh
<StevenK> 77 references to himself in our code -- most of them in doctests.
<StevenK> And referencing sampledata to boot
<lifeless> win
<lifeless> are you going to JFDI this wart?
<StevenK> lifeless: https://code.launchpad.net/~stevenk/launchpad/user-himself/+merge/90376
<benonsoftware> Hello
<benonsoftware> I am wondering how do I update my local install of Launchpad?
<nigelb> rocketfuel-get will work.
<benonsoftware> So just run that in the repo folder and it should work?
<nigelb> Well, how did oyu setup the first time?
<nigelb> If you did rf-setup the first up, rf-get should "Just Work"
<benonsoftware> nigelb: Ok, thanks very much
* wallyworld_ changed the topic of #launchpad-dev to:  | https://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugtasks: 4*10^2
<wgrant> stub: We've been running with targetnamecache search disabled for about a year.
<wgrant> And I have a branch to remove the only other use of it.
<stub> cool
<adeuring> good morning
<mrevell> Morning
* adeuring changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: adeuring | Firefighting: - | Critical bugtasks: 4*10^2
<benonsoftware> What does Firefighting mean?
<lifeless> bigjools: donw
<bigjools> lifeless: !  thank you
<StevenK> rick_h: Feel free to merge my approved convoy MP into trunk
<rick_h> StevenK: was there another one? I merged in your tests_require fix last night
<rick_h> ah, the copyright stuff
<rick_h> StevenK: sure thing, will do
<rick_h> wallyworld_: you still around?
<rick_h> StevenK: ok, merged and pushed. Thanks for that!
* bac changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: adeuring,bac | Firefighting: - | Critical bugtasks: 4*10^2
<wallyworld_> rick_h: hello
<rick_h> wallyworld_: hey, just a heads up that I commented on your MP
<rick_h> small notes on just conventions, nothing big
<wallyworld_> just saw, thanks. i used what i cargo culted from elsewhere for the wait
<rick_h> yea, definitely
<wallyworld_> i didn't want to use a simulate - i was black box testing the method which was supposed to raise the event
<wallyworld_> so all i needed to do in the test was subscribe to it and see that it was called with the right arguments
<rick_h> right, cool
<wallyworld_> i'll look at using wait/resume, although there should still be a timeout for dafety so the test doesn't hang
<rick_h> it won't
<rick_h> so what happens is that the test hits and errors that "wait called without resume" and it fails
<wallyworld_> ok, will look into it
<wallyworld_> the validation work is a little more problematic - there's no notion ahead of time how many validators there are and what the chaining order should be
<wallyworld_> with attaching the callbacks to the namespace, it was easy
<wallyworld_> so i'll give it some more thought
<rick_h> k
<rick_h> well glad at least some of it was easy :)
<wallyworld_> async events are good but they make some things a lot harder, especially collaboration between loosely coupled components
<wallyworld_> where you need ordering or other behaviours
<wallyworld_> how goes the convoy deployment stuff?
<rick_h> yea, you have to control the flow and document the expected chain so you can figure out what's up when it fails
<rick_h> wallyworld_: honestly not sure atm. Convoy is done and merged and in trunk and StevenK was putting together an updated package
<wallyworld_> and controlling the flow is hard when an arbitary component needs to participate
<wallyworld_> i really want a way to register validation callbacks - that's a common pattern for this type of thing
<rick_h> wallyworld_: well, it depends. In some ways it's easy to bring in an arbitrary bit just by listening for the same event, but yea, it's harder for that guy to then effect what tohers do
<wallyworld_> and the global namespace was a convenient point to register them on
<rick_h> wallyworld_: right
<wallyworld_> and i want them to fire one after the other, and if one fails, the others don't fire
<rick_h> where's this code at? I'll see if I can look at it and see an easy way to attack it
<wallyworld_> there is none yet - it's all done by registering the callbacks and looping over them
<wallyworld_> no event based implementation i mean
<rick_h> gotcha
<wallyworld_> i'll have a think about it over the w/e
<rick_h> cool
<rick_h> sorry, bit cloud headed this morning due to cold meds so having fun following
<wallyworld_> i'm off to bed, talk later
<rick_h> enjoy
<wallyworld_> will do, very tired tonight
<deryck> Morning, all.
<deryck> If anyone needs tips on staying motivated while working from home, I'm happy to oblige.
<rick_h> deryck: I feel like i need to grow my hair out so I can get styling tips
<deryck> I'm not sure I could do that anymore, even if grew it out again. ;)
<sinzui> You get use gel to put your hair into tips, but stay away from open flames
<benji> hmm, fetching any code.launchpad.net URL never finishes for me
<abentley> deryck: I'm assuming bug comments and code review comments are unicode.  Let me know if that's wrong.
<deryck> abentley, that's right.
<flacoste> bigjools: about question https://answers.launchpad.net/launchpad/+question/182545, i'm going to contact non-virtualised ppas owner, that's Archive where purpose = 2 and required_virtualized = False, right?
<bigjools> flacoste: yup
<flacoste> thx
<sinzui> adeuring, bac: do either of you have time to review https://code.launchpad.net/~sinzui/launchpad/mailing-list-name-field/+merge/90479
<bac> sinzui: i do
<sinzui> thank you
<bac> sinzui: when you had the pre-implementation call with yourself did you act it out with puppets?  any disagreements?
<sinzui> I quite. I wrote one thing 9 hours ago in a somnambulitic state. After 4 hours of sleep, I realised that +review on users was also broken so I told my previous self to fix the base view instead of creating a whole new view to support teams
<bac> sinzui: i'm confused by TeamAdminisiterViewTestCase -- the team doesn't (appear) to have a PPA or mailing list, so why is the field not there?
<sinzui> bac: the ppa/mailing lists tests are from the mixin
<sinzui> The tests I wrote verify the fields show to ~admin or ~registry
<bac> sinzui: gotcha
<bac> sinzui: based on that it looks good.  i'll finish up shortly.
<sinzui> bac: these are the tests I converted to the mixin: http://pastebin.ubuntu.com/819036/
<sinzui> diffs often no show what was really done
<rick_h> allenap: benji can either of you check and make sure this is the changes you decided as the *right* way at the end of yesterday? https://code.launchpad.net/~rharding/wadllib/fix_tests_922599/+merge/90492
<rick_h> allenap: benji and then while I'm getting you guys should we remove buildout from this since it doesn't work in py3?
<rick_h> james_w: ^^ as well
<benji> why is code.launchpad.net not working for me?  when I look at the source of the page it looks like it's all there; I wonder if there's some JS request that's hanging
<rick_h> benji: if there is should be able to see it in the dev tools
<SpamapS> OOPS-24aa7bfe6e5dbf33996f05a9614d7d02
<rick_h> benji: screenshot of the waterfall from the dev tools?
<benji> rick_h: yeah, that's what I was thinking of
<SpamapS> I believe this is a known bug...
<SpamapS> is anybody working on it? (regression where searching for New/Undecided bugs with ubuntu-server as the bug administrator times out
<rick_h> SpamapS: I don't think I've seen this one recently. adeuring this isn't any of the timeout stuff you were poking at right?
<james_w> rick_h, I don't think I would change the test. There's a test just below for a stream input, and changing it means that string inputs aren't tested
 * adeuring is looking...
<SpamapS> Its been over a month where ubuntu-server can't triage bugs.. :-/
<rick_h> james_w: ok, let me change that back then and see if that didn't help the tests pass for me
<SpamapS> I had a chat w/ lifeless in here about it.. he did some query plans and showed where it would take 7 minutes or something like that
<adeuring> no, that's nothing I touched. It's probably not easy to fix, I'm afraid
<SpamapS> I think we'll have to write a report to be able to triage then.. and just query each package that we supervise individually. :-/
<adeuring> SpamapS: yes, that could help. ANd actually, the slowest query is just a COUNT(*) for bugtasks related to ubuntu -- but the count is limited to bugs the user can see. Removing this filter should not do any harm
<SpamapS> adeuring: I'm being told this only happens because I'm in too many teams and so the number of bugs I can see is too high.
<adeuring> SpamapS: maybe -- but the Ubuntu bug searches tend to time out too often anyway.
<jml> gary_poster: I'm (finally!) watching Rich Hickey's talk on simple vs easy. The graph on development speed looks painfully, painfully familiar.
<SpamapS> Could this bug bug 901122 ?
<_mup_> Bug #901122: New bug listings need to preload more attributes <bug-columns> <regression> <timeout> <Launchpad itself:In Progress by deryck> < https://launchpad.net/bugs/901122 >
<gary_poster> jml :-)
<gary_poster> glad you are watching; hope you enjoy.
<adeuring> SpamapS: no, that's another issue that might crrep in once the cause for the timeout you are seeing is fixed ;)
<jml> gary_poster: yeah, I am enjoying it so far.
<gary_poster> great
<jml> gary_poster: it's good having such deep thinking clearly expressed
<jml> reminds me of the opening SICP lectures in some ways.
<sinzui> jcsackett, Maybe you need to fix a trivial bug that really irks you
<gary_poster> jml, clear expression of deep thinking: yes.  Hickey has impressed me with that repeatedly.  he gave another talk advocating hammock-driven development, which I think is related. :-)  I haven't taken time for SICP lectures; probably should.
<jml> gary_poster: the sicp book is a little blurred with the lecture in my mind. Am pretty sure it has most of the same stuff and can be consumed about 5x faster
<gary_poster> heh, cool
<gary_poster> will give it a whirl
<rick_h> james_w: ok, updated though line 55 seems like a giant line of hack https://code.launchpad.net/~rharding/wadllib/fix_tests_922599/+merge/90492
<james_w> rick_h, it does seem to solve the problem though :-)
<rick_h> james_w: yea, exactly, ok. I'm going to put this up for review and try to get the powers to review it. I *think* this is the only blocker for the release
<rick_h> the other items, fixing the buildout (or removing it) can come later
<james_w> yeah
<rick_h> cjwatson: allenap benji any of you guys check out https://code.launchpad.net/~rharding/wadllib/fix_tests_922599/+merge/90492 please?
<rick_h> and let me know what should be done from here on the buildout side?
<abentley> deryck: should BugComments and CodeReviewComments have separate config values for the maximum length before a "Read More..." link?
<deryck> abentley, I would tend toward saying no, that we should converge.  But if it's too hard to converge -- i.e. on form really needs longer snippets -- I'm fine to have two different configs.
<benji> rick_h: it looks good to me; want an approval?
<abentley> deryck: happy to converge.
<rick_h> benji: please
<deryck> abentley, awesome.
<benji> rick_h: done with one small suggestion
<rick_h> benji: awesome, ty
<benji> my pleasure
<jml> gary_poster: I don't think I understand what he's saying about using data for information, rather than objects.
<gary_poster> jml, it's been awhile since I've seen it now, but he, and Clojure, have an opinion that standard object oriented design has gone off the deep end.  We should have simple data structures that represent our "objects" and not code that has both data and behavior.  Clojure then has a simple data-y concept of type that can be applied to a data structure,  and generic functions and another simpler and faster dispatch syst
<gary_poster> em.
<gary_poster> I suspect that's the background for (and/or current best practical example of) what you mentioned but could be wrong.
<jml> gary_poster: hmm. interesting. I can begin to see what he's saying but don't grok it the same way as his other points. I wish I knew more about Clojure.
<jml> (not enough to actually *learn* more right now, you understand)
<gary_poster> lol, I understand
<rick_h> anyone know how to tell gpg which key to use to sign a file?
<gary_poster> rick_h, that sounds suspiciously like the first line to a joke.
<jml> rick_h: look into the --local-user and/or --default-key options
<rick_h> gary_poster: heh, looks like I have to change the --default-key
<rick_h> jml: yea, that seems to work thanks
<rick_h> this whole split private/work life is a mess sometimes heh
<deryck> rick_h, you mean you try to split work and private life? ;)  you're bordering on lifeless levels of online time. :)
<rick_h> heh
<rick_h> deryck: have to make up for being a bit dense :)
<rick_h> like where is this magic launchpadlib/bin/py
<rick_h> bah, that's a buildout reference I bet
<deryck> yeah, buildout create bin/py.  for most projects.  I don't know how launchpadlib is setup.
<rick_h> right, the bin/py threw me off hunting. I'm on the right track again I think
<rick_h> ok, maybe not. So the docs say I'm looking for: launchpadlib/contrib/upload_release_tarball.py
<rick_h> but my launchpadlib library has no contrib, seperate package?
<rick_h> deryck: do you know if anyone can upload the pypi package or do I need to get a special user credientials? pypi says the author is: LAZR Developers
<deryck> rick_h, I don't know.  My guess would be we have a login for LAZR dev.
<deryck> gary_poster, do you know ^^ ?
<rick_h> james_w: so I *think* I've got the 1.3.0 release all set on the LP side, just not in pypi yet
<rick_h> james_w: let me know if there's any issues and I can check it out
<james_w> rick_h, excellent, thanks
<rick_h> james_w: sorry for the delay on all that :)
<james_w> no problem, thanks for catching the bug and promptly fixing it
<gary_poster> deryck, rick_h, you'll need creds for pypi.  looking at pypi.
<rick_h> gary_poster: ty
<gary_poster> rick_h, is your pypi id rick_h?
<rick_h> gary_poster: no, actually not sure beena while
<gary_poster> rick_h, tell me what it is and I can give you permissions
<rick_h> gary_poster: sure thing, getting reminder sent one sec
<gary_poster> cool
<rick_h> gary_poster: mitechie
<gary_poster> rick_h, cool.  mitechie has owner permissions
<rick_h> ty much
<gary_poster> welcome
<rick_h> gah, py3 damn you!
<rick_h> gary_poster: I'll wait a bit to see if something needs time to update, but getting Server response (403): You are not allowed to store 'wadllib' package information
<gary_poster> rick_h, sounds like you need wadllib too! :-) I only did launchpadlib.
<gary_poster> rick_h, there is no big switch :-/
<gary_poster> rick_h, doing wadllib now
<rick_h> ah! sorry yea.
<gary_poster> rick_h, pypi is not a speed demon.  You are now added for wadllib.  any others?
<rick_h> gary_poster: ty much, that worked
<rick_h> gary_poster: not atm, wadllib was all I needed for now
<rick_h> james_w: so not sure if you need pypi, but should be up there soon as well
<gary_poster> cool welcome. rick_h I'm a one-stop-shop for a lot of our pypi privs; feel free to ask for more. ;-)
<james_w> thanks rick_h, we're fine with LP
<abentley> bac: Could you please review https://code.launchpad.net/~abentley/launchpad/attachment-timeout/+merge/90514 ?
<bac> abentley: sure.
<abentley> bac: thanks.
<bac> abentley: you know it seems we have this conversation about this time every friday!  :)  you're very consistent.
<abentley> bac: :-)
<abentley> rick_h: When you file bugs on Launchpad projects such as wadllib, please also triage them.
<bac> abentley: why did you decide to put the "Download" link at the top of the message instead of below?  i'm not asserting one is preferable but i was surprised to see it there.
<abentley> bac: I thought that if you want to download the comment, you probably don't want to read it in the Launchpad UI.
<bac> abentley: but you'll read the snippet before deciding, perhaps.
<abentley> bac: The snippet is 3200 characters long.
<bac> abentley: is the condition at line 557 correct?  shouldn't it be too_long_to_render?
<abentley> bac: No, the conditional is what was intended.  It seems like downloading is a reasonable option to present for long comments, even if they're not so long that they cannot be rendered.
 * bac looks again
<abentley> bac: Thanks!
<sinzui> bac: do you have time to review a follow up to my previous branch: https://code.launchpad.net/~sinzui/launchpad/administer-team/+merge/90530
<bac__> sinzui: in a bit.  i need to reboot.  and i may disappear due to flaky internet
<sinzui> understood
* gary_poster changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: adeuring | Firefighting: - | Critical bugtasks: 4*10^2
#launchpad-dev 2012-01-28
<olli> hi
<rick_h> howdy
<nigelb> The private team blogpost is so confusing!
<rick_h> heh, all the privacy stuff tends to lean that way I think. It's a hard thing to solve/get right
<nigelb> I read twice and it took me 5 minutes to understand what the blog post meant.
<nigelb> I'm still not confident yet.
<jml> lifeless: a day later than promised: https://code.launchpad.net/~jml/testtools/assert-raises-lambda/+merge/90574
 * jml gone
<james_w> lifeless, around by any chance?
<lifeless> fsvo
<james_w> hi lifeless
<james_w> I'm a bit confused by oops-tools currently
<james_w> I can trigger an oops at http://ec2-107-22-26-52.compute-1.amazonaws.com/pkgme/+oops and I can see it go across amqp and it's apparently inserted in to the db at http://ec2-107-20-69-109.compute-1.amazonaws.com/?oopsid=OOPS-884555d8407b0fe9fc62ce0f34094370 but it says there's no matching oopses
<lifeless> select * from oops_oops where oopsid = '
<lifeless> OOPS-884555d8407b0fe9fc62ce0f34094370'
<lifeless> ;
<lifeless> (e.g. have a direct poke at the DB)
<lifeless> james_w: what consumer are you using to pull it off of amqp ?
<james_w> amqp2disk
<james_w> hmm, maybe it's a setup issue actually
<james_w> (1 row)
<lifeless> ok, so if a literal lookup matches, in the DB, it is inserted
<lifeless> and the view should always be searching on the literal string regardless of any heuristics
<james_w> ah, I think it was running amqp2disk with a relative pathname
<james_w> the exists() check was failing
<james_w> http://ec2-107-20-69-109.compute-1.amazonaws.com/?oopsid=OOPS-c470f2678fd5ff65b48a522ae1a35816
<lifeless> yes
<lifeless> nice
<lifeless> shiny shiny
<james_w> it seems like timeline/oops-tools need to be generalised at bit?
<james_w> oops-tools assumes that everything is sql?
<lifeless> well
<lifeless> the UI certainly claims it does
<lifeless> and it does the %s %d substitution on everything
<lifeless> OTOH thats working quite well for LP for e.g. librarian downloads and so on - mapping different librarian requests to one pattern for aggregation
<lifeless> if you would like to make it better, would definitely love to see that
<james_w> I need to work out why the sql hooks aren't working here
<james_w> http://ec2-107-20-69-109.compute-1.amazonaws.com/?oopsid=OOPS-304f3e703162fa09f30065c7d582c890 is from when they were
<lifeless> it seems like you're finding this a bit tricky to debug / sort out
<lifeless> if you have suggestions for making that process easier, +1
<james_w> well, it's complicated by the fact that my internet is crappy today and I'm doing this remotely
<james_w> plus timeline_django is pretty damn hacky, so it's going to be fragile
#launchpad-dev 2012-01-29
<james_w> also, it seems oops-tools only shows a fixed set of things from the report?
<james_w> if you put interesting things in to wsgi_environ['oops.report'] you look at the raw oops to get the information?
<james_w> anyway, http://ec2-107-20-69-109.compute-1.amazonaws.com/?oopsid=OOPS-163ddd75a28f733d833626ba08f80c9d shows the sql logging working too
<lifeless> james_w: there is a bug open on that
<james_w> oh good, glad it's not desired :-)
<lifeless> james_w: looking good
<lifeless> I'll install django-timeline on lp-oops.c.c once you release it :)
<james_w> lifeless, do you think that oops_wsgi should get something like ++oops++ ?
<lifeless> that could be nice
<james_w> it needs a bit of cleanup before it can be released
<james_w> I haven't implement executemany yet, as I wasn't sure what it should put for the detail, given that it's not one statement
<james_w> but nothing in django core calls it, so I guess it's not too important :-)
<james_w> plus we still need to figure out how to get the oops id to the user for it to be excellent
<james_w> allocating the id early should be pretty easy I think, but would that break the amqp/datedir fallback?
<james_w> time for dinner, thanks for your help
<lifeless> so, getting id to the user; ideally the fixed django that lets oops-wsgi take over, but failing that we can allocate early, yes.
<james_w> isnotforpasswordsyouidiot
<lifeless> \o/ all non-launchpad projects retriaged.
<lifeless> meep
<lifeless> 127 operational bugs (losa tagged)
<spm> don't look at how old some of those are, fwiw. :-P
<lifeless> indeed
<lifeless> just noting that we have things that are costing us sysadmin time
<lifeless> 127 distinct such things.
<lifeless> 'aieeeee'
<wgrant> lifeless: Can we remove targetnamecache now?
<wgrant> Search has been disabled since around this time last year.
<wgrant> bah, still used for sorting by location.
#launchpad-dev 2013-01-21
 * StevenK stabs the blueprints tests
<StevenK> test_hasspecification is calling _valid_specification on a projectgroup with an open spec, and an obsolete spec, and asserting both are returned
<StevenK> Which makes me think the test is buggy, since valid specifications are not obselete or superseded ...
<StevenK> wgrant: How goes the BFJO murder?
<wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/handleStatus-refactor-1/+merge/144049
<wgrant> StevenK: As a bonus it makes buildd-manager logging suck a bit less
<StevenK> Hah
<wgrant> When a build completes it will actually log that the build completed, and what its status was
<wgrant> Currently you get, on some types of failure, '******* rother is MANUALDEPWAIT *******'
<wgrant> Without any reference to what the actual build was
<wgrant> And on success or a real build failure, you get no logging at all :)
<StevenK> 8- @classmethod
<StevenK> 10+ @defer.inlineCallbacks
<wgrant> Already fixed and retesting locally
<StevenK> Right
<wgrant> Was hoping you wouldn't notice :)
<StevenK> It was the first change in the diff, bit hard
<wgrant> It doesn't actually make any difference, but yeah
<StevenK> wgrant: That is my only concern with that branch
<wgrant> Thanks
<StevenK> I love the _handleStatus_generic_failure refactor
<wgrant> StevenK: It's pushed
<wgrant> And passes tests too
<StevenK> wgrant: r=me
<wgrant> Thanks sir
<StevenK> Right, hopefully the thinko I've fixed makes the tests pass.
<wgrant> :)
<StevenK> Aw.
<StevenK> It fixed one test, though
<wgrant> How many failures remain?
<StevenK> 4
<wgrant> ah
<wgrant> ez
<StevenK> From -m blueprints -m registry
<StevenK> And I've also bent {Distro,Product}Series to my will, and destroyed Specification.completeness_clause
<wgrant> Excellent
<wgrant> As I plotted
<StevenK>  14 files changed, 383 insertions(+), 765 deletions(-)
<wgrant> Hmm
<wgrant> Only a 50% win
<wgrant> Although I guess it also fixes some omissions
<StevenK> It drops a massive amount of duplication
<wgrant> Right, I just expected more
<StevenK> Perhaps I've missed some bits that can die, though
<StevenK> Hm, I think my LEFT JOIN on product is screwing up my productseries call
<wgrant> StevenK: bugtasksearch does the same thing
<wgrant> You can probably steal ideas from there
<wgrant> It might well omit the check if called in a product or productseries context
<StevenK> It does it differently
<wgrant> How differently?
<StevenK> Currently, I do it all the time, and bugtasksearch will only do that check if product, distribution, productseries and distroseries are all unset.
<wgrant> Right, that makes sense.
<wgrant> The way bugtasksearch does it
<wgrant> Possibly because I wrote it :)
<wgrant> distribution/distroseries are obviously fine because they can't have a product
<StevenK> Yeah
<StevenK> So it gets included for person/project group
<wgrant> And product/productseries are fine because they aren't accessible unless they're active or you can see inactive projects.
<wgrant> person/projectgroup/*, yes
<StevenK> Now I have to pass in the context
<wgrant> You weren't already?
<StevenK> No, I pass in a base clause
<StevenK> Specification.productID == self.id or so
<wgrant> Ah
<StevenK> I could check base_clauses[0].expr2.table, but you might murder me
<wgrant> Might?
<StevenK> :-)
<StevenK> wgrant: So handleStatus_OK is up next?
<wgrant> storeBuildInfo is being demolished
<wgrant> Well, substantially revised
<StevenK> Ugh, I *think* this will fix the productseries failures
<StevenK> Indeed, it does.
<StevenK> wgrant: Is http://pastebin.ubuntu.com/1554189/ a 2.7 failure?
<wgrant> StevenK: There shouldn't be any more 2.7 failures, but that's possible.
<StevenK> Hm
<wgrant> I've seen this before
<wgrant> Perhaps 18 months ago
<wgrant> But i don't remember quite what it was
<StevenK> I've ignored it for the time being, trying to figure out these person failures
<wgrant> StevenK: Oh, I wonder if it's proxied
<wgrant> Yeah
<wgrant> 2.7 doesn't like securityproxied dicts
<wgrant> 2.6 is fine
<StevenK> Haha, so it is a 2.7 failure
<wgrant> I wonder how that was missed
<wgrant> Perhaps it is new
<wgrant> Ah
<wgrant> A change in September
<wgrant>     - Issue #15801: Make sure mappings passed to '%' formatting are actually
<_mup_> Bug #15801: mozilla-firefox-locale-tr: new changes from Debian require merging <mozilla-firefox-locale-tr (Ubuntu):Invalid> < https://launchpad.net/bugs/15801 >
<wgrant>       subscriptable.
<wgrant> That would be how it was missed, I guess.
<wgrant> A regression in 2.7.something
<StevenK> I can rSP to work around it
<wgrant> http://hg.python.org/cpython/rev/2801bf875a24/
<wgrant> (I assume)
<StevenK> Plausible
<lifeless> StevenK: shouldn't need to, just make sure all the attributes needed for the new check pass
<adeuring> good morning
<cjwatson> wgrant,StevenK: I have a QAed DB patch - do you think I could have an FDT?
<czajkowski> having our only maintenace folks in the one timezone is not ideal :(
<cjwatson> I'm not in a desperate rush beyond "next couple of days would be nice"
 * mwhudson laughs at cjwatson getting 30k+ lines of LoC credit in one go
<cjwatson> the graph looks awesome :)
<cjwatson> http://people.canonical.com/~cjwatson/tmp/loc-cum.png
<mwhudson> would be bad if that was a stock market index
<lifeless> cjwatson: waaaa?
<bigjools> what did you nuke?
<cjwatson> database/schema/launchpad.html, last updated 2009
<cjwatson> make target to build it locally in a few seconds if anyone cares
<cjwatson> kind of cheating of course, but misleadingly obsolete doc is a maint problem imo
<maxb> Never hesitate to poke fun at the arbitraryness of a metric? :-)
<lifeless> well, its accurate so far :>
<lifeless> cjwatson: nice catch
<wgrant> cjwatson: Sure, we should be able to do it in a few hours
#launchpad-dev 2013-01-22
 * StevenK stabs specification filtering
<StevenK> Two methods use SpecificationFilter.{ACCEPTED,PROPOSED,DECLINED} for utterly different things
<wgrant> :)
<wgrant> Series and sprints?
<StevenK> Yeah
<StevenK> The sprint stuff doesn't use search_specifications(), it's just too wacky
<StevenK> Back down to one failure
<wgrant> :)
<wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/handleStatus-refactor-2/+merge/144203
<StevenK> wgrant: r=me, perhaps you could run update-copyright. I'm a little concerned about the BUILDERFAIL method now, you just reset the BQ record, and merrily go on your way.
<wgrant> StevenK: The thing is that BQ.reset already sets the status to NEEDSBUILD
<wgrant> But I think BUILDERFAIL is the case that causes some of the broken builds we occasionally see
<wgrant> So it probably already doesn't work for some reason anyway :/
<wgrant> It'll all be reworked when BFJO dies soonish
<StevenK> wgrant: Right
<StevenK> wgrant: I've been staring at this failure for too long. Can haz help?
<wgrant> StevenK: Sure
<StevenK> wgrant: http://pastebin.ubuntu.com/1557408/ is the diff (It's -r submit:, so should include everything)
<StevenK> wgrant: lp.registry.tests.test_person.TestSpecifications.test_in_progress is the failing test, which is failing since the query is returning both the STARTED and IMPLEMENTED specs
<wgrant> StevenK: Looking
<wgrant> StevenK: Look around line 1170 of the diff
<wgrant> in_progress was previously handled inside a "COMPLETE not in filter" guard
<wgrant> Which also added an INCOMPLETE filter
<StevenK> Bah, and that fixes it
<StevenK> Adding INCOMPLETE to the block was probably the kicker
<wgrant> Yeah
<wgrant> Is it really worth having a separate in_progress flag on the underlying method?
<wgrant> Should Person.specifications just map it to (STARTED, INCOMPLETE)?
<StevenK> I can drag that bit back to Person.specifications
<wgrant> It doesn't seem valuable to have that flag on the underlying infrastructure
<wgrant> Given only a single method exposes it
 * StevenK refactors the second if default_acceptance block out
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/use-specification-aag/+merge/143630 updated
<wgrant> StevenK: That's quite a diff
<wgrant> I may be a while
<StevenK> Haha
<StevenK> wgrant: Do you want a rope around the waist so you don't get lost? :-)
<wgrant> Can you do the neck instead?
<StevenK> Haha
<wgrant> StevenK: Are Milestone and Sprint likely to be portable?
<StevenK> wgrant: I've run out of steam
<wgrant> Sure.
<StevenK> Milestone.getSpecifications() is nothing like anything else, and wants people too
<StevenK> So that's a bit difficult
<wgrant> StevenK: For in_progress, I was hoping you could just add INCOMPLETE and STARTED to the filter.
<wgrant> Although I'm not sure if those filters are meant to be unioned or intersected.
<StevenK> Oh, spec_filter.extend([INCOMPLETE, STARTED]) and get_specification_filters adds get_specification_started_clause ?
<StevenK> Those filters are a mess
<wgrant> Depends if they intersect or union
<wgrant> But I was hoping to keep all those workflow filters in one place
<StevenK> AttributeError: type object 'SpecificationFilter' has no attribute 'STARTED'
<StevenK> No clean up for you
<wgrant> StevenK: Sure, it probably doesn't exist already
<wgrant> But that doesn't mean it shouldn't.
<StevenK> wgrant: Right, so that's been abstracted back into search_specifications
<StevenK> wgrant: http://pastebin.ubuntu.com/1557550/
<wgrant> StevenK: I think you double-join product a couple of times, don't you?
<wgrant> As search_specifications includes the Product.active filter
<StevenK> wgrant: Which callsites do you suspect do so?
<wgrant> StevenK: ProjectGroup.specifications
<wgrant> Though it doesn't rejoin Product, it does duplicate Product.active
<StevenK> wgrant: Right, dropped
<wgrant> StevenK: At the top of search_specifications, can you explicitly put both default/options sets in a branch?
<wgrant> ie. if not default_acceptance: default = ...; options = ...; else: default = ...; options = ...
<wgrant> It's very difficult to understand now
<wgrant> Also, that for option in options thing looks like someone reinvented set intersection
<StevenK> wgrant: http://pastebin.ubuntu.com/1557566/
<wgrant> StevenK: The series targets had a different interpretation of SpecificationSort.DATE, too
<wgrant> StevenK: After adding STARTED was a perfect place to commit :)
 * StevenK reads the SpecificationSort.DATE code in distroseries and groans
<wgrant> StevenK: Also, does get_specification_privacy_filter want an admin shortcircuit like I think the others do?
<StevenK> get_branch_privacy_filter has no admin shortcut
<wgrant> Indeed
<wgrant> Probably OK, then
<wgrant> StevenK: Does _preload_specifications_people still need the SQL() bit now that most callsites are ported?
<wgrant> Also, is that deliberately avoiding getPrecachedPersonsByID?
<StevenK> wgrant: I have *no* idea about that code
<StevenK> I touched it once, and the world broke, so I left well enough alone
<StevenK> Right, I get the series change for DATE
<StevenK> wgrant: STARTED code seperated out and commited, http://pastebin.ubuntu.com/1557578/ is remaining
<wgrant> StevenK: Don'y youy need the PROPOSED/ALL filter in the sorting bit?
<StevenK> I was ignoring that bit, since it acts the same
<wgrant> It's a change in behaviour
<wgrant> Previously if I asked for ALL or PROPOSED from a distroseries it'd order by datecreated
<wgrant> Now it will order by date_goal_decided
<StevenK> Bah
<StevenK> I misread date_completed as datecreated
<StevenK> wgrant: http://pastebin.ubuntu.com/1557587/
<wgrant> StevenK: Isn't that show_proposed check inverted?
<wgrant> Also, you use .intersection there but & in the previous hunk
<wgrant> In fact, I'd merge the show_proposed into the else
<wgrant> so 'if default_acceptance and not (show_proposed & spec_filter):"
<StevenK> wgrant: http://pastebin.ubuntu.com/1557595/
<wgrant> StevenK: Looks reasonable
<StevenK> wgrant: Diff on the MP updated
<stub> $ bzr branch lp:ubuntu/postgresql-9.1
<stub> bzr: ERROR: Revision {package-import@ubuntu.com-20120925054023-lv75ptpdzvjdhkbw} not present in "Graph(StackedParentsProvider(bzrlib.repository._LazyListJoin(([CachingParentsProvider(None)], []))))".
<stub> Anyone know if that means my repo is screwed, or if the lp branch is screwed?
<wgrant> StevenK: The LP branch
<wgrant> Er, stub
<StevenK> wgrant: Still looking?
<wgrant> StevenK: r=me with a comment
 * StevenK has fear
<wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/bfjo-shrink/+merge/144227
<StevenK> wgrant: The first paragraph of the description almost requires a diagram.
<wgrant> Quit.er
<wgrant> Quite.
<StevenK> However, r=me
<wgrant> Thankyou sir
<wgrant> Is your spec branch going through EC2?
<StevenK> No, I'm playing buildbot bingo
<wgrant> Aw
<wgrant> I was hoping for a 10 commit streak.
<StevenK> I've ran -m registry -m blueprints enough
<StevenK> So BFJO is dead. But only sort of, since you rename another class to its name
<wgrant> Well
<wgrant> BFJOD delegated to BFJO
<wgrant> And everything inherited BFJOD
<wgrant> But BFJO didn't do anything
<wgrant> So I merged them
<wgrant> And the new class is not terribly long for this world.
<StevenK> Holy crap. I won buildbot bingo.
<wgrant> Impressive
<wgrant> I guess the blueprint tests are crap :)
<lifeless> is bingo a false ok
<lifeless> or an ok ok
<StevenK> lifeless: An ok ok
<StevenK> IE, toss the branch at buildbot and see if it passes, since it's so much faster than ec2
<lifeless> cunning plan
<stub> Anyone know where the button to create a new sourcepackage recipe lives?
<wgrant> stub: On the branch page
<wgrant> It's not there for private branches, as recipes don't support private branches.
<stub> found it. not my branch so first step is working out where it was :)
<StevenK> wgrant: You lose buildbot bingo
<wgrant> I'm well aware :)
<czajkowski> morning
<lifeless> wait, what?
<czajkowski> lifeless: Goood morning
<lifeless> czajkowski: ;) good morning
<czajkowski> nothing good about snow mister nothing!
 * czajkowski cries at her inbox 
<czajkowski> it was all nice and tidy yesterday
<lifeless> czajkowski: heh
<czajkowski> how's the little one doing?
<lifeless> brilliantly
<adeuring> good morning
<lifeless> she was wandering around with my t-shirt on as an over-shirt this evening
<lifeless> cutest thing ever
<czajkowski> awww
<czajkowski> lifeless: is the heat bothering her at all ?
<lifeless> czajkowski: heat?
<czajkowski> you're in AU aren't you?
<lifeless> nope
<lifeless> NZ
<lifeless> AU == boiling heat, why I left there 2.5 years ago :)
<czajkowski> ahh
<czajkowski> now makes more sense
<czajkowski> I'll swap boiling for -8 and poxy snow!
<lifeless> much easier to warm up than cool down
<czajkowski> I like cjwatson and lifeless bugs, they can never be invalid or rarely a duplicate.
 * lifeless aims to please
<czajkowski> after this mornings lot of mails it makes me happy
<wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/ttbj-comply/+merge/144250
<wgrant> StevenK: Can you QA your blueprints thing?
#launchpad-dev 2013-01-23
<StevenK> wgrant: Ha. Hahaha
<wgrant> StevenK: Going well?
<StevenK> Still making a list of pages to hit up
<wgrant> Ah
<StevenK> Trying to find a projectgroup on qas that uses blueprints
<wgrant> https://qastaging.launchpad.net/heehee
<lifeless> !
<StevenK> wgrant: So that's Person, Product, ProductSeries, Milestone, Distribution, DistroSeries, and Sprint checked.
<StevenK> heehee has no blueprints, and something with private blueprints would be nice.
<wgrant> StevenK: It does, you might just not be able to see them
<wgrant> prop-auditorclient is in it
<wgrant> And I added a private blueprint to it last week to check this exact thing
<StevenK> Yeah, I can't view prop-auditorclient
<wgrant> I've made it public
<wgrant> The project group now correctly shows an empty blueprint list for an anonymous user
<wgrant> So it looks fixed
<StevenK> I still get the message for me
<wgrant> Which message?
<StevenK> "... Register the first blueprint in this project! If you have a proposal for a feature ..."
<wgrant> That's the empty blueprint list :)
<StevenK> Not so much a list as a wall of text
<wgrant> I believe it is the largest wall of text in the entire application, indeed
<wgrant> (ignoring help popups)
<StevenK> I still can't see the blueprint for prop-auditorclient, but eh
<StevenK> Oh, projectgroup didn't check for privacy. Right.
<StevenK> Now it does, since I made it do so.
<wgrant> Exactlyu
<wgrant> i've just shared the project with you
<wgrant> So you should be able to see the spec
<StevenK> And I can
<StevenK> Fantastic
 * StevenK goes to mark bugs as qa-ok
<StevenK> Did you want a deployment?
<wgrant> Indeed
<wgrant> I'm about to disappear out to lunch, so if you could arrange that...
<wgrant> Also, if it happens while I'm gone, do keep an eye on buildd-manager
<StevenK> I'll sort it out
<StevenK> wgrant: http://pastebin.ubuntu.com/1561501/
<wgrant> StevenK: Does it work?
<StevenK> The tests pass
<StevenK> I fear it doesn't close danilos bug
<wgrant> StevenK: H, though the privacy filter should be in the recursive query, not the outer one
<wgrant> eg. if we have A->B->C and I can't see B, I think this will do strange things
<wgrant> whereas if the filter's in the recursive bit, then I will just see A, not A and C
<StevenK> wgrant: Right
<StevenK> And both recursive_dependent_query and recursive_blocked_query are just a string
<wgrant> Maybe not for long :)
<StevenK> Yeah
<wgrant> Otherwise you can use that compilation thingy
<wgrant> convert_storm_clause_to_string
<StevenK> wgrant: The inner SELECT from the UNION requires the privacy check?
<wgrant> I haven't actually looked at the recursive query
 * wgrant looks
<StevenK> Which is on specificationdependency :-/
<wgrant> Yeah, so you'll probably want to adjust the second half of each union to do the filtering.
<wgrant> I didn't realise it was only over specificationdependency, which makes it a bit awkward
<wgrant> But it shouldn't be too bad.
<StevenK> So I have to join back to spec and then feed in get_specification_privacy_clause?
<wgrant> StevenK: Right.
<StevenK> Hmmm, on specification on dependency, though
<wgrant> Hm?
<StevenK> specificationdependency has specification and dependency, both of which are specification
<wgrant> Whichever one it returns
<wgrant> Which will be one for blocked, and the other for deps
<StevenK> Hm, my query is broken
<StevenK> wgrant: http://pastebin.ubuntu.com/1561618/
 * wgrant cries
<StevenK> wgrant: Hm?
<wgrant> StevenK: a) It's illegal to construct SQL with %. Legacy code is permitted as long as it's of the form "% sqlvalues(...)", but nothing else. Try using SQL() instead.
<wgrant> b) Have you verified the constructed SQL?
<wgrant> What does convert_storm_clause_to_string do when given multiple args?
<StevenK> It isn't, it's either an [Or()] or a [In()]
<wgrant> Ah
<wgrant> Still, examine the constructed SQL
<StevenK> wgrant: http://pastebin.ubuntu.com/1561630/
<wgrant> StevenK: lolwut
<wgrant> Oh, misread
<wgrant> StevenK: What doesn't it do?
<StevenK> Oh?
<wgrant> There's no actual SELECT there
<wgrant> Just a WITH RECURSIVE
<wgrant> Which is not completely helpful
<StevenK> wgrant: http://pastebin.ubuntu.com/1561633/
<wgrant> StevenK: and what's broken about it?
<wgrant> Other than some possibly missing parens
<StevenK> It was broken, I fixed it by reordering the FROM
<wgrant> Oh
<wgrant> I thought it was still broken :)
<StevenK> But I can't use %s
<StevenK> Is this a subtle hint that I should Storm-ify it?
<wgrant> You should be able to use SQL()
<wgrant> I don't think our current Storm With expression supports RECURSIVE
<StevenK> wgrant: The callsites do SQL(), the functions are expected to just return a string
<wgrant> You might want to rearrange that
<wgrant> Given the paucity of callsites that should be quite trivial
<StevenK> wgrant: http://pastebin.ubuntu.com/1561643/
<wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/bug-1100977/+merge/144432
<StevenK> 22if content.find(no_key_message) >= 0:
<StevenK> Sadface
<wgrant> StevenK: I think your linewrapping violates the UDHR.
<StevenK> Which linewrapping?
<wgrant> +        )""", params=(spec.id, convert_storm_clause_to_string(
<wgrant> +            *get_specification_privacy_filter(user))))
<StevenK> wgrant: And you didn't complain that I only fixed one of the methods? Disgraceful.
<StevenK> wgrant: http://pastebin.ubuntu.com/1561650/
<wgrant> I give up reading un-highlighted diffs easily :)
<wgrant> (--syntax=diff is your friend)
<wgrant> That's still a criminal offence in most jurisdictions, but less so
<StevenK> wgrant: params=(\nspec.id, ... )?
<wgrant> StevenK: Right :)
<StevenK> wgrant: http://pastebin.ubuntu.com/1561657/
<wgrant> StevenK: Actually, I'm surprised that works. 
<wgrant> Given you're passing a plain string into params
<wgrant> But trying to use it as SQL
<wgrant> It may work if you wrap it in SQL(), not sure
<StevenK> I've not actually tried it ...
<wgrant> Heh
<wgrant> You might actually be able to properly Stormify it with a bit of a hack
<wgrant> With('RECURSIVE dependencies(id)', Union(...))
<StevenK> DataError: invalid input syntax for type boolean: "Specification.information_type IN (1, 2)"
<StevenK> LINE 8:             WHERE sd.specification = d.id AND (E'Specificati...
<StevenK> Hahaha
<StevenK> Hmmm
<StevenK> ProgrammingError: can't adapt type 'In'
<StevenK> wgrant: I'm not sure about the Storm-ified version ...
<StevenK> Or indeed if I've done it correctly
<StevenK> wgrant: http://pastebin.ubuntu.com/1561722/
<wgrant> StevenK: The easiest way to be sure about it and that you've done it correctly is to see if the tests work
<wgrant> Bah
<wgrant> zeca was tested directly
<StevenK> Haha
<StevenK> I was about to point that out
<StevenK> CompileError: Don't know how to compile type Reference of <storm.references.Reference object at 0x795c850>
<StevenK> Thanks for being helpful, Storm.
<wgrant> StevenK: Did you do reference == reference?
<wgrant> That doesn't work
<wgrant> id == id
<wgrant> Or reference == id
<wgrant> works
<StevenK> Specification.id == SpecificationDependency.specificationID
<StevenK> That's the only ==, so I'm not sure which reference it's barfing on
<wgrant> Is it roughly simialr to the code that you pased?
<wgrant> +        'RECURSIVE blocked(id)', Union(Select(
<wgrant> +            (SpecificationDependency.specification,),
<wgrant> That might not work
<wgrant> You probably have to select the ID
<StevenK> wgrant: The only change is calling the new function further down
<StevenK> wgrant: 'RECURSIVE blocked', Select((id,), Union(... ?
<wgrant> StevenK: That'll try to select the builtin function 'id'
<wgrant> SpecificationDependency.specificationID
<StevenK> Which is already in the inner select?
<wgrant> StevenK: Is that a problem?
<StevenK> Just ... odd
<wgrant> Not sure how
<StevenK> Right, RECURSIVE blocked(id) is working because specificationID rather specification
<StevenK> Except it doesn't love blocked
<StevenK> So I get an actual statement that compiles but postgres doesn't love.
<wgrant> What does postgres complain about?
<StevenK> wgrant: http://pastebin.ubuntu.com/1561762/
<wgrant> StevenK: Ah, you should be able to see the problem there
<StevenK> Select((spec.id,), Union(... ?
<wgrant> No
<wgrant> Look at the first line
<StevenK> Oh, quoting RECURSIVE is a bad thing
<wgrant> Yes
<wgrant> With(SQL("RECURSIVE blocked(id)"), ...) might work
<wgrant> Not sure
<StevenK> ProgrammingError: syntax error at or near "("
<StevenK> LINE 1: WITH (RECURSIVE blocked(id)) AS (SELECT 13 WHERE (SELECT Spe...
<StevenK> Let's go with no
<wgrant> StevenK: Bah, it's too smart.
<StevenK> Heh
<wgrant> StevenK: Define your own With expression, I guess.
<StevenK> I don't think the Storm-ifying is buying us anything. If anything, it's far uglier
<wgrant> (or extend the one in our Storm branch, I guess, would be better0
<wgrant> Certainly
<wgrant> But it would have been a trivial way to integrate the privacy filter apart from the RECURSIVE issue
<StevenK> Toss it as a param is giving me a wierd error too
<wgrant> Well, params is for params
<StevenK> Oh
<StevenK> I feel %s calling, but I'll get stabbed
<wgrant> It seems that avoiding it here is somewhat non-trivial
<wgrant> So it may be permissible
<wgrant> But inserting the id like that as the original code did is strictly forbidden
<wgrant> Even in legacy code
<adeuring> good morning
<jtv> Good morning adeuring!
<adeuring> hi jtv!
<plomme> hi guys
<cjwatson> wgrant: I'd like to have a pre-imp chat about bug 1102870
<_mup_> Bug #1102870: Copies use naÃ¯ve ancestry check to calculate previous version for notifications and bug closures <package-copies> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1102870 >
<cjwatson> I *think* a sensible way to approach this is by moving NascentUpload.getSourceAncestry (and probably getBinaryAncestry too, for symmetry) somewhere more sensible
<cjwatson> However, it uses the deprecated DS.getPublishedSources method
<cjwatson> Rephrasing it in terms of Archive.getPublishedSources isn't too hard, but it means more potato programming since there's then no way to query multiple archives at once
<cjwatson> I kind of feel that the method belongs on DS anyway, since it's "give me the current published version of $package in this distroseries, for some set of pockets"
<cjwatson> But I'm not sure that there isn't a better place - do you have any strong feelings?
<cjwatson> (If I put it on Archive, then I have to do explicit version comparison between the results for each pocket, I think, and the method that does that will still have to live somewhere common - so I don't think putting it on Archive would help)
<cjwatson> It might be worth getting rid of one more use of DS.getPublishedSources in any case; something like http://paste.ubuntu.com/1563165/
<gmb> benji, bac, if either of you have time today, could you take a look at https://code.launchpad.net/~gmb/lp2kanban/encode-annotations-with-json/+merge/144372? It takes the description annotations work and uses JSON for encoding rather than hand-rolled key=value pairs.
<gmb> (per benji's suggestion)
<benji> gmb: sure (bac is a bit sickly at the moment)
<gmb> benji, Thanks.
<joey> dzin: hmm sinzui isn't here
<joey> dzin: maybe we need to go to #launchpad! :-)
<dzin> ok :)
<czajkowski> joey: he doesnt do LP stuff any more
<joey> czajkowski: ah ha
<cjwatson> wgrant,StevenK: I'd like to get https://code.launchpad.net/~cjwatson/launchpad/avoid-copy-archive-spam/+merge/144611 reviewed reasonably quickly, to ensure we stop spamming its victim
#launchpad-dev 2013-01-24
<StevenK> cjwatson: r=me
<cjwatson> Great, thanks
<StevenK> cjwatson: Do you want to cowboy it onto mawson while it plays through buildbot, or what is your plan?
<cjwatson> If you have bright ideas on disentangling the underlying cause (notes in the bug), that'd be lovely, or else I'll try to mock something up in tests
<cjwatson> StevenK: I was about to go to bed, TBH
<StevenK> cjwatson: If you want to crash, I can land it for you
<cjwatson> Will it be a problem if I don't QA it until tomorrow afternoon?  I'm going to be out tomorrow morning (hence wanting to go to bed)
<cjwatson> Well, I can land it
<StevenK> cjwatson: We have one revision waiting, and nothing urgent, so it should be fine
<cjwatson> Just QA will be with slightly less alacrity than usual
<wgrant> Given that we still have no builders and the change is isolated and trivial, QA on that seems less than essential.
<cjwatson> As I suggested in the MP, we can QA this without builders by injecting a production upload
<cjwatson> Probably quicker too
<wgrant> True
<cjwatson> Though I guess we need to mock up the build id in the directory name or something to get it into the right archive, but that's easy enough
<cjwatson> Anyway, it's on its way into buildbot, so I'll deal with that tomorrow; feel free to revert if it causes a problem, obviously
<StevenK> wgrant: How goes the BFJO murder?
<wgrant> I think all the failures are fixed
<wgrant> Another ec2 run is 3/4 done
<StevenK> Ah, how bad was the ec2 carnage?
<wgrant> Only about 60 failures.
<StevenK> Not bad.
<wgrant> StevenK: Looking
<wgrant> StevenK: Does the context of that specificationdependency.py change reveal another place that needs to respect privacy?
<StevenK> Unsure
<wgrant> Also, have you confirmed that this actually fixes the blocking issue?
<StevenK> I've confirmed it passes tests
<StevenK> Shall I cowboy it onto mawson?
<wgrant> If you can reproduce it on mawson today, that would indeed be a good idea
<StevenK> Reproduce the problem?
<wgrant> But the existing tests do not catch the blocking issue
<wgrant> Yes
<lifeless> 16:09 < ajmitch> heh
<lifeless> *Bah* copy-paste fail.
<ajmitch> quite
<StevenK> wgrant: https://blueprints.dogfood.launchpad.net/production-auditor/+spec/spec-c
<StevenK> I think I need to revert the change to make use of all_deps, rename dependencies to _dependencies, and write a method called dependencies that filters by visibility
<wgrant> StevenK: Why not just make dependencies filter by visibility?
<StevenK> 1) You need a user, and 2) dependencies is a SQLRelatedJoin
<wgrant> Sure
<wgrant> Does dependencies need to be an SQLRelatedJoin?
<wgrant> Hm, but if it's an SQLRelatedJoin then it shouldn't recurse
<wgrant> So how's it relevant?
<StevenK> We don't want it to recurse for the dep tree
<wgrant> Oh, does it only show immediate deps?
<StevenK> That is exactly what is causing bug 1095235
<_mup_> Bug #1095235: Bogus dependencies in Blueprint graph <lp-blueprints> <regression> <specifications> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1095235 >
<StevenK> Yeah, dependencies and blocked are immeadiate children and parents
<wgrant> Sure, but I thought the graph showed more than that
<StevenK> r16333 changed the dep tree to use all_deps and all_blocked, which is the entire tree
<StevenK> wgrant: It uses dependencies and blocked for each node
<StevenK> spec-c all_blocked shows 'spec-a' and 'spec-b' since it's recursive
<StevenK> spec-b all_blocked shows 'spec-a'
<wgrant> StevenK: So it just uses the recursive query to grab all the nodes, and then queries them all anyway?
<wgrant> If not, then what's the recursive query for in the first place?
<StevenK> I'm not sure, but the deptree didn't used to use them
<StevenK> OH
 * StevenK finally figures out this traceback
<StevenK> lib/lp/_schema_circular_imports.py, I will DESTROY you
<adeuring> good morning
<czajkowski> adeuring: morning
<adeuring> hi czajkowski!
<czajkowski> adeuring: cold and snowing over there?
<adeuring> czaqright, we have snow :) But "cold" is more a question of peception. It's not like in sebreia here ;)
<adeuring> s/sebreia/siberia/
<czajkowski> lol
<czajkowski> we have snow too :)
<czajkowski> poxy snow :(
<StevenK> czajkowski: It's currently 29 here ...
<czajkowski> SWAP
<lifeless> czajkowski: it was 45 a week ago
<czajkowski> tad warm :)
<cjwatson> wgrant,StevenK: Review of https://code.launchpad.net/~cjwatson/launchpad/avoid-copy-archive-spam-2/+merge/144727 would be good when you have a chance (and now I'm glad I bothered to QA that)
<plomme> hey
<plomme> I was wondering if there was anything new regarding bug 1100164 . I know you guys are crazy busy, just curious how far back this has been pushed.
<_mup_> Bug #1100164: Private projects are forbidden from having releases when only files are problematic <privacy> <private-projects> <releases> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1100164 >
#launchpad-dev 2013-01-25
<StevenK> steven@undermined:/tmp/summit% bzr grep dependencies
<StevenK> requirements.txt:# Non-standard dependencies
<StevenK> wgrant: ^
<wgrant> StevenK: great
<StevenK> Now I just need the webservice test to pass
<StevenK> ValueError: No value for required parameter 'user'
<StevenK> But it uses @call_with! Which is the whole point!
<wgrant> StevenK: What does the definition in the interface look like?
<StevenK> wgrant: http://pastebin.ubuntu.com/1567621/
<wgrant> StevenK: operation_parameters shouldn't be there, I don't think
<StevenK> Oh, that might be fighting with call_with?
<wgrant> Yes
<wgrant> operation_parameters specifies the parameters for the operation
<wgrant> Not necessarily the method
<StevenK> Ah
<StevenK> I thought you needed both. :-(
<cjwatson> I've pre-QAed r16450 on mawson this time, by way of a cowboy.  If it seems plausible to roll it out over the UK night, that would be cool.
<cjwatson> QueueInconsistentStateError: cjwatson QA bodge: refusing upload to copy archive
<cjwatson> 2013-01-25 00:07:48 DEBUG   Building recipients list.
<cjwatson> 2013-01-25 00:07:48 DEBUG   No recipients on email, not sending.
<cjwatson> (I couldn't get the actual QueueInconsistentStateError seen in the previous instances of this bug to happen on mawson.  I'm not sure whether it's gone away or I just wasn't trying hard enough.)
<wgrant> cjwatson: It might rely on the binary never making it into the primary archive
<wgrant> So there were no overrides to inherit
<cjwatson> Well, except it was in the primary archive on production ...
<cjwatson>        lha | 1.14i-10.4 | quantal/multiverse | source, amd64, armel, armhf, i386, powerpc
<cjwatson> Though it was removed from raring; I suppose that might matter
<cjwatson> Ah yes, the removal hadn't happened yet on dogfood
<cjwatson> I'll try that next time :)
<cjwatson> That would certainly explain a lot
<wgrant> Yeah, that'd be why it hasn't been noticed much before
<cjwatson> Indeed, all the other upload failures on http://people.ubuntuwire.org/~wgrant/rebuild-ftbfs-test/test-rebuild-20121221-raring.html were ones I removed in the same process-removals run on 2013-01-08
<wgrant> cjwatson: Great
<cjwatson> Good call
<wgrant> cjwatson, StevenK: Any objections to updating qastaging's DB over the weekend?
<wgrant> Will take qas down for about a day
<cjwatson> I'll be on leave and don't care
<cjwatson> I mean, physically away
<wgrant> Aha
<StevenK> It's what, 11 months out of date?
<wgrant> April, I think
<wgrant> So not quite
<StevenK> wgrant: However, no objections.
<plomme> hey wgrant. I know you're very busy so no pressure, I just wanted to know how far back you recon bug 1100164 would be pushed
<_mup_> Bug #1100164: Private projects are forbidden from having releases when only files are problematic <privacy> <private-projects> <releases> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1100164 >
<wgrant> plomme: I will hopefully be working on that this afternoon
<wgrant> it's a small fix.
<wgrant> I'd hope to work on it on Mon/Tue, but a lot of production issues have come up this week, sorry :(
<wgrant> hoped
<plomme> ok great thanks a lot =)
<plomme> yeah I know how that goes haha
<wgrant> Everything likes to fall apart at once :)
<plomme> yeah tell me about it. I've had a manic week as well. Issues never come alone
<StevenK> wgrant: https://blueprints.dogfood.launchpad.net/production-auditor/+spec/spec-c
<StevenK> wgrant: What do you see, if anything for the Dependency tree ?
<wgrant> StevenK: c â b â a
<StevenK> Hm, then you have access to B
<StevenK> wgrant: I thought I had removed you from seeing B, though :-(
<wgrant> StevenK: :(
<wgrant> I might be god at the moment
<wgrant> Though blueprints don't respect godpowers, do they...
<StevenK> Nope
<StevenK> I removed your APG on production-auditor
<wgrant> I can see B
<wgrant> But maybe that's because I'm an admin
<wgrant> Can I list B...
<wgrant> \Yes
<wgrant> StevenK: ~canonical has an APG
<StevenK> Ah ha
<wgrant> "permissions": {"PROPRIETARY": "ALL"}}
<StevenK> wgrant: ~canonical's APG is kaput, has B disappeared?
<wgrant> I AM BLIND
<wgrant> a and b are no longer on the graph
<StevenK> Right, that's what I expected
<StevenK> And it fixes danilos' Critical
 * StevenK marks r16450 as qa-ok
<danilos> StevenK, woohoo, cheers (though I am not here ;)
<StevenK> danilos: Haha! :-)
<wgrant> StevenK: Methods named 'dependencies' and 'blocked_specs' aren't a thing
<StevenK> Can't make them properties ...
<wgrant> Also, you uncached a few things -- deliberate?
<wgrant> StevenK: Sure
<wgrant> You can rename them :)
<StevenK> Bleh, I meant to revert back to cachedproperty
<wgrant> At the moment they look exactly like properties
<wgrant> Very confusing
<StevenK> wgrant: What would you suggest?
<wgrant> getDependencies, or something like that
<wgrant> Methods are verbs
<wgrant> StevenK: Hm, to retain webservice compatibility I guess you could leave dependencies as an unfiltered SQLRelatedJoin
<wgrant> lazr.restful will automatically filter out invisible things
<wgrant> (although the count will be wrong)
<wgrant> Not sure which is worse
<StevenK> Hmmm, neither
<lifeless> its all worse
<StevenK> lifeless: Thanks for your support. :-P
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/use-visibility-filter-spec-depends/+merge/144836 is updated
<StevenK> wgrant: Can haz review nao?
<wgrant> StevenK: r=me
<StevenK> wgrant: Thanks
<wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/bug-1100164/+merge/144856
<StevenK> 385+ tal:condition="context/information_type/enumvalue:PUBLIC"
<StevenK> wgrant: ^ Can that make use of the can_have_release_files that the other template uses?
<wgrant> StevenK: Sadly not, since that's an attribute on ProductRelease
<wgrant> I could move it to product, but meeeeh
<StevenK> That is my only niggle, r=me
<wgrant> Thanks
<StevenK> Curses, the blueprints on qas have one children
<StevenK> *child
<StevenK> And staging is in little pieces
<StevenK> Hmmmm, how do I find specs that have grandchildren
<adeuring> good morning
<gmb> benji, Do you think you'll get a chance to take a peek at https://code.launchpad.net/~gmb/lp2kanban/encode-annotations-with-json/+merge/144372 today, or is that too much to ask for?
<gmb> Ooh, that sounded a bit bitchy.
<gmb> What I meant was "or have you got too much on your plate?"
<gmb> Sorry benji :)
<benji> gmb: heh; sure, I can look at it in just a moment
<gmb> benji, Thanks :)
<benji> gmb: review done; looks great; I thought it might be nice to allow multi-line JSON so I sketched out some code to enable that
<gmb> benji, awesome, thanks.
#launchpad-dev 2014-01-24
<wgrant> cjwatson: Did I miss the sleep 15?
<cjwatson> ... oops
<cjwatson> you didn't see that.  (pushed)
<cjwatson> I think I'm the one who needs sleep 15 (hours)
<wgrant> Heh
<cjwatson> TBH it would probably have done fine anyway, given that ftpmaster pivots to the new dists pretty quickly
<wgrant> Yeah, non-instaneous retry is only valuable for !primary
<wgrant> But some PPAs can take a while
<cjwatson> Yeah
<wgrant> Also, did you leave it in WIP for four minutes, or do I have some email send delay bug to fix?
<wgrant> I only got the diff email 6 minutes ago
<cjwatson> no WIP state at any point
<wgrant> :/;
<cjwatson> might have taken me a few minutes to write the MP after pushing the branch, but I assume you don't mean that
<wgrant> Nah, I'm looking at MP creation time vs Date: header.
#launchpad-dev 2015-01-19
<blr> wgrant: privacy icon would work... thomi pointed out the other day who had filed that bug :(
<blr> didn't notice at the time...
<thomi> made me sad :(
<blr> there were some nice pieces on boingboing recently about him
<wgrant> Oh, indeed. I hadn't noticed that.
<thomi> well, now we're all sad, but not for long! it's beer time!
<blr> thomi: weren't you just complaining you drank too much at LCA? :)
<thomi> oh, except I have to drive to the airport.
<thomi> blr: yeah, but you can't quit these things too quickly
<thomi> gotta give your liver time to relax
<blr> right, have to taper off.
<thomi> besides, picking up tych0 in an hour, he's staying with us a few nights
<thomi> blr: wanna join us tomorrow after hackathon out on the town?
<blr> thomi: not sure I'm quite up for that yet, but I'll try to make it to the hackathon
<thomi> ok
<thomi> wgrant: I just noticed the lp team meeting on my calendar. Would you like me to attend?
<wgrant> thomi: Might as well, if you have time.
<thomi> sure
<cjwatson> Hi, this time I'm available just about on time
#launchpad-dev 2015-01-20
<thomi> so... I guess that filing a bug on launchpad at this time is somewhat risky :D
 * thomi files https://bugs.launchpad.net/launchpad/+bug/1412607
<mup> Bug #1412607: loggerhead does not wrap long lines or allow you to scroll <loggerhead> <ui> <Launchpad itself:New> <https://launchpad.net/bugs/1412607>
<blr_> thomi: the parent div just needs the css prop 'overflow: scroll'
<thomi> blr_: cool
<blr_> looks like the .diffinfo class has overflow: hidden set explicitly for some reason
<blr_> I don't see a good reason for that.
<blr_> could make sense in the context of another view though
<wgrant> It's probably for the expand animation on revision pages.
<wgrant> eg on http://bazaar.launchpad.net/~loggerhead-team/loggerhead/trunk-rich/revision/474
<blr_> ah..
<blr_> wrapping the lines would be better at any rate, horizontal scroll bars are not great.
<wgrant> YEah
<lifeless> just make the font smaller
#launchpad-dev 2015-01-21
<cjwatson> wgrant: In case you happen to look at it, I think any attempt to upgrade to ZTK 2.0a1 will run into https://github.com/buildout/buildout/issues/236
<wgrant> cjwatson: Odd, I don't remember that from last time I tried upgrading buildout.
<wgrant> I ran into buildout.cfg incompatibilities or removed features, I think.
<cjwatson> I tried it on lazr.sshserver.  I think 2.1.0 had the same problem, but generally buildout upgrades are massively confusing anyway ...  It could be that there weren't >= constraints in the relevant set when you tried, maybe
<cjwatson> (Though that seems unlikely)
<wgrant> Yeah, odd.
<wgrant> I didn't take good notes back then, unfortunately.
#launchpad-dev 2015-01-22
<cjwatson> wgrant: Please have a look at https://code.launchpad.net/~cjwatson/txpkgupload/fix-leak/+merge/247276 as soon as you get a chance; we've had three outages due to memleaks in the last couple of days, and while I don't fully understand why txpkgupload is suddenly so much worse than poppy was (possibly it's just slightly different organisation of the top-level classes causing different amounts of leakage, I'm not sure), this renders it ...
<cjwatson> ... stable for me and I'd like to get it rolled out before the weekend
<cjwatson> It's not desperately elegant use of Twisted before or after, but I argue it's at least less entirely unidiomatic now :-P
<wgrant> cjwatson: How very odd.
<blr> wgrant: cjwatson: going to need pygit2 for integration testing this charm, suppose I need to track down the libgit source tarball - do we want the one from jessie or sid?
<wgrant> blr: I'd backport the latest libgit2 from sid to the ~launchpad PPA.
<blr> wgrant: sounds good. want to have some functional tests for smart-http/ssh and possibly pack frontend/storage
<wgrant> blr: We have full end-to-end tests, but calling the git binary itself. It will be useful to be able to eg. inspect the repositories in the tests using libgit2, though.
<blr> wgrant: yep of course, but still think there's value in ensuring the services are exposed correctly after deployment
<wgrant> Oh, in the charm? Right.
<wgrant> Definitely.
<wgrant> s/charm/spec/
#launchpad-dev 2015-01-23
<cjwatson> wgrant: Could you supervise a txpkgupload rollout in 1h44m if I request it (and if we can find a webop)?
<wgrant> cjwatson: But of course.
<wgrant> cjwatson: We haven't deployed tmp-incoming yet, have we?
<cjwatson> We have.
<wgrant> Oh.
<cjwatson> RT#78156
<wgrant> Oh, I mean the actual code.
<wgrant> I know the dirs exist.
<cjwatson> Unless you mean we haven't deployed the txpkgupload that uses it yet, in which case that is correct.
<wgrant> But they're not yet used.
<wgrant> Right, so this is the first update deployment.
<cjwatson> Yup.  I got the deploymgr alias sorted out, so hopefully it's a two-liner.
<cjwatson> The memory use on pepo hasn't changed in the last hour or two.  I suspect it's correlated with script kiddies hammering on the ssh port, but hard to say.
<wgrant> Likely.
<wgrant> Thanks for fixing it.
<cjwatson> ... we hope
<wgrant> Heh
<wgrant> cjwatson: Did you look at deleting most of txpkgupload.filesystem? That gets rid of most of the Zopey deps.
<wgrant> Most of the methods have been obsolete since poppy classic died.
<wgrant> I guess all the launchpadlib stuff is pulled in by oops-datedir-repo...
<cjwatson> Not yet, no.  Good idea.
<wgrant> In fact that entire interface should just be shot and rewritten.
<cjwatson> Killing bzr from l.s's install_requires was some large percentage of txpkgupload's disk footprint all by itself ...
<cjwatson> But that was just a quick win
<wgrant> Seems like it should be easy to remove from the test deps too.
<wgrant> The only reference is in a test about logging, and there's no bzr-specific logging code, so that could all end up in lp.codehosting.
<cjwatson> Indeed it could.  That was just more than two minutes. :-)
<cjwatson> But it'd be a requirement for getting onto Python 3.
<wgrant> Yep.
#launchpad-dev 2015-01-24
<lifeless> wgrant: btw my account isn't being hacked, I'm poking at a new language mini-binding
#launchpad-dev 2015-01-25
<lifeless> wgrant: around ?
<wgrant> lifeless: Hi.
<lifeless> wgrant: I'm poking at LP with a non-python API client
<wgrant> lifeless: I'm going to be disappointed if it's not rust
<lifeless> wgrant: It's haskell FWIW
<wgrant> oh, that's acceptable
<wgrant> what's the issue?
<lifeless> wgrant: anyhow, the oauth library in it calls access-token with an Authorize header
<lifeless> rather than a body with the params
<lifeless> wondering how deep that is to allow w/LP
<wgrant> ah, xnox ran into a similar issue with something.
<wgrant> well, exactly the same issue
<wgrant> i don't recall why i didn't merge that
<wgrant> https://code.launchpad.net/~xnox/launchpad/devel/+merge/217079
<wgrant> lifeless: if you can't easily override the library's behaviour i can probably make LPmore forgiving in the coming week.
<lifeless> wgrant: I'm poking around at a patch, but since I'm still very rough at haskell its a bit of a cliff in analysis
<wgrant> heh, quite. i haven't done haskell properly since uni.
<lifeless> right now I'm staring
<lifeless> auth.hs:40:47:
<lifeless>     No instance for (MonadIO
<lifeless>                        (Control.Monad.Trans.Resource.Internal.ResourceT IO))
<lifeless>       arising from a use of âgetAccessToken''â
<lifeless> in the face
#launchpad-dev 2016-01-25
<lifeless> where is the bug tracker for the laptop testing tracker?
<lifeless> It keeps mutating bugs that are not part of Ubuntu, and I want to file a bug on it about that
<blr> lifeless: noticed your name cropping up in relation to that in my email... sadly laptop testing is something I know nothing about. Perhaps cjwatson will know when he's about.
<lifeless> blr: hmm, I haven't sent any email about that :) - just this ping here
<lifeless> blr: I presume wgrant is on leave or something ?
<blr> lifeless: you removed a tag from the qatracker supposedly
<lifeless> blr: oh yeah, I did
<lifeless> blr: from bug 10
<mup> Bug #10: It says "displaying matching bugs 1 to 8 of 8", but there is 9 <laptop-testing> <lp-bugs> <Launchpad itself:Invalid> <https://launchpad.net/bugs/10>
<blr> right
<lifeless> blr: which is an old lp bug
<blr> lifeless: stat holidays in au today and tomorrow
<cjwatson> I know nothing about this either, despite not being able to sleep again.  A bit of googling finds https://wiki.ubuntu.com/Testing/Laptop which may be of some help
<cjwatson> Possibly ubuntu-qa-website?
<blr> lifeless: this appears to be the offending item? http://laptop.qa.ubuntu.com/qatracker/milestones/352/builds/108233/testcases/1479/results
<cjwatson> blr: Are you planning to land https://code.launchpad.net/~blr/launchpad/bug-1536363-codereviewcomment-reply-git-oops/+merge/283382 today?
<blr> lifeless: is it linking bugs from the comment field? i.e. "10."
<blr> cjwatson: I would love to, but I've yet to resolve the gpg agent error I'm experiencing. :(
<blr> cjwatson: is it possible to land that on my behalf?
<cjwatson> Oh really?  Sure
<cjwatson> You should at least be able to turn off the agent and type in your passphrase every time though
<blr> yeah, quite perplexing, afaict, gpg is functioning perfectly fine, and the gpg-connect-agent responds to a health check
<lifeless> https://bugs.launchpad.net/ubuntu-qa-website/+bug/1537587
<mup> Bug #1537587: the bot "Ubuntu QA Website" keeps modifying bugs that are not Ubuntu bugs <Ubuntu QA Website:New> <https://launchpad.net/bugs/1537587>
<lifeless> blr: I don't know, its been doing it for years, and well, sigh.
<blr> cjwatson: if I enter my passphrase manually, bzr lp-land errors with: ERROR: Failed to GPG sign data with command "[u'gpg', '--clearsign', '-u', u'kit.randel@canonical.com']" - running clearsign manually is fine however.
<cjwatson> urgh
<cjwatson> blr: Anyway, edited the commit message to remove duplication with the tags that bzr lp-land will add, and sent to PQM.
<lifeless> cjwatson: thanks for digging up a plausible project
<blr> cjwatson: thanks for that. Hopefully can find the culprit this evening
<cjwatson> blr: damnit, that won't pass due to a pending db-devel exception run on buildbot
<cjwatson> blr: I've forced buildbot, will land that for you this coming morning
<blr> great, thanks
<daker> cjwatson: hi, is there a way to get the commit message from the webhook for bzr:push or git:push
<daker> ?
<cjwatson> daker: hm, not currently.  Could you file a bug explaining how you'd want to use this?  I don't think it would be too difficult to add - you can of course get hold of it using the information in the webhook payload but it would be more round-trips
<daker> cjwatson: sure i'll do
<cjwatson> At least in the git case we've already fetched that information immediately before dispatching the webhook, and I think that's true for bzr too
<daker> cjwatson: yes, i am building a launchpad integration for slack using webhooks
<cjwatson> neat!
<daker> cjwatson: https://i.imgur.com/bQrcu6i.png
<cjwatson> I've certainly got no problem including the information in the payload if it's going to be used.  We'd probably only send the new tip commit message though - is that sufficient?
<cjwatson> (since we similarly only send the new tip commit ID)
<daker> cjwatson: well that's enough but i would love to have the commit message
<cjwatson> that's what I said ...
<cjwatson> I meant the commit message only for the new tip commit, rather than for all commits between the previous state and the new state
<cjwatson> the latter is a bit awkward and potentially very large indede
<cjwatson> *indeed
<daker> cjwatson: ah sorry yes!
<daker> only the commit message for the new tip commit
<daker> cjwatson: bug 1537780
<mup> Bug #1537780: Provide the commit message in the webhook payload <Launchpad itself:New> <https://launchpad.net/bugs/1537780>
<cjwatson> Ah yes, db_branch.getTipRevision().log_body and err I think bzr_branch.repository.get_revision(bzr_branch.last_revision).message ?  I never really got on with bzrlib
<daker> cjwatson: i'll try that
<daker> also i'll the add the author of the commit
<cjwatson> well, that was more for my own reference
<cjwatson> unless you fancy trying to write the patch
<daker> cjwatson: no :D
<cjwatson> daker: could you give a rationale in the bug?  we may need to refer back to this years later wondering who was using this thing and why
<cjwatson> daker: so it's helpful to have more than a bare request
<daker> cjwatson: sure
<daker> cjwatson: good now bug 1537780 ?
<mup> Bug #1537780: Provide the commit message in the webhook payload <Launchpad itself:New> <https://launchpad.net/bugs/1537780>
<cjwatson> daker: yes, thanks
#launchpad-dev 2016-01-26
<daker> cjwatson: http://127.0.0.1:8080/2016/01/how-to-integrate-slack-with-launchpad.html
<daker> oupps :D
<daker> correct link : http://daker.me/2016/01/how-to-integrate-slack-with-launchpad.html
<cjwatson> daker: neat, thanks!  I assume you're working around your bug by checking out the branch in question or something
<daker> cjwatson: no for now i am not displaying the commit message
<daker> cjwatson: for the MR event, it would be great to have the person who had approved/merge the branch for ex
<cjwatson> I think we can only easily do the approver
<daker> cjwatson: yeah
#launchpad-dev 2016-01-27
<xnox> nice.
<xnox> daker, slack is this cool irc aka jabber done right or some such?
<blr> xnox: irc with animated gifs.
<xnox> blr, i guess it's one of those "SAAS" things =) -> Software as a Startup
<xnox> built using Startup as a Service
<xnox> wow it integrates with Ping... i was like wasn't that a dead iTunes social network for lady gaga monsters?! but no, it's "identity management" company.
<daker> xnox: yes :p
#launchpad-dev 2016-01-28
<s390x> "This action will reveal this team's name to the public."
<s390x> i'm trying to assign a private bug, in a private project to a private team.
<s390x> and I got this pop up.
<cjwatson>                 if (LP.cache.context.private !== undefined) {
<cjwatson>                     public_context = !(LP.cache.context.private);
<cjwatson>                 }
<cjwatson>                 if (private_person && public_context) {
<cjwatson>                     var yesno_content =
<cjwatson>                         "<p>This action will reveal this team's name to " +
<cjwatson>                         "the public.</p>";
<cjwatson> are you quite sure the prospective assignee is a private team?
<s390x> I go to the team page and i have a big red banner "The information on this page is private"
<s390x> however, i think said private team is not "part" of this private project.
<s390x> thus two privates, are exposed to each other.
#launchpad-dev 2020-01-20
<SpecialK|Canon> I need to bail on the standup later today because of a store meeting in ZA, sorry
<SpecialK|Canon> My main contributions: 1) I'm writing up OCI versioning notes for tomorrow 2) File your expenses from last week please
<cjwatson> Waiting for replacement phone to finish downloading a gazillion things over wet string before I do expenses :)
<SpecialK|Canon> zoom zoom zoom!
<pappacena> Tom is in vacation, and Ioana will take the day... there will be only me and colin at the standup today, then?
<SpecialK|Canon> pappacena: relatedly, you have two swap days!
<SpecialK|Canon> cjwatson: LP db/model structure as described in https://docs.google.com/document/d/1SheiqNbiiQu376XPWHEccLUupzpT_HPbADRtqp_boUY/edit#heading=h.wtrqreyakykt - how accurate is that now?
<pappacena> kristian, I'm thinking about taking Thursday and Friday...
<cjwatson> SpecialK|Canon: I've updated it a little
<SpecialK|Canon> cjwatson: fab, thank you!
<SpecialK|Canon> pappacena: excellent
<SpecialK|Canon> cjwatson: do we have a concrete decision on the OCIPS/OCIR linkage?
<cjwatson> SpecialK|Canon: I thought it was clear we weren't linking those at least until we had some practical experience with how people use bugs with OCI projects
<SpecialK|Canon> cjwatson: sorry, I've fallen into the trap of conflating the two kinds of series again
<cjwatson> Adding a recipe name to OCIRecipe allows us to have more than one recipe per project, which we can use for building for multiple channels
<cjwatson> I updated my DB patch for that this morning (https://git.launchpad.net/~cjwatson/launchpad/commit/?id=35651746e6218615c1f4259b64f0f0e6c7cbe3ab)
<cjwatson> Fairly sure that's in line with my discussions with William last week
<SpecialK|Canon> I think I entirely agree with you, I'm just finding all the places where it's important to be specific lest confusion occur, by walking into them again and getting confused - at least my edits should clarify this
<SpecialK|Canon> (Thanks!)
<cjwatson> Yep
<SpecialK|Canon> I've created a new doc that we all have edit rights to at https://docs.google.com/document/d/1vFuBS6ekm14udGFm4H6bHpuWT163kLUBavRM-uj5Aio/edit# and I'm merging in the predecessor with notes from last week
<SpecialK|Canon> Anyone happen to know why setting font to "Ubuntu Mono" occasionally makes GDocs treat it like a header in the TOC on the left?
<cjwatson> Afraid not.  Last time I hit that I think I worked around it by indenting or making paragraphs longer or something
<cjwatson> ultra-short standup :)
<pappacena> hahaha! Yes!
<cjwatson> probably just as well, I wasn't sure my network was going to hold up with phone + lxd downloading all the things
 * cjwatson considers Built-Using some more
<cjwatson> I think we probably need to track source name and version separately in the DB rather than using a SourcePackageRelease FK, because in general we don't have a particularly good way to know which SPR a given parsed relationship line refers to - things could get very confusing when PPAs are involved
<cjwatson> Hmm, OTOH we do want to check that the SPR exists at upload time
<cjwatson> So maybe this looks slightly like the multiple-archive stuff in DSCFile._getFileByName?
<cjwatson> Or maybe something involving expand_dependencies
#launchpad-dev 2020-01-22
<cjwatson> ilasc: Oh.  That test failure is actually from my fix-webhook-visibility-check branch I think
<cjwatson> So maybe it's my problem and not yours
<ilasc> cjwatson: :) ok, thanks for looking at it Colin! I'll definitely look at the failure in more detail once I'm done with the Turnip tests
<cjwatson> Light dawned immediately after the standup
<ilasc> :)
<cjwatson> ilasc: was that the only failure?
<ilasc> cjwatson: there was this too -  assumed it was because I was already running LP : https://pastebin.canonical.com/p/FnBZt5RXrY/
<ilasc> was going to re-run with LP not running today but never returned to the topic today
<ilasc> I noted it though - won't forget in case it is something real
<cjwatson> Yes, very likely just a clash with a running server
<cjwatson> ilasc: Fixed in fix-webhook-visibility-check.  You can rebase your branch if you like, but it's not necessary since my branch will necessarily be merged before yours anyway
<ilasc> cjwatson: sweet! thank you! will look at the fix - good opportunity to learn something (I didn't even know where the error was coming from :) )
#launchpad-dev 2020-01-23
<cjwatson> Could anyone look at https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/377974, which is a quick dependency upgrade?  And maybe also https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/377881 and https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/377946, which do some more Breezy porting?
<ilasc> cjwatson: they look good - one small comment / question on one of them (might be seeing double there though)
<cjwatson> thanks
<cjwatson> ilasc: also https://portal.admin.canonical.com/C123862/ FYI (that's your livefs webhooks DB patch)
<ilasc> cjwatson: thanks! was looking at the wiki to see what's process for db patches
<cjwatson> I have your turnip MP open for review though probably won't finish before your EOD
<cjwatson> ilasc: https://dev.launchpad.net/PolicyAndProcess/DatabaseSchemaChangesProcess is the master doc, but possibly confusing if you're coming to it cold.  The strictly-ordered steps from this point are (1) IS will do a fastdowntime deployment of your patch (2) we merge that revision from db-stable to master (3) you can land code that depends on that patch once it's otherwise good
<ilasc> :) thanks Colin, that's ok - I'm in the process of addressing comments from review of Webhooks for LiveFS - not blocked by Turnip
<cjwatson> I'm not completely sure (2) is documented clearly
<ilasc> cjwatson: oh, ok, your summary is far better then what I was reading :)
<cjwatson> But you might have noticed stuff like https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/377918 lately
<cjwatson> That was literally just "push a particular commit from db-stable to a branch in order to be able to raise an MP for it"
<cjwatson> (used to be a completely vile invocation of bzr pqm-submit that I always forgot ...)
<cjwatson> Anyway, at this point we wait for IS
<ilasc> :) ok, that example helps, thanks!
<cjwatson> Is anyone actually likely to be interested in reviewing the details of https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/377977 or https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/378001 ?  I'm somewhat inclined to just self-approve since they're almost entirely tests
<cjwatson> And giant
<cjwatson> (Beautiful Soup 4 porting work)
<SpecialK|Canon> I am largely through one of the bs4 ones
<SpecialK|Canon> I figured I'm deserving of some penance
<SpecialK|Canon> (by all means self-approve)
<SpecialK|Canon> I was just feeling curious
<SpecialK|Canon> (378001)
<cjwatson> to some extent this is mainly digging up stuff I did ages ago and never quite finished (and thus was still lying around in bzr branches)
<cjwatson> but beautifulsoup is currently the first thing that fails when one tries to bootstrap LP with py3, so it reminded me of it
<SpecialK|Canon> :D
 * cjwatson kicks off a full test run on the next branch in the series, which finishes off a few loose ends and actually removes Beautiful Soup 3
<SpecialK|Canon> niiiice
<cjwatson> SpecialK|Canon: Still around?  Would like a quick review of https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/378007 to unstick buildbot
<cjwatson> (local package fork vs. slight in-tree customisation ... on balance I think I still prefer the latter)
<SpecialK|Canon> cjwatson: done - cheers
<SpecialK|Canon> I'd definitely pick the latter as well
<cjwatson> thanks
#launchpad-dev 2020-01-24
<cjwatson> jelmer: Are there still remaining subvertpy blockers to getting bzr-svn or similar working with breezy?  If so can you explain how to reproduce them?  Maybe we can have somebody work on them ...
<cjwatson> jelmer: Even if we were to decouple codeimport, it would still mean doing the py3 transition at the same time as the port to breezy if we don't have svn support working before you drop py3 support, which sounds like it could add some challenges; so I'm keen to get that out of the way if it isn't completely impossible / doesn't involve a huge pile of extra work from you
<jelmer> cjwatson: there are basically two big challenges to getting bzr-svn working
<jelmer> cjwatson: subvertpy's working copy API needs to be fixed to run with a newer version of subversion
<jelmer> I got stuck getting that to work last time - it just segfaulted
<jelmer> I don't think there is anything in svn itself that uses the old wc API anymore, so it may just be fundamentally broken
<cjwatson> is that something subvertpy's own test suite shows up or is it only something higher-level?
<jelmer> subvertpy's tests show that once you uncomment some of the self.skipTest() invocations
<jelmer> the second big effort would be updating it to work with breezy's API
<cjwatson> I don't know if this is an unanswerable question, but do you know if that's going to require changes in breezy itself?  that is, could it in principle be done to work with breezy 3.0 even if it doesn't happen before you drop py3 support?
<cjwatson> Just trying to work out constraints
<jelmer> cjwatson: we have no immediate plans to drop Python 3 support, I'm happy to wait with that until it's in a state where Launchpad can migrate
<cjwatson> I have all the rest of LP ported (well, with the exception of utilities/community-contributions.py, which needs to be rewritten to work with git)
<jelmer> cjwatson: I can't think of major changes to Breezy that would be necessary, but it's possible there will be some
<cjwatson> ok, that sounds promising and gives me something concrete to dig into, thanks
<cjwatson> been a long time since I looked at subversion's innards
<StevenK> jelmer: Hopefully Python 2 support ...
<jelmer> StevenK: euhm, yes :)
<cjwatson> I made that exact same typo in my initial question, and caught it before hitting enter
<cjwatson> Err
<cjwatson> In fact I didn't catch it before hitting enter :)
<cjwatson> https://pypi.org/project/vermin/ is an interesting (if mildly terrifying) tool
<cjwatson> "Concurrently detect the minimum Python versions needed to run code"
<SpecialK|Canon> cripes
<cjwatson> It ... doesn't crash on Launchpad?
<cjwatson> I think it might be a nice way to find obvious stuff in advance of getting to the point of running the test suite
<SpecialK|Canon> Excellent :)
<cjwatson> http://lpbuildbot.canonical.com/builders/lp-devel-xenial/builds/970/steps/shell_9/logs/summary   wow, what even was that code
<SpecialK|Canon> I cannot read PackagePublishingPocket without hearing the opening line of http://holyjoe.org/poetry/anonE.htm
<cjwatson> It is a bit
<cjwatson> https://code.launchpad.net/~cjwatson/launchpad/+git/launchpad/+merge/378040 should fix that
<cjwatson> (and does in my tests)
<SpecialK|Canon> Oh wonderful!
<SpecialK|Canon> (approved)
<SpecialK|Canon> (heartily and enthusiastically)
<cjwatson> Heh
