#launchpad-dev 2009-04-10
<joey> so kfogel
<joey> I want to start using this channel!
<joey> :-D
<kfogel> joey: I've been thinking about that, but I'm not sure how it's practical before open-sourcing.
<kfogel> But we can certainly have the meta-conversation in this channel!
<kfogel> Basically, the internal company channel has a very wide topic spread -- lots of confidential stuff is said among the purely technical stuff.  And even most of the technical stuff is meaningless w/o access to the launchpad code.
 * joey nods.
<kfogel> Any ideas?
<kfogel> I think if we prod people to have their discussions over here right now, it'll last until the first time they need to talk about something non-public.
<kfogel> when we get external developer, now that's when this channel starts becoming important.  I think the main thing is, post-open-sourcing, to require that people at least be logged in here & addressable during core hours.
<kfogel> Then we'll set the channel topic to instruct outside contributors to ping a random name if they don't know anyone :-).
<kfogel> Hi, mat_t.
<joey> kfogel, well I think it would force us to be more open if we could at least move topics of conversation here for those things which are already public.
<joey> kfogel, it's not really easy to do in some cases since there is a lot of integration
<joey> kfogel, I was hoping we'd make a big "do" about the stuff as it lands and encourage conversation about it here
<kfogel> joey: when we have some kind of critical mass.  Trouble is, we're kind of opening all at once, except for things like LAZR etc -- but those things wouldn't really be talked about here either, because they're independent projects.  Only when their use in the Launchpad code is the topic would talking about them here be the thing to do.
<joey> kfogel, well that's an interesting statement
<kfogel> do expound...
<joey> kfogel, we have opened csvcs for example... and then lazr and other things to come. It would be nice to have an irc home for them so folks could come, feel welcomed, and participate.
<kfogel> joey: now, API talk, *that* cuold happen here!
<joey> and API talk I suppose
<joey> although API's are probably a help question which is #launchpad
<joey> kfogel, just so you know, I don't have any answers... I'm just thinking out loud :-)
<kfogel> joey: for example, with the lazr stuff, we made a deliberate decision not to put it in some kind of "launchpad.*" namespace.
<joey> kfogel, right.
<joey> kfogel, but without any sort of community wrapper around it, it's just there for show
<joey> kfogel, how do we get a community around it?
<joey> kfogel, my initial pondering is... can we not use #launchpad and #launchpad-dev?
<kfogel> we announce it in the right places, see who's using it, encourage its use where it would fit
<joey> I'd like for us to lead the community rather than someone from say Red Hat take it, fork it, and then have a big and active dev forum around it
<kfogel> I guess I'm saying, community starts with people being motivated to use the code.  *Then* if they need an IRC channel, they'll create one, or come looking here.  But the lazr people are not necessarily the same audience as future launchpad contributors.
<kfogel> That's a good point... is there a danger of that happening?
<joey> I agree with the lazr vs LP stuff
<kfogel> I mean, they're not going to fork unless it's worth forking, and if it's worth forking, we'll already see the community around it using it.
<joey> no more danger than anything else I suspect.
<kfogel> joey: you're at TLsprint in London right?
<joey> nope. I'm home this week
<joey> I've been super busy with my temporary job to do the TL stuff
<joey> which is why I'm only now thinking of this :-D
<joey> If I was on LP full time I'd have already talked your ear off on all this stuff
<joey> kfogel, In thinking about this more.... I think I'm sort of triggering off of attitudes
<joey> like....
<joey> I'd rather get people excited, use, and improve the software
<joey> vs just dump it over the fence for some show of good faith
<joey> the former of course requires some sort of manpower to do it
<joey> unless someone from the community (like wgrant or ScottK for example) decide it's something they want to help with
<kfogel> Well, I think we're doing more than just dumping the stuff over the fence.  AFAIK, the lazr stuff is registered in Pypi, and we've been making noises about it in the appropriate places.
<kfogel> And all these things are registered in launchpad, so there are binding surfaces for a community to attach to, if they're going to form.
<kfogel> e.g., mailing list, bug tracker, etc
<kfogel> (though lp mailnig lists can be hard to find, sigh)
<joey> Yeah, I concede you have a point there with the existing LP features
<kfogel> Community won't form because we want it to form; it'll form because *it* wants to form.  It may just be that the number of people who find these packages useful is not large enough (yet?).
<joey> So I would have agreed with you there if it wasn't for cscvs
<kfogel> joey: oh?
<kfogel> tell me more
<joey> so https://edge.launchpad.net/launchpad-cscvs
<joey> svn to bzr mirroring tool
<joey> it's been open for a few years now
<joey> afaik, nobody uses it
<joey> You'd think someone might use it....
<joey> but perhaps not
<joey> we're still the only ones hacking on it
<kfogel> oh, I know what it is... but aren't people using bzr-svn for that?
<kfogel> I mean, how many sites have the kind of mirroring need we have?
<kfogel> very few, I think.
<joey> for imports yes
<joey> we use cscvs for mirroring though
<kfogel> there are many people who want to use bzr client to talk to an svn repos, but that's totally different.
<joey> e.g. an svn branch on sourceforge
<joey> My only point though with cscvs is that I don't know how many people actually know it exists and that it is GPL
 * kfogel pokes around bazaar-vcs.org
<kfogel> http://bazaar-vcs.org/?action=fullsearch&context=180&fullsearch=Text&value=cscvs&go=go
<joey> actually I think mbt uses it
<kfogel> joey: hit that url and weep
<joey> :-)
<joey> 0
<joey> so Karl it seems the only thing that we may want/need to do them is to ensure any of those projects that we turn loose have mailing lists
<joey> turned on
<joey> so that we give the community a chance to form.
<kfogel> we at least want to do that, good point
<kfogel> (re cscvs: sigh, so let's look at this naming from an outsider's perspective:
<kfogel> a tool that mirrors svn to bzr, and it's called cscvs.  But actually, our version (which does bzr) is called "launchpad-cscvs", not even "bzr-cscvs", because we happened to use it for launchpad).
<joey> if you're looking for a new name for cscvs you could ask rockstar who is very intimate with the code although I don't know if he's on speaking terms with it :-)
<kfogel> yeah how about "svn-bzr-mirror" or something really wild like that?
<kfogel> I mean sheesh :-)
<kfogel> so, one sec
<joey> how about "cornpuffs"
<joey> :-)
<kfogel> https://wiki.canonical.com/OpenSource/Process mentions the mailing list thing now
<kfogel> (people seem to actually be following that doc, so this might -- _might_ -- have an effect.)
<joey> yeah, that looks good
<joey> do we have an equivalent on the dev wiki for the community? maybe we don't need one
<kfogel> the division of information between the internal and the dev wiki is a little awkward right now
<kfogel> unavoidably -- we have to either duplicate information, or have enigmatic references
<kfogel> joey: here's a question: why don't we create a mailnig list *automatically*?
<kfogel> when you create a project, you get a bug tracker, and code hosting, and you don't have to do anything special...
<kfogel> joey: so, the cscvs home page recommends a mailing list already -- bazaar's!
<kfogel> and presumably #bzr would be the IRC channel, therefore
<kfogel> though it doesn't say
<joey> kfogel, It sure makes sense
<joey> but I think we didn't initially because we weren't sure how much the servers could handle
<joey> maybe we're passed that.
<joey> barry would be the guy to ask I guess
<kfogel> I'll ask him
<joey> kfogel, I think one of the issues is that besides kiko and myself, there hasn't been a whole lot of thought to community building.
<joey> hence you kfogel :-)
<kfogel> yes :-)
<joey> I know a lot of people are interested in it...but we have other work to do
<kfogel> joey: the thing is, with the stuff we're releasing right now, it's not all one community -- it's spread across many communities, some of which already exist and have their own thing going on.  When we open source lp, then we really have to build a community ourselves.
 * joey nods in agreement.
 * joey goes on the offensive.....
<joey> ScottK, wgrant - I have a special assignment for you. :-)
<joey> ScottK, wgrant - Make noise on the launchpad-users list please about the need for an opensourcing party at OSCON :-)
<wgrant> joey: I think the reason that nobody has touched cscvs is that the code is old and not very nice to look at and breaks frequently and we have bzr-svn already.
<wgrant> I tried to have a look around at one point, but ran away fairly quickly.
#launchpad-dev 2009-04-11
<joey> wgrant, you're smarter than you look! ;-)
<wgrant> Heh.
<joey> wgrant, I think rockstar had the same impression.
<joey> wgrant, as in, he ran away
<wgrant> Now, LAZR looks interesting. And that code isn't awful.
<joey> yeah we've been hacking on LAZR for some time and it is continually being updated
<wgrant> Although I had to hack lazr.restful to make it work with Python 2.6, and I should probably file a bug.
<joey> wgrant, actually someone was working on that iirc
<joey> flacoste or barry
<wgrant> Ah.
<joey> you might file a bug though
<joey> not sure where they are with that
<joey> maybe we can use your patch
<wgrant> I saw somebody make the tests work with 2.4, but there are code changes needed for 2.6.
<wgrant> I might file a bug after breakfast.
<joey> that's like, 9 hours from now?
<joey> aren't you in the UK?
<wgrant> .au
<joey> oh, my apologies.
<wgrant> +11. It's just past 9am here.
<joey> I thought you were in the UK for some reason
<wgrant> By that I of course mean +10, because DST finished.
<joey> Where abouts? OZ?
<wgrant> Yes.
<wgrant> Melbourne.
<joey> Right on.
<ScottK> joey: When you opensource sure.  Sorry to be a party pooper, but you've not opensourced the part of Launchpad that most interests me.
 * wgrant agrees with ScottK
#launchpad-dev 2010-04-12
<IiLuminated> some project got open sourced http://code.google.com/p/macaw/ and http://drpetter.proboards.com/index.cgi?board=musagi&action=display&thread=82
 * mwhudson lunches
<lifeless> is there a standard environment variable for selecting EDGE/STAGING for API clients ?
<thumper> today seems to be full of fail
<thumper> mwhudson: do you know of a list like object that is fed by a generator but reusable?
 * thumper tries something
<mwhudson> thumper: there might be something in itertools
<thumper> mwhudson: look sane enough? http://pastebin.ubuntu.com/412900/
<thumper> mwhudson: if I were to put that into the launchpad tree, where should it go?
<lifeless> thumper: so, queued status doesn't seem to have any major side effects
<thumper> cool
<lifeless> thumper: are you happy with my using that on production ?
<thumper> if you break it, you fix it, how does that sound?
<lifeless> I'm not in the DBA group anymore
<lifeless> or I'd say yes,
<lifeless> thumper: https://bugs.edge.launchpad.net/launchpad-code/+bug/561160 would seem to be a blocker
<mup> Bug #561160: API: 'Code failed to merge' setting doesn't work <Launchpad Bazaar Integration:New> <https://launchpad.net/bugs/561160>
<thumper> lifeless: heh
<thumper> lifeless: convince jml that queues are a priority and I'll get it all working
<lifeless> thumper: slack time :)
<thumper> heh
<lifeless> thumper: do you have any thoughts about 561160 ?
<thumper> yes
<mwhudson> thumper: looks neat enough, probably lp.services.utils
<thumper> mwhudson: I've added an extra fix for when it hits completion
<thumper> mwhudson: shorter and simpler than the active state example
<lifeless> thumper: and your thoughts were ?
<thumper> lifeless: it is a trivial fix
<lifeless> cool
<lifeless> thumper: does that imply you'll do the trivial fix ?
<thumper> not right now, no
<thumper> I'm in the middle of three other things
<lifeless> Oh, I wasn't meaning instantly
<thumper> in which case, yes probably
<lifeless> is https://dev.launchpad.net/Getting still accurate about the supported versions?
<wgrant> It's Hardy, Jaunty, Karmic, and Lucid most of the time, though it sometimes breaks for a few hours, and two tests fail.
<lifeless> so the page is stale ?
<wgrant> In that Lucid should probably be added? Yes.
<lifeless> ugh wow
<lifeless> 650MB of pacakes rocketfuel-setup wants to isntall
<wgrant> lifeless: Disable Recommends installation.
<wgrant> (add '-o Apt::Install-Recommends=false' to the apt-get command-line)
<lifeless> does that work for aptitude command line too ?
<wgrant> Hopefully.
<jtv> hi folks
<wgrant> Morning jtv.
<lifeless> much better
<lifeless> perhaps that should be in rocketfuel-setup ?
<wgrant> Patches welcome :P
<spm> wgrant: I have to ask. how long have you been waiting to throw that line back at lifeless? ;-) months? years...?
<wgrant> spm: Heh heh.
<spm> must have worked. he's speechless. /me records the moment for posterity. something to tell the grandkids. "Yes, I was there when lifeless was rendered speechless."
<wgrant> But yes, seriously, I can't think of a reason not to have it in there.
<spm> sounds good to me as well; if anyone cares...
<lifeless> wgrant: rs=lifeless
<lifeless> spm: what were you saying about speechless? :P
<spm> lifeless: it depends on the person. so eg wgrant speechless == > 2-3 hours; you, 10-15 mins. me 2-3 seconds. it depends.
<jtv> lifeless: stop pretending... the "rs" means you weren't really reviewing.  You were speechless.
<wgrant> rs='really speechless'?
<lifeless> lol
<mwhudson> thumper: http://staging.launchpadlibrarian.net/42124669/mwhudson-linux-trunk.log <- log with my latest branch from the kernel import
<thumper> mwhudson: it still took over an hour to get the branch?
<mwhudson> "Getting exising bzr branch from central store." step took an hour :(
<mwhudson> thumper: looks like it
<thumper> 2010-04-12 03:04:41 INFO    29134781 bytes transferred | Fetching revisions:Inserting stream
<thumper> 2010-04-12 03:27:01 INFO    Fetching revisions:Finishing stream
<mwhudson> there's 20 minutes at the end of that step without output, dunno what's happening there
<thumper> what happens there?
<mwhudson> thumper: jinx!
 * thumper EODs
<thumper> good work though
<mwhudson> thumper: the import should complete overnight, i think
<mwhudson> then we can see how long the scanner takes to process it, that'll be good for a laugh
<wgrant> Why does the 'N bytes transferred' not increase? Is that only the volume transferred between messages?
<mwhudson> wgrant: something like that yes
<mwhudson> i actually forget the details
<lifeless> mwhudson: repack on the server
<lifeless> mwhudson: perhaps
<mwhudson> lifeless: it's fetching over sftp
<lifeless> mwhudson: ok, not that then.
<wgrant> Is the slowness also perhaps attributable to strawberry being slow?
<mwhudson> wgrant: some of it, yeah
<lifeless> spm: is there a standard for LP API credential files for services you maintain ?
<spm> lifeless: I'm not sure I appreciate the question? we deal with SSL certs - so I suspect you mean something else. Client side by implication? if Y, not that I'm aware of. we/losas have very little interaction with the API's.
<lifeless> spm: oauth cred files
<lifeless> spm: e.g. PQM, tarmac, other scripts that run and speak to LP API's
<lifeless> do you like to have the cached credentials files that they need in some specific place ?
<spm> short answer is they don't talk to the LP api's - again aiui; they're using ssh specific keys where we need to split user access. ???
<lifeless> spm: ok, weirdd ;)
<spm> legacy? :-)
<spm> keep in mind we were having to do this stuff before the api's existed.
<spm> actually i can only think of 1 method where we do access the api's - killing spam in bugs - and that's a pita to use.
<spm> PIA, more from a work flow perspective. request: "pls delete spam <url>" so we click on the url; verify it's spam. ssh to server X; sudo; setup api env; lookup docco on how to do this... ; run funky api command/login; disabled spam. Yukness.
<lifeless> spm: yes, I hate that too.
<spm> i personally find it easier to switch to DB and do it in sql... less faffing around.
<lifeless> spm: however, it is the blessed be official way to do stuff
<lifeless> spm: I figure documenting the path the creds file at should make it marginally easier
<spm> tho - from a pqm/etc perspective where we are scripting large amounts of "stuff" - there may be a case to do stuff in the API.... hrm....
<lifeless> spm: but perhaps you should also have a LP bug
<lifeless> 'create credentials for remote server with no browser hurts my bwain'
<spm> I am a sysadmni of very little brain, so the hurt is correspondingly smaller
<lifeless> spm: will it break you if I have a new PQM to deploy tomorrow that used LP API's ?
<lifeless> spm: we'd like to attach the error messages so that everyone can see the failure, not just the person that hit the big green button.
<spm> for bzr? probably - or rather *possibly*. we'd need to split the other users on the same box. I don't anticipate major drams.
<lifeless> spm: shouldn't need to split any users
<lifeless> spm: I can make the creds file a PQM config option
<spm> ahh, k.
<lifeless> spm: this is why I'm discussing it with you
<lifeless> spm: to know what you need
<spm> I'd want to ok with the 2nd group - we can't break them. Tho I suspect they'd be sympathetic.
<lifeless> they are ?
<spm> U1 :-)
<lifeless> I'll mail statik now
<spm> LS are configured, but not activly using
<spm> lifeless: cc losas@ pls too. if only so the other guys get a heads up.
<lifeless> c.c ?
<spm> aye
<lifeless> with or without s ?
<spm> um....
<spm> with :-)
<lifeless> I have both in my mail client :P
<spm> ha. wonder if without works....
<spm> Ahh that may go to just tom. as that's his title: losa of the losa's (apostrophe for clarity)
<lifeless> oh man
<spm> the 'l' is severely overloaded these days
<lifeless> don't say that aloud
<spm> heh, esp in a bad kiwi accent?
<lifeless> running make schema I get
<lifeless> ImportError: No module named tickcount
<lifeless> in a new setup following the current dev howto
<lifeless> on lucid
<lifeless> any advice ?
<noodles775> lifeless: yep, I think you're missing python-support from the launchpad ppa?
<adeuring> good morning
<al-maisan> Good morning noodles775 and adeuring
<adeuring> hi al-maisan, hi noodles775!
<noodles775> Hi al-maisan and adeuring
<mwhudson> holy ****: the kernel import completed!
<mwhudson> spm: is the puller running on tellurium, or is it broked by the config changes?
<spm> i'd have thought the fix universal on tellurium... checking...
<mwhudson> it which case it probably is actually running
<spm> ahh. the failed staging restores leave the maint file in place
<jelmer> mwhudson: \o/
<spm> mwhudson: should be working again RSN
<spm> mwhudson: which server? pear? or strawberry?
<mwhudson> spm: strawberry
<spm> holy ****!!!
<mwhudson> i guess the first pull will take ~1 hour
<spm> mwhudson: can we remove that bug report from against jelmer now this is working? :-P
 * mwhudson really has to go eat before his stomach implodes
<lifeless> noodles775: isn't rocketfuel-setup meant to do that ?
<lifeless> noodles775: I have the PPA python-support
<lifeless> where should I file a bug saying that the python-tickccount package wasn't brought in ?
<noodles775> lifeless: yes it should, and I think a bug was reported recently... hopefully you'll find it :)
 * noodles775 has a quick look
<maxb> Is launchpad-developer-dependencies actually still installed?
<maxb> Most (all?) of the problems people seem to be having are with the ubuntu upgrader removing it
<lifeless>   Installed: 0.71
<lifeless> maxb: this is a fresh developer configuration in a new vm
<maxb> launchpad-dependencies: Depends: .... python2.5-tickcount ...
<lifeless> thats broken
<lifeless> should be python-tickcount
<maxb> no, really, it's correct
<lifeless> python version deps went out back before hardy
<lifeless> $ apt-cache search -- -tickcount
<lifeless> python-tickcount - a python module to access the python interpreter tickcount.
<lifeless> there is a provides -
<lifeless> Provides: python2.6-tickcount
<maxb> Huh... sounds like you don't have the PPA version
<lifeless> I have the PPA version
<maxb> well that should provide python2.5-tickcount and python2.6-tickcount
<lifeless> no
<lifeless> lucid doesn't have python 2.5
<maxb> yes, that's why we have a PPA
<maxb> Version: 0.1-0ubuntu10launchpad1
<maxb> Provides: python2.5-tickcount, python2.6-tickcount
<lifeless> apt-cache policy launchpad-dependencies
<lifeless>   Installed: 0.71
<lifeless>  *** 0.71 0
<lifeless>         500 http://ppa.launchpad.net/launchpad/ppa/ubuntu/ lucid/main Packages
<lifeless>         100 /var/lib/dpkg/status
 * maxb is very confused
<maxb> unless your system actually has broken dependencies right now
<lifeless> not according to apt
<maxb> weird
<lifeless> noodles775: if its reported then I'm fine; I know how to install thigns ;)
<noodles775> lifeless: didn't find a bug, just this thread: https://lists.launchpad.net/launchpad-dev/msg03109.html
<lifeless> oh ugh
<lifeless> url generation is kindof annoying when working with a vm
<lifeless> noodles775: is there a FAQ about this - static ip address for the VM + etc/hosts on the host os ?
<wgrant> lifeless: https://dev.launchpad.net/Running/RemoteAccess?
<lifeless> mdns would be interesting
<mrevell> Morning
<wgrant> bigjools: Morning.
<bigjools> morning wgrant
<wgrant> bigjools: If you have time, can you please at some point do the necessary RT- or LOSA-wrangling to get the PPA log parser script running?
<bigjools> wgrant: I'll get to it when time allows, I have >1000 new emails to plough through :/
<wgrant> bigjools: Eeep. That is a few.
<bigjools> this is why having bots send email should not be allowed
<wgrant> Heh.
<bigjools> wgrant: I'm not fully up to speed with that buildd bug I pasted you a week ago, how was it fixed?
<wgrant> bigjools: lamont hadn't run 'debian/rules package' lately, so lib/canonical/buildd's copy of buildd-slave.py (which actually lives in daemons/) was out of date.
<bigjools> wgrant: ok, how can we avoid that problem in future?
<wgrant> bigjools: Split lp-buildd into a separate tree.
<bigjools> heh
<wgrant> The same thing can happen with tachandler.
<wgrant> But I believe production is all nice and up to date now.
<wgrant> Ooh. staging's linux import is scanning.
<wgrant> Oh, no, still mirroring.
<mwhudson> spm: say
<mwhudson> spm: is there a mirror-branch.py process on tellurium that's been running for ages?
<wgrant> Has it been at it for an hour and a half, or am I misreading?
<mwhudson> wgrant: potentially, yeah
<wgrant> (judging by last_mirror_attempt)
<mwhudson> oh that's set is it?
<wgrant> Yes.
<wgrant> 2010-04-12 07:44:04.505438+00:00
<mwhudson> then yes, it probably has been running for 1.5 hours
<mwhudson> or it crashed
<mwhudson> wgrant: is mirror_status_message set?
<wgrant> mwhudson: ah, yes, KeyboardInterrupt.
<wgrant> I forgot that was exposed.
<wgrant> That's a timeout, right?
<mwhudson> yeah
<mwhudson> weee
<mwhudson> hey bzr guys: make 2a fetch faster pls
<wgrant> 2a fetch really sucks at the moment.
<wgrant> But...
<wgrant> Didn't it only take an hour to copy over sftp from tellurium to strawberry?
<wgrant> So a local smart fetch should suck even less.
<mwhudson> the timeout is lower for the puller
<wgrant> Ah.
<mwhudson> though i'm surprised there's no activity at all
<deryck> Morning, all.
<wgrant> I am getting a test failure in lib/lp/registry/stories/webservice/xx-distribution.txt in both devel and db-devel, with clean DBs.
<james_w> what's the failure?
<wgrant> http://pastebin.ubuntu.com/413097/
<james_w> wgrant: I see it too
<wgrant> james_w: Uhoh.
<wgrant> Anyone have a Hardy machine to try it on?
<james_w> not to hand
<james_w> wgrant: the patch is getting 400, but not checking this
<wgrant> james_w: No obvious reason that this would be Lucid-specific?
<james_w> no
<james_w> I'd be looking at other things, such as whether buildbot is even running these tests
<wgrant> Again. Yay.
<wgrant> And EC2, of course... anyone have a recent EC2 log?
<wgrant> In the other couple of cases I've seen this happen, the test failure was obvious in the log.
<james_w> hmm, one thing
<james_w> what is happening is that the patch request is checking whether status == 'Official', which is being set in the same request
<wgrant> Aha.
<james_w> so if hardy/lucid differ in the order or something there then that will be the cause
<wgrant> Yep.
<james_w> wgrant: lazr.restful attempts to process the fields in a deterministic order, so that probably isn't the immediate cause
<james_w> oh no, I lied, it attempts to do it in one place, but then not in another
<james_w> got it, filing a bug
<james_w> bug 561521
<mup> Bug #561521: Success of PATCH request dependent on dict iteration order <lazr.restful:New> <https://launchpad.net/bugs/561521>
<wgrant> james_w: Aha! Thanks for tracking that down.
<james_w> np
<stub> Is the GPG tests under Lucid issue still blocked?
<jelmer> stub: there's a branch playing in ec2 at the moment
<jelmer> stub: ... after I landed a branch earlier that takes care of upgrading non-rich-root bzr branches in sourcecode/
<stub> w00t
<deryck> adeuring, let me know if we need to do a pre-imp about Bug 333521.
<mup> Bug #333521: Enable bugs expiration for Ubuntu <Launchpad Bugs:Triaged by deryck> <https://launchpad.net/bugs/333521>
<adeuring> deryck: yeah. that would be grat
<adiroiban> hi. in pages.py, what is the difference between test.globs['public_webservice'] and test.globs['user_webservice'] ?
<bigjools> anonymous vs logged-in at a guess
<adiroiban> bigjools: what is a âguessâ ? We also have user_webservice, where 'cprov' is the logged in user.
<adiroiban> can we have users authenticated using OAuth, but without an account on Launchpad ?
<bigjools> adiroiban: looking at the code, it's a read-only version of 'webservice' where the user logged in is "salgado"
<bigjools> user_webservice is nopriv, not cprov AFAICS
<adiroiban> bigjools: yes.
<adiroiban> but should we have public_webservice binded to nopriv
<allenap> leonardr: Can I ask a couple of questions about lazr.restful (and client)? First, can a collection return a heterogeneous mix of objects? Second, is it possible to choose what interface is used when exposing an individual object?
<bigjools> adiroiban: possibly, I don't know enough about all the tests that use it
<bigjools> at least it would be nice to rename
<leonardr> allenap: you can only return a mix of objects if they all implement the same base interface (ie. Person/Team)
<adiroiban> and user_webservice to salgado ?
<allenap> leonardr: If I had a compelling use-case, would it be possible to allow mixed types in collections?
<leonardr> allenap: it might be a lot of work. it could be as simple as just not specifying the collection type in the wadl, but i don't know what effects that would have
<leonardr> i don't know what you mean by exposing a specific object. you probably don't mean annotating that object's class's interface, but all the published interefaces are created when the classes are annotated
<allenap> leonardr: Perhaps I should explain my idea. Bug tasks can come in three forms: conjoined master, conjoined slave and not conjoined. A conjoined master and a non-conjoined bug task are functionally the same, but a slave is basically immutable, and can only be modified via its master.
<allenap> leonardr: It's confusing at the moment because some bug tasks are editable and some aren't. (There's more detail in bug 556515.)
<mup> Bug #556515: OOPS when editing conjoined bugtasks via API <oops> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/556515>
<allenap> leonardr: An idea was to return, from searchTasks(), a heterogeneous list of bug tasks. When the bug task is a slave, it would annotate the object such that lazr.restfulclient would only materialize a read-only object, without mutator methods too.
<leonardr> allenap: do you have separate launchpad interfaces for slave and non-slave bug tasks, or is the difference encapsulated in code?
<allenap> leonardr: It's currently encapsulated in code. Both slaves and masters and others are BugTask objects. I was thinking I could (not sure where) add a provided interface to slaves. Not sure if I could remove existing implemented-by-the-class interfaces via an object though.
<leonardr> alleenap: the capabilities of an entry resource are described by a wadl form
<leonardr> specifically we have a 'resource type' for every kind of resource we publish
<leonardr> if you want to publish a bug task-like resource that doesn't have any editable fields, either you will need to cause a new entry resource for that kind of thing to be created
<leonardr> or we will need to make it possible to describe a particular entry resource with a 'resource' stanza specific to that resource instead of a  'resource_type' stanza
 * allenap goes to look at the Launchpad WADL.
<leonardr> allenap: it's a machine-readable version of the apidoc
<leonardr> every section in the apidoc ('bugs', 'bugtask', etc) corresponds to a resource_type
<leonardr> so we're talking about adding a 'slave_bugtask' on a par with 'bugtask'
<leonardr> the alternative is to have something that's not mentioned in the apidoc and you only discover at runtime
<allenap> leonardr: I think having a slave_bugtask would be better. If I defined an ISlaveBugTask interface, annotated it appropriately, I guess lazr.restul* would DTRT with it when handed over the wire.
<allenap> leonardr: With objects providing it I mean.
<leonardr> allenap: it should, you can use IPerson/ITeam as an example
<allenap> leonardr: Of course, that's perfect. I have to go now to pick up my daughter. I'll be back later probably, but not for a few hours. Thanks for your help!
<leonardr> all right
<maxb> Hrm, I just AJAX-commented on a bug, and where my comment was supposed to appear in the page, I got "[object Object]" instead
<maxb> (on edge)
 * maxb stumbles across a cheeky use of http://launchpad.net/api/ in Ubuntu's python-software-properties
<maxb> Though given the alternative is oauth shenanigans, I find it hard to blame them
<maxb> Though this does raise the question of how long /api/beta is going to work for
<leonardr> maxb, can you give me some data?
<leonardr> we are planning to eol /beta/ at the same time as karmic, but that requires changing all existing karmic packages, and i didn't know about python-software-properties until just now
<maxb> http://bazaar.launchpad.net/~ubuntu-branches/ubuntu/lucid/software-properties/lucid/annotate/head%3A/softwareproperties/ppa.py#L68
<leonardr> maxb, do you know who is responsible for that package or did you just stumble across it?
<leonardr> there's nothing in there that couldn't be replaced with launchpadlib.login_anonymously()
<maxb> I just stumbled across it whilst thinking "I wonder how add-apt-repository ppa:blah/blah figures out the key"
<leonardr> ok, i need to talk to the maintainer
<maxb> mvo appears to be the most frequent recent uploader
<maxb> Is anyone aware of AJAX bug commenting being semi-broken on edge right now?
<maxb> ah yes, bug 541993
<mup> Bug #541993: Adding comments to a bug shows [Object object] instead of comment <javascript> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/541993>
<jml> g'night all
<mars> bac, ping, question about 'ec2 land' for you.
<bac> hi mars
<mars> hi bac.  Reading your mail to the list on 10/09/2009 about ec2 land
<mars> it says "The merge proposal must have a commit message set.  If it doesn't you can use '--commit-text' to set it."
<mars> bac, where would I find the "--commit-text" switch?
<mars> bac, and is this documented on the wiki?
<bac> ec2 land --help
<bac> mars: and my statement is a bit misworded.  using that switch will supply the commit message for the landing but does not affect the merge proposal
<mars> ah, I did that, saw "-s  blankblankthisisnotwhatIwant".  My bad.
<mars> right
<mars> bac, thanks
<mars> bac, I looked on https://dev.launchpad.net/EC2Test and did not see the command.  Is it documented elsewhere?
<mars> bac, fwiw, I think that page needs rework anyway.  It is outdated, and does not have enough examples.
<james_w> please merge with/improve https://dev.launchpad.net/LandingChanges
<bac> mars: i don't know if it is documented on the wiki or not
<bac> mars: i can look later
<mars> james_w, great! thanks for the pointer
<mars> james_w, looking at that page, does 'lp-land' also require a commit message on the merge proposal?
<james_w> yes, I believe so
<mars> bac, james_w's page has some 'ec2 land' instructions on it.
<james_w> and doesn't appear to have a way to override ir
<james_w> "./utilities/ec2 help land" should probably be on those pages
<mars> well, I don't know if you need the exact text, but "common examples" would be nice.  Like the wget and curl man pages have.
<james_w> I meant a pointer to the help
<mars> ah, yes
<james_w> but yes, it was just me writing down what I learnt to save the next person having to learn it from scratch. If everyone else did the same it would be great.
<mars> james_w, I had to troll the list archives with thunderbird to pull the same info you have on this page.
<mars> guess which is more convenient :)
<james_w> :-)
<bac> mars, james_w: the page is a nice start.  i think the part where it talks about using 'ec2 land' to land someone else's branch is wrong in that you supply the URL to the MP not a reference to the branch.  perhaps both work?  (i always use the MP url)
<james_w> I think both work
<bac> cool
<james_w> provided there is only a single merge proposal for the branch. Changing it to suggest the merge proposal way first would be good
<mars> bac, do you think your "ec2 land" pre-conditions should be added?  Or does the command do a good enough job of telling you that something needs to be fixed?
<bac> mars: it'll complain
<mars> bac, such as, I try "ec2 land" on an unapproved branch, and it will abort, and helpfully suggest I use "--force" if I am desperate?
<bac> mars: yes, i believe it does.
<bac> using --force is not uncommon since many reviewers don't change the final state of the MP
<mars> cool.  Self-documenting is nice.  Saves wiki words :)
<bac> mars: for example:
<bac> .../bac/launchpad/lp-branches/bug-559200> utilities/ec2 land
<bac> ec2: ERROR: Commit text not specified. Use --commit-text, or specify a message on the merge proposal.
<mars> beautiful
<bac> mars: that's all jml's work
<mars> hmm
<mars> bac, do you know what level of OAuth access ec2 land requires?
<bac> read-only i would assume
<bac> though i'm in the bad habit of clicking on the last button out of habit
<mars> bac, "read anything" then.  Depends on what "Private data" is.
<mars> eewww
<mars> that is a problem with OAuth
<mars> it is up to the site implementer to make a sane, helpful UI
<mars> and personally, I don't think the Launchpad OAuth page, or the lander itself, bothered to tell me what level of access they require
<mars> duplicate that potential problem across every OAuth app out there :/
<maxb> Best I could figure out from the code, "private" actually meant the same as it did in the rest of the UI - i.e. private bugs, branches, etc.
<mars> I wonder if there is some way for the client application to pass a string to the user saying, "If you give me access level X, I can activate capability Y"
<mars> So I hit the OAuth landing page,
<mars> and it says "Public Read Only: allows the lander to submit public branches to EC2"
<mars> "Private Read Only: allows the lander to submit private branches to EC2"
<mars> "Read and Write Anything: not required"
<mars> maybe you could pass a "do not use" hint instead of a string, and the landing page can disable the last button as a courtesy to the user.
<mars> maxb, if that is the case, then I hope it is easy to change the OAuth level after the fact :)
<maxb> Only by redoing the whole auth dance
<maxb> mars: An application can tell launchpad to display only a subset of the buttons
<SlonUA> could u point me the place where i can change path for 'lp://dev/' ...  i need bzr+ssh://bazaar.launchpad.dev:5022/ instead just 'bzr+ssh://bazaar.launchpad.dev/'
<maxb> Just type that explicitly. lp: is just a shortcut
<SlonUA> maxb: i see.. but for example if i have mapped branch to series then lp://dev/<project> show me .. whole branch ....
<SlonUA> maxb: yes i don't care i can work with full path .. but it's intresting .. also u can have dynamic branches for project =)
<wgrant> leonardr: launchpadlib is not really usable for that sort of application.
<wgrant> It takes forever to import and start, and downloads a lot of unnecessary stuff.
<leonardr> wgrant: understood, but if you don't use launchpadlib you also need to take responsibility for keeping your client up to date. i can't rely on knowing which ubuntu packages don't use launchpadlib but secretly depend on the web service
<sinzui> thumper, ping
#launchpad-dev 2010-04-13
 * maxb grimaces at tests which only fail in a full test-run
<maxb> lp.services.scripts.tests.test_all_scripts.ScriptsTestCase is the testcase of doooooom
<maxb> I worry that it's going to melt my laptop
<spm> maxb: it won't melt; but it might get a bit soft and wobbly/deform....
<maxb> It's when the laptop starts to smell a little like a soldering iron that I worry :-)
<spm> maxb: so long as the smoke doesn't escape? everything is gravy!
<mwhudson> spm: can i get you to apply a patch to strawberry pls?
<spm> sure
<mwhudson> spm: http://pastebin.ubuntu.com/413405/
<mwhudson> spm: *should* apply to the tree on there, not completely sure though
<mwhudson> if it doesn't i'll just get you to replace the entire file i guess
<spm> kk
<mwhudson> spm: in news that may please a sysadmin, this patch is essentially telling bzr to stop being clever and be very stupid instead
<spm> :-) applied
<mwhudson> spm: can you kill the running import on strawberry ?
<spm> mwhudson: with great and a deep sense of satisfaction and pleasure; done.
<spm> mwhudson: fyi. afk for a sec. delivery arrival...
<mwhudson> ok
<spm> back
<mwhudson> stupid > smart
<mwhudson> in this case
<spm> :-)
<mwhudson> noop import before patch: 1hr 10 minutes
<mwhudson> noop import after patch: three minutes
<jelmer> mwhudson: the create_tree_if_local argument to sprout() doesn't work as expected?
<spm> \o/
<mwhudson> jelmer: it doesn't?
<mwhudson> jelmer: the problem seems to be mainly that 2a fetch is terribly slow
<jelmer> mwhudson: I'm not sure, just trying to work my head around what your patch does differently
<mwhudson> jelmer: my patch uses copy_tree_to_transport rather than sprout
<mwhudson> jelmer: i think that counts as fairly different
<wgrant> Oh, nice!
<jelmer> mwhudson: heh
<wgrant> Do I spy a bzr bug coming soon?
<mwhudson> (credit to abentley for the idea)
<jelmer> mwhudson: I missed that, surprised bzr doesn't do that itself..
<jelmer> mwhudson: (in those cases where it's possible_
<jelmer> )
<jelmer> mwhudson: should the branch scanner work ?
<wgrant> The puller times out :(
<mwhudson> next on the hit list: having the puller use a similar hack for import branches
<wgrant> Hm. So it takes 37 seconds to copy directly, and slightly under 70 minutes for bzr to do it?
<mwhudson> something like that yes :(
<mwhudson> aragh
<mwhudson> Transport operation not possible: Transport <bzrlib.transport.http._urllib.HttpTransport_urllib url=http://localhost:10899/0000004d/> has not implemented list_dir (but must claim to be listable to trigger this error).
<wgrant> It pulls over HTTP!?
<mwhudson> yes
<mwhudson> actually not on staging, so this patch will probably work there....
<wgrant> Hm, but that's probably actually faster than using the smartserver here.
<mwhudson> yes, smartserver is /mostly/ a squeeze on latency
<mwhudson> and bandwidth in some cases i guess, but we have lots of one and very little of the other so...
<mwhudson> the error message seems to be lying fwiw
 * mwhudson hacks, mightily
<mwhudson> spm: hi, could you apply this patch to the tree on tellurium please? http://pastebin.ubuntu.com/413423/
<mwhudson> warning: do no attempt to read said patch
<mwhudson> *not
<spm> :-)
<spm> aaaaaaa. I am blinded and cannonit see
<spm> mwhudson: applied
<spm> need codehost restarted? looks like not....
<mwhudson> spm: no
<wgrant> It's taking a while..
<mwhudson> spm: is there a mirror-branch.py process running on tellurium?
<spm> 000      3944 97.1 18.5 653364 382120 ?       R    02:48   3:25      \_ /usr/bin/python2.5 /srv/bazaar.staging.launchpad.net/staging/launchpad/scripts/mirror-branch.py /home/supermirror/importd-push-branches/0004ea4b lp-mirrored:///~mwhudson/linux/trunk 322123 ~mwhudson/linux/trunk IMPORTED 1
<mwhudson> spm: okay
<mwhudson> seems maybe my hacks aren't helping all that much
<mwhudson> spm: can you du -sh /home/supermirror/importd-push-branches/0004ea4b /srv/bazaar.staging.launchpad.net/mirrors/00/04/ea/4b ?
<spm> 620M    /home/supermirror/importd-push-branches/0004ea4b
<spm> 61M     /srv/bazaar.staging.launchpad.net/mirrors/00/04/ea/4b
<spm> thats an impressive mismatch?
<mwhudson> bah
<mwhudson> spm: can you apply this patch to tellurium too? /home/supermirror/importd-push-branches/0004ea4b
<mwhudson> no
<mwhudson> spm: http://pastebin.ubuntu.com/413428/ <- this patch
<spm> sure
<spm> patched
<mwhudson> spm: can you zap the mirror-branch process?
 * spm reaches under the desk for the larger hammer.....
<spm> yo stub
<stub> put the hammer down...
<spm> <whine> but *stub* I was gunna make you plural!
<mwhudson> spm: ok, can you sync the puller logs from tellurium to devpad please?
<spm> mwhudson: 1000      4772 99.5  8.0 436900 165584 ?       R    03:04   0:53      \_ /usr/bin/python2.5 /srv/bazaar.staging.launchpad.net/staging/launchpad/scripts/mirror-branch.py /home/supermirror/importd-push-branches/0004ea4b lp-mirrored:///~mwhudson/linux/trunk 322123 ~mwhudson/linux/trunk IMPORTED 1
<spm> mwhudson: synced
<mwhudson> oh goody oops reporting is broken
<mwhudson> i guess i'll give up for now and have lunch
<adeuring> good morning
<lifeless> thumper: mwhudson: if you're still around, I need a pointer to the twisted vfs stuff in the lp code base - the policy on permitted file names specifically.
<lifeless> thumper: mwhudson: nevermind, found it, drive by patch submitted
<wgrant> bigjools: How many years is it since buildd-sequencer has been used?
<bigjools> since Jan 2009
<wgrant> Oh.
<bigjools> cprov wrote it in the back of the van at UDS Mountain View :)
<wgrant> I thought buildd-slave-scanner was used before buildd-manager, and buildd-sequencer came before slave-scanner.
<bigjools> wait
<bigjools> it's *not* been used since 2009
<bigjools> I mean cprov wrote b-m in the van
<bigjools> slave-scanner is the same thing as buildd-sequencer
<bigjools> we like to use as many different confusing names as possible
<wgrant> Er, is it?
<wgrant> lib/canonical/buildd/sequencer.py seems reasonably unrelated to cronscripts/buildd-slave-scanner.py
<wgrant> Or is buildd/sequencer not actually buildd-sequencer?
* wgrant changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 1 of 10.04 | PQM is open | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in irc://irc.freenode.net/#launchpad-reviews | Use http://paste.ubuntu.com/ for pastes
<bigjools> wgrant: I've no idea what the former is, since it's in the buildd dir then it's part of the slave
<wgrant> Ah:
<wgrant> The task sequencer is a simple twisted daemon which looks after making
<wgrant> sure that the buildd tasks (E.g. slave scanner, queue builder etc) are
<wgrant> only run one at a time, and potentially more often than once per
<wgrant> minute.
<wgrant> Sounds useless, is in the wrong place, and breaks slightly with Python 2.6.
<bigjools> no kidding
<wgrant> And hasn't been touched significantly since 2005.
<wgrant> bigjools: So, will anybody miss it if I remove it and its doctest as a step towards Python 2.6 support?
<bigjools> wgrant: where's the test?
<wgrant> bigjools: lib/lp/soyuz/doc/buildd-sequencer.txt
<bigjools> good grief
<wgrant> Hm?
<bigjools> delete it with prejudice
 * wgrant destroys.
<wgrant> It's impossible to remove config schema elements at the moment, right?
<bigjools> huh?
<wgrant> Didn't rollouts explode last week because a config key had been removed, so the production configs (which still had a value for it) were invalid?
<wgrant> Ah, yes, bug 557271.
<mup> Bug #557271: Unable to remove config entries from the schema <Launchpad Foundations:New> <https://launchpad.net/bugs/557271>
<bigjools> hmmm
<deryck> Morning, all.
<bigjools> howdy deryck
<sidnei> jml, where's that patch you wanted me to land again?
<jml> https://bugs.edge.launchpad.net/zope.testing/+bug/560259
<mup> Bug #560259: Subunit output formatter doesn't handle layer setup errors <zope.testing:In Progress by sidnei> <https://launchpad.net/bugs/560259>
<jml> sidnei: it's the branch linked to that, and the attached patch
<sidnei> jml, both? or either?
<jml> either
<jml> same patch
<sidnei> oh, looks trivial.
<sidnei> jml, you need a new release too?
<jml> sidnei: yes please
<jml> sidnei: and yes, it's really trivial :)
<sidnei> jml, done
<jml> sidnei: woot! thanks!
<sidnei> np!
<henninge> sidnei_: can you come over to #lp-review, please? ;)
<Ursinha_> leonardr, ping
<leonardr> hi ursinha_
<Ursinha_> leonardr, hi, I hit an error with one of my lpapi scripts, I wonder if you know what could it be
<Ursinha_> leonardr, "SyntaxError: no element found: line 16363, column 186"
<Ursinha_> this is the error, when I try to login_with
<leonardr> that sounds like there's a problem with the wadl
<Ursinha_> leonardr, yeah, but I have no idea why, since it works with another user
<Ursinha_> leonardr, invalid credentials?
<leonardr> you would get a 400 error. is that the entire error?
<leonardr> there's no headers?
<Ursinha_> leonardr, let me show you the traceback
<leonardr> no, i guess there wouldn't be since it's not an http error
<Ursinha_> leonardr, this is the traceback: https://pastebin.canonical.com/30486/
<leonardr> ursinha_: i suggest catching the exception in _browser.py#get_wadl_application and printing out 'content'
<Ursinha_> leonardr, sure, a moment
<Ursinha_> leonardr, this is weird, it seems that the end of that 'content' is missing
<Ursinha_> leonardr, line 16363, column 186 is the last line of that content, and it's not complete
<deryck> kfogel, ping
<kfogel> deryck: on phone, bbiab
<deryck> ack
<leonardr> Ursinha_ that content comes from a file on disk. lib/canonical/launchpad/apidoc/wadl-*-*.xml
<leonardr> check that file and see if it's truncated
<Ursinha_> aha
<leonardr> if so, remove that directory and make again
<Ursinha_> leonardr, hmm, thing is I'm running my script on devpad
<Ursinha_> works with one user and doesn't work with another
<Ursinha_> leonardr, and I can't reproduce the error locally
<leonardr> Ursinha_: i don't use devpad so i don't know what that means. you don't have access to the apidoc/*.xml?
<Ursinha_> leonardr, exactly
<leonardr> Ursinha_: i suggest you escalate to someone who does have access
<wgrant> Aren't the API scripts running against prod/edge?
<wgrant> So it's probably a client-side cache.
<Ursinha_> leonardr, but I'm running the script with edge lp api
<Ursinha_> what wgrant said
<Ursinha_> wgrant, cache huh
<leonardr> ursinha_: on phone, sorry
<Ursinha_> leonardr, no problem, thanks so far
<wgrant> Ursinha_: Try obliterating/moving ~/.launchpadlib/api.edge.launchpad.net/cache
<Ursinha_> wgrant, yeah, that worked
<Ursinha_> wgrant, thanks
<deryck> gmb, bug 294223 is done, right?
<mup> Bug #294223: Bugs missing after import from SourceForge <Launchpad Bugs:In Progress by gmb> <https://launchpad.net/bugs/294223>
<gmb> deryck, Yep.
<gmb> Forgot about that
<deryck> gmb, can you update and assign to the current milestone please?
<gmb> deryck, Sure
<gmb> Done
<deryck> gmb, thanks
<deryck> adeuring, can you update bug 261254 with linked branch, status, and assign to the current milestone?  I believe this is done, yes?
<mup> Bug #261254: Launchpad couldn't connect to ALSA Bug Tracker. <bugwatch> <oops> <story-reliable-bug-syncing> <ubuntu-qa> <Launchpad Bugs:In Progress by adeuring> <https://launchpad.net/bugs/261254>
<adeuring> deryck: sure
<deryck> adeuring, thanks!
<mars> sinzui, ping, do you have some time today to chat about changing some more /@@/ icons into sprites?  I have a question or two about it.
<mars> sinzui, if you have time, whenever you have time
<sinzui> mars, I will after 3:00+. I do not know much about how icons become sprites. EdwinGrubbs wrote the tools that generate the sprite and css
<SlonUA> maxb: how is going ? =)
<maxb> hello :-)
<mars> sinzui, ok.  EdwinGrubbs, do you have time to chat about sprites some time today?
<SlonUA> maxb: do u know how to enable karma .. just to see may digging =)
<EdwinGrubbs> mars: sure
<mars> EdwinGrubbs, what time for you?
<maxb> I know nothing more than it involves a cronscript somewhere
<EdwinGrubbs> mars: I can do it now, but it might be a little noisy since I'm at a coffee shop.
<mars> heh
<mars> EdwinGrubbs, that should be fine
<mars> EdwinGrubbs, skype or mumble?
<SlonUA> maxb: i see 'karma has expired.' all time
<EdwinGrubbs> mars: skype
<mars> intellectronica, online?
<intellectronica> mars: yes
<intellectronica> how can i help you?
<mars> Tom, I'm looking at turning some of the /@@/edit icons in the bug status table into sprites
<mars> intellectronica, can you see an issue with me doing so?
<mars> this is in the name of improved page performance, btw :)
<intellectronica> mars: nothing i can think of off the top of my head. looks like a net win to me.
<mars> intellectronica, cool.  I'll ask someone on the bugs team for a review when I'm done, just to make sure your team thinks my changes are sane.
<intellectronica> cool
<mars> thanks!
<jml> I'm forcing a rebuild of the recently failed 'lp' buildslave
<jml> "no space left on device" seems to be the originating error
 * jml discovers CachingIterator
<jml> it's just like lists in Haskell!
<jelmer> ooh, there is something like that?
 * jelmer has reimplemented it at least twice :-)
<jelmer> jml: where does it live?
<jml> jelmer, lp.services.utils
<jml> jelmer, which is the best known home for such a thing
<jelmer> oh, in Launchpad itself. It'd be a great thing to have in itertools...
<jelmer> jml: Thanks anyway, I'm sure this will come in useful sometime :-)
<jml> yeah
 * bigjools reboots from karmic into lucid
<jml> jelmer, I should also add synchronize and dichotomy to itertools, I guess
<jml> jelmer, I didn't write it, fwiw. I just found it.
<jml> given the docstring, I'll wager abentley wrote it
<jelmer> jml: you can tell that just from the docstring?
<jml> jelmer, "Some generators and iterators are expensive to calculate, like calculating the merge sorted revision graph for a bazaar branch"
<jelmer> ah, the reference to Bazaar
<jml> jelmer, nope, thumper.
<kfogel> deryck: pong
<kfogel> deryck: can't remember if your ping was internal or external channel :-)
<deryck> did I ping? :-)
<deryck> oh, this morning... right
<deryck> kfogel, are you working on stats for patch project?  Do I recall that correctly?
<kfogel> deryck: yup
<kfogel> deryck: https://code.edge.launchpad.net/~kfogel/tuolumne-lp-configs/patches-time-to-closing
<kfogel> deryck: queries there; not much code yet
<kfogel> deryck: abel is helping w/ queries btw
<deryck> kfogel, ok, that's all I needed to know. I've been meaning to get back to stats for that project and wondered if your work included lpstats stuff.
<deryck> kfogel, btw, you'll have to subscribe me to that branch to see it.
<kfogel> deryck: oh, it's private?  one sec
<kfogel> deryck: you want rev notifications too, or just branch attribute notifications?
<deryck> kfogel, just attrs.
<kfogel> deryck: done
<deryck> kfogel, thanks.  and thanks for getting the stats together.
<kfogel> deryck: you're welcome; but thank me when it's done -- it's hard to measure this thing!
<kfogel> not use, but benefit, I mean
<deryck> yeah, that's why I hadn't got round to working on it yet.
<sinzui> jpds, ping
<jpds> sinzui: Hello.
<sinzui> jpds, have you tested that you can control the official country mirror status using api?
<jpds> sinzui: Yes.
<sinzui> jpds, as you represent the primary user of the feature, you want to update this bug's tag to qa-oa: https://bugs.edge.launchpad.net/launchpad-registry/+bug/361650
<mup> Bug #361650: launchpad could know about official country mirror status <feature> <mirror> <qa-needstesting> <Launchpad Registry:Fix Committed by jpds> <https://launchpad.net/bugs/361650>
<jpds> sinzui: Sure, done.
<sinzui> thanks
<Ursinha> hi gary-lunch, I filed a bug now, bug 562486, are you aware of this issue? and, is that really foundations?
<mup> Bug #562486: accessing pending_gpg_keys using the api fails to some users <api> <oops> <Launchpad Foundations:New> <https://launchpad.net/bugs/562486>
<gary_poster> Ursinha: I was not aware of the issue.  It's definitely a foundations issue, and if it is a matter of how the code is exported rather than infrastructure, it is also a registry issue.  I'll ask leonardr for his opinion...once the oops tools are fixed :-)
<Ursinha> gary_poster, :)
<Ursinha> gary_poster, thanks
<gary_poster> thank you
<sinzui> gary_poster, Ursinha there are two issues that relate to this bug...
<sinzui> gary_poster, 1, the seem to be available in one database and not another, leading to many oopses....
<sinzui> but there is a second debacle from last release where gpgkeys were renamed to gpg_keys, not not all the code was updated
<sinzui> Ursinha, EdwinGrubbs is landing a fix for the latter.
<gary_poster> sinzui: "seem to be available in one database and not another": which different databases?  Do you mean replication slaves and masters?
<sinzui> gary_poster, yes, gpgkeys are manages via logintokens
<sinzui> gary_poster, we also saw this happen in the CoC web UI
<gary_poster> sinzui: does that maybe mean that you need to always connect to the master for these queries, using one of stub's context manager things?
<sinzui> gary_poster, I do not know
 * gary_poster doesn't really know context so is probably not helpful
<gary_poster> it sounds like it
<gary_poster> sounds like a race condition
<gary_poster> and if it is a race condition with the slaves, force using the master.
<gary_poster> if you think that sounds reasonable, and you are not sure about stub's context manager, I can hunt up an example for you, sinzui
<EdwinGrubbs> sinzui: regarding the bug with +claim, does this mean that the Account.activate() method is also problematic since it sets a password that might be different from what was entered on the LoginService?
<leonardr> james_w, can you give me the specific failure you get when you trigger bug 561521?
<mup> Bug #561521: Success of PATCH request dependent on dict iteration order <lazr.restful:New> <https://launchpad.net/bugs/561521>
<james_w> leonardr: it was in the mailing list post
<leonardr> james_w: ok, i'll find it and put it into the bug
<sinzui> EdwinGrubbs, I do not know. /me looks
<sinzui> EdwinGrubbs, isn't the account the SSO account, I do not think there is a Launchpad Account
<james_w> leonardr: note that it's a followon error that is in the thread. If you want the exception thrown by the mutator then modify the test mentioned in the bug to print the response from the patch request.
<sinzui> EdwinGrubbs, the use case we are trying to avoid is a non-login user working with profiles and account. So if the user can access Account.activate() without being logged into the SSO server, there is a problem
<leonardr> james_w: i can't find the relevant post in my mailbox. what was the subject?
<james_w> "Test failures on some platforms due to lazr.restful bug"
<EdwinGrubbs> sinzui: the reason I ask is that +claim gives you a login token that takes you to the ClaimProfileView in c/l/browser/logintoken.py and that calls emailaddress.account.activate(), so it would seem like the non-login user could access it.
<leonardr> ok, i got it
<sinzui> EdwinGrubbs, yes, that is taking the wrong path bad= person > email > account
<sinzui> EdwinGrubbs, good = account > email > person
<sinzui> EdwinGrubbs, so I think Account.activate is not a problem, but the callsite must know when/how to call it. claim does not
<EdwinGrubbs> sinzui: huh, does that just mean that we don't want to link a launchpad person that has the same email as the account but isn't already linked to that account directly?
<sinzui> EdwinGrubbs, we do want to link person to emails, we have to since emails are unique
<EdwinGrubbs> sinzui: I'm just trying to clarify why account>email>person is good.
<sinzui> EdwinGrubbs, I think this discussion is moot. there are two callsites, and you are removing one of them
<sinzui> EdwinGrubbs, So we can limit our discussion to Person.setPreferredEmail
<sinzui> EdwinGrubbs, Maybe we should change the guard in setPreferredEmail() and raise an exception instead of trying to fix the issue?
<EdwinGrubbs> sinzui: do you think it could be broken, or do you just want to get rid of the kludge?
<sinzui> EdwinGrubbs, We need to be careful in this method, I believe reset password switched the account from deactivated (by the user) back to activated. we may need to ensure the callsites (SSO!) has updated the account first.
<sinzui> EdwinGrubbs, ^ reset password switcheS THE ACCOUNT USING THIS METHOD
<sinzui> EdwinGrubbs, maybe I am over thinking this issue. setPreferredEmail() has no issues with active activated and deactivate accounts. suspend accounts cannot get here. so the problem is only when the account is not active.
<sinzui> EdwinGrubbs, I see scripts are calling setPreferredEmail. This could be very ugly.
<EdwinGrubbs> hmmm
<sinzui> EdwinGrubbs, I think we should remove the account hack in setPreferredEmail(), but I need to verify how we are resting password
<EdwinGrubbs> sinzui: well, I'll continue working on removing +claim for the time being.
<sinzui> yes, I suspect setPreferredEmail() is a separate bug
<sinzui> EdwinGrubbs, SSO reactivates the account itself. I think we can treat bug #248518 as a trivial deletion in a separate branch
<mup> Bug #248518: setPreferredEmail activated accounts <registry> <tech-debt> <Launchpad Registry:Triaged> <https://launchpad.net/bugs/248518>
<mwhudson> good morning
<sinzui> hi mwhudson
 * mwhudson tries to browse the code of the kernel import
<mwhudson> abentley, rockstar: https://code.staging.launchpad.net/~mwhudson/linux/trunk
<maxb> trunk != linux-2.6.31.y.git
<mwhudson> it's just the default in the form :)
<mwhudson> weee adding all the revisions in the scanner for that kernel import took 90 minutes
#launchpad-dev 2010-04-14
<jelmer> mwhudson: but it works \o/
<mwhudson> jelmer: yeah!
<wgrant> You know, 2a fetch is pretty slow.
<mwhudson> yes
<wgrant> Well, a bit worse than that, actually.
<mwhudson> heh
 * mwhudson files a bug
<wgrant> About?
<mwhudson> 2a fetch being slow
<mwhudson> wgrant: https://bugs.edge.launchpad.net/bzr/+bug/562666
<mup> Bug #562666: 2a fetch is very cpu intensive <Bazaar:New> <https://launchpad.net/bugs/562666>
<wgrant> mwhudson: I have a bzr.log for a checkout of that linux import, if you're interested... it took a little longer than I expected: 4 hours.
<jelmer> mwhudson: have you profiled it?
<mwhudson> jelmer: no
<mwhudson> wgrant: sigh
<mwhudson> wgrant: how big is the repo in the end?
<wgrant> mwhudson: ~400MB, it looks like.
<mwhudson> not bad
<mwhudson> git is 312megs it seems
<wgrant> .bzr in my shared repo is 383MB.
<mwhudson> wgrant: can you do repository/packs and repository/indices as well?
<wgrant> I am tmepted to try a local branch, but then I might be waiting all day.
<mwhudson> wgrant: it will probably take about an hour
<wgrant> wgrant@magrathea:~/dev/linux/bzr$ du -sh .bzr/repository/packs
<wgrant> 319M	.bzr/repository/packs
<wgrant> wgrant@magrathea:~/dev/linux/bzr$ du -sh .bzr/repository/indices/
<wgrant> 64M	.bzr/repository/indices/
<mwhudson> wgrant: thanks
<wgrant> mwhudson: Want bzr.log too?
<wgrant> It was 'get_parent_map'ing until 2086s :/
<jelmer> the --lsprof output would be interesting
<wgrant> THen fetched for 9000s.
<jelmer> wgrant: that is a lot of time spent to figure out it had to fetch everything...
<mwhudson> wgrant: did you fetch into the repo?
<mwhudson> if you fetch into a new branch with no repo you can skip that part i expect
<jelmer> mwhudson: it went straight to insert stream here
<wgrant> mwhudson: I did, yeah.
<wgrant> So I pay 2000 seconds because it retrieves the entire history map to check for anything I already have?
<mwhudson> something like that
<wgrant> Yay
<wgrant> Still, at least in-branch operations are pretty fast.
<jelmer> the smart server protocol should and could be a lot faster
<jelmer> s/faster/smarter/
<wgrant> As mwhudson showed yesterday, a completely dumb fetch took it down from 70 minutes to 37 seconds.
<poolie> mwhudson: hi
<poolie> bzr info bzr+ssh://bazaar.launchpad.net/~gz/bzr/trivial_testskipped -Dhpps seems to hang on the server side
<jelmer> wgrant: I guess it's partially a bandwidth usage/cpu usage/latency tradeoff
<poolie> mwhudson: i don't suppose we can get a traceback of the production server process?
<mwhudson> poolie: i expect so
<mwhudson> spm: can you copy ~codehost/.bzr.log somewhere poolie and i can see it?
<mwhudson> poolie: hm, it works for me
<mwhudson> i guess i'm seeing the hosted area of the branch though
<spm> mwhudson: sure, one sec
<spm> mwhudson: poolie: chinstrip:~spm/bzr.log.save.gz
<poolie> can we easily tar up a copy of the hosted disk directory for that branch?
<poolie> and attach it to https://bugs.edge.launchpad.net/launchpad-code/+bug/562672
<mup> Bug #562672: info on broken? branch hangs on server side <Launchpad Bazaar Integration:New> <https://launchpad.net/bugs/562672>
<mwhudson> poolie: spm can, i think
<spm> mwhudson: /srv/bazaar.launchpad.net/push-branches/00/04/f5/1b/ ?
<mwhudson> spm: yeah, and the mirror copy i guess
<poolie> yes
<spm> oki
<spm>  /srv/bazaar.launchpad.net/mirrors/00/04/f5/1b: No such file or directory
<poolie> i think he deleted it
<poolie> oh well
<spm> ew. that's a 43Mb tgz. do you really want that attached to the bug?
<spm> I ask as I'm not sure I'll be able to attach it without getting a timeout...
<poolie> sure, why not?
<poolie> erk
<poolie> well, maybe put it on people.c.c. and attach the url?
<spm> been down this path before
<spm> sure
<wgrant> mwhudson: My local branch has been going for 33 minutes, and has so far copied 99MB.
<mwhudson> wgrant: speed demon
 * mwhudson lunches
<spm> poolie: 15 min eta on that tarball landing on people.c.c.... 27% thru...
<poolie> spm, from inside the dc?
<poolie> or did you copy it down to canberra?
<spm> poolie: messing around within the dc tends to be sufficently confusing; it's easier to pull locally. if it was 100Mb? yeah, I'd setlle for the hassle and do it right.
<maxb> So here's a conundrum. Why would ~vcs-imports be granted launchpad.Admin on IProductSeries?
<mwhudson> maxb: to be able to set the linked branch?
<mwhudson> seems a bit overkill though
<maxb> oh. that makes sense... in a sort of sledgehammerish way
<mwhudson> sigh
<mwhudson> when your changes don't seem to be taking effect: check that you're running the tests in the right branch
<maxb> :-)
<mwhudson> wgrant: did your clone of the kernel import complete in the end?
<poolie> spm: how feasible is it for an appropriately authorized person to look at launchpad's http access logs
<poolie> for the apps, not for bzr or other things
<spm> poolie: easy I guess? in a technical sense. The hard part'd be the "authorised person"
<poolie> who could do that? or who could ask for it to be done
<spm> I have no idea.... I assume James is involved somewhere; but Francis'd be a good start.
<poolie> this is re bug 537432
<mup> Bug #537432: Save Playlist button does not point to (existing) playlist file <amd64> <apport-bug> <totem (Ubuntu):New for desktop-bugs> <https://launchpad.net/bugs/537432>
<wgrant> mwhudson: It did.
<mwhudson> wgrant: did you see how long it took?
<spm> poolie: ?? I'm not sure I follow the connection there?
<wgrant> mwhudson: I ran it with --lsprof, so it doesn't really count.
<wgrant> I am running it again now without it.
<poolie> spm, that's cause there isn't one :)
<poolie> bug 557432 is better
<mup> Bug #557432: Hot Bugs algorithm works poorly for some projects, making it a bad default listing <story-bug-heat> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/557432>
<poolie> just wanting to know what sort ordering people use most often
<wgrant> mwhudson: But it took about 96 minutes.
<wgrant> poolie: I'm guessing newest first.
<spm> poolie: ahh. that'd help then. :-)
<poolie> wgrant: i'm guessing most-recently-touched
<poolie> but yeah
<wgrant> Hm, true, or that.
<wgrant> But probably not heat.
<spm> poolie: ahh. i see. in that case, probably easiest if one of us runs the script directly; the logs are... um. not small. so moving the script to the logs would be a better idea.
<spm> poolie: vaguely related - we did similar with loggerhead/codebrowse a while back. The additional point is that we ran a sanitizer from CERT over the logs; as well as some home grown filtering to strip other bits that were sensitive. So for future ref I guess. we could do similar.
<poolie> i think in this case we'd just want the aggregate counts, so there should be nothing sensitive
<spm> I agree
<poolie> to write the script one would need to know the precise formats and ideally have a little sample
<mwhudson> wgrant: oh
<poolie> though it may just be whatever's in devel's apache config
<mwhudson> wgrant: i hope you kept the output -- can you attach it to the bug?
<wgrant> mwhudson: I have the lsprof output, yes.
 * wgrant attaches.
<spm> given you're dealing with URL's and stock apache - the URL's can be obtained by pretty much anyone who understands what that bug report is talking about. (I don't - exactly jic you were wondering - get the gist, not the fine detail :-) )
<poolie> spm, my question was, what field of the log is the url
<poolie> but i guess i can just assume 'it's the one that looks like a url' unless you're also logging referers or something :)
<spm> heh; is before the referrer. ip - - date tz "type url (http ver)?" ...
<spm> from memory....
<spm> generally is the 1st '"' field, sometimes the 2nd... /me goes to check.
<spm> yup that looks right. I know some of our logs use an earlier '"' field. one to be wary of.
<poolie> type being the http method, like GET?
<poolie> and the url is absolute?
<spm> 1.2.3.4 - - [13/Apr/2010:06:39:50 +0100] "GET /@@/footer-background.png HTTP/1.1" 200 404 ..... <== by way of example
<spm> or: 1.2.3.4 bugs.launchpad.net - [13/Apr/2010:06:39:50 +0100] "GET /@@/edit HTTP/1.1" 200 5100 .... <== for otehr domains not specifically logged on their own
<spm> url is always absolute, yes.
<poolie> ah but it's only the path part, not including the hostname
<poolie> ok
<spm> right
 * mwhudson spots awk lumbering over the horizon
<spm> i'd suggest 1 other part to look for - get a breakdown by Agent. it's not accurate, but *can* be instructive. if only to help identify noise to strip away. eg "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)"
<spm> mwhudson: heh; I'll just shove my logfiltering scripts at you guys and let you search'n'replace :-)
<spm> mwhudson: poolie: ancient robot filter: https://pastebin.canonical.com/30558/
<spm> as in ancient script; not ancient robots.
<wgrant> Can someone please land lp:~wgrant/launchpad/bug-344165-i-hate-buildd-sequencer? PQM rejected it while in testfix last night.
<spm> wgrant: can you come up with a more.... descriptive branch name? :-P
<poolie> spm: http://www.deviceguru.com/files/heath_he-robot.jpg
<poolie> >>             if ($6 !~ /bot([-./)(_s@ ]|$)|
<poolie> heh
<mwhudson> wow, that bug report is written in pure cprov :)
<spm> poolie: I wrote that and still can't figure out what the regex means :-D
<mwhudson> wgrant: so the branch passed ec2?
<wgrant> mwhudson: Yes.
<poolie> it's a steamroller threatening a wombat?
<mwhudson> wgrant: sent to pqm
<spm> okis. heading for some lunch and to let corbs run around and burn off school holidays energy. bbl.
<wgrant> mwhudson: Thanks.
<wgrant> spm: You're still in school holidays up there?
<spm> just started end of last week
<wgrant> Oh. Ours ended then.
<spm> yes. well. we can't have all states on the eastern seaboard as a 1st step align on school holidays! that'd be too logical! <== come and see the cynicism on display! step right up!
<mwhudson> mm
<spm> anyways, me and my cynicism are bbl :-)
 * mwhudson tries to remember how to turn statement logging back on in postgres...
<wgrant> mwhudson: /etc/postgresql/8.3/postgresql.conf
<wgrant> log_statement='all'
<mwhudson> wgrant: ta
<mwhudson> hmmm
<mwhudson> INSERT INTO Branch is blocking here
<wgrant> Awesome.
<mwhudson> stub: do you have any tips for seeing why a particular sql statement is hanging?
<stub> Look at pg_stat_activity to see what else is runnin
<stub> g
<mwhudson> i think i found it
<mwhudson> stub: all the other connections were <IDLE>
<mwhudson> but i guess one of them had taken a lock
<stub> If they were <IDLE>, they didn't have a lock.
<stub> '<IDLE> in transaction' might
<mwhudson> ah yeah, one was <IDLE> in transaction
<lifeless> mwhudson: is there an LP API call where i can pass in a url and it gives me the branch-if-any ?
<lifeless> mwhudson: I might have an http url, or a bzr+ssh one
<lifeless> thumper: ^ wgrant: ^
<lifeless> it might be a mirrored branch too
<adeuring> good morning
<mwhudson> lifeless: there's something like that at the python api level i think, getByURL ?
<mwhudson> don't know if it's exported
<lifeless> mwhudson: could you peek for me?
<mwhudson> lifeless: https://edge.launchpad.net/+apidoc/devel.html <- it is exported
<lifeless> mwhudson: thanks!
<lifeless> whats the 'right' service root to use at the moment ?
<james_w> lifeless: LPNET_SERVICE_ROOT, version='1.0' for stability or EDGE_SERVICE_ROOT, version='devel' for tracking changes I would say
<lifeless> james_w: so you need to pass two parameters these days ?
<lifeless> james_w: the code examples on the wiki are stale then
<james_w> lifeless: you don't have to, version has a default
<lifeless> ok
<lifeless> what is it ?
<james_w> lifeless: depends on the version of launchpadlib
<james_w> 1.0 currently I believe
<lifeless> mthaddon: ping
<lifeless> mthaddon: spm: http://paste.ubuntu.com/414192/  - I'd like to know if thats clear enough docs on setting up LP API keys for pqm
<wgrant> Interesting.
<wgrant> With Python 2.5, TALES obeys SourcePackageName's __unicode__.
<wgrant> With Python 2.6, we instead get '<security proxied blah blah blah>'
<lifeless> is that with an exception or regular object?
<lifeless> there were some changes in that area
<wgrant> lifeless: Regular object.
<wgrant> bigjools: Isn't the explanation in bug #562451 incorrect?
<mup> Bug #562451: developer-membership-board should be able to edit ArchivePermissionSet <packagesets> <Soyuz:Triaged> <https://launchpad.net/bugs/562451>
<wgrant> AFAICT, the TB can edit it because they are in ~ubuntu-drivers, which owns the primary archive, so they can do just about anything (which is pretty broken).
<wgrant> The only methods restricted to launchpad.Edit on ArchivePermissionSet are packageset-related, which is also pretty broken, but not the main problem here.
<bigjools> wgrant: yeah I was going to comment on that - the code is still wrong, but for other reasons
<bigjools> I mean "wrong" for philosophical reasons
<wgrant> AFAICT, it should all work if newPackagesetUploader and deletePackagesetUpload are moved to the <allow /> block, and the DMB added to ~ubuntu-drivers... but I really don't think most of the members of ~ubuntu-drivers should have the privs they do now...
<bigjools> yes - see the team description
<wgrant> In fact, their current permissions seem to somewhere between rather stupid and horrifyling dangerous.
<wgrant> I am leaning towards the latter.
<wgrant> Yes... but that has been there for a number of years.
<wgrant> lifeless: No other ideas?
<maxb> Oh gah, I'm sure I remember investigating __unicode__ vs. __str__ issues last python migration push, but I've forgotten the details :-(
<wgrant> The easy fix is to alter the expression to take /name, like all but two places in the code already do.
<wgrant> I am tempting to take that path.
<wgrant> Er, tempted.
<deryck> Morning, all.
<lifeless> wgrant: ?
<lifeless> wgrant: oh right, -_unicode_-
<lifeless> I think using explicit accessors is clearer
<lifeless> I could look at CPython if you're stuck
<wgrant> Certainly. I'm just a bit suspicious that something deeper has broken that I might be about to hide.
 * wgrant will trust the tests and just add /name.
<lifeless> I wonder if launchpad-code will blowup if i attach 40 MB comments to it ?
<intellectronica> lifeless: don't know about launchpad-code, but in malone we block comments above reasonable size
<allenap> lifeless: 50000 bytes is the limit I think, or 50000 characters.
<allenap> ... in malone.
<lifeless> allenap: intellectronica: So, I guess we need an attachment facility for merge proposals.
<lifeless> I'll file a bug requesting that.
<intellectronica> lifeless: sounds like a useful thing to have. as a short-term solution what we usually do it attach files to the bug corresponding to the MP and link to the attachment from a comment.
<lifeless> intellectronica: if there is a bug ...
<wgrant> lifeless: There already is one.
<wgrant> lifeless: You just have to submit them by email.
<lifeless> wgrant: for attachments ?
<wgrant> Yes.
<allenap> lifeless, intellectronica: Actually, I think the limit is on Bug.description (sane_description CHECK constraint). As far as the data model is concerned, it might be okay for comments to be bigger.
<wgrant> If you attach a diff, it will even render it nicely inline.
<lifeless> wgrant: oh, I've just sent a mail asking for them again then.
<lifeless> wgrant: whats the # ?
<intellectronica> allenap: oh, didn't realise that. that's a mis-feature, i think.
<lifeless> wgrant: and what do you mean 'you just have to submit them by email' ?
<wgrant> lifeless: As in it's already implemented.
<wgrant> lifeless: There is no UI to add attachments, but if one is attached to an email then it will work.
<allenap> intellectronica: I think there /might/ be a comment limit, but only implemented in the app.
<lifeless> wgrant: I can't see them on  https://edge.launchpad.net/+apidoc/devel.html#branch_merge_proposal
<wgrant> lifeless: The exposed API is not even close to complete.
<lifeless> wgrant: care to earn brownie points with me ? :>
 * wgrant has a few too many things on his plate already.
<lifeless> wgrant: if you can bear the agony, I'm happy to learn stuff
<wgrant> lifeless: Why do you want it exposed on the API?
<lifeless> wgrant: so that PQM can attach failure subunit streams, which can be many MB in size
<wgrant> Ahh.
<wgrant> Hm, actually, only diff comments are rendered at all at the moment. Others are stored, but rendered only as HTML comments.
<wgrant> And... hmmmmmmmmmmm.
<maxb> Is anyone knowledgeable in bizarre vcs-imports behaviour around? After being reviewed, the initial import of https://code.edge.launchpad.net/~maxb/irssi/trunk was executed 5 times immediately consecutively before it eventually "stuck" and completed for-real
<wgrant> maxb: bzr-svn and bzr-git initial imports are now done incrementally.
<wgrant> 1000 revisions in each attempt.
<maxb> oh...
<maxb> the UI is somewhat mystic about this
<wgrant> Hence the grey tick the first few times, which should have an informative tooltip.
<wgrant> (insufficiently obvious, but at least present)
<maxb> true (on both counts)
<lifeless> mthaddon: did you see the pastebin mentioned above ?
<lifeless> wgrant: and your conclusion on attachments?
 * mthaddon looks
<mthaddon> lifeless: that seems fine
<lifeless> wgrant: if you're interested in what I'm doing, have a look at lp:pqm - the recent commits
<lifeless> mthaddon: cool
<lifeless> mthaddon: I intend to ask spm to try a new pqm for bzr tomorrow (and roll it back if it won't work with the current launchpadlib)
<lifeless> mthaddon: I wanted to check the docs would be clear enough
<wgrant> mwhudson/thumper: Either of you still around?
<james_w> lifeless: it looks to me as though exposing attachments on merge proposals would be at least one day of solid effort on it
<lifeless> james_w: thanks for estimating it, thats good to know
<james_w> it's possible it would be quicker, but it's all a little odd, and so isn't just a matter of writing tests and slapping exported() on it
<wgrant> james_w: What do you see as being odd about it?
<james_w> you call a method getAtttachments that iterates through chunks on messages and checks their mime type and then returns two lists
<wgrant> You'd just need a bug-esque addAttachment (and probably linkAttachment).
<wgrant> Ah, right, for reading them.
<wgrant> Bugs do the same, but return only one list.
<james_w> so to add an attachment you need to add a message with a single chunk
<wgrant> Right, which is what Bug.addAttachment does.
<lifeless> so an API approach needs only addAttachment; the web display will need that logic
<wgrant> Isn't it?
<wgrant> Oh, it's not.
<wgrant> Odd.
<wgrant> That actually uses a separate BugAttachment table. Strange strange strnge.
<james_w> oh, it does attach it to a message though, so not as different as I thought
<james_w> once again, yay for reinventing comments/attachments/etc.
<wgrant> There's already a factory method to create the necessary message (I used it a few minutes ago), so it's probably just a matter of shuffling it around.
<james_w> yeah
<wgrant> And maybe strangling the next person who invents a new comment system.
<james_w> oh, the description of the attachment is actually the comment that it is attached to?
<james_w> interesting
<wgrant> Hm?
<wgrant> In which system?
<james_w> bugs
<james_w> IBug.addAttachement takes comment=Text()
<wgrant> That's the text of the comment to be attached to, right.
<james_w> ah, I'm talking nonsense, ignore me
<james_w> I was confusing comment and description
<wgrant> s/description/title?
<wgrant> It's title in the DB, at least.
<james_w> description in the API
 * wgrant is confused now too.
<wgrant> Yay.
<james_w> but yes, code attachments are sufficiently different that there's not a great deal you could take to implement code attachment addition
<wgrant> I forgot the two systems were so different :/
<deryck> gmb, is bug 546774 qa-able really?  Or just passing a test run is sufficient for this?
<mup> Bug #546774: BugWatchActivity.result should be NOT NULL <qa-needstesting> <story-bug-watch-error-tracking> <story-reliable-bug-syncing> <Launchpad Bugs:Fix Committed by gmb> <https://launchpad.net/bugs/546774>
<gmb> deryck, Well, we can quickly QA the db patch. I'll ask a LOSA once one comes free.
<gmb> deryck, But actually, there's probably no point, come to think about it
<gmb> deryck, I think we can mark it qa'd
 * gmb does
<deryck> gmb, yeah, that's what I wondered, if it really had any value.  Thanks!
<deryck> heh, we've emptied the landed/qa queue and filled it up again.
<deryck> at least we're being productive
<maxb> danilos: Can you provide a log of your 'make run' 'pydoc2.4' issue? FYI, the problem is that somehow the value of PYTHON_VERSION isn't being passed down from the root Makefile to the pygettextpo Makefile
<danilos> maxb, sure, though after a successful make it doesn't complain anymore so I am doing it in a new branch
<danilos> maxb, btw, it dies on the 'make' rather than 'make run'
<danilos> maxb, and the output I have seen was totally uninteresting (though, might give you a tip: it happens in pygettextpo)
<danilos> maxb, it seems it might have been stale sourcecode as well
<danilos> maxb, yeah, it works just fine now
<maxb> The pygettextpo sourcecode branch contains PYTHON_VERSION = 2.4 in its Makefile. This isn't a problem, because when the root makefile recurses into sourcecode, it passes its own PYTHON_VERSION as an override to the sub-make
<danilos> maxb, well, that's where it was trying to execute pydoc2.4; it doesn't anymore, perhaps I need to change some timestamps so make wants to run it again
<maxb> wfm in a completely clean environment, so I think it was just staleness hitting you
<danilos> maxb, yeah, I've tried after doing 'make clean' in pygettextpo and it still works and even rebuilds docs correctly
<maxb> jelmer (author), rockstar (reviewer): Regarding the cscvs "Set the 'converted-from' revision property." change landed on 2009-12-07. It appears it never got used, because sourcedeps.conf was not updated. I now have a later change to cscvs I want activated. Can you advise on the suitability of that change to go live?
<jelmer> maxb: heh, whoops
<jelmer> maxb: I guess any external users of cscvs would've been using it though
<maxb> Do any of those exist? :-P
<jelmer> maxb: There's no reason for the converted-from change *not* to go live
<maxb> I'm a bit unclear on the policy for bumping revnos in sourcedeps.conf. Does it need a review, or do I just need to get someone to rubberstamp and land?
<deryck> noodles775, ping
<noodles775> deryck: hi
<jelmer> maxb: I *think* you just need a review given the original change has been reviewed by a launchpad reviewer already
<jelmer> maxb: it's probably different for projects in sourcecode that aren't maintained by launchpad-pqm
<maxb> ok, I'll put a branch up that bumps the revno to include yours and my changes, and leave it to OCR discretion whether to r or rs
 * wgrant points at bug #562943, and bugs Bugs people to unbreak notifications
<deryck> wgrant, that bug is on MPs, not bugs, or I'm reading it wrong?
<wgrant> deryck: The bug is Code, but it is probably Bugs' problem that there will have been no notifications about it.
<wgrant> Security bugs against Launchpad go into a black hole.
<wgrant> Nobody except the reporter and a list that is never read get notified.
<deryck> ah, I get what you mean now.  Yeah, that is likely on us.
<deryck> wgrant, is there a bug about that then, about the black hole?
<wgrant> deryck: It's probably not so much a bug as a config issue.
<wgrant> A config issue which exists because the Launchpad notification system is pretty awful.
<deryck> wgrant, right, agreed.  But still would be nice to have a bug to track the issue and see if better general solution couldn't be found.
<deryck> intellectronica, can you do a quick pre-imp about bug 556499 now?
<mup> Bug #556499: Set user expectations for Ubuntu bugs after user reports a bug <Launchpad Bugs:In Progress by deryck> <https://launchpad.net/bugs/556499>
<intellectronica> deryck: sure. just give me a sec to get on skype again
<deryck> intellectronica, great, same for me.  thanks!
<wgrant> abentley: Bug #562943 may interest you.
<intellectronica> deryck: ready when you are
<Ursinha> hi gary_poster, I know you're busy now, but for you to answer later :) I've seen about 200 daily oopses like this one: https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1564CEMAIL1, I wonder if it's the same problem as bug 54005 but with a slightly different symptom?
<mup> Bug #54005: bugs.launchpad.ubuntu.com email ending up in Launchpad mailbox <email> <infrastructure> <oops> <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/54005>
<gary_poster> Ursinha: I have no idea, unfortunately.  stub, are you around and available to look at what Ursinha wrote and share a guess?
 * stub has a look
<stub> I'd say the OOPS system is rooted
<gary_poster> stub, uh?
<stub> The OOPS is almost entirely blank. There is an email listed though, which if I examine looks like it was sent to noreply@ubuntu.com but delivered by the mail system to launchpad@mail.canonical.com
<stub> Have a look at that oops
<stub> Traceback: None None  Traceback (most recent  call last): None
<stub> I guess it is a handled error that we are just dumping to an OOPS for reporting?
<gary_poster> stub, I did see oops.  I didn't know what to think of it either.  So you think we have a security compromise on the OOPS machine?  matsubara, any thoughts?
<stub> I don't see a security compromise anywhere.
<gary_poster> sorry, then what did you mean by "rooted"
 * gary_poster looks up australian definitions :-)
 * gary_poster thinks he understands
<gary_poster> "broken"
<stub> yes :)
<gary_poster> :-) k
<matsubara> hehe
<matsubara> I think it's the same issue described in bug 54005
<stub> It looks like a wiki update email was sent to launchpad@mail.canonical.com, and our mail processor doesn't know how to handle it.
<mup> Bug #54005: bugs.launchpad.ubuntu.com email ending up in Launchpad mailbox <email> <infrastructure> <oops> <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/54005>
<stub> I have no idea why that email was routed to us, but as it isn't our wiki and as it isn't our email infrastructure it is just another piece of spam
<gary_poster> stub, matsubara it felt lke it was something like that
<gary_poster> ok
<wgrant> 4
<wgrant> Argh.
<gary_poster> so Ursinha, yes, I think it is similar.  I'll add matsubara's and stub's comments.
<gary_poster> matsubara: why would that OOPS be blank?  Is the source file blank, or does this indicate a problem?
<matsubara> gary_poster, that's another bug that Ursinha found. bug 230106
<mup> Bug #230106: emails interface oops reports need better error messages <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/230106>
<Ursinha> yeah, that :)
<Ursinha> wgrant, irssi? :)
<stub> That is an old, old bug. Its a bit of an assumption to think they are the same (I have no idea if that old one is still valid - I don't even remember opening it 4 years ago!)
<wgrant> Ursinha: Of course.
<stub> 54005 I mean
<Ursinha> stub, c'mon, I don't remember bugs I opened last month...
<gary_poster> stub, ack.
<gary_poster> Ursinha, matsubara: 230106: ok, cool (for some definition of cool :-) )
<Ursinha> in any case, it would be good to have bug 230106 fixed
<mup> Bug #230106: emails interface oops reports need better error messages <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/230106>
<Ursinha> hi deryck :)
<gary_poster> heh
<stub> The referenced email in the bug is gone too - garbage collected a few years ago probably :)
<gary_poster> heh
 * deryck sees name and "heh" and gets nervous
<Ursinha> lol
<gary_poster> :-)
<bigjools> stub: can you make a dump of prod for me to refresh mawson please - it should be quicker than 2 weeks to restore this time I hope? :)
<Ursinha> deryck, we've been having 200 oopses with "None: none" content, how difficult would it be to fix that bug?
<Ursinha> deryck, bug 230106, that is
<mup> Bug #230106: emails interface oops reports need better error messages <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/230106>
<deryck> Ursinha, I really have no idea what's involved, sorry.  I wasn't aware of that bug until now.  Does this mean we have places in the bugs app where we just raise a bare Exception?  And we need something specific?
<Ursinha> deryck, to be hones I don't know why is it assigned to bugs, but that bug would be to avoid oopses like https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1563CEMAIL1 to be empty as they are
<Ursinha> *honest
<stub> bigjools: Better if the losas do - I don't have the keys setup
<stub> (this could probably be scheduled)
<bigjools> stub: it's somewhat awkward on mawson but maybe
<deryck> Ursinha, ah, it's to do with errors we raise when failing on bugs email, I think.
<bigjools> anyway I'll ask them, cheers
<stub> permission denied (public key). Yer.
 * stub wonders what he broke
<deryck> Ursinha, so good and bad news -- bad, we won't fix this ASAP, but we can look at it as part of our better subscriptions/notifications story coming next month.
<Ursinha> deryck, that's much better than nothing :)
<Ursinha> in the meantime, gary_poster and stub, could you investigate that issue, please?
<Ursinha> gary_poster, there's another oops to which I just filed a bug, bug 563055
<mup> Bug #563055: Mail parsing failing with ValueError <oops> <Launchpad Foundations:New> <https://launchpad.net/bugs/563055>
<gary_poster> Ursinha: 54005: There are a lot bigger things floating around, I'm afraid.  Maybe given the number of OOPSes we'll be able to get to it in a few weeks.  I added the notes to the bug.
<gary_poster> looking at other one
<gary_poster> Ursinha: 563055: that's a spam message.  low priority unless the OOPSes are high for this.  Are they?
<Ursinha> gary_poster, we had 36 yesterday
<gary_poster> ...
<Ursinha> gary_poster, but none today
<Ursinha> so it's intermittent
<Ursinha> (?)
<gary_poster> just not sure how to treat then :-)
<gary_poster> thank you
<Ursinha> gary_poster, no problem, thanks for taking a look in all these issues
<gary_poster> np thank you
<Ursinha> also thanks deryck and stub
<deryck> np
<Ursinha> gary_poster, re. bug 563055 I spoke too soon, I see 25 oopses today; not much, just correcting my previous statement
<mup> Bug #563055: Mail parsing failing with ValueError <oops> <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/563055>
<gary_poster> ack Ursinha thank you
<barry> does anybody (maybe leonardr, flacoste, or gary_poster) know anything about bug 512832 in lazr.restfulclient?  i'm getting hit by this but it's fix released, i'm assuming in lazr.rc 0.9.13.  however lucid appears to have 0.9.11.  do we need to push an update to lucid?
<mup> Bug #512832: launchpadlib caches based on full URL <lazr.restfulclient:Fix Committed> <https://launchpad.net/bugs/512832>
<gary_poster> barry: I know this keeps biting us in one way or another but AFAIK the current status is still fix released.  It does look like this was fix committed on 2010-02-16, and 0.9.11 was released on 2010-02-11, so it seems pretty clear that at least something sooner is needed.
<gary_poster> leonardr would need to provide more confidence.
<gary_poster> s/would need/could/
<barry> gary_poster: i think you mean s/sooner/later/ ? :)
<gary_poster> barry: yes, I was actually trying to say "more recent" :-)
<barry> gary_poster: :)
<barry> i'm happy to help push out 0.9.13 if you guys don't have time for it.  i'd really like to have this fixed asap
<barry> gary_poster: but i'll wait to see if leonardr responds...
<gary_poster> barry: your help would be much appreciated, of course.
<barry> gary_poster: cool. i'll take a look at this later today
<gary_poster> thanks
<leonardr> gary, are you sure it's not bug 545197?
<mup> Bug #545197: Cache filenames are (still) too long for ecryptfs <lazr.restfulclient:Fix Released> <https://launchpad.net/bugs/545197>
<leonardr> s/gary/barry/
<gary_poster> leonardr: that's what I meant by "it keeps biting us"
<leonardr> yeah
<leonardr> 0.9.13 should get rid of it for good, and if not, it makes it easy to hack the maximum file length
<barry> leonardr: could be.  563065 what what a quick search found.  i will at least put 0.9.13 in my ppa and give it a try
<barry> i do have my home dir crypted
<Ursinha> sinzui, hi, I'm trying to reproduce https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1564B963 but no luck, do you have an idea?
<wgrant> leonardr: Any chance of fixing bug #401990 some time soon? It makes it pretty awful for non-Ubuntu users to use launchpadlib.
<mup> Bug #401990: Depends on lazr.restful <lazr.restfulclient:New> <https://launchpad.net/bugs/401990>
<leonardr> gary, is that something we can fix given the current state of buildout?
<gary_poster> leonardr: ...I'm not sure on how buildout impacts it.  If lazr.restfulclient really doesn't need lazr.restful, we ought to be able to just remove the dependency from setup.py and move on.  Does it only depend on it for tests, perhaps?
<leonardr> gary: yes
<leonardr> it's a test dependency
<leonardr> i brought this up a while ago and iirc the conclusion was 'buildout sucks, let's do it later'
<wgrant> I'm sure I've seen other ZTK packages specify test deps.
<leonardr> wgrant, if you can point me to an example that would help a lot
<gary_poster> leonardr: I don't think buildout has anything to do with it.  We could easily have buildout require lazr.restful for tests, and then remove the core dependency.  However, philosophically, that means we don't have any assurance that lazr.restfulclient really works without that dependency (and even it's dependencies!).  Therefore, I kind of hate that approach.
<gary_poster> Perhaps there us a middle ground.
<gary_poster> is
<wgrant> gary_poster: Well, depending on lazr.restful brings in dozens and dozens of completely unnecessary ZTK.
<gary_poster> An option I've done in the past is to define two test suites
<leonardr> i was going to suggest something like that
<gary_poster> One test suite verifies basic operation with only standard dependencies...
<gary_poster> ok
<gary_poster> yeah, I'd be fine with that
<leonardr> we do have some tests in lazr.restfulclient that run against fake wsgi 'web services'
<leonardr> they don't do anything, but they prove that basic lazr.restfulclient functionality works
<gary_poster> I'd be much happier with that. then.
<gary_poster> (sigh, s/it's dependencies/its dependencies/)
<leonardr> if you're ok with me doing so, i will look into this after the first stage of performance work is done (which could be today)
<gary_poster> leonardr: I'm fine with that.  I added notes and comments to bug report.
<leonardr> ok
<jml> I have decided, strategically, that my day will be better if I have some chocolate.
<gary_poster> :-)
<james_w> I think sharing would be a good strategy
 * gary_poster envisions willy wonka and the chocolate factory, sending chocolate over TV^D^Dthe internet
<Ursinha> jelmer, hi, is soyuz team aware of this oops? https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1564L1960
<jelmer> Ursinha: hi
<jelmer> Ursinha: I can't remember seeing a bugreport about it, so I don't think we are
<Ursinha> jelmer, I'm filing one :)
<jelmer> Ursinha: Thanks!
<Ursinha> jelmer, bug 563176, thanks! :)
<mup> Bug #563176: OOPS accessing a distroseries binary package in a +source page <oops> <Soyuz:New> <https://launchpad.net/bugs/563176>
<gary_poster> rockstar or abentley: I tried to register code mirror of an svn: scheme and got  "The URI scheme "svn" is not allowed. Only URIs with the following schemes may be used: bzr+ssh, ftp, http, https, sftp".  That's new--I've imported svn: schemes many times before. :-/  Why the change?
<gary_poster> zope's svn traditionally only supported the svn: and svn+ssh: schemes
<rockstar> gary_poster, you don't want a mirrored branch, you want an import branch.
<gary_poster> rockstar I don't see that option on, for instance, https://code.edge.launchpad.net/zope.event/+addbranch
<rockstar> gary_poster, imports don't follow the same flow (stupid, I know)
<gary_poster> :-)
<rockstar> gary_poster, https://code.edge.launchpad.net/+code-imports/+new
<gary_poster> rockstar: ah.  I used mirrored branch to create https://code.edge.launchpad.net/~gary/z3c.recipe.filetemplate/svn and it worked fine
<gary_poster> so something changed
<rockstar> gary_poster, did you do it just now?
<gary_poster> rockstar: five days ago it worked (created that branch then)
<gary_poster> and just now it failed (from +addbranch)
 * jml back to hacking 
<rockstar> gary_poster, *shrug* no idea.  I'm pretty sure it shouldn't have worked the first time.
<gary_poster> huh
<gary_poster> ok
<gary_poster> so rockstar maybe last question then ;-) how can I tell if this is using bzr-svn or cscvs (or whatever it is)?  If it is using the old juice, how do I (or other people) convert it to the new bzr-svn juice?
<gary_poster> https://code.edge.launchpad.net/~vcs-imports/zope.deferredimport/trunk
<rockstar> gary_poster, all new svn imports are using bzr-svn
<gary_poster> rockstar: right, this was an old one, 2008.  so it is cscvs.  is the upgrade path to delete and recreate?
<rockstar> gary_poster, and unless the cscvs import is failing, we aren't currently converting them, since it's basically just rebasing the branches (bad bad bad bad bad)
<rockstar> gary_poster, if you're the only one using the branch, delete and recreate is probably fine.  Is there something wrong with the branch?
<gary_poster> rockstar, no, but zope folks seem to be interested in trying out more of the code-hosting bits, and in my experience the cscvs imports are much more fragile than the bzr-svn imports.  For example, with my bzr-svn imports I can push svn branches to LP and do MPs and everything Just Works in a reallt nice way (except externals, of course :-/ ).  I want them to have a similar experience.
<rockstar> gary_poster, this is absolutely true.  I agree that we should put our best foot forward.
<rockstar> If there's no real fallout from it, let's do it.
<gary_poster> rockstar: ack, thank you.  I'll see if I can determine fallout. :-)
<maxb> Given that cscvs and bzr-svn will generate different revision-ids, the *only* upgrade path is move-aside and recreate.
<sinzui> james_w, I jinked myself. a gentoo user reported another bug about source packages: Bug 563195
<mup> Bug #563195: distrotask: lp fails to recognize many gentoo packages in "source package name" <distrotask> <gentoo> <Launchpad itself:New> <https://launchpad.net/bugs/563195>
<james_w> heh
<jml> I'll fix the conflict between db-devel and stable
<jml> is there any particular reason the 'combine_css' operation in the Makefile isn't specified as the CSS file it creates?
<jml> hmm, I see. the 'combine-css' script uses a lazr.js method that duplicates some of make's logic.
<leonardr> gary or james_w, can you give me some guidance on setting up a test dependency? i'm following http://reinout.vanrees.org/weblog/2009/12/17/managing-dependencies.html but just setting up an extras_require stanza isn't working
<leonardr> can you point me to an example?
<leonardr> (tests_require also doesn't work btw)
<deryck> intellectronica, still around?
<leonardr> ok, i think i figured it out...
<jml> leonardr, what was the issue?
<jml> leonardr, worth adding a note to doc/buildout.txt?
<leonardr> jml: i didn't know how buildout worked
<jml> leonardr, ahh ok :)
<leonardr> i'm getting my test suite working and then i'll need to ask you some more questions about how buildout works
<leonardr> jml: ok, my question is now formulated
<leonardr> well, let me look at buildout.txt first. is that in launchpad?
<intellectronica> deryck: looks like i missed your ping earlier. sorry! still relevant?
<jml> leonardr, yeah, it's under the root doc/ folder
<jml> leonardr, it's been able to answer maybe 90% of my buildout questions
<deryck> intellectronica, hey, yeah, so please look at line 71 of the green of this rev:  http://bazaar.launchpad.net/~deryck/launchpad/bug-count-filters-503222/revision/10234
 * maxb chuckles at [r=jfdi]
<deryck> intellectronica, does that "<tr><td colspan="2"><br /></td></tr>" to create space bother you in this context?
<intellectronica> deryck: no, i think it's a fine pragmatic solution. if you want to be really pedantic you can add a tal:comment explaining why it's there.
<deryck> intellectronica, ok, I'll do that.  I want a better way, but give all the tables there anyway, it seems the cleanest for the moment.
<intellectronica> cool
<jml> maxb: :D
<jml> mwhudson, ping
<mwhudson> jml: hi
<jml> mwhudson, I've got a couple of branches that need review. I think you'd be the perfect reviewer.
<mwhudson> jml: ok
<mwhudson> jml: feel free to request a review on launchpad
<jml> mwhudson, done
<jml> mwhudson, one of them is slightly over 1000 lines, but that's just moving stuff around
<mwhudson> jml: yeah well, only people on your side of the world care about that
<jml> mwhudson, :\
<jml> mwhudson, I still haven't figured out European code review.
<jelmer> there is *a* European code review standard ? :-P
<jml> jelmer, yeah, those guys in Brussels think of everything
<jml> g'night all
<lifeless> notifications fail :) - http://imagebin.org/93049
<mwhudson> well, not exactly new
<mwhudson> thanks for doing those approvals though
<lifeless> I know
<lifeless> I just laugh every time it happens - I pointed it out in dev
<lifeless> and stub was 'meh, it will hardly ever happen'
<lifeless> the reason I'm donig them is that tres did something annoying
<lifeless> registered 50
<lifeless> and then renamed them all
<lifeless> so the email links we were sent were stale
<wgrant> Did stub's changes test commit changes make the test suite about an hour faster?
<wgrant> I have stuff landing in less than four hours now!
<wgrant> Any LOSAs around? buildd-manager is ignoring some buildds.
<wgrant> Total: 25792 tests, 25 failures, 9 errors in 204 minutes 11.495 seconds.
<wgrant> Python 2.6 approaches...
<mwhudson> lifeless: ah yes, that's (arguably) because we don't allow you to set the owner on the new code import page
<mwhudson> (there's a bug about this, i think)
<lifeless> ok nothing pendnig review
<mwhudson> lifeless: thanks
<lifeless> the magic of tabs
<wgrant> Do we have a LOSA at the moment?
<mwhudson> wgrant: mbarnett and/or Chex should be around
<mbarnett> wgrant: i just returned from feasting
<wgrant> mbarnett: Ah, your idle time did not raise my hopes.
<wgrant> mbarnett: Can you check buildd-manager's log for any obvious strangeness? ia64, powerpc and sparc builds are not dispatching at the moment.
<wgrant> Which does not make Ubuntu release managers very happy.
<mbarnett> yup, taking a peek
<wgrant> Thanks.
<wgrant> lamont has been pinged, but appears to not be around.
<mbarnett> wgrant: yeah, he has EODd.  i am not seeing anything obvious in the sequencer's logs
<mbarnett> wgrant: though my unfamiliarity might be a liability here  :)
<wgrant> mbarnett: No recent log entries mentioning artigas or sejong?
<mbarnett> wgrant: 2010-04-14 19:45:17+0100 [-] Processing successful build 1691253-3345351 from builder artigas
<mbarnett> wgrant: nothing recent from sejong
<wgrant> mbarnett: Argh.
<mbarnett> that doesn't sound happy at all!
<wgrant> Indeed... some archs are rather behind, and we are dangerously close to release.
<mbarnett> wgrant: do you think this warrants restarting the sequencer to see if that picks up the ignored jobs?
<wgrant> Without lamont, I guess the next thing to try is, in Python, "import xmlrpclib; xmlrpclib.ServerProxy('http://artigas.buildd:8221/rpc').status()" on cesium, and see what the buildd thinks its doing.
<wgrant> mbarnett: Unlikely. It's not *that* shoddy :P But we could try that if all else fails.
<elmo> artgias isn't doing anything
<mbarnett> wgrant: BuilderStatus.IDLE
<elmo> but it is getting poked by the build manager
<elmo> god I wish that thing had better logging
<elmo> I mean srsly
<wgrant> elmo: We improved logging a *lot* for 10.03.
<wgrant> But it might still need to be flipped into DEBUG for it to tell us why it is ignoring a few of them :/
<wgrant> mbarnett: Sad. That's normal.
<maxb> wgrant: Hi. Do you find attaching test logs to the wiki page is useful? I find that the layer-by-layer list of issues is what usefully displays progess and suggests action
<wgrant> maxb: I was going to get to that eventually. Just got to deal with a couple of other bits and pieces first.
<maxb> sure. I was thinking of tacking the tarfile nonsense next.
<maxb> *tackling
<wgrant> Yeah, that's the big one.
<wgrant> mwhudson: Any ideas on the Mercurial failures on https://dev.launchpad.net/PythonMigrationStatus?
<maxb> Fortunately it's a fairly shallow problem
<wgrant> I suspect that once we fix TestMercurialImport.test_import, the rest will magically vanish too, but I haven't dealt with import slaves before so I don't know where to start
<maxb> mwhudson: note that they seems to randomly oscillate between mercurial and git in different test runs
<wgrant> maxb: What do you make of bugs-emailinterface.txt? It looks to me like the test is testing for broken behaviour!?
<mwhudson> maxb: um, no not really
<maxb> wgrant: Well, I guess the question is "why has the behaviour changed?". I've not looked into that one.
<mwhudson> maxb: so it's not reliably reproducible?
<maxb> mwhudson: *something* fails in that layer every time I've seen, but it varies whether it's the git tests, the hg tests, or both
<wgrant> When running just the hg tests locally I've not seen them not fail.
<maxb> I am really starting to hate the author of the tarfile module
<james_w> it's not one of the most pleasant modules to use
<james_w> I dropped it in favour of shelling out to tar eventually
<maxb> It seems to break its interface every python version
<mwhudson> that's what code imports do too
<wgrant> Also, the author refuses to make it safe to extract things with.
<lifeless> thumper: ping
<lifeless> wgrant: I know the bug you're referring too; I don't think its that simple a refusal. IIRC he said roughly 'this is the same as tar itself'
<lifeless> wgrant: just do filename checks in a loop; its not hard.
<wgrant> lifeless: But it's not the same as tar itself.
<wgrant> It has had the symlink and directory traversal vulnerabilities fixed for years.
<mwhudson> maxb: ugh, i can only really suggest debugging via printf or something then
<wgrant> I've added a sleep just before the failure, and it looks like hg.tdb is there, but not hg-v2.db.
<wgrant> Odd.
<mwhudson> oh
<mwhudson> i guess you have python-tdb installed?
<wgrant> Ahahah.
<wgrant> And it's only for 2.6
<wgrant> So it uses tdb in preference?
<mwhudson> yeah
<mwhudson> though
<mwhudson> i thought we mostly avoiding importing things from the system locations now?
<mwhudson> maybe i'm confused
<wgrant> We all know that doesn't work very effectively.
 * wgrant removes python-tdb and reruns the tests.
<jelmer> tdb allows concurrent access and is a bit faster than sqlite
<mwhudson> sadly bzr-hg doesn't really work :)
<wgrant> Oh look, the tests pass now.
<maxb> ugh ugh ugh. system-environment-dependent test --
<wgrant> jelmer: Can we tell bzr-hg to not use tdb somehow?
<maxb> We could patch our sourcecode/ bzr-hg
<jelmer> wgrant: not at the moment, other than patching the source
<mwhudson> i think that's probably sane
<mwhudson> i guess the bzr-git code is now safe from this because it works at the directory level
 * mwhudson vanishes, back online in a bit (subject to cafenet working today...)
<wgrant> jelmer: Will we want to use tdb soonish?
<jelmer> wgrant: we is hoping that jam finishes his pack collection refactoring soon so that we can use native bzr objects
<wgrant> jelmer: Ah, OK. So I guess it is semi-reasonable to just hack the sourcecode copy temporarily until it goes away.
<jelmer> 44444444444jjj~.
<jelmer> wgrant: yes
#launchpad-dev 2010-04-15
<maxb> woot, with the tarfile fails cleaned off, it's looking quite good.
<maxb> I have a one-liner patch that I'd kind of like landed on devel. Is it appropriate to bother with a branch just for this?:
<maxb> -    >>> http = HTTPCaller(host='launchpad.dev')
<maxb> +    >>> http = HTTPCaller()
<wgrant> maxb: How're you dealing with tarfile? Just fixing the tests in the 2.6 branch?
<maxb> yes
<maxb> I've pushed that
<wgrant> Great.
<lifeless> maxb: nothing lands without a branch; but it doesn't have to be yours :P
<maxb> I guess my question is: is that too trivial for the reviewer overhead?
<lifeless> thats what rs= is for :)
<maxb> right :-)
<maxb> Hrm. valid_absolute_url('whatever://example.com/blah') ---> False in 2.5, True in 2.6, because of stdlb chanes
<maxb> *changes
 * maxb wonders what to do about that
<wgrant> I was wondering about that one.
<wgrant> Does it now ignore the scheme?
<wgrant> We already monkeypatch in extra schemes (sftp and bzr+ssh come to mind).
<wgrant> This may also cause the DB constraint to be relaxed, which would be seriously seriously bad.
<wgrant> OK, yes, it's seriously seriously bad.
<wgrant> Oh, wait, that *is* the DB constraint. Oops.
<wgrant> Or maybe not. i am confused.
 * wgrant reads.
<maxb> wtfness
<wgrant> I don't know. This seems like a Python bug.
<wgrant> Or at least a really dangerous behaviour change.
<maxb> py2.6 urlparse disregard the uses_netloc list when parsing urls and treats anything beginning with // as netloc based
<wgrant> Yep.
<wgrant> WTF
<maxb> Which is arguably saner than maintaining a list of specific schemes, but changing behaviour is just stupid
<wgrant> Not just stupid -- very dangerous.
<wgrant> Should we change both valid_absolute_url implementations to explicitly check the scheme against the monkeypatched uses_netloc?
<maxb> It looks to me like the only thing that uses this is c.l.interfaces.validation.validate_url, which already checks against a passed in list of schemes
<wgrant> maxb: What about valid_absolute_url in trusted.sql?
<maxb> That will suffer the same bug, but is an independent implementation to the one failing the tests
<wgrant> maxb: Right. But I would not fix the tested one until we've fixed the other one too, since it appears to be untested.
<wgrant> Although i guess it could be tested, but PostgreSQL still uses 2.5.
<wgrant> But no, PL/Python is built for 2.6
<wgrant> So the DB constraint is untested.
<wgrant> Yay.
<wgrant> http://bugs.python.org/issue7904
<wgrant> It's only a recent change.
<wgrant> So they must have decided to change behaviour like that in a stable update.
<wgrant> Awesomeness.
<maxb> wgrant: Hi, do you have something in the pipeline towards landing for the foreign branch tdb issues?
<wgrant> maxb: I don't. I guess I should prepare the two branches.
<wgrant> Although I wonder if I'm going to cause bzr-hg test failures by ripping out the TDB thing.
<wgrant> Hm, only 8 tests left to fix.
<maxb> I'm looking at the initZopelessTwice one
<wgrant> I looked at that before.
<wgrant> My initial thought was that the method had been renamed.
<wgrant> So the monkeypatch didn't work.
<wgrant> But it looks fine.
<maxb> Close - 2.6 is sending that codepath into a C extension, hence avoiding the monkeypatch
<wgrant> I saw that evil down the bottom, but then decided to have breakfast first.
<maxb> I'm in good humour, having returned to the computer having spent the evening in the pub :-)
<wgrant> AFAICT there are only two remaining issues that are actually mysterious.
<wgrant> The leftover thread one, and the _pythonpath one.
 * wgrant takes bugs-emailinterface.txt
<wgrant> OK, so, bugs-emailinterface.txt tests for the bug that http://bugs.python.org/issue7082 fixed.
<wgrant> That test is probably insane.
 * wgrant might talk to Bugs about that one tonight.
<maxb> hmm, is emailauthentication.txt really just complaining about a wrapped line?
<wgrant> maxb: yes.
<wgrant> Just looking at that now.
<wgrant> The point of the test is to test for whitespace; it tells doctest to not ignore whitespace changes.
<wgrant> I'm not sure what the point of the test is.
<wgrant> It says that we must be careful, because Python unfolds things.
<wgrant> It appears that Python no longer does.
<wgrant> I cannot see an obvious entry in NEWS about this.
<wgrant> And the test does not make it clear what the implications of it are.
<wgrant> Hm, I guess as long as the authenticateEmail succeeds it should be fine.
<wgrant> - Issue #1670765: Prevent email.generator.Generator from re-wrapping
<wgrant>   headers in multipart/signed MIME parts, which fixes one of the sources of
<wgrant>   invalid modifications to such parts by Generator.
<mwhudson> argh i hate the puller
<mwhudson> i guess i should be happy that i'm deleting chunks of it
<wgrant> maxb: Do you think that the carefulness it refers to is the manual boundary handling in lp.services.mail.signedmessage.SignedMessage._getSignatureAndSignedContent?
<maxb> that seems plausible
<maxb> So perhaps this test failure is inviting us to reconsider whether we still need to do that?
<wgrant> I guess I'll delete the special case and rerun the tests in 2.5.
<wgrant> I think so.
<maxb> Certainly I don't think the failing test has any operational significance
<wgrant> Hm, I guess we can't actually remove it.
<wgrant> Since only Lucid has the Python fix.
<maxb> Oh, is this a 2.6.x fix?
<wgrant> So it probably deserves to have the test for broken folding removed, a bug filed, and _getSignatureAndSignedContent XXXd.
<wgrant> 2.6.5.
<maxb> ugh
<wgrant> 2.6.5rc1, to be precise.
<maxb> ok, concur with your plaan
<mwhudson> biab, again
 * maxb wonders what the point of filing a bug for every XXX is
<wgrant> maxb: Turns out the bug isn't really fixed.
<wgrant> It's just less broken than before.
<maxb> :-/
<wgrant> as_string() on the entire message preserves folding, but on the part loses it
<wgrant> I'll alter the test to look at the part in question, which should work for both.
 * maxb is working on tolerance to the apt-ftparchive changes
<wgrant> I'm not sure if we want to be tolerant.
<wgrant> It's going to change the archive. Probably in a good way, but I'd prefer to disable SHA1 and SHA256 for now.
<maxb> hmm
<wgrant> But I can't work out how to do that -- it doesn't seem to respect the config options when they are in the apt-ftparchive config.
<wgrant> They may have to be in the apt config instead, and I'm not sure if we can pass a custom one into a-f.
<wgrant> Mind you, I haven't looked at it for two or three months.
<maxb> Interestingly, there's stuff left in the code from when the same issue affected Packages files, edgy->feisty
<maxb> So I was thinking we'd handle it the smae
<wgrant> What happened there?
<wgrant> Just ellipsised out the extra hashes?
<maxb> regexed them out, yes
<maxb> err, string-maniped them out, rather
<maxb> hmm, well, it's actually 2:20am here, so perhaps I should sleep instead :-)
<wgrant> Heh, probably.
<wgrant> Night.
<maxb> I lean towards letting lucid's apt-ftparchive write the extra sums for the production archive
<wgrant> If it's been done before, that's fine.
<maxb> It looks to have been done that way before for Packages files, we'd now be doing the analogous change for Sources files.
<maxb> Hrm, have you noticed that soyuz-upload.txt is now dropping a debian 'changelog' file into the root of the LP tree when run?
<wgrant> I hadn't noticed. But so it does.
<mwhudson> abentley: around?
<abentley> mwhudson, a bit.
<mwhudson> abentley: just replying to that merge proposal if you have time to look at that
<abentley> mwhudson, looking.
<poolie> spm, could you please run http://pastebin.ubuntu.com/414679/ across some of the logs and paste the first few lines of output?
<spm> poolie: http://paste.ubuntu.com/414681/
<poolie> urk, not quite enough
<poolie> i guess it has the refererer url at the end?
<poolie> spm how about http://pastebin.ubuntu.com/414683/
<spm> poolie: yeah it does: 1.2.3.4 bugs.launchpad.net - [14/Apr/2010:07:00:04 +0100] "GET /ubuntu/lucid/+bugs?field.searchtext=&orderby=-importance&search=Search&field.importance%3Alist=CRITICAL&field.importance%3Alist=HIGH HTTP/1.0" 200 90458 "-" "Python-urllib/1.17"
<spm> poolie: http://paste.ubuntu.com/414684/
<poolie> ok, and what about the whole thing piped through '|sort|uniq -c' ?
<spm> poolie: http://paste.ubuntu.com/414686/ <== might be worth checking for skew from robots. maybe.
<poolie> wow
<poolie> could you chain it together with your anti-robot filter that you showed the other day?
<poolie> though actually i think it's unlikely there will be too many
<poolie> because there's only a form linking to this url not (i think) generally an A ilnk
<poolie> link*
<poolie> that histogram is pretty damn suggestive btw
<poolie> i guess that's just over less than a day's data?
<spm> poolie: about half of 20 hours worth, for bugs.lp.net; not edge.
<spm> 1 of 2 servers
<poolie> so something like 10-20% of a day i guess
<poolie> i wonder if power users tend to use edge?
<poolie> thanks for helping btw
<spm> poolie: np; just don't mention it too much eh? I have a rep as a BOFH to live up to. k?
<poolie> heh
<poolie> so would you mind running that on edge, across several days data, just to be sure it's not skewed?
<spm> adding edge - same server/period - doesn't change the number much; sure.
<poolie> i mean catting several days of logs into it, if that's possible
<poolie> so that we get a few thousand hits altogether
<spm> oh yeah - I was 3/4 thru typiing the above. figured I'd finish and then do several days
<poolie> oh thanks
<spm> poolie: http://paste.ubuntu.com/414692/ about half of nearly 2 weeks worth
<wgrant> mwhudson: Hi. Half of the unsolved python2.6 failures are Codehosting threading things.
<mwhudson> wgrant: oh goodie
<wgrant> lp.codehosting.puller.tests.test_acceptance.TestBranchPuller.test_hosted_branch_stacked_on_mirrored_branch, in particular.
<wgrant> If the .join callback is there, it hangs at that point.
<wgrant> If it is not there, it leaves a thread behind.
<mwhudson> well, the latter isn't really a surprise, that's why the join is there :-)
<wgrant> Right.
<wgrant> Threads make me sad -- any ideas?
<mwhudson> one small one
<mwhudson> wgrant: i don't suppose it will make any difference, but if you go to bzrlib/tests/http_server.py:563
<mwhudson> wgrant: there's a comment about not having a join in a particular place
<mwhudson> does it work if you add the join there?
<mwhudson> wgrant: another idea would be to run gc.collect() before calling join
<wgrant> mwhudson: Both suggestions hang quite effectively.
<mwhudson> wgrant: :(
<mwhudson> wgrant: is there an easy ish way i can reproduce this myself?
<mwhudson> i have some terrifying gdb hacks lying around i can try to use...
<mwhudson> (https://code.edge.launchpad.net/~mwhudson/+junk/pygdb if you want to try to adapt them yourself)
<wgrant> mwhudson: https://dev.launchpad.net/PythonMigrationStatus <- just grab the integration branch, follow 'Setup caveats', and revert the patch which comments out the join.
<wgrant> Four tests remain broken, two of which we have solutions for. Yay.
<mwhudson> not too bad
<stub> What do BranchRevision rows with a NULL sequence signify again?
<mwhudson> stub: off mainline revisions
<wgrant> stub: *Ouch* at that column type change time.
<stub> difficult estimate to make though - it will be much faster on the kick arse hardware, but there are more nodes to update
<wgrant> It's a bit sad that 99% of that table is probably duplicated dozens to thousands of times.
<poolie> wgrant: https://bugs.edge.launchpad.net/malone/+bug/557432/comments/10
<mup> Bug #557432: Hot Bugs algorithm works poorly for some projects, making it a bad default listing <story-bug-heat> <Launchpad Bugs:Triaged> <https://launchpad.net/bugs/557432>
<wgrant> poolie: Yeah, I saw the paste.
<wgrant> Rather interesting.
<wgrant> We were both right, it seems :/
<wgrant> awkward.
<wgrant> poolie: Note, however, that since the default view is heat, there is less need to order search results by it.
<poolie> tue
<poolie> true
<poolie> but 42, being about 0.2% of the total, is awfully low
<wgrant> But I think the factor of 50 probably outweighs it by a bit.
<wgrant> Right.
<poolie> anyhow, i hope we can do more measurements like this
<poolie> they're not watertight but they bring something beyond just gut feel
<wgrant> Indeed. They're much better than guessing.
 * mwhudson fills up his hard drive with 2.6 versions of all the eggs
<wgrant> mwhudson: Weren't they all in download-cache anyway?
<wgrant> Although you may have issues with a missing i386 version of some egg that I forget.
<wgrant> The 2.6 i386 meliae egg is missing, that's right.
<mwhudson> wgrant: the download-cache mostly contains tarballs
<mwhudson> not built eggs
<wgrant> Oh, so it does.
 * wgrant gives up on checkwatches and leaves the last failure to Bugs.
<mwhudson> wgrant: now i can run the test!
<mwhudson> i wonder if that took long enough :/
<mwhudson> wgrant: the test passes for me :/
<mwhudson> wgrant: does it only fail as part of running the whole file or something?
<wgrant> mwhudson: Did you revert the commenting-out that is in the branch?
<mwhudson> wgrant: yes
<wgrant> Argh.
<wgrant> bin/test -vvt TestBranchPuller
<wgrant> That reproduces it for at least three people.
<mwhudson> ./bin/test -vvc  lp.codehosting.puller.tests.test_acceptance
<mwhudson> is what i'm running
<mwhudson> if using -t makes a difference, i am going to become very angry
<mwhudson> Tear down canonical.testing.layers.ZopelessAppServerLayer is hanging for me a _lot_ lately
<wgrant> Your machine hangs in the wrong place :(
<mwhudson> and not just on this branch
<cody-somerville> Whats the fix for the GpgmeError error in the testRepositorySignatureWithSigningKey test?
<mwhudson> i guess it's the sort of thing that rebooting, or at least logging out and in again might fix
<mwhudson> i may have stray processes
<wgrant> cody-somerville: http://paste.ubuntu.com/414737/
<cody-somerville> merci
<wgrant> mwhudson: With http://paste.ubuntu.com/414739/ on top of lp:~launchpad-committers/launchpad/python2.6, both my -t and your -c invocations hang.
<mwhudson> it seems test_mirror_imported_branch might have hung
<mwhudson> or is just very very slow
<mwhudson> looks pretty hung by now
<wgrant> But a different test.
<wgrant> Lovely......
<mwhudson> woot, i have some tracebacks
<wgrant> gdb?
<mwhudson> yes
<mwhudson> after a fashion
<mwhudson> wgrant: this is the hung thread: http://pastebin.ubuntu.com/414743/
<wgrant> Doesn't look very stopped to me.
<mwhudson>         # Support people who used socket.settimeout() to escape
<mwhudson>         # handle_request before self.timeout was available.
 * mwhudson is suspicious
<wgrant> Where's that?
<mwhudson> wgrant: handle_request in SocketServer is at least quite different between 2.5 and 2.6
<wgrant> Ah.
<mwhudson> i guess that if you're select()ing on a server socket that's shutdown, something different happens than if you're accept()ing on a socket that's shutdown?
<mwhudson> because that seems to be the difference
<wgrant> Could be, yes.
<mwhudson> oh grar
<mwhudson> i guess the select() in SocketServer should have self in the error list as well
<wgrant> It works if I force the timeout to 10 in SocketServer.
<wgrant> Hmmm.
<wgrant> Maybe.
<wgrant> It also works if self is in all three.
 * wgrant tries with it in wlist but not xlist.
<wgrant> It doesn't work in just rlist and xlist.
<mwhudson> i think this is perhaps the same issue: http://bytes.com/topic/python/answers/851825-socketserver-shutdown-deadlock
<wgrant> Similar.
<mwhudson> hm
<wgrant> So, it works OK with self in rlist, wlist and xlist. But no subset thereof.
<mwhudson> i guess it should be fixed in bzr
<mwhudson> can probably fix it by hacking in launchpad though
<mwhudson> well, it should be fixed in python
<mwhudson> but it's a bit late for that
<wgrant> Indeed... If you can work out the appropriate Launchpad hack, we can probably have the test suite passing under 2.6 in a few hours. Yay.
<mwhudson> wgrant: http://pastebin.ubuntu.com/414750/ maybe
<wgrant> mwhudson: Is that actually the right solution, though?
<mwhudson> well it doesn't seem to work :(
<mwhudson> oh no
<mwhudson> confusing slow and hanging tests there
<wgrant> mwhudson: http://bugs.python.org/issue2302 seems to be a bug about that shutdown thing you linked to earlier.
<mwhudson> wgrant: that seems a bit different
<mwhudson> wgrant: the problem that i saw was that the server thread went into a select that will never return
<wgrant> Right.
<wgrant> So server_forever was indeed running at the time, so it's probably irrelevant.
<mwhudson> noi
<mwhudson> server_forever is not called
<wgrant> Bah.
<mwhudson> by bzrlib
<wgrant> Anyway, lecture over... I should go home.
<mwhudson> haha
<mwhudson> wgrant: do you want me to push that patch somewhere, or .. ?
<wgrant> mwhudson: If it makes sense to you and probably doesn't just work by accident, I guess push it to the 2.6 branch.
<wgrant> At least it's only used in tests; it can't go horribly wrong.
<wgrant> OK, now I just have to get approval for one change from Bugs, advice from them on another, and we need to work out the best fix for validate_absolute_url.
<wgrant> Then we are DONE.
<wgrant> Thanks for fixing that one/
 * wgrant wanders off.
<mwhudson> hm
<mwhudson> not sure the fix is right actually :(
<adeuring> good morning
<maxb> *blink*
<maxb> regexes are really discouraged entirely in launchpad code?!
<maxb> is this really true?
<lifeless> first I've heard of it
<maxb> noodles775 attempted to convince me of this in a code review
<noodles775> I can't remember who had mentioned it to me, intellectronica perhaps - as I mentioned on the MP, it's not in the styleguide, so up to you.
<lifeless> re is generally faster and more reliable than adhoc stuff
<lifeless> so I would say 'whatever is clearer'
<maxb> noodles775: Yeah, I'm not implying you're being unreasonable, I'm just experiencing shock at the concept :-)
<lifeless> I don't see anything in the reviewer minutes about it either
<noodles775> lifeless: I don't even know that it's ever come up... it was just something that I was told for a branch I had ages ago that used a semi-complicated re.
<noodles775> lifeless: that's why I said it was up to maxb :)
<noodles775> maxb: in which case, do you want me to send it off to land as is?
<mrevell> Morning!
<maxb> noodles775: yes please, if you're happy with that
<noodles775> maxb: Sure. Done.
<wgrant> Any Bugs people around? I need advice on what to do with the section of bugs-emailinterface.txt beginning with 'Some mail clients append a filename to the content type of attachments.
<wgrant> '
<wgrant> It breaks in 2.6, because the behaviour that it says should not occur (extracting a filename from Content-Type, rather than Content-Disposition) now occurs due to a Python bugfix.
<wgrant> It appears safe to fix the test to expect that the filename is extracted from Content-Type... but I'm wondering why the broken behaviour was tested in the first place.
<intellectronica> wgrant: let me have a look
<wgrant> (this is one of the four remaining 2.6 failures)
<wgrant> intellectronica: Thanks.
<intellectronica> actually, adeuring might know more about this. i remember he did some work in this area lately
<maxb> wgrant: sorry, I just added another two in MailmanLayer to the wiki page :-)
<wgrant> maxb: Nooooooo.
<maxb> well, I added 3 and fixed 1
<adeuring> hrmmm, well, there is something....  main problem is my bad memory... let me look at the affected code/tests
<wgrant> I was running that just before I headed home, but it disagreed with being suspended for two hours.
<intellectronica> adeuring: maybe i'm wrong, this is about headers in mail. i remembered that you worked on attachments in the webapp.
<intellectronica> BjornT: do you maybe remember why this was done? ^^^
<adeuring> intellectronica: yeah, you are probably right...
<intellectronica> wgrant: i'm looking myself at the code, but my guess is as good as yours, which is why i'm looking for someone who might know the history of that code.
<lifeless> thumper: back, perchance?
<adeuring> wgrant: what breaks in this test?
 * maxb has a shocking thought: Python 2.6 for 10.04 == not unachievable
<intellectronica> wgrant: so i'm not sure if it's really important to ignore the filename supplied with content-type, as long as it doesn't change the behaviour of signatures, right?
<wgrant> adeuring, intellectronica: http://pastebin.ubuntu.com/414824/ is my fix.
<wgrant> intellectronica: And the signature depends solely on the MIME representation; not our interpretation of it.
<adeuring> wgrant: yes, that sould be fine
<wgrant> (well, assuming a lack of Python bugs, which there is not)
<intellectronica> wgrant: yes, that looks like the correct fix to me
<wgrant> intellectronica, adeuring: Thanks.
 * wgrant commits to the integration branch.
<wgrant> maxb: Ah, I see you still love the tarfile maintainer.
<maxb> :-)
<maxb> I suppose I have the consolation that after figuring it out last time, this was a pretty shallow bug
<adeuring> wgrant: this failure actually means that LP will process mails from Microsoft Entourage somewhat better, once we switch to Python 2.6
<adeuring> (another point is that this program should be banned)
<wgrant> adeuring: That is not a feature.
<wgrant> Heh, yes.
<maxb> lol
<adeuring> wgrant: I agree completely ;)
<wgrant> maxb: Ah, so the mailman timeouts are normal? I presumed they were from being asleep for hours.
<maxb> well, I'm going by Barry's comment on the wiki page that there are spurious failures if you run those tests not individually
<maxb> I ran the ones that failed individually and they passed
<wgrant> adeuring, intellectronica: Also, there is one other Bugs failure that I haven't got to the bottom of yet. Do you have any ideas about test_bug_496988 on https://dev.launchpad.net/PythonMigrationStatus?
<intellectronica> hmmmm ... it's worth asking allenap and gmb about this one, they might have an idea
 * gmb looks
<wgrant> gmb, intellectronica: Thanks.
<intellectronica> maybe the result is that we have to doctor the headers to satisfy the new version of xmlrpclib
<gmb> intellectronica, wgrant: Yeah, I think that might well be the case.
<wgrant> intellectronica, gmb: Any specific hints, or shall I go digging?
<gmb> wgrant, If you give me five minutes I'll have a look at the test. I suspect that some of our testing infrastructure for the XML-RPC tests, which is mostly a looks-like-xmlrpc-but-isn't object isn't playing nice.
<BjornT> intellectronica: can i remember why what was done?
<gmb> Okay, something weird's going on...
<intellectronica> BjornT: why we had a test for the broken behaviour where a filename specified in the content-type header of a mail is ignored. but unless something comes to mind don't worry about it, i think wgrant has a reasonable fix, now that the behaviour of email parsing was fixed in python2.6
<gmb> Oh, the test's moved. that's why I couldn't find it.
<wgrant> gmb: Yeah, it took me a while too.
<gmb> See, this is why we should never, ever refactor things and just keep the whole of LP in launchpad.py
<gmb> OH DEAR GOD.
<wgrant> Oh dear god?
<gmb> wgrant, Badly-written test.
<wgrant> gmb: So it *wasn't* just me.
<gmb> But, might be indicative of issues we'll ahve in the future when we switch to 2.6 with doing XML-RPC queries to remote trackers; I don't know.
<gmb> wgrant, nope.
<gmb> wgrant, that test actually tries to contact the gnome bugzilla.
<wgrant> gmb: Oh yes, I noticed that when I tried to run it on the train.
<wgrant> It was not happy.
<BjornT> intellectronica: no, i can't remember why. it seems like an odd test.
<gmb> wgrant, I think I wrote it, and once I've worked out how to fix it I'll throw myself on my sword.
 * gmb makes a note: buy a sword
<wgrant> Heh.
<wgrant> Apr 15 14:48:12 2010 (31672) Received these actions: create
<wgrant> Apr 15 14:48:12 2010 (31672) List creation error for team: itest-one
<wgrant> Apr 15 14:48:12 2010 (31672) An error occurred; the list was not created: itest-one
<wgrant> Thankyou, mailman, for your descriptive errors.
<gmb> wgrant, Actually, it's not all that bad, because I *think* it might show that we'll have a problem with XML-RPC and bugtrackers when we move to 2.6.
<dhastha> wgrant, Is there any tool which using launchpad API? like bzr-eclipse, tarmac, apport, ubuntu-dev-tools, and ilaunchpad-shell
<gmb> wgrant, So, one way to fix it is to monkeypatch lp.bugs.externalbugtracker.get_external_bugtracker to return a Bugzilla instance that doesn't try to connect out. See test_updater.py:131 for an example of us doing this.
<gmb> wgrant, in this case we want a get_external_bugtracker that returns a subclass of lp.bugs.externalbugtracker.bugzilla.BugzillaAPI whose getExternalBugTrackerToUse() method just returns self rather than contacting the remote server. I *think* that should do the trick.
<wgrant> dhastha: What do you mean? Aren't those all tools that use it?
<dhastha> wgrant, sorry.  I have to know, is there any application using launchpad api?
<wgrant> dhastha: All those examples you gave above do, don't they?
<wgrant> maxb: Um, I'm not sure those two mailman tests even pass in 2.5.
<wgrant> I get the same errors there.
<lifeless> wgrant: bzr-eclipse, tarmac don't
<lifeless> wgrant: last I heard, anyhow.
<wgrant> (yay for tests that are not run by default?)
<wgrant> lifeless: Hm, I'm pretty sure tarmac does.
<lifeless> oh, perhaps I mean 'doesn't use my pet feature'
<wgrant> lifeless: Merge queues that weren't meant to be exposed? :P
<lifeless> wgrant: meh, they are meant to be according to thumper
<lifeless> just not finished
<deryck> Morning, all.
<bigjools> apt-get dist-upgrade on lucid this morning wants to remove launchpad-developer-dependencies :/
<bigjools> ah, python-apt got bumped in lucid
<lifeless> what would cause
<lifeless>     raise HTTPError(response, content)
<lifeless> lazr.restfulclient.errors.HTTPError: HTTP Error 414: Request-URI Too Large
<wgrant> That's a new one.
<lifeless> I'm trying to lookup 126 branches by url at once
<wgrant> Ah.
<lifeless> squid allows several mb of URI IIRC
 * lifeless bugifies
<lifeless> launchpadlib, you think?
<wgrant> That looks like a server-side error to me.
<maxb> bigjools: so it did. uploading....
<maxb> I have this down to executing 'lpnochange python-apt' now :-)
<bigjools> maxb: :)  thanks
<wgrant> maxb: Those two mailman failures are indeed legit bugs in 2.5 as well. I'm fixing them now.
<bigjools> I have bigger issues, like the upgrader deciding not to install latest versions of some packages ...
<maxb> bigjools: !
<wgrant> bigjools: That's normal if there's arch skew.
<wgrant> And when the build farm breaks, there will be arch skew.
<bigjools> that's probably what it was
<bigjools> I need to install the task for the -desktop package
<bigjools> it  does explain all the frickin breakage!
<mwhudson> wgrant: i didn't seem to fix the http server hang :/
<wgrant> mwhudson: :(
<mwhudson> i'll look tomorrow when i'm more alert
<lifeless> jml: around ?
<lifeless> mwhudson: or are you still here ?
<lifeless> getByUrls is returning a dict mapping strings -> dicts rather than strings->Entries
<wgrant> lifeless: lazr.restful doesn't support it.
<wgrant> Well, actually, the WADL implementation doesn't support dicts, so lazr.restfulclient isn't told to interpret them as entries.
<lifeless> this is painful
<wgrant> Yes.
<lifeless> what does it take to fix
<wgrant> leonardr: ^^?
<lifeless> hes us based
<leonardr> i'm here
<lifeless> isn't he?
<leonardr> yes
<wgrant> He is. Eastern USians are around now
<wgrant> Some of.
<leonardr> i don't think it would be difficult to add support for that but it's also pretty low on my priority list. it's one of the things i'm going to look at for streamlining the web service
<wgrant> Or my timezones are wrong. That is more likely.
<wgrant> adeuring: The test is probably executed once for each bugtarget implementation.
<lifeless> hmm, 10 minutes to do the branch lookups
<wgrant> lifeless: This is Launchpad...
<lifeless> leonardr: this /really/ hurts. Can you suggest workaruonds.
<lifeless> wgrant: its ~ 1 minute with batches
<lifeless> wgrant: which I can't use...
<leonardr> my usual workaround seems to be the thing that seems to be hurting you--a dict with strings
<lifeless> leonardr: right; I want a collection with objects
<leonardr> is getByUrls the thing that's published now?
<lifeless> yes
<lifeless> leonardr: if you could describe how to approach the problem, in a lplib bug, I might take a peek, or get someone I know to take a peek,.
<lifeless> leonardr: I think guidance from you is a key step to someone else doing it though
<lifeless> leonardr: oh, while you are here
<leonardr> so, general steps
<lifeless> leonardr: do lplib objects compare == if they have the same canonical url ?
<leonardr> yes, as per https://bugs.edge.launchpad.net/launchpadlib/+bug/264098
<mup> Bug #264098: launchpadlib should implement __eq__ and __ne__ <launchpadlib :Fix Released by leonardr> <https://launchpad.net/bugs/264098>
<lifeless> leonardr: and are they hashable ?
<leonardr> i don't know
<lifeless> ok
<lifeless> what version of lplib is that fix in ?
<leonardr> the lazr.restfulclient NEWS.txt says 0.9.10
<lifeless> uugh, ok.
<lifeless> I'll comapre self_link then
<lifeless> (datacentre has 0.2.3)
<leonardr> is that even a real version? i don't see it in NEWS
<adeuring> wgrant: right
<adeuring> wgrant: thanks for the xplataion!
<lifeless> leonardr: I may have the number slightly wrong
<adeuring> ...explanation...
<lifeless> leonardr: probably 0.2~bzr35-0ubuntu1 or something
<lifeless> leonardr: its a port to hardy, after all
<leonardr> i see
<leonardr> maybe 0.9.3
<lifeless> no
<lifeless> probably one of these:
<lifeless> python-launchpadlib | 0.2~bzr25-0ubuntu1 | intrepid/universe | source, all
<lifeless> python-launchpadlib | 0.2~bzr35-0ubuntu1 | jaunty/universe | source, all
<lifeless> I saw the version a few days back, it was definitely 0.2something
<leonardr> it's possible the version numbers were very different before launchpadlib was split into lazr.restfulclient and launchpadlib
<leonardr> but pretty much nothing you want will be in there--including sensible support for dicts, which isn't anywhere yet
 * wgrant now understands why MailmanLayer isn't in normal test runs...
<lifeless> leonardr: sure
<lifeless> leonardr: I've filed an RT to get 1.5.1 on pqm
<lifeless> leonardr: you were going to describe how to approach supporting a dict of objects as a return value
<leonardr> right
<leonardr> 1. create @returns_dict_of similar to @returns_collection_of
<leonardr> 2. come up with a wadl description of what @returns_dict_of means and put it in the wadl
<lifeless> whats the difference between a dict and a collection ?
<leonardr> a collection is a list
<lifeless> ok
<lifeless> if we're using non python names, should we say 'returns_map_of' ? :P
<leonardr> 3. have lazr.restful properly serialize a dict to json (this might work already)
<leonardr> 'collection' is a term from the atom publishing protocol. there's no corresponding term for 'map' so i don't think it matters much
<leonardr> 4. have lazr.restfulclient pick up on @returns_dict_of and handle it properly. old versions won't be able to handle the return value so we should probably do this in a new version of the web service
<lifeless> leonardr: I can't see an obvious matching bug on launchpadlib
<lifeless> leonardr: should I file one with this conversation?
<leonardr> lifeless: https://bugs.edge.launchpad.net/lazr.restful/+bug/481090
<mup> Bug #481090: Cannot define a method that returns a dict <lazr.restful:Triaged> <https://launchpad.net/bugs/481090>
<lifeless> ah, ok
<jml> for this case, you don't want list semantics, you really want dict semantics, I think
<jml> .
<leonardr> jml, let me ask you this now rather than as part of a conversation later--how happy would it make you if i added this feature vs other features we've discussed?
<jml> leonardr: probably there would be a net change of zero to my happiness
<jml> leonardr: the stuff you've got scheduled now is more important. this particular bug annoys me though.
<leonardr> jml: how about once i get on to streamlining the web service? work on this first, on something else first, or talk to you then?
 * jml thinks
<jml> leonardr: by streamlining, are you referring to the "publish lazr.restful ops more naturally" task?
<leonardr> jml, yes
<leonardr> this falls under that task, i think
<leonardr> more urgently, i need to talk to you about performance at this point
<jml> leonardr: ok.
<jml> leonardr: I agree it falls under that task, despite the fact that the end-user benefit is greater perceived performance
<leonardr> jml: how much longer will you be around?
<leonardr> i wonder if i should talk to you first or clean up the performance wiki page
<jml> leonardr: probably another eight hours
<leonardr> ok, let me fix up the wiki and we can talk about performance next steps
<jml> leonardr: cool.
<jml> dammit, where's my hasbean delivery?
<leonardr> jml, i've sorted our performance interventions so far:
<leonardr> https://dev.launchpad.net/Foundations/Webservice/Performance
<wgrant> bigjools: Did you end up filing an RT for the PPA access log parser?
<jml> leonardr: regarding "Request service root conditionally", what was launchpadlib doing before your fix?
<leonardr> jml: launchpadlib was making a non-conditional request, thus your 'download the wadl every time' problem
<jml> leonardr: on every request?
<leonardr> jml: no, on every startup
<jml> leonardr: ahh, ok.
<jml> leonardr: sorry, mega-interrupty day
<jml> leonardr: can we talk in about three hours time?
<leonardr> jml, sure
<jml> leonardr: thanks.
<leonardr> or at some point--i have interruptions like doctors appointments today
<jml> ok
<jml> lifeless, you do know that the last element of an MP url is the db id, right?
<lifeless> jml: I know it happens to be
<lifeless> jml: I don't know that its guaranteed to stay that way
<lifeless> the self_link docs on branch_merge_proposal in the api doc don't claim either point
<jml> lifeless, ok. I'm not saying the bug you filed is invalid, just pointing to a potential workaround
<lifeless> jml: I put that precise workaround in place before filing it ;)
<jml> lifeless, good good
<mars> gary_poster, is the current buildbot configuration, specifically the purpose of the various build slaves, documented somewhere?
<gary_poster> mars, not to my knowledge but maybe.  This is the closest we have: https://dev.launchpad.net/Trunk/Glue
<wgrant> maxb: What do you think we should do about validate_absolute_url and co.?
<wgrant> That's about the only thing left.
<mars> deryck or gmb, think we can close bug #227824?
<mup> Bug #227824: BugPageTwoZero related branches table inline editing <ui> <Launchpad Bugs:Confirmed> <https://launchpad.net/bugs/227824>
<gmb> mars, No. You can't edit branches inline.
<maxb> wgrant: AFAICS It is not a problem that the valid_absolute_url that is being tested now accepts any scheme. It is only the DB constraint that matters.
<mars> gmb, ok.  The "BugPageTwoZero" stuff confused me.  I thought we were a bit beyond that :)
<maxb> In respect of the DB constraint, I think it is dodgy for a constraint named validate_absolute_url to secretly imply a limited list of schemes.
<gmb> mars, For BugPageTwoZero, read "ThingsWeWantedToDoForTwoZeroButDidntGetRoundTo"
<mars> hehe
<maxb> I would be in favour of any DB columns which actually need to be scheme restricted growing something more explicit to say so
<deryck> and there's this nifty inline title editing on launchpad, too.
<Ursinha> Chex, gary_poster, rockstar, bigjools, danilos, sinzui, allenap, production meeting @ #launchpad-meeting in 2 minutes
<bigjools> arg
<danilos> ack
<Ursinha> bigjools, c'mon, be nice
<cody-somerville> hmm... has anyone else started to get ImportError: No module named apt_pkg on lucid?
<cody-somerville> I'm getting it when trying to run some soyuz tests
<jpds> cody-somerville: Installed python-apt from the PPA?
<james_w> cody-somerville: did you install the python-apt from lucid and remove the 2.5 version from the PPA?
<cody-somerville> 0.7.94.2ubuntu6 superseded the version from the PPA it seems, 0.7.94.2ubuntu5launchpad2
<bigjools> maxb said he was going to upload a new one earlier
<maxb> grrrr
<bigjools> heh :)
<bigjools> cody-somerville: what are you using to install updates?  Mine failed because of the broken package.
<maxb> Would someone wield their buildd-admin powers and rescore this, please: https://launchpad.net/~launchpad/+archive/ppa/+build/1693771
<cody-somerville> I use update-manager
<bigjools> maxb: coming up
<bigjools> geez lp is slow
<cody-somerville> maybe a RBS should be set for that PPA too?
<bigjools> maxb: powers wielded
<maxb> RBS?
 * bigjools broke devel and is fixing it
<bigjools> except gpgme is screwed and my broken tests need it...GAH
<mrevell> night all
<maxb> bigjools: hmm?
<bigjools> might be that I didn't update sourcecode....
<maxb> If this is the lucid pygpgme problem, just bzr pull in the pygpgme directory
<maxb> My branch to actually bump the revno in sourcedeps.conf is stuck in ec2 somewhere
<bigjools> ok will try that if rf-get fails
<bigjools> I <3 my SSD
<maxb> Hrm, I've had 3 branches go through ec2 in ~3h20 today, and the fourth hasn't shown up yet
<bigjools> revno 49?
<bigjools> maxb: I did a pull and make in the pygpgme dir but no luck, still getting gpg failures in the test
<bigjools> the error changed from before though :)
<maxb> what is the failure?
<bigjools> GpgmeError: (7, 32816, u'Invalid argument')
<maxb> hrm
<bigjools> if you have a working box do you mind running a few tests for me so I can land my testfix?
<maxb> I can try
<bigjools> maxb: great thanks, it's here: lp:~julian-edwards/launchpad/ppa-quota-bug-413563
<bigjools> bin/test -cvvt TestPPAUploadProcessorQuotaChecks
<bigjools> maxb: actually
<bigjools> never mind - I did a make clean and it works now
<bigjools> thanks
<sinzui> barry, ping
<jml> leonardr-afk, some stuff came up -- let's talk tomorrow
<jml> g'night all
<barry> sinzui: pong
<sinzui> barry, does the holds on staging queue need a kick from an admin to send things to moderation
<sinzui> https://staging.launchpad.net/~launchpad-users/+mailinglist-moderate
<sinzui> ^ I sent three messages, expecting 1 to go to the archive (it did), one to go to moderation, and one to be discarded
<sinzui> barry, I wonder if both my missing messages were discarded
<barry> sinzui: it shouldn't.  i bet they were.  you can check the mailman logs from staging to know for sure
<sinzui> barry, the vete?
<barry> sinzui: check both vette and xlmrpc
<sinzui> thanks
<barry> np
<maxb> jelmer: Do you remember if your 'update pygpgme to trunk' ever got tested? I'm seeing failures in my python 2.6 environment
<mwhudson> good morning
<thumper> morning
<james_w> morning
<leonardr> flacoste, gary and i have a question
<leonardr> how reliable are the zope events that tell us that a launchpad object changed?
<leonardr> ie. can we be certain that every time an object changes, an event is sent out about it?
<flacoste> leonardr: unfortunately, that's not magic
<flacoste> it will only be sent out if the call sites, sends the event
<flacoste> so not totally reliable
<flacoste> although a lot of things break when that doesn't happen
<flacoste> unfortunately, things have broken in the past
<gary_poster> flacoste: but we rely on them already, yes?
<flacoste> yes
<flacoste> for notifications
<flacoste> and at some points in the API
<gary_poster> flacoste: we are intending to rely on them for invalidations of cached webservice representations.  Any reservations?
<flacoste> several
<gary_poster> heh
<flacoste> the first one is that not all objects will send events on modification consistently
<flacoste> because we don't rely on them for all objects
<flacoste> so the one for which we rely, we spot breakage
<flacoste> but for many objects, they don't systematically send events on mutation
<flacoste> i'd say the majority even
<gary_poster> ah :-(
<gary_poster> that's a biggie
<leonardr> flacoste: we're considering cache invalidation options
<leonardr> is there any amount of cache staleness we're willing to tolerate in the web service?
<flacoste> leonardr: i say it can work, but we'll have a lot of fixup to do, that's my opinion
<flacoste> i need to run now, ttyl
<gary_poster> leonardr, flacoste let's contonue this on email
<leonardr> all right
<mwhudson> hooray testfix
<mwhudson> oh updating sourcecode failed
<cody-somerville> mwhudson, does it complain about recursive symlink?
<mwhudson> cody-somerville: hm?
<cody-somerville> mwhudson, updating the sourcecode
<mwhudson> cody-somerville: no
<mwhudson> cody-somerville: at least, not in buildbot
<cody-somerville> ah
<cody-somerville> Does testRepositorySignatureWithSigningKey fail for anybody else on db-devel?
<maxb> oh, maybe. What revno of pygpgme do you have, though
<cody-somerville> 48
<maxb> ok. I did see that fail, but I wasn't sure whether it was part of the rest of the breakage that is r49 of pygpgme
<cody-somerville> r49? I just did ./utilities/update-sourcecode and it says 48 is the latest.
<mwhudson> failure in the devel buildbot, someone broke lib/lp/soyuz/browser/tests/archive-views.txt
<mwhudson> losa ping
<mbarnett> heya mwhudson
<mwhudson> mbarnett: can you tell me the state of the launchpad tree on strawberry?
#launchpad-dev 2010-04-16
<mbarnett> mwhudson:
<mbarnett> lp://staging/~mwhudson/linux/trunk  	Started: 48 minutes ago  	Last heartbeat: 14 seconds ago
<mbarnett> 2010-04-15 22:54:11 INFO    49969129 bytes transferred | Fetching revisions:Inserting stream
<mbarnett> 2010-04-15 22:55:11 INFO    43484334 bytes transferred | Fetching revisions:Inserting stream
<mbarnett> 2010-04-15 22:56:13 INFO    35991244 bytes transferred | Fetching revisions:Inserting stream
<mbarnett> 2010-04-15 22:57:14 INFO    31731071 bytes transferred | Fetching revisions:Inserting stream
<mwhudson> mbarnett: i meant like "log in and run bzr st", sorry
<mbarnett> mwhudson: hah, ok
<mbarnett> mwhudson:
<mbarnett> importd@strawberry:/srv/importd.staging.launchpad.net/staging/launchpad$ bzr st
<mbarnett> unknown:
<mbarnett>   lib/canonical/launchpad/apidoc/wadl-staging-1.0.xml
<mbarnett>   lib/canonical/launchpad/apidoc/wadl-staging-beta.xml
<mbarnett>   lib/canonical/launchpad/apidoc/wadl-staging-devel.xml
<mwhudson> mbarnett: oh ok
<mbarnett> revno 9233, if that is at all interesting
<mwhudson> mbarnett: can you run bzr merge  http://bazaar.launchpad.net/~mwhudson/launchpad/bzr-git-improvements ?
<mwhudson> if it has conflicts, please revert it
<mbarnett> mwhudson: Warning: criss-cross merge encountered.
<mwhudson> mbarnett: hmm
<mbarnett> still going though
<mbarnett> ah, 3 conflicts
<mbarnett> i shall revert
<mwhudson> thanks
<mbarnett> welcoem
<mwhudson> i guess i'll just wait until all the changes make their way through to db-stable
<mwhudson> 10721 Launchpad Patch Queue Manager	2010-04-15 [merge]
<mwhudson>       [testfix][rs=me][ui=None] Fix some tests broken in the branch that
<mwhudson>       	changes the default PPA quota
<mwhudson> but not all of them!
<mwhudson> guys!
<wgrant> mwhudson: Ew. What's left?
<mwhudson> wgrant:  lib/lp/soyuz/browser/tests/archive-views.txt
<mwhudson> i have a branch in pqm fixing it
<wgrant> Ah.
 * mwhudson lunches early
<thumper> lifeless: can you please tag any merge queue bugs with merge-queues?
<lifeless> thumper: sure
<thumper> ta
<lifeless> mwhudson: you said my 'cannot set status to 'code-failed-to-merge' bug was trivial'
<lifeless> mwhudson: care to point me at it ?
<mwhudson> lifeless: i think that was thumper
<thumper> ?
<thumper> BranchMergeProposal.set_status
<lifeless> mwhudson: well, the plural 'you'
<lifeless> thumper: yeah
<thumper> or setStatus
<lifeless> bug 561160
<mup> Bug #561160: API: 'Code failed to merge' setting doesn't work <code-review> <merge-queues> <trivial> <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/561160>
<lifeless> I pinged mwhudson because thumper was going out, I thought ;P
<lifeless> I'd just like a pointer at what to hit on, given its marked trvial
<thumper> trivial for me
<thumper> perhaps not trivial for you :)
<lifeless> thumper: so, gimme a couple pointers, and we'll see
<thumper> lifeless: setStatus needs a conditional to set mergeFailed
<lifeless> where do the tests live ?
<thumper> lp/code/model/tests
<thumper> test_branchmergeproposal.py
<lifeless> thumper: Looks to me like setStatus has a bug, if you do setStatus(Rejected) when it was Queued, it won't dequeue
<thumper> lifeless: it may well do, we haven't extensively tested queues
<mwhudson> spm: hey, could you trigger an update of tellurium and the staging code import slaves pls?
<spm> mwhudson: sure, gimme a bit...
<lifeless> spm: so, we haz keyfile ? :)
<lifeless> spm: I'm keep to get to the point that we find new rt tickets to file
<lifeless> spm: at which point stopping won't block things
<mwhudson> spm: update in progress? if you don't have time feel free to tell me to go away :-)
<spm> not yet....
<lifeless> thumper: you said the tests were in lp/code/tests/test_branchmergeproposal
<lifeless> thumper: that seems very small?
<mwhudson> lifeless: maybe it's lp/code/model/tests/test_branchmergeproposal.py
<lifeless> mwhudson: aha!
<spm> mwhudson: tellurium == codehost only?
<spm> mwhudson: and to what revno were you expecting? is currtently 9250
<mwhudson> spm: oh, 9250 is probably enough
<mwhudson> spm: what is strawberry at?
<spm> cool - that's tellurium btw; haven't checked the slaves yet
<spm> snap...
<spm> 9233. OLD as the hills... or me.
<mwhudson> spm: can you bully 9250 or newer on there?
<spm> mwhudson: tho if I update that - is that going to break if the DB isn't in sync?
<mwhudson> spm: it doesn't talk to the database any more
<spm> okis
<thumper> spm: do you have notes on why edge is so far out of date?
<spm> thumper: there's a bug somewhere.....
<thumper> spm: heh
<spm> ha. No. I mean a real one. not generally/amusing. hahaha :-D
<wgrant> There's the CSS bug, the lp.testing bug... any others?
<spm> https://bugs.edge.launchpad.net/launchpad-foundations/+bug/559128 <== basically auto updates are suspended as they've been breaking. We even a "Be Ware!" note doing CP's that they too can go spectacularly south.
<mup> Bug #559128: edge rollouts broke CSS <Launchpad Foundations:In Progress by flacoste> <https://launchpad.net/bugs/559128>
<thumper> plumbing emergency
<thumper> AFK
<lifeless> ok, and how do I run a single test ?
<lifeless> wgrant: ^
<wgrant> lifeless: I generally use bin/test -cvvt nameoftest
<lifeless> wgrant: what generates bin/py ?
<wgrant> lifeless: buildout, probably. Just run make.
<lifeless> ok
<lifeless> whats the magic to fix
<lifeless>   Getting distribution for 'zope.testing==3.9.4'.
<lifeless> Error: Couldn't find a distribution for 'zope.testing==3.9.4'.
<lifeless> ?
<wgrant> Update your download-cache.
<wgrant> Or just rocketfuel-get
<lifeless> make update-your-download-cache ?
<wgrant> bzr up ~/launchpad/lp-sourcedeps/download-cache
<wgrant> rocketfuel-get will do that for you, though.
<lifeless> wgrant: hmm, not finding the test
<wgrant> lifeless: Which test, and what are you running?
<lifeless> its a single test in a unittest test file, not a doctest
<lifeless> with -ct it seems to find it
<lifeless> I'm running 'testr run -- -ct testname' now
<mwhudson> spm: so are the code import slaves on a newer branch yet?
<spm> mwhudson: alas no...
<mwhudson> wgrant: i pushed a fix for the serveOverHTTP hang to ~launchpad-committers/launchpad/python2.6
<thumper> mwhudson: did your emacs change font when upgrading to lucid?
<mwhudson> thumper: maybe
<mwhudson> it certainly has done on upgrade before, don't remember if the lucid upgrade did
<thumper> mwhudson: because I don't like the current font but I can't remember what I had :(
<mwhudson> thumper: heh
<mwhudson> thumper: i seem to be using "DejaVu Sans Mono"
<thumper> seems good enough I suppose
<thumper> I don't feel happy with any of them
<spm> I used to use one called 'console' for terminals; bu tthe latest version of 'konsole' makes a *horrible* mess of that and a few other fonts; so switched to dejavu sans mono, which is the least worst...
<thumper> mwhudson: how is your afternoon going?
<mwhudson> thumper: ok i guess
<thumper> mwhudson: need to talk or are you chugging along nicely?
<mwhudson> thumper: still working on the no-hosted-area puller, i deleted a few too many tests
<thumper> mwhudson: I can't do the qa I wanted to do
<thumper> heh
<mwhudson> thumper: argh
<thumper> I could fix the issue that sinzui assigned to 10.04
<thumper> for us
<lifeless> thumper: mwhudson: for one of you; argh
<lifeless> timeout proposing the branch
<lifeless> https://code.edge.launchpad.net/~lifeless/launchpad/merge
<lifeless> ok got it in on 4th try
<lifeless> https://code.edge.launchpad.net/~lifeless/launchpad/merge/+merge/23521
<wgrant> mwhudson: Only 'slightly horrible'? :P
<wgrant> lifeless: There seems to be something really really wrong with that diff.
<thumper> the timeouts suck
<thumper> I wonder why you are getting those
<wgrant> I guess because you proposed to db-devel rather than devel.
<lifeless> wgrant: I proposed to the thing lp suggested I should
<wgrant> Yeah, the development focus is not actually the focus of development, in Launchpad's case
<lifeless> wasn't there a thread to fix that ?
<thumper> well...
<thumper> the development focus is the default stacking target
<mwhudson> wgrant: there's no monkey patching
<mwhudson> at least
<wgrant> mwhudson: True.
<mwhudson> lifeless: the timeouts are probably because the branch was still being scanned
<mwhudson> (yay branchrevision!)
<wgrant> maxb: You appear to have fixed bug #526826; you might want to assign it to yourself and mark it as Fix Committed.
<mup> Bug #526826: test_ftparchive failing on Lucid <soyuz-publish> <tech-debt> <Soyuz:Triaged> <https://launchpad.net/bugs/526826>
<wgrant> mwhudson: Is BranchRevision going to become less stupid some time soon?
<wgrant> stub: Is trusted.sql tested anywhere?
<mwhudson> wgrant: hopefully
<stub> make schema ensures it is syntactically correct.
<stub> Individual bits get tested when those application layer bits get tested.
<stub> such as the triggers etc.
<stub> check constraints etc. not so much, as they are just a safety net (the application needs to handle validation nicely, rather than have the database throw back exceptions to the end user)
<lifeless> wgrant: should I repropose against devel ?
<wgrant> lifeless: Probably. Otherwise ec2 land won't DTRT, and the diff will make people scream.
<lifeless> why doesn't devel turn up in the web form ?
<lifeless> done
<lifeless> thumper: your trivial fix is done, I think.
<mwhudson> grr fricking slow acceptance tests :(
<wgrant> mwhudson: Slow, or hanging?
<mwhudson> just slow
<thumper> do we have official redacted text?
<wgrant> thumper: Private teams used <hidden>.
<wgrant> But there's already special text for redacted import URLs, isn't there?
 * thumper shrugs
<thumper> mwhudson: I'll do the merge conflict if you like
<mwhudson> thumper: that'd be cool
<thumper> fix submitted
<thumper> what was the solution to SFTPError: Garbage packet received ?
<thumper> using ec2 land
<thumper> _ensure_ec2test_user_has_keys is in the stacktrace
<mwhudson> thumper: merge devel
<mwhudson> thumper: but that was quite a while ago, is this an old branch?
<thumper> mwhudson: no, newly created from devel about an hour ago
<mwhudson> or if it's branched of production-devel i guess you can run ./utilities/ec2 from devel;
<thumper> ah
<mwhudson> thumper: don't know then
<thumper> no it isn't
<thumper> it uses the LCA from devel and production-devel
 * thumper pqm submits
<lifeless> thumper: I know its near EOD; if you could tell me if I did roughly the right thing - even if its not a full review - that would be awesome
<thumper> which url?
<lifeless> https://code.edge.launchpad.net/~lifeless/launchpad/merge/+merge/23523
<lifeless> thumper: ^
<thumper> commented
<lifeless> thanks!
<lifeless> thumper: why does mergeFailed not dequeue ?
 * mwhudson submits no-hosted-area to ec2 test -- should be good for a laugh -- and EOWs
<lifeless> thumper: in fact, I'm deleting mergeFailed, because its cleaner.
<lifeless> thumper: if its not deleted, dequeue has to be a special case
<lifeless> thumper: if you haven't eod'd, I've replied.
<maxb> wgrant: thanks, didn't realize there was a bug for that
<adeuring> good morning
<wgrant> maxb: What were the 2.6 pygpgme failures that you mentioned earlier?
<wgrant> I don't see any.
<stub> Who prepared out Python2.5 packages? I'm missing setuptools and easy_install isn't there either for the quick fix (catch-22 there I guess...)
<maxb> wgrant: I submitted a branch for bumping to the existing r49 of the sourcecode/pygpgme branch, assuming it would have been tested before it got landed there. But actually, it breaks things
<maxb> stub: that would be me and wgrant
<wgrant> maxb: Ah, so it didn't actually land?
<stub> Is it quick to get the packages made (eg. copying the packages from hardy into the PPA)? Or should I just install setuptools from source?
<stub> c/hardy/karmic
<wgrant> Why do you need setuptools?
<stub> I need to generate 2.5 eggs
<wgrant> Ah.
<wgrant> A no-change rebuild should do it, I think.
<maxb> wgrant: specifically, pygpgme seems to be returning unicode strings instead of byte strings all over the place now, leading to doctests puking all over the place - plus some other fails :-(
<wgrant> Lucid?
<stub> Yup
<wgrant> maxb: Hm. Damn. We'd best fix that.
<stub> So I just find the Karmic package and copy it into the Launchpad PPA, with the rebuild option selected?
<maxb> stub: no!
<maxb> why do we need this, btw?
<stub> There is an open bug in distutils, the end result being we need to use eggs of pytz rather than tarballs.
<stub> I can't generate a new 2.5 egg of pytz at the moment because setuptools isn't installed on my box.
<maxb> oh, yes. I really need to got back to upstream on that
<stub> I can just install from source if it isn't a quick fix
<maxb> s/distutils/setuptools and distribute/
<stub> Yer - first part of installing from source is working out what needs to be installed ;)
<maxb> stub: Lucid has switched from setuptools to distribute. Installing setuptools on lucid would be.... potentially chaotic
<maxb> I wonder if I should do a no-change upload of distribute to the ppa
<wgrant> Worth a try.
<stub> It won't be a dependency, so it won't make anything explode
<wgrant> Uh... well...
<wgrant> I might point out that *buildout* is involved here.
<maxb> :-)
 * stub lunches
 * maxb lpnochange distribute
<stub> ta. I'll give that a spin when I'm back
<mrevell> Morning
<mwhudson> jml: so, how do i use subunit?
<mwhudson> i have this test results mail and i'd like to have a list of failing tests
<jml> mwhudson, you could open it in tribunal, use subunit2pyunit or use subunit-filter
<jml> mwhudson, is there no list of failing tests in the email?
<mwhudson> not that i can see
<mwhudson> also subunit-filter is breaking
<jml> mwhudson, can you please forward me the email?
<jml> mwhudson, how is it breaking?
<mwhudson> jml: http://pastebin.ubuntu.com/415439/
<mwhudson> jml: mail sent
<maxb> mwhudson: Your hackery regarding test hangs under python2.6 is.... intriguing :-)
<maxb> I am rather mystified, as I can't pin down the underlying cause of the problem appearing
<mwhudson> maxb: yeah, i couldn't reproduce the problem in a small example
<mwhudson> but my socket knowledge has bit rotted a bit
<maxb> oh, really? I did
<maxb> h=HTTPServer(); h.start_server(); h.stop_server(); h._http_thread.join()  <<<<HANG
<mwhudson> the issue is that in 2.5 socket.shutdown() gets called when another thread is in accept, and accept returns when this happens
<mwhudson> maxb: i meant 'just using the socket module'
<mwhudson> in 2.6 the other thread is in select(), not accept()
<maxb> oh, really? I didn't pursue the shutdown angle. I went straight to 'why doesn't closing the socket kick it out of select()'
<maxb> and the answer seems to be 'Python has a LAAAAME wrapper that attempts to implement platform-agnostic socket.dup() which doesn't actually close the socket'
<maxb> I think I might have a try at simplifying your changes - I postulate that adding a server._http_thread._sock.close() cleanup will do the job
<mwhudson> ah
<mwhudson> i wasn't very awake when i was digging :)
<maxb> hmm. oh gosh. I wonder if the very fact we're inside a select on the socket is what keeps alive the refcount which stops the socket being closed
<mwhudson> well, in 2.5 we're in an accept() when stop_server is called
<deryck> Morning, all.
<jml> mwhudson, I'm going to spend a little time digging around your subunit failure
 * jml is off the phone now
<jml> mwhudson, fuck
<jml> mwhudson, I think this is a \r\n problme
<bigjools> mwhudson: thanks for fixing my fsckup
<bigjools> jml: I responded to some of your questions on the pools LEP
<jml> bigjools, thanks.
<bigjools> some new mockups done
<bigjools> well, changed
<wgrant> bigjools: How inappropriate would it be to rSP bugs when closing them from a changelog?
<bigjools> wgrant: rSP?
<maxb> removeSecurityProxy?
<wgrant> bigjools: removeSecurityProxy. It's already pretty much done in the normal case, where process-upload.py uses a permissive security policy.
<bigjools> right
<bigjools> my initial reaction is: nofuckingwayareyoumad?
<bigjools> :)
<bigjools> the right approach is to get the security model correct
<bigjools> rSP is just papering over turd-smeared walls
<wgrant> What is 'correct' here?
<wgrant> It is.
<wgrant> But switching to a permissive policy during a webapp request sounds approximately infinitely worse.
<bigjools> we need to define "correct"
<maxb> When do you close bugs from a changelog from a webapp request?
<bigjools> queue accept
<maxb> ah
<bigjools> deryck: around?
<maxb> Why doesn't the accepting user have permission to close the bug anyway?
<wgrant> maxb: The bug is private.
<maxb> hmm
<deryck> bigjools, hi
<bigjools> hey deryck, can you check scrollback and give your opinion please?
<wgrant> (a similar thing can also now happen if a public bug is Won't Fix, although I believe all of ~ubuntu-archive has sufficient privs to avoid that problem)
 * deryck looks
<bigjools> deryck: just 3 mins back
<wgrant> Bug 564491
<mup> Bug #564491: Cannot accept package which closes inaccessible bug <oops> <queue-page> <ui> <Soyuz:Triaged> <https://launchpad.net/bugs/564491>
<henninge> wgrant, bigjools: Hi!
<henninge> ;)
<bigjools> guten tag henninge
<deryck> bigjools, do you mean how do I feel about using  removeSecurityProxy?
<wgrant> Morning henninge.
<henninge> Is bug 539185 valid?
<mup> Bug #539185: BuildBase implementation for build farm jobs <buildfarm> <Launchpad Translations:Triaged> <https://launchpad.net/bugs/539185>
<wgrant> That sounds like a noodles question.
<bigjools> deryck: that's a no-no AFAIAC, but it would be good to see if you have any good ideas on how to deal with this situation "properly"
<wgrant> AIUI he is reworking that all.
<bigjools> he is
<henninge> so I should not touch that atm?
<wgrant> henninge: That sounds sane.
<bigjools> henninge: have a chat with him, you might be able to help :)
<henninge> or that ;-)
<jml> mwhudson, it's definitely a \r\n bug
<danilos> bigjools, naughty, naughty boy... ;)
<bigjools> danilos: I know, I suck
<bigjools> assumptions are the mother of all fuckups, as they say
<danilos> bigjools, but sure, if it needs helping, and henninge can do it while making sure it works for us as well, excellent :)
<deryck> bigjools, what is the user when trying to close a bug from a changelog?  I'm afraid I need to understand better.
<henninge> danilos: I would have expected nothing else
<bigjools> danilos: well that would be beneficial to us all, yes :)
<henninge> noodles775: Are you out to lunch? ;-)
<bigjools> deryck: let's have a call
<bigjools> I can explain
<wgrant> Speaking of the buildfarm... can we have a launchpad-buildfarm project so we can stop randomly assigning bugs between Code, Soyuz and Translations?
<danilos> henninge, you are in the same country, just drop by him
<henninge> ;-)
<bigjools> wgrant: yeah, I'll try and set that up
<henninge> danilos: true, it's my turn. Last time he came by me.
<danilos> wgrant, no, let's just assign them all to soyuz
<deryck> bigjools, sure.  skype or mumble?
 * danilos puts on an innocent smile
<bigjools> deryck: given that I wasn't around when mumble accounts got handed out, skype :)
<bigjools> hopefully they'll be openid today or monday
 * wgrant hastily uses launchpadlib to reassign all soyuz-build bugs to danilos.
<bigjools> lol
<danilos> heh
<deryck> bigjools, give me 2 minutes to close something here.
<danilos> that API thing was a big mistake!
<noodles775> henninge: nope, I'm here.
<bigjools> deryck: ok call when you're ready, you can guess my skype id ...
<bigjools> mmm can't see any ash in the sky yet
<wgrant> I wonder when air traffic will resume.
<bigjools> this evening they say
<bigjools> but they said this morning, yesterday :)
<wgrant> Qantas here is saying Sunday.
<bigjools> depends on where each airline's planes are located I guess, they need to catch up
<wgrant> I guess, yeah.
<wgrant> But it probably depends more on what the eruption does...
<jml> we really should have "components" in the tracker
<jml> mwhudson, I guess you're beyond caring right now, but I can't track down why the critical '\r' character is being dropped
<noodles775> henninge, danilos, wgrant: bug updated (well, commented).
<wgrant> jml: +inf
<danilos> noodles775, thanks
<henninge> noodles775: great thanks!
 * henninge goes to lunch now ...
<henninge> ;-)
<danilos> noodles775, that does sound great as well, thanks :)
<noodles775> np.
 * noodles775 follows henninge 
<maxb> noodles775: Hi. Has the ec2 of lp:~maxb/launchpad/use-pygpgme-r49 hung, or has it died and just not sent me an email?
<jelmer> maxb: it would've died, that's what's happened to me twice now so I expect it to not be a coincidence
<jml> I thought we fixed that issue
<jelmer> It's a different one probably
<jelmer> I have no problems with other branches, just with this specific one
<lifeless> jml: you might like, in your 'advocating apis sub-hat', the recent commits to lp:pqm
<jml> lifeless, thanks for the heads up
<maxb> jelmer: Hi. I have run into test failures when I try to update pygpgme to the update you landed on the sourcecode branch. Do you know anything about that?
<bigjools> jml: do you know much about twisted's TimeoutMixin?
<jml> bigjools, I know a bit.
<bigjools> we appear to see it completely ignoring the timeout that was set
<jml> bigjools, show me the cdoe
<bigjools> and an exception ensues
<bigjools> jml: top of lib/lp/buildmaster/manager.py
<jml> bigjools, today's stable recent enough?
<bigjools> oh yes
<jml> bigjools, can you show me the error too?
<bigjools> yup one sec
<bigjools> jml: http://pastebin.ubuntu.com/415490/
<deryck> Could someone please subscribe me to bug 441039?
<mup> Bug #441039: Ubuntu One crashed on launch <apport-crash> <i386> <ubuntuone-karmic> <Android's Fortune:Invalid> <NULL Project:Invalid> <Ubuntu One Client:Fix Released by dobey> <ubuntuone-client (Ubuntu):Fix Committed by dobey> <https://launchpad.net/bugs/441039>
<jml> bigjools, have you reproduced this on a development environment?
<bigjools> jml: hahaha
<jml> bigjools, I can help you write a unit test for this.
<bigjools> sounds good
<jml> bigjools, couple more questions first though
<bigjools> wgrant: you have a local builder setup, can you re-produce this?
<jml> bigjools, what is config.builddmaster.socket_timeout set to on production?
<wgrant> bigjools: It doesn't actually need a builder at all. But I will try it.
<wgrant> Oh, wait, that one does.
<wgrant> Not the one I thought.
<bigjools> it's where it ignores the timeout when scanning it
 * wgrant firewalls.
<bigjools> jml: 180
<jml> bigjools, how did you figure that out?
<bigjools> jml: I looked in the production configs
<jml> bigjools, are we sure they are the ones being used in the deployment with this error?
<bigjools> jml: reasonably, why?
<bigjools> I'll get lamont to check
<jml> bigjools, well, if it's not set, or if it's set to a value higher than the system socket timeout, then no amount of debugging will help.
<bigjools> system is like an hour, I thought?
<bigjools> but maybe it's not the default any morie
<jml> bigjools, well, as far as I can tell from that log you pasted, it is timing out roughly three minutes after the process starts
<bigjools> which is 180 seconds as the config says
<jml> right.
<deryck> adeuring, can you subscribe me to bug 441039 please?
<mup> Bug #441039: Ubuntu One crashed on launch <apport-crash> <i386> <ubuntuone-karmic> <Android's Fortune:Invalid> <NULL Project:Invalid> <Ubuntu One Client:Fix Released by dobey> <ubuntuone-client (Ubuntu):Fix Committed by dobey> <https://launchpad.net/bugs/441039>
<jml> so what's happening that differs from what you are expecting?
<bigjools> I wasn't expecting a traceback
<jml> deryck, done
<wgrant> bigjools: So, I get a connection timeout after 20 seconds here. It looks like the dev timeout should be 40 seconds.
<bigjools> but I don't know twistd well enough to know what it's supposed to do when the timeout is set
<adeuring> deryck: "The following errors were encountered:Deryck Hodge has already been subscribed"
<bigjools> jml: is there a callback from the setTimeout() call if it does time out?
<deryck> adeuring, ah, ok.  someone must have already.  thanks.
<bigjools> wgrant: yeah dev is 40. ummm wtf
<jml> bigjools, yeah, timeoutConnection
<jml> bigjools, http://paste.ubuntu.com/415493/
<bigjools> jml: which is not defined in the buildd-manager ....
<bigjools> ho ho ho
<jml> bigjools, the default implementation disconnects
<bigjools> jml: ok I'm trying to work out what effect that would have on b-m
<jml> http://twistedmatrix.com/documents/current/api/twisted.protocols.policies.TimeoutMixin.html fwiw
<wgrant> bigjools: To be clear, I get a traceback like that, except with a normal socket exception.
<bigjools> ok
<bigjools> so I can't figure out if that's normal for the timeout that was set or if something else causes it
<jml> well, it's not an error that TimeoutMixin generates
<jml> but it's reasonable to expect xmlrpclib to explode if its socket is disconnected from under it.
<bigjools> we don't get the "Scanning failed with:" message so it's not hitting the scanFailed callback
<bigjools> I would have expected twisted to catch that and call the right callback in the Deferred
<jml> hmm
<bigjools> which is what we normally see
<bigjools> so I still wonder if it's a twistd bug
<jml> bigjools, what kind of object is "self.slave" in the traceback you pasted
<bigjools> heh
<jml> bigjools, nvm
<bigjools> this stuff is like black magic
<lifeless> its not that clean
<bigjools> ha
<bigjools> I know that it's an xmlrpc thingy but I can't find exactly where it's instantiated
<jml> bigjools, because it looks like it's using a non-Twisted xmlrpc client
<bigjools> because this code is so obtuse
<jml> I also can't find where the 'echo' method that is called in the traceback is defined
<bigjools> xmlrpclib.ServerProxy
<bigjools> in there ^
<jml> no it's not.
<jml> ServerProxy has magic __getattr__ evil
<jml> but where's the server-side implementation that it's trying to call?
<bigjools> jml: see lib/canonical/buildd/slave.py
<bigjools> it's magic, like I sid
<bigjools> said
<jml> ok.
<bigjools> :)
<bigjools> XMLRPCBuildDSlave specifically
<jml> bigjools, so the thing is, we hate transparent RPC in Twisted
<bigjools> s/in Twisted//
<jml> bigjools, well, we hate it enough to do something about it
<wgrant> This is why we have the even more transparent launchpadlib...?
<jml> bigjools, your hatred is like a candle to our sun
<bigjools> rofl
<jml> self.slave.callRemote('echo', ...) is the way it's normally done in Twisted
<noodles775> maxb: hey. I only received emails about trivial-bad-httpcaller..., py2.6-warnings and tolerate-lucid-apt... It was definitely running as I checked it's status when you asked yesterday, so I'd say it died :/
<lifeless> wgrant: launchpadlib makes me think of the movie Mystery Men
<bigjools> jml: basically this won;t work then until we make both sides use twisted's xmlrpc gubbins?
<maxb> noodles775: ok. no need to resumbit, it has further problems that need addressing anyway.
<jml> bigjools, so, I _think_ what's happening here is that the twisted client with the timeout mixin is doing its job and pulling the rug out from under the non-twisted client
<jml> bigjools, but that's only a guess
<jml> bigjools, yeah, I think so.
<bigjools> jml: I concur with you
<wgrant> Why does the slave need to?
<bigjools> jml: I wonder if there's anything we can do on the client side to handle this better in the timeout callback
<bigjools> where the slave == the server side (confusing I know)
<jml> bigjools, you mean, as a work-around?
<bigjools> yes
<jml> bigjools, nothing leaps to mind
<bigjools> I have no desire to start re-writing the xmlrpc innards :(
<jml> bigjools, and as for adding another obscure workaround to this code... the phrase "house of cards" leaps to mind.
<bigjools> :)
<bigjools> the stuff at the top of lib/lp/buildmaster/model/builder.py also deals with timeouts
<jml> bigjools, you could wrap the non-twisted xmlrpc calls in a try/except block -- that'd be your quickest way out, I think.
<jml> bigjools, yeah, that'll be what's triggering the error you see.
 * bigjools idly flicks the bottom card away
<jml> gah
<jml> I have no idea how this code works at all
<bigjools> is there anyone here in Sao Paulo who's prepared to do some kidnapping?
<wgrant> We really really need to refactor things a bit so RecordingSlave can DIE.
<wgrant> Hahah.
<wgrant> I know how it mostly works... but it's mostly evil.
<bigjools> every time I think I understand it, I promptly get confused again
<bigjools> RecordingSlave was a quick way to avoid doing what I hinted at earlier - re-writing the xmlrpc gubbins to be twisted
<jml> hmm
<jml> are there any bugs filed about cleaning it up?
<wgrant> It's not *that* hard. Just sometimes you'll get a RecordingSlave, sometimes you'll get a BuilderSlave, and sometimes it depends on who called this particular bit of code this time. And then you get fake responses which are fricking confusing and aaarrrrgh.
<jml> I might add them to my someday/maybe list.
<bigjools> it's basically proxying what's in BuilderSlave
<jml> you know
<jml> after I fix subunit, land the ssh server changes, make pretty graphs for bugs and make our test suite runnable more than once on a computer
<bigjools> can I have a flying car
<wgrant> Only if it's asynchronous.
 * jml returns bigjools a Deferred that will fire a flying car on success
<bigjools> ha
<jml> actually, maybe before the runnable-more-than-once thing
<bigjools> jml: I wonder if we just need to remove that timeout in builder.py and wait for the twisted one to work instead
<bigjools> it seems like they will compete
<jml> bigjools, they'll definitely compete
<jml> zoinks
<bigjools> jml: this is not using twisted xmlrpc at all, RecordingSlave is just setting up a load of Deferreds
<bigjools> the mixin seems like a total waste of time?
<jml> bigjools, well, dispatchBuild and checkDispatch use it
<jml> I think
<stub> maxb: That distribute package just seems to have 2.6 stuff in it. No 2.5.
<maxb> ugh
<jml> unfortunately, I've got a bucket load of stuff to do today. :(
<maxb> It must not respond to the general python version selections presented in python-defaults and python-support
<maxb> I am about to lunch. I'll take a look afterwards
<bigjools> jml: no worries, thanks for your time
<jml> bigjools, np. my pleasure.
<bigjools> jml: I think I can throw an extra catch in here to get that timeout but I've no idea how to test it!
<gmb> allenap, Do you have time for a call about bug 530113?
<mup> Bug #530113: Information on a bugwatch error is obscure <story-reliable-bug-syncing> <Launchpad Bugs:In Progress by gmb> <https://launchpad.net/bugs/530113>
<allenap> gmb: In about 3 minutes (ordering computer parts, woot).
<gmb> allenap, Cool, I'll sit around on t'mumble.
<jml> bigjools, I know how to test timeouts with pure Twisted systems
<jml> boo yah
<jml> the first of my four SSH branches just passed ec2 test
<jml> (they failed yesterday because devel was broken)
<bigjools> jml: I'll think of something hackish no doubt
<bigjools> jml: I apologise for breaking devel
<jml> bigjools, np. the real problem is the system that allows a simple human error to screw up other people's workflows
<bigjools> aye
<bigjools> jml: if you have a sec can you look at lib/lp/buildmaster/model/builder.py -  updateBuilderStatus()
<bigjools> and tell me what else it needs to catch to get that timeout?
<jml> bigjools, that ought to be it...
<bigjools> you'd think :/
<jml> bigjools, easy enough to figure out though. Make a TimeoutHTTPConnection work-a-like, give it a low timeout, see what error gets raised
<bigjools> right
<bigjools> shame it's not in the log :(
<bigjools> mmm this makes me hungry
<jml> I think it's more that socket module has crap errors.
<deryck> yay, gnome-bugs bugs are updating!
<deryck> gmb, error message question for bug watch update activity....
<deryck> gmb, see bug 44082
<mup> Bug #44082: GNOME Panel icons (on right side) move apparently randomly on session start in some situations <gloam> <qa-hardy-desktop> <qa-jaunty-desktop> <qa-karmic-desktop> <GNOME Panel:Fix Released> <One Hundred Paper Cuts:Confirmed for ryan.maki> <gnome-panel (Ubuntu):Confirmed for desktop-bugs> <gnome-panel (Ubuntu Hardy):Triaged by desktop-bugs> <https://launchpad.net/bugs/44082>
<gmb> Urr.
<nigelb> deryck, its more like "oh no! mail flood!"
<deryck> nigelb, for you. :-)  For me, it's "oh happy day, praise be to the internet almighty." :-)
<nigelb> deryck, haha.
<gmb> deryck, Whatsyerquestion?
<deryck> gmb, so expand the first task row.... that error there?  Is that from bug watch activity log?  It seems out of date, since the bug is in fact updated now.
<gmb> deryck, It's not from the activity log, no, it's from the last_error_type property of BugWatch. Let me just have a poke around and see if I can find out what's oging on.
<deryck> gmb, cool, thanks.  I want this expandable form to go away.  It causes no end of grief since it gets out of sync with the rest of the app.
<maxb> Is there any mechanism by which a ~launchpad-pqm sourcecode branch can be push --overwritten?
<gmb> deryck, allenap: AHAHAHAHA
<deryck> oh, do tell
<gmb> deryck, So, that error is a real error. The status sync succeeded but the backlinking failed (and our errors aren't smart enough to handle that yet; will fix that this pm).
<gmb> deryck, OOPS-1567CCW1152
<gmb> (Another win for BWA, though)
<deryck> indeed!  This logging is working out well.
<gmb> deryck, allenap: Looks like we don't have permission to backlink on the remote system.
<allenap> gmb: Oh, jolly good.
<gmb> allenap, I think it might be because we (used to?) backlink for every watch, whether the LP bug was relevant or not.
<gmb> So the gnome-bugs admins disabled backlinking.
<allenap> gmb: That sounds right.
<gmb> allenap, Is there an easy way to disable backlinking on our end to prevent the errors or is it not fine-grained enough for that? ICR.
<allenap> gmb: Not fine-grained enough.
<allenap> gmb: We could quite easily disable all back-linking by removing the ISupportsBackLinking interface from BugzillaAPI, but that could/would affect other trackers.
<gmb> allenap, Yeah.
<gmb> allenap, Though, hang on, if we can't sync comments why are we trying to backlink anyway?
<gmb> That seems like huffing on the bong pipe to me.
 * allenap looks
<allenap> gmb: It's not conditional on sync_comments.
<gmb> allenap, I think it should be. That would solve this problem for a start.
<allenap> gmb: The condition, in full, is: (bug not dupe and linked to at least one bugtask and supports back-linking)
<danilos> noodles775, hi, do you have any suggestions how could I QA https://code.edge.launchpad.net/~jtv/launchpad/bug-553077/+merge/22599?
 * noodles775 looks
<gmb> allenap, Right. Do you think that sync_comments should be used there too? I mean, I know it's a misnomer if we do that, but...
<stub> maxb: np. I worked around by installing from pypi. I then uninstalled it after generating the eggs, as buildout exploded ( https://bugs.edge.launchpad.net/launchpad-foundations/+bug/564680 ). I'm not sure if your package will cause the same issue, but maybe it is best not bothering (I can survive, and we should be on 2.6 sometime soonish)
<mup> Bug #564680: buildout explodes when distribute is installed <Launchpad Foundations:New> <https://launchpad.net/bugs/564680>
<noodles775> danilos: nope.
<noodles775> danilos: I mean, if the buildd's are still running, it's effectively qa'd.
<wgrant> But that code is not on production at the moment.
<noodles775> danilos: so, ensuring dogfo..
<danilos> noodles775, so, how do I get that tested on dogfood?
<noodles775> danilos: so, ensuring the buildd's used by dogfood are updated, and then that it still works.
<noodles775> danilos: as long as we ensure the buildd's are updated, we can just push a build through on df.
 * noodles775 checks with lamont to get the buildd's updated on dogfood.
<danilos> noodles775, cool, thanks (I totally haven't done anything about this, so I don't even know where to start)
<mars> danilos, looks like you found a bug in paramiko: line 163, "str(data[:chunk])" should be unicode(data...)?
<mars> danilos, don't know why it would start failing now though
<danilos> mars, it might not be recent, I haven't used ec2 for a while
<mars> hey, paramiko uses Launchpad! :)  https://edge.launchpad.net/paramiko
<danilos> mars, I can file a bug, if you think that's the thing to do :)
<mars> danilos, not sure if switching to unicode would help.  I just tried an experiement in ipython, and reproduced that error.  It is the str() call.
<mars> str(u'MÄris')  raises the error
<danilos> mars, right, it'd be worth seeing if it makes sense to .encode('utf-8') before passing it out to paramiko: it might be that unicode there won't help either and it'll barf later if no UTF-8 locale is used
<mars> danilos, worth a shot.  Looking at the docs, and sftp should support utf-8 encoding for filenames
<danilos> mars, well, UTF-8 wasn't called UTF for filenames for nothing on Plan9 I think :)
<mars> hehe
<danilos> anyway, me goes get some late lunch, will be back later for a short while
<mars> rocketfuel-branch is the wrong command to use when making a one-line patch :/
<noodles775> sinzui: do you have time in the next 0.5hr for a brief call regarding the style.css work?
<noodles775> (no problem if not).
<sinzui> I do
<mars> gary_poster, have a sec to review a one line patch?  https://code.edge.launchpad.net/~mars/launchpad/fix-ec2-email-encoding/+merge/23557
<gary_poster> mars, looking
<gary_poster> mars, I'm guessing you have empirical proof it works? :-)
<mars> gary_poster, just a traceback from danilos sent to lp-dev.  This same fix is present throughout the ec2test.py file.
<mars> gary_poster, I have have danilos try it out before submitting to PQM
<gary_poster> mars, alright.  I have vague memories of pain with UTF in email headers and the stdlib email packages, but even if things don't get better, this shouldn't make anything worse (and hopefully it is a fix, of course!).  Yes, would be happy if someone gave it a try before merging.  Will approve with that condition, since you already have it planned.
<mars> gary_poster, ok, thanks.  I'll send a mail to danilos saying as much.
<gary_poster> cool
<gary_poster> approved
<bigjools> jml`: still busy?
<jml`> bigjools, now is an ok interrupt time
<bigjools> jml`: ok should be quick :)
<bigjools> see lib/lp/buildmaster/model/builder.py "resumeSlaveHost"
<bigjools> versus
<bigjools> lib/lp/buildmaster/manager.py "checkResume"
<bigjools> the former is called when b-m starts up and reports the stdout/stderr if it fails
<bigjools> the latter is asynchronous and I don't know how to do the same
 * jml` pulls up the code
<bigjools> is stdout/err buried in response somewhere?
<jml`> bigjools, is "slave" a different type of object in the two methods?
<bigjools> I don't think so
<bigjools> unless one's the recording slave and the other's the BuilderSlave
<bigjools> who knows :/
<jml`> so uhh
<jml`> resume is what then?
<bigjools> actually yes checkResume is dealing with a RecordingSlave
<bigjools> resume is when we restart the VM
<bigjools> it ssh'es to the slave and calls a script
<bigjools> the slave's host I should say
<jml`> bigjools, where is it defined?
<bigjools> in config
<bigjools> ppa_reset_command or something
<bigjools> anyway
<jml`> my head hurts
<bigjools> I need to get at the output of running that - it uses run_process_with_timeout
<bigjools> ok let's keep it simple
<bigjools> checkResume gets a response from run_process_with_timeout
<bigjools> if the process failed I want its stdout/err
<jml`> oh, it's a callback attached to run_process_with_timeout
<jml`> ok, I'm on it.
<jml`> bigjools, run_process_with_timeout isn't very good.
<jml`> bigjools, or at least, it gives you know way of getting at stdout/stderr
 * jml` sketches up some code
<bigjools> \o/
<bigjools> :/
<bigjools> if it wasn't Friday afternoon I would be sprawled on my desk in despair
<jml`> run_process_with_timeout is using ProcessMonitorProtocolWithTimeout, which is actually intended for code that wants to monitor a process as it's running and report that information to some external system
<jml`> i.e. not at all what you want
<bigjools> I guess it was used because it has a timeout
<bigjools> and the name seemed right
<bigjools> *shrug*
<bigjools> is this nearer the mark? http://twistedmatrix.com/documents/current/api/twisted.internet.process.Process.html
<bigjools> doesn't seem to handle timeouts
<jml`> no... http://twistedmatrix.com/documents/current/api/twisted.internet.protocol.ProcessProtocol.html
<jml`> make a subclass that accumulates stdout & stderr, fires a deferred on disconnect and mixes in TimeoutMixin
 * jml` is doing a rough sketch right now
<jml`> Two minutes!
<bigjools> you da man
<jml`> bigjools, http://paste.ubuntu.com/415642/
<bigjools> lookng
<jml`> bigjools, ignore that function at the top
<bigjools> ok
<jml`> the clock business is so you can write unit tests that don't rely on the system clock
<bigjools> jml`: looks great, thanks
<jml`> you may want to inherit from ProcessProtocolWithTwoStageKill instead of ProcessProtocol, to get the improved killing behaviour
<bigjools> "improved"? :)
<jml`> bigjools, also, you could actually define "outReceived" and "errReceived", and have them reset the timeout. that'd give a different timeout behaviour which may be more appropriate for what you're doing.
<bigjools> right
<jml`> bigjools, "improved" as in, "send SIGINT, if it's not dead after a little while, send SIGKILL"
<bigjools> don't think it is, the script works in ~6 seconds
<bigjools> ah ok, don't really need that sophistication here I think
<jml`> bigjools, lucky you :)
<bigjools> it's an ssh - it can die die die :)
<jml`> bigjools, for the puller, we reset the timeout on activity -- doesn't matter how long a branch takes overall, just as long as we can reasonably expect it to keep making progress.
<bigjools> yer
 * bigjools hacks
<jelmer> maxb: still there?
<bigjools> jml`: in timeoutConnection you're using error and it's not defined
 * bigjools loves the vim pyflakes plugin
<mrevell> Guten nacht
<maxb> jelmer: back now
<jml`> g'night all
#launchpad-dev 2010-04-18
<thumper> morning
<mwhudson> sooo.... staging update?
<lifeless> mwhudson: thumper: - seeking an opinion. Is bug 562931 worth having as a separate bug [doing the API side and web display without doing web attaching], or would you necessarily do the whole enchilada at once?
<mup> Bug #562931: attachments in merge proposals <Launchpad Bazaar Integration:New> <https://launchpad.net/bugs/562931>
<thumper> lifeless: yes I think it is worth it
<thumper> lifeless: sorry, not enough coffee yet
<thumper> lifeless: well...
<thumper> lifeless: two bugs I think
<lifeless> would you like me to undup and clarify, or will you ?
<lifeless> thumper: ^
<thumper> lifeless: you can :)
<lifeless> what happens to attachments on emails sent to merge reviews, at the moment?
<lifeless> thumper: what importance do you want on it? the 'attachments via web' bug has high
<thumper> they should both probably be medium
#launchpad-dev 2011-04-11
<lifeless> I can has review? https://code.launchpad.net/~lifeless/lazr.batchnavigator/keyoffset/+merge/57091
<lifeless> huwshimi: hi
<huwshimi> lifeless: Morning
<lifeless> huwshimi: wondering if you could eyeball bug 119939
<_mup_> Bug #119939: strike through resolved bugs <lp-bugs> <ui> <Launchpad itself:Won't Fix> < https://launchpad.net/bugs/119939 >
<lifeless> huwshimi: I closed it wontfix because its clearly easy to do if we wanted to, and multiple ui refreshes haven't don it
<lifeless> but I don't have a UI analysis for why its a bad idea, and the filer is (reasonably) asking why we have chosen not to do this
<lifeless> huwshimi: if I was wrong to close it and you think it would be a good idea, we can reopen easily.
<wgrant> lifeless: I might point out that multiple UI refreshes also haven't made the UI not suck.
<lifeless> wgrant: you might
<lifeless> wgrant: and thus me talking to huw :P
<lifeless> it just seemed like a bug that would never get traction - do, or do not.
<lifeless> mwhudson: perhaps I can rope you in for that review ^?
<huwshimi> lifeless: So, I don't think we should do the strikethrough, but it would be super useful to have some kind of indication of the status of the linked bug (e.g. on tooltip or inline: bug #1234 [won't fix]).
<_mup_> Bug #1234: Gina is an unmaintainable mess of command line options, environment variables and shell scripts <lp-foundations> <Launchpad itself:Fix Released by debonzi> < https://launchpad.net/bugs/1234 >
<lifeless> huwshimi: do you mean in linkification, or other places?
<lifeless> fmt: and linkification have different algorithms [not necessarily for good reasons]
<lifeless> huwshimi: if you look here:
<lifeless> https://code.launchpad.net/~sinzui/launchpad/person-merge-job-4
<lifeless> huwshimi: the linked bugs have a mouseover status - 'critical - in progress'
<huwshimi> lifeless: I think this would be appropriate in bug comments/descriptions etc.
<lifeless> huwshimi: ok, thats in linkification. The good news is that I have a canned answer for you. The bad news is 'that turns out to be suprisingly expensive performance wise, back burner at best'.
<huwshimi> lifeless: Yeah I suspected.
<lifeless> its expensive because of bug privacy
<lifeless> huwshimi: anyhow, if you could comment on that bug [which is specifically about strikethrough] or perhaps alter it to talk about freeform bug linkfication and exposing more metadata, that would be cool.
<huwshimi> lifeless: I think it would be a good thing to do, but if the chances of it ever happening are slim to none then maybe we should kill the bug
<huwshimi> lifeless: OK will do.
<huwshimi> lifeless: maybe you should also make a note that we can not do it yet due to performance
<lifeless> huwshimi: all things are possible. I have no objection to a bug open saying 'linkified bugs would be more useful if they showed bug metadata such as title, status, milestone etc'
<huwshimi> lifeless: is "linkified" the correct term for this situation?
<wallyworld> huwshimi: hi. do you have any css you want to give me for the confirmation dialog?
<huwshimi> lifeless: As in, will other people know what we're talking about
<huwshimi> wallyworld: Not yet. But I do want to give you some :)
<wallyworld> np. thanks
<huwshimi> wallyworld: I should be able to send you some today
<wallyworld> excellent smithers
<lifeless> huwshimi: it may not be the official term but its close enough
<huwshimi> lifeless: Sure thanks :)
<lifeless> huwshimi: we have modelled links like 'duplicate' etc
<lifeless> huwshimi: and we have freeform stuff we infer by linkification
<lifeless> modelled stuff already shows the metadata (on mouse over)
<lifeless> wgrant: have you seen https://bugs.launchpad.net/launchpad/+bug/454307 ?
<_mup_> Bug #454307: indices/md5sums.gz doesn't match repository <lp-registry> <mirror> <Launchpad itself:Triaged> < https://launchpad.net/bugs/454307 >
<wgrant> thumper: Can you QA your branch-by-ID thing?
<thumper> wgrant: yeah
<thumper> I think
<wgrant> lifeless: Never seen that before, because it was mistriaged.
<thumper> lifeless: what do we have in place for codehosting on qastaging?
<wgrant> Fixed.
<lifeless> wgrant: I suspected as much - chalk another one up for silos
<thumper> or easier to just use staging?
<wgrant> thumper: Everything.
<wgrant> thumper: codebrowse you need to access by port forwarding, but everything else works.
<lifeless> wgrant: is it a genuine new bug and not a duplicate?
<wgrant> lifeless: I don't recall anything like it.
<lifeless> wgrant: win
<wgrant> ShipIt should be running for the last time right now :)
<wgrant> Then we can disable and delete it.
<wgrant> lifeless: Do you think https://qastaging.launchpad.net/ubuntu is OK? The "Latest derivatives" portlet is new, not flagged, probably undesirable, and mostly bad data.
<wgrant> Bug #747502
<_mup_> Bug #747502: Derived distributions must be discoverable from the "parent" distribution <derivation> <qa-needstesting> <Launchpad itself:Fix Committed by rvb> < https://launchpad.net/bugs/747502 >
<lifeless> wgrant: I would hesitate to ship that
<wgrant> Shall I flag it?
<lifeless> wgrant: please
<lifeless> wgrant: also a gentle reminder to the list 'if changing existing uis with unfinished stuff, feature flag it'
<wgrant> Sure.
<lifeless> mwhudson: ping
<mwhudson> lifeless: hi?
<lifeless> mwhudson: hi; did you see my query re a small lasr.batchnavigator review ?
<mwhudson> oh right, i clicked the link and then forgot about it
<lifeless> heh :) thanks!
<lifeless> https://github.com/defunkt/jquery-pjax#readme
<poolie> hi huwshimi?
<huwshimi> poolie: Hey
<poolie> hey there
<poolie> i had a random idea the other day; i'm not saying it's very urgent to actually do, but i just thought i'd ask you
<poolie> and that was, perhaps bug heat should just be hidden from display and used only as a sort key
<poolie> (and perhaps be in the api)
<poolie> that would avoid some number of complaints or bugs about the absolute values and the scaling onto flames being a bit weird
<poolie> (eg that they're inconsistent across pillars)
<lifeless> I think thats a bad idea
<lifeless> because it hides the machinery
<lifeless> which makes it less understandable, and thus more frustrating when its wrong
<lifeless> I think we should fix the machinery instead
<poolie> but the machinery is almost entirely hidden today
<poolie> it is not understandable at all
<lifeless> I'm not justifying what we have
<lifeless> I explaining why I think making it less visible would be a mistake
<huwshimi> Interesting. I don't really know much about bug heat. What do we use it for apart from the default bugs page?
<poolie> it's shown as a series of flames in some bug lists, and on bug pages
<lifeless> we permit a search sorted by heat
<poolie> and for sort ordering
<poolie> in fact that order is a common default
<lifeless> and the distro and some other teams use hot bugs as a health check
<lifeless> poolie: the default sort is importance
<lifeless> poolie: only the hot bugs widget sorts by heat by default
<poolie> really?
<poolie> because <https://bugs.launchpad.net/bzr> claims they are hot bugs
<poolie> the title may be wrong
<lifeless> poolie: right thats the widget
<lifeless> note that it doesn't show page navigation
<lifeless> it does link to a sort by heat
<poolie> ok, and the 'show me more' is by heat
<lifeless> but if you click on search at the top
<poolie> but, you're right, 'open bugs' is by importance
<lifeless> you get by importance
<lifeless> etc, yes.
<poolie> and, amusingly enough 'high bugs' sorts by importance too
<lifeless> there is/was a bug open saying 'please show the inputs to the heat on the flames'
<poolie> i might have filed it
<poolie> when you say people use it as a health check
<poolie> do you mean they count how many 4-flame bugs there are?
<poolie> or they just look at the open bugs sorted by heat
<lifeless> One of the complicating factors there is that the algorithm decays, so you can't statically calculate, you have to redo each time.
<lifeless> poolie: the latter
<poolie> well, my idea would keep that
<lifeless> stub is investigating heat as a possible cause of index bloat
<poolie> i think sorting is pretty worthwhile
<poolie> i suspect showing it on the bug page is pretty much noise
<lifeless> if it is, we'll need to revisit the entire implementation
<lifeless> I suspect we'll get the same sort with a /little/ fuzz without the decay
<lifeless> (because commenting on a bug removes the decay for a day)
<poolie> anyhow, it was just a random thought
<poolie> that perhaps making it serve the purpose of providing an ordering, and making it provide an absolute metric, and making it provide a scaled ranking
<poolie> may be asking too much
<wgrant> lifeless: https://code.launchpad.net/~wgrant/launchpad/bug-756983/+merge/57097
<poolie> lifeless, do you think anyone loves, or looks at, the flame icon?
<poolie> they may well do
<lifeless> wgrant: I think bigjools and raphael may be confused by your comment abot desirability
<poolie> but not me
<wgrant> lifeless: Possibly.
<lifeless> poolie: I think that if we don't expose it, we will have questions like your one about 'sort by date changed'
<lifeless> poolie: we don't expose date changed in the UI, so you can't decide if its buggy or not and you file a bug.
<poolie> lifeless, if you mean the bug i filed, i think that was actually a complaint about it not being in the ui
<poolie> gah, in the API
<lifeless> poolie: the bug you filed was that a particular bug was sorted at the top of 'my bugs sorted by date'
<poolie> well, that is just weird
<lifeless> poolie: bug 752169
<poolie> it seems like there for some reason the value is correct but the sort is wrong
<_mup_> Bug #752169: wrong ordering for date_last_updated? <Launchpad itself:Triaged> < https://launchpad.net/bugs/752169 >
<poolie> right
<lifeless> poolie: all I'm saying is that when there is a sort order and the data behind it isn't visible, folk will get confused.
<poolie> sure
<poolie> i don't think the bug is very analogous
<poolie> but i agree that being able to see "why is it doing that" can be good
<poolie> i don't think the flames are very helpful in understanding it
<poolie> i think it's more likely people will consider the numeric heat is wrong, than that lp will fail to sort integers once they have been calculated
<poolie> (which seems to be happening in 752169)
<lifeless> sure
<mwhudson> lifeless: i don't know if you care, but ListRangeFactory.getSlice will not DTRT in size==0 and forwards=False
<mwhudson> i think
<mwhudson> (as in, i think it will return the whole list)
<mwhudson> slicing is one of the points were one wants a negative zero
<lifeless> mwhudson: I don't think we need to care; size 0 is utterly bogus anyway.
<lifeless> of course, some one will trigger a timeout using that someday.
<thumper> :(
<thumper> lp://qastaging/ urls don't work
<mwhudson> lifeless: it's almost a natural law, any code around slices will get at least one edge case a bit wrong
<thumper> spm: what's the status of the staging code update?
<spm> should be able to kick that off shortly actually
<mwhudson> (i spent a long time fixing problems in CPython's code wrt this sort of thing)
<thumper> spm: it says it is doing one now
<spm> thumper: technically that's a lie, it's just the existing locks and blocks still in place from the fail on the w/e
<thumper> spm: ok, so ETA?
<spm> to kick it off, RSN, completion, no idea.
<thumper> ok
<spm> I think the last successful full restore was ages back, and rob's asked for a full in that case, so... 15-20 hours?
<lifeless> mwhudson: yup, I've at least one patch in CPython for this too - on find() IIRC
<lifeless> (which takes slice params, but didn't interpret them correctly)
<huwshimi> wallyworld: On the dialogue do you know how to get the green bar on below the title back?
<wallyworld> huwshimi: is that missing? let me look
<wallyworld> huwshimi: so it is. i will figure out how to make that stay. it may be that it's tied to the "steps" node which is hidden when the confirmation is shown
<huwshimi> yeah I wondered that. I had a look at the privacy dialog on the same page and it somehow manages to have it
<lifeless> win
<lifeless>     DataError: OFFSET must not be negative
<lifeless> StevenK: good morning
<StevenK> Morning
<thumper> lifeless: anyway to see the post params for the oops? https://lp-oops.canonical.com/oops.py/?oopsid=1131EC415
<lifeless> thumper: sure it
<lifeless> s
<lifeless> field.actions.* in the request variables
<wgrant> It's in the URL.
<thumper> ah... I see it now
<lifeless> http://infram.wordpress.com/kadosu-categorized-document-search/specification-\u2026put-of-pluginsspecification-for-filtering-text-output-of-plugins
<lifeless> is the problem
<lifeless> erm
<lifeless> wtf
<lifeless> # Branch: launchpad
<lifeless> # Revno: 7688
<thumper> ?
<lifeless> the revno
<lifeless> oh
<lifeless> 2009 oops
<thumper> :)
<lifeless> https://lp-oops.canonical.com/oops/?oopsid=OOPS-1926EE1366 looks more interesting
<thumper> :(
<thumper> it seems the middle click button on my mouse is dying
<lifeless> argh
<lifeless> sqlobject fail fail fail fail fail fail fail fail fail fail fail
<wgrant> lifeless: Oh?
<lifeless> partial implementation of slice protocol
<lifeless> resultset[-5:None] -> boom
<lifeless> mwhudson: pushing a tweak
<mwhudson> lifeless: whee!
<lifeless> rule #1 if you make it look like a slice, let it damn well implement slice
 * lifeless grumps
<mwhudson> lifeless: does resultset[-5:] work?
<thumper> WTF? we have our own urlparse?
<thumper> all it does is encode('ascii')
<mwhudson> thumper: something to do with unicode
<thumper> yeah, but I'm still trying to work out why we care
<wgrant> We are broken and generate Unicode URLs in a lot of places.
<mwhudson> the cache gets polluted with unicode objects if you feed it a unicode url
<mwhudson> or did, maybe that's fixed in 2.6...
<thumper> what cache?
<lifeless> mwhudson: the fail is because it passes the offset down to sql unaltered
<mwhudson> lifeless: lovely
<lifeless> mwhudson: in principle it could reverse all the sort elements automatically and adjust
<lifeless> mwhudson: -5:None = reversed[:5] afterall
<wgrant> lifeless: total_size is replaced with total_size_link in recent webservice versions.
<mwhudson> thumper: urlparse has a cache
<wgrant> (re bug #757007)
<_mup_> Bug #757007: total_size element returned in all batch retrievals <lazr.restful:Triaged> < https://launchpad.net/bugs/757007 >
<lifeless> mwhudson: I realise you're asking if __getitem__ in storm handles the second parameter differently; I haven't checked all the way
<lifeless> wgrant: our tests look for total_size
<lifeless> wgrant: and its still there
<thumper> mwhudson: really?
<thumper> WTF for?
<mwhudson> thumper: beats me
<lifeless> because parsing urls is slow
<wgrant> lifeless: They must be using an older version.
<wgrant> I forget the threshold.. possibly 1.0.
<lifeless> and rather than expose a parser object with state, - win -
<wgrant>     first_version_with_total_size_link = "devel"
<mwhudson> lifeless: yes because parsing the same url over and over again is a common case, obviously
 * mwhudson coughs
<lifeless> mwhudson: if you use damaged stacks, yes.
<mwhudson> thumper: looks like the bug launchpad is working around is fixed though
<thumper> mwhudson: yeah
<thumper> ?
<lifeless> encoding a url to ascii is sensible though
<lifeless> if it can't be encoded, its *not* a valid url and we should reject it.
<lifeless> the stdlib url stuff is broken in many more ways than mere caching
 * lifeless wouldn't ever use it by choice
<lifeless> even more sadly upstream just fundamentally don't understand what urls are.
<lifeless> See the 50 mail thread or whatever was on the *worse* breaking in 3.x
<thumper> lifeless: catching the unicode decode error in validate url seems pretty simple
<lifeless> thumper: yup, works for me
<wgrant> lifeless: A friend from uni tried to fix various bits of the 3.x stdlib URL unicode stuff... it was all pretty wrong and upstream was not at all receptive to fixing it in a reasonable way.
<lifeless> s/pretty/entirely/ :)
<wgrant> Yeah.
<lifeless> as your friend probably knows
<wgrant> I think it's less bad now, though.
<wgrant> Still awful, though!
<lifeless> py3 broke email, broke the web stacks.
<wgrant> The latest WSGI is 3.xable, isn't it?
<lifeless> I suspect so
<wgrant> They finally fixed that, IIRC.
<lifeless> yes
<lifeless> with class sensitive string manipulation functions
<wgrant> Yay
<lifeless> because bytes cannot be permitted to have a find() method, obviously.
<wgrant> I wish they'd backported more of the bytes API to 2.7 :/
<wgrant> Like, say, having a bytes type that indexes like a bytes, rather than making it index like a str.
<lifeless> other than a misleading type alias?
<wgrant> Grar.
<wgrant> Exactly.
<wgrant> It's an alias to something that is not at all like a bytes!
<lifeless> gustavo had a pretty solid rant on this
<lifeless> my opinion is that the bytes type is daft
<wgrant> I ended up casting lots of stuff to a bytearray to get compatible indexing.
<lifeless> mwhudson: can you check my incremental commit, see if still happy ?
<mwhudson> lifeless: looking
<lifeless> With that, at the cost of one more count(*) on Last links & only on Last links
<lifeless> it fixes the last of the test fallout
<StevenK> Is r12758 actually QA-able at all?
<StevenK> It's been top of the deployment report for a few days
<wgrant> StevenK: thumper was looking at it earlier.
<lifeless> probably got distwacted
<wgrant> The HWDB thing needs QA.
<thumper> StevenK: if this is the stacking issue, yes it is qa-able
<thumper> I'm wanting staging to update to poke properly
<lifeless> wgrant: revno?
<wgrant> r12787
<lifeless> thumper: thats going to be a day or more
<lifeless> thumper: use qastaging
<wgrant> Fixes the top OOPS.
<mwhudson> lifeless: wow, IFiniteSequence?
<mwhudson> but ok
<lifeless> mwhudson: yes, really - its what the z3batch code does
<mwhudson> lifeless: yes, still looks fine
<lifeless> mwhudson: so I assume there is an adapter registered for result sets somewhere or other
<thumper> is the branch scanner running on qastaging?
<thumper> or do we need to ask?
<wgrant> I'd ask.
<thumper> spm: qastaging branch scanner plz
<spm> thumper: it's not setup to run, is running manually atm
<thumper> spm: can you please run manually?
<spm> seems to be working even. /shock horr
<thumper> spm: I'll be asking again in a few minutes
<spm> or
<spm> oh. I didn;t type that clearly. "is running manually atm" means I've kicked off a manual run already
<spm> 2011-04-11 02:36:50 INFO    Adding 479 new revisions.
<spm> 2011-04-11 02:36:56 INFO    Deleting 0 branchrevision records.
<spm> 2011-04-11 02:36:56 INFO    Inserting 34782 branchrevision records.
<thumper> that isn't the one I wanted...
<thumper> is it done?
<spm> not yet
<thumper> how is qastaging codehosting set up?
<spm> branch scanner is "cronscripts/scan_branches.py" ?
<thumper> spm: aye
<thumper> spm: when you you get your new au losa?
<spm> thumper: not sure what you mean? it's setup and running on tellurium. lpqastaging.tellurium.canonical.com.
<wgrant> Less than a month until the new victims.
<thumper> spm: as in a completely separate hosting area?
<spm> I'm not sure when they start.
<wgrant> May 5?
<wgrant> Something like that.
<wgrant> spm: ShipIt's finished. Please remove the cronjobs.
<lifeless> thumper: yes
<lifeless> thumper: it has its own branches and copies the same set across on restores that stging does
<thumper> ok
<thumper> spm: please ping when scan complete
 * StevenK grumbles loudly at IDS.
<StevenK> If you pass in processor families to the package cloner, it will look up every SPPH for the destination distroseries and call .createMissingBuilds() on it.
<wgrant> Yes?
<spm> thumper: https://pastebin.canonical.com/45894/
<StevenK> Which is *slow*
<wgrant> Yes.
<StevenK> This makes me sad.
<wgrant> createMissingBuilds sucks, and needs to be fixed as the next phase for some timeouts.
<thumper> spm: very :((
<thumper> spm: that is because there was a scan job for a branch not copied to qastaging
<thumper> spm: and the oops config isn't set for the scanner on qastaging
<thumper> spm: if I said make all scan jobs as complete, could you do that without sql?
<StevenK> wgrant: It might be in-scope for us, because calling .createMissingBuilds() on 17,000 SPPHs makes mawson cry.
<wgrant> StevenK: Sure it's not the populate-archive bug?
<wgrant> StevenK: It may not be, but getPublishedSources now preloads huge amounts of useless stuff which makes any script using it on thousands of publications completely useless.
<StevenK> wgrant: lib/lp/soyuz/model/packagecloner.py, _create_missing_builds()
<StevenK> The package cloner suffers from Underscore Disease.
<wgrant> Yeah, p-a bug.
<wgrant> It's still slow, but not that slow.
<wgrant> Cowboy the patch from https://wiki.canonical.com/IncidentReports/2011-03-29-LP-populate-archive-slow
<wgrant> spm: Hi, I need a cronjob run on qas.
<wgrant> 'scripts/process-hwdb-submissions.py -vv', and I'd like the output.
<StevenK> wgrant: Because eager loading fixes everything. :-(
<wgrant> StevenK: Yes.
<lifeless> StevenK: its a key tool
<lifeless> the problem is that an object model assuming cheap lookups is broken in the real world.
<spm> thumper: "if I said make all scan jobs as complete, could you do that without sql?" Um. I dunno .but we can use sql on qas?
<thumper> spm: I'll get back to you shortly
<spm> np
<StevenK> lifeless: The problem is the eager loading breaks at least parts of Soyuz.
<StevenK> s/parts/two parts/
<spm> wgrant: do you expect lots of output, or just a little? ie log or screen gab
<lifeless> StevenK: thats the symptom
<wgrant> spm: I'm not entirely sure. Might be best to log.
<spm> okidoki
<lifeless> StevenK: the problem is that a single api was used when there are different usecases at hand.
<wgrant> spm: It depends how the DB was when qastaging was restored.
<wgrant> lifeless: The use-cases were identical.
<wgrant> lifeless: But now we have a policy of eager loading.
<wgrant> So they differ.
<lifeless> wgrant: they weren't really identical, the differentiation was around performance
<spm> wgrant: ftr: ~/tmp/wgrant-process-hwdb-submissions.log
<spm> ahh. cronscripts/process-hwdb-submissions.py would work better
<wgrant> spm: Er, yes, sorry.
<StevenK> lifeless: The only things that required the eager loading were the callsites that directly impacted on render times. .getPublishedSources() is called from other places, too.
<spm> wgrant: heh, is cool. it was third time lucky in any event. '/cripts/process...' didn't work either. :-)
<lifeless> StevenK: it required eager loading because it was making things slower for render times
<lifeless> StevenK: I agree that other places were negatively impacted
<spm> wgrant: argh. one sec. will fix. FATAL:  Ident authentication failed for user "hwdb-submission-processor"
<StevenK> Yes, hence my "directly impacted on render times"
 * spm pulls out the yak clippers
<lifeless> StevenK: the function I change directly impacted render times.
<StevenK> Perhaps we change the internal API of .getPublishedSources() to take a eagerload boolean, and it can return early if it's False (and it defaults to True)
<StevenK> It's not particularly elegant, but it solves both cases.
<lifeless> StevenK: sure, thats what we've done elsewhere.
<lifeless> StevenK: I didn't here because of oversight, not intent
<lifeless> StevenK: the scripts weren't visible to me when I grepped around quickly, for $whatever reason.
<spm> wgrant: https://pastebin.canonical.com/45895/
<wgrant> spm: Thanks.
 * StevenK looks for the p-a bug
<wgrant> spm: Do you want an RT for killing the shipit cronjobs? Or will the existing shipit cleanup one do?
<spm> the existing covers it
<StevenK> wgrant: Hm, does the p-a slowness even have a bug?
<wgrant> StevenK: Bug #744849
<_mup_> Bug #744849: populate-archive.py unusably slow <Launchpad itself:Triaged> < https://launchpad.net/bugs/744849 >
<StevenK> Bah, that didn't show up when I searched for populate-archive
<StevenK> wgrant: Thanks, I'll look at it after lunch.
<huwshimi> wallyworld: Did you have any luck with that border?
<wallyworld> huwshimi: yes. all good. i've improved the animations and also provided animation for when the user hits "no". all i need now is some css for the confirmation content and i can submit the mp
<huwshimi> wallyworld: Is all that pushed to your branch?
<wallyworld> huwshimi: yes
<huwshimi> wallyworld: Thanks I'll take a look
<wallyworld> huwshimi: the mp is also written but on hold so you can see that too if you want
<wallyworld> huwshimi: sorry, i wasn't aware you were waiting to hear back from me
<huwshimi> wallyworld: That's ok I just wanted to see how it looked with the bar added
<thumper> lifeless: https://code.launchpad.net/~thumper/launchpad/fix-unicode-error-in-urlparse/+merge/57103
<huwshimi> wallyworld: How about something like this: http://imgur.com/KVM6j
<wgrant> Do we want "Yes"/"No" rather than "Assign"/"Cancel"?
<lifeless> no
<lifeless> assign / do not assign
<wgrant> Right, or that.
<lifeless> huwshimi: ^
<wgrant> But definitely not "Yes"/"No",
<lifeless> huwshimi: gnome did a bunch of user testing
<lifeless> huwshimi: and yes/no prompts are very effective at confusing users
<wgrant> lifeless: $ACTION/"Cancel" is used everywhere else in LP, so I'm leaning towards that.
<lifeless> wgrant: the cancel is about as good as No
<wgrant> It is.
<lifeless> wgrant: it makes me take a second-glance all the time :(
<huwshimi> lifeless: wgrant: OK, but that's not what the screenshot was about at all.
<wgrant> Then there was that infamous HIG-compliant download manager a few years ago...
<lifeless> huwshimi: heh, what was it about ?
<lifeless> oh, I see
<thumper> lifeless: ta
<lifeless> wallyworld: ^ ^ ^ ^
<lifeless> thumper: de nada
<huwshimi> lifeless: I was just trying to improve some CSS
<lifeless> huwshimi: yeah, I'm caught up now
<huwshimi> lifeless: I think it's a fair call though
<lifeless> wgrant: http://projects.gnome.org/gwget/screenshots.html ?
<wgrant> lifeless: That's the one.
 * wallyworld just finished some food
<wgrant> But the screenshot is not there any more :(
<wgrant> https://bugs.launchpad.net/ubuntu/+source/gwget2/+bug/84215/+attachment/30855/+files/Screenshot-gwget.png
<_mup_> Bug #84215: Do you want to Cancel or Cancel? <gwget2 (Ubuntu):Fix Released> < https://launchpad.net/bugs/84215 >
<wallyworld> huwshimi: that screenshot looks pretty good
<lifeless> wgrant: ROTFL
<huwshimi> wallyworld: There was a small HTML change. I'll push my changes. Also did you see what lifeless and wgrant were just saying about the buttons?
<wallyworld> huwshimi: yes. so looks like i'll make it "Assign" / "Do Not Assign" ?
<wgrant> Sounds good.
<huwshimi> wallyworld: Actually in this case I'm not sure "Do Not Assign" makes sense. The action that it makes is "go back so I can fix it".
<lifeless> its a popup right ?
<huwshimi> lifeless: Yes.
<wallyworld> huwshimi: yes, you are right. so it should be something like "Choose Again" ?
<lifeless> wallyworld: sounds good
<huwshimi> wallyworld: Yeah something like that...
<wallyworld> huwshimi: thanks for the input btw
<huwshimi> wallyworld: np
<huwshimi> wallyworld: Here are my changes: https://code.launchpad.net/~huwshimi/launchpad/assign-non-contributor
 * wallyworld merges
<wallyworld> huwshimi: that css style is used elsewhere. i was just reusing an existing one. i'll make a new one with your style attributes
<huwshimi> wallyworld: Oh right
<huwshimi> wallyworld: Yeah, good plan :)
<wallyworld> :-)
<lifeless> mwhudson: https://bugs.launchpad.net/storm/+bug/239767
<_mup_> Bug #239767: Negative Indexing on results fails <Storm:New> < https://launchpad.net/bugs/239767 >
<lifeless> StevenK: I have updated my lazr.batchbavigator branch to fix test failures and use 1.2.4
<lifeless> StevenK: do you care to rereview?
<huwshimi> A review of this would be super great: https://code.launchpad.net/~huwshimi/launchpad/privacy-notification-firefox-753423/+merge/56872
<lifeless> jtv: hey, do you have write access on http://wiki.postgresql.org/wiki/Index_Maintenance ?
 * jtv tries
<jtv> Oh, that wiki?
<jtv> Err
<lifeless> jtv: if so the first query there is much more useful with 'ORDER BY pg_class.reltuples::bigint' rather than 'order by 2'
<jtv> If I could remember my login details, maybe.  :/
<jtv> Why would anyone want to order by 2?
<lifeless> that uses the second column
<lifeless> but its pretty printed
<jtv> Oh, yeah, unsurprisingly I wasn't aware of that feature.
<lifeless> so it orders 100MB before 2kb
<lifeless> jtv: old school sql ;P
<lifeless> jtv: none of these fancy column names
<jtv> Had to do so much of that before they ever let me touch a databaseâ¦ hundreds of select queries just written out on paper.
<jtv> Looks like the added the pg_size_pretty later and forgot to update the ORDER BY
<jtv> lifeless: you'll have better luck in #postgresql :)
<lifeless> yeah
<jtv> Where I also have some questions to ask, actually, but I've been too lazy or shy to do it.
<jtv> Such as: why does the background writer work out of the buffer cache instead of out of the WAL (and so, why do we need dirty bits for committed pages)?
<jtv> Sure, making it eat from the WAL would make it take longer for pages to get into tables, but so what?  Changes are already logged.  But I'm guessing that flushing could be more efficient if it weren't driven by eviction policy.
<lifeless> StevenK: hi
<StevenK> Damn it, I knew talking would get me on the hook for a review. :-P
<StevenK> lifeless: Link me
<lifeless> https://code.launchpad.net/~lifeless/launchpad/bug-752153/+merge/56505
<StevenK> lifeless: Which revisions are new, and can you provide a partial diff?
<lifeless> 12760->
<lifeless> isn't lp meant to do incremental diffs
<wgrant> lifeless: yes, but it's disabled because they're slightly broken.
<wgrant> IIRC they had excessive mode changes.
<lifeless> StevenK: http://bazaar.launchpad.net/~lifeless/launchpad/bug-752153/revision/12762?remember=12759&compare_revid=12759
<wgrant> But it was enabled for a few days, so it's disablement probably counts as a regression.
<StevenK> I wonder how I can hide an argument from the API
<lifeless> StevenK: give it a default value
<lifeless> StevenK: and don't list it in the export parameters list
<wgrant> StevenK: Which arg? For getPublishedSources?
<StevenK> Right, so I'm fine.
<StevenK> wgrant: Yup
<wgrant> StevenK: As lifeless says, make sure it has a default and don't list it in @operation_paramters
<wgrant> +e
<StevenK> wgrant, lifeless: http://pastebin.ubuntu.com/592452/
<StevenK> lifeless: You're ripping out a lot of code from lp.app.browser.root, can you explain a little context?
<lifeless> StevenK: that was the copied code that was fugly
<lifeless> StevenK: since I had to touch lazr.batchnavigator more, I refactored to let it be deleted.
<lifeless> see the comment with the pylint supressor
<StevenK> Right
<StevenK> Why self.default_size = 20 and then right after 'return 20' ?
<lifeless> StevenK: the function needs to set both
<lifeless> StevenK: thats explained in the docstring in the base class
<StevenK> lifeless: But if you return self.default_size, it only needs to be changed in one place?
<lifeless> StevenK: 1 line apart, and this is shorter.
<wgrant> wallyworld: Could you use an API call instead of BugContributorView?
<StevenK> Okay, it makes me grumble, but it looks good.
<lifeless> StevenK: thanks
<lifeless> next up, to teach lazr.restful to allow custom IRangeFactory
<wallyworld> wgrant: i considered that. i went with the view because the result of the call was related to getting data for display, a json dict of values
<wgrant> wallyworld: Don't you already have the person and pillar names?
<lifeless> that should so totally be an API, but lazr.restful doth not make it easy.
<wgrant> lifeless: Oh?
<wallyworld> wgrant: nope. not in the view
<wallyworld> lifeless: yes. hence the view :-)
<lifeless> wgrant: it wants to return typed objects only
<wgrant> lifeless: It'll happily return an untyped dict.
<wgrant> Soyuz does it in a few places.
<wgrant> It's not pretty, but it works fine.
<lifeless> wgrant: oh? I just recall jml headbutting making that work here and on the list in the past
<wgrant> wallyworld: The picker must already have the person displayname, and surely the task table knows the pillar displayname somewhere?
<lifeless> wgrant: whats the necessary dance ?
<wgrant> lifeless: Right, you can't return a dict of entries.
<wgrant> lifeless: They'll just end up as strings instead.
<wallyworld> wgrant: that's internal to the picker. the user of the picker only has the person uri
<wgrant> But if you leave the return type undeclared and just return a dict, the client will get it and just JSON decode it.
<lifeless> wgrant: cool
<wgrant> wallyworld: Hmm, that's pretty terrible. Not easily fixable to return the whole object? I imagine that could be generally useful.
<wgrant> Anyway, if you really can't refactor the machinery to let you get at the existing cached objects, this can easily be an API namedop, and that'd probably be better than manual JSON encoding and content-type hackery.
<wallyworld> wgrant: we don't have the whole person object. the picker has a vocab with terms and that's it
<wgrant> :(
<wallyworld> wgrant: my first implementation was via a named op - had to modify client.js slightly to allow sync calls - but i had trouble with lazr restful
<wallyworld> wgrant: can you poin tme to the soyez example?
<wgrant> wallyworld: Synchronous calls are never OK.
<wgrant> They block the browser UI in !Chrome.
<lifeless> jtv: Can I sanity check something with you
 * jtv seems an odd choice for sanity
<jtv> but yes, you can try
<lifeless> jtv: did you see my sketched fact table for bugtags
<wallyworld> wgrant: i need the validation call lthe block otherwise it all falls into a heap
<jtv> No
<wgrant> wallyworld: :/ We really really really want to avoid using sync anywhere.
<lifeless> jtv: ah. So, I've sketched a fact table out for bugtags, maintained with triggers I expect, and I can't get it out of my head.
<jtv> lifeless: "fact table"?
<lifeless> jtv: http://en.wikipedia.org/wiki/Fact_table
<jtv> thx
<lifeless> jtv: the centre table in a star schema
<jtv> lifeless: are you saying you've been looking at the same problems I wrote about yesterday?
<wallyworld> wgrant: hmmm. from memory that will make this change very hard to deliver without really fucking with the picker internals :-( i'll have a look
<lifeless> jtv: all week
<lifeless> jtv: as has wgrant
<jtv> ah! then sorry for the duplication
<jtv> or noise, or whatever
<lifeless> jtv: more eyeballs good
<jtv> yeah, I could do with some better ones
<lifeless> jtv: with timeouts, I do a sweep each day and make sure they are reported on  https://bugs.launchpad.net/launchpad-project/+bugs?field.tag=timeout
<lifeless> jtv: with the pageid from the oops summary in the bug title
<wgrant> wallyworld: It's particularly cool in Firefox, since the XUL runs in the same thread as page JS.
<lifeless> jtv: this particular one had not had the analysis updated in the bug :( - so looking there wouldn't have helped you know that folk were caring
<wgrant> wallyworld: So a synchronous call will actually freeze the browser.
<lifeless> jtv: but I like that you analysed it - good stuff
<wallyworld> wgrant: well that blows.
<jtv> lifeless: well I knew folks would care, but I thought maybe this was the outcome of a recent round of work.  And since it was a holiday, I didn't bother looking very far.  :)
<jtv> Anyway, about that fact table..?
<wgrant> wallyworld: Although it's possibly better now, and should be even better in 5.
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/getpublishedsources-too-eager/+merge/57107
<lifeless> jtv: anyhow, my question is, if one has an aggregate cached in a lookaside table, - which my volume column is, is it concurrency safe in postgresql (with repeated reads, which is I think the level we run at) to maintain that via triggers
<wallyworld> wgrant: seems like a hell of a lot of work for the sake of removing one minor sync call
<StevenK> lifeless: Can haz dropped timeout?
<lifeless> e.g. a trigger that on inserts/deletes into BugTag updates the running total for that tag in the fact table
<jtv> lifeless: you don't get "repeated read" as such in postgres
<lifeless> or will, if two people insert a reference to the same bug tag at the same time, the counter only go up by one, when it should go up by two
<lifeless> StevenK: no
<jtv> lifeless: you get "read committed" as the only ansi-standard isolation level
<lifeless> StevenK: we've got search timeouts happening
<lifeless> StevenK: e.g. https://bugs.launchpad.net/bugs/+bugs?field.searchtext=Ubuntu&search=Search+Bug+Reports&field.scope=all&field.scope.target=
<wgrant> wallyworld: Isn't the picker infrastructure already highly asynchronous? What's wrong with chaining another callback?
<lifeless> StevenK: and the tags portlet is right on the edge of going boom
<lifeless> StevenK: given its memcache cached, if it stops working, the timeouts won't increase a little...
<lifeless> StevenK: they will increase massively.
<lifeless> StevenK: so, we should get that fix from wgrant in place; then we can lower the timeout
<wgrant> Yeah, I'll have that finished tonight.
<jtv> lifeless: if you use read-committed for this, I think you'll lock the aggregate (which doesn't need to be problematic if you update very late).  In serializable, you may avoid that but take the occasional serialization failure.
<lifeless> jtv: right, so s/repeated read/whatever level we run at/
<jtv> serializable, I think.
<jtv> Hmm this has gone through changesâdon't trust me on this.
<wallyworld> wgrant: possibly nothing. my knowledge of the picker code is still not perfect and i couldn;t see how to easily do it. i'll take another look. when i last tried, the default save action happended in parallel with the validation
<lifeless> jtv: yah, and our retry-machinery catches serialiable failures and retries
<lifeless> jtv: I'll cross reference with stub
<jtv> lifeless: there may be call for a separate bit of infrastructure thereâ¦
<jtv> "I'm done, but if you could please add these numbers to these cached aggregates I'd be most grateful."
<lifeless> jtv: the big point though is that this can answer the portlet for anonymous users with only 10 rows read, and for logged in users with 10 + 10 * group membership [if the planner is real smart]
<jtv> If we can accept a tiny inconsistency window, the infrastructure could do it in a separate transactionâand only take the serialization errors there, without re-doing the request.
<lifeless> jtv: we could, and I'm happy for us to have that, but I think we can tolerate the few retries with existing infrastructure for now - shorter path to getting the primary win
<jtv> lifeless: some of these issues are very similar to what we had with translations.  One thing we did there was to update aggregates right away, but also have a background process check for and fix any discrepancies.
<wallyworld> wgrant: if you are interested https://code.launchpad.net/~wallyworld/launchpad/multicheckboxwidget-unescaped-items/+merge/57108
<wgrant> wallyworld: Will the item always have a displayname? I guess if not then the first callsite can fix it properly.
<wgrant> As long as it breaks instead of being insecure.
<wallyworld> wgrant: most if not all things we care about do afaik
<wgrant> StevenK: How many callsites actually need the eager loading?
<wgrant> StevenK: It should probably be off by default, I think.
<StevenK> wgrant: I was being cautious
<wgrant> lifeless: Could you mentor https://code.launchpad.net/~wallyworld/launchpad/multicheckboxwidget-unescaped-items/+merge/57108?
<wallyworld> wgrant: the sort of things we display in pickers etc all have displaynames so it's a pretty safe assumption
<wgrant> StevenK: Why does that direction deserve caution?
<wgrant> wallyworld: Yup.
<StevenK> wgrant: And if it's off by default, then the API doesn't get it.
<wgrant> StevenK: @call_with(eager_load=True)
 * wallyworld goes back to hacking picker javascript
<StevenK> wgrant: Looking at other callsites.
<wgrant> StevenK: Thanks. It seems to mostly be tests and a few things that use it to check if something is published, none of which need preloading.
<jtv> lifeless: I don't see any concurrency hazards with what you're doing, at any isolation level, but you're introducing a write lock on the row in the fact table.
<StevenK> wgrant: Er, then why was it added? Surely at least one of the callsites needs itl.
<wgrant> StevenK: The API and probably one or two UI callsites.
<wgrant> StevenK: Check the rev that added it.
<wgrant> It'll reference a bug.
<lifeless> jtv: yeah, understood
<lifeless> jtv: I think its simplest to start with triggers and move to async IFF the contention is a problem.
<lifeless> jtv: other than the bug import scripts, I don't see it being an issue - bug mail being async already
<lifeless> IMBW
<jtv> lifeless: that makes sense.  As long as we know that we'll recognize that if it happens.
<lifeless> $when the sky falls in, lifeless broke it
<wgrant> lifeless: Does http://paste.ubuntu.com/591153/ still perform adequately? It's 9s on DF.
<StevenK> wgrant: The linked bug only references the API.
<lifeless>  Total runtime: 36958.990 ms
<lifeless>  Total runtime: 3903.161 ms
<lifeless>  Total runtime: 3803.285 ms
<lifeless> wgrant: ^
<wgrant> lifeless: Hmm.
<wgrant> StevenK: Must only be the API, then.
<lifeless> wgrant: we can get spm to test on prod if you like
<lifeless> wgrant: staging is mid-restore so not exactly at its best
<wgrant> Not really. Need to get a realistic query first.
<lifeless> wgrant: do you mean 'authenticated'
<lifeless> wgrant: or that you want to do better ?
<wgrant> lifeless: And ideally with official tags in the same query.
<StevenK> wgrant: Changes pushing, I'll prod you when the diff is updated.
<lifeless> kk
<lifeless> wgrant: coalesce sort ?
<wgrant> lifeless: Possibly. I'm trying a UNION'd WITH first.
<wgrant> see how it works... probably not well.
<wgrant> But obvious solutions first.
<StevenK> wgrant: Diff updated.
<wgrant> StevenK: thanks, will look in a sec.
<lifeless> wgrant: I can't see a comment from you on https://code.launchpad.net/~wallyworld/launchpad/assign-non-contributor/+merge/55869
<wallyworld> lifeless: i think that's the wrong one
<wallyworld> lifeless: https://code.launchpad.net/~wallyworld/launchpad/multicheckboxwidget-unescaped-items/+merge/57108
<lifeless> wallyworld: sshh I was being crafty
<wgrant> lifeless: http://paste.ubuntu.com/592461/
<wgrant> lifeless: Could you try that on (qa)s?
<lifeless>  Total runtime: 4327.931 ms
<lifeless>  Total runtime: 3828.393 ms
<wgrant> Sounds OK, then.
<lifeless> cold, hot
<wgrant> Separate queries are far worse.
<lifeless> ah, because official tags don't have privacy
<wgrant> Hm?
<wgrant> This doesn't have privacy yet.
<lifeless> yeah it does
<lifeless> the private check
<wgrant> Well, yes, but it doesn't do authenticated users.
<lifeless> sure
<lifeless> anyhow
<wgrant> The officialbugtag stuff has nothing to do with privacy.
<lifeless> shipit
<wgrant> Don't ever say that word again.
<lifeless> shiiiiiiiiiiiiiiiiiiiiiiiiiiiipit
<jtv> wgrant: did you see my email about the bugtag count query?  I had it at 2s or so.
<lifeless> jtv: what db server?
<jtv> staging
<jtv> After that it gets harder, unfortunately.
<wgrant> lifeless: Could you mentor https://code.launchpad.net/~stevenk/launchpad/getpublishedsources-too-eager/+merge/57107?
<wgrant> jtv: Hmm, what improvements did you make over lifeless'?
<jtv> wgrant: I haven't compared.  Unaware that he was working on it, I played around with it a bit yesterday.
<lifeless> wgrant: no, I approved it already
<wgrant> lifeless: Oh, so you did.
<StevenK> wgrant: Thanks, ec2'ing
<jtv> wgrant: ah, I see that he followed up to my email.  One of my optimizations was to use inner joins instead of outer joins.  This saved a second on staging, but he says it won't help as much on "beefier hardware."
<jtv> Not sure why that would be.
<wgrant> The 3.5s query already uses inner joins :(
<lifeless> jtv: first thing we did :)
<lifeless> jtv: I had thought you were testing on dogfood
<jtv> Did a bit of that as well, tbh.
<lifeless> jtv: I've found that the left join/inner join plan cost vs actual cost difference is ~0 on prod
<jtv> Why would that be?
<lifeless> didn't dig terribly deeply sorry
<lifeless> except when  the plan is dramatically different (and it is sometimes)
<jtv> Seems strange though that this query would work out faster on staging than on production.
<lifeless> jtv: I've been testing on staging too
<lifeless> jtv: for added weird value
<lifeless> jtv: I don't have direct prod access
<lifeless> jtv: prod has different tuning parameters of course, vs staging - at 4 times the memory :)
<jtv> Oh.  Used to be the same.
<jtv> That'll make it hard to tune effectively for production.  I'd really like that plan logging feature for timeouts.
<jtv> Or oopses in general, perhaps.
<lifeless> jtv: same
<jtv> So we can use ++oops++ to check for flukes.
<jtv> Can't we ask one of the maintenance squads to do it?
<jtv> Or the TA?
<lifeless> yes, it does make predicting a little harder; I tend to have a losa check once I'm happy on staging
<lifeless> jtv: its a matter of triage
<lifeless> right now, the biggest analytic problem is that api oopses have no timeline
<lifeless> a regression
<poolie> o/ jtv
<lifeless> whereartthoustub
<lifeless> arrrrgh
<wgrant> Oh?
<lifeless> https://bugs.launchpad.net/launchpad/+bug/739051
<_mup_> Bug #739051: Product:+index timeouts <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/739051 >
<henninge> good morning
<henninge> And good day to you guys over there on the other side ... ;-)
<lifeless> hi henninge
<lifeless> wgrant: see the query.
<wgrant> Morning henninge.
<henninge> Can I ask for a review, please?
<henninge> https://code.launchpad.net/~henninge/launchpad/devel-744204-escaping-soyuz-base
<wgrant> henninge: Looking.
<henninge> javascript, quite simple
<henninge> oh, that was the branch url. nm
<henninge> https://code.launchpad.net/~henninge/launchpad/devel-744204-escaping-soyuz-base/+merge/56978
<jtv> hi poolie
<poolie> hi
<wgrant> henninge: Done.
<wgrant> lifeless: Still around for mentoring?
<lifeless> sure
<henninge> wgrant: thanks!
<wgrant> lifeless: henninge's MP just above, if you could.
<henninge> wgrant: seriously? I can have whitespaces before dots?
<henninge> also, I did not realized the YUI methods return the node. Thanks.
<lifeless> henninge: you can in python too
<wgrant> henninge: Yup.
<lifeless> henninge: e.h. '[] . reverse()'
<henninge> oh!
<wgrant> . can't begin an identifier, so it's OK.
<henninge> cool, learn something new every day ;)
<henninge> so '.' really is more like a binary operator
<spiv> lifeless: or even '[ ] . reverse ( )' for maximum ugly!
<lifeless> spiv: you forgot the \r\n
<rvba> wgrant: Hi! I guess you marking my bugs as qa-ok is due to the fact that I'm blocking the rollout and you know my developments are protected by the feature flag anyway ... right?
<StevenK> Except that r12786 is marked qa-basd
<StevenK> *qa-bad
<jtv> hey rvbaâisn't it very early for you?
<rvba> jtv: not really, 8:46
<rvba> StevenK: indeed
<rvba> just saw that
<wgrant> rvba: I checked that they didn't break anything on DF.
<wgrant> rvba: As for the qa-bad one, the fix should be on qastaging right around now.
<wgrant> I guess it's being slow because of the staging restore :/
<rvba> wgrant: appart from the one that you marked qa-bad ... I was in the process of trying to qa them on DF myself ... but I needed to upload a few bogus packages to properly test them.
 * wallyworld off to pick up new puppy, back later
<LPCIBot> Project windmill build #161: FAILURE in 1 hr 6 min: https://lpci.wedontsleep.org/job/windmill/161/
<wallyworld> wgrant: the assign-non-contrib mp now uses an api call. now i just need to sort of the async stuff
<poolie> wallyworld, i liked your video
<wgrant> wallyworld: Great!
<wallyworld> poolie: yeah, it was a learning exercise for me. i'm still new to yui
<poolie> wallyworld, when it checks people who already have bugs assigned, that includes closed bugs?
<wallyworld> wgrant: i basically just reverted to an earlier version and andded some tests
<rvba> wgrant: is a fix for the portlet display itself or did you add a protection using the feature flag?
<wallyworld> added
<wgrant> wallyworld: lazr.restful didn't stab you?
<wgrant> rvba: It's behind the feature flag for now.
<wgrant> rvba: It's not clear what fixing the portlet display entails.
<wallyworld> wgrant: nope. i thought it would. when i say reverted, i also added some extra stuff to include the person and pillar names.
<rvba> wgrant: ok
<wallyworld> so it's all good
<wgrant> wallyworld: Ah, right.
<wallyworld> wgrant: previously the version with the api cal ljust returned true/false without all the other detail
<wgrant> lifeless: How can I select columns from a With in Storm?
<wgrant> lifeless: Everywhere else seems to use the With in a condition :(
<lifeless> uhm
<lifeless> I think I did this in my latest with patch
<wgrant> I'm not sure it's possible :(
<lifeless> or perhaps its what gustavo is asking for
<lifeless> BranchRevision does it
<wgrant> lifeless: Hmm. Maybe I'll have to define a fake class.
<wgrant> It looks like it :(
<lifeless> wgrant: this is bugtag?
<henninge> Hm, were did the inline diffs in comments on MPs go?
<lifeless> wgrant: just select columns
<lifeless> wgrant: Alias(SQL("counts.foo"), "bar")
<lifeless> wgrant: or some such
<wgrant> lifeless: Can you do that in a findspec?
<lifeless> sure
<StevenK> lifeless: I also came to a conclusion over the weekend. You pushed back at me a little about making request-daily-builds a garbo job -- given the fun I've had with p-s-c, I'm glad I didn't.
<lifeless> StevenK: So the garbo now parallelises
<lifeless> StevenK: And will be getting more work put into it
<StevenK> lifeless: It still gets blocked for 5.5 hours per day.
<lifeless> StevenK: thats tunableloop that does it, and its configurable per loop
<lifeless> StevenK: all the tunable loops should get a timeslice every hour, guaranteed
<lifeless> if they aren't, its a but
<lifeless> bug*
<StevenK> lifeless: But they get blocked due to slony madness?
<lifeless> StevenK: they get blocked by a policy knob
<lifeless> StevenK: which says 'if this loop will write a lot of rows, running it during e.g. backup will permanently bloat the table and makeit run slowly'
<StevenK> lifeless: The logs disagree, but I hestitate to paste them.
<lifeless> StevenK: loops like psc should just tune that
<lifeless> StevenK: file a bug then
<lifeless> StevenK: we need to have a solid platform to do things with easily.
<lifeless> jtv: does serializable ever block on reads ?
<jtv> lifeless: not that I can think ofâactually the locking in serializable is the same as in read-committed by the way.
<jtv> There's a detailed listing of lock types in the pg documentation.
<lifeless> k
<lifeless> jtv: I'm looking at https://bugs.launchpad.net/launchpad/+bug/739051
<_mup_> Bug #739051: Product:+index timeouts <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/739051 >
 * jtv looks
<lifeless> jtv: which has an 11 second query, that takes 700ms on all three dbs in prod + similar on staging
<lifeless> though its gross that we're doing a fti search
<jtv> so what makes it an 11-second query?
<lifeless> + a %foo%
<lifeless> jtv: thats the million dollar question
 * jtv shakes fist at openid
<jtv> a 404 and a search form, just the things I expected from following a pair of automatically-generated oops links
<jtv> lifeless: wild guessâ¦ fast-changing table in there?  Stats go stale occasionally, query reverts to conservative plan with lots of seq scans?
<lifeless> 4 of the 5 timeouts are on the query
<lifeless> the last is high cpu
<lifeless> 22839 hits on the page
<jtv> It's reported as high CPU, but could this be one of those extreme db timeouts where the query isn't logged and so the time isn't counted as query time?
<lifeless> it may be an overloaded appserver
<lifeless> jtv: we don't have any of those
<jtv> Used to.
<lifeless> jtv: partial queries are reported properly now
<lifeless> jtv: https://lp-oops.canonical.com/oops.py/?oopsid=1926H1059#statementlog is the cpu one
<lifeless> jtv: you can see the gap
<lifeless> I think this was a crashing appserver
<lifeless> B E G H
<lifeless> all 4-threaded ones
<jtv> Yup, I see the gap
<lifeless> accounting for all the timeouts
<lifeless> I think the slow queries are the python code within the 'run the query' getting time starved
<jtv> That'd explain why the time seems to go arbitrarily into DB or non-DB time.
<jtv> Makes me wonder though: why relatively consistent delays and why specifically in this one place?
<wgrant> We see that on BugTask:+index a lot.
<adeuring> good morning
<jtv> hi adeuring
<adeuring> hi jtv!
<lifeless> I will ping mthaddon about progress on removing those servers
<jtv> Thanks for pushing through that long-desired change btw.
<lifeless> jtv: we had a bunch of servers on wampee crash out earlier today
<lifeless> jtv: rss up at 1GB, nonresponsive
<lifeless> jtv: my pleasure, like all things takes time
<wgrant> Deploy time, yay.
<lifeless> we can't rule out actual plan trouble
<lifeless> but the 22K successful renders
<lifeless> make me doubt that that its anything other than a sick appserver
<lifeless> so what we need to figure out is how to detect and eviscerate such appservers in advance (assuming the 2-thread config can still run into trouble)
<jtv> Well like I said, it's possible for a query to be planned differently while stats are out of date.  Could be a canary for vacuums falling behind.  (I think vacuum is where the stats are updated).
<StevenK> wgrant: Err, in r12792 you merged in the FF stuff -- however, one of them says tal:condition="features/soyuz.derived-series-ui.enabled" and the other has tal:condition="request/features/soyuz.derived-series-ui.enabled"
<lifeless> all palladium/potassium/gandwana - the old appservers
<wgrant> StevenK: 'features' is defined in base-layout-macros
<wgrant> StevenK: So you can only use 'features' directly in a top-level template.
<jtv> wgrant, StevenK: question about publish-distroâ¦ when publish-ftpmaster wants to process only security updates, it uses the script's -s option to indicate the security suites.  But not on the partner archive!  Doesn't partner have security suites?
<lifeless> stub: hi
<stub> yo
<lifeless> stub: 10 /    1  MaloneApplication:+bugs
<lifeless> stub: I think we may have index bloat again already :(
<stub> Bah. I'll check my shiny new report.
<lifeless> stub: how are you checking for it ?
<lifeless> stub: ooh is it online ?
<stub> I'm about to look :-) Should be
<stub> For the master at least. I need to set it up for the slaves.
<lifeless> stub: are there instructions for regenerating indices for the losas ? [
<lifeless> https://lp-oops.canonical.com/oops.py/?oopsid=1926D1311#longstatements
<stub> No. Its something I'll need to do manually. Not sure if I can automate it - parts have to be done outside transactions so needs someone who can workout how to back things out if things go screwy.
<wgrant> jtv: Partner only has the release pocket.
<lifeless> stub: if you can document it, then more folk could step through if needed, even if you're on leave / sick
<lifeless> stub: which is my main concern - higher bus factor
<jtv> wgrant: so security updates in partner go into the release pocket?
<wgrant> jtv: Yes.
<jtv> stub: he means tuk-tuk factor (he doesn't speak Thai)
<stub> Yes, but how do I document how to back out from an unknown state due to an unforeseen screwup?
<lifeless> stub: you don't
<lifeless> stub: but you can document the risks you know about, the things that would make you go hmmm
<jtv> wgrant: thanksâ¦ and once we start releasing the debug archive, that will have security suites again right?
<stub> Right. So I need to do it, or someone else who understands enough to do it such as our 3rd party support.
<wgrant> jtv: Yes, but they're not security-critical.
<lifeless> stub: sure; I mean the way I'd approach it is build a new index concurrently with a different name; then drop the old index
<jtv> wgrant: so the criterion for expediting security updates would be "everything in the security pocket on primary, everything at all in partner, and none in debug."
<lifeless> which would require a single short lock on the table
<jtv> wgrant: a suite wasâ¦ <archive>-<pocket>?
<wgrant> jtv: Probably nothing at all in partner, but either way.
<wgrant> jtv: suite = (distroseries, pocket)
<stub> Right. So I guess the failure there is the index not building, which can be documented how to recover.
<wgrant> But in practise it's really (archive, distroseries, pocket)
<stub> Of course, with the right reports we never need that process because we catch the problems proactively.
<lifeless> stub: right
 * stub tries to work out where the dbr is being generated from atm
<lifeless> ~lpqateam
<lifeless> crontab
<jtv> wgrant: thanksâ¦ this is turning out to be a nasty piece of hard-coding.
<jtv> wgrant: it would be a lot simpler if I could leave partner out of the expedited security updates.  So if you say I can, I will.
<stub> losaping: Can you please update the Launchpad tree on devpad for the ~lpqateam?
<stub> ECHANNEL
<lifeless> stub: you can sudo to lpqateam I think
<lifeless> stub: if you can't, I can
<wgrant> jtv: Please do.
<jtv> \o/
<jtv> wgrant: I had another questionâ¦ is the distscopy version of dists ever used for anything at all besides this script?  ISTM we could rsync the current dists to it _before_ moving it to a new place.  Just to narrow down the horrible-awkward-directory-shuffle-state window for failures a little.
<StevenK> wgrant: Have you filed a bug about that issue?
* henninge changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: henninge | https://code.launchpad.net/launchpad-project/+activereviews
<wgrant> jtv: I don't know of anything else that uses it.
<wgrant> StevenK: I have a half filled form... does that count?
<StevenK> Haha
<wgrant> Bug #757248
<_mup_> Bug #757248: poppy-sftp's signature checking relies on long-term survival of a directory in /tmp <poppy> <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/757248 >
<StevenK> wgrant: Excellent, thanks
<jtv> wgrant: while we're thinking about thisâ¦ if nothing uses the distscopy version of dists, why the à¸à¸±à¸ don't we run publish-distro on it in-place and simplify the directory shuffle in the publish-ftpmaster script?
 * jtv starts diagram to illustrate
<wgrant> jtv: No idea :/
<wgrant> Probably roughly "because someone thought that was a good idea 2005"
<wgrant> in 2005.
<jtv> wgrant: current â http://people.canonical.com/~jtv/publish-ftparchive-dists.png
<wgrant> bigjools: Morning.
<jtv> wgrant: proposed â http://people.canonical.com/~jtv/simplified-publish-ftparchive-dists.png
<jtv> hi bigjools
<bigjools> hello
<wgrant> bigjools: Did you know that cocoplum's /tmp pruner considers poppy-sftp's gpg.conf to be a nice snack?
<bigjools> wgrant: wtf is gpg.conf doing there?
<wgrant> bigjools: GPGHandler creates a GNUPGHOME in /tmp, with the configured keyserver.
<wgrant> bigjools: So it gets pruned, and poppy-sftp can no longer see any keys!
<wgrant> Fun for all.
<bigjools> GNUPGHOME should be set to the right place, I remember telling a losa to do that
<wgrant> Hmm?
<wgrant> GPGHandler overrides GNUPGHOME when it's initialised.
<wgrant> Setting it in poppy-sftp's env will do no good.
<wgrant> It will be clobbered.
<bigjools> that's cracj
<maxb> no it's quite sane really - encapsulate the needed setup in the code rather than mandating external setup
<wgrant> It's just not very helpful for daemons.
<bigjools> it's not mandating anything, it should be an override
<bigjools> so it's rather insane in fact
<bigjools> wgrant: did you sort it out?
<wgrant> bigjools: We've restored gpg.conf for now.
<wgrant> bigjools: It will happen again, though.
<wgrant> This week some time.
<bigjools> so it needs a code change I suppose
<wgrant> Yeah. I'm not sure what should be done.
<bigjools> we need to stop gpghandler using /tmp
<bigjools> simples
<jml> hello from millbank
<jml> product team is here / huw
<lifeless> \o/
<bigjools> jtv: did you work out what distscopyroot is for?
<bigjools> and did wgrant see those diagrams?
<jtv> bigjools: as far as I can make out from the script, all it does is keep the "alternate copy" of dists.
<jtv> Yes, I showed him the pictures.
<jtv> He can't think of anything else that distscopy might be used for.
<bigjools> jtv: so we have three copies of the indexes...
<jtv> Two.  But they take on 4 different paths.
<jtv> 3 each.
<jtv> They alternate.
<bigjools> so distscopyroot contains the indexes from the previous run
<jtv> So through one script run, each of these directories is renamed 4 times to assume each of those 4 paths, and they end up back where they started.
<jtv> Yes.
<bigjools> and we rsync to it at the start of each run
<jtv> Yes.
<jtv> It's an Ouroboros.
<bigjools> can't see a problem with the new method
<bigjools> bless you
<jtv> I wasn't sneezing, I was referring to the mythical snake that eats its own tail.
<jtv> It's also reminiscent of Heinlein's short story, All You Zombies.
<bigjools> jtv: the only difference now is that distscopyroot can get overridden before a successful run
<jtv> overridden?
<bigjools> with the old method it was only changed when the script completed
<bigjools> successfully
<jtv> Or unsuccessfully.
<bigjools> really?
<jtv> If it aborts prematurely, dists.new is moved back there.
<bigjools> interesting
<bigjools> I wonder if that was a bug!
<bigjools> I can't see any other reason to have dists.new
<wgrant> It's not really a bug.
<wgrant> The second copy is only a time saver.
<wgrant> It doesn't matter if it is broken.
<jtv> The rsync should take care of any broken crap in there.
<wgrant> Worst case the rsync will take a bit longer next time.
<bigjools> it depends on why someone wanted a copy of the indexes from the last run
<wgrant> bigjools: Because dists isn't regenerated from scratch each time, we need to work in a copy of it.
<wgrant> Because d-i images are big, we can't recopy each time.
<bigjools> I know. ..
<wgrant> So a second copy is kept, and brought up to date each time.
<bigjools> but the way it's done is weird
<wgrant> Oh?
<bigjools> jtv: I think your change is ok, FWIW. It doesn't seem to do anything differently
<bigjools> wgrant: having a backup the previous run's indexes may have been desirable
<bigjools> backup of*
<wgrant> bigjools: It has a special codepath for moving it back before publication has finished, so it was intended that it may not always be a valid copy of the indices.
<jtv> Hmm
<bigjools> wgrant: my point was, that could have been a bug
<bigjools> despite the intentions
<wgrant> It seems fairly deliberately.
<wgrant> Mm.
<wgrant> Maybe.
<bigjools> who knows, that script is a pile of crap
<bigjools> but it's been like it for many years so JFDI
<wgrant> Let's hope we can soon say it *was* a pile of crap.
<jtv> I wonder if this doesn't mean that we can improve turnaround time of security updates further by doing the rsync at the end of the previous run.  That way, its latency is out of the critical path for getting the fixes out.
<wgrant> jtv: No.
<wgrant> jtv: We don't want to push that frequently.
<wgrant> It's unclear whether expedited security processing is desirable in the first place.
<wgrant> THis predates s-i-s.
<jtv> It's in the spec.
<jtv> s-i-s?
<wgrant> security-in-soyuz
<wgrant> Back when -security was synced from a dak instance.
<bigjools> wgrant: this was done *after* s-i-s
<bigjools> and yes it's desirable
<jtv> And even if it weren't the spec is newer yet and specifies expedited security updates.
<bigjools> the spec's pretty old :)
<jtv> Well, "less ancient yet"
<bigjools> I wrote it with Celso about 2.5 years ago
<jtv> Anyway, I'll just focus on the simpler structure then.
<bigjools> cool, thanks for delving into that jtv
 * jtv loves to see complexity melt away
<jtv> This means no more broad cleanup in a "finally" or "atexit."
<bigjools> \o/
<jtv> bigjools: what in publish-ftpmaster exactly produces the Sources.* files?  Is it a-f, or publish-distro, or something else?
<bigjools> jtv: a-f for ubuntu, publish-distro for PPAs
<jtv> So a-f in this case.  Thanks.  Almost there with the cleanup, I hope.
<bigjools> woohoo
<jtv> Oh grumble.  What is it that runs a-f?
<bigjools> publish-distro
<jtv> So my question was wrong.
<bigjools> heh
<jtv> Curses.  Something in publish-distro doesn't seem to like the new location.
<bigjools> jtv: using -R ?
<jtv> Yes.
<bigjools> odd
<jtv> It's not generating the Sources files, far as I can tell.  But no hint of why.
<wgrant> jtv: Checked apt.conf?
<wgrant> It's generated in ftparchive.py, and is a bit bad.
<jtv> Noâ¦ where would I find it though?
<bigjools> wgrant: need to bounce an idea off you
<wgrant> bigjools: Sure.
<bigjools> wgrant: some of the derived distros need to work pretty much like PPAs
<bigjools> I think this can be as easy as changing the sources.list sent to the builders
<wgrant> Yeah.
<bigjools> if we have an archive flag, "overlay" or somesuch
<bigjools> then the initialisation phase will just include the small subset of packages needed
<bigjools> so it sounds sane?
<wgrant> Just change the archive dependency calculator to include the extra archive, and sources.list and retry-depwait will magically work.
<bigjools> yup
<jtv> Ohhh this is infuriating.  publish-distro fails to produce Sources.* in foo-distscopy/dists but does work if I mv foo-distscopy/dists dists.new ; publish-distro ; mv dists.new foo-distscopy/dists
<wgrant> jtv: a-f may require it to be within archiveroot.
<wgrant> Or our config of a-f.
<jtv> It's in archiveroot, I think, just not within distsroot.
<wgrant> 'foo-distscopy' is a sibling of the 'foo' archiveroot, right?
 * jtv checks
<jtv> Err yes it is
<jtv> Damn ".." in pathsâ¦
<jtv> Using different names works as well.  So it does seem to be a matter of location.
<jtv> wgrant: I'd be very interested to know why publish-distro can't work outside the archive root, but I'm not too confident that I can find out today.  :(
<jtv> Certainly nothing seems to be mentioning ".." explicitly.
<wgrant> jtv: It shouldn't be publish-distro... probably a-f or our config of it.
<jtv> Maybe it's relative softlinks that break?
<wgrant> I hope not.
<wgrant> It's possible... but unlikely.
<jtv> See diskpool.py: creates relative symlinks, but I moved the depth of the directory in the tree.
<jtv> Changed its depth, I mean.
<wgrant> Indices should go nowhere near diskpool.
<jtv> That's a relief, perhaps.
<deryck> Morning, all.
<LPCIBot> Project windmill build #162: STILL FAILING in 1 hr 5 min: https://lpci.wedontsleep.org/job/windmill/162/
* benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: henninge, benji | https://code.launchpad.net/launchpad-project/+activereviews
<benji> henninge: I'm going to do some of the reviews in active-reviews; none are claimed, are you activly working on any of them?
<henninge> Hi benji!
<henninge> benji: I am working on Huw's (the oldest in the list)
<benji> ok, I'm glad I asked, I was about to start on that one
<henninge> ;-)
<henninge> benji: I'll claim it now
<benji> thanks
<deryck> adeuring, can you join me in mumble for a second?  Just to mic check before the standup.
<adeuring> deryck: sure
<deryck> thanks!
<wgrant> wallyworld___: Thanks for fixing that up.
<wgrant> Although there is a conflict now :(
<wallyworld___> wgrant: yeah just saw it. am fixing
<wallyworld___> wgrant: i'm annoyed with myself that i had myself convinced that there was an issue with the async approach. well there was originally but it got fixed along the way and i didn't realise
<wgrant> wallyworld___: It's often hard to see that oneself :)
<wallyworld___> wgrant: yeah, especially when you've seen an issue and don't go back and revisit it later after making changes
<Ursinha> wgrant, are we suppose to use milestones or are we getting rid of that?
<Ursinha> supposed
<wgrant> Ursinha: lifeless wants to stop.
<Ursinha> wgrant, but we're not stopping yet
<Ursinha> ?
<wgrant> Ursinha: Well, now might be a good time, rather than fixing it to point at 11.05 instead... but probably best for lifeless to confirm that. So set it to 11.05 for now?
<Ursinha> wgrant, yeah, I'll do that now and talk to him late
<Ursinha> r
<Ursinha> :)
<wallyworld___> wgrant: merge conflict fixed
<wgrant> wallyworld___: Thanks. Will rereview in the morning, unless you really want it now.
<wallyworld___> wgrant: nope. morning is good
<deryck> henninge, ping for standup
<deryck> henninge_, ping for standup
<henninge_> deryck: coming
<bigjools> wgrant: steve was saying something about you saying you knew how to make createMissingBuilds quicker?
<wgrant> bigjools: I knew about the thing making IDS an order of magnitude slower than it needed to be (which StevenK has since fixed), and I have some ideas on optimising createMissingBuilds, but nothing revolutionary.
<bigjools> ok, thanks
<jcsackett> henninge, benji: can i get a review of https://code.launchpad.net/~jcsackett/launchpad/api-wants-questionset/+merge/57023
<benji> jcsackett: sure; I'm doing one at the moment, but if it's not claimed when I'm done then I'll do it then.
<jcsackett> thanks, benji. :-)
<abentley> deryck: Many or all of those failure were PEBKAC.  How do I run the YUI tests specifically?
<deryck> abentley, $BROWSER $PATH_TO_FILE, e.g. firefox lib/lp/app/javascript/test_foo.html
<abentley> deryck: no, all of them.
<deryck> ah
<deryck> abentley, ./bin/test -cvvt test_yuitest --layer=WindmillLayer
<abentley> deryck: cool.
<deryck> abentley, substituting the app-specific windmill layers if you only want some subset.
<wallyworld> deryck: too bad about the windmill tests being turned off again. i haven't had a chance to look. have you any more info?
<deryck> wallyworld, no, I just haven't had a chance myself either.  Did Natty upgrade and some pain there and 360 reviews.
<deryck> wallyworld, hope to look late today or tomorrow.
 * deryck is getting *very* frustrated with Windmill again
<wallyworld> deryck: yeah, natty upgrade pain for me too. let's touch base again in a few days
<deryck> wallyworld, sounds good.  I'm away W-F this week, if all goes well :-)
<wallyworld> deryck: cool. most likely next week then :-)
<deryck> indeed
<abentley> deryck: Now landing my JS work.
<deryck> abentley, awesome!  Thanks
<abentley> deryck: And failed out with a merge conflict :-(
<deryck> oh the joys!
 * deryck misses the xchat blinking icon in Natty
<jml> gary_poster: hi
<gary_poster> hey jml
<jml> gary_poster: on https://dev.launchpad.net/LEP/BetterBugSubscriptionsAndNotifications/FeatureReviewNotes, it says "unsubscribe in anger will be done" for Wednesday
<jml> gary_poster: what do you mean by "done" there?
<gary_poster> jml, yes, that was apparently optimistic :-/
<jml> gary_poster: I had assumed so :)
<gary_poster> jml, that was the only bad usage of "done" AFAIK.  I had intended it to mean coded.
<gary_poster> jml, I have a pending question of two for you...
<gary_poster> one of them is the email from gmb
<gary_poster> about a bug I can dig up
<jml> gary_poster: if a programmer tells me that an egg will be boiled in three minutes from now, I mentally increase that to six hours :)
<gary_poster> :-)
<jml> gary_poster: will look up that email
<gary_poster> the other is that I should give you an update on unsubscribe in anger expectations, but I maybe should wait on that one until tomorrow or Wed, when we will have more of that infrastructure in place.
<gary_poster> thank you
<gary_poster> jml, I wondered if it would be easier to convey the crux of gmb's email (titled "Bug 424849, batched notifications, fun and games, etc.") orally.  If you think so, I'm happy to Skype/mumble briefly.
<_mup_> Bug #424849: Launchpad should batch attachment notification emails <lp-bugs> <story-better-bug-notification> <story-better-notification-sending> <Launchpad itself:In Progress by gmb> <apport (Ubuntu):Invalid> < https://launchpad.net/bugs/424849 >
<jml> gary_poster: the main thing I don't get is why fixing bug 31586 would make it possible to batch multiple commented attachments.
<_mup_> Bug #31586: Malone comments are sent in email and forge the address of the person who filed them <email> <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/31586 >
<gary_poster> jml (I'm against that approach for a couple of reasons but...) it would help because then you could batch up *all* the comments on a particular bug.  So, solution 1 would let us batch comments/attachments from the same person for the same bug, and solution 2 lets us batch comments/attachments from the same bug--it should produce even less email.
<jml> oh, I see.
<jml> gary_poster: sorry, it took me a while to see the obvious, that the email has to come from someone :)
<gary_poster> :-)
<jml> gary_poster: multitasking is the mindkiller.
<gary_poster> heh, too true
<jml> gary_poster: how much work are we looking at if we take option 1?
<gary_poster> jml, we are guessing one developer, one week,  Maybe two devs, 3 days.  We really try not to make you have to inflate our guesses. :-)  The change itself should be relatively simple, a day-ish, but touching email sending can sometimes affect other parts of LP, so we are adding in time for problems of that nature.
<sinzui> jcsackett: do you have a few minutes to mumble?
<jml> gary_poster: ok. I think we should take option 1 then.
<gary_poster> jml, ack.  I'll put it on the board.  Thank you!
<jml> gary_poster: np.
<jcsackett> sinzui: sure.
<henninge> benji: I assume you claimed whatever you are working on?
<adeuring> deryck[lunch]: could you have another look at https://code.launchpad.net/~adeuring/launchpad/bug-746460/+merge/56840 ?
<gary_poster> henninge or benji, I have a small-ish JS branch for you.  https://code.launchpad.net/~gary/launchpad/bug750561/+merge/57195
<henninge> gary_poster: here!
<gary_poster> :-) cool thanks
<adeuring> ...or henninge, could you have a look at https://code.launchpad.net/~adeuring/launchpad/bug-746460/+merge/56840 ?
<henninge> adeuring: I just accepted gary_poster's but I guess you are closer to EOD. I'll have a look at how big it is.
<adeuring> henninge: ok, np
 * gary_poster is closer to lunch...and hungry, too!
<gary_poster> (that means nothing but silliness, in case it's not clear)
<deryck[lunch]> adeuring, looks fine to me.  I would ask henninge or abentley for a second opinion on that one line removal in lib/lp/translations/tests/test_pofile.py....
<deryck[lunch]> adeuring, I admit that I don't know the consequences of that either.
<adeuring> deryck[lunch]: yeah ;)
<adeuring> henninge, abentley: It's a small request ;)
<henninge> already looking at it
<henninge> adeuring: wow, didn't know about multiple "for" loops in list comprehension ...
<adeuring> henninge: erm.. where did you find this? (can't remember anything like that... ;)
<henninge> adeuring: I just saw you only moved so you probably didn't notice.
<henninge> +    >>> [hoary_package] = [
<henninge> +    ...     package for series in view.series_batch.batch
<henninge> +    ...     for package in series.packagings
<henninge> +    ...     if package.distroseries.name == 'hoary']
<adeuring> yeah, looks interesing ;)
<henninge> http://docs.python.org/tutorial/datastructures.html#list-comprehensions
<henninge> deryck[lunch], adeuring: This is the missing text:
<henninge>     + Change upstream link
<henninge>     + Remove upstream link
<henninge> adeuring: does that fit with the new permissions?
<adeuring> henninge: well, I can add it back, but it does not have any effect....
<henninge> Those two represent the edit and delete icons.
<adeuring> well, I would have to modify the test slightly
<henninge> hm?
<henninge> adeuring: no, that's not what I mean.
<henninge> adeuring: is the missing of the icons expected?
<henninge> the way the test is now.
<henninge> If so, all is well.
<adeuring> henninge: ah, the ellipis stuff? that's expected
<henninge> adeuring: yes, that is what the ellipsis represented.
<adeuring> henninge: and thanks for spottings the real difference ;)
<henninge> well, it's what deryck[lunch] asked for ... ;-)
<adeuring> henninge: there is and odd change in test_pofile. That's our main concern
<henninge> oh, right
<henninge> mixed that up
<henninge> adeuring: that one looks like a copy&paste error. Thanks for fixing it.
<bigjools> night everyone
<adeuring> henninge: THANKS FOR FIGURING OUT WHAT WAS GOING ON ;)
 * adeuring should remove the caps-lock key...
<henninge> adeuring: r=me on the incremental diff.
<adeuring> henninge: cool, thanks!
<benji> henninge: did I claim a review you were already doing? (the garbo logging one)
<henninge> benji: I had just started when gary and abel asked. So no doubled work.
<henninge> benji: but it was unclaimed at the time.
<benji> henninge: sorry about that, I'll have to be more careful
<henninge> np
<henninge> gary_poster: still lunching? ;)
<gary_poster> no henninge :-)
<henninge> gary_poster: I'd like to see your branch in action. Can you give me a URL on dev?
<gary_poster> henninge, sure.  It will involve feature flags.  I'll get you a pastebin with details.
<henninge> gary_poster: I already copied the feature flags from production if those are the ones needed.
<gary_poster> henninge, ok, well, here are the details anyway :-) http://pastebin.ubuntu.com/592749/
<abentley> henninge / benji, could you please review https://code.launchpad.net/~abentley/launchpad/fix-dummy-translations/+merge/57208
<benji> abentley: I'm finishing up a review now, so one of us will get it shortly.
<abentley> benji: great, thanks.
<henninge> gary_poster: ah yes, I can see it ;)
<gary_poster> :-) cool
 * henninge remembers that Saturday was international day of Astronomy ...
<henninge> I think it was international
<dobey> henninge: i think every day is a day of astronomy. :)
<henninge> yeah!
<gary_poster> henninge, FWIW, this is the follow-on branch to the one you are looking at.  https://code.launchpad.net/~gary/launchpad/bug750561-2/+merge/57216 .  It is very similar.
<gary_poster> I can ask someone else to do it though, I expect it is your EoD already?
<henninge> gary_poster: yes, I will stop working after this.
<gary_poster> cool
<henninge> gary_poster: This bit looks interesting but I don't know what it does.
<henninge> http://paste.ubuntu.com/592764/
<gary_poster> henninge, this is the machinery I mentioned in the MP that adds the ability of our LPClient test stub to halt execution.  That lets the test see what the state of things looks like after a call has been made to, say, named post, but before the named post has a response and calls the success or failure callback.
<gary_poster> You resume the success or failure callback by calling .resume on the stubbed function/method/whatever you want to call it.
<gary_poster> (And generally, that code is the heart of the LPClient test stub/mock, letting you specify for named_post and patch whether you want them to succeed or fail, and what arguments they should send back; and later letting you check the calls and arguments that it received)
<lifeless> moin
<lifeless> statik: are we on today?
<henninge> gary_poster: thanks for the explanations. r=me
* henninge changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: benji | https://code.launchpad.net/launchpad-project/+activereviews
<gary_poster> thank you henninge!
<gary_poster> benji, short-ish one for you.
<gary_poster> https://code.launchpad.net/~gary/launchpad/bug750561-2/+merge/57216
<benji> gary_poster: k
<gary_poster> thank s
<benji> gary_poster: review done
<gary_poster> cool thanks
<lifeless> sinzui: ping - bug 753306
<_mup_> Bug #753306: mailman doesn't shut down cleanly <canonical-losa-lp> <Launchpad itself:Triaged> < https://launchpad.net/bugs/753306 >
<sinzui> That looks hard
<lifeless> sinzui: tom has added a note to it about what the shutdown script runs
<sinzui> That does not look like anything to do with launchapd
<lifeless> sinzui: I'm figuring you probably know what it should run
<lifeless> sinzui: strictly speaking its not, but its part of our environment, and not an upstream script or issue
<sinzui> I know the pipeline. We have a nice test that shows we reordered it
<sinzui> I think the issue here is that I think mailman waits for all the handlers/pipelines to complete what they are doing before shutdown is complete
<lifeless> barrrrrry!
<lifeless> sinzui: so, what command *should* the losas be running to trigger a shutdown, and how long should they budget to wait
<lifeless> sinzui: oh, good thought bringing barry in. I'll get my butt out now :)
<sinzui> lifeless: I think I can make changes that barry recommends. He wrote the mailman-lp startup code. He may point out we lost something in a revision and advise me to put it back
<jcsackett> thanks for the review, benji.
<benji> my pleasure
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project windmill build #163: FIXED in 1 hr 4 min: https://lpci.wedontsleep.org/job/windmill/163/
* benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
#launchpad-dev 2011-04-12
<sinzui> thumper: mumble?
<huwshimi> sinzui: I think he's away today
<wgrant> wallyworld: I may be misunderstanding here, but picker.js' animate_validation_content and reset_form appear to be global. Is that intentional?
<wallyworld> wgrant: no, they should be global but i though because they were not added to the namespace they wouldn't be exposed??
<wallyworld> should *not* be
<wgrant> wallyworld: I'm not really sure how that works.
<wgrant> As I said, I may be misunderstanding.
<wallyworld> wgrant: i'm not sure either but i thought i was doing it correctly. that's why i've asked for a ui review cause whoever does that typically knows javascript better than me
 * wallyworld sighs. doctests make me sad
<wgrant> wallyworld: I just tried it myself... it's very pretty.
<wgrant> Although it has the issue that plagues most LP overlays: it resizes a lot.
<wallyworld> wgrant: it was a bit of a learning curve for me but i enjoyed it. the resizing i didn't think was too obtrusive. i thought it better than leaving the overlay way too big for the content
<wgrant> wallyworld: Yeah, it's a general problem that we don't really have a solution for.
<wallyworld> wgrant: the animation really helps i think. not sure why but it's easier on the eye when there's resizing involved rather than something just splatting onto the screen
<wgrant> Yeah.
<wgrant> Although it's possibly a bit long, it looks great.
<wallyworld> wgrant: i experimented with the timing. it';s easy to change. i didn't think it was too bad myself but i'll happily be guided bu someone else
<wallyworld> if it's too short you don't see the effect
<wgrant> wallyworld: Re-reviewed.
<wallyworld> wgrant: thanks. looking now
<lifeless> ok, so bugtag needs to be redesigned
<wallyworld> wgrant: the use of inline styles for the buttons div was copied from an existing usage
<wgrant> wallyworld: Ugh. OK.
<lifeless> sinzui: still around ?
<sinzui> I am
<lifeless> sinzui: oh hai
<lifeless> sinzui: there is a refactoring we need to do to fix performance in soyuz/recipes/translation template builds
<lifeless> sinzui: I don't know if wgrant has mentioned it to you yet (or not)
<wallyworld> wgrant: but we should propogate bad practice so i'll fix it
<wallyworld> *shouldn't*
<wgrant> wallyworld: Thanks.
<wgrant> lifeless: I hadn't.
<lifeless> sinzui: the BFJ/PB tables need to be folded into their adjacent tables (which are all 1:1) to permit context specific scans rather than all-builds-ever-history-scans
<sinzui> lifeless: I can make those related bug the next ones to be started
<wgrant> wallyworld: Oh, one other thing... are there spinners on both ends of the picker, if there is latency before it updates?
<lifeless> sinzui: its going to be a new-schema, migrate-over-a-week-or-so, then start using transition job
<lifeless> sinzui: I'll get a bug filed with supporting analysis and point you at it.
<wallyworld> wgrant: you mean when the confirmation info is loaded? no. i don't see an obvious place to put a spinner.  even though there probably should be one
<wgrant> wallyworld: Yeah, that was my thought, too... it's not clear where it should be.
<wallyworld> there no need for one when the user confirms the selection cause there's already one there as per a normal save
<wgrant> Right.
<wallyworld> if it were a desktop app i'd change the cursor
<wgrant> Ouch.
<wgrant>       138 /  434  Distribution:EntryResource:searchTasks
<lifeless> wgrant: btw we repacked the fti index last night
<wgrant> I saw that.
<wgrant> And the resulting timeouts :)
<wgrant> (looks like the lock was held for a little too long)
<lifeless> sinzui: bug 758258
<_mup_> Bug #758258: buildfarmjob schema is inefficient for reporting <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/758258 >
<sinzui> I assigned it to the green squad
<lifeless> thank you!
<lifeless> its almost small enough that I tackled it mysself
<lifeless> but not quite
<wgrant> lifeless: Could you please mentor https://code.launchpad.net/~wallyworld/launchpad/disabled-project-bugs-shown/+merge/57265?
<lifeless> wgrant: you missed a bit :)
<wgrant> Mm, I guess.
<wgrant> Well.
<wgrant> Something has completely destroyed my monospace fonts.
<wgrant> I think I may reboot and see if it is any better.
<wgrant> brb
<wallyworld> lifeless: the reason for just adding to the doc tests is that all the other testing for getBugTaskAndNominationViews is done there. i new unit tests were written, then the doc test would have to be ported over to keep it all together. i can do that, but it seemed like much extra work for marginal benefit
<lifeless> wallyworld: I don't buy the keep it together argument
<lifeless> wallyworld: given the following options:
<lifeless>  - a longer doctest
<lifeless>  - unaltered doctest and some unit tests
<lifeless>  - all unit tests
<lifeless> clearly the third option is best
<lifeless> but I argue that the second option is better than the first
<wallyworld> lifeless: i agree that #1 or #3 is preferable but #2 is bad. all tests for a given piece of code should be together so they are easily discoverable  etc imho
<wallyworld> you don't want to fragment the tests for a given module/class all over the code base
<lifeless> depends on what you value
<lifeless> right now, you can't safely change that function and only run that doctest
<lifeless> I think the tech debt of increasing the doctest outweighs the tech debt of having the tests in two different places
<lifeless> but thats up to you, I can only share why I would do it differently.
<wallyworld> lifeless: np. it's a matter of opinion in this case. i'll add a new unit test then
<lifeless> wallyworld: the other issues are significantly more important :)
<wallyworld> lifeless: with the filter - there was already filtering being done in the list comprehension. i just added an extra condition.
<lifeless> wallyworld: I think you are reading too shallowly
<lifeless> wallyworld: Thats a partition, not a filter
<lifeless> or 'thats a partition special case of multiple filters'
<wallyworld> my thinking was that the bugtasks were already eagerly loaded prior to the method being called. we were already extracting the ones we wanted from that list using a comprehension. why not add another condition to the filter?
<lifeless> wallyworld: because:
<lifeless>  - things that iterate on the bugtasks will show too many
<lifeless>  - we'll be eager loading too much data
<wallyworld> ah ok. so you want the initial loading of the self.bugtasks to be filtered
<wallyworld> although these com from the context
<lifeless> right, the context has the wrong bugtasks
<lifeless> searchTasks knows how to do all the filtering you're doing
<lifeless> (though, why are you filtering on experimental distroseries?!)
<wallyworld> ok. makes more sense now. i was trying to make the smallest change to fix the reported issue
<lifeless> wallyworld: I am totally in favour of doing the simplest thing that meets all the criteria
<wallyworld> but in this case it made sense to look a little further afield
<lifeless> I don't think that this code need sto change at all though, and the filtering rules need to be harmonised - we don't want to duplicate stuff
<lifeless> note this:
<lifeless>          self.bugtasks = list(self.context.bugtasks)
<wallyworld> yes, agreed. sometimes i'm more wary than needed for fear of breaking things like code i'm not 100% fmilar with
<wallyworld> lifeless: so self.context.bugtasks a resultset then?
<lifeless> wallyworld: *blink*
<wallyworld> is a
<lifeless> wallyworld: no, nor should it be
<wallyworld> ok. a collection. which is not loaded until the list() is constructed. but we want to so a search instead
<lifeless> yup
<lifeless> self.context is a Bug
<lifeless> so do a bugtask search with bug=self.context.id
<lifeless> listify the result
<wallyworld> right. i knew self.context was a bug but didn;t realise it's bugtasks would not already be loaded
<lifeless> wallyworld: one of them will be
<lifeless> wallyworld: (the one for the url we're on)
<lifeless> wallyworld: more /may be/ - but the key thing is the eager loading
<lifeless> wallyworld: see all the other work the constructor does
<lifeless> wallyworld: we dont' want to do that on unwanted bugtasks.
<wallyworld> sure. but i thought they would all already be loaded, hence not seeing the issue with the list comprehension. makes more sense now.
<wallyworld> thanks for the input
<lifeless> wallyworld: no worries
<lifeless> wallyworld: the trick here when looking at other cases
<lifeless> wallyworld: is to look at what *else* is done to the same data
<lifeless> wallyworld: sorry that I wasn't clearer in my review; I guess I had some hidden assumptions aout what you'd looked at
<wallyworld> lifeless: np. i had a wrong assumption about what had already been loaded and that lead me down an incorrect path
<lifeless> wgrant: I would analyse the current searchTasks timeouts but
<lifeless> bug 740750
<_mup_> Bug #740750: API timeouts broken and returning no useful data... <regression> <Launchpad itself:Triaged> <lazr.restful:Triaged> < https://launchpad.net/bugs/740750 >
<wallyworld> lifeless: turns out self.bugtasks = list(self.context.bugtasks) doesn't do any new sql cause all bugtasks are already loaded earlier. so doing a search actually *increases* the sql count. i'm tracking down where the earlier loading of all bugtasks is done
<lifeless> wallyworld: traversal
<wallyworld> lifeless: yeah, just got a hit on my conditional breakpoint. gotta figure out what to do about it
<lifeless> wallyworld: I'd ignore it; the traversal load almost certainly doesn't do the right query
<lifeless> wallyworld: and its not a significant cost
<wallyworld> lifeless: in the traversal code BugTargetTraversalMixin._get_task_for_context() it just loops through bug.bugtasks, hence loading them all
<lifeless> [compared to loading milestones etc for disabled rows]
<lifeless> wallyworld: yup
<wallyworld> lifeless:  ok, will do the search. bugtaskset.search() only filters on active projects. not distroseries or productseries. i guess that doesn't matter and it's really just the active projects we care about filtering?
<lifeless> wallyworld: right, I wasn't aware distros could be inactive
<wallyworld> lifeless: (product|distro)series have a read only property "active"
<wallyworld> set to false when the status is experimental etc
<lifeless> wallyworld: ah, this is different to the pillar 'active' field I think
<lifeless> wallyworld: the pillar active is set to false when we disable and hide a product/project
<wallyworld> ok. that's me not fully understanding the domain model then
<lifeless> or me, IMBW
<lifeless> second opinon time
<lifeless> wgrant: ^ second opinion plox
<lifeless> wallyworld: anyhow, if we should filter more, then searchtasks should be filtering more.
<lifeless> wallyworld: so useing searchtasks is right either way
<lifeless> jtv: hi
<lifeless> https://devpad.canonical.com/~lpqateam/qa_reports/deployment-db-stable.html
<jtv> hi lifeless
<lifeless> has a few patches from you needing qa
<wallyworld> lifeless: agreed. i'll just leave it as is for now (filtering only on active projects) since that appears to be the issue at hand
<jtv> lifeless: I shall see to it fortwithâthink all of it is probably safe-to-deploy regardless of actual Q/A.
<jtv> forthwith.
<lifeless> jtv: cool
<jtv> Thank you spell checquer.
<lifeless> jtv: we have a couple of weaks (sic) :)
<jtv> (sick)
<jtv> Wee half er Koppel
<wgrant> lifeless: Hi, sorry, having fglrx issues.
<wgrant> lifeless, wallyworld: IPillar['active'] is what we want here.
<wgrant> Product has a DB field, Distribution is always True.
<wgrant> But it's what we want.
<jtv> lifeless: the first one is the only one where deployment has any repercussions at all.
<lifeless> wgrant: bugtaskset.searchtasks is required to honour this anyway
<lifeless> wgrant: right?
<wgrant> lifeless: I believe it is fixed, yes.
<lifeless> right
<lifeless> so , searchtasks ftw
<lifeless> registry will be able to see the disabled tasks (which is appropriate)
<wgrant> Yeah.
<jtv> wgrant: the branch that gets rid of all the cache-flushing and gc'ing in the publisherâ¦ you tried it out on dogfood before it landed; did you see enough detail to know that it's safe to deploy?
 * jtv corrects himself
<jtv> "that it looks reasonably safe to deploy"
<wgrant> jtv: Let me just look through the diff again, but it all seemed to work OK.
<wgrant> I'd like to try fully publishing lucid to get a memory worst-case, but we don't have enough disk.
<wgrant> So natty will have to do, and it manages that fine.
<jtv> Then I think the only reasonable concern is memory usage, which is _probably_ solved and even if it's not, we'd ultimately be better off addressing afresh.
<jtv> Soâ¦ let's say that makes it qa-ok.
<wgrant> Right. I'm going to get it to publish natty again and watch the peak usage, but I would qa-ok it.
<jtv> Thanks!
<jtv> The effects won't be as dramatic as they were in the test, since we tested with the replacement publish-ftpmaster script.
<jtv> That means that publish-distro will have its memory to itself, and any flushing it fails to do won't affect anything else in the procedure.
<jtv> lifeless: you have a clear runway
<lifeless> \o/
<lifeless> we really should finish up the lmirror stuff
<wgrant> I haven't heard much about that in a while.
<lifeless> I haven't had time to optimise the parser, which was the only scaling problem I recall in it
<lifeless> wallyworld: ping
<lifeless> wallyworld: bug 756878
<_mup_> Bug #756878: Old bzr-builder <Launchpad itself:Won't Fix> < https://launchpad.net/bugs/756878 >
<lifeless> wallyworld: why did you close it wont fix ?
<wallyworld> lifeless: there's nothing to fix? nest-part is already supported since 0.4 and the current release is 0.6
<wgrant> The bug suggests we've reverted Launchpad's.
<lifeless> wallyworld: the version in natty has /nothing/ to do with what we're running on the web servers.
<lifeless> wallyworld: nor with the version we run on the build slaves.
<StevenK> Or on the buildds themselves, where it matters.
<StevenK> lifeless: Snap
<lifeless> wallyworld: both of which matter, because the one on the web server is where our validation code runs
<lifeless> wallyworld: and the buildds is where the actual builds happen
<wallyworld> lifeless: where do we get the package for the servers you just mentioned?
<wallyworld> maybe i misunderstood the bug, sorry :-(
<wgrant> The buildds use hardy-cat, the webapp should use sourcecode.
<lifeless> wallyworld: there are three dependency systems we have
<lifeless> wallyworld: launchpad-dependencies package
<lifeless> wallyworld: utilities/sourcedeps.conf
<lifeless> wallyworld: and versions.cfg
<StevenK> wgrant: hardy-cat? They're still hardy?
<wgrant> StevenK: Yes, until dapper dies.
<StevenK> Ah
<StevenK> Which is soon, I think
<wgrant> Yes!
<lifeless> wallyworld: so in this case, if you check utilities/sourcedeps.conf and look at the branch, you can see there is unused work on the branch
<lifeless> not to mention it may be arbitrarily out of date with upstream
<wallyworld> looking now
<lifeless> wallyworld: in general nothing in lp will depend on current distro versions - not because we don't use them - we do, but because we're version locked, in one of those dependency tools, and we have custom archives for things that aren't present on the LTS release of the distro we are running
<wallyworld> lifeless: ok. sorry. sourcedeps.conf does refer to a really ancient rev
<wallyworld> should we be updating that?
<lifeless> wallyworld: its only one rev back on the branch we run from
<lifeless> wallyworld: but yes.
<lifeless> bzr-builder is  alittle more complex because you need to upgrade the web ui & the buildds separately
<wallyworld> lifeless: rev 68 was around jan 2010. tip is 119
<lifeless> wallyworld: different branches
<lifeless> wallyworld: you can't compare the revnos
<wallyworld> lifeless: i thought i was looking just at trunk
<lifeless> wallyworld: sourcedeps.conf lists a branch and a revno
<lifeless> wallyworld: that branch is the lp branch, not the upstream branch
<lifeless> revno: 68 [merge]
<lifeless> committer: Launchpad Patch Queue Manager <launchpad@pqm.canonical.com>
<lifeless> branch nick: trunk
<lifeless> timestamp: Tue 2010-11-23 18:32:13 +0000
<lifeless> we're running a version merged in late november 2010
<lifeless> rev 69
<lifeless> timestamp: Tue 2010-12-07 21:24:13 +0000
<lifeless> message:
<lifeless>   [r=me] Update to lp:bzr-builder tip
<lifeless> is available
<StevenK> Isn't there some test harness thing that takes care of running scripts? I thought it was 'runscript' but I can't find it.
<lifeless> and presumably there is more work in upstream trunk too
<wallyworld> ok. so i'll re-open the bug
<lifeless> wallyworld: checking the work missing against trunk just requires branching our branch and running bzr missing lp:bzr-builder
<wallyworld> lifeless: ok. should the bug be marked as "high"? or "critical" since it affects a major lp feature?
<lifeless> I don't know if we ever actually supported nest-part
<lifeless> I suspect someone got mostly through upgrading but didn't quite finish it (that rev 69 in our branch)
<lifeless> it may be as simple as s/68/69/ in sourcedeps.conf and send to pqm
<wallyworld> well we can hope
<wallyworld> i'll take a look later
<huwshimi> Can someone explain to me the rules for what types of links/info we display in the sidebar?
<huwshimi> or point me to a doc
<wallyworld> lifeless: the gui does say "Assign", "Choose Again". but the code is generic. the labels to put on the buttons are specified by the caller
<lifeless> ah, kk.
<StevenK> Can haz review?
<lifeless> no
<lifeless> can't has
<lifeless> wgrant: want to, and I"ll mentor?
<wgrant> StevenK: Where is it?
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/ids-bugfixes/+merge/57270
<lifeless> https://mail.google.com/mail/u/1/?ui=2&shva=1#inbox/12f481611058e601
<lifeless> :P
<StevenK> I was so tempted to answer "On Launchpad. Duh."
<wgrant> StevenK: IDSJ takes forever... the job runner won't block on that?
<StevenK> wgrant: IDS or IDSJ?
<wgrant> Both.
<StevenK> wgrant: Oh, I should have added to the MP that I've QA'd this on dogfood before submitting.
<StevenK> wgrant: But run_jobs -v initialisedistroseries is happy enough on DF
<wgrant> StevenK: Oh, you give run_jobs a job type to run, it doesn't just run all of them?
<StevenK> wgrant: Exactly -- the run_jobs cronscript looks pretty much identical to the one I removed, it just pulls the source interface from the schema.
<StevenK> wgrant: The flush did seem to be required, when running the job runner it would OOPS saying the DAS could not be found.
<StevenK> wgrant: You're right, it seems it just wants ChildSeries.parent_series to get at the parent. I'd *love* to rewrite that horrible query, but it scares me.
<StevenK> wgrant: That error came from i-f-p. I got Julian'd into keeping the wording when I originally did the work.
<StevenK> wgrant: I've reverted the Store.find() change, and fixed the query. You feel nervous about the pending builds change?
<wgrant> StevenK: It has potentially disastrous consequences.
<wgrant> Copying an incomplete build set is terrible terrible thing to do in the current model.
<wgrant> In the worst case you can never copy that source again!
<StevenK> wgrant: You know IDS just uses the package cloner, right?
<wgrant> Yes. And?
<StevenK> But it doesn't touch BPB at all
<wgrant> createMissingBuilds does...
<StevenK> So, we copy a SPPH for a source, and a BPPH for the completed i386 build, since the amd64 build is NEEDSBUILD. The package cloner calls createMissingBuilds() which creates the missing amd64 build. And this is wrong?
<wgrant> StevenK: You now have two amd64 NEEDSBUILD builds in the same archive.
<wgrant> One can upload, the other will fail.
<wgrant> And you can never reconcile that.
<StevenK> Even if they are targeted to different distroseries?
<lifeless> so
<lifeless> big picture, IIUC
<wgrant> Same binary name in same archive = oh god what have we done
<lifeless> is that we need to do this $anytime
<lifeless> so waiting for quiesced == problem
<wgrant> lifeless: Sure.
<wgrant> But we cannot resolve this until we make binary copying not stupid.
<StevenK> wgrant: Right, so it's okay, IFF it's a different archive?
<lifeless> is this the same root issue as uploading to both poopies ?
<wgrant> StevenK: It's not okay, it's just not definitely going to fuck shit up.
<wgrant> lifeless: Not really.
<wgrant> lifeless: Not at all, actually.
<lifeless> wgrant: I asked because bigjools said that the upload issue and in-ui copying were the same problem
<lifeless> or at least, thats what I thought he said
<wgrant> lifeless: Yes, but that is a different in-UI copying issue.
<wgrant> They both have the same file conflict race.
<wgrant> But the race there is that the conflict will not be detected.
<wgrant> Here it will be detected, but you are stuck with it forever.
<lifeless> why would it create a missing build when there is a pending build?
<StevenK> Because .createMissingBuilds() is dumb.
<lifeless> ok
<wgrant> No, because our binary copying model is dumb.
<wgrant> And impossible to handle sanely.
<lifeless> so given my (admittedly shallow) understanding, we have more work todo before we can do this safely.
<lifeless> however
<wgrant> It's going to be amusing when we have armel PPAs.
<lifeless> given our isolation levels we're probably not entirely safe even when there are no pending builds
<wgrant> You will no longer be able to copy between all series.
<lifeless> [we only require serializable]
<wgrant> Upload to one series, builds (i386, amd64, armel). Copy to hardy, creates new lpia build. Can no longer copy either, because they don't have a superset of the archive's binaries.
<lifeless> argh bug 434733  comments
<_mup_> Bug #434733: marking public bug as duplicate of private bug leads to confusing UI <404> <confusing-ui> <disclosure> <lp-bugs> <privacy> <regression> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/434733 >
<StevenK> wgrant: Okay, so this obviously requires more thought. If I revert the NEEDSBUILD changes, are you broadly happy with the MP?
<wgrant> StevenK: I'll look over it again, but that is indeed the only deal- and distro-breaker.
<poolie_> does https://bugs.launchpad.net/launchpadlib/+bug/758460 ring any bells?
<_mup_> Bug #758460: ConfigParser.NoOptionError: No option 'consumer_key' in section: '1' <launchpadlib :New> < https://launchpad.net/bugs/758460 >
<StevenK> wgrant: Changes pushed, diff updated.
<wgrant> StevenK: You've fixed bug #744849, right?
<_mup_> Bug #744849: populate-archive.py unusably slow <Launchpad itself:Triaged> < https://launchpad.net/bugs/744849 >
<poolie> ah it's just the same as bug 745801
<_mup_> Bug #745801: system-based authorization doesn't store useful credentials in gnome-keyring <amd64> <apport-bug> <natty> <launchpadlib :New> <python-launchpadlib (Ubuntu):New> < https://launchpad.net/bugs/745801 >
<wgrant> lifeless: Can bug #637761 be closed?
<_mup_> Bug #637761: single-thread appserver experiment (rt41361) <lp-foundations> <Launchpad itself:Triaged> < https://launchpad.net/bugs/637761 >
<wgrant> lifeless: Given that it was the greatest success we have had in years?
<lifeless> in a sense
<lifeless> though we're not migrated yet
<lifeless> I don't mind either way
<wgrant> We've run the experiment.
<wgrant> => experiment bug is old.
<lifeless> wgrant: like I say, I don't mind either way
<StevenK> wgrant: The branch that fixes bug #744849 landed this morning.
<_mup_> Bug #744849: populate-archive.py unusably slow <Launchpad itself:Triaged> < https://launchpad.net/bugs/744849 >
<lifeless> man, conflict wise, its going to be a long month
<StevenK> Argh structual-subscription.js AGAIN?
<StevenK> wgrant: In fact, the fix hit stable about 30 minutes ago.
<wgrant> lifeless: Yeah, it's tempting to restrict features to a single branch :(
<wgrant> lifeless: Or just roll out more frequently.
<wgrant> s/roll out/DB-downtime/
<StevenK> wgrant: The qa-tagger would have done that in about 30 minutes? :-)
<wgrant> StevenK: Is it linked?
<StevenK> [r=lifeless, wgrant][bug=744849][no-qa] ...
<lifeless> wgrant: well, working on more often
<wgrant> StevenK: Ah.
<wgrant> lifeless: :(
<lifeless> wgrant: bigger branches have worse penalties
<wgrant> lifeless: One branch meaning either db-devel or devel.
<wgrant> Not a single feature branch.
<StevenK> Yes, a single feature branch would be crackful.
<wgrant> StevenK: pre-2008-Soyuz crackful, yes.
<StevenK> wgrant: You don't think the commit is needed?
<wgrant> StevenK: It wasn't before, was it?
<wgrant> What has changed?
<StevenK> I wanted to be explicit, I guess
<wgrant> Oh, did the thing at the end commit itself?
<wgrant> Whereas setParent does not?
<wgrant> I guess.
<wgrant> That's a reasonable justification :)
<StevenK> So you just withdrew your objection to the commit?
<wgrant> I asked for a reason.
<wgrant> You gave one!
<StevenK> Well, that's sort of an objection
<StevenK> The flush is due to a number of OOPS I saw on dogfood
<lifeless> wgrant: oh; hmmm
<wgrant> lifeless: Otherwise it's going to be a very long month on this side of the world :)
<StevenK> wgrant: According to Julian, large parts of Soyuz use parent_series as a "Yes, this series is initialised."
<wgrant> StevenK: Those are bugs.
<StevenK> Not according to Julian.
<StevenK> I've had this argument with him, I lost.
<wgrant> He is going to lose to me.
<wgrant> :)
<lifeless> if /any/ part of the system does not honour that, using it as such a flag would be a bug.
<wgrant> StevenK: Warty exists, so Soyuz does not require it.
<lifeless> As there must be at least one series with no parent.
<wgrant> StevenK: Warty even had lots of security uploads.
<lifeless> Some part of the system cannot honour it.
<lifeless> And thus using parent_series to pun with 'is initialised' is a bug.
<wgrant> Precisely.
<wgrant> It is both a stupid overloading and *impossible*.
<StevenK> wgrant: Julian also said that if Warty was touched Soyuz would blow up.
<lifeless> StevenK: add a field
<wgrant> StevenK: You couldn't +edit warty for a while, no.
<wgrant> StevenK: But you can do anything else to it.
<StevenK> lifeless: I like that idea even less.
<lifeless> StevenK: its entirely appropriate to be explicit about these things.
<StevenK> The other option is IDSJ.Job.status
<wgrant> DistroSeries.i_am_not_going_to_break_everything
<wgrant> StevenK: No.
<wgrant> StevenK: Hardy doesn't have an IDSJ.
<lifeless> StevenK: whats your objection
<wgrant> Even Natty probably doesn't.
<StevenK> I'm just moving the set parent call to the end
<lifeless> StevenK: does this run as one transaction ?
<StevenK> lifeless: To other useless field? DistroSeries is bloated already, and it's a use-once thing
<StevenK> Once the distroseries is done initialized it's pointless
<lifeless> StevenK: thats rather beside the point
<lifeless> StevenK: either this thing is atomic and always consistent
<lifeless> or its eventually consistent
<lifeless> if its eventually consistent then the rest of the system has to handle it being inconsistent
<lifeless> there are multiple ways you can do that
<StevenK> lifeless: No, it doesn't run as one transaction, the package cloner calls commit a lot
<lifeless> a flag which is propogated atomically once and only once the thing is consistent is a simple way to do this.
<lifeless> StevenK: distroseries - the whole table - is 16kb
<StevenK> Sorry, I'm not talking about the table, I'm talking about the *object*
<StevenK> IDistroSeries is *enormous*
<lifeless> StevenK: so, either design out the eventual consistency, or embrace it.
<lifeless> StevenK: conflating 'used in a particular way' with 'has had all its initial data copied in' seems guaranteed to create boobytraps
<StevenK> wgrant: So you think my reasoning is crack, and I should revert the placement of set parent?
<lifeless> what we did in bzr was take out a write lock on the object and only release that when the object was usable
<wgrant> StevenK: Yes. I will defeat bigjools on this if I have to.
<StevenK> lifeless: Given it takes 10 minutes or more to initialise a series, I suspect you'll shoot me.
<lifeless> StevenK: if you were to do one transaction, yes
<wgrant> Anyone want to review https://code.launchpad.net/~wgrant/launchpad/bug-736002/+merge/57278?
<jtv> wgrant: I'll trade you for this one, since you're OCR: https://code.launchpad.net/~jtv/launchpad/db-bug-752279/+merge/57277
<wgrant> I am intrigued to see how "Simplify and harden directory movements in publish-ftpmaster." works out to 819 lines.
<jtv> wgrant: there's one way to find out!
<wgrant> Indeed.
<lifeless> wgrant: what does the executed sql look like
<wgrant> http://paste.ubuntu.com/593003/
<wgrant> lifeless: WITH made the code terribly hard to work with, and this is just as fast AFAICT.
<wgrant> Although the SQL is much cleaner with WITH.
<lifeless> wgrant: how did it make it hard to work with ?
<wgrant> lifeless: I would have had to do the union inside gUBTWOC.
<jtv> wgrant: isn't "top 10 unofficial tags" a matter of querying the top 10+<#official tags> tags and eliminating the official ones?
<wgrant> jtv: It is.
<wgrant> But I doubt anybody will notice.
<wgrant> Speed > trivialities that nobody will notice
<lifeless> jtv: you'd need to query the top 20
<lifeless> jtv: and then take official + 10 more
<jtv> But is it a triviality or a specialized workflow?  I imagine there is a useful process of reviewing the most popular unofficial tags and blessing them.
<wgrant> jtv: I considered that.
<lifeless> in fact, you need to take 10 more than the count of official tags
<wgrant> But decided I didn't care enough. I can do it if someone convinces me to, but it seems to be of very limited utility.
<wgrant> lifeless: That's what jtv said.
<jtv> :-P
<lifeless> I'm slow :)
<jtv> wgrant: I'm not particularly bothered, just bringing it up.  Maybe we'll find we want a marker for "(unofficial)" instead or something.
<lifeless> Time: 4001.783 ms
<lifeless> cool
<wgrant> lifeless: Is that cold?
<jtv> Cool is sort of coldâ¦
<wgrant> Hot was more like 3700ms yesterday... although this is a different official tag set.
<wgrant> Also the DB has been restored since then.
<wgrant> So I guess that's OK.
<lifeless> wgrant: hot
<StevenK> wgrant: The last thing is the enormous if block -- can you suggest something? I touched it because I hated the idea of if <>: raise ; if <>: raise which is what is was before
<wgrant> StevenK: You could just remove the bool() bits.
<lifeless> also
<wgrant> Or use any()
<lifeless> only sqlobjectresultset has __nonzero__
<wgrant> Right.
<lifeless> don't use it. anywhere.
<wgrant> I did suggest is_empty
<wgrant> So...
<wgrant> if not all(map(lambda r: r.is_empty(), [foo, bar, baz]))!
<StevenK> Let me get a rusty spoon
<wgrant> Yes, lambda syntax is foul :(
<lifeless> any(map(methodcaller('is_empty'), [foo, bar, baz]))
<StevenK> wgrant: http://pastebin.ubuntu.com/593012/
<lifeless> if you're going to go functional, go all the way
<wgrant> lifeless: Bah, forgot about methodcaller.
<wgrant> I normally only have cause to use attrgetter and itemgetter.
<wgrant> StevenK: That is using __nonzero__.
<wgrant> Which I discourage, and lifeless forbids.
<lifeless> we're trying to migrate to stormresultsets everywhere
<lifeless> because of the storm migration
<wgrant> Right.
<lifeless> using __nonzero__ is a step backwards
<lifeless> not to mention horribly confusing
<jtv> wgrant: in your top-tags branch, is "factor" used for anything but sorting?
<wgrant> jtv: It's used as size in the tag cloud.
<jtv> AFAICS you could make the "bonus" that official tags clearer by adding max_count/2 to count, instead of adding 0.5 to count/max_count.
<lifeless> wwwwow
<lifeless> that was a fun test blowup
<lifeless>  File "/var/launchpad/tmp/eggs/bzr-2.2.2_lp_2-py2.6-linux-x86_64.egg/bzrlib/lazy_import.py", line 97, in _cleanup
<lifeless>    del self._factory
<lifeless> AttributeError: _factory
<lifeless> ------------
<jtv> wgrant: although maybe that just moves it closer towards the "how do we arrive at this" side at the cost of the "what does this imply for the tag clould" side.
<jtv> *cloud
<StevenK> Is methodcaller in 2.6?
<spiv> lifeless: threading race, I guess?
<wgrant> StevenK: Yes.
<lifeless> spiv: https://bugs.launchpad.net/bzr/+bug/396819
<_mup_> Bug #396819: AttributeError: _factory - lazy imports are not threadsafe <lazy-imports> <Bazaar:Confirmed> < https://launchpad.net/bugs/396819 >
<spiv> lifeless: threads racing to replace a lazy import stub with a real object I guess.
<jtv> Ye olde concurrent singleton update problemme
<spiv> lifeless: yeah
<StevenK> wgrant: Ah, it's hiding in operator. Sneaky.
<StevenK> Which leads to: ForbiddenAttribute: ('is_empty', <storm.sqlobject.SQLObjectResultSet object at 0xe6cab90>)
<lifeless> ughhhh
<lifeless> headdesk headdesk headdesk headdesk
<wgrant> StevenK: Ah, I guess SQLObjectResultSet doesn't have an is_empty :/
<wgrant> Or maybe it's just not on the interface.
<lifeless> its on the interface
<lifeless> bah
<lifeless> not on
<lifeless> it is on the class
<lifeless> line 563 of sqlobject.py
<lifeless> storm/zope/interfaces.py is bust
<lifeless> if you want to be lazy you can add is_empty manually in lp
<adeuring> good morning
<jtv> hi adeuring
<adeuring> good evening jtv!
<jtv> new year's eve :)
<adeuring> jtv: so, have a nice party!
<jtv> Thanks
<bigjools> good morning all
<wgrant> Morning bigjools.
<wgrant> jtv: Reviewed.
<jtv> wgrant: thanks, that was fast.  Reviewed yours as well.
<wgrant> jtv: Thanks.
<lifeless> stub: thinking about a fact table for bug tags
<lifeless> stub: I think we might also cater for milestones & bug status at the same time
<lifeless> more rows (because more unique combinations) but still very few
<stub> on phone
<lifeless> stub: sure, no worries
<wgrant> lifeless: But milestone and status are on BugTask... are they a problem?
<lifeless> wgrant: so will the bug tag sumamry
<wgrant> lifeless: When are milestones and statuses slow?
<wgrant> We can index directly into those.
<lifeless> wgrant: all the time
<lifeless> we have three portlets
<lifeless> all of which have now had *an* optimisation pass done
<lifeless> and none of which are snappy.
<lifeless> 53 /  101  Distribution:+bugtarget-portlet-tags-content you have just done
<wgrant> Ah, right, that sort of query.
<wgrant> I was thinking of searches.
<lifeless> wgrant: status is slow when the scope is huge
<lifeless> we've more work to do there, but I was thinking about the aggregate case specifically
<stub> yo
<stub> lifeless: This is moving towards a bugSearch table, which I've considered before.
<stub> ie. throwing together the information we query on in a denormalized format suitable for querying
<lifeless> stub: hmm
<lifeless> stub: so we'd what - add a link to the summary row in the bugtask; so we could pick the summaries we want and search for bugs matching ?
<stub> (bug, status, bugtag[], milestones[], fti, ...)
<lifeless> stub: ah so - no this is different
<lifeless> stub: opposite direction of denorm
<stub> lifeless: Something like that. GIN indexes on the arrays should allow us to match rows containing certain milestones
<stub> k
<lifeless> stub: we may well need both
<lifeless> stub: this is a bug that may benefit from a bugsearch table - 757426
<lifeless> bug 757426
<_mup_> Bug #757426: Distribution:+bugs timeout with combinator ALL <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/757426 >
<lifeless> (warning, eye bleeding and gouging may result)
<lifeless> also bug 750445
<_mup_> Bug #750445: MaloneApplication:+bugs timeout on 'any tag {pcert, blockshwcert}' <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/750445 >
<stub> lifeless: oic. This is purely for caching the aggregate values so we don't have to calculate on the fly.
<lifeless> yeah
<stub> lifeless: Sure. We have some similar structures already btw.
<lifeless> I'm filing a bug describing the problem clearly now
<lifeless> if there is an existing one I should update, I'd be delighted to
<lifeless> (existing structure)
<stub> karmacache and karmatotalcache are the ones I can think of atm.
<lifeless> we have some stuff shoved into pillars too
<lifeless> which I think is a locking mistake
<wgrant> And archives.
<wgrant> And D(S)PC
<jtv> wgrant: responded to your review â please let me know if you agree.  Also, since this is EOW for me, any chance you could assume the Q/A burden for the combined changes to publish-ftpmaster?
<wgrant> jtv: Sure.
<jtv> Thanks.
<wgrant> I was going to QA it myself anyway.
<wgrant> Something like this..
<wgrant> jtv: Looks good.
<wgrant> lifeless: Still around to mentor https://code.launchpad.net/~jtv/launchpad/db-bug-752279/+merge/57277?
<jtv> wgrant: thanks.  Yes, I've had to rely on you for the soyuz Q/A anyway â the difference is I may not be online to change the bug tags.  :)  Note that the branch actually attaches to two bugs.
<lifeless> wgrant: its late; my eyes are trying to shut :)
<lifeless> wgrant: I can look first thing tomorrow, or you can nab someone else to mentor
<lifeless> sorry
<rvba> I'd be happy if someone could give me a hand with the branch I'm working with. I've a strange problem with a test and I think the problem is either:
<rvba> a) something stupid that I don't see anymore because my nose is too close to my screen
<rvba> b) something wrong with the way feature flags are setup in tests
<rvba> here is the MP (still in the works) https://code.launchpad.net/~rvb/launchpad/dds-stats-portlet/+merge/57280
<rvba> my problem is that this test "./bin/test -cvv test_series_views test_differences_portlet_all_differences" is failing basically the feature flag is not set.
<rvba> if I remove the line "tal:condition="request/features/soyuz.derived-series-ui.enabled">" the test passes
<rvba> the feature flag in the test is set the same way it's set in other tests (I think) and that's why I don't understand why it's not working.
<rvba> Putting a breakpoint inside the feature flag controller code told me that, for some reason, the NullFeatureController is used ... I guess that's the reason why the feature flag is not set and the test is failing ... but can't figure out how to do it properly.
<lifeless> night all
<stub> jtv: So at the moment, we block dblooptuner stuff on long running transactions to avoid bloat.
<jtv> stub: I'm not complainingâonly about the check being cluster-wide rather than local to the database.
<stub> jtv: Instead, I wonder if it is sane to occasionally check dead tuples on a specified list of tables, and if there is a large enough number, block until autovacuum can clean them up?
<jtv> Risky.
<stub> We have had some huge bloat on indexes and tables that have been migrated, despite the check for long running transactions. I think autovacuum is backing off because the batch job wants access.
<jtv> I mean, if we get that wrong, isn't there a danger that we block just because autovacuum doesn't think it's worth cleaning up yet?
<stub> Yes, so we need a higher threshold than autovacuum.
<jtv> Once things get that bad though, is it really a matter for the dblooptuner?  It sounds as if it might be more something we should raise alarms about and fix.
<jtv> (Once the thresholds are high enough, obviously)
<stub> When we raise alarms, the bloat has already happened.
<stub> And cleaning it up is sometimes impossible (bloated primary key indexes for instance...) without rebuilding nodes.
<jtv> I see.
<stub> I suspect there is a flaw somewhere, or else autovacuum would check dead tuple counts rather than #updates. Maybe the dead tuple count doesn't get updated until autovacuum has had a look?
 * stub goes to do some research
<stub> Maybe I just need to stick in delays, which would suck.
<jtv> This is what I mean: I like the idea, but not the way you need to model the database's internals somewhat correctly to make it robust.
<stub> jtv: The alternative is to keep the existing long running transaction checks (to cope with vacuum not doing anything useful due to the backups being in progress) and manually vacuum in dblooptuner on occasions.
<jtv> BTW I'm not as deeply into this as you are, but it does sound sensible that we don't know the number of dead tuples until we've done a vacuum.  And dead tuples don't really matter I guess until a "vacuum full" because until then you can't reclaim the space anyway.
<jtv> I don't suppose there's some subtle tweak to count transaction time from the time an actual transaction id is assigned?
<stub> dead tuples matter because when dead tuples are not converted to free space quickly enough we get bloat.
<stub> jtv: Not sure if they are actually different. I just use what pg_stat_activity gives me, and if it is wrong tough because there is no other source of information :)
<stub> I suspect they are the same - <IDLE> connections don't have a transaction start time
<jtv> Late transaction id assignment was added pretty recently â 8.3 or so I think.  Shame.  :(
<jtv> Nowadays, transaction ids only get assigned when the transaction starts making changes.
<stub> oic.
<jtv> Which I'm guessing a backup doesn't do.
<stub> backup needs a consistent snapshot of the database, so vacuum can't clean up stuff until the backup transaction is completed.
<deryck> Morning, all.
<jml> deryck: good morning!
<jml> deryck: I wish to talk with you
<jtv> bigjools: oh!  I just realized I could just compare using "IS DISTINCT FROM."  That'll treat nulls as equal, but we're not going to get two nulls there.
<deryck> jml, ok, let's do.  mumble or chat?
<bigjools> jtv: that's the same as the old code?
<jml> deryck: chat.
<jtv> bigjools: yup, does the same thing.
<bigjools> jtv: the old *buggy* code?
<jtv> bigjools: no, not _that_ old :)
<jtv> This is internet time.
<jtv> It does the same as the "IS NOT TRUE" expression.
<jtv> But more succinctly.
<jtv> (And FTR it passes tests)
 * bigjools is confused!
<jtv> bigjools: IS DISTINCT FROM will return True or False.
<jtv> So "'foo' is distinct from null" â true, whereas "'foo' <> null" â null
<bigjools> jtv: what are you comparing?
<jtv> version numbers.
<jtv> One or the other may be null, that that's considered "different."
<jtv> *and that's considered "different."
<jtv> bigjools: oh I see what you mean nowâit's the same operator that the old, deployed code uses.
<jtv> Sorry, was a bit rushed.
<bigjools> jtv: ok, now it makes more sense :)
 * jtv blushes
<bigjools> jtv: approved the MP, thanks
<jtv> thanks
<LPCIBot> Project devel build #630: FAILURE in 5 hr 11 min: https://lpci.wedontsleep.org/job/devel/630/
<abentley_> deryck: http://pastebin.ubuntu.com/593119/
<abentley> bac: I am trying to extend lazr.FormOverlay, but getting a rendering bug.  I understand you've navigated these murky waters before.  Could you give me a hand?
<bac> abentley: sure.  the work we did is in registry/javascript/structuralsubscription.js
<bac> abentley: we didn't actually extend FormOverlay, though.  just instantiated and used one
<abentley> bac: This is what I have so far: http://pastebin.ubuntu.com/593119/
<bac> abentley: you're going down a completely different path.  it is probably the right thing to do i just haven't done it before.
<abentley> bac: Okay, thanks.
<jkakar> Can anyone think of a reason why my launchpadlib-based code is talking to staging when I've specified LPNET_SERVICE_ROOT as the endpoint...?  This is on natty with the latest updated.
<sinzui> jkakar: I am not sure, but I stopped using LPNET_SERVICE_ROOT and non-versioned scripts
<jkakar> sinzui: What do you use instead of LPNET_SERVICE_ROOT?
<sinzui> jkakar: I use this to be precise:
<sinzui> lp = Launchpad.login_with(
<sinzui>         'testing', service_root='https://api.launchpad.net', version='devel')
<jkakar> I have code like this: credentials = Credentials(); credentials.load(<path-to-credentials>); return Launchpad(credentials, LPNET_SERVICE_ROOT, getCachePath())
<jkakar> sinzui: ^^ I should be using login_with instead of that?
<sinzui> jkakar: I think that is a very old way of setting up credentials and oauth
<jkakar> sinzui: Okay, I'll try login_with, thanks.
<sinzui> I do not see LPNET_SERVICE_ROOT in the modern version of launchpadlib
<sinzui> I see staging and edge, and I know edge is dead
<jkakar> sinzui: It's in launchpadlib.uris.
<jkakar> sinzui: With a bunch of other odd looking stuff. :)
<sinzui> I see. Yes, that is odd
<bigjools> james_w: hey, got a moment?
<sinzui> I like to use the real URL so that I do not need to guess what I am talking too
<bigjools> jkakar: hello there, still hanging around? :)
<jkakar> bigjools: Hi :)
<james_w> bigjools, yeah
<jkakar> sinzui: That seems to have done the trick, thanks for the advice.
<bigjools> james_w: remember we talked about pockets in derived distros?  Would it be a blocker for Linaro if you had to use the same pocket paradigm as Ubuntu?
<james_w> bigjools, I don't think so
<bigjools> james_w: you made my day
<james_w> I expect OEM will be the difficulty with things like that
<bigjools> yes, they are
<bigjools> but we're like Cylons, we have a plan
<jml> gah, bad punctuation.
<bigjools> good night folks
<jcsackett> is anyone available to review https://code.launchpad.net/~jcsackett/launchpad/private-bugs-404
<jcsackett> it's some template changes and a test.
<jcsackett> sorry, mp link is https://code.launchpad.net/~jcsackett/launchpad/private-bugs-404/+merge/57244
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #631: FIXED in 5 hr 7 min: https://lpci.wedontsleep.org/job/devel/631/
<sinzui> A user has errors building a recipe. https://answers.launchpad.net/launchpad/+question/152534
<sinzui> ^ I think the solution is to delete then recreate https://code.launchpad.net/~vcs-imports/bugzilla/main so that the branch is modern
<jelmer> sinzui: the branch should be upgraded
<sinzui> jelmer: how do I upgrade a ~vcs-imports branch
<jelmer> sinzui: I've been wondering the same thing :-/
<jelmer> I think Bazaar experts can do upgrades, or at least LOSAs
<sinzui> jelmer, the bzr and Lp teams might all have permission to branch, upgrade, and push to ~vcs-imports...
<sinzui> which begs the question, if so, why haven't we done it for the 1000's of branchs
<jelmer> sinzui: the problem is that some users of ~vcs-imports might still be using an older version of bzr that doesn't support the 2a format
<jelmer> sinzui: and each upstream URL can only have one matching Launchpad branch
<sinzui> jelmer :(
<sinzui> I think users making recipes that using different formats will then need to do all the work to make alternate branches to ensure everything is the same format
<jelmer> sinzui: it's impossible to register alternate branches at the moment as launchpad only allows one vcs import for a particular upstream URL :-/
<sinzui> right
<sinzui> so the 1000s of imported branches might prove to be useless. Developers are keeping current, but we keep the formats to support old distros
<sinzui> I do mean old distros because a developer can always update his tools
<jelmer> sinzui: perhaps we should bring this up on the list
<sinzui> I think so. I will
<jelmer> sinzui: at this point or within a year from now, it's probably reasonable to require users to upgrade. It's something that we will keep hitting though as bzr adds new features
<jelmer> It might well be something we can fix on the Bazaar side, by not making format changes as rigorous as they are at the moment
<sinzui> Well I think the issue is a little different. Most of the vcs-imports exist to feed Ubuntu, and it only cares about new code. I think that is true of all developers. Lp should be using the latest version. bzr and Lp both need to inform users where they are using old tools and how to upgrade
<lifeless> moin
<lifeless> gary_poster: hi
<gary_poster> lifeless, hi
<timrc> sinzui: We talked awhile ago when I reported a problem with attempting to set a PPA private via launchpadlib (https://pastebin.canonical.com/46008/).  The problem was linked to https://bugs.launchpad.net/bugs/724522
<_mup_> Bug #724522: Retried transactions retry forever <canonical-losa-lp> <qa-ok> <regression> <Launchpad itself:Fix Released by wgrant> < https://launchpad.net/bugs/724522 >
<timrc> sinzui: it's still not possible to make a ppa private via the apiâ¦ are there plans to do this? or maybe there is a way, now, and I'm just unfamiliar with it
<sinzui> timrc: I do not know the  answer off hand
<sinzui> oh, maybe I do. I think we verified that no, it cannot be done now
<sinzui> timrc: There are no plans at the moment to allow setting a PPA private over the api. The state is bool, but making the PPA private actually requires several things to happen, among them you would need to provide a secret
<cody-somerville> sinzui, I believe the secret is automatically generated (it is for the form).
<timrc> sinzui: So how does launchpad achieve this?
<sinzui> The view is doing lots of things
<lifeless> sinzui: why can't the api generate a random secret on the server ?
<sinzui> A methods must be added to archive that allows an archive to be made private
<sinzui> ^ lifeless My answer remains the same
<lifeless> sinzui: possible in principle but not being worked on right now ?
<sinzui> lifeless: yes. I am channeling someone's voice from Soyuz.
<lifeless> sinzui: cool
<sinzui> I wanted to do the same and it was explained to me that there are missing steps to accomplish the bool state we see
<sinzui> Oh, I can see I have search for this bug before
<lifeless> bug 724740 ?
<_mup_> Bug #724740: setting a ppa private fails for commercial admins <oem-services> <Launchpad itself:Triaged> < https://launchpad.net/bugs/724740 >
<sinzui> lifeless: timrc: bug 724740
<_mup_> Bug #724740: setting a ppa private cannot be done over the API <oem-services> <Launchpad itself:Triaged> < https://launchpad.net/bugs/724740 >
<cody-somerville> So sounds like a simple fix.
<sinzui> bugzilla uses bzr now?
<lifeless> yes
<lifeless> davemiller (who registered launchpad.net/bugzilla and is a prime mover upstream) spearheaded a migration
<timrc> cody-somerville: you volunteering? :) *bats eyelashes*
<timrc> cody-somerville: while you're add it can you add support to enable arm builders?
<timrc> s/add/at/
<sinzui> lifeless: I think I am running in circles here since I am just trying to get our copy on format 2a so that a recipe will build
<lifeless> I think we need a mirrored branch
<lifeless> bzr://bzr.mozilla.org/bugzilla/trunk is trunk
<sinzui> yep: https://code.launchpad.net/~registry/bugzilla/trunk
<sinzui> I am waiting for the import to start
<lifeless> we have a typo
<sinzui> This is a unholy recipe using an import from github and buzilla
<lifeless> isHosted
<lifeless> on https://code.launchpad.net/~registry/bugzilla/trunk/+edit
<sinzui> I bet that is string concatenation in the interface. I fix my own mistake like that a few months ago
<lifeless> wow, so why are they using a github branch as well...
<lifeless> oh, I see
<lifeless> their branch should be in the packaging namespace
<lifeless> but I guess code imports don't support that yet
<sinzui> lifeless: the user only knows git and is a github user. I want to try new lp features
<sinzui> This has been an adventure moving from the concept of "I have code" to "lets share code"
 * sinzui just fized ishosted
<lifeless> thanks!
<sinzui> I have a trivial branch open as I make movies of how to use Launchpad
<lifeless> can anyone make heads or tails of Bug 759160
<_mup_> Bug #759160: normal search <Launchpad itself:New> < https://launchpad.net/bugs/759160 >
<sinzui> The experience is not pleasant. I keep feeling like I have to apologies tp users
<sinzui> looks like batching does not scale with that app. But is that app using Lp's API?
<lifeless> sinzui: I have *no* idea
<sinzui> ah
<sinzui> retarget the bug to webtrees
<sinzui> That project was registered recently.
<sinzui> The owner is not too familar with Lp
<groo_> hi/2 all
<groo_> im already using LP dev
<groo_> is there a branch i could use to be able to use git nested trees? and dont get : The repository you are fetching from contains submodules. To continue, upgrade your Bazaar repository to a format that supports nested trees, such as 'development-subtree'
<sinzui> groo_: I do not think bzr supports nested trees
<sinzui> groo_: See http://wiki.bazaar.canonical.com/NestedTrees
<lifeless> groo_: jelmer, in #bzr, can tell you more about where the progress has gotten to.
<groo_> tks lifeless, sinzui
#launchpad-dev 2011-04-13
<wgrant> sinzui: How close is the merge QA?
<lifeless> wgrant: thanks for backing out the batchnav landing
<wgrant> lifeless: I see you've fixed p-r-f. Thanks.
<wgrant> I'll be looking at the API timeout regression this morning.
<wgrant> Once I do a bit more QA.
<poolie> hi all
<lifeless> wgrant: awesome, thanks.
<lifeless> poolie: morning
<LPCIBot> Project devel build #632: FAILURE in 4 hr 46 min: https://lpci.wedontsleep.org/job/devel/632/
* StevenK changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: StevenK | https://code.launchpad.net/launchpad-project/+activereviews
<wgrant> poolie: OOPS-1929D68 (from bug #736421) doesn't sound like a qastaging OOPS...
<_mup_> Bug #736421: Person:+delete timeouts : Connect the UI the merge jobs <merge-deactivate> <qa-needstesting> <timeout> <Launchpad itself:Fix Committed by sinzui> < https://launchpad.net/bugs/736421 >
<poolie> maybe i used the wrong domain?
<wgrant> poolie: Could you test on qastaging again?
<poolie> i will
<wgrant> We need to get this QA'd ASAP.
<wgrant> Thankks.
<poolie> i blame url completion
<wgrant> Heh.
<poolie> either that or sunspots
<poolie> hm so qastaging still thinks it has a ppa, even though i deleted the ppa from lpnet yesterday?
<wgrant> poolie: qastaging updates manually, every month or so.
<wgrant> staging updates once a week.
<wgrant> s/updates/restores/g
<wgrant> You might need DB hacking to get the PPA deleted :(
<wgrant> But that is easily done.
<wgrant> poolie: So, request deletion of the PPA, and we'll SQL it.
<poolie> ppa deletion is broken only on qas?
<wgrant> Since the publisher doesn't run on qas, so it will stay DELETING forever.
<wgrant> The UI sets the status to DELETING, and then the publisher comes along, deletes it from disk, and sets it to DELETED.
<wgrant> qas doesn't have a publisher.
<poolie> right
<wallyworld_> lifeless: is this one ok now? https://code.edge.launchpad.net/~wallyworld/launchpad/disabled-project-bugs-shown/+merge/57265
<lifeless> wgrant: rereview ^ and I will mentor
<wgrant> lifeless: I'm rereviewing another of wallyworld_'s branches at the moment, but I'll get to it soonish.
<lifeless> wgrant: no panic
<wgrant> Not for this, at least...
<StevenK> wgrant: Would you accept http://pastebin.ubuntu.com/593393/ , or is it still dangeous?
<StevenK> *dangerous
<wgrant> StevenK: Still dangerous.
<StevenK> :-(
<StevenK> wgrant: Even though they don't share an archive?
<wgrant> StevenK: The initialisation and the builds it creates will work, but will live you in a bad state.'
<wgrant> We need to either decided that that bad state is OK, or fix it.
<StevenK> wgrant: What sort of bad state?
<wgrant> StevenK: The source has some binaries identical, some different.
<StevenK> wgrant: But we can even get that behavior with Ubuntu.
<wgrant> StevenK: Oh?
<wgrant> You can't have two different binaries with the same name in the same archive.
<wgrant> So no, you can't.
<StevenK> wgrant: Say 'libuser' is uploaded to Maverick, and it builds successfully on amd64, and i386, and fails on powerpc. The powerpc build isn't given back during Maverick's cycle. Natty opens, and the libuser amd64 and i386 binaries are copied in, and the powerpc build is re-created and is successful. So now we have some binaries identical and one different.
<wgrant> StevenK: Not different: unique.
<wgrant> The source is still copiable. There is no binary conflict.
<StevenK> Your concern is that if a series is derived while say, libuser is building we won't be able to copy the source?
<wgrant> Something like that.
<wgrant> This may be acceptable, but we need to think very carefully.
<wgrant> And see if there is a way we can fix it, because it's a problem for PPAs too.
<StevenK> wgrant: So even with that change, it's -1?
<wgrant> StevenK: Until I see discussion of the potential damage, yes.
<wgrant> Primary archive corruption is not something to be taken lightly.
<StevenK> I'm still failing to see how two seperate archives can impact each other.
<wgrant> They will not directly.
<StevenK> wgrant: Okay, would you mind explaining how this damage could occur? I'm afraid I don't see it yet.
<wgrant> StevenK: I can't think of a common case in this immediate feature that would be broken by the differing binary sets. But it is going to be very confusing, has potential to break future workflows, and is a problem that already plagues PPAs.
<wgrant> It is possible that the confusion is acceptable. We could declare that it is fine.
<wgrant> But we need to solve this at some point, and now is a good time to do that.
<wgrant> Until we've had this discussion it should not be permitted.
<wgrant> Copies will not work unless the copied source and series has a superset of the binaries that are in the target archive.
<wgrant> So creating conflicting sets makes some copies impossible.
<StevenK> wgrant: So, my current thought is this: I have a bug that states that derivation is impossible while builds are pending. The change I'm proposing helps with that, but I'm happy to open a new bug echoing your concerns and you, Julian and I can discuss it and come up with a solution?
<wgrant> StevenK: It helps with that by potentially creating bad state. The correct solution should be discussed in the bug.
<StevenK> Right, I can see that I'm only going to please you by having a fully correct solution.
<StevenK> wgrant: I've reverted to the __nonzero__ call that was the other push-back, do I need to fix that properly too?
<wgrant> StevenK: I will not let potential breakage in without a discussion of the potential breakage!
<wgrant> Particularly when it comes to primary archives.
<wgrant> Primary archives are serious business.
<StevenK> wgrant: I'm just frustrated, we've been arguing about this for hours!
<StevenK> wgrant: So, yes __nonzero__, or no __nonzero__?
<wgrant> StevenK: I do not strictly forbid it, but I suspect lifeless will.
<StevenK> lifeless: ^
<lifeless> hi guys
<StevenK> wgrant: The reason I ask was we saw that SQLObject doesn't provide .is_empty()
<lifeless> whats up
<lifeless> StevenK: it provides it
<lifeless> StevenK: the interface declaration is bogus
<lifeless> I looked this up yesterday
<StevenK> lifeless: I couldn't do it with methodcaller() yesterday when I tried?
<lifeless> StevenK: thats correct, because the security proxy is stopping you calling the method
<StevenK> Oh, sigh
<lifeless> StevenK: your options are:
<lifeless>  - removeSecurityProxy
<lifeless>  - allow the attribute in a lp config.zcml
<lifeless>  - fix the interface declaration in storm/zope/interfaces.py
<StevenK> The last one sounds like the proper solution
<StevenK> But likely requires a new storm
<lifeless> StormResultSet does not have __nonzero__, and I think its likely it won't ever, given the overhead of accidental evaluation.
<lifeless> I'm having one of those days
<StevenK> lifeless: lp:storm is fine to fix?
<lifeless> I'll put a new storm together for you after I enicreamenate
<StevenK> After you ... what?
<lifeless> StevenK: lp:storm will break lp on two counts: datetime marshalling has changed, and we're running with a WITH patch that is still under review.
<lifeless> enicecreaminate
<StevenK> Oh
<StevenK> No wonder dict(1) didn't know the word
<wallyworld_> poolie: 740338 (dup of 513591) doesn't seem to be an issue anymore. but 735290 still is. you last commented on the first bug mentioned on 2011-04-10. did you see that is was broken at that time?
<wallyworld_> just trying to see if something has been fixed in the past few days
 * poolie looks
<poolie> wallyworld_: hi
<wallyworld_> yello
<poolie> so jools was hitting bug 513591, and because of that he then hits bug 735290
<_mup_> Bug #513591: Assignee selector on +filebug trashes form input due to missing javascript <dhrb> <lp-bugs> <regression> <story-ajaxify-dupe-finder> <Launchpad itself:Triaged by wallyworld> < https://launchpad.net/bugs/513591 >
<_mup_> Bug #735290: changing Project drop down in project group +filebug loses all your work <ajax> <bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/735290 >
<wallyworld_> poolie: there around 9 dups of that bug
<poolie> right
<wallyworld_> poolie: the assignee selector is not blue anymore - the ajax link works
<poolie> i think it's the 735290 part that makes it super annoying
<poolie> right
<wallyworld_> on qastaging and lp.net
<poolie> but 735 can still be hit in other cases, as described on that bug
<poolie> there are several other things that are very nearly dupes of it
<wallyworld_> poolie: yes, so i'll fix the project drop down
<wgrant> wallyworld_: Still blue for me.
<wallyworld_> and others that i can see
<wallyworld_> wgrant: ??
<poolie> well, really https://bugs.launchpad.net/launchpad/+bug/553946 is the key thing
<_mup_> Bug #553946: JavaScript breaks ability to recover +filebug form data <bugjam2010> <lp-bugs> <story-ajaxify-dupe-finder> <Launchpad itself:Triaged> < https://launchpad.net/bugs/553946 >
<poolie> https://bugs.launchpad.net/launchpad/+bug/553946/comments/4
<_mup_> Bug #553946: JavaScript breaks ability to recover +filebug form data <bugjam2010> <lp-bugs> <story-ajaxify-dupe-finder> <Launchpad itself:Triaged> < https://launchpad.net/bugs/553946 >
<wallyworld_> wgrant: works fine for me. wonder what gives?
<wgrant> wallyworld_: The assignee selector on +filebug?
<wallyworld_> wgrant: yes
<wgrant> Which browser?
<wallyworld_> wgrant: ff4
 * wallyworld_ think he will have to open ff3
<wgrant> Intriguing! FF works.
<wgrant> Only broken in Chromium.
<wallyworld_> zh
<wallyworld_> ah
<wgrant> I didn't realise that.
<poolie> i would be thrilled if that is fixed
<wallyworld_> wgrant: still, even with js off or whatever, it's bad that people lose their work
<wgrant> Definitely.
<wallyworld_> wgrant: not sure how to solve it though in the non js case
<poolie> what happens in the non jscase?
<poolie> a separate form?
<wallyworld_> poolie: try ff :-)
<wallyworld_> poolie: i think a separate page, yes
<wallyworld_> poolie: haven't tried without js recently
<wgrant> wallyworld_: The link should be hidden without JS.
<poolie> anyhow imbw but i think the key thing there is not to dynamically add the textarea
<wallyworld_> wgrant: forcing people to type directly into the assignee text field?
<wgrant> wallyworld_: Yes.
<wallyworld_> poolie: i haven't looked yet at the implementation, but you're talking about the deleting of people's work when the project changes when you refer to the textarea?
<thumper> poolie: want a catch up chat?
<wgrant> No other way to do it, really...
<wallyworld_> wgrant: yeah, agreed
 * wallyworld_ wonders why it broke on chrome. will look into that
 * wallyworld_ needs food first
<poolie> thumper: sure
<poolie> want to try voip?
<thumper> poolie: sure
<thumper> poolie: mumble or sip?
<poolie> what's my name again?
<thumper> poolie: Martin
<poolie> 7806
<poolie> on sip
<poolie> can you try calling me?
<thumper> I can tru
<thumper> try
<lifeless> StevenK: https://code.launchpad.net/~lifeless/storm/bug-759384 - am merging it up to our storm now
<StevenK> lifeless: Are you self-reviewing, or are you looking for a +1?
<lifeless> StevenK: neither, just letting you know
<lifeless> StevenK: you can apply that locally to your unpacked storm egg to do your testing without waiting for me
<lifeless> wgrant: what was that egg-info chicken
<lifeless> nvm
<StevenK> lifeless: Love the docstring error that you corrected for __nonzero__.
<lifeless> StevenK: yah. c&p FTW
<lifeless> dist/storm-0.18.0.99-lpwithnodatetime-r392.tar.bz2 is in the download cache
<lifeless> StevenK: do a bzr update there, and update the version in versions.cfg; then bin/buildout and you are good to go.
<StevenK> I shall, after lunch
<lifeless>  https://bugs.launchpad.net/launchpad-project/+bugs?search=Search&field.status=New triage is falling behind
<wgrant> It would help if my bookmark was launchpad-project instead of just launchpad.
 * wgrant fixes.
<wgrant> I guess bug #758758 wants to be moved to bzr?
<_mup_> Bug #758758: bzr out of memory during auto-build <Launchpad itself:New> < https://launchpad.net/bugs/758758 >
<wgrant> poolie: ^^
<lifeless> iz dupe I think
<wgrant> Of a fix-released bug.
<wgrant> So not really.
<wgrant> wallyworld_: Rereviewed your picker branch. Sorry about the delay.
<wallyworld_> wgrant: np. i've had other stuff to keep me busy
<wgrant> Looking at your disabled projects one now.
<StevenK> lifeless: Your storm stuff has landed?
<lifeless> StevenK: no
<lifeless> StevenK: I've added an egg with it to the downloadcache
<lifeless> you just need to update your versions.cfg to use it
<lifeless> I've proposed a branch to storm
<lifeless> but we're running a fork anyway
<StevenK> Oh, you have to got to be kidding me.
<StevenK> IDistroSeries.section is a SQLObjectResultSet, but IDistroSeries.components is a list.
<StevenK> s/\(section\)/\1s/
<StevenK> And the XXX in IDistroSeries.components makes me sob.
<lifeless> oh grah
<lifeless> 'incomplete bugs' links to 'expirable bugs'
<lifeless> THESE ARE NOT THE SAME
<wgrant> StevenK: components is a list for cachability.
<lifeless> wgrant: so, in cases like this - task on bzr [core work happens there], task on lp [have to deploy the resulting work]
<StevenK> wgrant: But then I can't just call is_empty() on it :-(
<wgrant> StevenK: Yeah :(
<wgrant> lifeless: I guess.
<StevenK> wgrant: When IDistroSeries get Stormified, that point is moot, isn't it, since ResultSets can be easily cached?
<lifeless> wgrant: its what we do elsewhere
<lifeless> StevenK: ResultSet doesn't cache
<lifeless> StevenK: and you can use storm queries on sqlobject base classes
<lifeless> StevenK: so I'm confused about what you're saying/the problem you're facing
<StevenK> lifeless: I'm just wondering if we can drop the list() in IDistroSeries.components
<wgrant> No.
<lifeless> StevenK: only if we stop using it
<StevenK> Sorry, I wasn't being clear, I don't mean now, I mean 'at some point'
<lifeless> StevenK: thats what I mean
<lifeless> StevenK: if we do a persistence layer like I'm threatening, we'll get rid of all query objects in model code
<lifeless> StevenK: (so it wouldn't need listifying, it would /be/ a list)
<lifeless> StevenK: alternatively, if we setifiy all our logic, we don't need the attribute
<lifeless> StevenK: (but this makes quick-small hacks harder to write)
<poolie> hi wgrant
<wgrant> Hi poolie.
<StevenK> wgrant: +localpackagediffs on staging makes me sad
<StevenK> Bleh, more import policy violations
<StevenK> wgrant: Can you look over https://code.launchpad.net/~stevenk/launchpad/ids-bugfixes/+merge/57270 again?
<wgrant> StevenK: The lack of base versions?
<wgrant> bigjools and I were looking at that last night.
<StevenK> wgrant: That, but mostly the large number of DSDs with the same version in both Natty and Sid.
<wgrant> Ah, yes.
<wgrant> Has that fix landed yet?
<StevenK> jtv: ^
<wgrant> He's EOWed.
<StevenK> Ah, bah
 * StevenK checks kanban, then
<wgrant> Ahh.
<wgrant> He ec2'd it 10 minutes before my testfix.
<wgrant> So it probably just missed.
<wgrant> I might throw it through again.
<wgrant> Or you could. We probably don't want to wait until Monday.
<StevenK> Yeah, I'm happy to.
<StevenK> It's tiny
<wgrant> Thanks.
<wgrant> lifeless: https://code.launchpad.net/~wallyworld/launchpad/disabled-project-bugs-shown/+merge/57265 and https://code.launchpad.net/~wallyworld/launchpad/assign-non-contributor/+merge/55869 would both like to be mentored.
<lifeless> 1.4 second tag summaries so far - with visibility but doublecounting on redundant subscriptions
<wallyworld_> wgrant: poolie: that assignee link on the bug page works fine for me in Chrome :-/ wonder why it's broken for you?
<poolie> which one?
<poolie> on filebug?
<wallyworld_> poolie: yes. the one that says "Find..." in blue. except for me it doesn't. it says "Choose..." in green :-)
<wallyworld_> so, so far i'm unable to see the reported problem
<poolie> it is blue for me and sends me to /people
<poolie> which seems kind of weird for a non-js fallback
<poolie> but i think i've never wanted to click that
<poolie> i know the ids of every person i want to assign bugs to
<wgrant> wallyworld_: Let me see.
<poolie> my complaint with this page is just the more general issue that it breaks history
<wallyworld_> poolie: agreed. i'm just curious as to why we're seeing different behaviour. but i think i need to add an option to not show the Find... if there's no javascript
<wgrant> wallyworld_: Which URL?
<poolie> that would be fine
<wallyworld_> wgrant: lp.net/someproject/+filebug
<wallyworld_> lp.dev even
<StevenK> wgrant: Right, after fighting with utilties/ec2, I've tossed it successfully.
<wgrant> wallyworld_: Which domain? launchpad.dev, or bugs.launchpad.dev?
<wallyworld_> wgrant: sorry, bugs
<lifeless> +            qs = Y.lp.client.append_qs(qs, 'name', vocabulary);
<lifeless> 300	+            qs = Y.lp.client.append_qs(qs, 'search_text', search_text);
<wgrant> Although neither work for me, we've had cross-domain JS problems like this before.
<lifeless> 301	+            qs = Y.lp.client.append_qs(qs, 'batch', BATCH_SIZE);
<lifeless> 302	+            qs = Y.lp.client.append_qs(qs, 'start', start);
<lifeless> does the javascript client manually track the batching implementation ?!
<wgrant> lifeless: Yes, pickers have their own batchnav UI.
<wgrant> I didn't realise they did it like that, though :/
 * lifeless deadhesks
<StevenK> Haha
<jtv> wgrant: a db-devel merge caused some trouble, so it's back into EC2
<wgrant> jtv: It's a race!
<poolie> wallyworld_: oh btw i like the assignee thing a lot
<wallyworld_> wgrant: poolie: actually, it works for me using bugs.lp.dev but fails on bugs.lp.net. so something must have been fixed
<StevenK> jtv: Ah, should I stop mine? :-)
<wallyworld_> poolie: me too :-)
<poolie> (i realize there are details to sort out but thumbs up anyhow)
<wgrant> wallyworld_: What about qas? Which version Chromium?
<jtv> StevenK, wgrant: did you try to land it as well?
<wallyworld_> poolie: yeah, it's a first cut. it certainly can be improved but good to get something out there
<wgrant> jtv: I asked StevenK to.
<wgrant> jtv: Since you're not here.
<jtv> Which I'm not.
<StevenK> jtv: I *just* threw it at ec2, so I'm happy to kill it.
<jtv> Kill.
<wallyworld_> wgrant: broken on qas too. 11.0.696.43 beta
<wgrant> poolie: Oh dear, that whole form is constructed in JS?
<wgrant> poolie: This is revolting.
<poolie> yus
<lifeless> pop quiz
<lifeless> is subsecond tag / count aggregates acceptable ?
<poolie> cunningly designed to disable browser history mechanisms
<wgrant> No, needs to be <1Âµs
<wgrant> poolie: Indeed!
<wgrant> Let's see.
<poolie> i spent probably 20m typing into it the other week
<poolie> was very sad
<wgrant> I think we've all done that :(
<lifeless> wgrant: Time: 17.281 ms
<lifeless> wgrant: for bzr
<lifeless> wgrant: Time: 743.434 ms
<wgrant> lifeless: Âµs, not ms.
<lifeless> wgrant: for ubuntu
<wgrant> Insufficient.
<wgrant> What :/
<StevenK> wgrant: I get a Unicode error, you mean microsecond?
<lifeless> wgrant: Time: 135.709 ms
<lifeless> wgrant: ubuntu anonymous
<wgrant> Why is activateConstrainBugExpiration called on *every* page?
<wgrant> When it's only used on Product:+configure-bugtracker?
<wallyworld_> wgrant: since it works on lp.dev and not qas or lp.net, must be some issue with how we run prod vs local?
 * StevenK manages to get ec2 list to hang, so checks AWS by hand
<wgrant> wallyworld_: Yeah... I'm trying to work out where we actually hook it up.
<wallyworld_> wgrant: a lot of it uses the lp formk infrastructure
<wallyworld_> the assignee is a IPersonChoice and there's a standard widget for that
<wgrant> We need a new creation form system.
<wallyworld_> wgrant: vocabulary-picker.js.template is used and has some pararameters wired in
<wallyworld_> wgrant: lp.app.widgets.popup.py is also part of it
<wallyworld_> wgrant: it provides VocabularyPickerWidget
<wgrant> aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa
<lifeless> https://bugs.launchpad.net/launchpad/+bug/758587/comments/10 for folk interested in the summary table
<_mup_> Bug #758587: summarising bugs is expensive <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/758587 >
<StevenK> "You should not import PrefixFilter from lp.services.log.logger" -- but it's in __all__? I'm confused
<wgrant> StevenK: Not here it's not.
<StevenK> wgrant: Here, r12811 of devel it is.
<wgrant> StevenK: Erm.
<wgrant>     'Nullhandler'
<wgrant>     'PrefixFilter',
<wgrant> Wrong caps, and missing comma.
<StevenK> Ah ha
<wgrant> So we now have a NullhandlerPrefixFilter
<StevenK> wgrant: Right, fixing. Thanks.
<wallyworld_> wgrant: <tal:chooseLink replace="structure view/chooseLink" /> in form-picker.pt inserts the javascript from the vocabulary-picker.js.template into the page
<wallyworld_> wgrant: but i'm not sure why it all works locally on lp.dev and not elsewhere
<wgrant> wallyworld_: Maybe a race?
<wgrant> wallyworld_: The form is loaded in a separate request.
<wallyworld_> wgrant: maybe. looking into it as we speak.
<wallyworld_> wgrant: you mentioned something though about setting cross domain issues?
<wallyworld_> seeing
<wgrant> wallyworld_: That's not the problem here.
<wgrant> Just something we've seen elsewhere.
<wgrant> So thought it was best to be exactly sure about where you were seeing it.
<wallyworld_> np. didn't think so but just wanted to double check in case i was being stpuid again
<wgrant> Ah hm.
<wgrant> I wonder if what we're doing is actually legal.
<wgrant> I saw some similar strange JS-ignoring behaviour in your branch yesterday.
<wgrant> An XSS attempt failed.
<lifeless> SpamapS: oh btw
<wgrant> <b>, <i> etc worked fine, but <script> didn't.
<wgrant> Perhaps Chromium doesn't like inserting JS like this.
<lifeless> SpamapS: that api thing you had that was slow
 * StevenK waits for 90 second PQM
<lifeless> SpamapS: was because you were querying 'related bugs'
<lifeless> SpamapS: thats partly fixed in our 'next' schema
<SpamapS> lifeless: OOH haha
<SpamapS> lifeless: so "please show me all bugs and all their related bugs" ?
<wgrant> StevenK: 90s? You mean 45s?
<lifeless> SpamapS: show me all bugs I (filed, commented on, assigned to me, subscribed to)
<lifeless> SpamapS: the commented-on clause required filtering across both ends of a 4 table join.
<StevenK> wgrant: I've not seen that quick, it tends to take a bit to actually get the mail
<lifeless> SpamapS: bug table seqscan is 4 seconds on its own.
<wgrant> wallyworld_: http://poeticcode.wordpress.com/2007/10/03/innerhtml-and-script-tags/
 * wallyworld_ looks
 * wallyworld_ hates IE even more if that's possible
<wgrant> wallyworld_: It's not just IE, though.
<wallyworld_> wgrant: ok. but it's still fun to hate IE :-)
<wgrant> Yeah.
<StevenK> lifeless: Can haz mentor on https://code.launchpad.net/~stevenk/launchpad/ids-bugfixes/+merge/57270 ?
<wgrant> Ah, sorry, forgot about that one.
<wallyworld_> huwshimi: are you able to +1 on a ui review?
<huwshimi> wallyworld_: Sure.
<wallyworld_> huwshimi: it's that picker confirmation one you help me with https://code.edge.launchpad.net/~wallyworld/launchpad/assign-non-contributor/+merge/55869
<lifeless> StevenK: whats source_interface: lp.soyuz.interfaces.distributionjob.IInitialiseDistroSeriesJobSource for ?
<StevenK> lifeless: It's so the IDSJs can be run using cronscripts/run_jobs.py, and allows me to remove cronscripts/initialise_distro_series.py
<lifeless> when would it have a different value ?
<StevenK> When the interfaces moves, if ever
<StevenK> lifeless: Keep in mind, run_jobs is generic, multiple job runners make use of it.
<lifeless> this seems cracked
<lifeless> yes, i get that
<StevenK> TBBH, the entire job system is crack
<lifeless> we should not have things in the config system that are other than knobs-we-want-admins-to-change.
<lifeless> thats not config. its code.
<StevenK> lifeless: I like being able to drop the cronscript, I've had to kill everything else I liked about that branch due to wgrant. :-(
<lifeless> StevenK: I like that too
<lifeless> but for 'non end user config' we have zcml
<wgrant> lifeless: The generic job runner takes a config section name. So the interface has to be specified there. Yes, this is crap, like DB users.
<wgrant> It can be fixed in the post-critical world.
<wgrant> Which may happen this year.
<lifeless> not saying I like - or dislike - zcml, but we definitely should not be putting non admin knobs into the the lazr.config.
<lifeless> right
<lifeless> not blocking this on it
<lifeless> but - tag, you're it - please file a bug
<StevenK> Er, which one of us?
<lifeless> StevenK: you
<StevenK> Bah
<StevenK> lifeless: High? Low?
<wgrant> Low tech-debt, IMO
<StevenK> Done, bug 759476
<_mup_> Bug #759476: run_jobs shouldn't require source interfaces in the schema <tech-debt> <Launchpad itself:Triaged> < https://launchpad.net/bugs/759476 >
<lifeless> thanks
<huwshimi> wallyworld_: Reviewed. Sorry for my comment in advance :)
<wallyworld_> huwshimi: thanks. actually the lack of space was bothering me too :-) so i'll fix it for  sure before landing. i'd rather out in the effort to get it as right as we can
<wallyworld_> put
<huwshimi> wallyworld_: Yeah, thanks a lot. I don't like giving such trivial reviews, especially if there's been a bunch of back and forwards already.
<wallyworld_> huwshimi: i don't see it as trivial per se. if something needs fixing, it should be fixed. much easier to do it now before landing :-)
<huwshimi> wallyworld_: Yeah I agree, it's not a full stop in a comment or something, but it was just once space that I was complaining about :)
<wallyworld_> huwshimi: one space that made the buttons look too squashed up and would annoy users :-)
<huwshimi> wallyworld_: I know, I know. It's worth doing. I still feel bad though :)
<wallyworld_> huwshimi: actually, do you know off hand what css i need to say "every element inside this containing div should have some right padding"?
<wallyworld_> i'm thinking i'll use that in the extra-form-buttons class to add the padding to the buttons
<wgrant> Is a normal space not sufficient?
<wgrant> I guess it might not be wide enough.
<wallyworld_> wgrant: i thought nbsp was verbotten?
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #633: FIXED in 5 hr 22 min: https://lpci.wedontsleep.org/job/devel/633/
<wallyworld_> and that it is best to use css for layout
<wgrant> wallyworld_: Doesn't even need to be nbsp
<wgrant> A plain old ' ' would do fine.
<huwshimi> wallyworld_: You can reference them by their type e.g. button or input or input[type="submit"] depending on what they are and what else is in the div
<huwshimi> wallyworld_: For this, if a space is wide enough then I don't think it's bad
<wallyworld_> huwshimi: wgrant: ok. i'll use a ' '.  much simpler :-) i just didn't want to the css style nazis to get mad at me :-)
<huwshimi> wallyworld_: You will be gotten
<wallyworld_> using css is actually cleaner cause then you can just add in other buttons and not worry about also adding in the ' ' each time
<wallyworld_> and there's more control and flexibility to adjust it in one spot
<wgrant> wallyworld_: Did you have the misfortune to come across subscribe_form_body in lib/lp/bugs/javascript/filebug_dupefinder.js during your investigations earlier?
<wallyworld_> wgrant: not in detail. but i know it's there waiting for me :-(
<wgrant> This, here, is a good example of why I am pedantic about JavaScript now.'
<wgrant> To prevent people from doing... that.
<wallyworld_> wgrant: dogfood also shows the issue on chrome so i may need to money patch some stuff for diagnosis perhaps. i'll see how i go.
<wgrant> wallyworld_: Interesting.
<wallyworld_> monkey. gof i can't type today
<wallyworld_> god aaaahhh
<wallyworld_> gotta put in a stupid ' ' for huwshimi first though :-P
<wgrant> Heh.
<StevenK> rvba: Shoot
<rvba> StevenK: https://code.launchpad.net/~rvb/launchpad/dds-stats-portlet/+merge/57280
<rvba> StevenK: I think you're the right person for this ;).
<wgrant> StevenK: Any ideas on the lack of base version?
<StevenK> wgrant: I've not investigated
<wgrant> Ah.
<StevenK> rvba: Reviewed
<StevenK> wgrant: The script appears to do the right thing
<StevenK> wgrant: Which saddens me
<rvba> StevenK: thank you.
<rvba> StevenK: do you have a suggestion to avoid adding the two booleans to IDistroSeries?
<StevenK> rvba: I suspect we don't have a way to get around it -- I suspect the information has to live *somewhere*, I'm just concerned that IDistroSeries is already bloated.
<StevenK> rvba: So ignore that bit for now, and fix up the rest. :-)
<rvba> StevenK: all right ... so I guess I'll have to leave them where they are. I'll fix the rest.
<wgrant> StevenK: Why's it the right thing? Unparsable changelogs?
<StevenK> wgrant: What I mean is that the code in the script appears to do the right thing
<wgrant> StevenK: Right.
<StevenK> More logging would be awesome
<wgrant> That's what I Thought.
<wgrant> I'll attack mawson once I've finished with this lazr.restful business.
<lifeless> actually we can just ignore dupes
<lifeless> no point counting them towards closed etc
<StevenK> rvba: Anyway, I want to remove that horrible webservice doctest, no fair adding to it.
<lifeless> grrrrrr
<lifeless>    944 |     22
<wgrant> Oh?
<lifeless> 949 In-progress bugs
<lifeless> out by 5
<rvba> StevenK: ok.
<StevenK> In fact, I'm working on that now.
<StevenK> Since lazr.restful seems to have set out to annoy me.
<rvba> StevenK: I was about to propose to do it as part of the fixing I'm doing now ... but since you're on it, I'll just convert my new doctest to a unitest.
<lifeless> wgrant: if you want to bend your mind a little
<lifeless>  https://bugs.launchpad.net/launchpad/+bug/758587/comments/14
<_mup_> Bug #758587: summarising bugs is expensive <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/758587 >
<lifeless> OTOH this queries Ubuntu bugs in 235ms
<lifeless> which I'm kinda happy about
<wgrant> Heh.
<lifeless> including privacy
<wgrant> Do you have temp table privs on qas?
<lifeless> yes
<wgrant> Handy.
<lifeless> yes.
<wgrant> I didn't know that.
<lifeless> 22MB of data.
<lifeless> about 100MB with indices
<wgrant> Not bad.
<lifeless> biggest is the unique index @ 35MB
<wgrant> lifeless: So, I know you're not really Zopey, but have a look at RequestExpiredView in lib/canonical/launchpad/webapp/error.py
<wgrant> lifeless: It clears the request data in __init__.
<wgrant> This seems fairly crackful.
<wgrant> I've moved it to initialize(), and everything still works.
<lifeless> just cooking dinner; I will come look at it in ~ 15-20
<wgrant> k'
<wgrant> Could also *possibly* fix lazr.restful to use queryMultiAdapter instead of getMultiAdapter to look up the view, but then it's not obvious how to replace the IWebServiceExceptionView adaption check, except perhaps by forcing the view to implement it directly.
<wgrant> But I think dangerous view __init__s should probably be avoided.
<adeuring> good morning
<wgrant> Morning adeuring.
<adeuring> hi wgrant!
<StevenK> Didn't we have a better way to call the webservice in tests rather than webservice_for_person() ?
<wgrant> StevenK: Interesting. DF has 7209 natty base_versions set, 8054 not set. That's more set than staging has, and staging should have more changelogs..
<StevenK> Staging looks to have close to zero set
<rvba> StevenK: I've tried to find a way to do what DistributionJob.getJobs does without having to had a new method but no luck ... could you take a look for me?
<wgrant> StevenK: bigjools said it had 19000 (yes, exactly 19000) unset when I asked him last night. But the script wasn't done yet.
<wgrant> And staging is down, yay staging.
<StevenK> rvba: getUtility(IInitialiseDistroSeriesJobSource).iterReady(),
* StevenK changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<StevenK> wgrant: I wonder if it completed.
<wgrant> StevenK: It probably did. This was a second run after the first run did the same thing.
<wgrant> But we should check with the Julian.
<StevenK> Right
<StevenK> And do it again after jtv's fix lands
<StevenK> Or just update dogfood
<StevenK> (The database, not the code)
<StevenK> Julian seems to be velemently against that, though
<wgrant> DF needs to update in two weeks anyway.
<wgrant> We can do it then.
<StevenK> Perhaps we should cowboy more logging onto staging, so we can get some idea
<wgrant> Well, we should check if the script finished first.
<lifeless> wgrant: that seems appropriate, *IFF* lazr.restful is creating such a view *before* the oops is written
<rvba> StevenK: I did not consider iterReady because it does not look to allow to get the jobs for one distro.
<lifeless> wgrant: wgrant the regular publishing machinery does this:
<lifeless>  - log the oops
<lifeless>  - figure out how to show the error
<StevenK> rvba: I doubt there are going to be many IDSJs
<lifeless> wgrant: why doesn't the api stuff do that [not to mention, why does the api have its own publisher]
<wgrant> lifeless: The lazr.restful exception handler is a bit special, since it needs to handle errors that should be returned as 400s, for example.
<wgrant> lifeless: So it looks up the view for the exception, and checks if it can be adapted to IWebServiceExceptionView
<wgrant> lifeless: If not, it reraises.
<wgrant> lifeless: I'm not sure how avoidable this duplication is.
<lifeless> wgrant: I understand its special; I don't understand why it needs to be different to the main publisher
<lifeless> e.g.
<rvba> StevenK: ok ... so I guess you mean I should get the right job in python.
<wgrant> lifeless: Neither.
<lifeless> more code to delete.
<lifeless> after criticals.
<rvba> StevenK: I'll do that but I don't like it.
<wgrant> Yeah.
<rvba> StevenK: iterReady fetches only jobs with status Job.ready_jobs and I think I really need all jobs with a status in PENDING_STATUSES (WAITING, RUNNING, SUSPENDED)
<StevenK> rvba: Then I suggest a better name than a generic getJobs
<rvba> StevenK: all right ... getPendingJobsForDistribution ;-)
<bigjools> wgrant: hello
<wgrant> bigjools: Hi.
<bigjools> wgrant: I'm interested to hear about your discussion with StevenK regarding initialising from distros with builds in progress
<wgrant> bigjools: Well, for initialisations within an archive you will never be able to copy that source in that archive ever again.
<bigjools> wgrant: what do you mean by within?
<wgrant> bigjools: For inter-archive initialisations you end up with confusion and binaries that are mostly copiable between archives, except not quite.
<wgrant> bigjools: eg. a new series of Ubuntu.
<wgrant> Doesn't cross archives.
<bigjools> so firstly, we're never going to allow intra-archive copies with builds in progress
<wgrant> Well, that original code did.
<bigjools> StevenK wasn't listening to me the other week then :)
<bigjools> secondly, can you expand on the inter-archive problem?
<bigjools> the plan was to reset BUILDING to NEEDSBUILD
<wgrant> bigjools: A source has built on i386 and amd64 in Ubuntu, but is still pending on armel.
<wgrant> bigjools: We initialise a new distribution from this Ubuntu series.
<wgrant> The series eventually have identical i386 and amd64 binaries, but the two armel builds produce different binaries.
<bigjools> how?
<wgrant> There are two builds... they will produce different binaries.
<bigjools> they are using the same dependencies, I don't see how
<bigjools> but anyway, so what?
<wgrant> The functional differences may not be significant, true. But regardless those two sources become incompatible, as each has a subset of the other's binaries. This is a problem we see fairly often in PPAs already, and is going to become far worse the next time we add another PPA architecture.
<wgrant> It is probable that none of the derivation workflows at the moment have a problem with this situation.
<bigjools> I still don't see why that's a problem
<wgrant> bigjools: For PPAs it breaks copies... Upload to a series with (i386, amd64, lpia), copy to a series with (i386, amd64, armel) which creates a new armel binary, and you can never copy that source again.
<wgrant> I can't see where that is going to come up immediately in the derivation case.
<bigjools> how does it create a new armel binary?
<wgrant> bigjools: Because there is no existing armel binary, a new build will be created.
<wgrant> (the upload series did not have armel)
<bigjools> I can't remember if we do that automatically or not
<wgrant> We don't have the pending build issue in PPAs because we forbid such copies.
<wgrant> We do.
<wgrant> All source copies run createMissingBuilds.
<wgrant> Similar things happen when copying between PPAs with incompatible sets of architectures.
<bigjools> what part of the copy checker prevents copying the source again?
<wgrant> The whole of it.
<wgrant> That's what CopyChecker does.
<bigjools> specifically which check?
<wgrant> Oh, you mean the build thing?
<bigjools> yes
<wgrant> It's crackful and broken and wrong, but let me find it...
<wgrant> Hmm. It looks like it relies on the destination archive having the pending builds, which only works for intra-archive copies.
<wgrant>             if build_summary['status'] in building_states:
<wgrant> Around line 315.
<wgrant> So, I can be convinced to allow inter-archive copies of pending builds if we acknowledge the issue, but the fact that the check was removed unconditionally strongly suggested that it had not been discussed.
<wgrant> And I will not permit undiscussed corruption of primary archives :)
<bigjools> yes sir
<wgrant> (I was the reviewer)
<bigjools> can you explain in English why the copy is prevented? :)
<lifeless> [no]
<lifeless> :P
<wgrant> bigjools: For the inter-archive case? Not directly, no. It has definite undesirable consequences, but none are fatal.
<StevenK> wgrant: The check was removed unconditionally was because I thought it was fine. I'm happy to admit I'm wrong
<wgrant> None are *obviously* fatal.
<wgrant> In present workflows.
<bigjools> wgrant: let me re-phrase this so I get the answer I am looking for.  What check, in English, prevents re-copying the same source in a PPA after the above situation?
<wgrant> Oh.
<wgrant> That check.
<wgrant>             if not copied_binaries.issuperset(published_binaries):
<wgrant>                 raise CannotCopy(
<wgrant>                     "binaries conflicting with the existing ones")
<wgrant> (is that broken? possibly. the whole binary copying system is a bit iffy :/)
<bigjools> what part of "in English" passed you by? :)
<bigjools> in any case, I'm trying to work out why that check is in place
<wgrant> bigjools: The check exists to prevent copying in duplicate binaries.
<wgrant> You can end up with some archs' binaries matching, and others differing.
<wgrant> If you permitted the copy then you would have multiple binaries with identical names.
<bigjools> so if we have the extra armel in the target PPA and we copy again,  why is that bad?
<wgrant> If there is only one set of armel binaries, nothing.
<wgrant> s/nothing/it's not/
 * bigjools has to run off for 10 mins
<bigjools> bbr
<bigjools> brb
<lifeless> headdesk
<lifeless> select count(*) from (select distinct bugtask.bug from bug,bugtask where bug.id=bugtask.bug and bugtask.distribution=1 and (not bug.private or exists (select true from teamparticipation,bugsubscription where bugsubscription.bug=bug.id and teamparticipation.person=2 and teamparticipation.team=bugsubscription.person)) and bugtask.status=22 and bug.duplicateof is NULL) as _tmp;
<lifeless>  count
<lifeless>    944 |     22
<lifeless> -------
<lifeless> -so close-
<lifeless>    943
<lifeless> but I think this is the duplicate subscription miscounting
<lifeless> which I can live with.
<bigjools> wgrant: sorry, back
<bigjools> wgrant: considering we'll never copy binaries when syncing from the parent distro, I don't think that check will ever trigger will it?
<bigjools> the binary copy will only happen on init
<wgrant> bigjools: At the moment we won't, no.
<wgrant> But it's possible in OEM's insane structure that they'll want to.
<bigjools> wgrant: it's not possible to do in the current design
<wgrant> Say you have a hierarchy, and add a new arch in the top shared project, and copy down a layer.
<bigjools> I also think each project is single arch
<wgrant> You probably don't want to rebuild stuff in the second layer unless necessary.
<wgrant> bigjools: Well, an OEM could conceivably have common branding and utilities on i386 and ARM projects.
<wgrant> It's not *too* contrived a situation.
<wgrant> But it's not immediately applicable.
<bigjools> wgrant: I still don't really grok why that check is written like it is
<wgrant> But it needs to be considered.
<wgrant> bigjools: It's a cheap way of checking that the files aren't duplicated, I guess.
<bigjools> too cheap it seems
<wgrant> Indeed, although some of the issues would still fall afoul of a duplicate file check.
<wgrant> It gets really messy when you can have partial copies :/
<bigjools> wgrant: are you happy with the workflow we have now?
<bigjools> I need to get this resolved.  Not being able to init with builds in progress is a blocker, I can't see anyone ever being able to initialise otherwise.
<wgrant> bigjools: As long as the issue is recognised I am happy enough with inter-archive copies.
<wgrant> It is not a non-issue, and it's certainly never OK to permit it intra-archive, but it's probably acceptable for inter-archive for now.
<bigjools> wgrant: and do you think resetting BUILDING to NEEDSBUILD for the binary copies will suffice?
<wgrant> bigjools: Resetting? New NEEDSBUILD builds will be created.
<bigjools> that's what I mean
<wgrant> Right.
<wgrant> It's the right way to go. It will result in bad things happening, but we can resolve that when it becomes a problem as long as we acknowledge the issue now.
<bigjools> ok thanks for raising the point
<bigjools> StevenK, got all that?
<wgrant> bigjools: I gather that the DSD population script failed again :(
<bigjools> wgrant: yes.
<bigjools> wgrant: missing changelogs I expect, but I need to investigate more
<bigjools> and jtv's branch didn't land apparently.  Darn
<wgrant> bigjools: He sent it off to ec2 again this afternoon.
<bigjools> ah good lad
<wgrant> Should be landing in half an hour ago.
<wgrant> :/
<bigjools> e_timewarp_error
 * bigjools needs caffeine to cope with this
<wgrant> bigjools: It just landed.
<bigjools> \o/
<wgrant> Half an hour later than I expected, but it's there.
<StevenK> wgrant: So, with the issue discussed, the change I mentioned this morning with only checking builds for the same distribution is fine?
<wgrant> StevenK: Not "fine", but "OK".
<jtv1> bigjools: the fix for bug 757483 has landed on devel.
<jtv1> My other branch still needs a bit of nursing, evidently.  Merging in a fresh db-devel broke some things.
<bigjools> jtv1: cheers
<jtv1> I honestly don't think librarian.txt has any right to fail on that branch.
<wgrant> jtv1: That's spurious.
<wgrant> jtv1: If it's returning the wrong status code.
<jtv1> Ohâ¦  the "this is some data" failure?
* danilos changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: danilos | https://code.launchpad.net/launchpad-project/+activereviews
<jtv1>     UploadFailed: Server said: 200
 * danilos is always bad at updating the topic with his OCR info :(
<jtv1> excuses
<jtv1> Why can't you be more like meâjust accept that you can't be bothered
<wgrant> jtv1: That's the one.
<wgrant> jtv1: If that was the only failure, lp-land!
<jtv1> wgrant: no, but it looks like the fixes for the merge-induced problems were simply never pushed
<wgrant> Heh.
<jtv1> re-ec2'ing now
<jtv1> wgrant: it's detached.  I'm off.
<lifeless> jkakar: thanks for merging my fix
<jkakar> lifeless: Thanks for providing it. :)
<lifeless> de nada
<jkakar> Que beuno!
<jkakar> bueno, even.
<lifeless> :)
<danilos> anyone knows of good examples of collapsible divs in LP UI?
<danilos> ah right, PPA page is a good one
<bigjools> danilos: ....
<bigjools> beat me to it :)
<danilos> bigjools, heh :)
<bigjools> we're doing the same thing on the derived distros pages
<bigjools> wgrant: turns out a load of packages in natty don't have changelogs. Grar.
<wgrant> bigjools: Or a lot of recent ones in debian?
<wgrant> bigjools: Do you have an example?
<bigjools> not looked at debian yet, the script is still running
<wgrant> Again? :(
<bigjools> StevenK tells me so
<bigjools> aalib (which handily has a SPN id of 1) has no changelog in natty for version  1.4p5-38build1
<wgrant> When was that uploaded?
<wgrant> I guess I could check.
<bigjools> checking
<wgrant> But that would mean not being lazy.
<bigjools> "26 weeks ago"
<wgrant> Hmm.
<wgrant> And its changelog is null?
<bigjools> apparently
<wgrant> Can you verify that in the DB?
<bigjools> looking at the query output right now
<bigjools> http://pastebin.ubuntu.com/593544/
<wgrant> datecreated
<StevenK> On *dogfood* the SPR's changelog is None
<bigjools> this is staging
<bigjools> there are 2994 packages in natty that don't have it set, compared to 15042 that do
<StevenK> Yes, but when I checked for you on the call, I was prodding dogfood
<bigjools> sure
<wgrant> bigjools: Oh, so most have it set, OK.
<bigjools> I am checking staging too
<bigjools> yeah, most
<bigjools> trying to work out the remaining ones
<wgrant> We started setting it in April last year, except for gina which was Nov/Dec.
<bigjools> ah
<bigjools> aalib was uploaded 2010-03-05
<bigjools> the sid version does have one BTW
<wgrant> It may be one of the last few, then.
<bigjools> I guess we could just say there's no diff until it's updated for those few
<wgrant> Which should be some time this week, right StevenK?
<bigjools> not for old natty versions?
<StevenK> I am going to check on the progress tomorrow afternoon
<StevenK> It should be done by Friday, hopefully
<wgrant> bigjools: It should be importing everything without a changelog, AIUI.
<wgrant> bigjools: Except for expired things.
<bigjools> ah cool
<bigjools> let's wait until it's finished then
<StevenK> And staging is updated
<StevenK> So Monday or Tuesday, I guess
<bigjools> the thought makes me hungry
<maxb> Um.... is it just me, or is the AJAX bug "Subscribe" link broken on production?  (Spins briefly and does nothing)
<wgrant> maxb: Are you in the malone alpha team?
<maxb> yes
<maxb> mute_link is null
<wgrant> Yeah.
<wgrant> Broken for the alpha team for now.
<wgrant> I thought that fix had been deployed, but maybe I was wrong.
<wgrant> Bug 753387?
<_mup_> Bug #753387: Can't subscribe to bug: Uncaught TypeError: Cannot call method 'get' of null <Launchpad itself:Triaged> < https://launchpad.net/bugs/753387 >
<maxb> Sounds like the same thing, just reported differently by firefox
<bac> danilos: would you have a moment to do a review?  should be quick.
<danilos> bac, sure thing
<bac> https://code.launchpad.net/~bac/launchpad/lptestrequest/+merge/57484
<bigjools> for the love of .... Google chose *TCL* as a scripting language for Chrome?!
<danilos> bac, r=me
<danilos> bigjools, isn't it already the 13th? :)
<bigjools> danilos: which century?
<danilos> bigjools, well, not that far off, but I think April 1st is gone by already :)
<bigjools> which is a shame
<jcsackett> danilos: do you have time to review https://code.launchpad.net/~jcsackett/launchpad/private-bugs-404/+merge/57244, by any chance?
<danilos> jcsackett, sure thing, let me take a look
<jcsackett> thanks!
<bac> bigjools: really?  oh that is awful.  i had to do a lot of tcl/tk back in the day and it was painful
<bigjools> bac: me too.  It's abominable.
<bigjools> bac: although I may have misread slightly, I think the TCL folks are the first to produce an interface that works with the Chrome script API
<bigjools> I'd love to use Python instead of JS
<bac> oh
<bac> i'd love to use X instead of JS where X not in [TCL, perl]
<danilos> bigjools, unfortunately, world is turning towards JS :( even epiphany, which you could write plugins for in python before, now only supports either natively compiled or JS
<bigjools> danilos: it makes me sad to hear that
<danilos> bigjools, yeah, me too
<abentley> danilos: on the plus side, PyPy seems to be doing well, and it has a JS backend :-)
<danilos> jcsackett, r=me
<jcsackett> danilos: thanks!
<danilos> jcsackett, you are welcome
* danilos changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<bac> danilos: in one of your yuitests i get a failure
* abentley changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: abentley | https://code.launchpad.net/launchpad-project/+activereviews
<danilos> bac, which one is it? node construction ones with waits?
<bac> danilos: test_escaping_url
<danilos> bac, hum, let me try it out in different browsers
<bac> i used ff3
<danilos> bac, right, I can see it failing... damn YUI "compatibility" layer... it's probably blindly using browser's implementation of escape() function which seem to differ between browsers
<danilos> bac, I don't see a good way around it other than asserting that the text is *not* unescaped (i.e. as-is)
<danilos> bac, what do you think?
<bac> danilos: i haven't looked at the details of the test yet
<danilos> bac, oh, this just makes sure that node.set('url', '...') escapes the URL properly; it's not even a big deal because it's all URLs we produce, but I just wanted to be extra safe
<danilos> bac, fwiw, the test passes in webkit browsers like epiphany
<bac> danilos: you have other escaping tests.  why do those pass and this one fails?
<danilos> bac, the only thing I can think of is that others are for node content, and this one is for node attribute value (ff seems to transform stuff into URL encoding vs. HTML character entities)
<danilos> (that might be YUI and not FF, of course)
<danilos> bac, basically, what I am doing in the test is reading node.get('innerHTML') and that behaves differently
<danilos> (at least)
<renata> Hello everybody, I am starting to work with launchpad api, and without launchpadlib, I would like to obtain a list of packages from a specific ubuntu release.
<renata> I know that source_package_publishing_history shows me a package based on a package ID. But I would like to get a direct link with a list of all packages, without knowing their IDS.
<renata> is it possible?
<renata> thanks in advance :)
<abentley> renata: there's no central list of what sourcepackages are present in a release.  In our model, sourcepackages are arbitrary combinations of a sourcepackagename and a distroseries.
<abentley> renata: You could try archive.getPublishedSources.  Doesn't look like sourcepackagename is required.
<renata> abentley ahhh ok!! so I'll try to see getPublishedSources.. if that doens't work I figure something out. thank you very much for the help :)
<danilos> bac, I've fixed the test case, fwiw, I am asserting that it's not equal to the originating string now
<bac> danilos: ok, that seems reasonable
<danilos> bac, do you perhaps want to go through the branch on the phone?
<bac> danilos: sorry, i know you're in a rush.  i don't think that would help right now.
<danilos> bac, np, just checking :) I am not in a big rush, I just want to help as much as I can
<BjornT> i've gotten a lot of oopses today while reviewing merge proposals. well, not a lot maybe, but 2  out of 5 or so page loads have oopsed (e.g. OOPS-1929AW110)
<adeuring> abentley: could you have a look at this mp: https://code.launchpad.net/~adeuring/launchpad/bug-758902/+merge/57502 ?
<gary_poster> I was about to say the same thing, BjornT.  Looks like a librarian might be down.
<abentley> adeuring: sure.
<gary_poster> abentley, here's another one when you have a chance.  https://code.launchpad.net/~gary/launchpad/bug744888/+merge/57499
<abentley> adeuring: r=me
<adeuring> abentley: thanks!
<abentley> gary_poster: What's the effect of doNotSnapshot?
<gary_poster> abentley, it means that Snapshots, which are temporary objects we use typically to send out modify events to show what an object looked like before a webservice change, will not include that particular attribute.
<abentley> gary_poster: so that's server-side event handling, not directly related to the web service?
<gary_poster> abentley, yes, except that AFAIK, only the webservice changes use them.  I could be wrong on that, but that's certainly where they are used the most consistently.
<gary_poster> BjornT, there might have been some misconfiguration for the new LP appservers.  We think it's fixed.  Could you let us know if things seem better?
<abentley> gary_poster: Cool.  I think we also use snapshots to generate attribute change emails, e.g. for branches and merge proposals.
<gary_poster> ah, cool abentley.
<abentley> gary_poster: would StormStatementRecorder work for your test?
<gary_poster> abentley, me looks for it.
<BjornT> gary_poster: second page load: OOPS-1929AR131
<gary_poster> BjornT: :-( ok thanks
<abentley> gary_poster: testing/__init__.py
<gary_poster> abentley, yeah, that looks simpler, thanks.  I'll switch to that
<abentley> gary_poster: I wonder whether there's risk of your SQL check hitting false negatives?  Would it be crazy to just look for the string 'message' in the entire statement?
<abentley> gary_poster: I think PEP8 requires an extra blank line at 55.
<gary_poster> abentley: looking for message: well...there are also "bugmessage"s in theory...it's obviously a balance between possible false negatives and false positives.
<gary_poster> I guess we could try being a bit more aggressive.  I'd be OK with simply splitting on whitespace and making sure there are no strings that start with "message" (which would include tables).  If people find that too restrictive in the future they can adjust the test.  You good with that variation?
<gary_poster> abentley: line 55 of test_bug.py?
<abentley> gary_poster: line 55 of the diff.
<gary_poster> oh sorry
<gary_poster> looking
<abentley> gary_poster: line 25 or so of test_bug.
<abentley> gary_poster: immediately after metaclass = __type__
<gary_poster> gotcha.  huh, I'm not familiar with that, and lint didn't complain, but I'm fine with it.  I'll make the change.
<gary_poster> And review at PEP 8 later :-)
<gary_poster> Oh, line 25 of test_bug?
<gary_poster> that has two returns now
<gary_poster> though I changed that after the initial MP
<gary_poster> maybe you are looking at the email version?
<gary_poster> of the diff?
<abentley> gary_poster: My apologies, PEP8 says "Separate top-level function and class definitions with two blank lines." and is silent on how to separate other top-level items (except imports).
<gary_poster> oh ok cool
<abentley> gary_poster: So with your proposed change ("splitting on whitespace..."), r=me.
<gary_poster> great, thanks very much abentley!  Glad to know about StormStatementRecorder for this purpose especially.
<abentley> gary_poster: np.
<sinzui> jcsackett: do you have time to mumble now?
<jcsackett> sinzui: now is excellent.
<henninge> no branches on qastaging ... :(
<henninge> ah, just no lp: support ;)
<jelmer> I've filed bug 759928
<_mup_> Bug #759928: linked MP is inaccessible <Launchpad itself:New> < https://launchpad.net/bugs/759928 >
<sinzui> jelmer: I am still looking for an oops I can read
<maxb> It's not OOPSing any more
<jelmer> sinzui: you can't access the OOPS id mention in the bug report?
<jelmer> I can pastebin the traceback if that would help
<sinzui> jelmer: no, nor the others I have been given. I think oopes are not syncing
<sinzui> jelmer: a TB would be very helpful
<jelmer> sinzui: I've pasted one in the bug report as a comment
<sinzui> thanks
<jelmer> sinzui: you can't reproduce the issue with the first URL?
<sinzui> ah yes! I followed the wrong link the first time.
<sinzui> I am an idiot
<sinzui> jelmer: I am a different kind of idiot than I supposed. I did click on the right link.
<sinzui> jelmer: The access to the librarian is intermittent. I believe we have a connectivity issue with one of more app servers
<jelmer> sinzui: ah, that would explain some things
<jelmer> I can sometimes get the page if I refresh often enough
<sinzui> I am engaging a losa now
<maxb> There definitely seems to be some sort of pattern in the oops prefixes
<sinzui> jelmer, lets load that mp url 5 times each to see it is now reliably loads
<sinzui> not for me
<jelmer> sinzui: just refreshing it 5 times here it failed 3 out of 5
<sinzui> i get the same
<maxb> I have stuck a load of OOPS codes on the bug
<maxb> to demonstrate that it looks to have node affinity
<sinzui> maxb: I saw. thanks
<sinzui> mthaddon: took the new webapp offline. I think the issue is fixed for now
<bac> abentley: i like the changes to 'bzr lp-land', i assume you did them.
<abentley> bac: I changed it to prompt before launching an editor, and to not write the commit message to the proposal.  Is that what you like?
<bac> abentley: indeed!
<abentley> bac: Cool!  You're welcome.
<sinzui> I recall we changed out ftp server recently. I think I make related to https://answers.launchpad.net/launchpad/+question/152715
<sinzui> Have we blocked passive-ftp
<sinzui> lifeless: thumper, abentley ^
<abentley> sinzui: I don't know anything about ftp.
<lifeless> sinzui: morning
<lifeless> sinzui: we did, poppy died and poppy-sftp now listens on ftp as well
<lifeless> sinzui: this is the issue - valid GPG signature: Verification failed 3 times: ['General error', 'General error', 'General error'] :
<lifeless> sinzui: the poppy-sftp service *also* tries to verify gpg sigs while the client is connected
<lifeless> sinzui: it uses GPGHandler
<lifeless> sinzui: which uses a /tmp dir
<lifeless> sinzui: which the /tmp reaper deletes old files from
<lifeless> sinzui: so this means that the gpg.conf for the running poppy-sftp on ppa.launchpad.net has been deleted and needs to be recreated.
<lifeless> sinzui: I'm looking up the bug
<sinzui> yuck
<lifeless> its pure win
<lifeless> bug 374019 is what was fixed causing the regression
<_mup_> Bug #374019: bad keys result in no response on ppa uploads <email> <lp-soyuz> <oops> <poppy> <qa-ok> <soyuz-upload> <Launchpad itself:Fix Released by julian-edwards> < https://launchpad.net/bugs/374019 >
<lifeless> bug 760147
<_mup_> Bug #760147: poppy-sftp service has its gpg conf reaped <Launchpad itself:New> < https://launchpad.net/bugs/760147 >
<sinzui> fab
<lifeless> workaround is to contact losa and have them repair
<sinzui> I am starting that now.
<sinzui> lifeless: Does the fix for this entail preventing reaping or automatic recreation?
<lifeless> sinzui: long term is a code change to either stop using one long gpghandler or put it somewhere else
<lifeless> the problem isn't that its in /tmp, its that its written at daemon startup and untouched for 5 days
<lifeless> or 4 days or whatever the reaper config is
<Ursinha> http://i.imgur.com/y7Hm9.jpg
<benji> news flash from 2024: the "Sassy Grass Green" squad has moved to maintenance rotation
<lifeless> benji: thats 'the stoners' right ?
<benji> or people who drive Plymouth cars from the 70s
<benji> http://www.flickr.com/photos/paddyspig/5434998200/
<lifeless> benji: shiny
<lifeless> benji: I want the gocart in the background
<Ursinha> lifeless, I wonder why that oops wasn't synced to devpad
<benji> lifeless: heh, really; I guess once you run out of room for cars you start hanging them on the wall
<lifeless> Ursinha: 12 hour cron IIRC
<lifeless> Ursinha: should be synced now, I think
<Ursinha> lifeless, I thought sync happened like every 10 minutes
<lifeless> Ursinha: for some unknown reason, not on codehosting
<lifeless> Ursinha: since you are here
<lifeless> Ursinha: I have a small qatagger request
<lifeless> I was going to put a patch together, but trunk failed tests for me and I got distracted by a tech review for statik
<lifeless> Ursinha: can we stop it setting milestones ?
<lifeless> Ursinha: per the thread on -dev about releases being decoupled from downtime
<jcsackett> i think i have killed my computer. i can't seem to run tests...
<lifeless> we don't need milestones anymore
<Ursinha> lifeless, I guess, I haven't stopped because I see a few people still setting it even after your email
<lifeless> Ursinha: well, qatagger is the driving force
<lifeless> Ursinha: if it stops, I think folk will naturally stop
<Ursinha> lifeless, ok, so I'll disable it
<LPCIBot> Project windmill build #173: FAILURE in 1 hr 6 min: https://lpci.wedontsleep.org/job/windmill/173/
<lifeless> Ursinha: thanks!
<Ursinha> np :)
<sinzui> jcsackett: mumble?
<jcsackett> sinzui: sure, one moment.
<jcsackett> sinzui: can you hear me?
<sinzui> jcsackett: me no hear
<jcsackett> sinzui: mumble appears to not be working for me. i did not hear you either.
<jcsackett> one moment.
 * sinzui kills mumble
<lifeless> sinzui: is there anything I can do for you guys today ?
<jcsackett> lifeless: make mumble work reliably? :-P
<lifeless> use ekiga/empathy with the canonical voip system
<sinzui> and unity/compiz. I went belly up and it looks like I am ignoring people
<lifeless> it works beautifully, even here where mumble is terrible
 * jcsackett makes note of that.
<lifeless> if you don't have an extension (check your directory page) contact #is and they can allocate one
<lifeless> [and get them to put it in the directory so you can find it later :P]
<sinzui> lifeless: We do not need anything at the moment. I think we  need to talk about how to improve the velocity of the teal squad and do a on-going maintenance well.
<lifeless> sinzui: when would you like to do that ?
<sinzui> I may ask for your help later today
<lifeless> sinzui: sure thing.
<lifeless> I have a little hr adminny stuff to do, and other than that a clean slate
<jcsackett> and now my whole system just crashed.
<jcsackett> perhaps my problem isn't with mumble...
<sinzui> jcsackett: I cannot get mumble to restart :(
<jcsackett> sinzui: skype? i have that on my phone, which i'm trusting a bit more this second.
<sinzui> jcsackett: I am trying skype
<jcsackett> sinzui: okay. i'm online with it now.
<jcsackett> sinzui: i can hear you shouting. :-)
<jcsackett> sinzui: i take it you could not hear me. and the call died.
<jcsackett> ...perhaps i have a crappy network connection today...?
<sinzui> I could hear you
<sinzui> It was not clear you could hear me
 * jcsackett grows angry with technology.
<lifeless> rage powered development. win.
<lifeless> flacoste: I've tweaked your edit to the rotation (there are two bug triage searches needed :( - to deep link directly to the them on the bugtriage page)
<lifeless> flacoste: if you still prefer them inline on the maintenance page, I'll do that for you and add bidirectional pointers so they don't accidentally get out of syn
<lifeless> c
<lifeless> grr
<lifeless> why isn't bug expiry expiring stuff
<lifeless> https://bugs.launchpad.net/launchpad/+bug/121859
<_mup_> Bug #121859: RFE: Url for posting private, non-security bugs <lp-bugs> <Launchpad itself:Incomplete> < https://launchpad.net/bugs/121859 >
#launchpad-dev 2011-04-14
<sinzui> wgrant: mumble?
<sinzui> lifeless: I thought expiration was 60 days. That bug was commented 33 days ago
<lifeless> sinzui: I thought bugs went 'eligible for expiry in X days' and then 'marked for epxiry Y days ago'
<lifeless> sinzui: I may be misunderstanding
<lifeless> sinzui: https://bugs.launchpad.net/launchpad/+bug/616704 is a better example
<_mup_> Bug #616704: Security package copy should copy packages to -updates immediately <lp-soyuz> <Launchpad itself:Incomplete> < https://launchpad.net/bugs/616704 >
<sinzui> lifeless: Yes that should be expired
<lifeless> sinzui: (not that that should be in this state, because bigjools was just asking a question)
<lifeless> but  it demonstrates the issue well
<sinzui> lifeless: The days marked is from the moment it qualifies to be expired...incomplete, un assigned, only one task etc...
<lifeless> sinzui: so its not 60 + count, its 'will expire when count == 60' ?
<sinzui> oh, did our db merging of projects break expiration for us?
<sinzui> yes equality
<lifeless> the phrasing is confusing then
<poolie> o/ lifeless, sinzui
<lifeless> hi poolie
<lifeless> sinzui: I thought it used to say 'will expire in N days'.
<lifeless> sinzui: in fact, I'm pretty sure it does.
<sinzui> lifeless: it does say that
<lifeless> sinzui: at what point does it switch to 'marked for expiry' ?
<sinzui> when all the conditions are met when the status incomplete is set
<sinzui> https://dev.launchpad.net/MaintenanceRotationSchedule
<lifeless> sinzui: I don't see anything on that page relevant to this
<sinzui> lifeless: sorry, that was for teal
<lifeless> ;)
<lifeless> sinzui: I think the 'marked for X days ago' turns up on the day it can expire
<lifeless> sinzui: so I think X days ago is last-changed + 60
<lifeless> 702022	Once a project is modified you can no longer modify the status		2011-01-14	
<lifeless> is 91 days ago
<lifeless> and it shows 'marked 31 days ago'
<lifeless> so, if its marked X days ago, it should have expired if bug expiry is working
<sinzui> lifeless: Changed could be ambiguous. I think there is a specific date, like date_incomplete_without_followup
<lifeless>  date_last_updated      | timestamp without time zone | not null default timezone('UTC'::text, now())
<lifeless>  date_made_private      | timestamp without time zone |
<lifeless>  who_made_private       | integer                     |
<lifeless>  date_last_message      | timestamp without time zone |
<lifeless> date_last_message is the one used
<lifeless> but note that adding a message sets date_last_updated
<lifeless> so date_last_message can only ever be older than date_last_updated
<sinzui> lifeless: These three methods look like the issues affecting expiration, but I do not see what is a miss yet: http://pastebin.ubuntu.com/593809/
<sinzui> Sorry. I was missing the forth method from bgutaskset:http://pastebin.ubuntu.com/593811/
<lifeless> days-before-expiration is unset in prod, so default value of 60
<lifeless>     enable_bug_expiration = BoolCol(dbName='enable_bug_expiration',
<lifeless>         notNull=True, default=False)
<lifeless>  is a simple lookup on product
<lifeless> no fancy check-group stuff
<lifeless> sinzui: it may be something simple, like not running the cronjob
<sinzui> I was thinking the same
<lifeless> or
<lifeless> it may be that its running in a very slow loop over one project at a time
<lifeless> with 20K+ projects that will perform very poorly.
<sinzui> I see it in loganberry-launchpad to run "17 04 * * * "
<sinzui> 10,000 projects have nothing in them
<lifeless> yeah
<lifeless> but its still 10K lookups
<lifeless> when the DB knows
<lifeless> and can do it in one hit.
<lifeless> sinzui: it must be getting late for you
<lifeless> sinzui: when did you want to talk ?
<sinzui> lifeless. Nothing I have nothing conclusive to talk about it seams. I think teal needs to do a better job watching scope of the tasks it undertakes
<sinzui> lifeless: Are you available tomorrow?
<lifeless> sinzui: sure am
<sinzui> I will ping you tommorow then
<lifeless> cool
<sinzui> or maybe tomorrow.
<lifeless> Tomorrow, when the OOPS ended
<cinerama> lifeless: lol
<wgrant> lifeless: :( CPU graphs are odd this morning.
<lifeless> wgrant: oh?
<lifeless> cinerama: :)
<wgrant> lifeless: There was a big drop right after the new appserver was activated, but not long later they returned to normal :(
<lifeless> wgrant: well, thats to be expected
<lifeless> wgrant: when the new appserver was turned off
<wgrant> lifeless: Oh, it was turned off?
<wgrant> Maybe there was another incident I haven't read about yet.
<lifeless> ~2am your time
<lifeless> yeah
<wgrant> Aha.
<lifeless> linked in -ops
<wgrant> Yeah, found it.
<wgrant> It's still down? :/
<wgrant> Hmm.
<wgrant> Odd.
<lifeless> we should be able to run the smoketest and so on and bring it up
<sinzui> lifeless: wgrant: mthaddon: was at the EOD when he decided to take the new server offline. He had made changes to give the server access to the internal librarian, but jelmer and I did not see them take affect
<wgrant> sinzui: Do you know which rules?
<lifeless> sinzui: yah, it was the right thing to do
<wgrant> I suspect the restricted ports were forgotten.
<wgrant> (I wish we had transparency here)
<sinzui> wgrant: I think the same, but I do not know for certain
<wgrant> Hopefully we will run out of fires in a few hours and be able to debug.
<huwshimi> lifeless: I'm looking at a bunch bugs related to user profiles. Would it be appropriate to tag these all with "profile" or something?
<lifeless> huwshimi: if you like
<lifeless> huwshimi: basically, if its useful to you, do it.
<huwshimi> lifeless: Just in regards to your email, I don't want to add more clutter (I know this is not an official tag).
<lifeless> huwshimi: folk that are trying to avoid scope creep will often not want to fix clusters of bugs (because the cluster can include a ringer)
<lifeless> huwshimi: I have no objection to large numbers of tags
<lifeless> huwshimi: official tags are special because they always show up everywhere, even if not used much (or used so much they are not valuable, like the lp-* ones)
<lifeless> huwshimi: I hope that helps
<huwshimi> lifeless: Yes, thanks.
<lifeless> mwhudson: hey
<mwhudson> lifeless: hello
<lifeless> mwhudson: does the branch rewriter script have a db statement timeout ?
<lifeless> I suspect it doesn't
<mwhudson> lifeless: i believe not, i think there may be a bug for this already
<lifeless> there is
<lifeless> its critical
<lifeless> I'm proposing to make it 500ms
<lifeless> how badly will the rewriter handle a timeout exception ?
<lifeless> as in, will it die
<lifeless> or shrug and move on?
<mwhudson> lifeless: i don't know, i'd have to look at the code
<lifeless> mwhudson: it looks like it will be ok to me.
<mwhudson> actually, i'm pretty sure it will shrug
<lifeless> mwhudson: do you think this needs tests? http://pastebin.com/3HHjiQkL
<lifeless> mwhudson: [please say no]
<mwhudson> lifeless: i dunno, could you monkey patch _getBranchIdAndTrailingPath and check that there is a request when it's called?
<lifeless> mwhudson: I could, but it seems like a comment at the call site is as good insulation
<huwshimi> lifeless: Is that bug about strikethroughs on closed bugs now a duplicate of bug #1924 now that we've changed the description?
<lifeless> huwshimi: indeed
* henninge changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<huwshimi> Should this bug be unassigned:https://bugs.launchpad.net/launchpad/+bug/58784 ?
<_mup_> Bug #58784: Spec page refers to non-existent "Specifications in grey" <feature> <lp-blueprints> <ui> <Launchpad itself:Triaged by sabdfl> < https://launchpad.net/bugs/58784 >
<lifeless> heh
<lifeless> I suspect so
<lifeless> wgrant: dunno about you but today was a write off
<StevenK> I managed to get 1.5 branches done. I was aiming for 2.5
<wgrant> lifeless: Yeah, it was pretty hopeless.
<wgrant> I'll hopefully get some stuff done tonight, though :/
<lifeless> StevenK: nice
<LPCIBot> Project devel build #638: FAILURE in 5 hr 5 min: https://lpci.wedontsleep.org/job/devel/638/
<adeuring> good morning
<wgrant> Not really.
<huwshimi> wgrant: Everyone's a bundle of laughs tonight :)
<bigjools> morning
<jml> huwshimi: hi
<huwshimi> jml: Hey mate
<bigjools> anyone know a way of making pdb break on a class?
<lifeless> on definition?
<lifeless> or construction ?
<bigjools> lifeless: running any code inside a class
<bigjools> it seems to want either filename:lineno or a function
<lifeless> yeah
<bigjools> the best I can do is put the class in its own file
<lifeless> probably won't do base classes for you
<bigjools> which is exactly what I want :/
<bigjools> If I have an object that inherits from IHasOwner, I can't seem to use a custom launchpad.Edit security adapter because the EditByOwnersOrAdmins adapter is used - is there a way around that or do we have to just not use lp.Edit?
<henninge> jtv: Hi!
<jtv> hi
<henninge> jtv:  I forgot: what is needed to trigger pottery processing on a branch, i.e. automatic template creation?
<henninge> jtv: I know the branch needs an intltool setup. What else?
<henninge> jtv: Is that usable for projects?
<jtv> henninge: template imports enabled, POTFILES.in present, can't think of anything else right now.
<jtv> Also, I'm not here.
<henninge> jtv: thanks ;)
<lifeless> bigjools: sorry, I don't know the answer. perhaps curtis/benji or gary would be good folk to ask
<bigjools> lifeless: thanks, we can work around it my removing IHasOwner
<lifeless> bigjools: perhaps justhave a more specific class first ?
<bigjools> lifeless: yeah it's almost certainly down to ordering, but inherited classes come first it seems, not much you can do
<jtv> bigjools: I marked the DSDs-with-identical-versions fix as qa-ok since it's safe to deploy, but haven't done proper Q/A on it.
<jtv> Ah.
<bigjools> jtv: I literally just tested them ok
<jtv> There it goes.
<bigjools> :)
<jtv> Thanks.
<jtv> Damn I hate it when this happens.  Spend hours chasing down a report of a tiny memory leak in libpqxx against one particular postgres version, and it's just valgrind crying foul murder over libnss caching two layers down.  :/
<danilos> allenap, hi, are you OCR today? ;)
<allenap> danilos: Yep. Whatchagot?
<danilos> allenap, a nice little JS branch: https://code.launchpad.net/~danilo/launchpad/bug728370-descriptions/+merge/57656 :)
<allenap> danilos: Tip top.
<danilos> allenap, thanks :)
<danilos> allenap, I'll be logging out and back in, should be back shortly (if xchat keeps working with gnome3 running :)
<allenap> danilos: Hehe :) No worries.
<wallyworld> henninge: hi. there's a few translation imports that need reviewing. i'm not sure how to do that. are you or jtv still the best ones to ask?
<henninge> wallyworld: probably. and danilos ;)
<wallyworld> henninge: the last time i checked i don't think the required doco on what to do for us noobs had been fully developed?
<henninge> wallyworld: I thought jtv had written something up.
<wallyworld> henninge: yes, right you are. i'll take another look through it
<wallyworld> henninge: "For now, if you get a request to review and approve a translations upload, it's still best to forward it to one of the former Translations crowd: Danilo, Henning, and Jeroen". i'll give it a go but be prepared that i may need to as questions :-)
<danilos> allenap, hi, having any questions about the branch?
<allenap> danilos: None yet; I just had my lunch :) I'm going through it now.
<danilos> allenap, ok, thanks
<henninge> wallyworld: we can look at a couple of them now if you want but I'll have my stand-up call in 15 min
<wallyworld> henninge: it's ok. i'm reading through all the doc i can find. i'll see how i go and perhaps ask tomorrow if i get stuck. i just don't want to do something dumb
<henninge> wallyworld: there are 13 entries in the queue so it should not be too hard
<wallyworld> henninge: yep. and some are not that old so may yet be done by the cron job
<henninge> wallyworld: possibly. Let's have a look at it together tomorrow.
<wallyworld> kk
<jml> beuno: did you write up UI guidelines for Launchpad?
<jml> mpt: did you write up UI guidelines for Launchpad? Do they still exist? (I know of UserInterfaceWording only)
<beuno> jml, nothing besides: https://dev.launchpad.net/UI/
<jml> beuno: thanks.
* jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jcsackett | https://code.launchpad.net/launchpad-project/+activereviews
<mpt> jml, I wrote <https://launchpad.canonical.com/MatthewPaulThomas/BetterDesign/> (sorry, Canonical-only link) in 2008, but didn't have the opportunity to propose them as LP guidelines
<jml> mpt: ok, thanks.
<mpt> (Actually, most of that's from 2006)
* allenap changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jcsackett, allenap | https://code.launchpad.net/launchpad-project/+activereviews
<allenap> jam: Would you mind reviewing https://code.launchpad.net/~jstpierre/loggerhead/css-changes/+merge/56393? I don't really understand what he's doing :-/
<adeuring> allenap, jcsackett: could one of you please review this mp: https://code.launchpad.net/~adeuring/launchpad/bug-746460-productseries/+merge/57685 ?
<jcsackett> adeuring: sure
<adeuring> jcsackett: thanks!
<abentley> jml: can we chat about bug #758857?
<_mup_> Bug #758857: Linked branch in translations sharing details page shows branch URL without lp:// prefix <exploratory-testing> <upstream-translations-sharing> <Launchpad itself:Triaged by abentley> < https://launchpad.net/bugs/758857 >
<jml> abentley: sure
<abentley> jml: mumble?
<jml> abentley: yep.
<jam> allenap: I'll give it a shot, though if his screenshots are accurate, something is wrong
<allenap> jam: Cheers :)
* allenap changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: jcsackett | https://code.launchpad.net/launchpad-project/+activereviews
<rvba> jcsackett: Hi, could you have a look at https://code.launchpad.net/~rvb/launchpad/fix-package-diff-equal-versions/+merge/57686 ?
<jcsackett> rvba: sure, it's next in the queue. :-)
<rvba> jcsackett: great.
<jml> abentley: https://launchpad.net/ubuntu/natty/+source/ssss/+edit-packaging
<jml> abentley: https://bugs.launchpad.net/testtools/+subscriptions
<sinzui> bigjools: do you have any insight into question https://answers.launchpad.net/launchpad/+question/151985
<bigjools> looking
<wgrant> sinzui: I could not reproduce that locally with what I believe to be a broken sudo.
<wgrant> I may try harder to verify the bad umask.
<bigjools> it is certainly very coincidental with the broken sudo
<bigjools> sinzui: since it's happening as far back as lucid, I suspect it's not the broken sudo actually.
<bigjools> he's probably found a new bug, but my knowledge of recipes is not good enough to help without spending more time on this
<sinzui> bigjools: I was thinking this permission issue related to the umask problem last week, but lamont believes that is fixed
<bigjools> sinzui: the umask was done to paper over the sudo problem AFAIK
<bigjools> and yes it was fixed
<bigjools> fairly quickly :)
<bigjools> it was natty only
<sinzui> bigjools: understood
<jcsackett> adeuring: r=me.
<bigjools> sorry I can't be of more help
<wgrant> bigjools: It wasn't natty-only :(
<wgrant> bigjools: It's the system sudo.
<bigjools> ah ok, thanks, I misunderstood then
<bigjools> so it was the sudo outside the chroot?
<wgrant> Right.
<bigjools> oh dear
<wgrant> In the original incident a few weeks ago only natty was majorly affected, because it was the majority of the builds.
<jcsackett> rvba: r=me.
<rvba> jcsackett: thanks.
<danilos> allenap, hi, responded to your review :) I'd like to get your comments before I proceed to land this, because I mostly haven't changed anything (did say why not :)
<danilos> allenap, also, I just checked, the slide-out/in effects take 0.4s, which is why 0.5 worked for me consistently
<allenap> danilos: Cool, I'll take a look.
<danilos> allenap, sorry if I missed anything you said, gnome3 crashed with keyboard layout change :/
<allenap> danilos: Hah, no.
<allenap> danilos: setup_bug_subscriptions() calls fill_in_bug_subscriptions() which appends to a list that *is* in the page's DOM.
<danilos> allenap, right, but it doesn't get called before show_* function is called, and that one is called in 'domready'
<allenap> danilos: Oops, I get it.
<danilos> allenap, I originally had it inside 'domready' but figured there's no need... perhaps a comment would do :)
<danilos> with "would do" == "is a must" ;)
<allenap> danilos: Thanks for the response though. I'm still not a fan of using IDs as much as we do, but it's interesting to hear other people's approaches :)
<danilos> allenap, heh, yeah  I'd love them even more if web page somehow failed to render if you've got duplicate IDs
<adeuring> gary_poster: do you have time for a pre-imp call?
<gary_poster> adeuring: yes, in...15 min or so?  Is that ok?
<allenap> danilos: You said that there's no way for a browser to use a selector, except when a browser has the Selectors API. Can they not fall back to XPath? I assumed that's what YUI, jQuery, et al did.
<adeuring> gary_poster: sure, thanks!
<gary_poster> np, talk to you then
<allenap> danilos: Perhaps I should code up a cheeky checkers that does a document.write("<h1>Boom!</h1>") when duplicate IDs are found ;)
<danilos> allenap, yeah, selectors API is the querySelector* stuff; I assume that gets much easier for the platform when you narrow the scope because it's not just IDs it has to go through
<danilos> allenap, heh, I wouldn't mind :)
<danilos> allenap, anyway, I personally feel better asking in a more constrained set, perhaps it's totally irrelevant, but please don't make me do it in the entire document scope! pleaaaseee! ;)
<danilos> allenap, also, doing it on a node is a sanity check at the same time: you haven't moved the element someplace else
<allenap> danilos: I agree, always start from the narrower scope. Although I suspect that query by ID might be quicker in document scope (because the browser may optimize for that case), I think the difference is going to be minimal.
<danilos> allenap, yeah, but there's also getElementById on the DOM node, so that should be even faster :)
<danilos> allenap, btw, I've added a comment explaining the sequence of events, and now I am going to ec2 land the branch :) thanks again for the review!
<allenap> danilos: Cool, and you're welcome :)
<gary_poster> adeuring, actually I'm ready now.  I was overestimating for once, it seems. :-) I'm on Skype (garyposter) and mumble.  mumble is sometimes weird for me, but sometimes its fine, so either is worth a try. :-)
<adeuring> gary_poster: cool. shall we  mumble?
<gary_poster> sure
<gary_poster> I'm in Yellow: 1-on-1.  Join me or drag me somewhere
<jcsackett> benji: ping.
<jcsackett> does anyone know how to restrict version when exporting a webservice collection? i know you can use as_of when exporting an entry, but that's not supported for collection.
<jcsackett> sinzui: can you chat for a few minutes?
<sinzui> yes
<abentley> jcsackett: Do you mean a resource type?  Or an attribute of an object that is a collection?
<jcsackett> abentley: the former, i think. e.g. exporting IQuestionSet using "export_as_webservice_collection"
<abentley> jcsackett: It's a known issue that there is no way to restrict that to a particular version.
<jcsackett> abentley: dig. thanks.
<jcsackett> abentley: is there a bug already filed for it, do you know? i can't seem to find one for on lazr.restful
<abentley> jcsackett: don't know.
<jcsackett> abentley: okay, thanks. i'll dig around a bit.
<jcsackett> sinzui: i have put up the MP and requested you as reviewer. https://code.launchpad.net/~jcsackett/launchpad/api-wants-questionset/+merge/57723
<sinzui> jcsackett: thanks
<rvba> jcsackett: another small fix ... could you have a look? https://code.launchpad.net/~rvb/launchpad/fix-packagediff-request-link/+merge/57727
<jcsackett> rvba: sure
<jcsackett> rbva: r=me. i mention in my comment there's an alternative way to do some of your testing, but it's just a suggestion, not block to landing.
<rvba> jcsackett: thanks, I'll have a look right now.
<rvba> jcsackett: it's a nice suggestion ... I think I'll keep it this way because I've factored the 2 asserts in a method that does a little bit more than just testing ... but I'll definitely do it next time.
<jcsackett> rvba: dig.
<LPCIBot> Project windmill build #174: STILL FAILING in 1 hr 5 min: https://lpci.wedontsleep.org/job/windmill/174/
<adeuring> gary_poster: about the canAccess(), canWrite() methods we discussed: Any suggestions for an exception that should be raised if the number of participations is wrong?
<adeuring> gary_poster: ValueError sounds quite generic to me... but I have no better idea
<gary_poster> adeuring: if the number of participations in the current interaction is wrong, that is a NotImplementedError from the perspective of what we discussed before
<gary_poster> Because if you make your own interaction, you can get the answer irrespective of the current one.
<gary_poster> So there is a fix for this
<adeuring> gary_poster: ah right, seems that it is too late here to ask such questions ;)
<gary_poster> We just are not bothering with it for now.
<gary_poster> :-) np at all
<sinzui> jcsackett: do you have a moment to mumble?
<jcsackett> i do.
<lifeless> morning
<adeuring> abentley: lp:~adeuring/launchpad/api-query-permissions-on-object (not yet submitted for review; will do that tomorrow -- it's alread 9pm for me)
<abentley> adeuring: cool.
<jcsackett> sinzui: http://paste.ubuntu.com/594169/
<LPCIBot> Project windmill build #175: STILL FAILING in 1 hr 4 min: https://lpci.wedontsleep.org/job/windmill/175/
<lifeless> flacoste: hi
<flacoste> hi lifeless
<lifeless> so, major outage
<lifeless> I think we should talk about this on the blog
<flacoste> lifeless: the switch thing? sure, care to write something? otherwise, we could ask mrevell, but he will need guidance
<lifeless> I'll write something in mail and send it to a couple of internal lists for commentary
<jcsackett> sinzui: changes pushed to the review branch.
<jcsackett> i've also started in on the launchpadlib thing.
<sinzui> jcsackett: thanks
<thumper> morning
<sinzui> jcsackett: r=me
<poolie> jcsackett, the questionset api might be good to mention on the blog as a new feature
<jcsackett> poolie: indeed. but first, finishing it. :-P
<jcsackett> sinzui: thanks for the review. :-)
<sinzui> poolie: We cannot do anything with the questions, nor can you search for them. It is only useful for hiding a comment on a question you know exists :(. I hope to hack on the seaarch and workflow methods of the next two weekends to make something I can personally use
<poolie> heh, ok
<poolie> you can't even look at them if you know the number?
<sinzui> poolie: let me check. I think we can do that in prod right now
<poolie> it looks like you can, if you guess the url
<poolie> (which is not so hard)
<sinzui> yep
<sinzui> I want to get the questions for a project or a group of distribution source packages
<poolie> that would be useful
<sinzui> Indeed. I cannot see all the open questions for all the things I am an answer contact for
<flacoste> sinzui: do you know how the ProductWithLicense SQL expands to?
<flacoste> iow, how to i get the related rows as an array column in the main query
<sinzui> I do not know
<sinzui> flacoste: I think I want to make that query obsolete.
<flacoste> ok
<lifeless> flacoste: more graphs?
<flacoste> lifeless: yep :-)
<lifeless> flacoste: oh, and did you have a chance to look at the uncommitted changes stub removed ?
<lifeless> from the PPR tree
<flacoste> lifeless: not yet, but i'll get to it
<lifeless> flacoste: no rush :)
<sinzui> I am sitting on data that provides the missing licenses for about 800 projects. I then want to set the remaining projects to have I_DONT_KNOW
<flacoste> i think it's fine
<lifeless> monday hopefully we can change the timeout again
<flacoste> since i recall commiting it at some point
<flacoste> i want to show # of unreviewed projects
<flacoste> and track the 'special' licenses one separately from the regular ones
<lifeless> flacoste: I remember what I wanted to ask you
<lifeless> flacoste: bug count aggregates, what were your thoughts
<flacoste> lifeless: ideally, i think exact below 50 is desireable
<flacoste> above that it can be fuzzy
<flacoste> i liked the render fuzzy, get exact number async
<flacoste> approach
<flacoste> but i'm also fine with fuzzy only
<lifeless> I think there are a few ways we can be really precise
<lifeless> I think its not worth it in the short term
<lifeless> - the polls are pretty seriously weighted towards speed
<lifeless> - precision will definitely still have a cost
* jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<james_w> lifeless, will the system know if the number is fuzzy?
<lifeless> james_w: in principle yes
<james_w> so the rendering could include that information? I think that would help alleviate some concerns
<james_w> "3 bugs" vs "about 500 bugs"
<lifeless> yes, thats one way to handle it
<lifeless> I agree its not ideal to present a wrong count as fact; however...
<lifeless> note that /all/ our counts are usually wrong by the time someone clicks through.
<james_w> still doesn't help if you want to know the exact number, but avoids confusion if you click what looks like a precise number to find a different number of results
<james_w> well
<james_w> on some projects
<lifeless> james_w: we have broken memcached rules
<lifeless> james_w: before you guess at the cause.
<lifeless> james_w: we cache private results as publically visible, and anonymous likewise.
<lifeless> james_w: for these portlets specifically.
<lifeless> We noticed last week.
<lifeless> Its been like that for ... some time.
<james_w> ok
<lifeless> james_w: I'm trying to separate out:
<lifeless>  - things we can do to make it better than it is now
<lifeless>  - things we must to do to increase performance
<lifeless> So while I accept and agree that putting tasteful caveats around the place, (?) popups and so on are all good things
<lifeless> I am not convinced that they are necessary to make switching to precalculated aggregates a net win.
<lifeless> And if something is a net win, I think we should take an iterative approach rather than biting off more than needed.
<lifeless> drive velocity up and get out of the 200+ critical bug zone.
<lifeless> bac:  around? bug 753965 needs qa
<_mup_> Bug #753965: An IStructuralSubscriptionTarget that does not use LP for bug tracking should not present a subscription link <qa-needstesting> <Launchpad itself:Fix Committed by bac> < https://launchpad.net/bugs/753965 >
<lifeless> abentley: likewise - Fixes: Bug:702477
<flacoste> http://pastebin.ubuntu.com/594235/ is what i came with
<flacoste> any other suggestions
<flacoste> ?
<lifeless> flacoste: looking
<LPCIBot> Project devel build #639: STILL FAILING in 5 hr 35 min: https://lpci.wedontsleep.org/job/devel/639/
<lifeless> flacoste: seems reasonable to me
<lifeless> mtaylor: poolie mentioned a jenkins issue you're having?
<mtaylor> lifeless: yeah- I can't remember what it was right now
<lifeless> he showed me a java log
<lifeless> looks like slave reuse race condition to me, not a bzr plugin issue
<lifeless> IMBW
<LPCIBot> Project windmill build #176: STILL FAILING in 1 hr 10 min: https://lpci.wedontsleep.org/job/windmill/176/
<wallyworld_> lifeless: i was going to test "recipe request daily build script triggers OOPS on disabled archive" but getting the make_daily_builds script to be run on qas and check the log. you don't think this is needed?
<lifeless> wallyworld_: I think its fine not to - I looked at the change
<wallyworld_> lifeless: np. saves me some work :-)
<lifeless> sinzui: hi
<lifeless> sinzui: shall we chat?
<sinzui> yes
<lifeless> skype?
<sinzui> yes
<abentley> lifeless: I'm missing context.
<lifeless> abentley: ah there is a patch with you as assignee marked needing qa
<lifeless> abentley: there were two, but I wasn't sure about this one
<abentley> lifeless: yeah, that one's tricky and the steps to reproduce never went into the bug report.
<lifeless> abentley: I would like to do a nodowntime deploy during asiapacs day; if you can qa that we'll know whether we can deploy everything or just up to the commit before yours
<thumper> ah fuck
<thumper> :(
#launchpad-dev 2011-04-15
<wallyworld_> sinzui: me and thumper are sick of each other's company in mumble. you around?
<sinzui> 2 minutes please
<wallyworld_> ack
<thumper> sinzui: time's up
<abentley> lifeless: I can't QA this.  I need henninge.
<lifeless> abentley: thanks for trying
<abentley> lifeless: np
<lifeless> abentley: is it possible to check it doesn't break anything (vs fixing the issue) ?
<abentley> lifeless: I don't believe so.
<lifeless> ah well, ok.
<lifeless> I will mail henninge a ping
<abentley> lifeless: To check it doesn't break anything, I would need to get it to display the thing I changed, and that's what I need hennninge for.
<lifeless> abentley: is it a permissions thing, or knowledge?
<abentley> lifeless: knowledge
<lifeless> k
<lifeless> I might see if I can figure it out later today, and if so qa it
<wgrant> lifeless, abentley: https://translations.qastaging.launchpad.net/ubuntu/hardy/+source/gnome-applets/+pots/gnome-applets-2.0/bs/+translate, down the bottom.
<wgrant> Doesn't seem to be fixed.
<lifeless> wgrant: have you left rosetta experts etc?
<wgrant> lifeless: I logged out.
<wgrant> Actually, I guess this might be a different bug.
<wgrant> Since the bad string still shows up under Shared when I'm logged in.
<abentley> wgrant: the english when logged out is "To prevent privacy issues, this translation is not available to anonymous users,
<abentley> if you want to see it, please, log in first.", which is correct.
<abentley> wgrant: the fact that it says the bit about dummy translations under "shared" is another issue.
<abentley> wgrant: the bug was not exhibited when users were logged out.  AFAIK, they were always getting ""To prevent privacy issues, this translation is not available to anonymous..."
<abentley> wgrant: it was logged-in users without write access who were getting the issue.
<wgrant> abentley: As an unprivileged user I see the contributions.
<wgrant> But same on production.
<abentley> wgrant: I don't see how that could happen.  Do you see entry fields where you could put translations?
<wgrant> abentley: No.
<wgrant> It's read-only.
<wgrant> Oh.
<wgrant> Hmm.
<wgrant> Do suggestions count? I guess they probably do.
<abentley> wgrant: they do.
<wgrant> Ah.
<wgrant> I guess I can make it read-only if I change my licensing settings.
<wgrant> Apparently not.
<abentley> wgrant: I believe that will prevent you from seeing it at all.
<wgrant> I no longer have suggestion widgets, but the contributions are still there.
<abentley> wgrant: my steps to reproduce involve setting the pillar to "closed" translation permissions.
<sinzui> lifeless: skype again?
<wgrant> abentley: I might revert the fix from DF, close Ubuntu there, and see what happens.
<lifeless> sinzui: please
<lifeless> sinzui: 5 minutes first, gotta ring a supplier
<sinzui> okay, 5 minutes
<wgrant> lifeless: Still can't reproduce :(
<lifeless> sinzui: https://bugs.launchpad.net/launchpad/+bug/745512
<_mup_> Bug #745512: product-release-finder broken <qa-untestable> <regression> <Launchpad itself:Fix Committed by lifeless> < https://launchpad.net/bugs/745512 >
<poolie> lifeless: the privacy banner thing makes me think it would be nice to have a scope that matches user-agent
<wallyworld_> lifeless: do you know why lp/app/widgets/configure.zcml is not listed in canonical/launchpad/configure.zcml? hence it is not being loaded :-(
<wallyworld_> i added something to lp/app/widgets/configure.zcml and took me a bit to figure out why it wasn't working
<lifeless> poolie: it would
<lifeless> wallyworld_: well
<lifeless> I'd expect lp/app/configure.zcml to list it
<lifeless> wallyworld_: I don't know why it isn't listed
<wgrant> OK, now why is buildout crashing with an assertionerror...
<wallyworld_> lifeless: np. i'll add it. just wanted to check if there was a known reason for omitting it
<lifeless> wallyworld_: not that *I* know of, but I'm not as cluey as say gary or curtis
<lifeless> whee
<lifeless> 771 Time Outs - yesterday
<lifeless> 6926 Exceptions
<wgrant> Heh, yeah.
<wgrant> grar
<lifeless> ?
<lifeless> also
<lifeless> sob
<lifeless>  SourcePackage.setpackaging() now deletes an existing Packaging record and creates a new one, if one exists,
<lifeless> -> index bloat.
<wgrant> Yay
<lifeless> looks like we can deploy to 12821
<wgrant> lifeless: Can you make compile on natty?
<wgrant> I could until this morning :/
<lifeless> wgrant: I don't know
<lifeless> wgrant: (because I develop on lucid, our deployment environment)
<wgrant> lifeless: Ah, of course.
<LPCIBot> Project windmill build #177: STILL FAILING in 1 hr 3 min: https://lpci.wedontsleep.org/job/windmill/177/
<james_w> poolie, was bug 761327 supposed to be against launchpad?
<_mup_> Bug #761327: would like a feature flag scope on browser user agent <feature-flags> <launchpadlib :Triaged> < https://launchpad.net/bugs/761327 >
<poolie> oops yes
<poolie> 'too many matches' fail
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #640: FIXED in 5 hr 7 min: https://lpci.wedontsleep.org/job/devel/640/
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/drop-psc/+merge/57807
<StevenK> lifeless: It doesn't matter. staging should get everything due to the DB restore, and if qastaging if missing some changelogs, we don't care.
<wgrant> StevenK: Let's not drop it yet.
<wgrant> StevenK: Leave it running for a week to make sure we don't miss anything.
<StevenK> Meh, it was five minutes work, it can wait.
<wgrant> It's possible (although rather unlikely) that there is still bad stuff being created.
<lifeless> StevenK: I had a similar migration I turned off eagerly.
<lifeless> StevenK: then it broke staging
<lifeless> StevenK: *trust* me on this.
<StevenK> And DF
<StevenK> :-)
<lifeless> once all systems have either caught up on their own or been restored to it safe to nuke
<LPCIBot> Project windmill build #178: STILL FAILING in 1 hr 7 min: https://lpci.wedontsleep.org/job/windmill/178/
<StevenK> Rargh, now it's rvba!
<rvba> StevenK: ?
<StevenK> rvba: There's a 'hello-rvba' in dogfood's NEW queue!
<StevenK> I think it's your doing!
<rvba> StevenK: right ... nice it went through.
<StevenK> Well, it's in NEW, what do you want me to do with it?
<StevenK> wgrant: Can haz a hand?
<rvba> StevenK: would be great yes ... and you could update df while you've at it?
<wgrant> StevenK: 'sup?
<StevenK> DF has been updated for 15 minutes.
<rvba> StevenK: ok then, I'll be able to qa yesterday's work
<StevenK> wgrant: I'd like to force a maverick build record into NEEDSBUILD without it building.
<StevenK> rvba: What I mean is that hello-rvba is blocking my QA -- shall I accept it, or reject it?
<wgrant> /srv/launchpad.net/codelines/buildd-manager stop
<rvba> StevenK: accept it please
<StevenK> wgrant: No such file or directory?
<adeuring> good morning
<StevenK> wgrant: Right, buildd-manager killed
<StevenK> rvba: Accepted, er, about 4 minutes ago
<StevenK> rvba: I've only process-accepted, so it's out of the ACCEPTED queue
<wgrant> StevenK: Hm, maybe it's just in plain /srv/launchpad.net.
<wgrant> I can never remember.
<rvba> StevenK: anyway, I'm stuck because the page https://dogfood.launchpad.net/ubuntu/natty/+localpackagediffs is borked
<StevenK> wgrant: Tis
<StevenK> wgrant: Anyway, what's the next step?
<wgrant> StevenK: Set one to NEEDSBUILD.
<wgrant> SQL is your friend.
<StevenK> wgrant: In BPB, right?
<wgrant> StevenK: BFJ
<StevenK> Bleh, BFJ has no distroseries
<wgrant> PB
<wgrant> Er.
<wgrant> distroarchseries is on BPB, yeah.
<wgrant> You'll need to join BPB->PB->BFJ
<lifeless> or merge them
<dpm> hi all, could someone give me a hand? I'm trying to use the API to manage the import queue entries in Ubuntu. While on the web UI I can change any entry's status, I don't seem to have the permissions when trying to do it with launchpadlib -> http://pastebin.ubuntu.com/594384/ Perhaps I'm not changing the status the way it's supposed to be done. Any ideas?
<wgrant> dpm: Is import_into set?
<dpm> wgrant, what's import_into? I cannot find it in the api docs, and I did not set it
<wgrant> dpm: An import queue entry can't be approved until it has somewhere to be imported into. I'm not sure you can set that through the API.
<dpm> wgrant, ah so perhaps the setStatus call is missing an import_into parameter, you think?
<dpm> (I mean in the api)
<dpm> or the import queue entry is missing the attribute
<wgrant> dpm: Well, you'd need to set the target POFile or POTemplate.
<wgrant> Which means we'd need to export those two onto the API, which isn't done yet.
<dpm> ok, thanks wgrant, I think I'll file a bug, then
<henninge> wgrant, dpm: that is correct
<henninge> (missing API export for approval)
<henninge> And "Good morning" ;)
<henninge> Hi lifeless
<henninge> lifeless: looking at that QA
<dpm> thanks henninge, and good morning ;) I'll file a bug then - any additional info I should mention there?
<lifeless> henninge: thanks!
<henninge> dpm: "Enable queue approval through API" should be enough. You can mention what wgrant just said about being able to set potemplate and pofile
<dpm> ok, thanks henninge
<henninge> dpm: but there may be more to it because POTempalte and POFile entries might need to geet created during approval.
<dpm> ok, I'll add all the info
* adeuring changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: adeuring | https://code.launchpad.net/launchpad-project/+activereviews
<dpm> henninge, ah, I've just noted I had filed a very similar bug a while ago, so I've just updated the info (bug 671329)
<_mup_> Bug #671329: Enable translation import queue approval through API <api> <lp-translations> <Launchpad itself:Triaged> <Ubuntu Translations:Triaged> < https://launchpad.net/bugs/671329 >
<dpm> *noticed
<LPCIBot> Project devel build #641: FAILURE in 5 hr 14 min: https://lpci.wedontsleep.org/job/devel/641/
<allenap> adeuring: Are you in the mood for performance branch? It's a bit big, but reasonably straightforward... I hope :) https://code.launchpad.net/~allenap/launchpad/localpackagediffs-performance-bug-751321-queries/+merge/57778
<adeuring> allenap: sure, I'll look
<allenap> adeuring: Thank you :)
<lifeless> mrevell: you have mail
<mrevell> Thanks lifeless!
<lifeless> mrevell: re the downtime
<lifeless> mrevell: do you mean to say 'rob, you should tweet on identi.ca on monday morning about this'
<lifeless> mrevell: or 'cannot do monday as there isn't enough warning' ?
<mrevell> lifeless, Sorry that I wasn't clear. I meant that I am happy to announce this for Monday, if that's when you want to do it, because I think that a full weekend's notice is plenty for such a short and localised disruption.
<danilos> adeuring, hi, got time to review https://code.launchpad.net/~danilo/launchpad/remove-primary-duplicate-reason/+merge/57839?
<adeuring> danilos: I'll look at your mp when I've finished a review for Gavin
<danilos> adeuring, cool, thanks
<danilos> adeuring, fwiw, this one is 40 lines, so shouldn't be too hard
<adeuring> allenap: very nice work, r=me. Just one details puzzled me: There are two XXX comments in a template "this cell needs 0 QUERIES per row". Zero queries doesn't sound like a bug ;)
<allenap> adeuring: Yeah, I left those there for completeness, but if they're puzzling then I shall remove them. Thank you for the review!
<jml> https://answers.launchpad.net/testresources/+addquestion is 34 queries for me. wtf.
<adeuring> danilos: r=me
<danilos> adeuring, thanks
<jml> lifeless: I've got a guy asking questions about testresources on an old blog post. He has asked a question that's a bit tougher than I'm willing to help with right now. Which forum should I direct him to for help?
<jml> http://code.mumak.net/2008/10/testresources-some-examples.html fwiw
<LPCIBot> Project windmill build #179: STILL FAILING in 1 hr 1 min: https://lpci.wedontsleep.org/job/windmill/179/
<bigjools> jml: got a minute to talk about testtools?
<jml> bigjools: in a few minutes, need to grab some sandwiches from the kitchen first.
<bigjools> ok - just trying to work out how to use MatchesStructure and failing hard
<jml> bigjools: ah ok.
<jml> bigjools: I haven't used it before. Taking a look now.
<jml> bigjools: What are you trying to do?
<bigjools> jml: ok I think I got it now.  The docs are wrong I think.
<jml> bigjools: what's the prob?
<bigjools> http://readthedocs.org/docs/testtools/en/latest/for-test-authors.html#matchers
<bigjools> it says:
<bigjools> matcher = MatchesStructure({'a', Equals(1), 'b', Equals(2)})
<bigjools> I had to use = MatchesStructure(a=Equals(1), b=Equals(2))
<bigjools> seems like I am the first to use this matcher in LP :)
<jml> wouldn't surprise me :)
<bigjools> it's very handy though
<jml> mwhudson wrote it for some Linaro stuff he was doing
<bigjools> I can use it to test new model objects
<jml> a thing that I often do with matchers is make new ones using lambda
<jml> IsAThing = lambda x, y: MatchesStructure(a=Equals(x), b=Equals(y))
<jml> bigjools: just landed a doc fix. thanks.
<bigjools> jml: cool, thanks :)
<bigjools> it was invalid syntax anyway :)
<jml> hah, yes.
<jml> just goes to show that the best documentation review is from someone who's trying to figure something out.
<lifeless> mrevell: thanks please do
<lifeless> jml: anywhere I chat on
<adeuring> gary_poster: again a question about canAcess(): How can we say in the @operation_parameters decorator that the object can have more or less any schema? (I tried "obj=Reference(schema=Interface)", but that's a bit too naive...)
<gary_poster> adeuring...I don't know. :-/  benji, any ideas?
 * benji looks.
* bac changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: adeuring, bac | https://code.launchpad.net/launchpad-project/+activereviews
<bac> hi adeuring
<adeuring> hi bac!
<benji> adeuring: I'm not certain, but I think using IField will give you a parameter that can take any value
<adeuring> benji: let me check again, but I think I tried that too,. just a second...
<adeuring> benji: InvalidInterface: Concrete attribute, IField
<benji> hmm, let me look at it a little more
<adeuring> benji, gary_poster: but I think meanwhile that we could as well use obj.canAccess(user, attrs), instead of user.canAccess(obj, attrs). All we need then is a simple mixin class
<adeuring> ...and no hassles with a "generic object" parameter
<benji> adeuring: oh, it wants a schema field, not an interface; if you still want to try something try using an instance of zope.schema.Field
<adeuring> benji: Ah, thanks!
<adeuring> benji: "@operation_parameters(obj=Field())"  results in "zope.interface.exceptions.InvalidInterface: Concrete attribute, Field". Or did I misunderstand your suggestion?
<benji> That was my suggestion, unfortunately.
<benji> adeuring: if the parameter is an object, I'm pretty sure this will work: @operation_parameters(foo=Reference(schema=Interface, title=_('Foo')))
<adeuring> benji: zope.interface.exceptions.InvalidInterface: Concrete attribute, Field
<benji> that's really strange because LP uses code like that all over the place
<benji> I wonder if your getting that error from something else; try commenting out the method that you're working on and see if the error persists.
<benji> s/your/you're/
<adeuring> benji: can you give me an example where Reference(schema=Interface, title=_('Foo') is _really_ used? That's quite often used to avoid circular imports; the schema is later changed, I think
<benji> adeuring: you may be right; I had forgotten about the "change the interface later" thing we do
<abentley> henninge: https://code.launchpad.net/~abentley/launchpad/sharing-spinners
<abentley> henninge: https://bugs.launchpad.net/launchpad-project/+bugs?field.tag=exploratory-testing+upstream-translations-sharing&field.tags_combinator=ALL
<jcsackett> benji: are you the right person to bother if i'm hacking on launchpadlib?
<benji> jcsackett: most likely, what's up?
<jcsackett> benji: do you know if the docs in launchpadlib are just intended as examples of usage, or if they actually run as doctests? b/c they don't appear to be pulled in when you run bin/test
<benji> jcsackett: unfortunately the docs are not tested; I would love for that to change though.
<jcsackett> benji: ah. dig. that is a shame, as the docs looked like the right place for me to add a change for what i'm doing. i'll just add a unit test then.
<jcsackett> thanks!
<benji> np
<abentley> adeuring or bac: could you please review https://code.launchpad.net/~abentley/launchpad/sharing-spinners/+merge/57888 ?
<adeuring> abentley: I'll look
<abentley> adeuring: thanks.
<abentley> adeuring: I noticed a conflict.  Updating...
<adeuring> ok
<abentley> adeuring: updated.
 * adeuring is looking
<jcsackett> adeuring, bac: can one of you take a look at https://code.launchpad.net/~jcsackett/launchpadlib/add-questionset/+merge/57891
<bac> yes
<bac> jcsackett: done.  easy peasy.
<jcsackett> thanks, bac.
 * jcsackett wonders what the landing procedure is for launchpadlib...
<bigjools> commit, pray
<jml> jcsackett: yeah, that's an interesting question, isn't it.
<jcsackett> jml: it is. i worry that bigjools is right. it looks like you just commit to a checkout version and push...
<bac> adeuring: so we don't step on each other's toes would you be sure to hit the 'claim review' button?
<adeuring> bac: whoops... ok
<bac> thanks!
<bac> jcsackett: are you going to update the version of launchpadlib LP uses?  i see we're way back on 1.9.2
<bac> er, 1.9.3
<jcsackett> bac: er, i haven't thought about the version that launchpad uses.
<benji> jcsackett: https://dev.launchpad.net/HackingLazrLibraries#Landing%20your%20branches
<jcsackett> thanks, benji.
<bac> jcsackett: sorry, lost my internet
<bac> jcsackett: it isn't required but we should bump it at some point
<jcsackett> bac: dig.
<adeuring> abentley: r=me
<abentley> adeuring: thanks.
<jcsackett> sinzui: have some time to mumble?
<sinzui> jcsackett: sorry, screen locked up
<jcsackett> sinzui: all good.
<sinzui> jcsackett: I think my session is corrupted. I need a few more minutes
<jcsackett> sinzui: ok.
<jcsackett> sinzui: i could hear you. i think you cannot hear me?
 * jcsackett sees sinzui disconnected.
<beuno> I can't assign bugs to people
<beuno> is that a known problem?
<jml> beuno: no, it's not a known problem.
<jml> beuno: it works for me.
<beuno> jml, can I debug somehow?
<beuno> I click on the person and nothing happens
<jml> beuno: what browser?
<beuno> jml, FF 3.6, stock 10.10
<jml> hm.
<jml> beuno: that's just weird.
<jml> beuno: I'm in a bad position to help you right now.
<jml> beuno: sorry.
<beuno> jml, np
<jml> g'night all.
<jml> Will see you around.
<abentley> bac: could you please review https://code.launchpad.net/~abentley/launchpad/sharing-overlay-buttons/+merge/57935 ?
<bac> abentley: sure, you'll be next
<abentley> bac: thanks.
<abentley> bac: resubmitted due to missing prerequisite branch: https://code.launchpad.net/~abentley/launchpad/sharing-overlay-buttons/+merge/57938
<bac> abentley: looking now
<abentley> bac: ack
<bac> looks fine abentley
<abentley> bac: thanks.
<abentley> bac: Can you do one more?  It's real short: https://code.launchpad.net/~abentley/launchpad/fix-branch-precondition2/+merge/57945
<bac> sure
* bac changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | https://code.launchpad.net/launchpad-project/+activereviews
<lifeless> moin
#launchpad-dev 2011-04-16
<poolie> i'm getting a failure like
<poolie> keyring.tests.test_core: module references __file__
<poolie> Got keyring 0.5.1.
<poolie> Getting distribution for 'lazr-js==1.5DEV-r191'.
<poolie> i guess this is some kind of dependency out of date thing
<poolie> any clues? running update-sourcecode just once did not seem to fix it
<poolie> also lazr.restful...
<poolie> np, got it
<lifeless> \o/
<lifeless>  * 201 Time Outs
<lifeless> still a long tail
<wgrant> lifeless: But 43 of those are already fixed.
<wgrant> 450 soft too, wow.
<wgrant> We could almost get away with dropping to 9s right now...
<wgrant> Surprise surprise, all but one or two of the BugTask:+index OOPSes were from gandwana/potassium.
<LPCIBot> Project windmill build #180: STILL FAILING in 1 hr 1 min: https://lpci.wedontsleep.org/job/windmill/180/
<wgrant> Oh no, not again.
<lifeless> wgrant: windmill or something else ?
<lifeless> wgrant: and yes, the better hardware is making quite a difference
<wgrant> lifeless: structsub conflict.
<lifeless> \o.
<wgrant> Submitting now....
<lifeless> coolio
<lifeless> I think the bug aggregates will need rabbit
<wgrant> Oh?
<lifeless> unless I get some metric together on the volume of aggregate-field changes in ubuntu bugs
<wgrant> Hmmm.
<lifeless> I may be jumping at ghosts
<lifeless> bbiab
<StevenK> wgrant: Structul subscriptions *again*? :-(
<StevenK> lifeless: I wonder if we can drop the timeout now due to single-threadness plus new appserver of awesome?
<wgrant> searchTasks is an issue :(
<wgrant> Apart from searchTasks we can drop 2s (to 9s) without a problem, AFAICT.
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #642: FIXED in 5 hr 38 min: https://lpci.wedontsleep.org/job/devel/642/
<lifeless> indeed
<lifeless> we need to address searchtasks - perhaps by result size estimation
<lifeless> or possibly an exception
<wgrant> I don't think we can immediately make any vast improvements there :/
<lifeless> better search schema
<wgrant> "immediately"
<lifeless> fsvo
<lifeless> tag search is being a pain
<LPCIBot> Project windmill build #181: STILL FAILING in 1 hr 0 min: https://lpci.wedontsleep.org/job/windmill/181/
<LPCIBot> Project windmill build #182: STILL FAILING in 1 hr 1 min: https://lpci.wedontsleep.org/job/windmill/182/
<LPCIBot> Project devel build #643: FAILURE in 5 hr 14 min: https://lpci.wedontsleep.org/job/devel/643/
<lifeless> meep
<lifeless> 'bug' gets written to 30K times a day
<lifeless> for ubuntu alone
<lifeless> 60K across the board
<lifeless> BugTask:+index is our 10th most popular codepath : 160K hits a day
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #644: FIXED in 5 hr 30 min: https://lpci.wedontsleep.org/job/devel/644/
<LPCIBot> Project windmill build #183: STILL FAILING in 1 hr 0 min: https://lpci.wedontsleep.org/job/windmill/183/
#launchpad-dev 2011-04-17
<wgrant> Hmmm.
<wgrant> I think someone needs to make dpkg -b stab people unless I_SWEAR_THAT_I_AM_DEBUILD=1 is set.
<lifeless> hmm
<lifeless> need to make incomplete-* actual statuses.
<wgrant> :( 105 timeouts.
<wgrant> So close.
<wgrant>        36 /   18  Distribution:+bugtarget-portlet-tags-content
<wgrant> :/
<lifeless> thats pretty good
<wgrant> Hmmmm
<wgrant> 20 of the timeouts are from edge.
<lifeless> yup
<lifeless> gotta get that redirect in
<wgrant> aaaa, it executes the query twice
<wgrant> *That's* not meant to happen.
<wgrant> Oh god.
<wgrant> It's not checking tags_cloud_data's emptiness in TAL, is it...
<wgrant> Yes, yes it is.
 * wgrant blargs.
<lifeless> wgrant: double query? I did mention that :>
<wgrant> lifeless: So, grant an exception to Distribution:EntryResource:searchTasks and drop to 9s on Monday? :D
<wgrant> Hmm.
<wgrant> Not all the soft timeouts are listed.
<wgrant> There's 65 that aren't on the list... I guess it doesn't show if there aren't any hard timeouts :(
<wgrant> lifeless: https://code.launchpad.net/~wgrant/launchpad/bug-736002-seriously-this-time/+merge/58016
<lifeless> wgrant: are you sure it works?
<wgrant> Yes.
<lifeless> why does it work?
<wgrant> Because it's a dict, not a resultset. So the define evaluates it, and now we have a non-lazy object.
<wgrant> And I have a postgres log.
<wgrant> Thanks.
<lifeless> ah, it wasn't a cached property?
<wgrant> No. Could do that, I guess, but it's so easy to avoid..
<lifeless> possibly useful belt and bracer
<lifeless> given that there is no test
<wgrant> It's hard to give a useful test, since it's dominated by variable numbers of other queries :/
<lifeless> \i/
<wgrant> :(
<wgrant> chaenomeles still has no 'logs' rsync module :(
<lifeless> tell losa on monday
<wgrant> Oh hm.
<wgrant> These OOPS counts are wrong :/
<wgrant> oops-tools doesn't know about the new wampee prefixes, nor chaenomeles at all.
<lifeless> heh
<lifeless> bug xyz about that.
<wgrant> That explains why they're mostly edge.
<wgrant> lifeless: I should file a bug on oops-tools about that?
<lifeless> there are bugs for auto adjusting already
<lifeless> there is a wiki page explaining how to update things laready
<lifeless> uhm
<lifeless> if its hard to find or whatever then yes, a bug.
<wgrant> I gather it has to be done through the web UI now.
<wgrant> By an unknown set of privileged people.
<wgrant> Ah, there we go.
<wgrant> New prefixes added.
<wgrant> You can import them from lp-p-c without auth.
<lifeless> nicely done
<lifeless> if only it was in cron
<wgrant> .....
<wgrant> IE10 will not run on Vista.
<wgrant> Grar.
<wgrant> lifeless: Do you know how to rescan the last couple of days' OOPSes?
<lifeless> AIUI each scan scans all oops prefixes known
<lifeless> runs every 7 minutes
<lifeless> so may have a bit of a backlog, but should catch up pretty soon
<wgrant> Ah, true.
<StevenK> As expected p-s-c has done nothing over the weekend.
<lifeless> StevenK: on any of the 4 services?
<StevenK> lifeless: Er, it runs on loganberry only?
<lifeless> + staging
<lifeless> + qastaging
<lifeless> + dogfood
<StevenK> dogfood doesn't run it, and we need to re-import the database onto mawson in 1.5 weeks anyway
<lifeless> :)
<StevenK> staging won't need to after this weekends re-import
<StevenK> And qastaging, "meh" ?
<StevenK> lifeless: And also, it doesn't hurt anything if it stops running early -- it isn't like your index change, where stuff completly breaks if it's NULL
<lifeless> you sure about that ?
<StevenK> The only thing that uses SPR.changelog is DSD base version calculation -- if it can't find the ancestry, it will return early, not OOPS.
<wgrant> lifeless: He's right.
<wgrant> It's not like BugMessage:+index.
<wgrant> Er.
<wgrant> BugMessage.index
 * lifeless shrugs
<StevenK> lifeless: I can see I've not convinced you, so meh
<lifeless> I would be most comfortable with a situation where we don't have partial migrations done
<lifeless> but its your call; you are doing the work.
<StevenK> As you would have seen on the list, I'm somewhat around this week, so I'm just going to not care until Friday
<StevenK> lifeless: I also seriously question having to perform this (very expensive) migration on four seperate instances
<lifeless> wow
<lifeless> my net is so farked
<LPCIBot> Project windmill build #184: STILL FAILING in 1 hr 0 min: https://lpci.wedontsleep.org/job/windmill/184/
<LPCIBot> Project devel build #645: FAILURE in 5 hr 33 min: https://lpci.wedontsleep.org/job/devel/645/
<StevenK> Sigh. That test needs a rewrite to not be racy.
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build #646: FIXED in 5 hr 34 min: https://lpci.wedontsleep.org/job/devel/646/
<LPCIBot> Project windmill build #185: STILL FAILING in 1 hr 0 min: https://lpci.wedontsleep.org/job/windmill/185/
<lifeless> morning
<jelmer> Has something changed in the ~vcs-imports owner recently?
<jelmer> I now get a rejection email from launchpad-bugs@ every time I change a vcs import
<lifeless> I think we'll nuke that team (-bugs)
<poolie> hi all
<lifeless> thumper: hey
<thumper> lifeless: hi
<lifeless> thumper: how does enum sorting work with queries
<thumper> what do you mean?
<lifeless> thumper: I'm looking at making INCOMPLETE_WITH_RESPONSE and INCOMPLETE_WITHOUT_RESPONSE real status values in the db
<lifeless> bug 759467
<_mup_> Bug #759467: incomplete-with-response searches require complex searches <Launchpad itself:Triaged> < https://launchpad.net/bugs/759467 >
<lifeless> see lib/lp/bugs/interfaces/bugtask.py and BugTaskStatus/BugTaskStatusSearch
<thumper> ok...
<lifeless> so from my brief checks we just issue order by status
<lifeless> where status is the enum column
<lifeless> an int in the schema
<lifeless> and we have no special handling
<lifeless> *but*
<lifeless> BugTaskStatusSearch
<lifeless> has a sort_order attribute
<lifeless> and I'm lke 'wtf'
#launchpad-dev 2012-04-09
<rick_h> morning
<bac> hi rick_h
<bac> i suspect it may be very quiet here today
<rick_h> bac: yea, probably.
<rick_h> it was pretty darn quiet friday
<rick_h> almost eerily so
<rick_h> bac: are you running on precise? I'm still trying to figure out the way around my rabbitmq setup issues I replied to the -dev list about on friday
<nigelb> Morning rick_h & bac :)
<bac> good evening nigelb
<bac> rick_h: i am.  didn't see your email.  but i haven't run the test suite locally in a while
<rick_h> bac: I'm working on getting rocketfuel setup past this point
* benji changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: benji | Firefighting: - | Critical bugtasks: 4*10^2
<StevenK> bac: O hai. You can remove the EC2 image with id 521.
<bac> StevenK: okay, thanks
<rick_h> wgrant: any word on the rabbitmq dep stuff that breaks in precise? Stupid me did a fresh install and can't get rf-setup past that
<wgrant> rick_h: Ah, indeed, I should fix that. For now I'd just suggest manually installing the old rabbitmq-server and rabbitmq-erlang-client, which will let the other plugins install.
<rick_h> wgrant: ok, thanks
<jcsackett> sinzui, you around?
<lifeless> sinzui: how does registry admins end up assigned? Is it person deletions ?
<sinzui> lifeless, yes
<sinzui> We I think some linaro teams got deleted
 * sinzui needs to see if ~registry now owns linaro projects
<lifeless> sinzui: would it be reasonable for the delete code to unassign and unsubscribe?
<sinzui> yes. I already reported that bug suggesting that as the solution to the problem
<sinzui> bug #613603
<_mup_> Bug #613603: team merge subscribes/assigns bugs to ~registry <lp-registry> <merge-deactivate> <Launchpad itself:Triaged> < https://launchpad.net/bugs/613603 >
<lifeless> cool cool
<lifeless> btw those bug numbers are -old-, way before linaro
<lifeless> hwenablement or something
<sinzui> yes, but the bugs and branches I unsubscribed ~registry mentioned linaro from the maverick period
<lifeless> interesting
<lifeless> shrug, no matter; I'm glad you cleaned it up
<lifeless> thank you
<sinzui> I am always happy to stop email arriving to my inbox
<sinzui> particularly this week. I am using my backup computer and I think some mail and calendar settings are 6 months too old
<timrc> Speaking of registry, I'm imbued with its magical powers,  but have yet to receive my wizard's hat
<sinzui> timrc, I think ~registry conveys more responsibility than power. You have some extra moderation powers to work with projects and teams, but it means you are on call to fix data that stop other users from serving themselves
<timrc> more responsibility than power sounds a lot like parenting... bummer :)
<abentley> deryck: Could you please review https://code.launchpad.net/~abentley/launchpad/celery-everywhere-2/+merge/101284 ?
<deryck> abentley, sure
<lifeless> abentley: I have a new job in a local branch; do I need to change anything in it to work with celery ?
<lifeless> abentley: s/do I/is it likely that I/
<abentley> lifeless: Yes, you need to provide a "config" member that points to the config section for that job type, so that we can determine the dbuser.
<abentley> lifeless: does the job access branches?
<lifeless> abentley: no
<lifeless> abentley: its a stub implementation of the notification API, I wanted a low-key way to check that the API is completish
<abentley> lifeless: If it's not derived from BranchJobDerived, there's extra work needed so that LP can look its class up by Job.id
<abentley> lifeless: I'm working on supporting BranchMergeProposalJobDerived right now, actually.
<lifeless> abentley: ok. the job I have is super simple; its just got a single job type in its table
<lifeless> abentley: I wouldn't care if it has to be run non-celery in fact, as it should only exist long enough to validate stuff and bring up the external service.
<lifeless> abentley: but OTOH I don't want to hold back the cleanup of dead code
<abentley> lifeless: Okay, I wouldn't worry about it, then.
<lifeless> cool, thanhks.
<kirkland> is anyone around who can help me with a private/commercial launchpad project?
<lifeless> perhaps
<lifeless> !ask :P
<deryck> abentley, r=me. nice work.
<kirkland> lifeless: I have registered a new private/commercial project last week and I'm trying to get the bit flipped to make the code hosting private
<abentley> deryck: cool, thanks.
<kirkland> lifeless: normally, I ask flacoste to do that, but I think he must be away on holiday or something
<lifeless> did you do the entitlement registration thingy?
<kirkland> lifeless: yepper
<kirkland> lifeless: but my trunk is publicly visible
<lifeless> ok, let me hook you up with a -ops
<kirkland> lifeless: okey
<lifeless> kirkland: you should be pm'd in a sec by thedac
<kirkland> lifeless: kthx
<abentley> deryck: I'd like to retrieve the BranchMergeProposalJob where BranchMergeProposalJob.job = x OR the BranchJob where BranchJob.job = x.  Do you know how to do that with a single SQL statement?
<lifeless> abentley: they are different tables?
<abentley> lifeless: Yes.
<lifeless> abentley: if so you need to find (BranchMergeProposalJob, BranchJob) where BranchMergeProposalJob.job = x and BranchJob.job is NULL or BranchJob.job = x and BranchMergeProposalJob.job is NULL
<lifeless> (because its a cross join by default)
<lifeless> abentley: and then in your code check for which object is NULL
<lifeless> abentley: (this won't scale all that gracefully to many job types)
<abentley> lifeless: Cool, thanks.
<lifeless> hmm, you might be able to say BranchMergeProposalJob.job is not distinct from x OR BranchJob.job is not distinct from x
<abentley> lifeless: but I guess I can generate it programmatically from a list of job types?
<lifeless> you'd want to test that
<lifeless> abentley: possibly; I'd worry a little that it will slow down as you start getting hundreds of columns returned
<lifeless> s/little/lot/
<abentley> Why hundreds?
<lifeless> most sql queries bring back a few dozen columns at most
<lifeless> but each type in the storm query will be at least 2 columns (id, metadata), and more depending on the job's FK's.
<lifeless> things outside the common case are often slower, so I'd be worrying because we'd be doing something rather unusual.
<abentley> lifeless: I doubt there are more than 10 job types.
<abentley> lifeless: Most of our "types" are actually BranchJob + a distinguishing job_type attribute, rather than distinct tables.
<abentley> lifeless: Or similar for merge proposal jobs, etc.
<lifeless> yah
<lifeless> well, its something to note I think
<lifeless> we're reconstructing data other parts of the system had, with a lookup pattern its not tuned for; thats something to at minimum put on the known defects pile
<lifeless> address it is obviously not urgent, nor even necessary at this stage
<abentley> lifeless: Fair enough.  I had assumed that such a lookup pattern would not be a problem.
<abentley> lifeless: ISTM that since Job.id is a common field in all our job types, it would be elegant to be able to look things up by Job.id.
<abentley> lifeless: if that actually is a problem, we could use a tuple of table and id to look up Jobs.
<lifeless> the problem is the schema; consider bugtask, which acts as an intermediary between bug and product/productseries/distro/distroseries
<lifeless> we have the FK of the related object in bugtask
<lifeless> so only ever have to do one index walk to find the other object for a given bugtask
<lifeless> abentley: if you could do that, it would be better I think
<lifeless> (that == tuple of table+id)
<abentley> lifeless: We were going to go with a design inspired by your earlier suggestions, in that the job-running code was not even aware that the jobs were coming from a database.
<abentley> lifeless: Including the table name as a parameter is going to mar that a bit, but maybe not too much.
<abentley> lifeless: I'll chat with adeuring about how to adjust our approach.
<lifeless> ah, I see
<lifeless> I think if the job running code gets something opaque and hands it to a factory, and gets back a job, it shouldn't expose the db'ness of of it all
<abentley> lifeless: We've been printing the ID, and assuming it is short and integery.
* benji changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugtasks: 4*10^2
<lifeless> pop quiz, does simple sendmail handle unicode values in headers (E.g. from, subject) appropriately ?
<maxb> I'd be surprised if it did, it seems like it ought to be the submitting program's job to produce a valid MIME message
<lifeless> maxb: simple_sendmail is an internal API in the lp source tree
<maxb> ah. The _ clarifies much
#launchpad-dev 2012-04-10
<cjwatson> benji: thanks for your review of https://code.launchpad.net/~cjwatson/launchpad/api-build-builder/+merge/101162; would you mind landing it for me?
<lifeless> StevenK: enterpriseid probably needs a fixup approach
<lifeless> StevenK: otherwise it will become a source of circular imports
<lifeless> cjwatson: I'm loving your recent stuff :)
<lifeless> cjwatson: are you finding the LoC constraint problematic ?
<lifeless> wgrant: StevenK: wallyworld: one of you reprobates should be around and able to land cjwatson's branch ?
<cjwatson> lifeless: I'm not sure; I alternate between finding it a tedious source of busywork and thinking it's a useful way to make the codebase less of an inaccessible monster
<wallyworld> lifeless: sure, even though i'm not sure what a reprobate is
<cjwatson> lifeless: so far I've had enough sources of crap to remove that it's not been too bad
<cjwatson> (in practice)
<wallyworld> cjwatson: which branch do you need landed?
<cjwatson> wallyworld: https://code.launchpad.net/~cjwatson/launchpad/api-build-builder/+merge/101162
<lifeless> wallyworld: thanks
<wallyworld> np
<lifeless> wallyworld: lossely, scoundrels
<wallyworld> lifeless: clearly i need to have a thesaurus handy when talking to you :-P
<cjwatson> lifeless: Francis muttered something a few months back about possibly giving me PQM access; what oaths would I need to swear in blood for that?
<lifeless> cjwatson: an rt asking for it, webops can talk you through the bits
<cjwatson> this branch will take me to joint second non-LP Canonical contributor :)
<cjwatson> lifeless: OK, great, I'll put that in tomorrow
<spm> cjwatson: it's tomorrow, today.
 * spm messes with colin's mind
<cjwatson> my family are about to go away for a few days, so I fully expect to be on weirdo timezones
<nigelb> spm: that took a bit to understand... future ma.
<nigelb> *man
 * spm bows, todays troll achievement, unlocked. *ding*.
<lifeless> spm: don't forget to do it daily; tis classic grinding
<spm> lifeless: indeed. this is the cross I bear
<StevenK> lifeless: If you mean fixup like _schema_circular_imports, I object.
<lifeless> StevenK: I imported enterpriseid_to_object from lp.services.mail and the world ended
<StevenK> lifeless: Sure, so it needs a solution, but I *hate* _schema_circular_imports as a pattern and do not want to repeat it.
<lifeless> sure
<lifeless> wallyworld: thesaurus :
<lifeless> https://www.google.com/search?q=reprobate&client=fs
<StevenK> He is calling us villianous
<wallyworld> lifeless: i was being rhetorical before :-)
<lifeless> wallyworld: :P
<StevenK> lifeless is an expert at answering questions you weren't asking.
<cjwatson> lifeless: I am curious whether you expect the LoC constraint to actually result in a net reduction of code by some point, I must say :-)
<lifeless> cjwatson: if it stops the bleeding, I will be content
 * cjwatson nods
<lifeless> cjwatson: I expect the cultural values to shift over time
<wallyworld> StevenK: perhaps he knows they need answering before we do
<lifeless> cjwatson: and once that happens, we've won
<lifeless> cjwatson: I see rules as, at most, a way to flag things that are likely problematic from the desired values
<StevenK> lifeless: How do you suggest we fix enterpriseid?
<lifeless> StevenK: there are three general patterns I know of; one is to have the module provide framework but no registrations - and then register into it
<lifeless> StevenK: the second is to use something lazy - e.g. lazy_import, or local imports in factory-functions, to let the module not import the world but still know about it
<lifeless> StevenK: the third is to put all the needed imports at the end of the module after the things it exports
<lifeless> e.g.
<lifeless> imports
<lifeless> public thing
<lifeless> public thing
<lifeless> more imports
<StevenK> I dislike the imports at the end as a pattern too.
<lifeless> my first inclination is to use the solution we have used elsewhere
<lifeless> and IFF that works worse for this case than those cases, hunt for something else.
<lifeless> but I'm lazy
 * StevenK stabs _schema_circular_imports and then twists the knife.
<lifeless> wgrant: lp:~lifeless/launchpad/notificationapi is my <rough and minimal> sketch
<wgrant> aaaa
<wgrant> phew, it's not scanned yet
 * lifeless goes hunting for test case code matching 'please run jobs'
<wgrant> yay, go's next stupid package manager reimplementation has debconf questions
<lifeless> there was a prior one?
<wgrant> I mean "next" as in "next in the line of programming language designers ignoring nearly 20 years of prior art and doing it all again themselves because fuck every other language"
<StevenK> Sounds like Go.
<StevenK> Those who don't understand history are doomed to repeat it.
<StevenK> (Is my bitterness about Go showing yet?)
<wallyworld> StevenK: we (Canonical) seem to be jumping on the Go bandwagon?
<StevenK> Only because Gustavo is pushing it.
<wallyworld> sure one person doesn't make policy though
 * StevenK pulls the messaging indicator out, and stabs it through the heart.
<StevenK> The message it considers unread is OPEN in Thunderbird
<lifeless> wgrant: StevenK: https://bugs.launchpad.net/launchpad/+bug/30496 seems half-or-more-done, could I impose on one of you to either translate whats remaining for me, or do so in the bug ?
<_mup_> Bug #30496: [feature-request] status pages for uploaded packages <feature> <lp-soyuz> <Launchpad itself:Triaged> < https://launchpad.net/bugs/30496 >
<lifeless> StevenK: and Mark is too
<lifeless> dunno exactly how the wagon is structured.
<StevenK> lifeless: I'm trying to contain my bitterness since this is a public forum.
<lifeless> StevenK: indeed
<wgrant> lifeless: It wants something like http://qa.ubuntuwire.org/ftbfs/ but with more queue integration etc
<StevenK> And expiring old entries
<wgrant> That does expire old entries :)
<lifeless> so, can you edit the summary to reflect this ?
<lifeless> right now, my eyes glaze and roll back in my head
<StevenK> Welcome to Soyuz
<bigjools> heh
<lifeless> well, its more the description is awkward
<nigelb> I realy need to setup a launchpad qdb
<nigelb> and, of course, put everything spm says.
<StevenK> Haha
<lifeless> it should either describe the symtoms - the things users can't do - or if its going to ask for something, ask directly rather than in semi-passive phrases
<spm> :-)
<bigjools> are you saying that everything that spm says is a joke?
<ajmitch> nigelb: but he's so entertaining
<nigelb> bigjools: everything he says in launchpad-dev. I suspect he does work in other channels ;)
<nigelb> ajmitch: Indeed!
<lifeless> wgrant: StevenK: will you guys be drive-by fixing https://bugs.launchpad.net/launchpad/+bug/249112 ?
<_mup_> Bug #249112: Changing security status requires clicking "This report is public" <confusing-ui> <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/249112 >
<bigjools> "work"
<bigjools> ;)
<wgrant> lifeless: Yes
<wgrant> StevenK has it fixed.
<StevenK> "Fixed"
<lifeless> its fixed? Can close ?
<wgrant> ... in a branch
<StevenK> lifeless: It's in progress.
<StevenK> Assign to me and mark it so if you wish, I'm working on it this week.
<lifeless> done
<wgrant> Oh
<wgrant> Awesome
<wgrant> We're still using YUI 3.0.0's cssgrids
<wgrant> Rather than 3.3's.
<wgrant> Deliberately.
<wgrant> I wonder why
<wgrant> jelmer: bzr daily builds have been failing for a while. Looks like 04_serve_address is in bzr.dev, so it can be dropped from the packaging branch?
<poolie> hi wgrant
<poolie> bit later even for jelmer in cet, i think
<poolie>  //hope
<wgrant> Hi poolie
<wgrant> I hope it's a bit late, yes :)
<poolie> *late
<wallyworld> wgrant: do you know, if i'm defining a new ws collection using export_as_webservice_collection, what version string do i use for export_as_webservice_entry and export method operation_for_version in order for it to all work?
<wallyworld> atm, wadl generation fails because the collection adaptor generated by the ws infrastructure has no entry_schema defined
<lifeless> wallyworld: devel
<wallyworld> lifeless: tried that :-(
<wallyworld> i double check though since i've tried a few things
<wgrant> wallyworld: beta
<wgrant> It has to be the earliest version
<wallyworld> tried that too :-(
<wgrant> diff?
<wallyworld> one sec
<wallyworld> wgrant: https://pastebin.canonical.com/64011/
<wgrant> wallyworld: looking
<wallyworld> wgrant: thanks. i need to rejig the web service stuff for services so that services can be exposed nicely via launchpadlib. hence i need a service collection which i didn't need before since i relied on traversal
<wgrant> yep
<bigjools> the LP api stuff is a lesson in how not to design an API, right from how it's coded to how it's presented
<wgrant> wallyworld: Are you sure this is going to work at all, given the different types that can be returned?
<wgrant> It'll work if you hardcode .services in launchpadlib
<wgrant> But not if you use API methods on IServiceFactory
<wallyworld> wgrant: the return types will all implement IService
<wgrant> (saying lp.services.getByName('sharing') in launchpadlib will probably just return an IService, without the implementation-specific methods.
<wallyworld> i've added .services to launchpadlib but it doesn't work without a collection being defined
<wgrant> wallyworld: You'll need to do something like PillarSet
<wgrant> Not PersonSet or BugSet.
<wgrant> Because of the different types.
<wallyworld> i'll look at that. does the PIllarSet pattern allow something like lp.services['sharing'].foo() to be used?
<wgrant> That's exactly what it allows.
<wallyworld> what's the collection name? 'pillars' is assume?
<wgrant> (note that these are client-side *Set definitions, independent of the server ones)
<wgrant> No collection.
<lifeless> bigjools: well, it had a set of specific goals, which it reached.
<wgrant> They're client side utter hideousness.
<wgrant> But they're the only way to do this sensibly.
<lifeless> bigjools: sadly, some things we consider very important weren't there as goals, metrics - nor deliverabels
<wgrant> wallyworld: Can't get past this:
<wgrant> LocationError: (<lazr.restful.tales.WadlServiceRootResourceAPI object at 0xfda208c>, 'top_level_resources')
<wgrant> wallyworld: Even after removing the ServicesLink
<bigjools> lifeless: it's what happens when you leave someone alone to do something important
<wallyworld> wgrant: that's exactly my problem, and i too removed the link
<wgrant> Ah
<wgrant> I thought you'd got past that
<wgrant> So I will dig more
<bigjools> lifeless: it's just crap on so many levels I despair
<wallyworld> wgrant: so would i be wrong is assuming we cannot defined a new collection atm?
<wgrant> It's pretty elegant, pretty and nice.
<wgrant> It's just completely ineffective at anything ata ll :)
<lifeless> bigjools: nah, it wasn't because it was one person; it was because of fundamental approaches - like the choice to expose internal model objects
<lifeless> bigjools: which was, AIUI, mandated
<lifeless> anyhow
<lifeless> we have it now
<lifeless> and need to migrate gracefully
<wgrant> wallyworld: We can define a new collection. And lp.services.getByName('sharing') will work. But the object you get back will have no methods.
<wgrant> wallyworld: With the client-side hack, lp.services['sharing'] will work and have methods.
<wallyworld> bigjools: it's one of the reasons i did the services stuff - to try and keep internal model things hidden
<wallyworld> wgrant: client-side hack = pillarset?
<wgrant> I am wondering if the pragmatic approach may be to register all the services as top-level entries
<wgrant> wallyworld: Yes
<bigjools> wallyworld: fuck yes
<bigjools> lifeless: well I am sure that was the case but there's other human factors involved which I don't want to discuss on a public channel
<wallyworld> bigjools: note though that it does use the restful infrastructure so there are some concessions
<bigjools> lifeless: anyway I'd love to see us migrating to something else.  I doubt we'll find the resources though :(
<bigjools> wallyworld: restful is fine if used correctly
<wgrant> Mmm
<wallyworld> wgrant: not sure about exposing all services as top level entities
<wgrant> It's not clear what "correctly" is
<wgrant> For our needs, correctly RESTful is difficult.
<wallyworld> wgrant: services aside, i still wonder why defining a new simple collection of entities didn't work for me
<wgrant> wallyworld: It shouldn't fail to build, and I'm trying to work out why it is.
<wgrant> But it also won't be useful once it does build.
<wallyworld> wgrant: is that because it checks the ws.op client side and complains if it doesn;t think there's a server side end point to handle it?
<wgrant> wallyworld: I believe so.
<wgrant> It will respect the collection's resource type
<wallyworld> well, what it believes it to be
<wgrant> What you declared it to be :)
<wallyworld> so it's static
<wgrant> Basically.
<wgrant> You have to explicitly downcast.
<wgrant> By saying lp.load('someuri')
<wallyworld> which i'm trying to hide from the user
<wgrant> And it's not possible with the current implementation.
<wgrant> Unless you do something like PillarSet
<wgrant> Which implements client-side URL generation
<wgrant> Which it then spits into lp.load directly.
<wallyworld> so i wonder if it's worth doing this then. i guess so to make it user friendly
<wgrant> wallyworld: Oh, the error message was obvious all along.
<wallyworld> i didn't know how to fix it though
<wallyworld> or why the entry_resource was None
<wallyworld> which was the root cause
<wgrant> You are truly ignorant of the world's cultures. "LocationError: (<lazr.restful.tales.WadlServiceRootResourceAPI object at 0xfda208c>, 'top_level_resources')" is TALESlish for "IService isn't imported by lp.registry.interfaces.webservice"
<wgrant> Everyone should know that.
<wgrant> (I suspect it's hiding an AttributeError or KeyError under that LocationError, as TALES is wont to do)
<wallyworld> it wasn't before either, but i guess adding a collection made it matter
<wgrant> It wasn't involved in the root before.
<wgrant> I suspect only the root relies on the webservice registrations without anything else.
<wgrant> Also, fuck TAES.
<wgrant> TALES
<wallyworld> wgrant: right, will try that. thanks for looking. i wasted too much time on that :-(
<bigjools> lifeless: is there any support in testtools for re-running tests under different setup conditions?
<wgrant> bigjools: scenarios
<wgrant> eg lib/lp/poppy/tests/test_poppy.py
<bigjools> I knew someone would say that
<wgrant> Although that uses bzrlib's, not testscenarios
<bigjools> there's a clone_test_with_new_id thing, but no explanation of how to use it
<wallyworld> wgrant: turns out i don't need to do any client side hacking - it all just works without
<wallyworld> ah, it depends
<wallyworld> launchpad.services['sharing'].foo() works
<wallyworld> launchpad.services('sharing').foo() doesn't
<wallyworld> that's pretty sad
<wgrant> Why would services('sharing') work?
<wallyworld> wgrant: it doesn't since it checks that the method exists and it doesn;t as far as the client knows
<wallyworld> but why does services['sharing'] work
<wallyworld> it must not do the same client side checks
<lifeless> bigjools: testscenarios
<lifeless> bigjools: (python-testscenarios)
<bigjools> lifeless: any online docs?
<lifeless> bigjools: what test-poppy is using is an earlier edition of that
<lifeless> http://pypi.python.org/pypi/testscenarios
<bigjools> thanks
<lifeless> bigjools: obviously if you have q's I'll be happy to answer
<bigjools> lifeless: doc seems good so far thanks!
<bigjools> lifeless: ok question: the scenarios list isn't explained very well, what is supposed to be in it?
<lifeless> a list of (name, dict-of-things-to-apply) tuples
<lifeless> e.g. ...     test.scenarios = [
<lifeless> ...         ('scenario_1_0', {}),
<lifeless> ...         ('scenario_1_1', {})]
<lifeless> the dict maps attribute names to values
<lifeless> e.g.
<bigjools> ,lifelessall,
<bigjools> grear
<lifeless> {'foo': 'bar'}
<bigjools> lifeless: apply to what?
<lifeless> will make self.foo be 'bar' once the test is setUp
<bigjools> ok
<lifeless> see the dynamic scenarios example
<lifeless> that makes two scenarios, one that sets self.hash=md5 and one that sets self.hash=sha1
<bigjools> right
<wallyworld> wgrant: so setting collection_of=None and forcing it to lookup what the resource entry provides seems to do the trick. but a trap that [] and () both return service resources which behave differently
<wgrant> wallyworld: () isn't documented and really shouldn't work at all
<wgrant> I don't know why it does
<wgrant> I'm surprised collection_of=None works
<wgrant> Wait.
<wgrant> collection_of=None where?
<wallyworld> i'll keep using [] in my test code and at least launchpadlib will work in both case
<wallyworld> in the ServiceSet class
<wgrant> Oh
<wgrant> I thought you were doing this without the client-side hack :)
<wgrant> 16:22:38 < wallyworld> wgrant: turns out i don't need to do any client side hacking - it all just works without
<wallyworld> i am
<wgrant> confuse
<wgrant> d
<wallyworld> but i need to define a client side set
<wallyworld> but only the one - not one for sharing, one for ...
<wgrant> That's the client-side hacking we wanted to avoid :)
<wallyworld> there's also BugSet too etc
<wgrant> Since it requires a client upgrade, which is less than ideal. But it's probably unavoidable.
<wgrant> Sure.
<wgrant> Precedent doesn't make it less nauseating.
<wallyworld> so i thought that having a ServiceSet was kosher (like BugSet etc). and that avoiding SHaringServiceSet, FooServiceSet like was done for pillars was not using any hacks
<wgrant> It's accepted, but still evil since it means we can't evolve the webservice without client changes.
<wgrant> Which sort of destroys the entire purpose of WADL
<wallyworld> yes. i didn;t think we could anyway
<wgrant> Hm?
<wgrant> As we saw before, you can add new types, methods, top-level entries, top-level collections without changing the client.
<wgrant> But not this sort of top-level collection
<wallyworld> given our current implementation (ie BugSet, XXXSet all defined in the client side lib) i thought we needed to make that compromise
<wgrant> 'all'
<wgrant> Aren't there only 3 of them?
<wallyworld> Bug, Pillar, Product, Person, Distribution etc
<wgrant> Those are only needed because we wanted foos['somefoo'] to work
<wgrant> Collections with normal methods don't need one of those client-side hacks.
<wallyworld> so it's really a limitation of our infrastructure
<wallyworld> since all the client side classes do is define _get_url_from_id
<wallyworld> and that could be put in the wadl
<wallyworld> as an attribute
<wgrant> In this case we want services['someservice'] to work, partly because due to lazr.restfulclient limitations the ServiceFactory's methods will only return objects with IService's methods
<wgrant> Yes.
<lifeless> there is a bug
<lifeless> about allowing arbitrary interfaces on each api object
<lifeless> rather than predicting the interface for collection elements
<wgrant> But in this particular case we only want that workaround to get that functionality to workaround another limitation
<wgrant> lifeless: Right, that would work too
<wallyworld> so for now i'll do a new version of launchpadlib
<adeuring> good morning
<lifeless> bigjools: it making more sense now?
<bigjools> lifeless: yes thanks - rvba is on it :)
<rvba> Hi lifeless.
<czajkowski> good morning
<lifeless> cool
<rvba> lifeless: was this about the testscenarios thingy?
<lifeless> yah
<rvba> lifeless: I've been bitten by 872887.  Any advice on how to use testscenarios with nose?
<lifeless> rvba: use load_tests with nose2
<lifeless> rvba: or don't use nose, its terrifying
<lifeless> rvba: or figure out how to make nose do a custom callback before test execution, it probably has a hook somewhere for that, it just won't be compatible with anything else
<rvba> lifeless: ok, I'll try to do that thanks.  (too late, we're already using nose extensively.  It's worked fine so far to easily integrate django tests, twisted tests, etcâ¦.  But maybe it's terrifying inside ;) )
<lifeless> rvba: it is; are you using nose's 'twisted' thing, or testtool's twisted async module ?
<lifeless> rvba: (it is terrifying inside)
 * lifeless *far* preferrs using testtools.run / subunit.run to nose.
<rvba> lifeless: testtool's
<lifeless> rvba: phew :)(
<lifeless> s/(//
<rvba> testtools*
<rvba> lifeless: fwiw load_tests (module level function) is called without any argument by nose.
<lifeless> yay, totally incompatible
<rvba> indeed.
<rvba> I guess I'll have to resort to nose-specific hooks.
 * rvba starts to hate nose.
<lifeless> http://pypi.python.org/pypi/discover - see load_tests on that page
<lifeless> rvba: welcome to my world
<rvba> I guess nose stinks :)
<rvba> lifeless: in fact, nose is simply treating load_tests as a module level test function.
<lifeless> yup
<rvba> jam: Hi (I'm from the Launchpad team). I'm struggling quite a bit trying to get nose working with testscenarios and I see that you've been trying to do that as well (https://bugs.launchpad.net/testscenarios/+bug/872887). Do you happen to remember what you ended up doing to generate the scenarios?
<_mup_> Bug #872887: TestwithScenarios incompatible with nose 1.1.2 <testscenarios:Invalid> < https://launchpad.net/bugs/872887 >
<cjwatson> argh doctests with sampledata.  hate hate hate
<cjwatson> things I do not like include reading current-dev.sql to discover exactly what should be guaranteed in a given test
<cjwatson> also, the "bob the builder" joke in the test suite gets very old after (a) you've read it a few hundred times (b) your child watches enough of it that you get the theme tune in your head every time you see it
<lifeless> cjwatson: current-dev.sql has bad data in it
<cjwatson> indeed, but I'm not quite sure of your point :)
<lifeless> whats guaranteed in a test isn't well demonstrated by current.sql
<lifeless> (and current-dev.sql isn't used at all for tests ;))
<cjwatson> uh, current, ok.  but I meant doctests that are basically pretty-printing objects straight out of the sampledata.
<lifeless> okies
<lifeless> kill crush destroy
<cjwatson> quite.  need to make this branch pass for now, though :)
<wgrant> Is this webservice doctests for build.builder?
<cjwatson> Yes
<cjwatson> I've pushed a fix
<cjwatson> http://bazaar.launchpad.net/~cjwatson/launchpad/api-build-builder/revision/15073 is no more evil than the crap around it, I think
<StevenK> cjwatson: A few hundred, you say? I was sighing about it after the tenth time, and I have no young child. :-)
<StevenK> I don't know whether to blame Julian or Celso for that particular piece of Soyuz.
<cjwatson> you sure it's not a Daniel thing?
<wgrant> I believe it was Daniel
<cjwatson> most of the puns are his fault
<StevenK> Oh.
<StevenK> Is he still in Melbourne?
<StevenK> I have a few ... debts to settle.
<cjwatson> Silverstone, not Stone
<StevenK> Oh, bah.
<cjwatson> AFAIK he was never in Melbourne
<cjwatson> arguably transitively my fault for feeding him salami while he wrote chunks of Soyuz at my house, I guess
<StevenK> Daniel Stone was, though.
<jam> rvba: basically, I stopped using nose w/ testscenarios...
<jam> Specifically, nose doesn't seem to support the python 2.6+ test feature of "load_tests"
<jam> if you look here: http://bazaar.launchpad.net/~jameinel/u1db/c-oauth/view/head:/u1db/tests/__init__.py#L304
<jam> That is the adapter function I wrote, which allows you to do: "from ... import load_with_scenarios as load_tests
<jam> and then python 2.6/7 unittest and testtools itself
<jam> will load the tests in your module and parameterize them with testscenarios
<jam> nose doesn't support having "load_tests" defining how to extract tests cases from a module
<jam> and I never figured out how nose does that sort of thing
<rvba> jam: thanks.  I've been trying to use nose-specific stuff to generate the scenarios but no luck so farâ¦
<jam> rvba: testscenarios lets you do the parameterization by either inheriting from its TestCase or by using a load-time hook
<jam> I tried the test case approach, but it was broken for a lot of cases
<jam> a lot of runners, I should say
<jam> specifically, there are ones that assume the number of tests that will be *run* with one call to .run() is always going to be 1
<jam> but the test-case based approach creates them on-the-fly
<jam> so I recommend going with the load-time version
<jam> but I don't know how to get nose to not try and load your tests normally, and only accept them through your loader
<rvba> Yeah, but an assertion in nose makes that approach impossible to use with nose.
<jam> rvba: exactly
<rvba> I even tried to monkey patch nose to remove that assertion but then there is something wrong with how the tests are run.
<rvba> jam: anyway, thanks for sharing the result of your experiments.  I'll try again for a few hours and I guess I'll not use testscenarios if I can't find a proper way to use it with nose.
<rick_h> rvba: have you tried to file a bug/hit up the mailing list? I know Jason's been working on putting out some nose2 releases. Helpful guy.
<jam> rvba: yeah, I didn't need much from a test runner. I ended up writing a plugin to hook into bzr's test infrastructure, as it is the best I've found yet :)
<rvba> rick_h: I haven't tried that.  I'll look into it thanks.
<jam> (partial loading, progress bars, regex matching, etc, etc.)
<lifeless> jam: the only runner I know that assumes one TestCase == 1 test is nose
<lifeless> jam: everything else i"ve see, however awful, honours the protocol
<jam> lifeless: sure, though everything else is subtle shades of broken in different ways :)
<lifeless> jam: I'd love bug reports; I commented on all your current ones I think
<jam> lifeless: yeah, its mostly problems with the actual runners
 * lifeless goes back to bed.
<jam> like not great support for --verbose, etc
<rick_h> I love the nose --with-id
<rick_h> use that one all the time
<jml> hey
<jml> I've seen a fair bit of chatter about testtools & friends over the last couple of days
<jml> I haven't followed it all closely, but I'd like to know: how can I help?
<wallyworld> cjwatson: i've merged your branch directly since the fixes you did were just test fixes so you can update locally and your changes will be there
<cjwatson> wallyworld: excellent, thanks
<wallyworld> np
<cjwatson> jtv: if my changes to https://code.launchpad.net/~cjwatson/launchpad/remove-query-distro-pending-suites/+merge/101174 meet with your approval, do you think you could land that for me?
<jtv> cjwatson: I'll have a look.
<deryck> Morning, all.
<rick_h> morning
<jtv> cjwatson: I know this is terrible but I don't even know what the state of the landing machinery in LP is.  Is just setting the MP to Approved enough now?  If not, you may need to get help from someone who's actively on LP duty for the landing.  My setup isn't in a usable state.
<StevenK> jtv: Tossing at ec2 for cjwatson and you.
<jtv> Thanks!
<jtv> And those drop bears are sugar-free by the way.
<StevenK> Haha
<jtv> Good to see code shrinking into something better, no?
<cjwatson> jtv: ah, I didn't realise you weren't actively on LP at the moment
<cjwatson> StevenK: thanks
<czajkowski> StevenK: herrrro
<StevenK> czajkowski: O hai
<StevenK> cjwatson: I've marked it no-qa, you didn't have a bug linked, but feel free to do some QA on DF when it lands.
<cjwatson> I didn't really think there was much plausible QA, as if it broke anything it should have failed tests
<cjwatson> but I can run the publisher through on DF if you like, I guess
<StevenK> cjwatson: If you're happy with no QA, then it's fine.
<cjwatson> cool
<rick_h> adeuring: let me know what symlink/js file there you have out of whack.
<rick_h> the file I think you're looking at should be in build/lp/app/languages.js
<rick_h> sorry, build/js/lp
<adeuring> rick_h: no, its build/js/lp/app/worlddata
<sinzui> jcsackett, I will be afk for about 75 minute because I am picking up my computer.
<rick_h> adeuring: so that used to be a symlink to the worlddata directory, but the launguages.js was moved into app
<adeuring> rick_h: OTP
<rick_h> adeuring: ok
<StevenK> rick_h: Oh, bah.
<jcsackett>  sinzui: ack.
<rick_h> StevenK: huh?
<rick_h> StevenK: heard back from sidnei yesterday so we've got to redo the meeting. Should we just try again for next week? Any day work best for you since you're the late one?
<StevenK> rick_h: The symlinks we craft in combo-rootdir
<rick_h> StevenK: I didn't think we did any symlinks any more
<rick_h> StevenK: I was pretty sure I removed all those
<rick_h> StevenK: yea, yui is the only symlink any more
<StevenK> Oh, neat.
<StevenK> rick_h: Any day is fine with me.
<rick_h> StevenK: ok, will get another email out today.
<rick_h> StevenK: yea, that's the bug/issue, adeuring has a symlink for some reason
<bac> hi czajkowski
<czajkowski> bac: howdy
<czajkowski> bac: skype/G+ ?
<jml> hello
<jml> I've just uploaded fixes for the failing tests in my branch, could someone please attempt to land it?
<jml> is no one around?
<rick_h> jml: I'll try to pull that down and land it again. thanks for updating
<adeuring> rick_h: I removed the bad symlink build/js/lp/app/worlddata and now "make jsbuild" ends with IOError: [Errno 2] No such file or directory: 'build/js/lp/app/worlddata'
<jml> rick_h: thanks.
<rick_h> adeuring: do you have the traceback from that IOError?
<adeuring> rick_h: http://paste.ubuntu.com/923374/
<adeuring> rick_h: actually, the _is_ a symlink: ls -l build/js/lp/app/worlddata
<adeuring> lrwxrwxrwx 1 abel abel 41 2012-01-02 10:02 build/js/lp/app/worlddata -> ../../../lp/services/worlddata/javascript
<rick_h> adeuring: rm -rf you whole build dir please and then run make clean make
<adeuring> rick_h: same as before. BTW, the date of the new symlink is odd: 10:02, but my local time is 16:30
<adeuring> rick_h: well, the "creation date" is 2012-01-02...
<rick_h> adeuring: so the build dir got recreated but the symlink is still ther?
<adeuring> rick_h: yes...
<rick_h> is there a build/js/lp/app/languages.js file and -min?
<rick_h> both with current modifeid dates on them?
<rick_h> adeuring: can you dbl check your bin/combo-rootdir script? http://paste.mitechie.com/show/610/
<rick_h> adeuring: that is what used to create the symlinks
<adeuring> rick_h: that's what I have. "diff" output is empty
<adeuring> rick_h: ahhh....  there is this bad symlink already in lib/lp/app/javascript
<rick_h> adeuring: ah, that would explain it
<rick_h> adeuring: yea, there should be no symlinks in there
<rick_h> adeuring: and the build script copies the files out of there and minimizes them in the build location
<adeuring> rick_h: I removed it; "make jsbuild" now works
<rick_h> adeuring: awesome!
<rick_h> adeuring: I guess at some point that symlink was there and we didn't explicitly cleanup/remove it
<rick_h> I've had to redo my devel/etc so many times didn't hit it
<adeuring> rick_h: seems so.
<rick_h> jml: ok, up to the ec2 gods now
<jml> rick_h: thanks.
<jcsackett> sinzui: got time for a quick chat? just need to clear up a few points.
<sinzui> okay
<timrc> question: do I programatically add a comment to a bug via newMessage()? Something about the use of the word "message" and not "comment" makes me nervous, here
<cjwatson> timrc: yes
<timrc> cjwatson, perfect, thanks
<cjwatson> hey, if somebody wants to review https://code.launchpad.net/~cjwatson/launchpad/remove-services-unicode-csv/+merge/101315 for me, I can test my shiny new landing access
<lifeless> gary_poster: so, testtools and subunit
<gary_poster> lifeless, good call thanks
<lifeless> hum
<lifeless> subunit has the idea that tag changes happen at one of two places
<lifeless> between tests
<lifeless> and for one test
<lifeless> changes that happen between tests are applied to all subsequent tests
<lifeless> changes that happen for one test are not
<lifeless> testtools should be broadly compatible with that, if its not, its an unintended incompatability
<lifeless> I think testtools TestResult not being per-test aware is probably a naivety bug
<lifeless> rather than intention
<gary_poster> lifeless, cool.  We might or might not clarify that in testtools in a patch then.  thanks!
<lifeless> de nada
<lifeless> flacoste: you around today?
<flacoste> lifeless: i am, but booked wall-to-wall with calls
<lifeless> no worries
<StevenK> sinzui: http://pastebin.ubuntu.com/924059/
<sinzui> StevenK, Your approach is right. We just need to make the simpleterms to ensure we get the right value, token, and title: here are examples: http://pastebin.ubuntu.com/924071/
<sinzui> wallyworld, I see my comment is still in lazr.restful and that the code is guessing top-level collections. The code recovers from exceptions: http://pastebin.ubuntu.com/924082/
<sinzui> wallyworld, https://launchpad.net/moovida
<sinzui> wallyworld, http://people.canonical.com/~curtis/disclosure/
<rick_h> woot, yui 3.5 is official http://www.yuiblog.com/blog/2012/04/10/announcing-yui-3-5-0/
#launchpad-dev 2012-04-11
<lifeless> cool
<wgrant> lifeless: I'm slightly concerned that the LOC policy is encouraging people to put massive (multi-thousand lines of diff) unrelated changes into a single branch.
<lifeless> reviewers should still be pushing back on that
<wgrant> eg. spot the real change in http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/revision/15074
<lifeless> I presume you have examples?
<StevenK> Oh, bloody hell. That was my suggestion.
<StevenK> wgrant: archive model changes, but I know the branch :-P
<lifeless> so, this is an example of doctests sucking
<lifeless> I would have made a new login_person that was better and left the old one for doctest specific usage, myself.
<lifeless> landing it separately would have been nice indeed.
<lifeless> wgrant: such permutations during review have happened even before the LoC thing
<lifeless> wgrant: That specific one doesn't flag the LoC policy per se, for me.
<wgrant> Possibly not.
<lifeless> its a related change
<lifeless> if it was purely unrelated I would definitely feel tweaked
<lifeless> anyone know why we special case no-bug-supervisor situations ?
<wgrant> In which context?
<wgrant> Bug notifications?
<lifeless> yah
<cjwatson> anyone fancy a quick review of https://code.launchpad.net/~cjwatson/launchpad/remove-services-unicode-csv/+merge/101315 so I can test my landing access?  it's just removing two now-unused files
<wgrant> lifeless: Because to do otherwise would not confuse people sufficiently.
<lifeless> oh no whatever will we do
<lifeless> cjwatson: don't forget to set a commit message and go via ec2 land
<cjwatson> Oh yes, commit message.  I was going to use ec2 land, yes.
<cjwatson> Thanks, will attempt that now
<wgrant> rick_h: Hi, did you get a UI reviewer or Dan to look over the new email notifications?
<wgrant> The one you added doesn't respect our normal conventions and has incorrect capitalisation in a number of places.
<rick_h> wgrant: yes, dan has sent me some feedback I've got to incorperate
<rick_h> I'm working with him on it while I try to get tests passing
<rick_h> wgrant: it's not final yet.
<wgrant> confused
<wgrant> it landed overnight
<lifeless> maybe flagged?
<rick_h> it shouldn't have landed, tests have been failing nad I just pushed another ec2 run during the day today
<rick_h> I've not seen the tests run yet
<wgrant> http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/revision/15075
<wgrant> [r=gmb][bug=959482] Add event hooks to send a notification email to a user whenever significant changes to their account are taking place.
<lifeless> https://bugs.launchpad.net/launchpad/+bug/854405
<_mup_> Bug #854405: Private bug subscriptions <disclosure> <Launchpad itself:In Progress by wallyworld> < https://launchpad.net/bugs/854405 >
<lifeless> wallyworld: ^ could you perhaps give that a more descriptive subject?
<wgrant> rick_h: So, if it wasn't meant to land we should probably unland it.
<wallyworld> lifeless: sure. i'm not sure why that one is not marked as released
<rick_h> wgrant: email hung and didn't get the notice.
<wgrant> heh
<rick_h> wgrant: so I screwed up, should have ec2 test vs land. If youcould let me know what's wrong that would be helpful. Dan's notes so far have been more generic
<rick_h> wgrant: so I'm not aware of what's not to standard atm.
<rick_h> wgrant: I'm getting kicked out of the library here in a couple of min sorry. Going to be afk.
<lifeless> wallyworld: thanks!
<wgrant> "irc" -> "IRC", "Freenode" -> "freenode", needs general rewording to say "if you didn't make the change" rather than "if you didn't meant to make the change", URLs should be in <> or nothing at all rather than []. I'd check existing templates to see the general style.
<wgrant> It also seems slightly odd to not say what the new or removed email address is.
<wgrant> And I'd prefer an assertion that there's only one changed field, rather than just dropping everything except the first one, given the security implications.
<lifeless> totally
<lifeless> or list all the changed fields
<wgrant> Sure, but in the current implementation that's stated to be an impossible situation :)
<wgrant> So it just takes the first one from the list, ignoring any others that probably aren't there.
 * StevenK stabs layers.
<StevenK> Awww, a Maverick EOL notice.
<wgrant> Oh, right, 10.10.10
<wgrant> Wasn't expecting it for a couple more weeks
<cjwatson> So how do I go about debugging things when the EC2 instance that 'ec2 land' starts up won't accept SSH connections?
<cjwatson> strace shows it sitting in connect()
<wgrant> cjwatson: It'll normally retry a couple of times. Takes a while to start
<cjwatson> Last time it timed out; this time it looks well on its way to timing out
<StevenK> cjwatson: Gasp, you're going to land your own branches
<StevenK> ?
<cjwatson> If EC2 will ever work for me, yes
<StevenK> Heh
<StevenK> Can you use ssh to connect to the instance it has started?
<cjwatson> No; also hangs in connect()
<lifeless> check the ingress rules
<lifeless> ec2 land may have misdetected your source ip address
<StevenK> Yeah, log into AWS and check your security groups.
<cjwatson> ah yes, you're quite right
<cjwatson> bogus RFC1918 source addresses
<lifeless> I see a patch to ec2 land coming up
<cjwatson> not sure it's ec2's fault - why on earth is checkip.amazonaws.com telling me my RFC1918 address?  (how does it even know?)
<wgrant> You're not setting an outbound X-Forwarded-For or something like that?
<cjwatson> ah, could be a proxy, yeah
<cjwatson> works better with http_proxy unset
<StevenK> Heh
<lifeless> mmm haha
<lifeless> cjwatson: I wonder if there is a bug tracker for checkip.amazonaws.com
<lifeless> is bug 63000 still true?
<_mup_> Bug #63000: IBugTask permissions are not in security.py and are obtuse <lp-bugs> <tech-debt> <Launchpad itself:Triaged> < https://launchpad.net/bugs/63000 >
<cjwatson> running now, phew
<wgrant> lifeless: Yes
<StevenK> wallyworld: I guess https://code.launchpad.net/~wallyworld/launchpad/launchpadlib-services-974139/+merge/101349 is tied to the mail you sent to -dev?
<StevenK> cjwatson: \o/
<wallyworld> StevenK: yes.
<StevenK> wallyworld: Do you want to set it to WIP if you want to wait for the discussion?
<wallyworld> right, will do
 * StevenK stabs buildbot. Six or seven times.
<wgrant> StevenK: rick_h's branch conflicted with jml's.
<StevenK> Ah.
<StevenK> Handy.
<lifeless> does anyone know a reason why we have UNKNOWN *and* UNDECIDED  bug statuses?
<StevenK> I think UNKNOWN is for imported bugs?
<lifeless> s/status/impostance/
<lifeless> StevenK: well, thats what we use it for
<lifeless> StevenK: but why do we have it
<rick_h> wgrant: sorry, like I was saying, kicked out of library post LUG meeting. Thanks for the notes. Taken down to go over in the morning.
<lifeless> we have a rule that says projects which use lp should always use undecided, and ones that don't should always use unknown (because their only bugs hsould be watches)
<rick_h> lifeless: it does take from a list of fields, but the event is fired on one of them not all so it 'assumes' the first one. I'll add a check to it to make sure and fail otherwise
<lifeless> rick_h: Wouldn't it have been easier to code the notification directly in
<rick_h> lifeless: well, I was shown the license change notification stuff in subscribers at the start and took that path.
<lifeless> I have two rules of thumbs
<lifeless> if an event has only one user it doesn't need to be an event
<lifeless> if you have to add special case code because you've lost context it probably shouldn't be an event
<lifeless> rick_h: I can eyeball your branch if you like, rather than talking in generalities
<rick_h> lifeless: well this does have three users, but you're right in the sense that it probably doesn't have to be an event.
<rick_h> lifeless: definitely up for suggestions. It's my first real bit of code on teh zope side so learning more > *
<lifeless> url me up
<rick_h> lifeless: but I'm finally back from LUG dinner/meeting and it's past my bed time. Can you email me notes for the morning?
<rick_h> https://code.launchpad.net/~rharding/launchpad/email_notice_959482
<rick_h> lifeless: ^^
<lifeless> rick_h: thanks
 * wallyworld has an appointment, back in a bit
<lifeless> wgrant: StevenK: is bug 250003 still relecant ?
<_mup_> Bug #250003: package-cache should be optionally limited by distribution <lp-soyuz> <soyuz-core> <trivial> <Launchpad itself:Triaged> < https://launchpad.net/bugs/250003 >
<StevenK> lifeless: If that populates DSPC and other madness -- it could be, yes. wgrant and I want to completly rip all that crap out and have DSP in the DB directly, but ETIME
<bigjools> there's that "should" thing in a bug again
<StevenK> Indeed.
<StevenK> And it mentions hateful things that shouldn't have existed.
<lifeless> bigjools: yes, there it is. I hates.
<bigjools> the bug does not clearly state the problem
<lifeless> indeed
<lifeless> this is not uncommon :)
<lifeless> I'm trying to find a bug
<lifeless> there is a bug about showing folk that are important to a project as such
<lifeless> in bug pages and so on
<lifeless> there is another bug that also says this
<lifeless> I want to dup them
<lifeless> I can't find the former bug ><
<StevenK> We have lots and lots of bugs. News at 11.
<lifeless> wgrant: is bug 372643 fixed by your once-over ?
<_mup_> Bug #372643: Import queue error display: IE7 makes it all one line <easy> <ie> <javascript> <lp-translations> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/372643 >
<lifeless> also bug 376186 is about the most opaque I've seen today
<_mup_> Bug #376186: private bug implicit subscription <disclosure> <lp-bugs> <privacy> <story-better-bug-notification> <ubuntu-qa> <Apport:Won't Fix> <Launchpad itself:Triaged> < https://launchpad.net/bugs/376186 >
<lifeless> [the subject I mean]
<wgrant> lifeless: That one is pretty transparent.
<wgrant> It wants private bug implicit subscriptions
<wgrant> (we'll be fixing it with disclosure)
<wgrant> lifeless: 372634 needs to be checked in a more recent IE
<wgrant> If it works in IE8 it's fine
<wgrant> Since IE7 has no reason to exist
<wgrant> rick_h: Anyway, we probably need to unland the mistaken landing. Any objections?
<lifeless> wgrant: the /subject/ isn't transparent :)
<wgrant> I knew immediately from the summary what the problem was
<wgrant> Prepending 'No ' would possibly have made it a bit clearer, however.
<lifeless> I don't understand the bug, and I've read it
<lifeless> it says
<lifeless> new apport bugs have u-c-u subscribed, but members of u-c-u don't get notifications.
<lifeless> That sounds like 'the system is working but it isn't'
<lifeless> perhaps it means 'we have muted u-c-u via an external contact address and we want folk to have to have a different subscription to get notifications, and they want structural subscriptions to work'
<wgrant> lifeless: u-c-u deliberately doesn't imply notifications, because that would be insane.
<lifeless> so, bug 376186
<_mup_> Bug #376186: structural subscriptions to private bugs don't work even when the subscriber has visibility on the bug <disclosure> <lp-bugs> <privacy> <story-better-bug-notification> <ubuntu-qa> <Apport:Won't Fix> <Launchpad itself:Triaged> < https://launchpad.net/bugs/376186 >
<wgrant> lifeless: But people who have explicitly opted into the package's bugs (through a structural subscription) don't get notified, even if they have access
<lifeless> thats clear.
<wgrant> Right, but bug is that private bugs don't have implicit subscriptions.
<lifeless> no
<lifeless> implicit subscriptions aren't a thing anywhere, they don't exist
<lifeless> the bug is that structural subscriptions aren't working with private bugs - they shouldn't grant visibilty, but they should still grant notification.
<lifeless> thats even the symptoms described, once you know that u-c-u doesn't imply notification
<wgrant> lifeless: Structural subscriptions are one type of implicit subscription
<wgrant> Others include subscriptions through duplicates, or for notification by assignment.
<lifeless> granted
<lifeless> bug 376186
<_mup_> Bug #376186: implicit (e.g.structural) subscriptions to private bugs don't work even when the subscriber has visibility on the bug <disclosure> <lp-bugs> <privacy> <story-better-bug-notification> <ubuntu-qa> <Apport:Won't Fix> <Launchpad itself:Triaged> < https://launchpad.net/bugs/376186 >
<lifeless> which isn't entirely true
<lifeless> because we handle assignment already I think
<lifeless> but I accept your precision; do you accept my critique of the prior subject?
<wgrant> Assignment is handled by some very inconsistent and non-performant and broken special cases, yes.
<lifeless> poolie: bug 660340
<_mup_> Bug #660340: [wishlist] Bug report message editor: want some markup to make the text bold/italic <lp-foundations> <Launchpad itself:Triaged> < https://launchpad.net/bugs/660340 >
<poolie> hi
<poolie> yup, that'd be arguably a dupe
<StevenK> wgrant: Hai. https://code.launchpad.net/~stevenk/launchpad/information_type-vocab/+merge/101486 if you please?
<wgrant> StevenK: Looking
<wgrant> StevenK: Done
<StevenK> wgrant: SimpleTerm doesn't seem to take a description, I went pawing through zope.schema
<wgrant> StevenK: Perhaps description is of our own invention :(
<lifeless> cjwatson: since you're spending some time on publishing things - https://bugs.launchpad.net/launchpad/+bug/616704
<_mup_> Bug #616704: security unembargoes leads to security.u.c overload because primary archive does not have a copy of the debs <lp-soyuz> <Launchpad itself:Triaged> < https://launchpad.net/bugs/616704 >
<StevenK> wgrant: MatchesStructure() is pretty awesome, first time I've used it.
<wgrant> StevenK: That's the point :)
<bigjools> we did a test helper in Maas where you can do assertAttributes(model, attr=, attr2= ...) which uses MatchesStructure
<wgrant> lifeless: Can I nuke the 'specification' bug search order on the grounds that it's stupid and not used by the UI?
<wgrant> (I believe it was added for a (likely false) constraint designed into dynamic bug listings, but never used)
<StevenK> wgrant: Sort by blueprint? Oh, that sounds SO useful.
<StevenK> s/Sort/Search/
<wgrant> It sorts by the alphabetically first linked blueprint name
<StevenK> Oh, I can think of ... no use case for that whatsoever. :-)
<wgrant> (only slightly less useful than the 'tag' sort order, which sorts by alphabetically first tag)
<wgrant> But tag is a bit harder to remove, since it's used by the UI.
<StevenK> wgrant: I vote killing it, since it's just sounds dumb.
<StevenK> s/'s//
<StevenK> wgrant: And in fact, once we drop both feature flags, the entire vocab can just die.
<wgrant> StevenK: Ah, true
 * StevenK peers at Steam.
<StevenK> Now Valve is doing Indie bundles?
<wgrant> They do sometimes.
<wgrant> There were a lot about 18 months ago
<lifeless> wgrant: if it has no search lozenge I see no reason to keep it
<lifeless> wgrant: (spec search order specifically)
<wgrant> lifeless: Yeah
<lifeless> mpt: hey
<wgrant> tag does, unfortunately :(
<lifeless> mpt: I am convinced that bug 522741 is a dup, but I can't find the other copy
<_mup_> Bug #522741: Highlight comments from certain people <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/522741 >
<lifeless> wgrant: sort by tag makes no sense to me; is it getting used ?
<lifeless> wgrant: (e.g. what do the web logs say)
<wgrant> lifeless: Probably only by accident
<wgrant> But I could check
<wgrant> But it has a lozenge, so it's not trivial to remove
<lifeless> data >> nodata
<lifeless> right, but you can start a discussion on the list
<lifeless> where you can document issues like 'can't use it to find an unknown tag, because it doesn't repeat the row per-tag, just sorts with the alpha-lowest tag'
<wgrant> StevenK: You didn't address my concern about duplication with LP.cache.context
<wgrant> In other news, 450 line functions irk me
<StevenK> wgrant: Bah, sorry.
<StevenK> wgrant: I killed it.
<StevenK> I'm going to have fun merging devel into my UI branch, given it's 35 revisions behind.
<mpt> lifeless, I don't remember that being reported before (though I did think of bug 58670 before seeing that it was already linked)
<_mup_> Bug #58670: Highlight comments from the reporter <easy> <feature> <lp-answers> <lp-bugs> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/58670 >
<lifeless> mpt: hmm, I was sure there was something, ah well. I have at least cleaned up some more dross in the tracker
<czajkowski> aloha
<adeuring> good mooriig
<adeuring> ...morning
<lifeless> I need someone to translate bug 253934 for me
<_mup_> Bug #253934: Please bring back the source info to the bug page <feature> <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/253934 >
<lifeless> what do they mean 'source info'
<mpt> lifeless, when a bug page was viewed in the context of a source package it would contain the source package portlet. I removed it as a small step towards (hi wgrant!) making the page the same regardless of what context you were in
<lifeless> thanks
<wgrant> Heh
<wgrant> I think I filed a bug in 2006 or 2007 about it being context-dependent
<cjwatson> lifeless: well, mostly, I second comments #1 and #2
<cjwatson> lifeless: although it occurs to me that a launchpadlib reimplementation of our current script could do a better job of reacting to -security publications without even needing to touch the security team workflow
<cjwatson> maybe
<cjwatson> right now it relies on the package being completely published
<lifeless> wgrant: is zopeless buried yet ?
<wgrant> lifeless: Apart from ZopelessLayer and its derivatives, yes.
<wgrant> So unless it's referring to tests, the bug is a lie.
<lifeless> bug 781014
<_mup_> Bug #781014: View tests should probably not use Zopeless layers <tech-debt> <Launchpad itself:Triaged> < https://launchpad.net/bugs/781014 >
<wgrant> Still present
<wgrant> The main reason they shouldn't use it is the main reason it still exists
<wgrant> LaunchpadSecurityPolicy vs PermissiveSecurityPolicy
<cjwatson> lifeless: anyway, doing *something* with copy-report is on my list, since it needs to be moved off cocoplum
<lifeless> cjwatson: I'd like to think LP has all the bits needed for that bug, just a matter of using em differently
<cjwatson> I think that's probably true, although the changelog comparison we do in copy-report would be a bit tedious pre-publication
<cjwatson> but it can reasonably wait until I reach this particular bit of the grand refactoring of all our tools
<cjwatson> (which is, hopefully, resourced for Q, if secure boot doesn't entirely eat my life which would make me sad)
<cjwatson> oh, sigh, when EC2Test says you need a publicly accessible SMTP server it really means public, doesn't it
<wgrant> cjwatson: Everyone just uses smtp.canonical.com
<cjwatson> yeah, changing it according to EmailSetup now
<lifeless> wallyworld: should https://bugs.launchpad.net/launchpad/+bug/885503 be F-R?
<_mup_> Bug #885503: Its not obvious on the UI how to choose to have all bugs reported against your project marked private by default. <bugtracker> <disclosure> <documentation> <privacy> <qa-ok> <Launchpad itself:Fix Committed by wallyworld> < https://launchpad.net/bugs/885503 >
<cjwatson> lifeless: linked from https://wiki.ubuntu.com/FoundationsTeam/ReplaceArchiveAdminShellAccess so I don't forget
<lifeless> cjwatson: thanks!
<lifeless> mpt: hah - bug 251479
<_mup_> Bug #251479: bug pages should list the published versions <feature> <lp-bugs> <Launchpad itself:Triaged> < https://launchpad.net/bugs/251479 >
<mpt> lifeless, so if people still want that, and while we still have multi-task bug reports, the challenge then would be to work out how to present it sensibly no matter how many packages are there
<lifeless> mpt: I've suggested in the bug to have the expander that currently shows the old style form, show details about the context.
<mpt> good idea
<wgrant> No no no
<wgrant> Don't encourage making multi-task bugs more workable
<wgrant> Introduce obstructions to their functionality so we can better justify their abolition :)
<lifeless> I suspect we could close every bug with 'should' in the title with no degradation of the LP bug tracker
<lifeless> wgrant: you might want to check bug 415223 and see if its still broken, after all the subscription rework
<_mup_> Bug #415223: "Subscribed by Some Person" tooltip after inline subscription is wrong <lp-bugs> <story-better-bug-notification> <Launchpad itself:Triaged> < https://launchpad.net/bugs/415223 >
<wgrant> lifeless: That work never happened :)
<nigelb> /ws/ws 20
<lifeless> wgrant: did you end up nuking lazr.restfuls representation cache ?
<wgrant> lifeless: No, just the LP integration
<lifeless> more LoC just hanging around
<lifeless> bigjools: why did you keep bug 613642 open given Ubuntu saying 'no never' ?
<_mup_> Bug #613642: PPA pages have two different sets of instructions for adding them, rather than providing a button <doc> <lp-soyuz> <ppa> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/613642 >
<jml> do I need to take any action to deal with my buildbot rejection?
<nigelb> jtv: Everything okay out there?
<jml> Hmm.
<jml> I see no one has fixed it.
<jtv> Hi there nigelb!
<jml> So I guess that means "yes".
<nigelb> jtv: Ohai. Context of that question - earthquake!
<jtv> jml: hi there you too.  Buildbot problems?
<jtv> nigelb: Just heard about it.  Didn't notice it.  Mind if I go private?  Don't want to distract the channel from Jono's problem.
<nigelb> oh sure
<lifeless> rick_h: I have added a review to https://code.launchpad.net/~rharding/launchpad/email_notice_959482/+merge/99995
<lifeless> gmb: you may want to read my comment on it too, as you reviewed it initially ^
<jml> jtv: yeah, a submtted branch failed. wgrant said earlier it was a conflict, but I can't tell if someone has already addressed the issue.
<bigjools> lifeless: I missed your ping earlier and now I switch clients, what did you want?
<wgrant> jml: It seems that rick_h inadvertently landed a branch that was not ready
<wgrant> jml: And, unrelatedly to that, it conflicted with your ignored = branch
<wgrant> Well, not a bzr conflict.
<jml> wgrant: yeah, I understand.
<wgrant> But some tests that it added are broken
<lifeless> bigjools: there is a bug 613642 , but AIUI Ubuntu have said 'we do not want just-a-button for PPAs'
<_mup_> Bug #613642: PPA pages have two different sets of instructions for adding them, rather than providing a button <doc> <lp-soyuz> <ppa> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/613642 >
<wgrant> I was hoping to talk to rick_h to get that branch reverted
<wgrant> But he probably went to sleep
<lifeless> bigjools: you said in the bug you were keeping it open because it was clear, but as Ubuntu don't want it, I'm not sure why clarity is a reason to keep it open/.
<jml> wgrant: OK. Maybe it's a good idea to fall back to asynchronous communication?
<bigjools> lifeless: I'm sure I had a good reason but IBF if I can think of it now
<wgrant> jml: I intend to talk to him when he comes online in a couple of hours.
<lifeless> bigjools: I'll leave it for now, let your brain sleep on it
<bigjools> lifeless: I'd be happy to close it I think
<jml> wgrant: ok.
<lifeless> bigjools: would you lke me to click the button for you ?
<bigjools> lifeless: done :)
<lifeless> \o/
<lifeless> I've been procrastinating on allhands.c.c with this
<jml> procrastination, is the answer, procrastination, doing it later! dig it.
<nigelb> Quick question - also probably stupid question - is there a way I can run LP on OS X?
<lifeless> virtualbox
<nigelb> Aww. ok. that's what I'll do then.
<lifeless> in principle you can run it natively, yes
<wgrant> It's probably possible to run LP directly on OS X, as long as you have libapt and the like
<lifeless> but frankly it would basically be a nontrivial time investment for no reward
<wgrant> But it hasn't been done since 2005/2006.
<nigelb> lifeless: Yeah, that's what I was not sure about.
<nigelb> The good thing is I can let rocketfuel take over the virtualbox.
<nigelb> So it should be trivial at that end.
<lifeless> I do all my dev in lxc
<wgrant> Me too
<lifeless> e.g. I don't even run LP on Ubuntu directly
<wgrant> LXC probably doesn't run on OS X, though :)
<lifeless> some porting required
<nigelb> wgrant: Damn. I was hoping it would work.
<lifeless> nigelb: how much memory does your machine have?
<nigelb> lifeless: 2 GB *cough*
<lifeless> nigelb: so, run a server install, no x or display manager
<wgrant> i386, not amd64
<lifeless> you'll need the X client
<lifeless> ssh into the vm to do stuff
<lifeless> you can use a native X server if you want to run up firefox etc within the vm
<nigelb> I wanted to setup a headless VM
<nigelb> and then hit it from outside
<nigelb> that probably needs some hacking to get the domains right.
<lifeless> its documented on the wiki
<nigelb> I have a few LP MP/bugs that I want to close.
<nigelb> oh cool. this should be fun.
<lifeless> (naturally enough, because thats how everyone developing with LXC gets at their dev instance)
<nigelb> Oh right.
<lifeless> I really need a 'this will make wgrant happy' template for bugs
<nigelb> heh
<wgrant> lifeless: Oh?
<lifeless> bug 978768
<_mup_> Bug #978768: read-only mode code is now dead code <tech-debt> <Launchpad itself:Triaged> < https://launchpad.net/bugs/978768 >
<wgrant> Hah
<wgrant> That will make me slightly sad, actually.
<lifeless> oh?
<wgrant> Although maybe the current implementation is not worth keeping, I guess.
<lifeless> the current implementation is not at all what you have advocated for
<wgrant> Indeed
<wgrant> It's also horribly buggy.
<lifeless> so, we should toss it.
<lifeless> *if* we go for a restricted, real-time, no-writes-permitted thing, it would need to be a lot leaner and more reliable to be better than FDTW
<wgrant> Yep
<poolie> lifeless, ok, markdown baby tossed
<lifeless> its a bit sad, but perhaps someone will pick it up
<poolie> maybe we will finish it saturday
<poolie> it's not that much more actual code but it takes some chasing to get it landed and qa'd
<poolie> lifeless, gc-ing it after 2 months makes sense
<rick_h> wgrant: sorry, yes went to bed. I've got to get hte boy to day care and I can look at the rollback process.
<rick_h> lifeless: thanks for the review. I did discuss the code LoC increase with deryck and it was thought that with some previous JS changes and hte fact that this is a hot security issue that getting out faster was better at the moment
<wgrant> rick_h: Hm, wasn't expecting you for another 2.5 hours anyway :)
<rick_h> wgrant: ok, well should be < 1hr. Sorry about the builtbot blockages
<lifeless> rick_h: its the overall thing we look for, but the absence of discussion in the review is a red flag
<wgrant> rick_h: Great.
<lifeless> rick_h: it either means it was done out of band, which is ok, or wasn't considered, and an external party cannot tell:)
<rick_h> lifeless: right, I should have noted the discussion in my MP, sorry about that.
<lifeless> rick_h: nevertheless, it can be much smaller ;)
<wgrant> :(
<wgrant> Someone just registered a few dozen distinct imports under kdemultimedia
<wgrant> Ah, only a dozen, just looked like a lot more
<jelmer> wgrant: that would be the project neon folks
<wgrant> So it seems
<wgrant> That is even more worrying
<wgrant> As we're pushed for buildd time already
<rick_h> wgrant: should this back out be sent through ec2 then? or just pqm submit?
<wgrant> rick_h: pqm-submit
<rick_h> with [rollback=15075] in the commit message, is there anything else I need to make sure is in there?
<wgrant> Nope, just [r=foo][rollback=bar], where foo is usually yourself
<rick_h> thanks
<czajkowski> wgrant: see over there --->
<rick_h> wgrant: oh, need [testfix] as well?
<StevenK> rick_h, wgrant: I was about to lp-land my vocab branch. Shall I wait until your rollback lands?
<wgrant> rick_h: Ah, yeah, you'll need testfix since buildbot finished with the broken branch
<rick_h> is there a web ui view for pqm aside from the deployment report I can monitor for this hitting?}
<wgrant> rick_h: pqm.launchpad.net
<wgrant> But it'll only be there for ~45seconds
<rick_h> wgrant: ah ok
<rick_h> I'll keep pushing my email. offlineimap ftw except when you want an email *NOW*
<rick_h> bah, testfix has to be the first one, *sigh*
<rick_h> ok, rollback success from pqm
 * rick_h crosses fingers it works out from here
<StevenK> rick_h: You can't really self-review, you're still being mentored. But I think we can let it slide.
<rick_h> StevenK: thanks, if it makes you feel better jcsackett has started looking at getting me out of mentorship :)
 * StevenK makes a note to fix that on the call tomorrow. :-P
<rick_h> doh
<StevenK> Haha
<nigelb> it's a trap :P
<jml> gary_poster: let me know if https://code.launchpad.net/~jml/testtools/forward-current-tags/+merge/101538 addresses your issues with testtools tag support.
<gary_poster> jml, cool will review thanks.  benji would also review, as he's been doing a lot of the work in the area, but he has internet connectivity issues today.
<gary_poster> jml, that looks very good to me in terms of subunit contract and in terms of the tag context approach.  typo line 477 of diff: restorted. I don't understand meaning of Python27Contract subclass and have not explored, but it implies to me that this the contract will not be expected/maintained for 2.6.  Do I understand correctly?  If so, why would we only want this in 2.7?  testresult.real.ThreadsafeForwardingResult a
<gary_poster> lso does not conform to subunit tag contract correctly, but we have work in that direction, as well as another test result implementation, so we can run with your branch once it is landed and hopefully use your testcases to test those test result classes
<jml> gary_poster: Python27Contract is the contract for stdlib unittest.TestResult in Python 2.7
<jml> gary_poster: and likewise for 2.6
<gary_poster> jml, but I thought your MP said that tags were not in 2.7...rereading
<jml> gary_poster: the previous tests implied that Python 2.7 unittest.TestResult had tags()
<jml> gary_poster: and thus our test double object had tags()
<jml> gary_poster: but that was wrong, making it a bad double.
<jml> gary_poster: I'm also fairly sure that this branch fixes ThreadsafeForwardingResult to conform to subunit tag contract.
<gary_poster> jml, right, ok.  So...TagsContract will be expected to be followed in both 2.6 and 2.7, right?  That's my only concern, really, despite the fact that I'm still in a bit of a fog about the intent of that subclassing (reading more code would help, I suspect(
<jml> gary_poster: the subclassing is sort of a kludge for not using testscenarios
<gary_poster> Threadsafe...: really!  lemme look again
<gary_poster> subclassing: ok
<jml> gary_poster: by inserting TagsContract into the tree where we did, that tests against TSFR.
<gary_poster> jml, ok cool.  I believe ThreadsafeForwardingResult still has some issues, but perhaps they are mitigated somehow.  benji and/or I will review later.  However, _not_empty_tags is simply wrong AFAWCT (minimally should be boolean or, not boolean and, and arguably should be "bool(tags[0] - tags[1])" from some kind of purity perspective).  Similarly, _add_result_with_semaphore reverts the _global_tags at the end of the
<gary_poster> function, which we believe is a typo/thinko
<jml> gary_poster: oh right, the forwarding might well be busted. I don't understand it, tbh.
<gary_poster> As I said, we've been staring at this for a while, so we'd be happy to review again once this has landed (especially if it lands soon)
<gary_poster> (review the forwarding bits I mean)
<jml> gary_poster: I was sort of hoping that making TagContext would have helped me understand / get rid of it, but no, not really :\
<gary_poster> heh
<jml> gary_poster: I should have been more insistent at the code review point.
<gary_poster> jml, you can be insistent when we propose some changes to it, and that will help everything along :-)
<gary_poster> We'll see if we can use TagContext in there too; I suspect we can.
<jml> gary_poster: thanks :)
<gary_poster> jml, any idea how soon this will land?  I'll suggest to benji that we work from your branch if it won't be within an hour or so
<jml> gary_poster: I'll try to get another testtools person to review
<gary_poster> cool jml, thanks
<jml> gary_poster: oh. oops. lifeless is the only other person with commit access.
<gary_poster> :-)
<abentley> rick_h: Your r15075 broke buildbot.  You added login_person calls to doctests, but r15074 makes login_person return a value, which breaks doctests.
<rick_h> abentley: it's been rolled back this morning
<rick_h> abentley: I replied to the email, I've got work to do on it still. Should hopefully fix soon
<abentley> rick_h: Ah, okay.  The mail was more about buildbot failing to notify you.
<rick_h> abentley: ok, just grabbed one of the failure emails to reply to this morning.
<abentley> rick_h: We're talking about different mails.  I just sent a mail complaining about buildbot blaming *me* for this.
<deryck> abentley, adeuring, rick_h -- can one of you paste me the link for the standup hangout?
<abentley> deryck: https://plus.google.com/hangouts/_/extras/talk.google.com/orange-standup#
<deryck> thanks!
<abentley> https://dev.launchpad.net/PolicyAndProcess/MaintenanceCosts
* abentley changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: abentley | Firefighting: - | Critical bugtasks: 4*10^2
<jml> flacoste: btw, it would be great if the MaintenanceCosts doc included or linked to some helpful suggestions on what to do to reduce maint costs to pay for a patch.
<flacoste> jml: you mean like a task list?
<jml> flacoste: yeah
<flacoste> indeed, that's a good idea
<jml> flacoste: but not necessarily exhaustive
<flacoste> right
<jml> e.g.
<flacoste> just a getting started
<jml> - remove sample data
<jml> - convert a doctest to a unittest
<jml> that sort of thing
<flacoste> ah, so more "principles" based
<flacoste> rather than fix bug 12345
<_mup_> Bug #12345: isdn does not work, fritz avm (pnp?) <isdnutils (Ubuntu):Fix Released by doko> < https://launchpad.net/bugs/12345 >
<flacoste> or bug 123456
<james_w> both would be good
<_mup_> Bug #123456: podcast crashes amarok <Amarok:Invalid> <xine-lib (Ubuntu):Fix Released> < https://launchpad.net/bugs/123456 >
<jml> flacoste: well, that might be nice too.
<jml> flacoste: basically, something to get someone started who isn't heavily involved in the code-base and feeling the costs.
<nigelb> Er, since when did we have a reduce LoC mission?
<cjwatson> Since 2012-02-10.
<nigelb> Ah, that's why it felt new.
<nigelb> I did wonder, I hadn't heard of this when I was actively writing code.
<cjwatson> https://lists.launchpad.net/launchpad-dev/msg08923.html
<nigelb> Also. Should learn to scroll. That info was right on the page. :|
<abentley> jml: We could also have a list of dead code, but that raises the question "why not just delete it now?"
<jml> abentley: well, why not?
<cjwatson> https://lists.launchpad.net/launchpad-dev/msg08903.html had some specific suggestions from lifeless
<abentley> jml: Because if we save it up, then we don't have to work so hard later to balance a LOC addition :-)
<cjwatson> If people start saving things up, the policy is arguably counterproductive ...
<jml> right
<abentley> cjwatson: Yes, therefore the smiley.
<jml> abentley: is there a way to discover dead code?
<nigelb> Hrm, I'm curious, as an external contributor, how can I help? Because most of my MPs were adding code. Not deleting them. Not sure I know enough about Lp to delete code.
<cjwatson> abentley: fair enough, it can be hard to tell
<cjwatson> jml: https://bugs.launchpad.net/launchpad/+bug/420338 ;-)
<_mup_> Bug #420338: Tool for identifying dead code <build-infrastructure> <feature> <lp-foundations> <tech-debt> <Launchpad itself:Triaged> < https://launchpad.net/bugs/420338 >
<abentley> jml: Nothing foolproof, but checking the coverage of the test suite would give hints.
<cjwatson> nigelb: I've been doing a fair bit of deletion as an external contributor
<jml> nigelb: yeah, likewise. that's what started off this conversation ;)
<abentley> jml: In some cases, though, the test suite is the only caller.
<nigelb> cjwatson: *cough* you know more about LP :)
<nigelb> jml: Ah!
<cjwatson> nigelb: in my case it's just come down to intuition ("hmm, that looks a bit crufty") and grep to confirm
<jml> nigelb: the only way to learn is by randomly deleting stuff.
<nigelb> jml:  Or turn into StevenK :D
<jml> cjwatson describes it better.
<jml> nigelb: please don't. one is enough.
<nigelb> haha
<nigelb> I actually would love to see tagged bugs or something.
<nigelb> If there's low hanging fruits
<cjwatson> there's the tech-debt tag, which has some useful ideas
<cjwatson> although with a wide range of difficulty
<jml> oh yeah, that one.
<nigelb> I should resucitate my ubuntu laptop to try this out.
<sinzui> jcsackett, you make a good point about the Ubuntu bug supervisor
<wallyworld> sinzui: hello
<wallyworld> bug 978212
<_mup_> Bug #978212: Sharing batch navigation shows ugly 404 error. <disclosure> <ui> <Launchpad itself:In Progress by wallyworld> < https://launchpad.net/bugs/978212 >
 * jml considers doing a test run with coverage.py
<sinzui> jcsackett, The ubuntu bug supervisor is not subscribed when the private bug is created, apport is the only party that can access it. When a user changes the bug to private, the ubuntu bug supervisor is being subscribed?
<wallyworld> sinzui: it's caused by the PillarNavigationMixin added for the sharing details page. the mixing processes traversal of xxx/+sharing/name
<wallyworld> sinzui: but the batching js sends urls like xxx/+sharing/++model++ and the navigation mixin gets confused
<wallyworld> sinzui: i have a fix but i'm not happy with it. i'd rather change the url for the sharing details page to xxx/+sharingdetails/name
<jml> sinzui: want to have a call w/ james_w & I today?
<sinzui> wallyworld,  I think We want to extend the @stepto. The URL looks natural...if I hack the URL I get a 404
<wallyworld> sinzui: the url looks natural but what is sent includes the ++model++
<wallyworld> sinzui: my lame fix which does work is https://pastebin.canonical.com/64098/
<sinzui> wallyworld, I think that is the correct fix...the application should not dictate the URL
<wallyworld> sinzui: the browser location doesn't show the ++model++ even though that's what is sent via the post
<sinzui> jml yes. I am available starting at 16:00 UTC
<wallyworld> sinzui: so you favour retaining xxx/+sharing/name for the sharing details view url?
<sinzui> yes.
<jml> sinzui: thanks.
<jml> re dead code: http://pypi.python.org/pypi/vulture might be interesting.
<wallyworld> sinzui: ok. is my first attempted fix which does work but i know is wrong in the right area at least?
<sinzui> wallyworld, The underlying issue is that ++model++ which is an internal mechanism was exposed in the URL is an nasty way
<wallyworld> sinzui: that's out of my control - it's what the js batch navigation infrastructure sends through
<wallyworld> and it was harmless until we introduced the extra traversal stuff for the details page
<wallyworld> i think the idea is that the js batch navigation stuff is simply asking the view to init and populate the json cache
<wallyworld> and the ++model++ namespace is used to do this
<wallyworld> so it seems an established pattern
<sinzui> wallyworld, it was harmless except it was introduced in ignorance that we use @steptrough in a lot of places and we frustrate users try to make that hack work... ~team/+members and ~team/+member/sinzui is bong
<wallyworld> sinzui: sorry, i should have typed "harmless"
<sinzui> wallyworld, also, try putting to batch navigators on the same page as is also done on +members and you will see it is truly fuxked
<wallyworld> sinzui: so seems like we have a form of infrastructure friction here
<sinzui> yes
<sinzui> wallyworld, I think the real issue is that @stepthough should ignore ++ because it is a special rule
<wallyworld> that sounds reasonable i think, but if it ignored it i wonder if the ++model++ contract would still be met
<sinzui> wallyworld, consider using ++oops++ on a page with an @stepthrough...it will fail
<sinzui> wallyworld, so this issue is not new.
<jml> flacoste: thanks!
<wallyworld> yes, it would wouldn't it
<wallyworld> ok. i'll get myself off to bed and revisit in the morning. thanks for providing the input. i didn't realise the issue was sort of known
<sinzui> jml: I cannot meet until 16:30 UTC. I need to get my son from school
<jml> sinzui: no worries. can you please ping us when you're back?
<sinzui> fab
<abentley> jml: Vulture looks interesting.
<rick_h> abentley: quick ? so I wanted to pull devel with my branch to keep working on it
<rick_h> abentley: except devel has a rollback that undoes my branch
<rick_h> abentley: any clue on what I *should* have done?
<jml> abentley: yeah. I just ran it. It generates warnings for methods that are part of an exposed API.
<rick_h> I'm assuming there's a right order/process to keep my work while getting devel's changes so I can fix the tests/etc.
<abentley> rick_h: You want to merge, not pull.  Merge up to the rollback first.
<abentley> rick_h: Then merge the rollback, but revert all its changes.
<rick_h> abentley: ok
<rick_h> ty
<abentley> rick_h: So: bzr merge -r 15076; bzr commit; bzr merge -c 15077; bzr revert ".";  bzr commit;
<abentley> rick_h: If you've pulled devel, you can reset your branch to its old revision with "bzr pull -r 15010.1.25 --overwrite"
<rick_h> abentley: ok, thanks
<abentley> rick_h: np.
<sinzui> jcsackett, I think your reply to the branch in the review hints that if we make the change I propose, Ubuntu's bug subscription behaviour will change
<jcsackett> sinzui: yeah. the current behavior, as regards bug supervisor, wasn't changed; just cleaned up.
<jcsackett> mind you, that doesn't mean the behavior is *right*. :-P
<sinzui> jcsackett, Canonical noticed the change in behaviour...not ubuntu
<jcsackett> sinzui: dig. so ... leave it? i think your right that the current behavior is actually wrong.
<sinzui> jcsackett, I think the rules are wrong. wgrant might say the rules are right because the apport is not using the chunks of code we are changing
<jcsackett> sinzui: well, i can sit on it till tonight, and we can talk to wgrant.
<sinzui> jcsackett, I am going to make the review as needs information, and we can discuss this as a group
<jcsackett> sinzui: dig.
<adeuring> abentley: could have a look this mp: https://code.launchpad.net/~adeuring/lazr.jobrunner/slow-lane/+merge/101576 ?
<jcsackett> i'll leave off changing it then, and not worry about landing it today.
<abentley> adeuring: with pleasure.
<jcsackett> sinzui: i was thinking of tackling bug 968253 today. any landmines you can think of, or is this a pretty straight forward case of just getting the banner to display?
<_mup_> Bug #968253: +archivesubscriptions pages do not clearly state that they contain private information <disclosure> <privacy> <Launchpad itself:Triaged> < https://launchpad.net/bugs/968253 >
<sinzui> jcsackett, I think so. You might notice that there is a related bug. the bread crumbs are missing/wrong
<jcsackett> sinzui: i did not notice that, but it might explain why the banner isn't showing if its supposed to.
<jcsackett> sinzui: oh, or i can take the details closed bugs issue; did y'all come to a decision on that last night?
<sinzui> right, without the breadcrumb adapter, lots of things are subtly broken
<abentley> adeuring: Did you consider making fallbackToSlowerLane a contextManager?  Then it could re-raise the exception itself if there was no fallback available.
<sinzui> jcsackett, that needs research. I see some closed bugs are listed...this issue might also be about the fact that duplicates are often omitted from searchTasks()
<jcsackett> sinzui: ah, right.
<jcsackett> sinzui: ok, i'll go after the privacy banner/breadcrumb bit first. if that's a relatively quick resolution, i'll pry into the details page issue some.
<sinzui> I cannot be certain at the moment but bug search is really mediocre for researching privacy issues, and team leads can not access the staging databases anymore...I have no idea which bug Lp thinks the example user has access to
<abentley> adeuring: 119 looks like acccidental whitespace.
<adeuring> abentley: fixed. (the whitespace) ANd no, I did not consider to define fallback/oSLowerLane as a context manager.
<adeuring> abentley: I think we should keep the exeption handling in one place though, in runJob()
<abentley> adeuring: cool.
<adeuring> Otherwise, it will be hard to keep an overview that the job status is correctly updated
<abentley> adeuring: You don't actually need RunFileJobNoResult.  You can just pass ignore_result=True to apply_async.  This works for most (all?) Task properties.
<adeuring> abentley: ah, ok, I'll try that
<adeuring> abentley: this does not work. With this change: http://paste.ubuntu.com/924985/ , I get again errors about messages that are stuck in rabbitmq...
<abentley> adeuring: Wacky.  Okay, I guess leave it as is for now.
<adeuring> yeah...
<abentley> adeuring: It might be helpful to distinguish between the noun and verb forms of queue by using "enqueue" for the verb.
<adeuring> abentley: yes... where exactly?
<abentley> adeuring: I think 147?
<adeuring> abentley: ok, I'll fix this
<adeuring> abentley: ah, you mean the function name? that would requires more change in the main LP code...
<abentley> adeuring: Let me look at this.
<abentley> adeuring: My bad.
<abentley> adeuring: We might want to change both, but we can't change just one.
<adeuring> abentley: right...
<adeuring> abentley: to add fun for such a name change: "grep -Ir job.*queue lib/lp" shows this result: lib/lp/answers/notification.py:            self.job = self.enqueue()
<abentley> adeuring: why is runJob accepting a callback now?  It could just call reQueue, which would return False if no FALLBACK_QUEUE was configured.
<adeuring> abentley: JobRunner does not know about the _task_, it knows only about the job. But reQueue is a task method
<abentley> adeuring: Makes sense.
<jml> am having trouble firing up an instance with ec2 demo: getting a socket timeout during the ssh connection.
<abentley> adeuring: r=me
<adeuring> abentley: thanks!
<jml> any known gotchas there? anyone else able/unable to use 'ec2 demo'?
<jcsackett> jml: i think wallyworld had issues doing ec2 demo as well. there was some discussion of using it (and possible bit-rot?) on the launchpad-dev list. look for wallyworld posting about mockups and best practices.
<jml> jcsackett: thanks.
<jml> jcsackett: nothing particularly helpful there, I'm afraid. I'll see if I can get 'ec2 test' to work
<jcsackett> jml: sorry it wasn't useful.
<jml> jcsackett: no worries
<jml> jcsackett: I really appreciate getting any response at all
<jcsackett> well, we try to not let this channel be completely like shouting into a hole. :-)
<abentley> adeuring: Does this change require additional packages?
<adeuring> abentley: in theory, no. But the tests require a package like rabbit-management or so, and this is currently not available in precise. the LP PPA needs to be updated...
<adeuring> well, it is avaibale, but depends on the wrong version of rabbitmq-serverm IIRC
<abentley> adeuring: Could you specify the exact package so I can run the tests, please?
<adeuring> abentley: I _think_ it is rabbitmq-management
<adeuring> but htere are several broken rabbitmq dependencies...
<abentley> adeuring: Okay.
<abentley> adeuring: Given the circumstances, I think it makes sense to make the queue check optional.  Otherwise, I won't know if any of the TestCeleryD tests start failing.
<adeuring> abentley: like a try/except around the checks in setUp() and tearDown()
<abentley> adeuring: Yes.
<adeuring> ok, I'll add that
<abentley> adeuring: thanks.
<jml> so I'm not going to do a coverage run, because I can't launch an ec2 instance for Launchpad.
<cjwatson> jml: last night lifeless and others helped me fix a problem I was having with ec2, which turned out to be that ec2 was detecting my IP address via a proxy which caused it to enable a private IP address in the instance's security policy, which didn't work so well.  'env -u http_proxy ec2' fixed it.
<jml> cjwatson: interesting. thanks.
<cjwatson> jml: you might see what checkip.amazonaws.com says
<gary_poster> I'm stumped.  In an sh script, if dir is defined, what is ${dir##*-} ?  I can't find a value for dir that gives anything, and I have not been able to find docs on that syntax
<cjwatson> gary_poster: strip the longest segment matching the pattern *- off the start of $dir
<cjwatson> gary_poster: man dash, search for "##"
<cjwatson> (or bash)
<gary_poster> cjwatson, awesome thank you much
<cjwatson> (to be clear, that doesn't modify $dir)
<gary_poster> cjwatson, ack, thanks.
<jml> cjwatson: it's giving me 10.45.43.105, which does look internal (my shell server says I'm connecting from 91.189.88.12)
<jml> cjwatson: but http_proxy is unset
<cjwatson> there might be a "transparent" proxy in the way I guess; anyway, that's probably the problem
<cjwatson> worst case hack ec2 to ignore what checkip says ...
<rick_h> anyone else getting 503 from codehosting? https://code.launchpad.net/+loggerhead/~rharding/launchpad/email_notice_959482/diff/15040/15035
<jml> cjwatson: yeah, I was just giving that a try  :)
<jml> if it works, I'll add an command-line option to override the public IP
<sinzui> hi jml.
<jml> sinzui: oh hi
<jml> sinzui: good timing
<jml> sinzui: which voice communication technology works for you?
<sinzui> hangout and mumble
<jml> sinzui: let it be hangout!
<jcsackett> sinzui: are you around? need to bounce an idea off someone.
<sinzui> in a meeting at the moment
<jcsackett> sinzui: dig.
<benji> jml: a quick question: I branched from lp:~jml/testtools/forward-current-tags and added a test (http://paste.ubuntu.com/925149/) and it failed (http://paste.ubuntu.com/925150/).  Should that have suceeded, or am I missing something?
<jml> sinzui:         # The software center agent can view commercial archives
<jml>         if self.obj.commercial:
<jml>             return user.in_software_center_agent
<jml> ViewArchive
<jml> benji: looking...
<jml> benji: TestThreadSafeForwardingResultContract already exists
<jml> benji: and is tested with StartTestRunContract, which includes DetailsContract.
<jml> benji: what's the failure?
<benji> jml: heh, indeed it does; it's slightly different though
<benji> http://paste.ubuntu.com/925150/
<jml> benji: and it passes.... perhaps that's the difference? :P
 * jml looing
<jml> looking
<benji> jml: ok, I /think/ that the failure is to be expected, test_addUnexpectedSuccess_was_successful is overridden in a subclass and apparently different things are expected in different contexts
<jml> benji: ah right. yes, that's it.
<jml> benji: sorry about the contract subclass thing. I think it would be clearer if we used testscenarios and each thing listed their contracts explicitly
<mpt> oy crikey, Launchpad bug comments contain <table> elements with negative margins
<mpt> Why? Why not?
<jml> benji: there's also TestThreadSafeForwardingResult, which I think deserves some tests to show that tags are forwarded correctly.
<benji> jml: yeah, I'm thinking the same thing, I just removed a bunch of tag-forwarding code from TestThreadSafeForwardingResult (http://paste.ubuntu.com/925170/) and the tests still passed
<jml> benji: right. I sort of leaved that untouched because I couldn't quite grasp the intent of what's going on in TSFR
<jml> benji: I'm guessing that it's buffering up tags() calls and then playing them back on stopTest() in a way that makes them equivalent for single isolated test.
<benji> jml: I think some of what it does is somewhere between broken and unneccesary.
<jml> benji: a plausible hypothesis!
<benji> jml: that's the spirit of how it works
<benji> jml: ok, I'll see about adding some tests for TSFR; I need something much like it and the tests can probably work for both TSFR and my new thing
<jml> benji: cool.
<jml> benji: what's the thing you need?
<jml> benji: btw, as a general thing, please flag any bugs in testtools, subunit, testrepository etc. that are getting in the way or causing friction. I'm happy to work on them, am easily distracted and won't get punished for helping colleagues during work time.
<benji> jml: heh, ok
<benji> jml: I just noticed that you asked about what I'm working on.  Since we're doing parallel runs, we want all the tests that were run together in a given process to have a unique tag so we can figure out which tests were run together, primarily for debugging test interactions and isolation failures
<jml> benji: ah right. and presumably the tagging is easy but when you draw the streams together the tagging info gets lost/mangled?
<benji> jml: hmm, well, I suspect you know something I don't.  Why do you presume the tagging is easy?  The easiest way I can find to do it requires tweaking ConcurrentTestSuite to insert the thing I'm making which will add a tag that identifies the worker to the stream.
<jml> benji: is anything more than http://paste.ubuntu.com/925211/ necessary?
<benji> jml: perhaps that would work; does the fact that the tag is global in process_result translate correctly when the streams are aggrigated?
<jml> benji: ahh, I see, probably not. you want to be able to tag each and every test individually.
<benji> right
<jml> (maybe it should, but that sounds unreasonably difficult)
<jml> Hmm.
<benji> I would expect that all the global tags would be added to the aggrigates stream.
<jml> Actually, tell a lie, such an aggregator would be easier but strictly more work than your approach
<jml> s/easier/easy/
<jml> benji: the way I'd go about this is to make a TestResult decorator that was given some tags in the constructor and then tagged every test with those tags.
<jml> benji: and then tweak ConcurrentTestSuite to have a result_factory or something
<jml> benji: but that's only the thing I thought of just then. ymmv.
<benji> jml: yep, that's the apprach I'm taking (if you consider TSFR a TestResult decorator, because my thing will be structurally similar)
<jml> benji: hmm.
<jml> benji: I forgot that there's a thing called TestResultDecorator in subunit/test_result.py (I thought it was in testtools)
<jml> it *should* be in testtools, dammit.
<jml> and there it is: https://code.launchpad.net/~jml/testtools/test-result-decorator/+merge/101621
<jml> I think I've hit on something.
<jcsackett> sinzui: out of meetings?
<sinzui> jcsackett, yes sorry for not saying so last hour
<jcsackett> sinzui: all good, gave me time to investigate some approaches. now i have an idea of what will and won't work, but i'm not sure what's best. :-P
<sinzui> hangout?
<jml> benji: I hate to be pulling you in different directions, but I've just uploaded https://code.launchpad.net/~jml/testtools/test-by-test-result/+merge/101622. If that and the two other branches were in trunk, I would try rewriting TSFR in terms for TestByTestResult, and would probably consider writing your thing in similar terms
<benji> jml: looking
<jml> benji: but I'd hate to build a thing on top of three unlanded branches -- too much WIP -- so I'm not going to attempt that until tomorrow.
<jml> benji: I'm packing up now. Leaving in another 5m
<benji> jml: I think I see how that might be used to do what I want.  I'll see which approach turns out better.
<jml> benji: cool.
<jml> benji: please be in touch via email if you have any questions.
<jml> IRC backlog kind of works but isn't so great.
<benji> jml: will do, thanks
<abentley> deryck: I'm the OCR, so could you please review my branch? https://code.launchpad.net/~abentley/launchpad/celery-everywhere-3/+merge/101626
<deryck> abentley, sure.  I'm finishing reviews today, so it may be a bit before I get to it if that's okay.
<abentley> deryck: no problem.
<rick_h> lifeless: morning if you get a chance I wonder if you can peek at and possibly sign off on the updated branch from last night? https://code.launchpad.net/~rharding/launchpad/email_notice_959482/+merge/99995
<lifeless> rick_h: did you try without events ?
<rick_h> lifeless: no, that came up in our discussion, but I went off of the comments in the MP and went for straight code dupe reduction with the current setup.
<lifeless> ok, so its definitely a bit leaner - thanks.
<lifeless> I'll do a spike today for a no-event version, I want to see how much better (or not) it is
<lifeless> rick_h: e.g. lines 406 to 420 in the diff are there to deal with bad events
<rick_h> lifeless: right, I can go that route. When it came up was when you mentioned it having '1 call place' when it had 3 so I kep the event bit to trigger.
<rick_h> lifeless: but yea, you could probably drop those and a few lines of zcml/import with non-event method
<rick_h> lifeless: thanks for poking at it, I'm definitely happier with things overall than I was the other day.
<lifeless> cool. If you're not EOD and want to try a non-event version of this, great; if you are, I'll put the effort in to see what it looks like
<lifeless> e.g. I don't want to block you, but I do want to see what it looks like before approving it :)(
<lifeless> s/(//
<rick_h> yea, I've got to run and get the boy from day care, but I'll be at the coffee shop hacking later tonight with friends so can chat for a little bit then if we need to
<lifeless> righto, experiment is on me then ;)
<abentley> lifeless: Currently, my tests start and stop a celeryd instance for each test.  I'm thinking that I should switch to using a layer, but is that the current best practise?
<lifeless> the current general thing to do is to create a Fixture for the daemon
<lifeless> and have a layer that brings that up and tears it down
<lifeless> you can use the per test calls in the layer to reset the state of the daemon without restarting it, if thats desirable
<lifeless> so yes, we still use layers, but we try to have no logic in them
<abentley> lifeless: I have a context manager.  Is is easy to create a Fixture for a context manager?
<lifeless> yes, Fixtures are context managers
<abentley> lifeless: But last I heard, context managers were not fixtures.
<lifeless> right
<lifeless> I don't think there is an adapter today
<lifeless> you could write one, but you might find your context manager is smaller if written as a fixture
<abentley> lifeless: I'd be surprised if it could get any smaller.
<lifeless> was just a thought :)
<abentley> lifeless: I find the contextlib.contextmanager decorator makes for very readable and succinct context managers.
* abentley changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugtasks: 4*10^2
<lifeless> deryck:   https://dev.launchpad.net/PolicyAndProcess/MaintenanceCosts#preview updated
 * deryck looks
<deryck> lifeless, yeah, that's probably good enough.  I can clarify beyond that with my squad in standups.
<lifeless> it allows self-history, squad history, whatever
<lifeless> I thought about saying 'recent branch'
<lifeless> but then we're down into precise definitions
<lifeless> I think folk will appreciate that using a 5 year old branch to offset a current LoC increase isn't really kocher
<deryck> yeah, I think we need to just have a better sense of our squad state on this, too.  which is on me.
<poolie> hi all
<lifeless> hi
<nigelb> morning
<nigelb> this is probably my cue to figure out this sleep thing.
<huwshimi> Morning
<poolie> o/ huwshimi
<timrc> Perhaps a super dumb question, but can you upload to components other than main with a ppa? I have a situation where it would be great to upload to a restricted component :)
<bigjools> you can't
#launchpad-dev 2012-04-12
<timrc> bigjools, thanks :)
<wgrant> timrc: Why do you need to?
<wgrant> timrc: PPA packages by default build against all components.
<timrc> wgrant, You won't like it... we want to upload proprietary packages to project PPAs and when we pull them into our archive "downstream" route them into a private component
<timrc> wgrant, it would be easy to do this if we could publish packages to different components in the PPA
<wgrant> timrc: You're not wrong :)
<wgrant> Perhaps use different PPAs?
<timrc> wgrant, that's an option.. was hoping for something more "elegant"
<wgrant> I campaigned years ago for custom component support in PPAs.
<wgrant> But ended up satisfied when they relented and implemented multiple PPAs per person
<wgrant> It works well for most things
<timrc> Yeah, there is a certain appeal as it naturally enforces a good separation
<timrc> I'll list it as an option for my team to discuss... we basically don't have a good way of handling this scenario right now, though
<timrc> people should quit making proprietary software :)
<wgrant> That sounds like the easiest solution :)
<timrc> Hm my 8 mo old daughter is trying to eat the heads off the meerkats on this 10.10 shirt... guess I better go feed her.  Thanks for the assistance
<lifeless> it would be a step towards consistency to let ppas do components
<wgrant> Letting PPAs have Ubuntu's components is not particularly useful
<wgrant> Letting them have custom components might be
<cjwatson> Except for PPAs intended to be copied to Ubuntu
<cjwatson> I mean, custom components wouldn't hurt if done right, but in that case letting them have Ubuntu's components is useful
<lifeless> letting them have the same components would permit removing special case code
<cjwatson> components do still need to be per-archive
<cjwatson> (e.g. Debian)
<lifeless> yes indeed
<wgrant> Exactly.
<wgrant> They need to be per-archive
<wgrant> (DDs had some of this vaguely specced out, even up to dependencies between components and archives)
<lifeless> one month f changes to devel:
<lifeless> 586 files changed, 13951 insertions(+), 5928 deletions(-)
<lifeless> not so much stable on loc
<wgrant>  3659 files changed, 97472 insertions(+), 95512 deletions(-)
<wgrant> 2011-12-01 -> 2012-03-31
<wgrant> Not so bad
<lifeless> thats better yes
<wgrant>  2345 files changed, 58185 insertions(+), 59554 deletions(-)
<wgrant> 2012-01-01 -> 2012-03-31
<cjwatson>  1025 files changed, 40735 insertions(+), 35055 deletions(-)
<cjwatson> since new policy (so 2012-02-10 -> now)
<bigjools> lifeless: PPAs used to have components and we removed them
<bigjools> it confused the heck of out a lot of users
<wgrant> lifeless: https://lpstats.canonical.com/graphs/SourcherryCPU/20120314/20120413/ is interesting
<wgrant> 9.1 stabilised CPU usage and cut it by 40%
<wgrant> I wonder if it was the new slony
<wgrant> Nope, not slony
<lifeless> pg 9.1 perhaps
<wgrant> It was clearly at the time of the upgrade, and slony wasn't upgraded at the same time, so it was probably 9.1 itself, yes.
<lifeless> did pages get faster tho?
<timrc> in theory I could just use a derived distribution overlay with no packages and upload a package to restricted, right :)?
<lifeless> yes
<wgrant> timrc: Hahahahaha
<wgrant> lifeless: We don't have a staging PPR, do we?
<lifeless> don't think so
<lifeless> pick a search thats slow hot and try it
<lifeless> on both prod & qas
<timrc> cjwatson, when creating a PPA you could define the PPA as an Ubuntu archive or a Customer archive or something, but seeing as how this will never get developed, probably not worth discussing
<timrc> Customer? Custom
<wgrant> lifeless: Bug searches aren't faster AFAICR
<timrc> At any rate we'll figure out some way to horribly abuse LP, jk
<lifeless> flacoste: http://www.forbes.com/sites/stevedenning/2011/07/08/the-five-big-surprises-of-radical-management/
<lifeless> wgrant: do you know where the bug is about lazr.restfulclient not respecting the type of resources in collections?
<wgrant> lifeless: wadllib
<wgrant> Bug #340935
<_mup_> Bug #340935: Resources are only instantiated using the WADL predicted type, not the actual found type <wadllib:Triaged> < https://launchpad.net/bugs/340935 >
 * wallyworld__ is sad that his computer has started to spontaneously reboot after the latest updates were applied :-(
<lifeless> wallyworld__: maybe its building up credit
<lifeless> wallyworld__: so it won't have to reboot layer
<wallyworld__> hah
<wallyworld__> like LOC
<lifeless> indeed
<wallyworld__> if i'm lucky i can work for say 10 minutes each time
<lifeless> anything in syslog?
<lifeless> you could try putting tail -f /var/log/syslog in a terminal, pinned to top
<lifeless> might catch a glimpse
<wallyworld__> haven't looked yet, just trying to quickly land some branches first
 * bigjools hides the Boris Spike code from wallyworld__
<wallyworld__> who is Boris?
<bigjools> have you seen Goldeneye?
<wallyworld__> a long time ago
<lifeless> rick_h: around ?
<rick_h> lifeless: yea, just got home
<lifeless> hey
<lifeless> so I'm curious why you used PersonNotification manually
<lifeless> like
<lifeless> its a background sending thing
<lifeless> other uses of it use PersonNotificationSet.add(..) and don't call send
<lifeless> (I'm also curious why you used it at all vs just sending the message)
<wallyworld__> anyone know where traversal stuff like stepthrough is tested?
<wallyworld__> or the ++model++ stuff
<rick_h> lifeless: so for part 1, not sure. I just loaded up the object and that seemed to be the use. I could have misused it I suppose. loading it up now.
<rick_h> lifeless: for part 2. I used it after discussing ways of sending emails. A background process sounded good so requests weren't hung up on smtp/in band stuff
<rick_h> lifeless: I forget who pointed it out to be as a pretty light easy to use way to get emails scheduled
<wallyworld__> found the ++model++ tests
<rick_h> lifeless: either deryck or sinzui I think since I chatted with both of them in pre-impl.
<rick_h> lifeless: but yea, you're right, I shouldn't have called send and I shouldn't have patched send to override the email since it's not backgrounding like I wanted and defeats the purpose
<rick_h> compleetely missed that
<rick_h> except spelled better
<lifeless> so the reason we have background stuff is for teams
<rick_h> lifeless: ah, I see where I went wrong. So most code uses the Set object, but I wanted to override the To addr so I went after the raw PersonNotificatoin object
<lifeless> that may notify thousands
<StevenK> wgrant: LP.cache.context.information_type == undefined :-(
<lifeless> this is for notifying of changes to individuals
<rick_h> lifeless: ok, I just assumed it's good practice since things like slow/delayed smtp servers could cause request timeouts/etc
<lifeless> rick_h: we have local smtp on every server
<lifeless> rick_h: sending hundreds of mails is a problem, sending a fixed number isn't
<lifeless> rick_h: thanks, this helps me out
<rick_h> lifeless: ok, I did notice there seems to be a few ways to send emails out and when I mentioned background this was pointed out to me
<wgrant> StevenK: Ah, possibly because this is a bugtask, not a bug.
<wgrant> StevenK: Hopefully the bug is there as well. If not, put it there :)
<rick_h> lifeless: but yea, I did it wrong because when a user wants to change their preferred email I needed to send to their previous addr and PersonNotification doesn't support it
<rick_h> which is why I couldn't copy what other uses were doing directly
<StevenK> wgrant: self_link is the bug.
<StevenK> wgrant: Just trying to find where is defined
<wgrant> StevenK: Try LP.cache.bug.information_type
<StevenK> Ooh, that looks good.
 * StevenK nails wallyworld_ to IRC.
<wallyworld_> can you make my pc stop rebooting?
<rick_h> lifeless: so while I'm fixing that in the morning the goal is to ditch the event as well?
<StevenK> I'd watch tail -f /var/log/syslog like lifeless suggested
<StevenK> Or downgrade what you recently upgraded
<wallyworld_> it just happens to quickly for me to see what syslog says
<wallyworld_> i will try downgrading but it was a large number of packages that were upgraded
<lifeless> rick_h: well, I'm doing a sketch on it as you're past EOD
<StevenK> Do you have another machine?
<wallyworld_> > 50 or 60
<wallyworld_> nope
<rick_h> lifeless: yea, I've got to hit the sack soon. Couple of late nights for me
<StevenK> wallyworld_: No even running Windows?
<wallyworld_> my home theatre pc
<rick_h> lifeless: thanks, shoot me a message on status and I'll pick up/finish whatever needs doing in the morning. Thanks for pointing out my misuse of the notification.
<rick_h> missed in code review and my own look through it today
<StevenK> wallyworld_: Install PuTTy on it, ssh into your laptop and tail syslog over it?
<wallyworld_> StevenK: you are an ideas man. i may just try that. thanks
<StevenK> Joustling sticks?
<StevenK> 'Ideas man' just reminds me of The Castle.
<wallyworld_> tell him he's dreamin'
<wallyworld_> that's going straight to the pool room
<wallyworld_> why would you eat out?
<StevenK> wallyworld_: Does 'if (some boolean) {' do what I expect in JS, or should I say === true ?
<wallyworld_> StevenK: it will evaluate to false if undefined also
<wallyworld_> StevenK: so if that's what you want then that's ok
<lifeless> wow we are so inconsistent about From:
<lifeless> rick_h: btw, why does this generate multiple emails in those doc tests?
<lifeless> rick_h: are we also notifying separately to the new address?
<lifeless> rick_h: nvm, figured it - we're doing a token challenge
<lifeless> what stops teams getting ssh keys / gpgkeys ?
<wgrant>         if params.distroseries is not None:
<wgrant>             parent_distro_id = params.distroseries.distributionID
<wgrant>         else:
<wgrant>             parent_distro_id = 0
<wgrant> wat
 * wgrant blames lifeless
<wgrant> wtf is this
<lifeless> I don't recognise that
<lifeless> parent_distro sounds like UDD
<wgrant> It's in the structsub searching that you reworked
<wgrant> No
<lifeless> sorry, I mean DD
<wgrant> Still not :)
<wgrant> It's just the distro that owns the series
<wgrant> In this case
<wgrant> The constraint that uses it doesn't make much sense
<wgrant> Not to mention the whole defaulting to 0 thing
<lifeless> you've annotated the pre-move search to see it was me ?
<wgrant> No/
<lifeless> revno: 14485 [merge]
<lifeless> I was on lave
<lifeless> leave
<lifeless> wgrant: sinzui
<wgrant> Still your fault :)
<lifeless> hah!
<lifeless> rick_h: btw telling folk that an ssh key *can't* be used anymore isn't important
<lifeless> rick_h: it doesn't open up security holes ;)
<wgrant> I think it's still interesting.
<lifeless> *adding* one is crucially important
<wgrant> Certainly
<rick_h> lifeless: yes, deryck thought that if someone removed your ssh key and potentially locked you out of things like bzr access you'd want to know why.
<rick_h> lifeless: but I agree, it's on the lower end of the notification scale
<lifeless> I think its nice in an auditord sense
<lifeless> I'm not sure its worth an email
<lifeless> not to worry, I'm keeping it
<rick_h> lifeless: yes, this discussion brought out a new bug I filed on adding some auditing support because if someone does change your email we can't verify what it was/you are who you say you are easiliy
<lifeless> stevenk has an auditord nearly ready
<rick_h> lifeless: or that we can generate a list of all the data a malicious user changed so that we make sure we can revert all the changes, etc
<lifeless> rick_h: log files :>
<rick_h> lifeless: ah, good to know then
<rick_h> lifeless: on the sql queries? Not sure what all we have there.
<rick_h> lifeless: it was brought up that we could try to do some sql backup/etc pulling, but painful
<rick_h> lifeless: but yea, that was deemed outside the scope of the current branch and the idea is to prevent it from happening
<lifeless> huh, no.
<lifeless> we email on these things
<lifeless> they are all important
<lifeless> so there should be email logs for them all
<lifeless> not in smtp, in our email engines.
<lifeless> that said, its 3 settings
<lifeless> preferred mail
<lifeless> (and all contact addresses arguably)
<lifeless> ssh keys
<lifeless> gpg keys
<rick_h> right, but we can restore data based off the email logs?
<lifeless> I think saying to someone 'we've nuked the settings, please set up the way you want' is a simple and solid answer
<lifeless> its not like restoring complex data
<rick_h> lifeless: ah ok
<lifeless> or to put it another way
<lifeless> even if we *did* restore it
<lifeless> any Ubuntu dev worth their salt would manually verify it all anyway.
<rick_h> right, I think the idea was more someone jumps in irc "I can't login!" "who are you? what's your email?...that doesn't match what I've got, let's see if it's changed and what it was?"
<lifeless> rick_h: another thing we should notify on is adding oauth tokens
<lifeless> rick_h: an audit trail for that would be good, and I'm +1 on having an audit trail
<rick_h> lifeless: ok, haven't seen taht
<lifeless> (again, StevenK :))
<rick_h> lifeless: yea, cool
<lifeless> rick_h: when you allow an app access to your account, would you object to being told that that has happened ?
<rick_h> sorry, haven't seen the oauth stuff, cool on the WIP audit trail
<rick_h> lifeless: not sure, I've got to check the oauth use. Is that just for apps? scripts using api?
<rick_h> lifeless: I guess it should be rare I get these right? Changing keys, email is pretty rare. Annually maybe in practice worst case?
<lifeless> right
<lifeless> generally its one-time setup
<lifeless> rick_h: did you do GPG keys when you did this ?
<lifeless> rick_h: or do they already notify ?
<rick_h> yea, they already get an email because you have to decrypt your email with the key to verify
<rick_h> so gpg keys aren't in this
<lifeless> does that go to the preferred address
<lifeless> or the address in the key
<rick_h> preferred
<lifeless> cause if it goes to the address in the key it would be useless
<rick_h> I believe, I'll have to dbl check I guess
<lifeless> I believe it goes to the address in the key
<lifeless> so we know they aren't impersonating someone via the key
<rick_h> k, I'll double check. It's hidden in some browser action so I've got to find it again.
<lifeless> now where is that doc on Launchpads voice
<rick_h> https://dev.launchpad.net/UserInterfaceWording/CanonicalStyleGuide?highlight=%28voice%29 looks like what you might mean?
<lifeless> https://dev.launchpad.net/UserInterfaceWording
<rick_h> bah, I fail at going to bed during the right day
<rick_h> I'm really gone to bed this time. Will look over the wording docs lifeless, thanks for pointing out. Looks like I wrapped wrongly off the bat
<lifeless> rick_h: :P
<lifeless> rick_h: gnight
<lifeless> oh wow
<wgrant> ?
<lifeless> thats a fun hole
<lifeless> we validate a preferred but unvalidated email address when a different preferred email address is set
<wgrant> Preferred but unvalidated?
<wgrant> what?
<wgrant> lifeless: Anything setting an unvalidated email to PREFERRED without first validating it is a bug.
<wgrant> Preferred > validated
<lifeless> wgrant: as I said
<lifeless> 'thats a fun hole'
<lifeless> person.py
<lifeless> look for existing_preferred_email.status = EmailAddressStatus.VALIDATED
<wgrant> lifeless: An existing preferred email is by definition already validated.
<wgrant> Because we're setting it from preferred to validated
<lifeless> wgrant: oh, I see, right
<lifeless> so this could be more clear
<wgrant> And preferred is stricter than validated
<lifeless> also, we have to totally different concepts conflated. Win.
<wgrant> Yes.
<wgrant> Win
<lifeless> I would model this with preferred as a pointer from person, constrainted by trigger checks to be validated.
<lifeless> anyhow
<lifeless> thats then, this is now
<wgrant> The idea was to avoid triggers, I believe.
<wgrant> Or you could also have a preferred flag
<lifeless> and a composite unique index
<lifeless> or whatever
<wgrant> With UNIQUE emailaddress(person) WHERE preferred
<wgrant> Right
<wgrant> It's insane.
<wgrant> But 2005
<wgrant> So we sort of have to live with it
<StevenK> It doesn't sound that hard to shift away.
<lifeless> its straight forward
<lifeless> little returns
<lifeless> probably not the most significant thing to improve right now
<StevenK> More straight-forward than bug.{private,security_related} -> bug.information_type :-)
<wgrant> No
<wgrant> Only a few places touch that
<wgrant> There are dozens upon dozens of places that touch emailaddress.status == 4
<lifeless> rick_h: I've pushed an opinionated branch to https://code.launchpad.net/~rharding/launchpad/email_notice_959482/+merge/99995
<lifeless> rick_h: its about half the size, a significant chunk of which was the event overhead + the need for dedicated per-event tests.
<StevenK> wallyworld_: O hai.
<lifeless> rick_h: let me know what you think, I obviously don't want to be blocking you in any way.
<StevenK> wallyworld_: Since I moved the ChoiceSource stuff into its own module, Y.one('body') seems to return the enclosing div.
<wallyworld_> StevenK: that seems strange at first thought
<StevenK> wallyworld_: It seems to be strange. If I change it to a private type, the enclosing div turns red.
<wallyworld_> StevenK: Y.one('body') should return the html element <body>
<StevenK> Yes, agreed.
<wallyworld_> fiik what's up then :-(
<wallyworld_> have you looked at the html source in the browser to make sure there isn't anything stupid in there?
<wallyworld_> or tun it through an html validator?
<wallyworld_> run
<StevenK> Now it works. I think it's toying with me.
<wallyworld_> like my pc hardware
<StevenK> wallyworld_: Oh, could I get your help with the JS fallback? If I set href to the link in the TAL, the JS never fires.
<wallyworld_> StevenK: you have set the onclick handler?
<StevenK> I thought the ChoiceSource does that for me?
<wallyworld_> yes, should do
<wallyworld_> i have a quick look at the source code to see if it is doing anything dumb
<wgrant> Instance i-4ea81529 starting................................................................................................................................................................................... started on ec2-67-202-7-98.compute-1.amazonaws.com
<wgrant> Started in 15 minutes 46 seconds
<wgrant> odd
<StevenK> wgrant: Orsum.
<StevenK> wallyworld_: Sigh, that works now too.
<wallyworld_> StevenK: maybe browser caching issue?
<StevenK> Hm. Maybe.
<StevenK> AssertionError: The vocabulary must implement IEnumeratedType
<StevenK> :-(
<wgrant> blink
<wgrant> lp.bugs.model.bugtasksearch._build_pending_bugwatch_elsewhere_clause
<wgrant> First bit, 'if params.product'
<wgrant>                     AND RelatedBugTask.id = BugTask.id
<wgrant> Am I not thinking it through correctly, or is that inverted?
<wgrant> Ah
<wgrant>     @property
<wgrant>     def pending_bugwatches_url(self):
<wgrant>         if not IProduct.providedBy(self.context):
<wgrant>             return None
<wgrant> So that codepath is never exercised, I guess
<lifeless> wgrant: did you sstart a thread on tag sort order on the list ?
<wgrant> No, I'll do that now.
<adeuring> good morning
<gmb> Does anyone have a spare second to run a test for me in their lp dev environment?
<lifeless> gmb: p'rhaps
<gmb> lifeless, bin/test -t externalbugtracker-bug-imports.txt if you'd be so kind.
<lifeless> hah, out of date schema
<lifeless> making new
<gmb> lifeless, Oh ,wait.
<gmb> No need.
<gmb> I see why it's failing
<gmb> It's because I'm stupid.
<lifeless> do -not- give me straight lines like that
<gmb> :)
<lifeless> I won't be able to help myself if you do
<gmb> Turns out that it helps if you're running tests in a Lucid (or Oneiric) container, not on Precise. My lxc container had exited and I hadn't noticed.
<lifeless> ah yes, that will mess you up
<lifeless> I don't have a non-lxc postgresql, helps avoid that confusion
<gmb> Yeah, I think this is my cue to nuke mine :)
<gmb> Otherwise I'm going to do this again.
<lifeless> ah, insanity!
<gmb> :)
<czajkowski> it's wrong to be so happy with only having 1 ticket in my RT queue but I am :D
<czajkowski> rt queue has been fully smacked :D
<StevenK> jml: Hai. Are you able to QA r15074 today?
<jml> StevenK: I guess.
<jml> StevenK: I'm surprised that's how it works though
<StevenK> jml: Really? Developers are mostly responsible for their own QA. In this case, you should be able to ask the webops to make you an admin on qas for short amount of time.
<jml> StevenK: I'm trusted as a judge of behaviour but not of code quality?
<jml> anyway
<StevenK> jml: In what way, that you need a review?
<jml> StevenK: yes.
<StevenK> jml: Everyone has their branches reviewed, except if the change is small and well-understood, in which case, graduated reviewers can self-review. So don't feel like it's just you.
<StevenK> jml: And you wrote the feature so you should have a very good understanding of how it is supposed to behave.
<jml> Yeah, that's exactly how it works.
<jml> StevenK: anyway, I can QA the feature today (UK).
<StevenK> jml: Thanks. If you don't feel comfortable QA'ing, then please feel free to hand it off to a LP dev if they agree to do it -- we just tend to assume the developer who wrote it will do the QA.
<benji> jml: are you available to review my branch which will (soon) bring the TSFM tests inline with the improved tests on your branch?
<jml> benji: yes, roughly speaking
<jml> benji: I mean, I can do it today. A synchronous review is probably beyond me.
<benji> jml: that's cool, thanks
<benji> jml: while I have you on the line, do you know anything about TestThreadSafeForwardingResult.test_forwarding_methods and why it does what it does (testtools/tests/test_testresult.py)
<benji> I don't understand the underlying intent of the test.  And the sparse comments in the test seem to disagree (or at least are irrelevent) to what the test actually verifies.
<jml> looking
* rick_h changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: rick_h* | Firefighting: - | Critical bugtasks: 4*10^2
<jml> benji: No idea, sorry.
<benji> jml: thanks anyway
<jml> benji: perhaps ask lifeless when he gets back online.
<benji> yep
* jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: jcsackett, rick_h* | Firefighting: - | Critical bugtasks: 4*10^2
<adeuring> abentley: mumble?
<abentley> adeuring: coming
<abentley> adeuring: https://pastebin.canonical.com/63588/
<rick_h> abentley: comments on teh review, please also make the LoC note in the MP.
<deryck> rick_h, hey, dude.  I can chat about your branch now.  2 minutes to warm coffee and meet in a hangout, cool?
<rick_h> deryck: sounds god
<rick_h> good
<benji> jml: https://code.launchpad.net/~benji/testtools/modernize-tsfr/+merge/101749 awaits when you have a moment.
<jml> benji: ta
<jml> benji: have looked at it long enough to know that I need to have a detailed look.
<benji> jml: ok; thanks
<jml> benji: will try to get to it today (UK)
<benji> jml: I appreciate it.
<abentley> rick_h: Thanks.  done.
<abentley> jcsackett: The imports in runner are explained by the comment "Avoid importing from lp.services.job.celeryjob where not needed, to avoid configuring Celery when Rabbit is not configured."
 * jcsackett looks again
<jcsackett> abentley: so it is, ignore my blind eyes. :-)
<abentley> jcsackett: cool.  I'll add a comment to the other one, though.
<jcsackett> thanks, abentley
<adeuring> rick_h: could you please have a look at this mp: https://code.launchpad.net/~adeuring/launchpad/abort-transaction-in-job-queue/+merge/101762 ?
<rick_h> will do adeuring I'll put it next on my list
<adeuring> rick_h: thanks!
<deryck> gary_poster, hi.  got a sec?
<sinzui> Looks like wordpress's autosave feature is broken on Lp's blog and I just lost 90 minutes of writing when it's JS went tits up. It then insults me by saying "howdy" when I log back in. If I could accomplish one thing today, please let me stab the cowboyism to death.
<jml> benji: have replied. am available to chat in real-time if you want.
<benji> jml: cool, I'll read it now
<benji> jml: interesting; if your understanding of TSFR is correct, I don't see that it ever achieved it's aim, which is entirely possible
<jml> benji: I'm pretty sure it's got very close.
<benji> my understanding (gained only though reading the code) is that it intended only to make sure individual calls did not overlap, not that calls for different tests didn't overlap
<jml> benji: startTest() sets the test start tim, and then on receiving $RESULT, then runs some semaphore-protected code that does (ignoring tags):
<jml>  * time(start_time)
<jml>  * startTest(test)
<jml>  * time(now)
<jml>  * add$RESULT(test, *args, **kwargs)
<jml>  * stopTest(test)
<jml> multiple threads, each with their own TSFR, but with a shared semaphore and a shared target result object, ...
<jml> imagine the target result object dumping subunit output
<jml> it *has* to be:
<jml> test: foo
<jml> success: foo
<jml> test: bar
<jml> success: bar
<jml> and since the parallel runner isn't completely broken, I'm assuming that's how it behaves now.
<jml> granted, that might be quite a hefty assumption
<benji> :)
<benji> jml: right, it works now, and I think I can preserve that contract (start/result/stop occur atomicly), but it won't be able to comply with the other general contracts, for example, start/tags/success/tags/stop won't work correctly unless we go to great lengths (as far as I can tell)
<jml> hmm.
<jml> benji: I haven't really thought about tags in that context, so I couldn't say one way or another.
<jml> benji: I *think* I can address your original concern about a thing that applies a constant set of tags to each test though.
<benji> jml: now that I have a deeper understanding of what TSFR needs to do, let me take a stab at it and follow up with you when I succeed or fail
<jml> benji: ok.
<jml> benji: I think all you need is something like this though: http://paste.ubuntu.com/926640/
<benji> I hope it's that simple.
<jml> benji: I've merged forward-current-tags into testtools trunk, fwiw.
<rick_h> adeuring: ping, so the MP talks a lot about the abort_transaction, but what about hte manage transaction?
<adeuring> rick_h: that paramater existed a bit longer; it was just not yet tested.
<adeuring> rick_h: we need this parameter because...
<rick_h> adeuring: right, but just added to the start/complete/fail, etc.
<rick_h> it seems that it'd be the same for all methods of a job. More of a job "starts_transaction" parameter on the job itself
<rick_h> can you have that set on start but not complete for instance?
<adeuring> the job runner in the main LP code managed the transactions, while the lazr.jobrunner does not do this, so we had to move this to Job.start(), complete() etc
<adeuring> rick_h: no, the settings should be consistent in the call sites, and it is
<rick_h> ok, so I'm not seeing the tie in. Ok
<rick_h> adeuring: yea, just looking here it seems you could have a mix up of manage_transaction in one place but not another and I worry about that being consistant
<rick_h> rather than method params it checking an object level attribute that it has a transaction to manage or something
<adeuring> rick_h: you are right, but there is only a minor risk. Most status handling is done now in lazr.jobrunner, which always sets manage_transaction=True; there are a few calls in the main LP code like Job.wueue(), where manage_transaction is not specified at all, i.e., it is False. (to avoid "premature" commits)
<adeuring> rick_h: Using something like Job.teansaction an interesting suggestion, but I would prefer to implement that in a differnt branch...
<rick_h> adeuring: ok, thanks
<gary_poster> deryck[lunch], sorry, was lunching myself
<rick_h> adeuring: ok, noted my question, can you reply with that possibly follow up branch and the LoC notes and I'll mark ok
<adeuring> rick_h: ok
<rick_h> adeuring: ty
<adeuring> rick_h: answered
<rick_h> adeuring: ok, thanks for the discussion. Ok'd
<adeuring> rick_h: thanks!
<deryck> hi, gary_poster. just wondering a bit about our fork of python-openid and seeing branches from you and wgrant.
<deryck> gary_poster, wondering why we forked and stayed behind current release. if any patches were applied upstream. that kind of thing.
<gary_poster> deryck, hey.  um, I don't remember openid forks but that doesn't mean I didn't do it. :-)  I just did python-memcached.  Lemme see if versions.cfg says anything interesting
<deryck> gary_poster, versions suggests wgrant has the latest fork. which looked to be off your branch.
<jml> benji: how goes it?
<benji> jml: I've decided to leave TSFR be for the moment and implement my worker ID tagging result wrapper and then see if the bugs I expect in TSFR's tag handling really exist and actually bite me; if they do, then I'll at least have a concrete idea of what is wrong with TSFR and what I want it to do
<jml> benji: a sound plan.
<gary_poster> deryck, I'm afraid I think I need to point you to flacoste.  Perhaps he remembers.  According to wgrant's branch (https://code.launchpad.net/~wgrant/python-openid/python-openid-2.2.1-fix676372) he branched off of flacoste's branch from Nov 2008, which was very soon after I started work here.  bzr log versions.cfg only mentions what wgrant did ("[r=lifeless][bug=676372][rollback=12290] Use a custom python-openid") AF
<gary_poster> AICT.  I'm still digging more, but perhaps it would help to know why you thought it was my branch?  Maybe I can dig from there
<deryck> gary_poster, ah, guilt by assumption then. :)
<gary_poster> heh, deryck :-)
<deryck> gary_poster, I looked at https://code.launchpad.net/python-openid/ and saw your branch there was older.
<gary_poster> deryck, ah.  Well, that looks like it had a reason at the time, but is no longer needed
<deryck> gary_poster, I can email wgrant on list then and see if he recalls the details here.  thanks.
<gary_poster> cool, welcome
<deryck> fwiw, I need a fixes in a recent python-openid, and either need to backport or apply our forked patches.
<sinzui> jcsackett, what is blocking us from showing the links to view bugs and branches that are shared with users?
<jcsackett> sinzui: not sure what you mean. +sharing does have links to the details page, details page does have links to the various bugs and branches. at least on qastaging.
<jcsackett> sinzui: i'm assuming i'm misunderstanding your question.
<sinzui> not in production? did you not ask for the feature flag to be enabled?
<sinzui> https://launchpad.net/launchpad/+sharing
<lifeless> rick_h: hi
<jcsackett> sinzui: we hadn't turned it on b/c there were some bugs to be addressed, but it looks like wallyworld_ took care of them.
<jcsackett> so, nothing's blocking us. shall i make the request?
<rick_h> lifeless: howdy (/me ducks from sinzui)
<sinzui> jcsackett, yes pleased. I will second it
<lifeless> rick_h: what did you think of my sketch?
<rick_h> lifeless: I've only had a quick change to glance at it so far. Busy OCR day.
<lifeless> kk
<rick_h> lifeless: think it's good and I'll be using that to clean up and try to get my branch finished up by tomorrow.
<lifeless> if you want to talk about it at any point, I'll be around, slightly offset-earlier day today
<rick_h> lifeless: yea, np. I think it's been a process, but good stuff.
<lifeless> rick_h: so the '1 callsite' comment i made the other day - thats a heuristic - didn't apply in this case, but when it does, its just another bit of evidence that an event may be the wrong fit
<rick_h> lifeless: yes, and I mentioned during impl calls with deryck that it didn't feel quite right since it wasn't really an event off of the Person as it started out once we hit things like logintoken/ssh keys
<rick_h> lifeless: but change some names and you try to make yourself feel better.
<rick_h> but it's been good, I've learned a lot about how stuff works in this branch
<rick_h> lifeless: appreciate the time you've taken with me over the last few days on it
<lifeless> my pleasure
<bac> jcsackett, rick_h: have time for a review of a short change?  https://code.launchpad.net/~bac/launchpad/bug-974617/+merge/101797
<rick_h> bac: looking
<bac> thanks rick_h
<lifeless> flacoste: hiya
<rick_h> bac: ok'd and commented
<bac> rick_h: thanks
<bac> thanks jcsackett
<bac> rick_h: i implemented both of your suggestions
<rick_h> bac: cool, thanks
<lifeless> flacoste: http://www.infoq.com/presentations/1000000-Daily-Users-and-No-Cache
<lifeless> and
<lifeless> flacoste: http://www.forbes.com/sites/stevedenning/2011/07/08/the-five-big-surprises-of-radical-management/
<gary_poster> lifeless, we have some tests that fail if bzr whoami is not set (and not guessable).  Is this an indicator of a bug in the test, or should we set up a whoami value?
<lifeless> those tests are incorrectly isolated
<gary_poster> lifeless, cool, we will file bugs, thanks
<lifeless> they need to either isolate a number of things themselves, or derive from one of the bzrlib TestCase base classes (e.g. TestCaseWithTransport or TestCaseWithTemporaryDirectory)
<gary_poster> ack
<lifeless> I wonder if we should special case subscription of open lists
<lifeless> that is, make subscribing, or assigning, an open team to anything be a privileged (admin) action
<lifeless> (see the recent shenanigans with the debian-ppas bug where the launchpad-dev *mailing list* was subscribed by a random list member
<abentley> rick_h: could you please review https://code.launchpad.net/~abentley/launchpad/celery-job-layer/+merge/101806 ?
<rick_h> abentley: sorry, EOD.
* abentley changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugtasks: 4*10^2
<rick_h> thanks, was just editing that. jcsackett might still able to/around
<abentley> jcackett: are you up for solo reviewing?
<jcsackett> abentley: sure. be a little bit but i can get to it before EOD.
<abentley> jcsackett: cool, thanks.
<jcsackett> np.
<jcsackett> abentley: looked at the branch. r=me.
<timrc> I'm getting an oops on staging... 1d44c531f28a13c40cb562e401dd7cf2
<timrc> effecting my ability to test a build in a PPA
<timrc> hm ok, 3rd or so reload at least showed me the PPA page
<lifeless> timrc: the OOPS- is part of the id
<lifeless> timrc: just copy and paste the whole thing please
<timrc> lifeless, It would just be OOPS-1d44c531f28a13c40cb562e401dd7cf2 but I was finally able to copy / re-build packages so I'm fine
<SpamapS> OOPS-71384b1baaee6bfd54172741bf834dc7
<SpamapS> Anybody know why retrying a build would cause a timeout?
<lifeless> SQL time: 8962 ms
<lifeless> Non-sql time: 141 ms
<lifeless> Total time: 9103 ms
<lifeless> Statement Count: 35
<lifeless> 1. 	8887.0 	1 	SQL-main-master 	
<lifeless> UPDATE BuildFarmJob
<lifeless> SET builder=NONE, date_finished=NONE, status=$INT, date_started=NONE, log=NONE
<lifeless> WHERE BuildFarmJob.id = $INT
<lifeless> row contention
<SpamapS> lots of builds going on right now
<lifeless> yet another symptom of the builddmaster being not-an-API-client
<SpamapS> seems like that should be async entirely. like I'm not saying "tell me when you're done updating your database" I'm saying "tell me that you're going to retry that build"
<lifeless> well, duh
<lifeless> :)
<SpamapS> same thing happens when I try to build anything it seems
<lifeless> if you try a couple of times does it come good?
<SpamapS> no
<lifeless> if it doesn't, then we may have a melting down buildd manager, and I'll go chase that
<lifeless> ok
<SpamapS> after 3 I figured I'm part of the problem
<lifeless> oh you are, fo'sure
<wgrant> lifeless: Unrelated to buildd-manager
<wgrant> SpamapS: That means it's already been retried by retry-depwait, but not committed yet
 * timrc shakes fist at staging.launchpad.com... I hit reload and got the "code update in progress"
 * timrc will try this all again after his child goes to sleep, ttyl
<StevenK> Tis not .com
<lifeless> timrc: staging.launchpad.net
<lifeless> timrc: of course, if you mistyped, then again, COpy-paste is your friend ;)
<timrc> I meant .net
<lifeless> timrc: as for the timeout
<lifeless> 1. 	8355.0 	1 	SQL-main-master 	
<lifeless> SELECT *
<lifeless> FROM ((
<lifeless> SELECT BinaryPackageBuild.distro_arch_series,
<lifeless> you can see it https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1d44c531f28a13c40cb562e401dd7cf2#longstatements
<timrc> lifeless, thanks
<wgrant> ">From a data integrity perspective, we must not have two different
<wgrant> sorts of data arguing: if there is a subscription, the user must have
<wgrant> a grant."
<wgrant> lifeless: false
<wgrant> lifeless: That is at most an argument from usability
<wgrant> Not integrity
<lifeless> wgrant: actually it is integrity if you think from a services perspective.
<wgrant> It's not.
<wgrant> Implicit subscriptions mean we pretty much have to do permission-based filtering of subscriptions anyway
<wgrant> Probably at the service level
<lifeless> nope
<lifeless> I have a flat out day
<lifeless> so I can't spend a lot of time digging into this, sorry.
<lifeless> The notification service will do precisely *one* callback to LP to expand team subscriptions. The rest is assumed accurate
<lifeless> it is up to LP to maintain its integrity.
<wgrant> Then LP will filter the subscriptions based on privacy
<wgrant> It already has to do that for implicit ones.
<lifeless> I'd like you to operate on that basis.
<lifeless> you've got the proposed language in mail and in my branch
<lifeless> if you can't represent what you need, we'll need to iterate that.
<lifeless> What we can't sensibly do is multitenant with multiple services determining visibility of notifications
<lifeless> its a denorm, it needs to be treated as such
<SpamapS> OOPS-936e37e9194377d4c49cfcf1314896b5 ...
<SpamapS> this is trying to tell a recipe to build now...
<wgrant> SpamapS: It's trying to build against maverick
<wgrant> maverick is no longer supported
<SpamapS> Shouldn't that just automatically fall out?
<wgrant> You'd think.
<SpamapS> Had to click the widget to select series and then save it unchanged. That worked. Thanks.
<lifeless> SpamapS: please file a bug
<lifeless> SpamapS: I suspect we're going to have hundreds of those soon
#launchpad-dev 2012-04-13
* wallyworld changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: wallyworld | Firefighting: - | Critical bugtasks: 4*10^2
<poolie> where did the ec2 script go?
<poolie> i'm guessing lp-dev-utils
<wgrant> Yes, in there
 * poolie is trying to finish https://code.launchpad.net/~mbp/launchpad/925597-dkim-from/+merge/98976
<lifeless> have a good weekend y'all
<StevenK> wallyworld: Can I borrow your eyes for a moment?
<wallyworld> StevenK: sure, so long as you return them, and it depends on what you use them to look at
<StevenK> wallyworld: http://pastebin.ubuntu.com/927290/ is one of my JS files, make lint reports "59: Expected ';' and instead saw '}'."
<wallyworld> StevenK: you need to terminate the } with a ;
<wallyworld> since that block is a statement and all statements need to be terminated with a ; in js
<StevenK> Ah.
<wallyworld> one of the reasons to prefer python :-)
<StevenK> Oh, bah. Y.one('[name="field.information_type"]'); is only matching the first radio button with that name. :-(
<wallyworld> do you want all of them or a specific one?
<rick_h> StevenK: that's what .one does :)
<StevenK> rick_h: Silence! :-P
<rick_h> /one/all
<wallyworld> what he said
<StevenK> rick_h: s/one/all/   # perhaps? :-)
<rick_h> sorry, was this in vimscript or something else :P
<StevenK> rick_h, wallyworld: .all returns an array?
<wallyworld> rick_h: ignore the pedant :-)
<rick_h> hey, 'pseudocode' is legit
<wallyworld> a NodeList
<wallyworld> which you can call .each() on etc
<rick_h> nodelists are nice, act like nodes
<StevenK> Can I say .on('change', ) on it?
<wallyworld> i'd use event delegation
<wallyworld> instead
<rick_h> you can say that on the nodelist, but yea, delegate ftw
<wallyworld> attach a handler to the containing div or whatever and use a selector in the delegate() to choose what to listen to
<rick_h> assuming they're all from a common parent node
<wallyworld> StevenK: look at examples in shareetable.js
<StevenK> The radio button list is coming from Zope's form handling
<wallyworld> there should be a parent node you can use
 * wallyworld has to run away to drop the kid at the occupational therapist. back in ~30
<StevenK> wallyworld: It's in a table, so the parent node is ... difficult to get.
<StevenK> Oh, wait a sec, there's a table inside another.
<StevenK> rick_h: Hmmm, maybe I'm going about this all wrong
<rick_h> how so?
<StevenK> rick_h: If I put the change function on all nodes, I'm not sure how to reach in and get which radio button is selected
<bigjools> wallyworld: fancy a liquid lunch today?
<rick_h> so when you delegate you tell it you only care about 'input[name...]' nodes that change
<rick_h> StevenK: and then when your callback is called, 'this' is the node that changed
<rick_h> StevenK: http://yuilibrary.com/yui/docs/event/#delegation
<poolie> is there an easy way to get a person by email, requiring that the email be verified?
<rick_h> poolie: there's 'admin = getUtility(IPersonSet).getByEmail(ADMIN_EMAIL)' usage
<wgrant> I don't think that validates.
<rick_h> but you'll have to check the validated status of the email after you get hte person back I think
<poolie> in my test for bug 925597 my current (not live) code is accepting mail from unverified addresses
<_mup_> Bug #925597: From address is ignored if DKIM is supported <email> <regression> <Launchpad itself:In Progress by mbp> < https://launchpad.net/bugs/925597 >
<poolie> which what a simple getPrincipalByLogin(addr) does
<poolie> mm i think i should just check it
<poolie> thanks
<StevenK> rick_h: I'm not doing delegation yet.
<poolie> wallyworld, could i get an incremental review on https://code.launchpad.net/~mbp/launchpad/925597-dkim-from/+merge/98976 ?
<wallyworld> poolie: sure
<wallyworld> bigjools: i ate on the way back from dropping lachie off at the dr :-(
<bigjools> wallyworld: what part of liquid didn't you understand? :)
<wallyworld> lol
<poolie> :)
<poolie> yay bne
<wallyworld> bigjools:  i have to play soccer tonight but i could be persuaded to have one or two or three
<poolie> btw i'm probably going to be up there on about the 11th of june
<wallyworld> poolie: excellent. make sure you keep some time free to come over for dinner or something
<bigjools> I trust we'll see you for a wee drinkie
<poolie> :) please keep some beer for me
<bigjools> Dan Murphy's is going to get more custom
<wgrant> Hmmm
<wgrant> We have a lot of old private bugs that probably shouldn't be private.
<wallyworld> poolie: 2 comments 1. preload_dns_response. i'd prefer response_type to default to 'valid' instead of None
<poolie> good idea
<wallyworld> poolie: 2. i'd like to see tests for all the new code paths introduced for checking the email address
<wallyworld> not just the 'this is not active'' case
<poolie> oh, person with no account, etc
<wallyworld> yeah
<poolie> mm
<poolie> fair point
<wallyworld> bigjools: when you going to Dan's?
<wallyworld> poolie: sorry to be picky on your last day
<bigjools> when I run out of OSH
<wallyworld> i'm surprised you have any left
<lifeless> you Mosh ?
<bigjools> war rationing
<poolie> hm
<poolie> i wonder if those can actually be hit or if i'm just over cautios
<wallyworld> poolie: i'm not sure tbh
<wallyworld> poolie: you could always put in asserts if we think those conditions should be impossible to hit
<poolie> plain 'assert' or if/raise pseudo assert?
<lifeless> if raise
<lifeless> is what our style guide says
<lifeless> OTOH the odds of running with -O are < 0.
<StevenK> lifeless: What time is your flight?
<lifeless> 1510
<poolie> i'm guessing he's in the lounge
<lifeless> don't ask when I started work this morning :)
<StevenK> poolie: Given the airnz.co.nz hostname ...
<timrc> I have a package waiting to build on staging in 59 seconds for about 4 hours
<lifeless> are there live builders for it ?
<poolie> can anyone tell me off hand if address.person = None or person.account = None is likely to happen?
<wgrant> address.person = None is impossible
<lifeless> ahh :)
<wgrant> person.account = None is likely
<lifeless> this is a special area of hell reserved for conflated services
<lifeless> which we're trying to untangle
<wgrant> timrc: staging doesn't do builds
<lifeless> poolie: a valid person has an account
<StevenK> No Soyuz services on staging at all, in fact.
<lifeless> poolie: (which may not help you)
<wgrant> StevenK: buildd-manager runs, for TTBJs
<StevenK> wallyworld: css-selector should be what?
<poolie> the question is should i add tests for getting mail from an address corresponding to a person with no account
<timrc> $?@!
<timrc> k
<poolie> at the moment it will just basically reject it but it's not tested
<StevenK> wallyworld: You've got span in there
<poolie> well
<wallyworld> StevenK: should be what you need to capture clicks from the nodes you want
<poolie> if it's not tested, perhaps i should say the behaviour is undefined :)
<lifeless> poolie: that will happen in real life, so I'd say yes.
<lifeless> auntil we fix up the cruft
<StevenK> wallyworld: Right, but I've not used them before, so how do I match the radio buttons?
<poolie> so test for person with no account, not for an address with no person?
<poolie> k
<wgrant> poolie: EmailAddress.person is NOT NULL since January
<wallyworld> StevenK: something like "input[name='field.foo']"
<poolie> ah ok
<wgrant> You may still see some code that assumes it can be null. I didn't catch everything.
<poolie> the interface still says required=False
<wgrant> Like that.
<poolie> shall i change it in this branch?
<wgrant> Please.
<wgrant> There were about 10 branches in that series; there was a *lot* to change.;
<StevenK> required=True may have some test fallout
<poolie> or maybe separately?
<wallyworld> StevenK: should be no need to further qualify with type=radio
<wallyworld> poolie: i'd do it separately given the unknown test fallout
<wallyworld> and since it's your last day and ideally we want this to land first time :-)
<StevenK> poolie: I'm a little annoyed at this UI branch I'm working on, I'm happy to cowboy a branch that changes it and toss it at ec2
<poolie> indeed
<poolie> k
<wallyworld> StevenK: let me know if you want me to look at anything with your ui stuff
<StevenK> poolie: This is in EmailAddress ?
<poolie> yes IEmailAddress.person (required=False)
<poolie> in my branch
<poolie> it may just be out of date
<StevenK> wgrant said he missed it accidently
<wgrant> mm
<wgrant> I'd be realised surprised if there was test fallout
<wgrant> But maybe
<lifeless> well the DB requires it
<lifeless> so the only fallout could be of the form construct; assign; flush
<wgrant> Exactly
<wgrant> And I doubt anything uses the interface
<wgrant> We don't use formlib on EmailAddresses
<wgrant> uhm
<wgrant> r15086 is odd
<wgrant> I think someone mistakenly commented out bouncer.start()
<lifeless> wahuh
<wgrant> The revision was an attempt to make the test more reliable
<wgrant> But I think it made it always fail instead :)
 * wgrant fixes
<wgrant> That works a bit better
<poolie> wallyworld, ok, i added a test for a person with no account
 * wallyworld looks
<StevenK> wallyworld: Ah ha, it doesn't work because Y.one('#radio-button-widget'); is null
<wallyworld> StevenK: sounds plausible depending on how the ui is set up
<wgrant> I hope this is a YUI test :)
<StevenK> wallyworld: # is for class?
<wallyworld> StevenK: # is for id
<wallyworld> . is for class
<wallyworld> poolie: looks pretty good i think. 2 nitpics. imports need to be alphabetical, person = self.factory.makePerson(...) - person is not used so you don't need to make the assignment
<poolie> k
<wallyworld> thanks for making all the requested changes
<poolie> which import is wrong?
<wallyworld> +from lp.services.mail.incoming import (
<wallyworld> + InactiveAccount,
<wallyworld> + authenticateEmail,
<wallyworld> + )
<poolie> ah, case-insensitive sort is required?
<wallyworld> i thought so but now i am doubting myself
<lifeless> there is a tool that will do it
<lifeless> and yes, case insensitive
<wgrant> utilities/format-imports
<wgrant> Also utilities/format-new-and-modified-imports
<lifeless> they should be in lp-dev-utils
<poolie> which machine runs incoming.py again?
<wgrant> loganberry
<poolie> wallyworld, ok, i'll send it up
<wallyworld> poolie: i was just about to mark it approved in anticipation of the last few changes being made :-)
<wallyworld> poolie: i've approved it. did you want me to land it so that i can follow up if there's any issues?
<poolie> that'd be good, thanks!
<poolie> i have one more mail thing in progress so i'll go on to that
<poolie> gl
<wallyworld> np. i'll wait for your changes to be pushed and then throw it at ec2
<poolie> ok, pushed
 * wallyworld lands
<poolie> Getting distribution for 'storm==0.19.0.99-lpwithnodatetime-r406'.
<poolie> An error occurred when trying to install storm 0.19.0.99-lpwithnodatetime-r406. Look above this message for any errors that were output by easy_install.
<poolie> does this mean something?
<wgrant> poolie: Is your download-cache up to date?
<poolie> i did just update it..
<poolie> i have an r406 tarball
<poolie> ah it's ok
<poolie> i made a new chroot and it didn't have ccache, and my $CC points to it
<wgrant> Ah
<wgrant> That would do it indeed.
<wallyworld> wgrant: just to double check - i got 3 ec2 runs rejected because of a pgbouncer error - that's fixed right?
<wgrant> wallyworld: Yup
<wallyworld> thanks, will lp-land the fuckers
<wgrant> Why does each test in test_bugtask_search execute more than 400 queries :/
 * StevenK stabs TAL, and then twists the knife.
<poolie> wallyworld, my attempted landings of that branch get errors connecting to postfix
<poolie> OperationalError: could not connect to server: Connection refused
<poolie>        Is the server running on host "localhost" (127.0.0.1) and accepting
<poolie>        TCP/IP connections on port 52425?
<poolie> maybe it's just transient
<wgrant> poolie: There should be a single failure like that
<wgrant> I fixed it a couple of hours ago
<wallyworld> poolie: there was a bad commit by someone which commented out the startup of pgbouncer
<wallyworld> poolie: that has been fixed now
<poolie> i launched these a couple of hours ago
<poolie> ok
<poolie> but probably before your fix
<poolie> thanks
<wallyworld> poolie: if that's the only error then you are good to lp-land
<poolie> i'll leave it with you
<poolie> 23m to go :)
<wgrant> :(
<wallyworld> poolie: np. what's the url? this is a different one than the mp i landed earlier?
<poolie> it's the one you sent earlier
<poolie> depending when you launched, you may have already caught the fix
<poolie> i fired off an ec2 test earlier in parallel with your reviews
<wallyworld> oh right, i haven't got the email from ec2 yet
<wallyworld> still 1 hour to go before i get the failure email
<poolie> you can look on its web server
<wallyworld> poolie: sure, i was just saying out loud why i didn't realise about the failure yet :-)
<wallyworld> wgrant: sorry, just noticed your branch, looking now
<wallyworld> wgrant: did you know the current storm egg has the Row function?
<wgrant> Oh, does it?
<wgrant> That's handy.
<wallyworld> yeah :-)
<wallyworld> wgrant: perhaps it would be better, in searchClause() in TestBugTaskTagSearchClauses, to only call convert_storm_clause_to_string if the thing in not None and return None otherwise and then test_empty() can just assertIsNone which seems more natural and doesn't need the explanatory text
<wgrant> wallyworld: Possibly, yeah
<wallyworld> poolie: goodbye and good luck
<poolie> thanks, same to you
<wallyworld> poolie: hopefully see you around 11th June :-)
<poolie> yep :)
<adeuring> good morning
<wgrant> poolie: Likewise, good luck and hopefully I'll run into you at some point!
<poolie> yep
<StevenK> poolie: It's a bit sad that I'm tied up for the next two weekends. :-(
<poolie> one more critical bugfix for the road https://code.launchpad.net/~mbp/launchpad/893612-mail-too-big/+merge/101865
<poolie> yeah it would have been nice to see you too
<StevenK> poolie: But, have a great time on your travels, and have fun storming the Googleplex.
<poolie> but, someone else will need to land it
<poolie> thanks
 * wallyworld looks at poolie's one for the road
<StevenK> wallyworld: Right next to the poolie's one for the kerb?
<wallyworld> lol
<StevenK> And then another one for the other kerb?
<StevenK> And then you still have two nature strips, footpaths, street lights ...
<wallyworld> poolie: with incoming size check, shouldn't there be a place where the check is not done anymore and so some code should be cut out?
<poolie> mm good point
<wallyworld> poolie: i'd understand if you didn;t want to do it. i can pick it up if you like
<poolie> i added a comment apologizing for doing it twice
<poolie> but we should just delete it
<wallyworld> i think so. so long as all mail processing goes through the code path that now does the check
<poolie> ok fixed
 * wallyworld hit refresh and taps fingers waiting for the new diff
<wallyworld> as the blackboard said to mr squiggle - "hurry up"
<poolie> upside down!
<poolie> now that would be a nice april fools hack
<poolie> at least for aussies
<poolie> i wonder if you can get there in css
<wallyworld> wgrant is too young to get that reference :-)
<wallyworld> poolie: r=me, i will land. thank you.
<poolie> thanks mate
<poolie> good night and good luck
<poolie> see you in a while
<wallyworld> yes, will do
* wallyworld changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugtasks: 4*10^2
<wgrant> wallyworld: I just get it :)
* adeuring changed the topic of #launchpad-dev to:  http://dev.launchpad.net/ | On call reviewer: adeuring | Firefighting: - | Critical bugtasks: 4*10^2 abgeÃ¤ndert
<StevenK> jml: No QA? :-)
<StevenK> :-( , rather
<jml> StevenK: sorry, no. I got caught up in other things.
* bac changed the topic of #launchpad-dev to:  http://dev.launchpad.net/ | On call reviewer: adeuring, bac | Firefighting: - | Critical bugtasks: 4*10^2 abgeÃ¤ndert
<bac> morning adeuring  ... much reviewing today?
<bac> adeuring, it is the first friday we've both been here in a very long time
<adeuring> morning bac, not much to review today. ANd I must admit that simply forgot to add myself to the topic the last weeks...
<jml> benji: I decided to take the nuclear option on ThreadsafeForwardingResult: testing it so hard that the bugs glow. See https://code.launchpad.net/~jml/testtools/tsfr-fixup/+merge/10189
<benji> jml: interesting, looking at it now
<benji> jml: the branch looks good, the comment at the top of  test_only_one_test_at_a_time is probably the single most valuable thing in the branch and would have saved me many hours, perhaps even a day of flailing
<jml> benji: :)
<benji> jml: the other test methods could really use similar descriptions of why the test is important and what it demonstrates
<jml> benji: I tried to tell you that thing on IRC yesterday.
<jml> or in the MP or somewhere
<benji> jml: you succeeded :)  the day of flail preceded that
<jml> having it in the code ++
<jml> heh, good to know
<jml> benji: have just pushed some documentation and comment tweaks: r263
<benji> I'll take a look.
<jml> benji: you might also be interested in https://code.launchpad.net/~jml/testtools/wrap-result-in-concurrent-suite/+merge/101902 and https://code.launchpad.net/~jml/testtools/tagger/+merge/101904
<jml> anyway, now all of that is out of my head, I will return to my irregularly programmed schedule.
<benji> jml: is there any reason not to always tag tests run concurrently?
<jml> benji: I don't know, to be honest.
<jml> benji: I thought about it.
<benji> hmm, I don't know if subclassing will work for us because we don't controll the code running subunit (it's testr)
<jml> benji: well, I don't think there's a reason for testr not to always tag tests that run concurrently :)
<jml> benji: the reason (to give it too grand a name) I didn't tag tests automatically is because this approach felt more like "here's a bunch of components that you can assemble at will", which is more in keeping with some internal, vague, design aesthetic I have
<benji> jml: I suppose we could do it that way.  When do you expect these three branches to land?
<jml> benji: testtools has a 24hr timeout on code reviews.
<benji> jml: meaning that if they aren't reviewed in 24 hours they are assumed approved?
<jml> benji: so, I guess some time tomorrow, unless lifeless responds with deep objections.
<jml> benji: right.
<benji> k
<jml> benji: obviously we allow for post-merge reviews, and I'd certainly like to get a second opinion from lifeless before a release.
<benji> makes sense
<deryck> Morning, folks.
<rick_h> morning
<jml> hmm.
<jml> I guess this leaves a natural opening for a subunit tool that takes multiple input streams and generates one output stream.
<deryck> abentley, ping for standup
<jml> I'm getting tracebacks running webservice tests, to do with the google service
<jml> http://pastebin.ubuntu.com/927931/
<jml> ah, maybe that's because I didn't update my /etc/hosts file
<jml> (wow, the things you forget!)
<benji> the things we forget are the things we shouldn't be expected to remember <0.2 wink>
<jml> https://code.launchpad.net/~jml/launchpad/expose-commercial-on-create-ppa/+merge/101907
<jml> 1 line patch.
<jml> flacoste: hello
<flacoste> hi jml
<jml> flacoste: may I please have a LoC exception for https://code.launchpad.net/~jml/launchpad/expose-commercial-on-create-ppa/+merge/101907
<jml> flacoste: also, if you could review & land it, that'd be great too.
<flacoste> jml: i'm kind of swamped today, so i may look for the exception, but review and land will have to be delegated
<jml> flacoste: no worries. it's a one-line patch, if that helps.
<jml> abentley: you moved ec2test to lp-dev-utils, right? how do I run the tests there?
<abentley> jml: I was using nosetests.
<jml> abentley: any PYTHONPATH mangling required? (it seems to depend on 'lp'?)
<abentley> jml, no.  I run "nosetests ec2test" and it seems happy.  I think there are two failing tests.
<jml> abentley: oh, there are failing tests in trunk? What failures do you  get?
<abentley> jml: https://pastebin.canonical.com/64277/
<jml> abentley: cool. those are the ones I get. thanks.
<abentley> jml: IIRC, fakemethod is also provided by bzr-pqm.
<jml> abentley: ta.
<jml> abentley: different API, it seems.
<jml> FakeMethod in bzr-pqm (at least the version on the system that comes via rocketfuel-setup) doesn't have call_count.
<abentley> jml: doh.  I guess we should have been more aggressive about switching this code to bzr-pqm originally.
<jml> abentley: well this particular thing is just about using a home-rolled mocking library, right?
<abentley> jml: Yes.
 * jml is tempted to re-implement
<jml> brb
<jml> abentley: https://code.launchpad.net/~jml/lp-dev-utils/fix-tests/+merge/101916
<abentley> jml: I said earlier that PYTHONPATH does not need to be set.  What makes you say it does?
<jml> abentley: because setting it makes the test pass.
<abentley> jml: Which tests does it make pass?
<jml> abentley: ec2test.tests.test_remote.TestDaemonizationInteraction.test_daemonization
<jml> abentley: the error indicates that it can't find ec2test.remote
<jml> abentley: but of course, ec2test.remote exists and is perfectly easy to find, if you have PYTHONPATH set so that the 'ec2test' package is discoverable.
<abentley> jml: When I want subprocesses to have the same pythonpath as the current process, I usually set it in the code that invokes the subprocess.
<jml> abentley: well, not when you're hacking on Launchpad you don't. You use ./bin/py.
<abentley> jml: I can use bin/py when I'm hacking on Launchpad.
<jml> abentley: so you suggest converting turning sys.path into an env var and passing that through in the code that spawns the subprocess?
<abentley> jml: Yes.
<jml> abentley: got a shell quoting function handy?
<abentley> jml: I don't understand why I would need one.  Are you saying that python shell-expands the contents of PYTHONPATH?
<rick_h> deryck: ping, do you have a sec? I'm not seeing tests for any of these cases and I'm feeling like I must be missing something
<rick_h> I'm wearing grep out I think
<abentley> jml: Testing shows python does not shell-expand PYTHONPATH.  It does not expand $PETER in this example: http://pastebin.ubuntu.com/928037/
<rick_h> oh man, nvm...having a monday for a friday. limiting to .py files won't find a bunch of tests :/
<rick_h> deryck: ^
<jml> abentley: what if sys.path has paths that have a space or a colon.
<jml> space seems to work
<cjwatson> shell quoting when spawning subprocesses is a really bad idea
<cjwatson> use the functions that don't require shell quoting instead ...
<abentley> cjwatson: agreed.
<jml> OEK
<jml> OK
<cjwatson> e.g. subprocess.Popen and friends let you pass env=, so you copy os.environ and replace the keys you want directly
<jml> Pretend I didn't say shell quoting. How do you safely turn sys.path into a value for PYTHONPATH that works regardless of what crazy paths are in sys.path?
<abentley> cjwatson: Sure.
<deryck> rick_h, sorry, was on call.
<deryck> rick_h, I'm just about to step in another call.  can we chat after that?
<rick_h> deryck: I think I'm ok now. Sorry for the ping
<abentley> jml: I don't know how to escape colons in a PATH.  Looking into it.
<deryck> rick_h, dude, no worries.  always ping when in doubt.  sorry it took me so long to reply.
<cjwatson> AFAICS there's no way to wedge an entry containing os.pathsep into PYTHONPATH.
<cjwatson> The good news is that, since it can't have got into sys.path via PYTHONPATH, the only way such a directory would be on sys.path is if you put it there yourself.
<jml> fsvo 'you'
<abentley> cjwatson: Exception: https://pastebin.canonical.com/64293/
<jelmer> is it still impossible to rename teams with PPAs that have been deleted, but were published in the past?
<jml> anyway, I've pushed up an os.pathsep.join(sys.path) version and updated the docs.
<abentley> jml: I think you can also update testr.conf
<jml> abentley: I tried. It fails when cwd isn't in PYTHONPATH
<jml> abentley: I can't speak for nosetests, but I'm pretty sure trial adds '.' to the sys.path as a convenience.
<abentley> jml: I saw 123 tests.  Are you seeing only 84?
<jml> abentley: trial shows 88
 * jml runs w/ nosetests
<jml> abentley: 125 w/ nosetests
<jml> abentley: I think that's because nosetests incorrectly discovers the TestRequest tests (from test_remote) inside remote_daemonization_test.py
<jml> abentley: which imports TestRequest in order to get at a helper function.
<abentley> jml: Okay, cool.  So long as we know why it happens.
<jml> abentley: it's a little fiddly to extract, because the helper relies on bzrlib.TestCase*'s make_branch_and_tree
<jml> I really should have pushed harder to get that stuff extracted out of the TestCase inheritance hierarchy.
<jml> brb
<abentley> jml: r=me
<jml> abentley: thanks. How do I land it?
<abentley> jml: We just push to that branch.
<jml> \o/
<jml> abentley: I don't have commit access to that branch.
<abentley> jml: I will do it, then.
<jml> abentley: thanks.
<abentley> jml: I am going to add .testrepository to the list of ignored files.
<jml> abentley: good call. I think I have it in my global ignore
<jml> oh that reminds me
<jml> bac: thanks for the review. can you please submit my branch to ec2 test?
<abentley> jml: landed.
<jml> abentley: thanks.
<rick_h> 362877
<abentley> rick_h: 32568
<rick_h> abentley: bah, my secret it out
<abentley> adeuring: chat?
<adeuring> abentley: sure. mumble?
<abentley> adeuring: sure.
<rick_h> deryck: sanity check time, I'm not seeing the sendmail.simple_sendmail work through the IMailer and so stub.test_emails no longer works.
<rick_h> deryck: in checking out how the sendmail it seems to be straight monkey patching/revert? http://paste.mitechie.com/show/616/
<rick_h> deryck: so I should be doing this in the doctest for checking the notification emails? or am I not seeing the magic pretty way to do it like stub.test_emails?
 * deryck is readying back, thinking.....
<deryck> rick_h, stub.test_emails is how we normally inspect mail in doctest.  and that doesn't work when calling sendmail in process.  I get that problem right?
<rick_h> deryck: right, sendmail.simple_sendmail doesn't hit the stub.test_mails bit
<rick_h> deryck: and it looks like it's because it doesn't use/implement the IMailer interface that stub is working against
<deryck> rick_h, right.  it's just a function that calls sendmail directly, IIRC.
<rick_h> deryck: basically updating hte doctest to monkey patch and restore sendmail.simple_sendmail is ugly as can be and tossing out a last ditch effort I'm missing something
<rick_h> deryck: right, looks like it
<deryck> rick_h, yeah, 1) I wouldn't monkey patch in doctest. I'd create a doctest helper if needed, and 2) we have to have something for this already, let me look more....
<rick_h> deryck: rgr on 1, figured I'd need to do that since I'm updating several doctests to get this tested for each notification
<deryck> rick_h, so I really thought when someone is subscribed to a bug, that we send them a notification in process in this same way....
<deryck> rick_h, kind of a wart I'm recalling that we sent mail in process.
<deryck> rick_h, and the testing is the same it seems. see: lib/lp/bugs/stories/bugs/bug-add-subscriber.txt
<deryck> rick_h, so either, I'm wrong and we don't do it in process, or you can look there and work out what is different about that test that it works.
<rick_h> deryck: ok, will look there. Probably using a different method of sending the emails than the simple_sendmail
<rick_h> deryck: thanks
<deryck> rick_h, so the subscription stuff does use the event system to trigger sending mail, but it still just wraps a callback that calls sendmail directly, which is at heart the same thing you're doing....
<deryck> rick_h, see lp.bugs.subscribers.bug.send_bug_details_to_new_bug_subscribers to see how the mail is sent....
<deryck> rick_h, and the stub mailer works fine there.
<rick_h> deryck: looking
<deryck> rick_h, should we high bandwidth this in a hangout?
<rick_h> deryck: sorry, trying to pdb to see if I'm actually hitting the email code like I think I am and so is stub emails blank because simple_sendmail doesn't hit it, or my code isn't working like I think it should
<rick_h> except in doctest having a hard time seeing if I'm hitting pdb because of hte block of code I think I am
<rick_h> pdb doensn't show me the line num of the doctest in the path so not sure if it's a test before/after the one I'm trying to see
<deryck> rick_h, yeah, that's why I was thinking hangout.... we could step though the paths together.
<rick_h> deryck: sorry, long way of saying give me a few min
<rick_h> deryck: k, hangout is ok then
<deryck> rick_h, yeah, let's do, just to possibly save you some frustration.
<rick_h> deryck: ok, we can try :)
* adeuring changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: bac | Firefighting: - | Critical bugtasks: 4*10^2
<rick_h> ok, if in doubt, transaction.commit!
<bac> hi abentley , i'm finding when i run out tests within an lxc that many (perhaps all) of the celery tests fail with a timeout.  can you give me some advice on tracking this down?
<bac>  s/out/our
<abentley> bac: with you in a minute.
<abentley> bac: Okay.
<abentley> bac: So, the timeout is emitted from a line that says something like responses[0].wait ?
<bac> http://paste.ubuntu.com/927046/
<bac> abentley, yeppers
<bac> abentley, ignore the 'whoami' errors, those have been resolved
<bac> only interested in the timeouts
<abentley> bac: So this means that rabbitmq did not send a message saying the job was processed in 30 seconds.
<abentley> bac: possible causes: celeryd did not start correctly
<abentley> bac: celeryd was not listening to the correct rabbitmq instance
<abentley> bac: Not sure what else.
<bac> abentley, should celeryd be running outside of the tests?
<abentley> bac: No, the tests that need it will start it.  lp.services.job.tests.celeryd is a context manager that starts and stops celeryd.
<bac> ok
<abentley> bac: does this happen if you restrict the tests to just ones using celery (e.g. "-t Celery") ?
<bac> abentley, let me try
<abentley> bac: I am guessing that if all the celery tests run on process, they work, but if they run on different processes they don't.
<abentley> s/run on process/run on one process/
<abentley> bac: rebooting.  Back in a sec.
<abentley> bac: My best guess is that celeryd will be using the RabbitMQ config associated with another process.
<abentley> bac: To test this, we could print celeryd's Popen.stderr.  That will indicate the broker config.  The Popen object is yielded by lp.services.job.tests.celeryd.
<bac> ok
<abentley> bac: in some tests, it's referred to as "proc".
<bac> abentley: Total: 12 tests, 2 failures, 5 errors in 5 minutes 25.011 seconds.
<abentley> bac: Okay, cool.  We can reproduce it, then.
<abentley> bac: lp.code.model.tests.test_branch.TestBranchJobViaCelery.test_branchChanged_via_celery show a "Function not implemented" OS error.
<abentley> bac: That may be the root cause of all of this.
<bac> abentley, where do you see that FNI error?
<abentley> bac: line 279 of your paste.
<abentley> bac: reconstructed: http://pastebin.ubuntu.com/928292/
<bac> abentley, hey, that makes sense as there is a problem with lxc and semaphores
<abentley> bac: yay?
<bac> abentley, cool, thanks for the pointer
<abentley> bac: np.
<abentley> bac: if celeryd is incompatible with lxc, I guess the options are 1. fix lxc, 2. fix celeryd, 3. run celery tests outside lxc.
<abentley> 4. run celeryd outside lxc, run tests that use it inside lxc.
<SpamapS> Hey we're getting ready to bump the 'charms' distribution from oneiric->precise .. which means a LOSA running the script that does that. I forget who I need to ping to get that done.
<SpamapS> flacoste: any ideas? ^^
<james_w> SpamapS, copying branches, or just setting the precise as the default release?
<flacoste> james_w: he probably wants to copy branches also
<flacoste> or rebase them
<james_w> SpamapS, either way saying "webops ping" will get you a friendly webops (neÃ© LOSA) to help when it is time
<SpamapS> I want to do both
<james_w> (sorry webops)
<SpamapS> we need to shift dev focus to precise, and copy all the oneiric branches to precise.
<SpamapS> so they'll know the script to do it?
<SpamapS> also will the mass copy process just ignore anything that has already diverged in precise?
<SpamapS> I asssume yes
<james_w> SpamapS, https://wiki.canonical.com/InformationInfrastructure/OSA/LPHowTo/BranchUbuntu?highlight=%28branch%29|%28distro%29
<james_w> that's the script we use for Ubuntu
<james_w> I don't know how it handles existing branches though
<james_w> it looks like it might not handle it all that well
<bac> abentley, i just saw this commit message of yours and had to laugh:  "There's no j in Launchpad."
<abentley> bac: :-)
* gmb changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugtasks: 4*10^2
#launchpad-dev 2012-04-14
<wgrant> jml: Hi, how goes your QA?
<wgrant> jml: QA delays block the whole team, so they should generally considered just about top priority.
<jml> well, I'd reply to wgrant.
<jml> but he's not heree
<jml> the perils of IRC as a communication medium.
<wgrant> jml: Ah, thanks.
<wgrant> Apparently I managed to unknowingly suspend irssi a few hours ago...
<wgrant> That bug is actually qa-ok -- it may not fix the bug, but it doesn't break existing functionality, so it can be deployed.
<jml> wgrant: I was surprised that you weren't around. :)
<jml> ok
<jml> hmm.
<jml> I'm kind of stuck in other things right now, but it would be good if the docs were updated to make that clearer.
<wgrant> Indeed.
<wgrant> It all sort of changed when nodowntime happened.
<jml> yeah
<jml> oh
<jml> that reminds me
<jml> james_w hacked up a prototype base on lpqateam that tracks QA based on revisions, not bugs.
<wgrant> Yeah, I saw that.
<wgrant> Makes a lot more sense.
<StevenK> Does it publish anywhere?
<StevenK> wgrant: How did you suspend irssi? I thought it trapped Ctrl-Z?
<wgrant> StevenK: So did I.
<wgrant> I have no idea how I did it.
<StevenK> Hm, it doesn't trap it.
<nigelb> Yeah, this has happened to me. Multiple times.
<StevenK> I don't tend to hit Ctrl-Z by accident, though.
<nigelb> I do. Instead of Ctrl + A.
<nigelb> Btw, tracking qa based on revisions is what the deployment report does, right?
<nigelb> Or is there something else that jono's talking about?
<wgrant> It's similar.
<wgrant> Our current one tracks on an awkward hybrid of revisions and bugs.
<wgrant> Which has some pretty strange issues.
<jml> StevenK: what do you mean "does it publish"?
<jml> StevenK: it's a web service with a pretty looking web page
<nigelb> oh? what kind of issues?
<StevenK> jml: Which is what I meant. What URL is it at?
<jml> StevenK: oh right, I can never remember, he hosts it on ec2 or canonistack or something.
<jml> http://ec2-184-73-64-233.compute-1.amazonaws.com/
<jml> http://ec2-184-73-64-233.compute-1.amazonaws.com/project/Launchpad
<nigelb> bah. openid.
 * nigelb hopes he has access.
<nigelb> Not allowed here.
<StevenK> Neither, I suspect I should have ticked the team checkbox
<cjwatson> I didn't get a team checkbox.
<cjwatson> Maybe it's restricted to ~launchpad or something.
<wgrant> It asks for launchpad
<wgrant> Not canonical or anything like that
<cjwatson> Or anything that might let nigelb see it.
<StevenK> Now I'm not sure how to actually tell SSO to let me see it.
<nigelb> logout and log back in guess, with the team checked.
<StevenK> Which doesn't help, sadly.
<StevenK> Since the site itself still knows about me. Maybe I need to delete cookies.
<nigelb> Oh by logout, I meant logout of the site.
<nigelb> hrm, maybe deleting cookies is a good idea :D
<StevenK> nigelb: How does that help, every page gives "Not allowed here."
<G> StevenK: I love it when sites do that
<G> *cough*
<StevenK> Hey, that site does look pretty.
<G> wgrant: btw, maybe instead of ^Z you did ^Q (XON Flow control) ;)
 * nigelb waves to G 
<nigelb> StevenK: I meant, flush the cookies, so it forgest your login, so you can re-login, giving it your team credentials.
<G> nigelb: howdy
#launchpad-dev 2012-04-15
<wgrant> Does someone have a Gmail account with which they can test bug #925597?
<_mup_> Bug #925597: From address is ignored if DKIM is supported <email> <qa-needstesting> <regression> <Launchpad itself:Fix Committed by mbp> < https://launchpad.net/bugs/925597 >
<mwhudson> wgrant: probably
<mwhudson> wgrant: afk for 5-10 mins though
<wgrant> mwhudson: That would be great, thanks.
<mwhudson> wgrant: so what do i need to do in words of 1 syllable?
<mwhudson> wgrant: send an email from a gmail account that is not registered in lp with a from: that is registered in lp?
 * mwhudson tries to remember the password for his other account...
<wgrant> mwhudson: Yep, that's right.
<wgrant> mwhudson: Which probably just means removing your gmail address from your account on qastaging
<mwhudson> wgrant: to 1@bugs.qastaging.canonical.com or something?
<mwhudson> ah yeah that might be easier...
<mwhudson> qastaging appears to need a fresh hamster
<mwhudson> wgrant: ok, two mails sent
<mwhudson> wgrant: how often does it get looked at on qastaging?
<wgrant> mwhudson: */15, I think. Let me check.
<mwhudson> (sent to https://bugs.qastaging.launchpad.net/pydoctor/+bug/409996)
<_mup_> Bug #409996: nevowhtml needs tests <pydoctor:Triaged> < https://launchpad.net/bugs/409996 >
<wgrant> */10
<wgrant> Thanks
<wgrant> Did they contain commands?
<wgrant> The no-commands case is probably the only really interesting one.
<mwhudson> no
<wgrant> Great
<mwhudson> command-containing would need to be gpg signed in this case?
<wgrant> Not from gmail.com, if the address is known
<wgrant> This change should still trust commands from gmail.com, but not if the From is registered but the @gmail.com is not.
<mwhudson> do we trust the from: for all gmail emails?  even if the sender: address is unknown
<mwhudson> ah
<mwhudson> right
<wgrant> Previously we didn't fall back to the From if the authenticated sender wasn't registered, so the problematic emails just got dropped.
<wgrant> This change should fall back to the From, but remain unauthenticated.
<mwhudson> right
<mwhudson> this waiting is pretty tedious :)
<wgrant> 40s to go :)
<mwhudson> dun dun dun
<mwhudson> wgrant: seems to have worked
<wgrant> Ah
<wgrant> I was looking at prod
<mwhudson> yay mup
<wgrant> Because I used _mup_'s link
<wgrant> Yeah
<mwhudson> the mail from: the unknown address seems to have fallen into a hole
<wgrant> mwhudson: The other mail had a @gmail.com from?
<mwhudson> i guess that's expected?
<mwhudson> yeah
<wgrant> Great
<wgrant> qa-ok
<wgrant> Thanks!
<mwhudson> np
#launchpad-dev 2013-04-08
<StevenK> wgrant: http://pastebin.ubuntu.com/5688104/
<wgrant> StevenK: Why does LP_JS_BUILD depend on YUI_DEFAULT_SYMLINK, and why does YUI_DEFAULT_SYMLINK depend on YUI2_BUILD?
<StevenK> wgrant: So that we have one rule that will do all population via dependency resolution
<wgrant> StevenK: Isn't that jsbuild?
<StevenK> wgrant: I can expand it out into jsbuild if you wish
<wgrant> StevenK: That seems better than having random deps like you have now
<StevenK> wgrant: http://pastebin.ubuntu.com/5688136/
<wgrant> StevenK: That seems more sensible, though it looks like you can probably now merge JS_OUT, JS_ALL and JS_BUILD
<wgrant> Er
<wgrant> s/JS_BUILD/jsbuild/
<StevenK> JS_ALL and JS_OUT can not be merged
<StevenK> The JS_ALL rule can be flattened
<wgrant> They can be merged, because we never used launchpad.js now
<wgrant> I don't think support YUI <3.5, and we only support 3.5 when it's comboloaded
<StevenK> wgrant: You want me to rip out generation of launchpad.js?
<StevenK> And yes, launchpad.js only supports YUI 3.3.0
<wgrant> StevenK: Given you're doing this rework now, and it will be purely deleting code...
<wgrant> 3.3 support was only kept so we could do a gradual migration to 3.5
<wgrant> That migration is done
<wgrant> Multi-version support should probably be preserved for future upgrades, but we don't need to preserve the non-comboloader approach
<StevenK> js.combo_loader.enabled can die too
<wgrant> Yes
<wgrant> And the code that it disables
<wgrant> Which is probably just two lines in the root template
<StevenK> Yeah
<StevenK> wgrant: It's a bit more than two lines
<StevenK> % bzr di lib/lp/app/templates/base-layout-macros.pt | diffstat -s
<StevenK>  1 file changed, 1 insertion(+), 20 deletions(-)
<wgrant> :)
<StevenK> Grrrr
<StevenK> combine-css depends on jsbuild implicitly, since it wants all of the css files in icing
<wgrant> StevenK: Can you do the CSS copying outside jsbuild?
<StevenK> wgrant: Sorry, where jsbuild means bin/jsbuild
<StevenK> wgrant: http://pastebin.ubuntu.com/5688275/
<wgrant> StevenK: Doesn't js.yui-version want to stay?
<wgrant> You can also drop YUI_VERSIONS down to just 3.5.1
<StevenK> wgrant: js.yui-version isn't referenced anywhere else
<StevenK> wgrant: And I can not.
<StevenK> YUI 3.3.0 is linked into icing, and combo.css uses one of the files
<wgrant>     @property
<wgrant>     def yui_version(self):
<wgrant>         """The version of YUI we are using."""
<wgrant>         value = getFeatureFlag('js.yui_version')
<StevenK> Haha
<wgrant> StevenK: There's no corresponding file in 3.5.1?
<StevenK> Note the disparity
<wgrant> Indeed
<wgrant> The name in the docs was wrong
<StevenK> wgrant: Fixed the docs
<StevenK> wgrant: cssreset/reset.css 3.3.0 and 3.5.1 has a slight difference
<StevenK> wgrant: http://pastebin.ubuntu.com/5688372/
<wgrant> StevenK: YUI_DEFAULT_SYMLINK probably wants a -n, and won't combo.css now get the 3.5.1 reset/grid CSS because of the default symlink change?
<wgrant> Actually, we may still be using a copy of the 3.0.0 grid CSS...
<wgrant> Yeah
<wgrant> So it might just be reset that we care about
<wgrant> Baaaah
<StevenK> wgrant: Hmm?
<wgrant> StevenK: Tried to reproduce the process-upload logging unicode issue this morning
<wgrant> Finally realised what I was doing wrong now
<wgrant> It's failing to write to the file stream, not the stderr steam
<wgrant> stream
<StevenK> wgrant: So the 3.5.1 link is okay in terms of reset.css?
<wgrant> StevenK: I don't know; we'll want to test
<wgrant> But it's probably fine.
<StevenK> wgrant: I've fixed the -n for DEFAULT_SYMLINK. Shall I push this mess up?
<wgrant> StevenK: Sounds like a plan
<wgrant> I've wanted to fix this for a while :)
<StevenK> wgrant: Which, the process-upload, or the YUI building thing that I got annoyed enough to take an axe too?
<wgrant> StevenK: Both, but I was talking about the latter.
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/silence-yui-build/+merge/157272
<wgrant> StevenK: Hmm
<wgrant> StevenK: I think we need to always run lpjsmin, don't we?
<StevenK> wgrant: Why?
<wgrant> The launchpad.js call was guarded before, but the combo-rootdir one was not
<StevenK> Yeah
<wgrant> Because launchpad.js was either minified or not
<wgrant> Whereas for the comboloader it produces separate .min versions
<StevenK> It does, yes
<wgrant> So we probably just want to always run it
<wgrant> Unless there's a reason not to?
<StevenK> Then the variable just dies
<wgrant> Exactly
<wgrant> And we remove more dev/prod crap from the makefile :)
<StevenK> wgrant: Do we want to sprinkle in YUI 3.9.1 after this branch?
<wgrant> StevenK: I imagine there'll be some porting work, but we should at least see how many tests break
<wgrant> The 3.5 upgrade wasn't that bad, apart from the comboloader
<StevenK> wgrant: 3.9.1 in YUI_VERSIONS will blow up tests?
<wgrant> I've been meaning to modernise our Zope stack for a while
<wgrant> StevenK: No, but once it's there we'll want to run the tests over it
<wgrant> I guess that just means changing the default, jsbuild, then run the tests?
<StevenK> Yeah
<StevenK> I can add it to download-cache and prep a throwaway branch to toss at ec2 to do so
<wgrant> I wouldn't bother ec2ing it. The tests don't take long to run locally.
<wgrant> And easier to debug
<wgrant> Then we can hopefully migrate to the new calendar and kill off yui3.3
<StevenK> And 2
<wgrant> Er
<wgrant> That's what I meant, yeah
<wgrant> 3.3's already dead :)
<StevenK> :-)
<StevenK> Not until you approve my branch it isn't
<wgrant> Ah, didn't realise JS_BUILD had perished.
<StevenK> I got distracted by URL-hacking to find a 3.9.1 zip that I didn't tell you I'd pushed
<wgrant> I got distracted by failing to find a YUI3 changelog
<StevenK> http://yuilibrary.com/download/yui3/ doesn't mention 3.9.1, but the zip file does exist
<wgrant> The 3.6.0, 3.7.0, 3.8.0 and 3.9.0 announcements look safe enough
<wgrant> So there shouldn't be much fallout, AFAICT
<StevenK> wgrant: http://pastebin.ubuntu.com/5688475/
<wgrant> StevenK: Leave 3.3.0 so current devel is buildable..
<wgrant> But 3.3 and 3.4.1 can go
<StevenK> wgrant: That's 3.3.0.zip
<wgrant> Orright
<StevenK> -rw-rw-r-- 1 steven steven 12M Jul 17  2012 yui_3.3.0.zip
<StevenK> -rw-rw-r-- 1 steven steven 33M Jul 17  2012 yui_3.5.1.zip
<StevenK> -rw-rw-r-- 1 steven steven 30M Apr  8 14:57 yui_3.9.1.zip
<wgrant> :)
 * StevenK watches bzr upload 30MiB
<StevenK> wgrant: Can haz +1?
<wgrant> StevenK: Is icing/yui's sole remaining purpose to feed the CSS combiner?
<StevenK> Yes
<wgrant> Yay
<wgrant> r=me
<StevenK> We could grab reset.css and drop that requirement
<wgrant> Let's not do that right now
<wgrant> This branch avoids evil so far
<StevenK> Avoids? It outright removes it
<StevenK> wgrant: -vvm javascript, or which tests were you thinking?
<wgrant> StevenK: -vv --layer YUI
<StevenK> wgrant: They're all failing with AssertionError: JS timed out.  The test may never have started.
<StevenK> Oh, not all of them
<wgrant> StevenK: Some of them may reference files that don't exist any more. Try running the failures in a browser
<wgrant> Also make sure you've actually jsbuilt.
<StevenK> Failure in /home/steven/launchpad/lp-branches/test-391/lib/lp/registry/javascript/sharing/tests/test_sharingdetails.html.Sharing Details.test_display_error_bug: Unexpected error: 'undefined' is not a function
<StevenK> Passed tests:     93
<StevenK> Failed tests:      7
<wgrant> That's not bad, considering how much that page does
<wgrant> How many test files fail vs pass?
<StevenK> 91.1% pass, 6.9% fail
<StevenK> 102 total
<wgrant> Great.
<StevenK> And 2 skipped, for some reason
<wgrant> That's from testr?
<StevenK> bin/test -vv --layer YUI --subunit, and then I ran subunit-stats over the stream
<wgrant> Right
<wgrant> So, sounds like the upgrade will be easy.
<wgrant> Feel like finishing it off now?
<StevenK> I don't think we'll jump to it by default, but I can fix the tests
<wgrant> yeah, we'll want to trial it ourselves for at least a few days, but I think we can probably get the tests fixed and landed today
<wgrant> Unless something terrible happens
<StevenK> The YUIXHR tests all fail
<StevenK> lib/lp/app/javascript/choiceedit/tests/test_choiceedit.html passes in my browser, but failed under bin/test
<wgrant> THey're fragile at the best of times
<wgrant> Do they even pass with 3.3?
<wgrant> Or 3.5?
<wgrant> Hmm
<StevenK> I thought buildbot ran them?
<wgrant> Yeah
<wgrant> But I mean on your system
<StevenK> devel gives the same failures against 3.5.1
<StevenK> (WRT yuixhr)
<wgrant> Right
<wgrant> What about 3.3?
<StevenK> wgrant: 3.3.0 (IE devel r-1) works
<StevenK> Ah ha, they die on buildbot too
<wgrant> Indeed. Do you want to look at that, or do you want to look at the eg. choiceedit failures while I poke at YUIXHR?
<StevenK> wgrant: I'll see if I can work out the other 4 if you grab YUIXHR
<wgrant> kk
<czajkowski> jtv: does this sound right to tyou ??? https://bugs.launchpad.net/launchpad/+bug/1162192
<_mup_> Bug #1162192: Partial pot-files destroy translations <lp-translations> <Launchpad itself:New> < https://launchpad.net/bugs/1162192 >
<jtv> czajkowski: hi.  No, that does not sound right to me.
<jtv> Maybe somebody got clever and decided we can delete TTIs instead of setting their sequence number to zero.
<czajkowski> StevenK: wgrant has anything been done to translations in the last week or so ?
<wgrant> jtv, czajkowski: I discussed that with danilos last week
<jtv> TTI stands for TranslationTemplateItem.  It's basically an entry that says "this particular translatable message, or POTMsgSet, occurs in this particular template, and here's its message number in that template."
<wgrant> It regressed in mid-2011
<wgrant> When a POTMsgSet pruner was added
<wgrant> It also prunes TTIs with sequence == 0
<jtv> !
<wgrant> Which I believe is insane
<wgrant> But I was hoping to get input from you or danilos on that :)
<czajkowski> jtv: morning btw :)
<jtv> When a message was once in a template but no longer is, we keep it and set its sequence number to zero (not null unfortunately, for historical reasons) exactly so that it can be restored easily, with all its translations.
<jtv> So wgrant, here's my input: yes that's insane.  :)
<wgrant> jtv: That's what I thought, yeah. http://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/view/head:/lib/lp/scripts/garbo.py#L1230 is the code
<jtv> There should be at least a grace period.
<wgrant> Until last week I assumed that it just pruned POTMsgSets
<wgrant> But it prunes obsolete TTIs too
<wgrant> Yes
<wgrant> That would be more reasonable :)
<wgrant> But I'm more inclined to just not prune obsolete TTIs at all
<wgrant> There shouldn't be that many of them
<jtv> Well you can't prune POTMsgSets without getting rid of the TTIs.
<wgrant> Well, POTMsgSets from deleted templates will get pruned
<wgrant> But that's about it
<wgrant> Though can you even delete templates...
<jtv> You can mark them as deactivated, not delete them.
<jtv> Or you can merge them into other templates.
<wgrant> So perhaps we want to set a date when we obsolete a TTI
<wgrant> Then kill them after 6 months or something
<jtv> And then only prune POTMsgSets that don't have any TTIs.
<wgrant> Well, FKs should ensure that.
<wgrant> But yes
<jtv> I mean: the two can be entirely separate processes.
<jtv> Where one _only_ prunes long-unused TTIs, and the other prunes unreferenced POTMsgSets.
<wgrant> Oh, I thought that's what it did already, but you're right.
<wgrant> It is entirely TTI-driven
<wgrant> I hadn't read the code beyond seeing the sequence == 0 bit and crying.
<jtv> I'll bet the join is complicated by having to deal with both sequence == 0 *and* there being no TTI.
<jtv> I wanted null for the unused sequence number.  Would have made it easy to sort them first or last as desired, for one.
<wgrant> Though it's certainly faster to be TTI-driven, but there are few enough POTMsgSets that it might not matter
<jtv> I would also expect obsolete POTMsgSets etc. to pile up slowly enough that it's more a matter of a one-off cleanup followed by reruns at glacial intervals.
<wgrant> I think it currently runs daily mostly because there is no less frequent schedule
<wgrant> But I was going to introduce a garbo-weekly or garbo-monthly for this
<jtv> Nice.
<jtv> For those jobs where the cost of checking for cruft vastly outweighs the cost of cleaning it up.
<wgrant> Yep
<wgrant> And the cost of the cruft is so minimal, except over huge intervals.
<wgrant> StevenK: buildbot's just run the first yuixhr test, and it succeeded. the milestone one needed another YUI 3.5 fix, but I think they're all good in devel now. Hopefully that'll fix your issue with 3.9
<wgrant> Huh
<wgrant> The other 3 tests failed.
<wgrant> Still pass locally :/
<wgrant> Even in a clean branch
#launchpad-dev 2013-04-09
<wgrant> StevenK: I'm out for a while, but I have a lead and can reproduce it on ec2. Hopefully get it fixed soonish
<StevenK> wgrant: The subscription YUI issue is because it is the only call in the codebase that calls Y.substitute() with a 3rd argument of a function to do the replacement.
<StevenK> According to upstream, that means "We have a harder migration", which is pointing to Y.Template, but Y.Template was only added in 3.8.0 and I can't figure out how to have two code paths depending on the YUI version.
<wgrant> StevenK: Where's the call?
<StevenK> wgrant: lib/lp/bugs/javascript/subscription.js line 849
<wgrant> StevenK: Can't you just manually preprocess the keys?
<StevenK> wgrant: As in call Y.Lang.sub() with additional_vars as well?
<wgrant> StevenK: The function just takes the key and the value
<wgrant> So you should be able to replicate the present behaviour by just manually mapping the function over vars, and giving Y.Lang.sub the result
<wgrant> StevenK: Can you try running YUIAppServerLayer locally?
<wgrant> I've managed to reproduce it once on a single test locally, and two ec2 runs last night failed with it, but I can't repro it on ec2 manually. I guess it must be a race.
<StevenK> wgrant: The function also uses additional_vars from the outer function
<wgrant> StevenK: Sure, is that a problem?
<StevenK> -    return Y.substitute(subscription.reason, subscription.vars, var_replacer);
<StevenK> +    vars = var_replacer(subscription.vars);
<StevenK> wgrant: Like that?
<StevenK> +    return Y.Lang.sub(subscription.reason, vars);
<StevenK> It even fits inline
<StevenK> wgrant: YUIAppServerLayer against 3.5.1 or 3.9.1?
<wgrant> StevenK: 3.5.1
<wgrant> StevenK: var_replacer takes a key and a value
<wgrant> you'll need to map it over the keys and values of cars
<wgrant> vars
<StevenK> So it does
<StevenK> Where did the keys come from?
<wgrant> I'm confused
<wgrant> subscription.vars should be a map
<StevenK> wgrant: The plot thickens. key is only used for additional_vars
<wgrant> StevenK: Sure, because it already has the value from subscriptions.vars
<wgrant> It's given the (key, value) pair from subscription.vars
<StevenK> Ah ha, we do have to call it twice
<StevenK> It will only act on additional_vars if vars is undef
<wgrant> That presumably means you want to merge the keys of subscription.vars and additional_vars
<StevenK> bin/test -vv --layer YUIAppServerLayer --subunit against 3.5.1 looks fine
<wgrant> Yeah :(
<wgrant> I might just disasble the layer by default for now...
<wgrant> Does it work with 3.9.1 too?
<wgrant> I have YUIAppServerLayer repeating on four machines now, hopefully one of them will give me a failure soon...
<StevenK> wgrant: Your leads all ended up being nothing? :-(
<StevenK> wgrant: 3 failures with 3.9.1
<StevenK> Looks suspcisiously like the buildbot failures
<StevenK> Hmmm
<StevenK> I can't just call it with subscription.vars since that's an object too
<StevenK> wgrant: I have all failures sorted for 3.9.1, except for the 3 XHR failures
<wgrant> StevenK: Can you paste the failures?
<wgrant> I will be very happy if you can reproduce them :)
<StevenK> wgrant: I can copy the subunit stream to carob or so if you wish
<wgrant> StevenK: I'm more interested in whether they are reproducible
<wgrant> My four continuous runs have yielded nothing so far :/
<StevenK> Re-running --layer YUI against 3.9.1
<StevenK> wgrant: It happens against on a re-run
<StevenK> *again
<wgrant> StevenK: Same errors? Success on test_yuixhr_fixture_facet, a timeout before test start on test_milestone_creation, and undefined errors on the other two?
<StevenK> Yes
<wgrant> Can you push up your 3.9.1 branch so I can try to get them locally?
<StevenK> wgrant: I've split it into two
<StevenK> You should be able to merge the one I'm about to push, change default to 3.9.1 and make jsbuild
<StevenK> wgrant: lp:~stevenk/launchpad/sprinkle-in-yui-391
<wgrant> :S
<wgrant> All four still pass
<wgrant> On both my machines
<StevenK> wgrant: build/js/yui points to 3.9.1?
<wgrant> Yes
<StevenK> wgrant: Odd. They still fail for me
<wgrant> actual    = ['Failure in /home/wgrant/launchpad/lp-branches/yuixhr-3.5/lib/lp/app/javascript/choiceedit/tests/test_choiceedit.html.nullable_choice_edit.test_action_icon: Add icon is not visible when it should be\nExpected: inline-block (string)\nActual: inline (string)']
<wgrant> That's the only failure I get with 3.9.1
<wgrant> (ignore the branch name)
<StevenK> wgrant: What other changes have you made?
<wgrant> Nothing. That's just the branch that switched the default.
<wgrant> It's identical to devel
<wgrant> The ec2 runs where I can't manually reproduce it *are* devel
<wgrant> :/
<StevenK> wgrant: sprinkle-in-yui-391 doesn't change the default, but okay
<StevenK> wgrant: Are you able to try on Quantal, rather than Lucid?
<wgrant> Oh yeah, I changed the default too. but that's it
 * wgrant tries on precise
<wgrant> Still works fine :/
<wgrant> And that precise container is even amd64
<StevenK> wgrant: This saddens me
<wgrant> StevenK: Can you reproduce the test_milestone_creation issue in a browser?
<wgrant> (copy yuitest.zcml from configs/testrunner-appserver to configs/test-playground, bin/run -r librarian -i test-playground, visit https://launchpad.dev/+yuitest, select test_milestone_creation... it will fail because it can't reset the database, but you should at least see it running the tests)
<StevenK> wgrant: On 3.5.1 or 3.9.1?
<wgrant> StevenK: Doesn't really matter, if it fails on both
<StevenK>     lp.registry.javascript.tests.test_milestone_creation
<StevenK> Want to re-run your test?
<wgrant> You don't get a test console at the bottom after a few seconds?
<StevenK> Nope
<wgrant> :D
<wgrant> Check the browser's network/JS console
<wgrant> For errors
<StevenK> Hmm
<StevenK> GET meta.js ended up with a 500
<wgrant> Oho
<wgrant> You've run combobuild, right?
<StevenK> The path is bong
<StevenK> https://launchpad.dev/+yuitest/build/js/lp/meta.js
<wgrant> That should be fine
<wgrant> +yuitest/build forwards to the comboloader dir
<wgrant> So it doesn't have to use a real comboloader
<StevenK> Passed:2 Failed:2 Total:4 (0 ignored)
<wgrant> What'd you change?
<StevenK> I ran make combobuild
<wgrant> Ah
<wgrant> Do your tests pass now?
<wgrant> Make sure you shut down test-playground before you rerun YUIAppServerLayer
<StevenK> test: lp/registry/javascript/tests/test_milestone_creation
<StevenK> WARNING: gnome-keyring:: couldn't connect to: /run/user/steven/keyring-hlnCV9/pkcs11: No such file or directory
<StevenK> successful: lp/registry/javascript/tests/test_milestone_creation
<StevenK> Ah ha
<wgrant> Damn.
<wgrant> Can you paste the failures you got before the combobuild?
<StevenK> Just make clean && make and you should have no combobuild ?
<StevenK> Or rm -rf build/js && make jsbuild
<StevenK> Failed tests:      0
<wgrant> I'd expect a missing meta.js to cause all four suites to fail to start
<wgrant> but omfg
<wgrant> that was it
<wgrant> It's just a missing meta.js
<wgrant> fuuuuu
 * wgrant lands fix
<StevenK> Haha
<wgrant> Thanks
<StevenK> What is the fix, then?
<wgrant> I guess the other tests specify just enough direct deps to get them stat
<wgrant> build needs to depend on combobuild
<wgrant> The reason it worked whenever I tried it on ec2 is that I ran 'make' before testing...
<StevenK> That will blow up
<StevenK> You can't rely on convoy being available
<StevenK> Which was the whole reason combobuild was split out
<wgrant> Ah, right
<wgrant> Might just put that in check, then
<wgrant> I think buildbot and ec2 both use check, right?
<StevenK> Not sure
<StevenK> ec2 does
<StevenK> Since I just checked
<wgrant>         command = ['make', 'check']
<wgrant> Yeah
<wgrant> Hmm
<wgrant> buildbot won't
<wgrant> Because it does it parallel
<wgrant> lxc-clean codehosting-dir build schema
<StevenK> We could do it in build, but the only way I can think to flag if convoy is available is -f /usr/share/convoy/convoy.wsgi or something equally hideous
<wgrant> Well
<wgrant> StevenK: Is there a reason to not do it in build now, other than the fact that python-convoy isn't installed everywhere?
<wgrant> Easiest thing to do may just be to add it to launchpad-dependencies
<wgrant> Otherwise we can change the buildbot config
<StevenK> wgrant: We don't need to do so everywhere
<StevenK> Only the frontends really need it
<wgrant> But is there any reason to not?
<StevenK> Save a little time
<StevenK> I guess
<wgrant> We already do the JS building etc. unnecessarily on the backends and non-web machines
<StevenK> Not anymore we won't
<StevenK> wgrant: I think python-convoy is only on banana/nutmeg and not everywhere
<wgrant> Sure
<StevenK> But we sync built trees, no?
<wgrant> Yes.
<wgrant> But then they're remade at the destination
<StevenK> Hah
<StevenK> wgrant: I seem to recall you objecting to needing convoy everywhere, but I could be recalling wrong
<wgrant> So do I, but I can't find the IRC logs.
<StevenK> wgrant: If we want convoy everywhere, we just move combobuild into jsbuild
<wgrant> Yes.
<wgrant> I suspect my objection was just that the easiest way to get the world unbroken was to split combobuild back ouit
<StevenK> Right
<wgrant> Now that combobuild is mandatory, having it separated from jsbuild probably doesn't make sense.
<wgrant> But buildbot should have convoy installed, so we could unbreak it now by merging them with a guard.
<wgrant> And then push out the dependency changes, or stop building the JS everywhere, later.
<wgrant> I don't remember if we actually use make build on prod anywhere
<StevenK> I *think* banana and nutmeg have been hacked to also run combobuild
<StevenK> And HTF knows what asuka does any more
<wgrant> staging uses make build
<wgrant> rocketfuel-built on carob doesn't have any JS stuff
<wgrant> Looking at deploymgr configs, codebrowse uses make build
<wgrant> For no reason
<wgrant> I don't think...
<StevenK> stevenk@carob:~$ dpkg -l python-convoy
<StevenK> No packages found matching python-convoy.
<wgrant> But other than that it's just banana/nutmeg
<wgrant> StevenK: Yeah, but carob has no JS at all. It must just use compile, not build
<StevenK> Ah
<wgrant> Actually, it probably only uses build_eggs
<wgrant> Not even compile
<wgrant> And asuka must already have python-convoy, since its comboloader works
<wgrant> So it might just be codebrowse that needs fixing to not call build
<StevenK> I think the upgrade scripts call combobuild
<wgrant> Upgrade scripts/Z
<wgrant> ?
<StevenK> The staging/qas new revision scripts
<wgrant> Yeah, they do
<wgrant> modules/launchpad/files/staging_restore.sh:    ssh launchpad@asuka "make -C /srv/staging.launchpad.net/staging/launchpad-new build combobuild LPCONFIG=staging" >> $LOGFILE 2>&1
<wgrant> modules/launchpad/templates/qastaging-update.erb:make combobuild LPCONFIG=qastaging
<StevenK> Right, so those can die
<StevenK> How do you want to proceed?
<wgrant> We need to think of anything else that might use build
<wgrant> And kill it
<wgrant> I think it's just babaco that needs fixing
<StevenK> Does it actually need anything from the built tree?
<wgrant> NAFAIK
<wgrant> It uses CSS from launchpad.net, I believe
<wgrant> Yeah
<wgrant> So I think it can just use compile
<wgrant> Which leaves only the frontends and staging using build
<wgrant> Which means we can merge combobuild into jsbuild
<StevenK> wgrant: I can destroy the combobuild target and move to jsbuild for my sprinkle-in-yui-391 branch if you wish
<wgrant> We'll want to leave combobuild as an alias for jsbuild for now, I guess
<wgrant> Until we remove references to it
<StevenK> That's one puppet branch only, though
<wgrant> Yes
<wgrant> But two phases
<wgrant> Mm, I guess not
<StevenK> And we probably need one anyway to add python-convoy
<wgrant> If we just deploy it at the right time
<StevenK> wgrant: I have a meta-lp-deps branch that promotes convoy
<StevenK> In fact that shows a problem with 0.120
<wgrant> StevenK: What's the problem?
<StevenK> -  python-egenix-mxtools, python-amqplib, python-gmpy, libpq-dev, unzip, lzma
<StevenK> -  ${misc:Depends}
<StevenK> +  python-egenix-mxtools, python-amqplib, python-gmpy, libpq-dev, unzip, lzma,
<wgrant> (and while we don't necessarily need convoy everywhere, it's so trivial that we might as well just install it everyhwere)
<StevenK> +  python-convoy, ${misc:Depends}
<wgrant> Oops
<StevenK> Haha
<wgrant> That was mine, wasn't it?
<StevenK> Yeah
<wgrant> But I bet misc:Depends expands to nothing
<wgrant> So it didn't actually matter
<StevenK> Yes, so it's fine
<StevenK> wgrant: So I should push up my death-of-combobuild commit to sprinkle-in-yui-391 and my meta-lp-deps branch?
<wgrant> StevenK: Please do
<wgrant> StevenK: That branch doesn't make 3.9.1 the default, right? Just adds it as an alternate version?
<StevenK> wgrant: Right
<StevenK> And fixes the 3.9.1 carnage in the JS
<wgrant> Right
<wgrant> Though that is safe back to 3.5
<wgrant> And possibly even 3.3
<StevenK> I checked 3.5.1, but 3.3.0
<StevenK> wgrant: Shall I just land meta-lp-deps?
<wgrant> 3.3.0 is dead, fortunately :)
<wgrant> Indeed
<StevenK> meta-lp-deps r149 pushed
<wgrant> StevenK: Is there an MP coming up?
<wgrant> sprinkle-in-yui-391 looks good, but I can't approve it :)
<StevenK> https://code.launchpad.net/~stevenk/launchpad/sprinkle-in-yui-391/+merge/157805
<StevenK> wgrant: Oh?
<wgrant> StevenK: Hm, what's with the test_choiceedit change?
<wgrant> That didn't fail on the last buildbot run
<StevenK> That was the failure
<wgrant> Won't that break on 3.5?
<StevenK> On 3.9.1
<wgrant> And only pass on 3.9?
 * StevenK tests again
<StevenK> wgrant: Failed tests:      0
<wgrant> StevenK: But how
<wgrant> StevenK: Are you sure your symlinks are correct?
<StevenK> lrwxrwxrwx 1 steven steven 9 Apr  9 14:56 build/js/yui -> yui-3.5.1
<StevenK> lib/canonical/launchpad/icing/yui -> ../../../../build/js/yui-3.5.1
<wgrant> Hmmmmmm
<StevenK> 3.5.1 fails without that change
<StevenK> Oddly
<wgrant> Maybe it's the CSS?
<wgrant> 'cause it fails for me *with* that change
 * StevenK runs make clean && make
<wgrant> clean_buildout is probably sufficient
<StevenK> Eh
<StevenK> Tests runnning
<wgrant> I'm looking at upgrading us to a more modern ZTK
<wgrant> Since ZTK 2 will be out soonish
<wgrant> And we're way behind
<StevenK> Failed tests:      0
<wgrant> StevenK: Still fails here with 3.5.1: Expected: inline-block (string)\nActual: inline (string)
<wgrant> And buildbot agrees with me
<StevenK> Very strange
<wgrant> StevenK: So, do you want to revert the inline-block change so we can get buildbot unbroken?
<StevenK> wgrant: I did
<StevenK> I just forgot to ping
<wgrant> Ah, great
<wgrant> StevenK: r=me, remember to [testfix] it
 * wgrant is disentangling ancient zope stuff
<StevenK> Aye, I have not forgotten
<StevenK> wgrant: I think almost all of our Zope stack is ancient
<wgrant> Well
<wgrant> One bit I just removed has been unused since 2005
<wgrant> We use so much deprecated stuff that I need to take an axe to a few things before I can upgrade to ZTK 1.1.5
<StevenK> wgrant: testfix is r16556
<wgrant> Once we're up to 1.1.5 future upgrades should be relatively painless
<wgrant> Thanks.
 * wgrant watches buildbot
<StevenK> Which one is ZTK in versions.cfg?
<wgrant> The main product of ZTK is the KGS
<wgrant> http://download.zope.org/zopetoolkit/index/1.1.5/ztk-versions.cfg
<wgrant> So ZTK 1.1.5 is just the aggregate of those version of the zope.* packages
<StevenK> Ah
<StevenK> zope.testing = 3.9.4-p17
<wgrant> So it's most of our Zope stack, except that we have a broken implementation of zope.app.apidoc which is deprecated
<StevenK> That's going to be fun
<wgrant> And probably doesn't work
<wgrant> So our Zope apidoc (not to be confused with the lazr.restful apidoc) will probably die tonight
<StevenK> wgrant: And no one will miss it ever.
<wgrant> I don't think it's worked in a couple of years
<wgrant> Since our last Zope upgrade
<wgrant> StevenK: One of the key changes in recent ZTK releases is that zope.app is basically dead
<wgrant> it
<wgrant> We still use bits of
<wgrant> But in most cases they're just deprecated imports that have been moved elsewhere
<lifeless> wgrant: StevenK: btw, new testrepository & subunit are out; you guys may want to consider upgrading at some point
<wgrant> lifeless: Is that with the v2 protocol?
<lifeless> yes
<lifeless> the testr data store is still v1, but the backend -> coordinator is v2
<lifeless> I'm flushing the last of the 'TestResult' API use within testr's internals
#launchpad-dev 2013-04-10
<lordadamson> hello
<lordadamson> anybody here?
<czajkowski> mgz: does this ring any bells with you https://bugs.launchpad.net/launchpad/+bug/1166854
<_mup_> Bug #1166854: bzr http URL stores evaluated URL instead of the link <Launchpad itself:New> < https://launchpad.net/bugs/1166854 >
<mgz> czajkowski: yeah, I'll look for a dupe. there were several fun things going on with that brokenness yesterday though.
<czajkowski> mgz: dont break things then ;p
<mgz> czajkowski: bug 569361
<_mup_> Bug #569361: Remembering an URL different from the one the user entered is surprising <Bazaar:Confirmed> < https://launchpad.net/bugs/569361 >
<czajkowski> mgz: so dupe that against the first one right
<mgz> well, I'm not really sure
<czajkowski> well it is now
<Laney> Is it possible to get the source archive of a copy, given a package_upload object?
<lifeless> wgrant: / StevenK: I think milestones have regressed ...
<lifeless> consider https://bugs.launchpad.net/testtools/+bug/1130429
<lifeless> and
<_mup_> Bug #1130429: ConcurrentTestSuite silently eats exceptions from run(result) <testtools:Fix Released by lifeless> < https://launchpad.net/bugs/1130429 >
<lifeless> oh, -weird- now the milestone is counting them
<lifeless> nvm
#launchpad-dev 2013-04-11
<adeuring> good morning
<gotwig> hey there
<gotwig> I wanna run launchpad on my own machine
<gotwig> when I run "make run" I get the info that it runs, but I dont know where it runs. I use a lcz (or whatever) container
<gotwig> http(s)://launchpad.dev does not work for me
<pindonga> hi there, I'm wondering if there is any way to get the openid identifiers back from LP via it's api
<pindonga> the need I have (for summit) is to map a list of lp usernames to their openids
<pindonga> but I can't seem to see any openids unless I'm logged in, and then I can only see the openid for the logged in user
<pindonga> just want to confirm if that's the way it goes, or if it's possible to have a privileged user that can see all openids
#launchpad-dev 2013-04-12
<StevenK> wgrant: lib/canonical/launchpad/icing/yui3-gallery -> ../../../lp/contrib/javascript/yui3-gallery ; but I think it's broken
<wgrant> StevenK: Yeah, that symlink is obsolete.
<StevenK> wgrant: So, I've been trying to get Y.PopupCalendar to work, and I think Y.Calendar is enough
<wgrant> :)
<StevenK> Healing: 26222 lines of code
<wgrant> yui2 is dead?
<StevenK> Yeah, that's removing it and the Makefile targets
<StevenK> And large poritions of lp.app.calendar
<wgrant> StevenK: You'll also be able to drop the two yui2 comboloader client config sections
<StevenK> wgrant: The event fires, the calender popups with the work I have. Things missing: It looks like rubbish, and is right over the text;  an X to hide the calendar; handling of time input as well
<StevenK> Clicking Choose will create a new calendar each time; the selection function either doesn't fire or doesn't work
<adeuring> good morning
<stub> adeuring, wgrant: Either of you respond to david re: translation coverage stats?
<adeuring> stub: I missed his mail. reading now...
<stub> Assuming it hasn't atrophied, this can just land and run with the --output= option right now.
<stub> And I think it is a minor change to get the librarian url reported rather than the file alias id
<cjwatson> I don't understand the point of process-pending-packagediffs being limited to processing 50 packages per run
<cjwatson> Wouldn't it always result in no worse and often better ETA if it always processed the entire queue?
<cjwatson> Possibly repeating the query at the end of the run to improve ETA for diff requests that arrived during processing
<cjwatson> The whole thing behaves pretty abysmally when a batch of language packs have just been uploaded
<stokachu> could someone help me understand why this api call times out: https://api.launchpad.net/1.0/ubuntu/+archive/primary?pocket=Release&ws.op=getPublishedBinaries&status=Published
<maxb> stokachu: Because you're trying to retrieve a huge dataset
<stokachu> the limit was set to 76 records
<stokachu> i think that was an automatic set as well
<maxb> Generating the dataset in order to find the first 76 probably didn't perform very well
<stokachu> ill try limiting to like 5
<maxb> Are you sure you don't want to specify a specific series or package?
<stokachu> ive been reading the api and that was my next step
<stokachu> ill narrow it down some more
#launchpad-dev 2014-04-13
<RoelV> !
#launchpad-dev 2015-04-06
<rabbitskynet> qweqwe
<rabbitskynet> e
#launchpad-dev 2015-04-07
<wgrant> cjwatson: Morning.
<wgrant> Oh, right, you're not here today.
#launchpad-dev 2015-04-08
<wgrant> cjwatson: Moved the LP meeting back to its traditional time for you, let me know if that isn't ideal.
<cjwatson> wgrant: Thanks, it'll do for now and we'll see for the next one in a couple of weeks
<cjwatson> Good holiday?
<wgrant> Yep, pretty good
<wgrant> Nice lunar eclipse on Saturday night, and perfect weather for it.
<cjwatson> Oh, shiny.
<cjwatson> Any talk of putting together a sprint agenda in last night's meeting?
<wgrant> It was discussed, indeed. I need to put my list in writing and then the three of us should compare notes.
<cjwatson> Righto.
<wgrant> Receiving objects:   0% (4588/4587073), 1.74 MiB | 6.00 KiB/s
<wgrant> My connection to Canonical is marvellous tonight.
<wgrant> Hmm
<wgrant> I wonder if we're exacerbating the LFP problem somehow.
<wgrant> cjwatson: Are the ddeb-retriever changes largely done except for that LP MP?
<cjwatson> wgrant: Yes
<wgrant> cjwatson: Excellent. Do I want to know how long it will take to catch up when a new series is initialised?
<cjwatson> wgrant: Not sure yet, can't do much realistic testing until the webservice extension is in place
<wgrant> cjwatson: Indeed, but we can guess.
<cjwatson> It won't re-download everything, but it'll have a whole bunch of BPPHs to look at
<wgrant> eg. does it skip a BPPH early if it sees it has that version already?
<cjwatson> It's not as early as it should be yet.
<cjwatson> At the moment I'm just going by whether the file exists in the pool to avoid having to stand up a separate database; I want to see how long it would take to get to that point when initing a new series before deciding how much to optimise that.
<cjwatson> Oh, blast, did I break turnip ...
<cjwatson> [2015-04-08 11:39:18,904: DEBUG/PoolWorker-2] "GET /repo/4/refs HTTP/1.1" 503 None
<cjwatson> probably :-/
<wgrant> Heh
<wgrant> That sounds like the WSGI app failed to start.
<wgrant> Which is odd.
<cjwatson> VersionConflict: (pygit2 0.22.0 (/usr/lib/python2.7/dist-packages), Requirement.parse('pygit2>=0.21.0,<0.22.0'))
<wgrant> If you check the name, version and arch of each BPPH as you see it and compare it to the pool, you should be able to throw away all the duplicates before making any O(bpphs) calls at all.
<cjwatson> Oi, turnip, I upgraded you
<cjwatson> Comparing to the pool requires getting at the source name.
<wgrant> That is a good point.
<cjwatson> Well, unless I map it through the already-published Packages.
<cjwatson> Which is a possibility.
<cjwatson> That's my next optimisation candidate.
<wgrant> cjwatson: YOu set requirements.txt, but I didn't see a setup.py change in the diff.
<cjwatson> Oh, blah, thanks
<wgrant> In fact, the change you made doesn't get used by the charm at all :P
<cjwatson> Didn't notice that
<wgrant> pygit2-requirements.txt is separate precisely so the charm can avoid it.
<wgrant> cjwatson: btw, not sure if you saw the Asana notifications, but the work to get merge diffs from pygit2 turns out to be 0, since merge_commits was added in 0.21.4 and merge_trees in 0.22.1.
<wgrant> So the feature was added a couple of weeks after I determined it didn't exist.
<cjwatson> Ah, I was also using the wrong mojo manifest, sigh.
<cjwatson> wgrant: I didn't notice that, thanks.  Glad that this upgrade work has value :-)
<cjwatson> turnip's better again.
<wgrant> cjwatson: unset SUDO_USER; Ctrl+R upg; is what I usually do.
<wgrant> Which manifest did you try?
<cjwatson> The default because I did C-r mojo rather than C-r upg.
<cjwatson> And didn't look closely enough :)
<wgrant> Heh
<wgrant> So just a very long no-op.
<cjwatson> Exactly.
<cjwatson> So noisy it's hard to notice.
<wgrant> The best thing about turnip's ssh frontend is that it doesn't invoke a new Launchpaddy Python process per startup, so checking updatedness is reasonably quick...
<wgrant> Rather than spending 4-7 seconds on top of the SSH negotiation.
<wgrant> blr: What's this Charm Framework thing?
<blr> wgrant: the new name for the services framework
<blr> wgrant: the api will most likely not be backwards compatible, we'll have to update our charms.
<wgrant> Ah, good.
<wgrant> Hopefully the API is less broken!
<blr> it does look better
#launchpad-dev 2015-04-09
<cjwatson> wgrant: Thanks, that looks like excellent advice on db-index-bpph-datecreated.  The COUNT(*) did actually come from a real query, but that's because I was using list() on the collection; not totally unreasonable since I wanted the whole collection anyway, but I'd forgotten that it concealed a race.
<cjwatson> (And yes, I know it's a silly time.  Fell asleep getting the children to bed and in a slightly odd state of sleepiness as a result; I'm going back to bed soon.)
<wgrant> Heh
<wgrant> cjwatson: The racy version probably wouldn't be too bad, but the fixes to make it non-racy should be pretty easy I think.
<cjwatson> I agree.
<cjwatson> Perhaps it makes sense to add a similar ordering parameter to getPublishedSources as well.  We don't need it in this case, but it's easy to imagine, and it's better to keep APIs symmetric where they don't need to be asymmetric.
<wgrant> cjwatson: I don't know, anyone using the API would probably be upset by such consistency or sense.
<wgrant> The LP API just wouldn't feel the same without smatterings of exact_match=False and the like.
<cjwatson> We should be applying PHP design patterns?
<wgrant> Precisely.
<cjwatson> Noted.
<cjwatson> I should probably Stormify getAPB too before my eyes bleed too much.
<wgrant> IIRC it has very few internal callsites, so it should be easy enough to port.
<cjwatson> Two.
<cjwatson> (One of which is annoyingly *almost* test-only.)
<cjwatson> Oh, well, two for _getBinaryPublishingBaseClauses, probably more for getAllPublishedBinaries.
<blr> pit y            console.log(Y.Node.DOM_EVENTS.key.eventDef.KEY_MAP);
<blr> err pity. Only 5 keys in that map
<blr> kind of loath to use the charCodes in tests, but adding a new KeyEvent map there seems redundant.
<blr> wgrant: anyhow, have some integration tests for nav, and found a bug in the process which was nice.
<blr> wgrant: afaict we don't have js unittests?
<wgrant> blr: There aren't many proper unit tests, no.
<blr> wgrant: a bit more noise in the diff now, I fixed some js lintiness.
<blr> wgrant: if we can modernised our js build process, we could consider adopting something like karma.
<blr> have used that in the past and it is rather good.
<blr> did you have a chance to look at babel?
<wgrant> It sounded too good to be true :P
<blr> hah right
<blr> wgrant: I have a friend at mozilla, I should ask him about it - they're using it apparently.
<blr> wgrant: transitioning to something like webpack/babel/gulp seems sensible, but I'm not certain how we do that while also supporting yui3. Suspect it is going to be messy.
<wgrant> I suspect it would be reasonably possible to use webpack to load YUI3, but it won't be totally trivial.
<blr> something to experiment with a bit later perhaps.
<wgrant> Yep.
<wgrant> tsk
<blr> wgrant: for some reason lp-land wants to uses a curses browser, but firefox is set as my x-www-browser alternative - have you seen that?
<wgrant> blr: You're not accidentally running it inside a container?
<blr> err... maybe? >.>
<blr> thanks
<wgrant> Heh, I do that all the time...
<cjwatson> wgrant: Hm, inserting StormRangeFactory here is less than trivial; all the uses to date have been in browser code, not webservice.  I think lazr.restful might need some extensions here - perhaps by adding a range_factory to ICollection?
<cjwatson> And maybe an optional parameter to export_as_webservice_collection
<wgrant> cjwatson: Hmm, you may well be right...
<wgrant> It's also possible that launchpadlib won't like it.
<wgrant> I remember we ran into this years ago, but we may not have solved it.
<cjwatson> I can start in on the lazr.restful changes, though I have the sense of a rabbit-hole
<wgrant> Quite.
<cjwatson> All I want is ddebs damnit
<wgrant> Still, most of the benefit can be gained without SRF
<cjwatson> We can't run it reliably without SRF, can we?
<wgrant> Because the traversal will be shallow with the collection sanely ordered.
<cjwatson> Because the memos will be wrong
<wgrant> Oh, yeah.
<cjwatson> Although the batch is only going to be shuffled backwards
<wgrant> Yeah, in this case it's probably safeish.
<wgrant> It can be shuffled the other direction if things get deleted.
<cjwatson> That is, the ordering means that the worst case is that elements 76-150 (say) in the batch were previously elements 75-149 or earlier
<wgrant> (which they often do, nowdaays)
<cjwatson> Oh?  Bah, good point.
<cjwatson> So I think I need to at least try to attack this.
<wgrant> The alternative, of course, is to let the status filter be omitted.
<cjwatson> We arguably want that anyway
<wgrant> Quite.
<cjwatson> For more reliable retracing, ddeb-retriever wants to catch up with everything, not just what's Published right now
<wgrant> Well, that will be achieved eventually with the retracers not using ddebs.u.c at all.
<cjwatson> Yes, but not for a while.
<wgrant> But I've long wanted a getPublishedBinaries that was designed to be used.
<cjwatson> How about I timebox this: I'll spend until end of day today seeing how far I get with inserting SRF
<wgrant> Works for me.
<cjwatson> If the madness continues I can give up and go for the stupid and duplicative approach.
<cjwatson> wgrant: Could I have PyPI access to lazr.batchnavigator, please, so that I can finish up bug 872086 while I'm buried in all of this anyway?
<mup> Bug #872086: Distribution:+ppas and Distribution:+questions issue unlimited queries when memo=0&direction=backwards <qa-ok> <regression> <timeout> <Launchpad itself:Fix Released by rvb> <lazr.batchnavigator:Fix Committed by rvb> <Storm:Fix Released by rvb> <https://launchpad.net/bugs/872086>
<cjwatson> wgrant: Maybe also lazr.restful
<wgrant> cjwatson: You're cjwatson?
<cjwatson> wgrant: Yes.
<wgrant> cjwatson: Done.
<cjwatson> Thanks
<cjwatson> wgrant: Looking harder, I'm not sure what you mean about the status filter.  It can be omitted already and you get publications with any status.
<cjwatson> wgrant: I think I have a first cut at the lazr.restful changes, although I'm sufficiently unfamiliar with the lazr.restful test suite that I won't be especially confident in them until I've tried hooking that up to LP.
<wgrant> cjwatson: Oh, forgot status was optional.
#launchpad-dev 2015-04-10
<cjwatson> wgrant: I'm giving up on the StormRangeFactory bit for now.  I do actually now have something that appears to work - haven't tried with launchpadlib, but the next_collection_link entries have elaborate-looking memos with full dates in them.  The tricky bit is that it breaks if you pass the "wrong" parameters, because e.g. BPN isn't in the resultset so SRF can't cope with a resultset ordered by BPN, and at the moment I'm inserting the ...
<cjwatson> ... range factory in the interface so it doesn't get to be parameter-sensitive.
<cjwatson> Maybe this is fixable with a DecoratedResultSet or something, but I've run past the end of my timebox for this.
<cjwatson> I'll just push up something today that implements order_by_date without that.
<wgrant> cjwatson: I think launchpadlib is a bit magical, since it supports slicing.
<wgrant> But yeah, SRF for the default sort order is awkward.
<wgrant> (and pointless, since it can't be sorted by an index)
<cjwatson> Ooh, I hadn't noticed launchpadlib.testing.launchpad before.
<cjwatson> Should probably convert cdimage to that.
<cjwatson> (It has a much less general version of the same idea)
<cjwatson> wgrant: Is binarypackagepublishinghistory__archive__datecreated__id__idx on dogfood yours?  It's not in database/schema/.
<cjwatson> wgrant: And I assume it would need to be created concurrently (and live)?
<stub> It doesn't exist on production
<stub> cjwatson: ^
<rgammon51_> I am up to date today on Ubuntu 15.04 with all updates applied
<rgammon51_> My issue is with nfs server
<rgammon51_> There are NO messages on the mount -a, exportfs, server restart
<rgammon51_> and showmount shows no messages - no exports
<jpds> rgammon51_: Surely this is a question for #ubuntu-server ?
<rgammon51_> No this is Desktop, not server
<mapreri> rgammon51_: then this is a question for #ubuntu, but surely not for this channel
<rgammon51_> I am an individual working on my own computrers, not for abusiness
<rgammon51_> launcpad, said this one
<mapreri> rgammon51_: this is for development of the launchpad service itself, not for project hosted there
<cjwatson> stub: thanks
<cjwatson> wgrant: I'm seriously tempted to not bother making a separate class for GitMergeProposal at all and just teach BranchMergeProposal to handle both, notwithstanding the confusing name.  It'd need to gain a few columns for source/target/dependent GitRef instead of Branch and for commit OIDs instead of revnos, and some complicated constraints to make sure that you can't mix-and-match, and the UI would need to use different macros to ...
<cjwatson> ... render revisions etc.  But it's that or clone-and-hacking a huge pile of both DDL and code, only to need similar branch-or-ref stuff in all the tables that refer to BMP; having started to write out all the DDL, I'm not sure the latter approach actually gains us much.
<cjwatson> That is, although we have an Asana task to split out a base class, I think we might be better off just having the same class handle both.
<cjwatson> I admit that widening 200000-odd BMP columns might be interesting.  I'm not sure that should force the design though.
<cjwatson> Maybe not a problem since the new columns wouldn't have defaults.
<cjwatson> wgrant: The decision to use (repository, path) as a natural primary key for GitRef makes it really inconvenient to reference it as an FK in other tables.  Maybe we should rearrange that to have a normal id.
<wgrant> cjwatson: BMP very explicitly must not use an FK to GitRef.
<wgrant> cjwatson: Since refs will be deleted, while MPs must not.
<wgrant> cjwatson: (oops, yes, that index was mine. got distracted writing that big reply so forgot to delete it)
<wgrant> cjwatson: Widening BMP isn't an issue, as it's a small table and reasonably sensibly bounded. And it's already very wide.
<wgrant> cjwatson: I'm all for reusing the table if you think it'll be more efficient. In fact I think it was you that convinced me not to, after your experience with not being able to share much code with your initial work.  So go ahead.
#launchpad-dev 2015-04-11
<cjwatson> wgrant: Good point about refs.  I'll have a think about what a decent model would be, but I guess it just has to involve two columns for everything ...
<wgrant> cjwatson: More than two, since it also needs to record the commit ID. But yeah.
#launchpad-dev 2016-04-11
<lblythen> hi. advice pls re. launchpad documentation team questions?
<lblythen> i joined the team, its mailing list's been inactive for five years
<lblythen> its home page says post questions to launchpad-dev mailing list.
<lblythen> that's been inactive since january and i'd have to apply for dev membership to post anyway.
<wgrant> ~launchpad-dev is an open team.
<lblythen> worth doing, or should i just ask doc-team questions here?
<wgrant> Or ask here.
<wgrant> We normally use IRC over mailing lists.
<lblythen> tks, might need to use dev mailing list as slightly complicated question :-)
#launchpad-dev 2016-04-17
<tsimonq2> I'm getting timeout errors when I try to submit a bug report: Error ID: OOPS-b5bac976dfc33190c19e492ca5ea9798
#launchpad-dev 2017-04-13
<artox_> Hello
<artox_> What is the guidelines for debian/changelog en upload it to launchpad? myApp (1.0-1) UNRELEASED; urgency=medium.
<artox_> must i put unstable instead of UNRELEASED?
<artox_> or something else?
<artox_> for upload it to launchpad*
<artox_> In other words: what should i put in debian/changelog for a first release? unstable or something else?
<artox_> If i'm not on the good irc just tell me :)
<artox_> irc channel*
<cjwatson> artox_: (#launchpad would be better, but anyway.)  That field should contain an Ubuntu series name, e.g. xenial.
<cjwatson> Or zesty or whatever.
<artox_> so if I want available for xenial and trusty I must upload 2 files?
<artox_> cjwatson: up
<artox_> Let's say for xenial, yakkety and zesty and futur releases
<artox_> Should I create a new deb and upload it every 6 months?
<artox_> Or just upload it with xenial field and it will be availlable for the futur releases?
<cjwatson> artox_: You can either upload one source package to each release, or you can upload to the oldest you want to support and use the "Copy packages" interface to copy it forward to other series once it's built and published.
<artox_> cjwatson: thank you :)
<cjwatson> See https://help.launchpad.net/Packaging/PPA/BuildingASourcePackage
<artox_> cjwatson: understood :)
#launchpad-dev 2018-04-11
<tsimonq2> cjwatson: Do you have any feedback in particular on bug 1763174 that I can use when I get to fixing it in less than a month?
<mup> Bug #1763174: Soyuz should document acceptions/rejections from the queue <Launchpad itself:New for tsimonq2> <https://launchpad.net/bugs/1763174>
<tsimonq2> e.g. pointers on where this is actually implemented, etc.
<xnox> tsimonq2, this must have been filed before.
<tsimonq2> xnox: I was told it hasn't.
<xnox> interesting. As long as I remember using Linux, infinity was ranting about that.
<tsimonq2> xnox: Well, I'm going to fix it.
<tsimonq2> And yes, Steve has been vocal about it as well. :
<tsimonq2> *:)
<xnox> go tsimonq2 go =)
<tsimonq2> \o/
#launchpad-dev 2018-04-12
<tsimonq2> xnox: You were right, Adam just marked it himself. :P
<cjwatson> tsimonq2: Difficult.  We have a part-completed thing called 'auditor' in the tree that was supposed to be for this kind of thing; but I've never looked at how far along it got, and whether it's still a sensible approach.  And it would probably require some deployment work if we stuck with that approach since IIRC it involves a separate Django app
<cjwatson> (which seems a kind of bizarre choice for LP, which is one reason I wonder if it's still the right approach)
<StevenK> Well, it was supposed to be a microservice because apparently they are the new hotness, but yes, I wonder that too.
<cjwatson> Yeah, I was more thinking of Django in particular being (nowadays; possibly not when you did it) not really the obvious choice for a microservice - it needs a fair bit of ongoing upkeep
<cjwatson> (also, hi)
<StevenK> cjwatson: Hi! Django was not even the right fit when I wrote the thing to begin with
<cjwatson> Heh
<cjwatson> I've been considering a little Flask app to be an API for codehosting to be consumed by LP
<cjwatson> Bit like the git API
<cjwatson> (though that's pyramid.  Gotta catch 'em all)
<StevenK> My only experience with Flask is for https://github.com/pjf/rickastley
<cjwatson> ...
<tsimonq2> cjwatson: OK. Would this take a particularly long time to get working if I wanted to take a shot at it?
<tsimonq2> (Probably, eh?)
<cjwatson> tsimonq2: Quite possibly?  I haven't looked at it much myself
<tsimonq2> cjwatson: OK.
<tsimonq2> cjwatson: I'll ping you when I'm at the preimplementation review. :)
<tsimonq2> Thanks.
<cjwatson> I'd at least consider dropping the microservice and folding it back into the main DB.  It was sort of meant to be a trial balloon I believe, but we have better examples of broken-out services these days (notably git) and it's not at all clear to me that the benefit will outweigh the operational cost of a whole separate service with a single table.
<StevenK> cjwatson: The point there was that it could be extended to be an audit log for everything, logins, password changes, etc etc
<cjwatson> Acknowledged, but I'm still not sure it exceeds the is-it-worth-it bar for me.  wgrant might disagree.
<StevenK> wgrant and lifeless were the ones who pushed me down the microservice rabbit hole :-P
<StevenK> Because yes, my original plan was another database table
<cjwatson> I certainly agree that a generic audit facility is worth it
<cjwatson> If we stick with auditor it'll need a reasonable amount of time from either William or me to get it actually deployed - I don't think that's something an external contributor can really do, at least not mostly
<tsimonq2> cjwatson: Ack. Let me know once that's taken care of (even if a bit down the line) and I can work on that bit, unless it's just easy to add.
#launchpad-dev 2018-04-15
<tsimonq2> What time does the usual publisher maintenance typically start?
<tsimonq2> I'm working on something in a PPA and I'm wondering how long I'll need to work on something else. :)
<wgrant> tsimonq2: 05:59Z
<tsimonq2> wgrant: That's UTC+/-0?
 * tsimonq2 googles
<tsimonq2> Ah, TIL.
<tsimonq2> Thanks.
