#launchpad-dev 2009-10-12
<lifeless> bug 449105
<ubot3`> Malone bug 449105 in launchpad "logging into sourceforge with openid breaks" [Undecided,New] https://launchpad.net/bugs/449105
<mup> Bug #449105: logging into sourceforge with openid breaks <Launchpad itself:New> <https://launchpad.net/bugs/449105>
<mup> Bug #449105: logging into sourceforge with openid breaks <Launchpad itself:New> <https://launchpad.net/bugs/449105>
<mup> Bug #449105: logging into sourceforge with openid breaks <Launchpad itself:New> <https://launchpad.net/bugs/449105>
<lifeless> oh joy
<lifeless> happy mup
<wgrant> lifeless: I think it's SF.net misbehaving.
<wgrant> lifeless: It's only half-respecting the delegation.
 * thumper away doing emails
<lifeless> wgrant: happens with apache2-mod-openid
<lifeless> wgrant: which I was trying to setup last week, looks exactly the same
<wgrant> lifeless: The proprietary Canonical fork, or an open one?
<lifeless> apt-get install
<wgrant> Aha.
 * wgrant checks the source.
<lifeless> wgrant: what proprietary canonical fork?
<wgrant> lifeless: https://launchpad.net/apache-openid
<lifeless> wgrant: looks to be all handed back to me
<lifeless> wgrant: oh, and I think its a different code base anyhow - there are *two* apache openid modules
<lifeless> yeah, thats mpopenid, not openid
<lifeless> wgrant: (mod python openid is what we use, apache2-mod-openid is written in C I think)
<wgrant> Ah.
<wgrant> The author of libopkele seems to have had a vendetta against whitespace.
<lifeless>  http://freshmeat.net/projects/libopkele/ ?
<wgrant> That one.
<lifeless> you're tracking down a root cause?
<wgrant> I am.
<lifeless> cool, thank you
 * mwhudson lunches
<wgrant> lifeless: c-i-p bug. It's not present in the copy that I of course didn't retrieve from lp:launchpad's history, because I'm not allowed to do that.
<wgrant> lifeless: All the declarations in the page are fine, but c-i-p now provides XRDS for OpenID 1.0 and 1.1, as well as 2.0 (which it always has)
<wgrant> The XRDS overrides the declarations on the page itself, and doesn't specify the identity to delegate to.
<wgrant> Just the server.
<lifeless> c-i-p ?
<lifeless> oh, canonical-identity-provider?
<lifeless> wgrant: I presume you'll put the details into my bug?
<wgrant> lifeless: Yep, doing so now.
<lifeless> thanks!
<wgrant> I was very confused for a while before I noticed the XRDS link.
<wgrant> Because identical delegation code worked fine elsewhere.
<lifeless> kfogel: http://www.pmease.com/features/screentour/
<lifeless> kfogel: /nice/ 'tour' presentation
<spm> wgrant: that buildd issue - *should* be sorted now
<wgrant> spm: Thanks. What was the issue?
<spm> firewall upgrade and subtle funkies thereof
<wgrant> Lovely.
<spm> excluding the ones I did - which always worked perfectly. natch - I've never known a firewall upgrade/change to go perfectly. ;-)
<spm> my mates at $job-1 went thru 3-5 iterations of pain trying to work with the gateway provider (as it so happens $job-2 :-) )
<spm> where each iteration is around 5-8 hours from midnight
<wgrant> Urgh.
<spm> and that was *just* the final firewall to the corporate network
<spm> not including the 4 or so DMZ ones just for their webserver farm
<spm> each of which has 13-17 active interfaces. \o/
<spm> I *so* don't miss working at $job-2 :-D
<adeuring> good morning
<mrevell> Morning!
<wgrant> bigjools: I need some hints on writing tests for the ddeb stuff.
<bigjools> heh
<bigjools> you mean you didn't write the tests first? :)
<wgrant> No, no, I am writing them first.
<wgrant> I'm not that crazy.
<bigjools> ok
<bigjools> you need to figure out all the places in the code where we're likely to need to keep a ddeb change in parallel
<bigjools> I hope this is mostly in BPPH
<wgrant> It should be.
<bigjools> and if not we should move code into BPPH
<bigjools> I suspect nascentupload will be a pig
<gmb> Does anyone mind if I set fire to checkwatches? No? Good.
<bigjools> and PackageUpload too
<bigjools> once you figure that out, write tests to match your expected behaviour, in fact you can probably just alter the existing ones on BPPH for copy, override etc
<wgrant> nascentupload gets a bit hard, as BPR is immutable but BPRs will now need to refer to other BPRs created in the same upload. I guess I'll need to create the DDEBs first.
<bigjools> hmmm interesting, what is making BPRs immutable?
<wgrant> DB and Zope permissions.
<wgrant> No user has more than SELECT+INSERT on BPR.
<wgrant> Which seems sane.
<bigjools> well we can change it if it makes sense
<wgrant> Actually, that can't be right.
<wgrant> Because queue overrides modify the BPR, don't they?
 * wgrant looks again.
<bigjools> nascentupload runs zopeless
* mrevell changed the topic of #launchpad-dev to: This is Launchpad Development Channel | Week 1 of 3.1.10 | PQM is OPEN | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in #launchpad-reviews | Use http://paste.ubuntu.com/ for pastes | This channel is logged: http://irclogs.ubuntu.com/
<wgrant> OK, so queued is the only real thing that can UPDATE BPR. I guess uploader could be allowed to.
<wgrant> But anyway.
<bigjools> wgrant: you probably won't need to
<bigjools> change the sql config I mean, the uploader won't have committed anything so you're only updating the storm copy
<wgrant> It will have flushed it.
<wgrant> Because storeInDatabase does queries of some kind.
<bigjools> bah
<wgrant> I've tested this; it is problematic in practice.
<bigjools> ok
<bigjools> fix security.cfg then
<wgrant> OK.
<wgrant> Now, the tests of new nascentupload functionality involve checking state at various points over a sequence of about 5 uploads. I think it might be neatest to have it all as one big doctest, although they seem to be losing favour...
<jml> hello.
<bigjools> yeah, I'm unsure of which is the best direction, but possibly a combo of both
<bigjools> hi jml
<bigjools> did you know there was a TV station called JML in the UK? :)
<jml> no, I didn't.
<jml> or maybe mrevell told me once
<jml> or twice
<bigjools> wgrant: celso was moving towards using more unit tests as they are usually more focused on the actual test conditions
<mrevell> bigjools: To my shame, I've told him all about the products of JML.
<mrevell> repeatedly
<mrevell> :)
<bigjools> wgrant: there is one doctest that's an end-to-end test though, but it's currently disabled
<bigjools> mrevell: ha :)
<wgrant> bigjools: Now, this nascentupload-supertest will also end up testing for correct supersedure and overriding in the course of what it needs to do. So it's not strictly nascentupload-only. Ew.
<bigjools> wgrant: that will be already tested somewhere (I forgot where)
<wgrant> bigjools: So I should integrate the more specific DDEB tests alongside the existing tests for those methods?
<bigjools> wgrant: yes, I think so, that would be the first place I'd expect to see them I think.
<mwhudson> jml: good morning
<jml> mwhudson, hello
<jml> mwhudson, I just sent you an email.
<mwhudson> jml: so you did
<mwhudson> jml: thanks for the mail
<jml> mwhudson, np.
* jml changed the topic of #launchpad-dev to: This is Launchpad Development Channel | Week 1 of 3.1.10 | I am Zero OOPS and So Can You! http://is.gd/4fkLl | PQM is OPEN | https://dev.launchpad.net/ | Get the code: https://dev.launchpad.net/Getting | On-call review in #launchpad-reviews | Use http://paste.ubuntu.com/ for pastes | This channel is logged: http://irclogs.ubuntu.com/
<bigjools> thumper: still up?
<wgrant> bigjools: I see you are working on a branch to allow customisation of the first PPA's name. Does it still default to 'ppa'?
<bigjools> no
<wgrant> bigjools: That seems like a bad idea. The display names that people come up with are bad enough.
<jml> gary_poster, good morning
<gary_poster> jml, good morning.
<jml> gmb, if I wanted to learn about how our external bug tracking system works and what it does & doesn't do, where'd be the best place for me to look?
<gmb> jml: Well, there's actually not all that much in-depth documentation for it (I've been meaning to write some). But there are some wiki pages that are relevant. Let me find them for you.
<gmb> jml: Also, if you want to hurt yourself. there's lib/lp/bugs/externalbugtracker/* and lib/lp/bugs/scripts/checkwatches.py. The latter is a bit of a mind-bender.
<jml> gmb, heh :)
<jml> gmb, maybe a good idea is for me to read what docs there are & have a chat w/ you sometime after.
<gmb> jml: Hmm, turns out that my dream of relevant wiki pages was optimistic... I can only find https://help.launchpad.net/FeatureHighlights/BugWatches, which says nothing useful.
<gmb> jml: Although, I did write this just the other day, which should tell you a bit about what's wrong with the current setup: https://dev.launchpad.net/Bugs/CheckwatchesNG
<jml> gmb, thanks.
<bigjools> wgrant: they are *Personal* PAs, and it can't be any worse than "ppa" ... !
<wgrant> bigjools: The old default ppa/'PPA for Some User' seems better than what people are coming up with now. There are no examples for what is appropriate.
<bigjools> why do you think it's necessary to do that though?  I'm not against it, just keen to know why.
<bigjools> as an aside, I want to change the interface description to make it obvious that the name appears in the URL and the description in the GPG key
<wgrant> I bet a lot of people are going to start naming their first PPA with their username.
<wgrant> Is the display name used in the OpenPGP key? I thought it was 'Launchpad PPA for $owner'... but I haven't checked the code.
<bigjools> from memory it is
<bigjools> but I've been wrong before :)
<wgrant> Urgh. You're right.
<wgrant> That sounds like a bug.
<wgrant> As it's now shared between all PPAs.
<bigjools> hmmm good point
<barry> gary_poster: ping
<gary_poster> barry: hey are you taking today off?
<barry> gary_poster: nope, swapping it.  how about yourself?
<gary_poster> barry: me too, cool!  salgado has today off though.  on call will ping you
<barry> gary_poster: np
<gary_poster> barry: yo, let's start
<gary_poster> barry, want to go to another channel?  #launchpad-sprint or something silly?
<barry> gary_poster: +1
<gary_poster> cool
<rockstar> barry, am I safe to assume there is no call today?
<barry> rockstar: yep
 * rockstar has been listening to smooth jazz for too long...
 * jml has been listening to _The Score_ for too long.
<barry> rockstar, abentley: https://code.edge.launchpad.net/~launchpad/launchpad/python-migration
<barry> rockstar, abentley i'm deleting that branch, so nm ;)
<rockstar> barry, okay.
<rockstar> barry, I think abentley is off on a national holiday today.
<barry> rockstar: oh right.  anyway the branch had a big warning that said just: MemoryError
<rockstar> barry, eep.
<barry> rockstar: might be reproducible by: branching off of stable; pushing w/no stacking (or with a --stacked-on that lp will deliberately ignore); wait for the puller to bomb
<rockstar> barry, yeah, I could see how that might be it.
<rockstar> I wonder if there's an open bug about that.
<leonardr> adeuring, i have a question about bug 402126
<mup> Bug #402126: top level publications must be public <lazr.restful:New> <https://launchpad.net/bugs/402126>
<ubot3`> Malone bug 402126 in lazr.restful "top level publications must be public" [Undecided,New] https://launchpad.net/bugs/402126
<mup> Bug #402126: top level publications must be public <lazr.restful:New> <https://launchpad.net/bugs/402126>
<mup> Bug #402126: top level publications must be public <lazr.restful:New> <https://launchpad.net/bugs/402126>
<adeuring> leonardr: yes?
<leonardr> presumably you tried to publish the hwdbapplication object as a top-level object, and it didn't work
<leonardr> what was the failure?
<adeuring> leonardr: I must admit that I can't remeber the details...
<adeuring> leonardr: give me some time to reproduce.
<leonardr> adeuring, sure
<leonardr> rockstar, i have a question about bug 326307
<mup> Bug #326307: Need @property equivalent for the API <api> <lazr.restful:Triaged> <https://launchpad.net/bugs/326307>
<ubot3`> Malone bug 326307 in lazr.restful "Need @property equivalent for the API" [Low,Triaged] https://launchpad.net/bugs/326307
<mup> Bug #326307: Need @property equivalent for the API <api> <lazr.restful:Triaged> <https://launchpad.net/bugs/326307>
<mup> Bug #326307: Need @property equivalent for the API <api> <lazr.restful:Triaged> <https://launchpad.net/bugs/326307>
<leonardr> what is the accessor method you'd like to publish as a property?
<leonardr> i'd like an example for a doc i'm writing
<rockstar> leonardr, looking.
<rockstar> leonardr, hm, it's been a while since I wanted that.  Lemme look around.
<rockstar> I'm sure I can find something.
<leonardr> rockstar, thanks
<rockstar> leonardr, how about IBranch.getPullURL exposed as IBranch.pull_url
<leonardr> ok, great
<adeuring> leonardr: I believe this patch re-enables HWDBApplication as a top-level publication (with access restriction): http://paste.ubuntu.com/291685/ Results in this error: http://paste.ubuntu.com/291687/
<leonardr> adeuring: can you just except link_name from launchpad.View protection?
<leonardr> it might be better not to show hwdb at all, but then it wouldn't show up in the wadl
<adeuring> leonardr: any hint for how to exempt link_name from the <require> rule? A simple addition of "allow attrinbutes="link_name" leads to the error "Failed to load application: Conflicting configuration actions"
<leonardr> adeuring: unfortunately the only way i've found to do that is to restrict every field *except* the one you want to allow
<leonardr> there's a slight chance gary will have a better idea
<adeuring> leonardr: that would be good. Defining the complete set of restscited attributes would not be very convenient...
<gary_poster> adeuring: no, you can either arrange the interfaces to suit your needs better, or define the set, I'm afraid.
<gary_poster> that's something we could theoretically improve.  if you wanted to put a bug into foundations I could see how much support there is for it.  describing what you want in a clear, compelling, and general way would probably help the sales pitch when I present it to others.  Think of it as trying to sell it to your team leads. :-)
<adeuring> gary_poster: OK; I'll add a comment to bug 402126
<ubot3`> Malone bug 402126 in lazr.restful "top level publications must be public" [Undecided,New] https://launchpad.net/bugs/402126
<mup> Bug #402126: top level publications must be public <lazr.restful:New> <https://launchpad.net/bugs/402126>
<mup> Bug #402126: top level publications must be public <lazr.restful:New> <https://launchpad.net/bugs/402126>
<mup> Bug #402126: top level publications must be public <lazr.restful:New> <https://launchpad.net/bugs/402126>
<gary_poster> heh, mup + ubot3` == excitement!
<gary_poster> adeuring: it's more of a problem in the zcml spelling
<adeuring> gary_poster: yes, maybe
<gary_poster> IOW, we would be redoing some zope bits to try to do what you want.  I think. :-)  eh, write up what you think you want and we'll see.  ;-)
<mrevell> Night all
<gary_poster> bac, are you approving CPs?  Is anyone to your knowledge?
<gary_poster> (and hi btw :-) )
<bac> gary_poster: i am not.  my RM powers expired a while back.  you'll have to talk to flacoste
<gary_poster> bac ok thanks.  I figured, but was worth a check.  (flacoste's out on nat. holiday afaik)
 * rockstar lunches
<maxb> barry: You appear to have removed all my commentary on why particular tests failed
<barry> maxb: dang.  sorry for the hamfisted wiki editing.  i'll try to revert that
<barry> maxb: btw, you are very welcome to join gary_poster and myself on #launchpad-sprints!
<maxb> s/sprints/sprint/
<barry> right, sorry.  and changes reverted
<EdwinGrubbs> barry: hi, can you mark this mp as reviewed so I can use "ec2 land" on it? https://code.edge.launchpad.net/~edwin-grubbs/launchpad/bug-430708-registry-windmill-layer/+merge/13148
<barry> EdwinGrubbs: done
<EdwinGrubbs> thanks
<rockstar> mwhudson, when you're around, I'd like to chat with you about upgrading stacked branches.
<mwhudson> rockstar: ok
<mwhudson> rockstar: i'm definitely not caffeinated enough for that yet, will let you know :)
<rockstar> mwhudson, okay.  I'm assuming we'll have to be on the phone together for it.
<thumper> rockstar: are you working today?
<rockstar> thumper, why wouldn't I be?
<thumper> rockstar: the email said it was a public holiday
<rockstar> thumper, what email?
<thumper> the staffing email
<rockstar> thumper, ah, well, I guess I should have read that then.  :)  Yes, I'm indeed working today.
 * thumper runs to drop off Maia
<mwhudson> i guess someone should look at the launchpad failures with bzr.dev at some point
<rockstar> mwhudson, yeah, I keep wondering that.  I get those emails and wonder if maybe one of use should respond about them.
<mwhudson> :)
<mwhudson> thumper did an accurate summary of the failures a while ago
<mwhudson> the part that's really sigh-inducing is the plugin api versioning nonsense
<thumper> mwhudson: skype?
<mwhudson> thumper: as usual i'm online
<mwhudson> thumper: did you try calling?
<wgrant> Intriguing. Breaking LP OpenID is Low, I see.
<lifeless> wgrant: ?
<wgrant> lifeless: The bug I diagnosed yesterday was marked as a duplicate, and the original became Low.
<thumper> rockstar: I guess we are done then
<rockstar> thumper, oops. I didn't mean to click the hang up button.
<thumper> :)
<thumper> np
<rockstar> Stupid touchpad sensitivity...
<lifeless> wgrant: meep :(
<lifeless> wgrant: bug number?
<lifeless> found it
 * mwhudson -> into town, back online in a bit
#launchpad-dev 2009-10-13
 * mwhudson is in that fun "run tests. edit security.cfg, rinse later repeat" cycle
<lifeless> <
<lifeless> jml: ping
<wgrant> noodles775: Won't your anti-buildd-monopolisation branch pretty effectively break start time and queue length estimations?
<wgrant> (it's also completely over-restrictive except for a few PPAs)
<adeuring> good morning
<noodles775> wgrant: yes - as per the comment on that bug, we'll need to update the estimations etc.
<noodles775> wgrant: and I've created bug 450124 for improvements...
<mup> Bug #450124: Improvements to IBuilder.findBuildCandidate() <Soyuz:New> <https://launchpad.net/bugs/450124>
<ubot3`> Malone bug 450124 in soyuz "Improvements to IBuilder.findBuildCandidate()" [Undecided,New] https://launchpad.net/bugs/450124
<mup> Bug #450124: Improvements to IBuilder.findBuildCandidate() <Soyuz:New> <https://launchpad.net/bugs/450124>
<mup> Bug #450124: Improvements to IBuilder.findBuildCandidate() <Soyuz:New> <https://launchpad.net/bugs/450124>
<noodles775> wgrant: so pls add any issues/improvements you can think of there (I've put one easy solution, but you might have better ideas :) ).
<stub> eek - double bots. Danger of feedback loops.
 * thumper shoots mup
<al-maisan> Good morning
<bigjools> thumper: yo
<thumper> bigjools: hey
<bigjools> thumper: call?
<thumper> bigjools: yo
 * thumper fires up skype
<bigjools> FAIL
<thumper> bigjools: first time I hadn't plugged in the head phones
<bigjools> thumper: ah yeah, skype doesn't like it when you plug them in when it's already running
<bigjools> bites me all the time
<mrevell> Morning!
<bigjools> thumper: so I can't find a blueprint for build from branch, I'll register one on Soyuz as a start since most of the work is there
<gary_poster> salgado: I'm hoping you'll join us on working on Python 2.5/2.6 on a sprint-like thing.  We're hanging out in #launchpad-sprint .  (Others are welcome too!)
<gary_poster> BjornT: hey.  You want to call?  I just got on Skype
<BjornT> gary_poster: yes, i'm starting skype now
<gary_poster> BjornT: cool
<BjornT> gary_poster: https://code.launchpad.net/~bjornt/lazr-js/buildoutification/+merge/13003
<BjornT> gary_poster: http://code.google.com/p/js-test-driver/
<barry> sinzui: network fail
<sinzui> I see
<barry> sinzui: try me again
<sinzui> barry:I cannot connected you
<barry> sinzui: restarting :(
<barry> sinzui: i guess we're done, eh? ;)
<sinzui> barry: are indeed done
<barry> sinzui: okie dokie
<barry> danilos: ping
<danilos> barry: hi
<barry> danilos: hi!  i think you're the only translations guy online atm.  do you have a few minutes to help me with something?
<danilos> barry: sure, I can give it a shot, though I might be way behind things after two weeks of sprinting
<danilos> (though, just imagine the wonders it did for my line :)
<barry> danilos: :).  what i'm looking for is the code in poexports that actually creates the tarfiles.  tests such as poexport-request-productseries.txt are failing in the python2.5 branch because the tar format changed.  i want to take a look at how export creates those tarfiles, but i'm having some trouble locating the exact code
<barry> danilos: actually looking for where files are being added to the tarfile
<barry> danilos: do we use python's tarfile module or do we shell out to tar to do that?
<danilos> barry: somewhere in lp.translations.utilities, let me check
<danilos> barry: we use tarfile module
<barry> danilos: cool, thanks
<danilos> barry: translation_export in there
<barry> danilos: ExportFileStorage?
<barry> TarballFileStorageStrategy?
<danilos> barry: LaunchpadWriteTarFile I think
<danilos> barry: add_file method if that's what you are looking for
<barry> danilos: got it.  thanks!  i think that's what i'm looking for
<barry> danilos: let me poke around in there.  i may return with more questions.  in the meantime.... thanks!
<danilos> barry: sure, you are welcome
<leonardr> adeuring, i have some more questions about the problem publishing IHWDBApplication through the web service, which i asked you about yesterday
<adeuring> leonardr: yes?
<leonardr> who has permission to see that object?
<adeuring> leonardr: a special team called hwdb-team
<leonardr> ok
<leonardr> is this internal to canonical?
<leonardr> i'm trying to figure out whether it makes sense to publish information about the hwdb object in the public api documentation
<adeuring> leonardr: it is internal to Canonical
<leonardr> adeuring: is this bug stopping those people from doing useful work?
<adeuring> leonardr: No; the point is that you need to know how to create the HWDB object "manually", via lp.load().
<leonardr> adeuring: i see
<adeuring> leonardr: so, it is more an inconvenience, since lp.hwdb does not work
<adeuring> (i mean, in scripts that use launchpadlib)
<leonardr> right
<leonardr> adeuring: is the same true of objects like h_w_device and h_w_device_class? is there no way for normal users to get to them?
<leonardr> because if so, they are cluttering up the api doc
<adeuring> leonardr: yes. Though I hink that we should open at least parts of the HWDB at some time. But the details are quite difficult...
<leonardr> i see
<leonardr> i've been telling people about the hwdb publishing as though it was something they could access. that's pretty emberassing
<adeuring> well, it was open for some time, but the was, erm, some demand to close it...
<leonardr> ok
<bigjools> anyone got any ideas why this might happen? http://pastebin.ubuntu.com/292507/
<mrevell> Night all
<jml> bigjools, no. however I will gladly rewrite said software in Twisted, if you can somehow wrangle me the time to do so :)
<bigjools> jml: oh *please*.... with lots of icing and cherries
<bigjools> actually I *think* Celso did some of that before he left, I need to poke through his unlanded branches
<jml> bigjools, time-wrangling is the trick.
<bigjools> jml: I'm sure it's only 30m work for a man of your calibre
<jml> bigjools, nothing that involves running tests in Launchpad takes less than 30m.
<bigjools> jml: it's ok poppy doesn't have tests :)
 * bigjools EODs
<bigjools> g'night
<jml> bigjools, g'night.
<mwhudson> good morning
<dobey> hey mwhudson
<lifeless> hey
<jml> hello
<thumper> hi jml
<mwhudson> hi jml
<mwhudson> jml: thanks for the review
<jml> np
<thumper> rockstar: skype ping
<rockstar> thumper, pong
<beuno> flacoste, intellectronica, http://www.456bereastreet.com/archive/200910/remove_the_outline_from_links_on_active_only/
<lifeless> hi jml
<lifeless> jml: whats your opinion about patches adding the extended test protocol I've been designing to testtools?
<jml> lifeless, unformed, generally favourable.
<mwhudson> jml: is there a bug for the branchingubuntu work?
<jml> mwhudson, I don't think so.
<mwhudson> or, gosh, maybe a blueprint!?!
 * mwhudson acquires ridiculous amounts of karma by using blueprints
<mwhudson> thumper, jml: does https://blueprints.edge.launchpad.net/launchpad-code/+spec/branching-ubuntu/ look ok?
<jml> mwhudson, what exactly am I looking for?
<mwhudson> jml: sanity
<jml> mwhudson, the blueprint metadata looks fine
<mwhudson> jml: conspicuously wrong things
<mwhudson> jml: ok thanks
<jml> mwhudson, I'm pretty sure I wrote the spec, so I'm happy to assume that it's relatively sane :)
<mwhudson> jml: heh
<rockstar> thumper, did you land the branch index windmill test?
<jml> mwhudson, actually, do you have a moment to talk about the code import stuff?
<thumper> rockstar: yes
<mwhudson> jml: sure
<jml> mwhudson, can you call my landline?
<mwhudson> jml: sec
<rockstar> thumper, it's failing on my machine looking for xpath='//div[contains(@class, "yui-ichoicelist-content")]'
<thumper> ??
 * thumper tries to run the test
<rockstar> thumper, http://pastebin.ubuntu.com/292652/
<thumper> flacoste: did we have a call?
<flacoste> thumper: we should
<simon-o> Hi, I asked this in launchpad-sprint before, but they are busy at the moment (the problem is related to the sprint). Does anyone know, when and where output elements are escaped?
<thumper> simon-o: which elements?
<thumper> flacoste: I just get your skype voicemail
<simon-o> thumper: links to feeds (<link rel="alternate" type="application/atom+xml" ...>)
<flacoste> thumper: that's because i needed to hang up with Gary first!
<flacoste> thumper: i'm free now!
<simon-o> this is the test case feeds/xx-links
<thumper> simon-o: what is failing exactly?
<wgrant> (note that this only fails with Python 2.5)
<simon-o> thumper: http://paste.ubuntu.com/292664/
<simon-o> wgrant: yes, it's only py2.5 but I trying to fix this, but don't know where to search
<wgrant> The escaping is still fine.
<wgrant> It just does it differently.
<wgrant> I'm not sure changing it just to match the tests is a good idea.
<simon-o> wgrant: ok I saw this while debugging. So this isn't related to py2.5
<wgrant> simon-o: Doesn't it only happen in 2.5?
<simon-o> wgrant: Sure, what I wanted to say was that this isn't a bug somewhere in python 2.5 it's just a changed behavior
<simon-o> :)
<wgrant> simon-o: Oh, right.
<mwhudson> hooray doctests
<mwhudson> simon-o: you could test that the unescaped output is the same?
<mwhudson> maybe
<simon-o> mwhudson: the test is just about correct escaping
<mwhudson> i see
<simon-o> and I think title='Announcements for Bad displayname"&gt;&lt;script&gt;alert("h4x0r")&lt;/script&gt;' is escaped correctly
<wgrant> Right - it just prefers to use different quotes rather than escape them.
<simon-o> wgrant: yes. now I would just be interested where exactly this happens (i.e. which method does the escaping)
<wgrant> simon-o: Whatever XML library TAL uses, I would guess.
<simon-o> wgrant, mwhudson, thumper: thanks, you helped me a lot
#launchpad-dev 2009-10-14
 * mwhudson shoves branching-ubuntu into ec2 land (at last!) and goes to lunch
<james_w> \o/
<james_w> thanks mwhudson
<mwhudson> james_w: you can see the diff at https://code.edge.launchpad.net/~mwhudson/launchpad/branching-ubuntu/+merge/13269 if you want
<mwhudson> james_w: shouldn't you be asleep?
<james_w> I should
<james_w> so I will look tomorrow
<mwhudson> cool
<mwhudson> thumper: didn't i worry at you about which zcml was executing in scripts vs the webapp?
<mwhudson> thumper: looking at the errorreportingutility lookup failures
<thumper> mwhudson: yes you did
<mwhudson> oh well, use globalErrorReportingUtility and don't worry over much i guess :/
<thumper> mwhudson: my branch adds the registration of the utility for scripts
<thumper> mwhudson: after talking with flacoste_afk
<mwhudson> thumper: ah cool, that's a better solution
<thumper> mwhudson: the code should be moved from c.l.webapp to lp.services.error
<thumper> but hey
<mwhudson> thumper: world hunger also needs fixing
<thumper> mwhudson: :)
<thumper> my desire to file bugs about ubuntu drops significantly as soon as things work again, even if intermittantly
<mwhudson> hmm
<mwhudson> the commit message area in https://code.edge.launchpad.net/~mwhudson/launchpad/branching-ubuntu/+merge/13269 looks seriously ugly
<thumper> mwhudson: yes it does
 * thumper has a cunning plan
<thumper> mwhudson: https://pastebin.canonical.com/23283/
<thumper> https://code.edge.launchpad.net/~andrea-bs/launchpadlib/fix-set_data_store/+merge/619
<thumper> has a null date review requested
<thumper> causing the oops
<thumper> now how the hell did that happen?
<mwhudson> thumper: that's really olkd
<mwhudson> old
<thumper> 619? yeah
<thumper> I wonder if it slipped through early before we knew what we were doing
<thumper> ...
<thumper> rejected the proposal and now we have a working active reviews again
<thumper> should look into that at some stage
<thumper> maybe add a DB constraint
<thumper> mwhudson: http://pastebin.ubuntu.com/292816/ ideas?
<mwhudson> thumper: apt-get install python-boto ??
<thumper> what is boto?
<mwhudson> like launchpadlib is to launchpad, boto is to amazon web services
<mwhudson> thumper: have you run ec2 on this laptop yet?
<thumper> just about to
<thumper> mwhudson: does land have --dry-run?
<mwhudson> thumper: yes
<thumper> heh, it tells me the mp is not approved :)
<thumper> if I approve it does that make it r=me or you?
<mwhudson> thumper: it looks at the votes
<mwhudson> thumper: finish autoapprove already :-)
<thumper> mwhudson: help me come up with the db schema and we can land it this cycle
 * thumper needs to copy across aws credentials
<mwhudson> sigh
<mwhudson> so it turns out that writing a simulator for a confusingly concurrent situation is, in and of itself, confusingly concurrent
<stub> Can mocker help?
<mwhudson> stub: several processes involved so maybe not
<mwhudson> hmmm straage, two successive calls to acquireBranchToPull are returning the same branch
<mwhudson> well, that's a bug to fix i guess
<mwhudson> argh
<mwhudson> there's no way to tell the difference between a branch which failed last time that the puller is currently working on and one that failed last time :(
<mwhudson> i can think of various stupid ways that you could encode this, but maybe we should just use jobs
<mwhudson> right *now* however, i'm going to the pub
<adeuring> good morning
<mrevell> morning
<wgrant> bigjools: I think I am attempting the impossible (establishing the matching publication for a deb's ddeb).
<bigjools> wgrant: it should be possible
<bigjools> you need to join via the BPR
<wgrant> bigjools: I know, but there is no way to identify which of the potentially several BPPHs matches (think overrides).
<bigjools> wgrant: the ddeb should not be overridden differently
<noodles775> wgrant: let me know if you've got any improvements for bug 450124
<mup> Bug #450124: Improvements to IBuilder.findBuildCandidate() <soyuz-build> <Soyuz:In Progress by michael.nelson> <https://launchpad.net/bugs/450124>
<ubot3`> Malone bug 450124 in soyuz "Improvements to IBuilder.findBuildCandidate()" [High,In progress] https://launchpad.net/bugs/450124
<mup> Bug #450124: Improvements to IBuilder.findBuildCandidate() <soyuz-build> <Soyuz:In Progress by michael.nelson> <https://launchpad.net/bugs/450124>
<mup> Bug #450124: Improvements to IBuilder.findBuildCandidate() <soyuz-build> <Soyuz:In Progress by michael.nelson> <https://launchpad.net/bugs/450124>
<bigjools> GAH
<wgrant> bigjools: Matching based on overrides sounds horribly fragile.
<bigjools> who keeps putting bots in here
<wgrant> And it's not been donebefore.
 * wgrant hunts out jussi01.
<bigjools> it's not an override, it's a matching publication in the same series, component and pocket
<wgrant> But there's nothing unique about that.
<bigjools> apart from the archive
<wgrant> I mean, I can easily have duplicate (bpr, archive, distroseries, pocket, component, section)
<jussi01> wgrant: just get one of your ops to quiet it for now is probably the best course of action
<jussi01> wgrant: I dont have access to ubot3, thats nalioth, but if you just quiet it, it sits here and isnt a problem
<wgrant> jussi01: ubot3 says you own it :(
<jussi01> wgrant: unfortunately thats because it syncs the factoids from ubottu
<wgrant> jussi01: Ahh.
<bigjools> wgrant: well yes any model code can screw up the DB, but if it's encapsulated properly and the rules are self-contained then I don't see a problem
<wgrant> bigjools: I have a deb and ddeb, in universe. I promote them to main, then demote them again.
<wgrant> Now there are three publications for each, but two are duplicates.
<wgrant> I now remove the deb, and it tries to drag along the ddeb.
<wgrant> But how does it know whihc publication is the right one?
<bigjools> wgrant: the one that's PUBLISHED
<wgrant> bigjools: But I did all three override changes semi-accidentally in a single publication cycle.
<bigjools> or the one with the most recent ID
<wgrant> (it has happened)
<bigjools> then you need to supersede old publications if you find inconsistencies
<wgrant> Inconsistencies?
<bigjools> gimme 10 I need to have a call
<wgrant> Sure.
<jml> hello
<maxb> Hi, has anyone else seen failures on    lib/canonical/launchpad/pagetests/webservice/xx-wadl.txt ?
<jml> hmm
<wgrant> That name is familiar. What sort of failure?
<maxb> wadl_schema.validate(wadl) returning False
<noodles775> maxb: passes here locally (r9691 on karmic)
<maxb> noodles775: How "interesting" :-)
<BjornT> maxb: the wadl file is generated automatically, so my guess is that something went wrong generating it
<jml> maxb, I get that failure when running the test on the stable branch.
<jml> updating download-cache & running 'make build'
<jml> still get the error
<bigjools> wgrant: ok back
<bigjools> so, you just need to take the publication with the most recent ID
<bigjools> the publisher should supersede the others
<wgrant> bigjools: Even if the latest one is no longer active? That sounds convenient.
<wgrant> Although still a bit ew.
<bigjools> yeah
<bigjools> which makes me think, you might want to check that the publisher will work with debug archives :)
<wgrant> Actually, this isn't safe.
<wgrant> deb_bpph.supersede() should also supersede the corresponding ddeb.
<bigjools> yes
<wgrant> This is to handle the case that a package ceases to produce a ddeb.
<wgrant> But what if it doesn't cease to produce it?
<wgrant> Hmm.
<wgrant> I guess it should be safe as long as we only look within the build.
<wgrant> Yeah, I guess that will work.
<bigjools> I don't quite follow this case
<wgrant> I wasn't thinking properly. There is no problem.
<bigjools> ok :)
<wgrant> It is guessing, but very safely.
<bigjools> if we assume the publisher will deal with old publications, taking the most recent one is always safe
<wgrant> Right. I had forgotten about other restrictions on the search.
<wgrant> Maybe it's not safe. How I hate guessing.
<wgrant> I have a deb and ddeb, both with Published publications. I promote them, giving them a new Pending publication each.
<wgrant> The publisher runs.
<wgrant> It sees that the deb is superseded, and calls deb_bpph.supersede()
<wgrant> The Pending ddeb publication screams and dies.
<wgrant> This is not correct behaviour.
<wgrant> s/deb_bpph/deb_old_bpph/
<jml> bigjools, are there docs anywhere describing how the buildmaster works?
<wgrant> Ha ha ha.
<wgrant> jml, this is Soyuz.
<bigjools> ROFL
<jml> fair point.
<noodles775> buildd-dispatching.txt ?
<jml> wgrant, do _you_ have docs describing how the buildmaster works?
<bigjools> thge buildmaster was oringally hacked up by the early LP devs (daniel etc) and more recently re-written by Celso in a hurry
<bigjools> yes, the doctest is the closest we have
<wgrant> jml: Although I know it internally fairly well now, I only have docs on how to run it.
<jml> yeah, I remember him being in a hurry. :)
<bigjools> when wasn't he? :)
<bigjools> jml we have a plan on how to generalise the buildmaster side but it was written before the re-write of the scanner
<wgrant> bigjools: Any ideas on the above example?
 * bigjools missed that, just a sec
<jml> bigjools, is the plan written down?
<wgrant> https://dev.launchpad.net/Soyuz/Specs/BuilddGeneralisation
<jml> I might have a poke around the code once I do today's CHR duties & finish off this damn imports branch and review al-maisan's branch.
<jml> I guess that means tomorrow.
<jml> wgrant, thanks.
<bigjools> wgrant: why would it scream and die?
<maxb>     <wadl:method name="PUT" id="HostedFile-put"/>
<maxb>     <wadl:method name="DELETE" id="HostedFile-put"/>
<maxb> ^ Well, oops :-)
<wgrant> bigjools: Because it has the latest ID, so gets superseded.
<bigjools> jml: the code is pretty bad, most of it was written before we had real coding standards.  Feel free to hassle me to explain stuff.
 * al-maisan takes a peek at lp:~jml/launchpad/upload-permission-joy
<bigjools> wgrant: ok I see
<bigjools> wgrant: if the last publication is pending then it should be clever enough to go to the previous one
<wgrant> bigjools: This is getting more and more into inaccurate guess territory, and sounds like it will go horribly wrong if there are multiple pending publications at once.
<jml> bigjools, I remember someone asking me for a generic sftp server based on the one we have in codehosting...
<maxb> I'd like one of those too :-)
<jml> bigjools, I guess that's for a different thing
<bigjools> jml: yeah we could do with replacing poppy
<jml> maxb, remind me to actually put license text in txsshserver one of these days.
<bigjools> wgrant: there is already code like this in the publisher
<maxb> jml: Your verification of the xx-wadl.txt failure was on jaunty/karmic? py2.4?
<bigjools> wgrant: I suggest we try and map out all of the operations that can happen like this, and work through them carefully
<jml> maxb, karmic.
<jml> maxb, ./bin/test -1cvvt xx-wadl.txt in stable branch
 * maxb wonders if this is a mysterious jaunty vs. karmic oddity
<jml> maxb, I'm deleting the unversioned files in lib/canonical/launchpad/apidoc and running the test again.
<maxb> jml: the wadl tested in the test is generated on the fly
<jml> maxb, ... as my experiment just demonstrated :)
<maxb> I'll have to dig out a jaunty system and retest there
 * bigjools laughs a lot at https://answers.launchpad.net/soyuz/+question/85773
<bigjools> jml --^
<maxb> How does the wadl generation actually work? There is stuff in the comments in the wadl which I don't find by grepping anywhere in the source?!
<jml> bigjools, heh heh.
<jml> bigjools, what's the blueprint URL?
<wgrant> maxb: Possibly stuff in lazr.restful(client)?
<bigjools> jml: I haven't done one yet, I will sort it out soon
<jml> bigjools, no worries.
<bigjools> jml: I cleaned up the buildd manager one yesterday as that's more important to me right now :)
<maxb> Hmm... looks like the bugged wadl is copied verbatim from wadl-root.py in lazr.restful
<jml> bigjools, ahh sure.
<jml> bigjools, I can just register the blueprint and link it to the already-written spec
<bigjools> jml: I can do it, it's on my list of things to do anyway
<jml> bigjools, ok.
 * jml would really love to scrub the launchpad-code blueprints at some point
<jmux> Hi - is there a good way to debug traversal errors? I'm getting a backtrace from base-layout-macros.pt for "json" and I didn't really find a way to find the original error.
<jml> hmm
<jml> jmux, nothing leaps to mind.
<jml> https://dev.launchpad.net/Debugging might help.
<maxb> eww. lazr.restful has no tags :-(
<james_w> does logger.exception in LP log sys.exc_info() or similar?
<jml> james_w, it really depends.
<james_w> 258	+ except:
<james_w> 259	+ ok = False
<james_w> 260	+ self.logger.exception(
<james_w> 261	+ "Unexpected error checking %s!", db_branch)
<jml> james_w, that behaviour comes standard in Python :)
<james_w> I'd like to know that this logs the exception, otherwise we are going to be scratching our heads when something goes wrong
<jml> james_w, http://www.python.org/doc/2.4.4/lib/node341.html
<james_w> ah, so it does
<wgrant> eg. try to catch an exception and call logging.exception("blah")
<jml> james_w, I'll need more context
<james_w> thanks, will be useful in future
<jml> james_w, because it's the log _handlers_ that really make the difference
<james_w> I'm looking at https://code.edge.launchpad.net/~mwhudson/launchpad/branching-ubuntu/+merge/13269
<james_w> def checkNewBranches(self):
<james_w> so in this case logger comes from LaunchpadScript
<jml> james_w, which in turn uses canonical.launchpad.scripts.logger.logger
<jml> james_w, which uses a default Python StreamHandler
<jml> which is fine.
<james_w> cool, thanks for checking
<jml> np
<jml> thanks for asking :)
<wgrant> bigjools: http://williamgrant.id.au/f/1/2009/ddeb-supersedure-cases.html describes most supersedure cases, I think.
<bigjools> wgrant: ok, thanks, I'll look through it in a bit
<jml> bigjools, the buildd-generalizing spec is for running arbitrary jobs, right
<bigjools> jml: yes
<jml> bigjools, is there another spec for generalizing across machine type? (physical machine, local vm, ec2 instance etc)
<bigjools> there's a bug
<bigjools> well
<bigjools> there's a bug for building any type of arch/virtual on the existing builders
<bigjools> there's no spec for going as far as what you say there
<jml> ok, thanks.
<bigjools> which I think is slightly crazy BTW :)
<jml> perhaps it is.
<jml> what I'm actually worried about is build capacity
<bigjools> yes, me too
<jml> s/worried/thinking/
<bigjools> we'll need a lot more builders
<jml> yeah.
<bigjools> we'll also need some more optimisation done to the buildd-manager
<bigjools> because finished builds are uploaded synchronously
<jml> oh right, I remember talking to cprov about that.
<bigjools> it's a very hard problem to solve
<elmo> err, why?
<bigjools> error conditions mainly
<bigjools> I forgot the exact details
<elmo> mmk
<elmo> I may be overlooking something, but Debian has done uploads asynchronously for over a decade
<jml> bigjools, ISTR it as more being a case of getting round to it.
<bigjools> well don't let me stop you fixing the buildd-manager then ;)
<elmo> it just means the extra section of the state machine where a package can be either 'uploaded' or 'installed' (i.e. the upload completed successfully) and you just have to make sure nothing stays in 'uploaded' for overly long
<elmo> also, why are we worried about capacity?
<bigjools> ah I remember more about this now
<jml> dammit. etsu lunch again.
<bigjools> the upload processor is single-threaded
<wgrant> But it's run in a subprocess.
<bigjools> with a lock
<wgrant> Oh. Right.
<bigjools> and no, it's not a subprocess when uploading binaries
<wgrant> Then why does the builddmaster have a config option for the command to run to upload the binaries?
<bigjools> NFI
<wgrant> It spawns process-upload.py.
<wgrant> This is why it takes /forever/.
<bigjools> hmmm last time I looked it didn't do that
<bigjools> but anyway, it's still has a bunch of issues that prevent it running more than one copy at once
<jml> I am full of unhappiness and empty of food.
<jml> brb.
<bigjools> and there are other issues with the buildd-manager just dumping the binary in an upload queue - for example how to deal with upload failures
<wgrant> bigjools: lp.buildmaster.buildergroup.BuilderGroup.buildStatus_OK certainly seems to spawn a subprocess.
<jml> bigjools, how do you deal with them now?
<bigjools> it marks the build as failed
<jml> bigjools, I see.
<bigjools> which the upload processor can't do, since it doesn't know about the build
<wgrant> It does, actually.
<jml> bigjools, you could write the builder asynchronously
<wgrant> The upload processor sets the build to Successfully Built, and attaches the binaries.
<wgrant> If it completes successfully, the buildmaster attaches the build log, sets the finish time, unassigns the builder, deletes the buildqueue.
<wgrant> If it fails, it attaches the upload log and marks the build as Failed to Upload.
<wgrant> But the upload processor does (and has to) know about the build.
<bigjools> wgrant: of course, you're right.  I think I need more coffee.
<sinzui> EdwinGrubbs: barry: bac: standup in two minutes
<EdwinGrubbs> sinzui: standup?
<barry> reviewers, interested developers, and lurkers: ameu reviewers meeting -> #launchpad-meeting in 5m
<barry> ameu reviewers meeting -> #launchpad-meeting
<barry> abentley: -> #launchpad-meeting.  i'm about to talk about pipelines :)
<Fox_1_> hi all
<Fox_1_> people do you know the place where I can get documentation about putting Launchpad on Debian
<Fox_1_> ?
<sinzui> Fox_1_: you are in unexplored territory. I am not aware is has been done, let alone documented
<sinzui> bac: ready for our call?
<bac> yes
<jml> leonardr, were you making a trusted client for the API?
<leonardr> jml: yes, it is on my list of things to do, but pretty far down the list
<leonardr> i can point you to the bug if you want to take over
<jml> leonardr, what if I don't want to take over but just want to leer speculatively, one day dreaming of having enough time to want to take over?
<leonardr> i think pointing you to the bug would still be appropriate
<leonardr> it might not be as difficult as you think, i did write the client, i just didn't write the tests for it
<jml> heh.
<jml> leonardr, in that case, please point me to the bug report.
<jml> I'm actually planning on working with launchpadlib on Canonical time during the Bazaar sprint in November.
<jml> so I might do trusted client work on the plane or something. maybe.
<leonardr> jml: https://bugs.launchpad.net/ubuntu/+source/ubuntu-dev-tools/+bug/387297
<mup> Bug #387297: manage-credentials should not ask for Launchpad password directly <ubuntu-dev-tools (Ubuntu):Triaged> <https://launchpad.net/bugs/387297>
<ubot3`> Malone bug 387297 in ubuntu-dev-tools "manage-credentials should not ask for Launchpad password directly" [Medium,Triaged]
<leonardr> https://code.launchpad.net/~leonardr/launchpadlib/trusted-client
<mup> Bug #387297: manage-credentials should not ask for Launchpad password directly <ubuntu-dev-tools (Ubuntu):Triaged> <https://launchpad.net/bugs/387297>
<jml> I thought we asked ubot3 to leave. :(
<jml> leonardr, thanks.
<intellectronica> bigjools: halp. i don't know anything about upload permissions. can you give me some pointers? do you know how the driver permission for a distroseries is calculated?
<bigjools> intellectronica: I don't know that, I only know about upload permissions
<jml> intellectronica, for a distroseries?
<jml> hm
<jml> wait. I'm focusing.
<intellectronica> jml: nm, i think i just found out
<jml> intellectronica, glad to hear it.
<mrevell> night all
<barry> danilos: ping
<Ursinha> barry: danilos is ill today, and it's pretty late for him, I don't think he'll answer in a reasonable time :)
<barry> Ursinha: okay, thanks.  danilos hope you feel better soon!
 * jgay attempts to install launchpad. 
<jgay> Does anybody know if I will likely get a successful build and install on intrepid?
<mwhudson> jgay: unlikely
<mwhudson> jgay: nothing fundamental, but some of the packages needed haven't been built for intrepid i think
<jgay> mwhudson: ok, thanks. I'll see how far I get before things start to breakdown.
<rockstar> mwhudson, abentley, who's hosting today?
<mwhudson> rockstar: good question!
<rockstar> mwhudson, would you like me to?
<abentley> rockstar: Go for it.
<mwhudson> rockstar: sure
<rockstar> The windmill tests don't run concurrently as threads, do they?
<barry> rockstar, mwhudson thumper's off line today.  do you still want to meet today?
<rockstar> barry, yeah, abentley said today was pretty exciting.
<barry> rockstar: okay cool.  i'll update the minutes and we can meet in 19m
<rockstar> barry, ack
<mwhudson> barry: ok
<barry> rockstar, mwhudson, wgrant  -> #launchpad-meeting
 * rockstar goes on a short walk
<wgrant> intellectronica: jml refactored the upload permission stuff recently, so it should be pretty easy to remove the duplicated logic from the bug nomination permissions.
#launchpad-dev 2009-10-15
<adeuring> good morning
<mrevell> Hi!
<wgrant> bigjools: Morning...
<bigjools> morning!
<wgrant> So, I found something in Dominator this morning which is indeed rather similar to the ddeb issue - atomic arch-indep domination, which I didn't really know existed before reading the code.
<bigjools> yeah I've not delved into that code
<wgrant> Turns out it suffer(s|ed) from the same problems I identified with the ddeb situation: bug #178102, bug #192547.
<mup> Bug #178102: (Pro|De)motion loses arch: all binaries <Soyuz:Fix Released by cprov> <https://launchpad.net/bugs/178102>
<mup> Bug #192547: Doubly-overridden binaries get eaten <Soyuz:Triaged> <https://launchpad.net/bugs/192547>
<bigjools> aha
<wgrant> (I reported both, but forgot about them, funnily enough)
<bigjools> :)
<wgrant> The first was fixed by doing what you suggested last night: matching component/section/priority.
<bigjools> oh jeez, see #lanchpad :)
<wgrant> Hahha.
<wgrant> The second was marked as a duplicate of another bug, but I do not understand how a fix for one fixes the other.
<bigjools> I'm just reading it
<wgrant> Presumably Celso had some idea, but there is no public record of that.
<bigjools> it's ok I know where he lives, I can kidnap his dog
<wgrant> Heh.
<bigjools> wgrant: the duped one could be un-duped and fixed as easily as you suggest
<bigjools> I don't quite understand why it's a dupe wither
<wgrant> bigjools: But I didn't suggest...
<bigjools> either
<bigjools> wgrant: in the description for it you suggest not allowing identical overrides
<wgrant> That was before I saw the code.
<wgrant> That's not feasible.
<bigjools> lol
<bigjools> ok I think I see why it's a dupe
<wgrant> In the initial case that I saw, the binary had been consecutively overriden identically twice.
<wgrant> But the same problem will occur even if there are intermediate overrides.
<wgrant> Oh, really? Please explain!
<wgrant> I considered that the override could instead mutate an existing Pending publication, which would probably fix this.
<wgrant> But that seems a bit evil.
<bigjools> yeah can't do that
<bigjools> or the publisher would have some issues
<wgrant> No, there are no publisher problems there.
<wgrant> But you seem to have a better solution.
<bigjools> well it would need to re-write indexes
<wgrant> If it's Pending, it hasn't written it out in the first place.
<bigjools> anyway I don't have a solution
<wgrant> Oh, right, you just had a reason for it to be a dupe. What is that?
<bigjools> if it's published, it has
<wgrant> If it's published, it won't cause itself to be dominated. => not a problem
<wgrant> Hmm.
<wgrant> Anyway. Why is it a dupe?
<bigjools> I am not particularly familiar with the code there, but you'd surely need to know where it was overridden from
<bigjools> I think it's to do with the fact that the whole overriding mechanism is a bit bong and has many race conditions
<bigjools> I'm still getting it straight in my head
<wgrant> If this problem hadn't popped up in ddeb territory too, I would be more likely to place the blame on the fairly strange way that arch-indep publications work.
<jml> wgrant, "should be pretty easy" -- famous last words :)
 * bigjools nods
<wgrant> jml: Pfft.
<wgrant> Admittedly I didn't see ddeb stuff getting even close to this hairy.
<simon-o> Hi, when I merge a branch is and push it. Is the status of the branch I merged from automatically set to merged?
<jml> simon-o, I think so.
<jml> simon-o, but I think it depends on the branch that was merged into.
<simon-o> jml: thanks, I guess it just takes some time
<jml> simon-o, only a few minutes, normally.
<BjornT> jml: it depends on into which branch you're merging into, no?
<jml> BjornT, yes, that's what I meant to say :)
<jml> also, we don't mark series branches as merged, since that turns out to be a real pain.
<BjornT> simon-o: which two branches did you merge?
<simon-o> BjornT: I merged lp:~simono/launchpad/xx-links into lp:~launchpad-dev/launchpad/python-migration
<simon-o> and the MP was set to merged and I wondered what happens to my branch now
<BjornT> simon-o: i suspect the status won't change. it's a bit odd, though, since the MP was set to merged, so i would file a bug about it.
<wgrant> I don't think it's safe to automatically change the branch status in that cas.e
<wgrant> Or I will start hiding everybody's branches by merging them into mine.
<jml> yeah.
<simon-o> wgrant: that's correct, but in which case can you change the status automatically? Because I never changed the status of my branches myself
<jml> the "Merged" status sucks.
<jml> simon-o, when they are merged into trunkp.
<BjornT> wgrant: why not? he said was intending to merge the branch.
<simon-o> jml: ok, that makes sense. And it explains why I never saw this before, because I nearly always merge into trunk
<wgrant> BjornT: Perhaps, but that may not be final. I think it's best to be conservative when making branches disappear.
<BjornT> wgrant: i said it was odd because there was an MP indicating the intention to merge. if you would merge my branch, without there being an MP to merge into your branch, we should definitely not change the status
<jml> simon-o, right :)
<jml> just to be clear
<jml> having a status of a branch called "Merge" is a misfeature.
<jml> "Merged", rather.
<BjornT> wgrant: why show the branch, if there are no new revisions in that branch?
<BjornT> jml: i agree that the branch status story needs a bit of a re-design
<wgrant> BjornT: define "new"
<BjornT> wgrant: revisions in A that aren't in branch B (where B is the branch that branch A was to be merged into)
<wgrant> Perhaps. It all needs a big rethink.
<noodles775> thumper: did you mention something recently about counting queries during tests? If so, where can I find it?
<noodles775> (sorry to disturb your evening if you're already gone!)
<jml> noodles775, it's in the base TestCasue
<jml> lp.testing.TestCase
<noodles775> jml: great, thanks.
<jml> noodles775, np.
<mwhudson> hello jml
<mwhudson> (and everyone else!)
<jml> mwhudson, hi
<jml> mwhudson, mwhudson, http://code.mumak.net/2009/10/command-line-apps.html
<jml> sinzui, I'm working on a script that maybe will make reviewing projects easier
<sinzui> Do you want mine? auto-approve does a lot of nice things
<jml> sinzui, http://pastebin.ubuntu.com/293935/ is the output of the first chunk of work: a nice display of useful bits of a project
<jml> sinzui, oh, yes please.
<jml> sinzui, I couldn't find it in lp-dev-utils.
<sinzui> It is very personal. I codified the rules I observed I used
<jml> oh.
<jml> sinzui, well that's good. I was planning on making something that was still manual, just less manual than the web ui.
<sinzui> jml: http://pastebin.ubuntu.com/293940/ <- The one aspect that is missing is announcements, They are not exposed in the UI, but I tend to approve when I see them.
<jml> sinzui, what's lpapi?
<sinzui> jml, the undecided license scenario should be automated in Launchpad because we really need to communicate with the owner that they can easily violate the terms of hosting if they write or choose a non-standard license.
<jml> sinzui, agreed.
<sinzui> jml: lpapi came from launchpadlib
<jml> sinzui, is projects.licensing_search restricted in some way?
<sinzui> It is the same search that is used on the review page?
<jml> sinzui, it's what auto-approve uses.
<jml> sinzui, I don't know if it's what the review page uses.
<sinzui> the page that lists the projects uses the license search.
<jml> sinzui, I'm getting a 401 Unauthorized error when I try to use it in APIs
<jml> does it require lp.Edit maybe?
<jml> ahhh, yes it does.
<jml> that's probably a bug.
<sinzui> it certainly is if you can use the page, but not the api
<jml> sinzui, well, I can use the API, but I need to grant my application "Change" permissions on the oauth page.
<jml> sinzui, I shouldn't have to do that to run a search.
<sinzui> hmm
<sinzui> That script I sent does lots of changes.
<jml> sinzui, yes, but me doing a search shouldn't require any changes
<sinzui> agreed. I do not think that feature reveals confidential information, but that may need to be vetted
<jml> sinzui, even so...
<jml> sinzui, that's not what launchpad.Edit is for.
<sinzui> jml: consider this feature was for a very restricted team of two users until recently
<sinzui> I am sure we want to review the rules given that we have opened it up to more users
<jml> sinzui, fair enough.
<jml> sinzui, the script I've written downloads the entire list of projects to review, and then lets you select '(s)kip (a)pprove (q)uit (d)one'
<sinzui> sweet.
<jml> sinzui, and then, when you are done, it will (not done yet) submit all of the changes to Launchpad
<sinzui> Does it send emails? yet?
<jml> sinzui, I'm assuming that the "Approve" action is just "mark approved, mark reviewed, clear whiteboard"
<jml> sinzui, do we need to send emails for approval?
<sinzui> No
<jml> sinzui, ok. then we'll worry about that when I add a '(r)eject' option :)
<sinzui> Nor do I think we should send emails to feedback if we want the user to provide more information, but we approved it anyway...because we do not intend to followup wit that
<jml> sinzui, yeah. there should be a more info action too.
<matsubara> gary_poster, rockstar, bigjools, danilos, sinzui, allenap, Chex: LP production meeting in 14 min @ #launchpad-meeting
<gary_poster> ty
<danilos> matsubara: if I must, sure :)
<matsubara> danilos, you can send someone else, if the timing is not good for you
<danilos> matsubara: nah, everybody else (apart from Ursula and me) is on vacation, I am just not feeling too good, but IRC meeting should be something I can handle
<matsubara> danilos, ok
<jml> beuno, sinzui, is there any particular reason that we didn't put the series / milestone graph on distros?
<sinzui> jml: development time
<beuno> jml, IIRC, missing API
<sinzui> beuno: API + code
<jml> what we like to call "work" :)
<beuno> yes, missing work
<sinzui> jml: we did not consciously reject it. it is a matter of thousands of project verses 6 distros
<jml> sinzui, I guess this means that distros & projects present substantially different interfaces re series & milestones?
 * sinzui thinks more code needs to move into PillarMixin and SeriesMixin
<jml> sinzui, well, I was thinking more that there should be an ISeries interface that the graph could be written in terms of
<sinzui> jml ^ Not at all, We need to dismantle the impelmentation into their common parts
<jml> hmm.
<jml> sinzui, when you implement a series- or milestone-related feature now, do you in general have to do the work twice: once for projects, once for distros?
<sinzui> jml: I did not focus on distros, but distros, projects, and series all use the same milestone mixin to implement IHadMilestone
<jml> ok.
<sinzui> jml: I think you need to keep in mind that these very early objects need more than love, sometimes bombs are needed to remove the cruft. Once the team faltered in February, we focused on the greatest need
<jml> sinzui, *nod*
<jml> sinzui, my questions were aimed at figuring out whether we should allocate time to cleaning up the model here
<sinzui> jml: They really do need to be dismantled to bring consistency. There are blocking issues too, like I cannot make compatible urls for distros, projects, and projectgroups without very invasive work
<jml> sinzui, I am interested in your ideas and wish to subscribe to your newsletter.
<sinzui> jml: 15% of registry bugs about removing unused features, or reconcilling behaviour
<sinzui> jml: Removing the cruft from the models will make launchpad easier to learn because it is consistent. We are maintaining this cruft too.
<jml> sinzui, easier to learn is good, but it's not quite as interesting to me as "faster to develop" or "harder to miss things"
<jml> sinzui, although they aren't exactly mutually exclusive :)
<sinzui> jml: my team was create to think about this. It did take me a year to digest every bug and touch all the code to see the big problem. It can also be challenging since these are core object that other teams are making assumptions about
<jml> sinzui, I'll bet! :)
<jml> sinzui, I imagine that sometimes the other areas of the app are making crappy assumptions.
<jml> sinzui, do you have a roadmap for cleaning up the registry, or is it mostly in your head still?
<sinzui> yes. but there is still the case were one team does a fab implementation for one pillar, but does not consider that it can be applied to all pillars
<sinzui> No map. My hopes were dashed in March. I really thought the series and project work would let me fix the models
<jml> *nod*
<sinzui> The bugs are tagged with cleanup, tech-debt, infrastructure.  I can get you a report (using tags) for next week
<jml> sinzui, ok. no rush, but it'd be nice to have by UDS.
<jml> sinzui, regarding good implementations for one pillar, but not for others
<sinzui> for some reason these core bugs keep showing up in my searches. maybe that is why I think they are core
<jml> sinzui, that's a problem we really need to solve exactly once.
<sinzui> yep
<jml> sinzui, where we = you, BjornT & I and whoever else is interested
<bigjools> sinzui: so where did you get to with the script failure?
<sinzui> bigjools: I am subscribed to the list.
<bigjools> what list?
<sinzui> My filter is on nightly and on (Daemon|script-failures)
<sinzui> bigjools: I cannot find how I am getting the email. I recall subscribing to a list and configuring to to send only certain classes of emails
<bigjools> isn't it sent to the internal list?
<mrevell> okay, g'night friends
<sinzui> barry, bac, EdwinGrubbs: release meeting in 2 minutes
<dobey> hrmm
<dobey> kfogel: ping
<barry> thumper, mwhudson ping
<mwhudson> barry: good morning
<barry> mwhudson: hi.  do you have a few minutes to join a discussion going on in #launchpad-sprint?
<barry> we're working on the py2.4 -> 2.5 upgrade and there are soem questions about a codehosing test
<barry> er, code hosting
<barry> :)
<mwhudson> jml: did you get my twisted talk slides?
<EdwinGrubbs> sinzui: ping
<sinzui> Hi EdwinGrubbs
<mwhudson> rockstar, abentley: i feel like i'm lagging horribly on this call
<mwhudson> back to os x tomorrow then i guess
<EdwinGrubbs> sinzui: I was thinking about removing the "PPA Packages" link from the person/team index page since there is already a link next to it to the "Related projects" page, and that page has a menu with a link to the "PPA Packages".
<sinzui> hmm
<sinzui> There are two problems here isn't there? One link is to *related* PPAs which I do not own and *Owned* PPAs
<EdwinGrubbs> sinzui: correct. The related PPAs are ones that contain a package that was created by that person, although it may have been copied into that PPA by someone else.
<sinzui> yes, so we need to keep those links distinct. I would prefer to remove all *related* paages since no one can say what they are for
<EdwinGrubbs> sinzui: that sounds like a soyuz bug to me.
<sinzui> EdwinGrubbs: they say we own it, which is why I think I have the right to remove them
<bigjools> sinzui: I
<bigjools> 'm sure I told you a few times
<sinzui> EdwinGrubbs: https://edge.launchpad.net/~launchpad <- the links under kiko need to read
<sinzui>     (i) Related project  (i) Related PPA packages
<sinzui> bigjools: You have said that user can see packages and software. But since there are no actions that can be taken from that information I do not value it
<bigjools> sinzui: I said that the MOTUs use it to review new MOTU applicants' work
<sinzui> bigjools: https://edge.launchpad.net/~sinzui/+related-software
<sinzui> ^ is a bug fat lie and I do not want anyone to every see that lie
<sinzui> bigjools: so do we need to see projects and PPAs? Packages are the motu work?
<sinzui> bigjools: And what is the action item? Add this user to the motu team?
<bigjools> yes
<bigjools> or not add
<bigjools> but I suggest you talk to wgrant about it as he's our MOTU contact
<sinzui> Surprisingly I did not get a persuasive reason.
<bigjools> and dare I say it, ScottK
<sinzui> Given that wgrants opinions are viable/doable items, I took the ambiguity to  mean it is not important
<bigjools> what is it you want to remove exactly?
<sinzui> +uploaded-packages,  +ppa-packages, +related-projects because they seem to guess wrong most of the time about what is truely related
<sinzui> If I had a real story/use case, I think I would create an very different feature from how these fail to operate
<bigjools> ok well my advice is to talk to more people first, since I had some very irate users in the past complaining about the performance of those pages
<sinzui> EdwinGrubbs: I do not think you can trust the related portlet to always display. I only appears if a team owns a project, but a team can manage packages and PPAs with out ever owning a project
<EdwinGrubbs> sinzui: I was referring to the "Related projects" link and not the portlet. According to the sourcecode, the "Related projects" link is always displayed above the yui grid.
<sinzui> That is true
<EdwinGrubbs> sinzui: do you want me to track down the people involved or just send out an email to launchpad-dev?
<sinzui> EdwinGrubbs: Fixing those pages are out of scope for this release. I think packaging is in scope for 3.1 and we should fix that feature
<EdwinGrubbs> sinzui: are you talking about packaging as in the planning meeting we just had or as in the "PPA packages" link?
<sinzui> EdwinGrubbs: be mindful when we remove links or information from a page because there is no information, users then file bugs that they spent 2 hours searching launchpad for the information. When I reply that is because there was nothing to be found, the user states if we knew that we should have made that clear before he wasted 2 valuable hours of his life.
<sinzui> EdwinGrubbs: You are working on a UI bug and we need to keep the scope at that. We start new work next release
<EdwinGrubbs> sinzui: it sounds like you are saying, "don't spend a lot of time on it, but don't just remove the link." This brings us back to your suggestion to use "Related PPA packages", which sounds fine to me, but the conversation after that confused me.
<sinzui> sorry
<sinzui> Edwin: there are two issue with the bug. 1) two links with the same name. we need to fix the name of +ppa-packages. The other +archive/ppa should not be shown if there the user does not own a PPA
<sinzui> EdwinGrubbs: and the +archive/ppa can be empty
<sinzui> EdwinGrubbs: we might able to remove all but the related software links from person and teams. We rename the single link (i) Related software and packages.
<jml> mwhudson, I did.
<mwhudson> jml: great
<jml> mwhudson, I've replied to exactly three emails that required thinking today.
<EdwinGrubbs> sinzui: the +archive/ppa link only appears if that ppa exists. Compare /~launchpad and /~launchpad-dev. An empty PPA needs to be listed just like an empty series does. +1 on the "Related software and packages" link.
<mwhudson> jml: :)
<mwhudson> jml: no real hurry
<mwhudson> well, the conference guy wants slides by monday, but i have a long and proud history of ignoring deadlines like that
<sinzui> EdwinGrubbs: I am sure we are showing all the links to avoid changing lots of tests that test their presence from the old nav menu. Feel free to delete the redundant tests
<EdwinGrubbs> ok, cool
<wgrant> sinzui, EdwinGrubbs: +uploaded-packages is critical to Ubuntu developer review workflows.
<sinzui> wgrant: great how is it used what should link to it and what should it link to or actions it should contain
<EdwinGrubbs> wgrant: our current plan is just to have a single link to +related-software, and that page will keep its links to +uploaded-packages, +ppa-packages, and +related-projects.
<wgrant> sinzui: It just needs to be somehow accessible, and needs to contain links to all SPRs in the primary archive where the person is mentioned in Changed-By.
<wgrant> So exactly what it does now.
<sinzui> Why do you want to see it?
<wgrant> So that the MOTU Council (or new Developer Membership Board) can see what a prospective developer has done.
<sinzui> wgrant: Fab
<sinzui> Do you use the ppa or releated software views?
<wgrant> PPA occasionally.
<wgrant> But projects are not relevant, and usually incorrect.
<sinzui> wgrant: I think these pages might be better if they contained information about motu with links about how to become a motu.
<wgrant> sinzui: I don't think so.
<wgrant> That's not the sort of thing LP does.
<sinzui> maybe it should. Launchpad has packaging and package linking features that do a poor job of explaining their value. Launchpad could do  a better job mobilising users and getting bug and code upstream
<sinzui> maybe the person and team page should include latest packages uploaded like it should include latest branches and merge proposals
<wgrant> That would indeed make sense.
<sinzui> The distroseries page is not what I wanted to do. The top of the right column should be something that needs to be done or can be done right now to contribute. I think  linking package to upstream might be what belongs there.
<sinzui> I wonder how much labour is needed to repurpose the sparkline code to show other activities link package uploads, linking, bug fixes.
<mwhudson> !!
<mwhudson> no gpg-agent in karmic?
<maxb> ?
<mwhudson> well, not installed any more it seems
<sinzui> mwhudson: I had some issues with it a few days ago. I need a good kick. bac also had issues with it
#launchpad-dev 2009-10-16
<EdwinGrubbs> wgrant: ping
<wgrant> EdwinGrubbs: Hi.
<EdwinGrubbs> wgrant: hi, I was wondering if you had any opinion about hiding the links at the top of the +related-software page if they are empty. There would always be a link on the person/team page pointing to +related-software, but the +uploaded-packages, +maintained-packages, +ppa-packages, and +related-projects links would be hidden if they were empty.
 * mwhudson tries to pound the puller into breaking locally
<spm> mwhudson: :-)
<mwhudson> completely unsuccessfully so far :(
<wgrant> EdwinGrubbs: That's how I imagined it was intended to be.
<spm> mwhudson: "if it was easy, we wouldn't have given it to you" <== story of my life and one you could take to heart as well ;-)
<wgrant> What is the normal slave replication lag, and what is the maximum before requests all go to the masteR?
<kfogel> dobey: pong
<spm> wgrant: bugger and all; don't recall offhand - but not that long. a few mins max.
<wgrant> spm: A few minutes is enormous...
<spm> tradeoffs. most folk aren't modifying.
<wgrant> API requests will now use a slave where possible. This is going to cause a lot of clients to get very angry.
<gary_poster> wgrant: I don't think that is the case.  What makes you think that?
<wgrant> gary_poster: See devel r9703
<gary_poster> I think I know what that is, but double-checking.
<wgrant> Oh. session cookie only.
<wgrant> I guess that's less unreasonable.
<gary_poster> right.  this is for JS browser users.
<mwhudson> wgrant: spm is wrong btw, lag is usually 5-10 seconds
<gary_poster> mwhudson: hi.  I have a bzr plugin problem with eggs in the branch where I'm trying to apply voodoo to make the build happy on karmic.  I've "fixed" the horrible namespace package problem.
<gary_poster> Now I only have two failures, both for the same reason. test_successful_start_then_stop and test_unexpected_error_logs_oops fail because they start up bzr, which tries to load plugins from all possible places including the system's site-packages, even though I'm not using the system 's bzr.
<gary_poster> If you have plugins installed then bzr complains, because they cannot be imported.  This causes test failures in assertFinishedCleanly: return code is 0, yes; stdout is empty, yes; but stderr is not empty: in my case, it is the following string:
<gary_poster> "No module named dbus.bus\nUnable to load plugin 'dbus' from '/usr/lib/python2.4/site-packages/bzrlib/plugins'\nNo module named dbus.bus\nUnable to load plugin 'avahi' from '/usr/lib/python2.4/site-packages/bzrlib/plugins'\n"
<gary_poster> I have stepped through the bzr code and I do not see a way to tell the bzr commandline to please only use the plugin path I'm providing, and don't look anywhere else, especially not in the system site-packages.
<gary_poster> Therefore, what I'm planning on doing is changing assertFinishCleanly to check that the return code is 0; check that the stdout is empty; and check that stderr is entirely comprised of 0 or more of those plugin error messages, using a regex.
<gary_poster> Can you get behind that?  The only other option I can see is making another LP-specific bzr egg change, which I am loathe to do.
<gary_poster> (done)
<spm> mwhudson: heh. that's a variation of definition. 5-10 secs to me is bugger and all. :-)
<mwhudson> gary_poster: where are these tests?
<wgrant> mwhudson: It's sometimes a lot more, though.
<mwhudson> spm: you said "a few minutes max" -- it doesn't get as high as a minute very often though does it?
<gary_poster> mwhudson: lp.codehosting.tests.test_lpserve.TestLaunchpadServe
<mwhudson> wgrant: generally when something breaks, i think
<mwhudson> gary_poster: looking
<gary_poster> thank you
<spm> mwhudson: very rarely - is something critical that we alert on.
<mwhudson> gary_poster: ararargh!
<gary_poster> mwhudson: well, yes
<mwhudson> gary_poster: i guess your solution is ok
<mwhudson> gary_poster: so long as there's a big fat XXX pointing at a bzr bug i'll find in a sec
<gary_poster> mwhudson: heh, that sort of XXX would make me feel better, yes.
<mwhudson> wth it's Fix Released
<mwhudson> oh, but very recently
<mwhudson> gary_poster: i think your patch is ok then, we should remove the hack when we upgrade bzr
<mwhudson> gary_poster: https://bugs.edge.launchpad.net/bzr/+bug/316192 fwiw
<mup> Bug #316192: bzrlib.plugin.load_plugins() loads system plugins even for non-system bzrlib <Bazaar:Fix Released by vila> <Launchpad Bazaar Integration:Fix Released by thumper> <https://launchpad.net/bugs/316192>
<gary_poster> mwhudson: awesome that is fixed.  OK, thank you very much.
<mwhudson> gary_poster: np
<mwhudson> spm:  has u1 stopped exploding yet>
<mwhudson> ?
<wgrant> It's working enough for me to have discovered 5 XSS vulnerabilities in the past hour.
<wgrant> (I am not amused)
<mwhudson> :(
<spm> mwhudson: sorta. in that the immediate stuff is now under control; now I move to the ZMOG 9999 priority stuff. I can do small simple things; else... busy :-/
<mwhudson> spm: ok
<mwhudson> spm: btw the thing i want to talk to you about is one of those things where priority -> \infty as we get closer to karmic being released :-)
<spm> mwhudson: heh. figures.
 * mwhudson updates his puller bashing to be less wasteful
<spm> noodles775: 1. Heyo! 2. Bug 452730 for your attn :-)
<mup> Bug #452730: lp_publish/ppa-generate-keys is broken atm <Soyuz:New> <https://launchpad.net/bugs/452730>
<noodles775> looking...
<noodles775> spm: do you know what the latency between the main and slave db servers is?
<noodles775> (just chasing one possibility for the above bug)
<wgrant> noodles775: Oh, I know exactly what that bug is.
<wgrant> devel r9687
<noodles775> wgrant: it's related to r9687.
<noodles775> yep...
<wgrant> There are now PPAs in production for users without a PPA named 'ppa'.
<noodles775> wgrant: sure, but the new method that's been added doesn't assume the name is ppa - it just grabs the first one.
<wgrant> noodles775: But that code is only on edge.
<wgrant> noodles775: Not on whatever machine runs ppa-generate-keys
<noodles775> wgrant: right.
<wgrant> So Person.default_ppa still looks up a PPA named 'ppa', always.
<noodles775> yep - I'll chat to bigjools about getting r9687 cp'd.
<noodles775> Thanks wgrant
<wgrant> That's what I suspected. Thanks.
<wgrant> spm: U1 rollout not done yet, I hope?
<spm> wgrant: partially.
<adeuring> good morning
<wgrant> spm: Great. I presumed it would be done after 3ish hours, but hoped it wasn't, given that it still seems broken.
<spm> the app servers are done; but i'm having a mild pebkac issue with one of the back end services - unsure how related that is to the problems
<wgrant> Uhoh.
<spm> believed I didn't have the keys to push the new hotness up. turns out I did - in a different directory. le-sigh.
<awilkins> jelmer: Why isn't Launchpad using subvertpy instead of python2.4-subversion  .... I'd guess it's an immense fiddle to have both subversion 1.6.5 and launchpad-dependencies because of the python bindings?
<wgrant> awilkins: Launchpad doesn't use bzr-svn. It uses a cscvs, horrible horrid horror which uses some strange svn bindings and predates subvertpy by several years.
<awilkins> wgrant: I'm presuming that means that LP-monitored SVN branches are essentially trapped and the only way to get patches back to the original project is to ... send patches?
<wgrant> awilkins: Yes. There is an intention to move to bzr-svn soon.
<awilkins> wgrant: Is there a roadmap the public can see? How about git integration? When will this pestering get too annoying? ;p
<wgrant> awilkins: I am the public. I do not know of such a roadmap.
<wgrant> 'git integration'?
<awilkins> LP using git in the same capacity it uses bzr, where possible
<wgrant> That is not on the roadmap even if it does exist.
<wgrant> win 23
<gmb> Morning padlaunchers.
<mrevell> Morning!
 * wgrant wonders whether to consider it safe to dive back into Canonical-produced code...
<wgrant> maybe as long as it isn't webapp code.
<awilkins> That on IP grounds, or just because it's scary? :-)
<wgrant> Scary in bad ways.
<jml> wgrant, that's simply not fair.
<wgrant> jml: Hm?
<wgrant> What have I done now?
<jml> wgrant, Canonical-produced code isn't that scary
<jml> and it's not a fair category.
<wgrant> jml: It is scary in that I don't know how many more fucking obvious huge vulnerabilities I will discvoer.
<jml> wgrant, oh ok.
<jml> wgrant, yeah, that sucks. :)
<wgrant> Although LP has less of them than U1, or they are less obvious. This is good.
<jml> allenap, hi
<jml> allenap, do you have any idea how to stop PQM from sending success messages to the Launchpad list every time we do an automatic merge from stable?
<jml> (you don't have to, but I thought I'd ask someone)
<jml> allenap, I've read your test parallelization email & lifeless's reply.
<jml> allenap, I don't have much to add to what Rob said, except to say that I'm happy for you to call to talk about stuff if you think that'll help.
<gmb> jml: Do you have any idea why I'd see http://pastebin.ubuntu.com/294573/ when running ec2 demo?
<gmb> What do I need to do to ec2 to know who I am in LP?
<jml> gmb, the unhelpful answer to your first question is that our ec2 tools lack tests.
<gmb> Hah.
<jml> gmb, give me a minute to finish what I'm doing now and I'll have a look & see what's going on.
<gmb> jml: Ta muchley.
<mrevell> Can anyone remind me where to find the API uses wiki page?
<mrevell> Of course, it's https://help.launchpad.net/API/Uses
<noodles775> lol
<jml> gmb, ok. I'm all yours.
<jml> gmb, can you paste me the full log of what you're doing, from command through to error?
<jml> gmb, also, what do you get when you type 'bzr launchpad-login'
<gmb> jml: So, full log: https://pastebin.canonical.com/23397/
<gmb> jml: `bzr launchpad-login` gives me `gmb`
<jml> gmb, thanks.
<jml> gmb, I'm going to try reproducing the problem locally.
<gmb> ok
<jml> "I told him to go away and reproduce his problem, but not in those words exactly"
<jml> (why I'm not Woody Allen)
<jml> gmb, try applying this patch http://pastebin.ubuntu.com/294591/. You'll need to do it in a branch that's not the branch that you want to demo (I recommend stable)
<gmb> jml: Okay, bear with me a sec...
<gmb> jml: Works. Thanks!
<gmb> jml: r=me on the patch if it needs landing ;)
<jml> gmb, https://code.edge.launchpad.net/~jml/launchpad/fix-ec2-demo/+merge/13461
<gmb> jml: Approved.
<jml> gmb, thanks.
<jml> gosh ec2 land is handy.
<bigjools> :)
<bigjools> is there a mode on it to send straight to PQM?
<jml> bigjools, no, I thought of doing that.
<bigjools> 'twould be useful for us heathens
<jml> bigjools, but then mwh said "what's the point?" and "your branch is too big already"
<bigjools> I can think of many points
<jml> bigjools, I'll happily review a patch for adding a --no-test option or similar.
<jml> alternatively
<bigjools> ok when I have some time I'll ask you for a pre-imp on that
<jml> I'll trade you that for a vfs backend on soyuz so people can rename their accounts if they have PPAs.
<bigjools> heh
<bigjools> I suspect that would push the renaming problem elsewhere, but I am not unsympathetic to it
<bac> bigjools, jml: i though of adding a --no-test option, but it seemed a little perverse since the point of the script is to send things off to ec2
<bac> bigjools: you can do 'ec2 land --print-commit' and combine the results with 'bzr pqm-submit'
<bigjools> I think its use lies more in the automation of the perverse options we have to put in the commit string
<bac> bigjools: i understand.  i've had the need myself but it still felt odd.
<bac> bigjools: so an alias of:  bzr pqm-submit -m "`ec2 land --print-commit`" or some variation may work
<bigjools> bac: that's so sick I love it :)
<bac> :)
<bac> bigjools: and it doesn't have to go through code review!
<allenap> jml: I don't know how to prevent PQM from sending success messages, but I'll have a look.
<allenap> jml: Thank you for reading through my message. I think Rob's reply is very useful to help me approach the problem, but I'll ask you about implementation as I go along.
<jml> allenap, cool. don't worry too much about the PQM thing, I've now just set up a rule so I don't get those emails any more :)
<allenap> jml: Jolly good :)
<bac> sinzui: https://edge.launchpad.net/gnome
<sinzui> bac: are you hinting that I fix the licenses for all those projects?
<sinzui> bac: This page looks better
<sinzui> bac: Do you know why the projects state they are proprietary when they are have a subscription: https://edge.launchpad.net/launchpad-project
<sinzui> bigjools: What is the license for https://edge.launchpad.net/launchpad-buildd
<bac> sinzui: nothing so deep.  i was showing you that our tiny little fix allows the gnome page to load without OOPSing
 * sinzui nods
<bigjools> sinzui: good question
<sinzui> https://edge.launchpad.net/launchpad-registry/+milestone/3.1.10
<Ursinha> err, better here
<Ursinha> salgado: ping :)
<Ursinha> salgado: hey, I'm checking the oopses here and I see occurrences of UnicodeDecodeError in +filebug
<Ursinha> salgado: OOPS-1384C2001
<Ursinha> err
<Ursinha> salgado: https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1384C2001
<Ursinha> salgado: is this related to bug 61171
<Ursinha> ?
<mup> Bug #61171: +login page OOPSes if query string has accented chars encoded as ASCII <oops> <Apport:Invalid> <Launchpad Foundations:Fix Released by salgado> <https://launchpad.net/bugs/61171>
<salgado> Ursinha, no, this is something else
<dobey> hey guys
<dobey> is it possible to change the comment string on a GPG key that was generated for a user/team for PPAs?
<Ursinha> salgado: I've filed another bug, bug 453203
<mup> Bug #453203: UnicodeDecodeError in +filebug: unexpected code byte <oops> <Launchpad Bugs:New> <https://launchpad.net/bugs/453203>
<Ursinha> maybe a bugs' person could take a look on it
<Ursinha> hi allenap
<allenap> Ursinha: Hi!
<Ursinha> :)
<Ursinha> allenap: do you have some time to take a look at bug 453203, please?
<mup> Bug #453203: UnicodeDecodeError in +filebug: unexpected code byte <oops> <Launchpad Bugs:New> <https://launchpad.net/bugs/453203>
<allenap> Ursinha: Looking now. That's an odd one.
<Ursinha> thanks allenap
<jcinpv> Running Ubuntu 9.04 in Virtualbox on an Intel Mac; After a recent update, the Window Manager does not launch on start up or restart, but does launch after logging out and back in. Going to System->Preferences->Windows, get error message: 'Cannot start the preferences application for your window manager; Window manager "unknown" has not registered a configuration tool.'  I have deleted .gnome2, .gnome2_private, .gconf, and .gconfd to
<jcinpv> no avail and I have reproduced the problem in a new user account. Where to look?
<Ursinha> jcinpv: I think you may find more useful advices on #ubuntu
<mrevell> Have a good weekend all.
 * rockstar lunches
<jcinpv> Ursinha: Been there. No help.
<jcinpv> They suggested I submit a bug.
<jcinpv> Where do I submit a bug?
<Ursinha> jcinpv: https://launchpad.net/ubuntu/+filebug
<jcinpv> Ursinha: Thanks. Bug submitted.
<Ursinha> np jcinpv, sorry not being that helpful
<jcinpv> Ursinha: You were very helpful in pointing me to the right place to file a bug. My first time.
<Ursinha> jcinpv: in this case, glad that helped :)
<rockstar> abentley, chat?
<abentley> rockstar: 1 sec
<rockstar> abentley, just call when ready.
<barry> are there any bugs folks still around?  allenap, gmb, intellectronica, deryck
<rockstar> abentley, that lag is terrible.
<abentley> rockstar: and unpredictable...
<gary_poster> night all
#launchpad-dev 2009-10-17
<maxb> Is there any magic in launchpad's tests which causes warnings to become fatal exceptions?
<maxb> oh, there it is
<maxb> The good: We have the tests all passing on Python 2.5!
<maxb> The bad: The likelyhood of Python 2.6 happening any time soon is somewhat remote
<lifeless> maxb: how so?
<maxb> It looks like it'll force a huge amount of upgrading of Zope and other deps
<rockstar> maxb, I don't think that's necessarily a bad thing.
<malloc_test> Hi to all, I'm trying launchpad code on my machine, but I have a problem with bazaar use. I followed the developer instructions, all works good, but when I register a new branch I don't able to push code with bzr. with the shell command "bzr push lp://dev/~pippo/+junk/devel" this get "bzr: ERROR: http://bazaar.launchpad.dev/~pippo/%2Bjunk/devel/.bzr/smart is redirected to http://launchpad.dev". Where I'm wrong? Thanks and excuse me for my English
<maxb> malloc_test: hm. I wonder if your /etc/hosts and/or apache vhost config is not right
<maxb> I can believe bzr might say that if yoou landed a bzr+http request on a browser vhost
<malloc_test> maxb: thanks for your answer. I don't modified manually /etc/hosts and apache conf, but I used launchpad script. I think that launchpad doesn't create project or/and branch folder. Where I can find ~pippo/%2Bjunk/devel folder?
<maxb> malloc_test: What does 'host bazaar.launchpad.dev' report?
<malloc_test> maxb: in /etc/hosts there is "127.0.0.99      bazaar.launchpad.dev"
<maxb> yes, but what does 'host bazaar.launchpad.dev' say :-)
 * maxb gone
<malloc_test> maxb: I misunderstood, host command report this: bazaar.launchpad.dev.fastwebnet.it mail is handled by 10 mx3.fastwebnet.it.
<maxb> !
<maxb> you have really weird dns
<maxb> Oh, hangon, you're *pushing* ?
<maxb> You can't push over http
<maxb> You ought to use make-lp-user.py to create a user in your dev launchpad with the same id as your real launchpad account, and tell bzr of the id with "bzr launchpad-login <id>"
<malloc_test> Yes you're right, this is my italian network dns...:'(.  Ok with those commands all ok, I've create a new branch but with command "bzr push lp://dev/~antonio-litterio-gmail/+junk/dev" return this " bzr: ERROR: Parent directory of lp://dev/~antonio-litterio-gmail/+junk/dev does not exist.", and with command "bzr push lp://dev/~antonio-litterio-gmail/cnit/devel --create-prefix " return this "bzr: ERROR: Permission denied: "~antonio-litterio-gmail/c
<maxb> Hmm, bit weird
<malloc_test> maxb: yes it is very weird!!
<malloc_test> maxb:I'll try to download and reconfigure all,I'll tell you later, thanks a lot.
<maxb> Is there anyone around who can explain what this error means:
<maxb> $ bzr push lp://dev/~maxb/+junk/foo
<maxb> bzr: ERROR: Server sent an unexpected error: ('error', "The transport '<lp.codehosting.vfs.transport.SynchronousAdapter url=lp-79819856:///~maxb/+junk/foo/>' is only accessible within this process.")
<maxb> hmm
<maxb> console logging says it runs bzr lp-serve, but who knows what happens then
<maxb> uhoh
#launchpad-dev 2009-10-18
<malloc_test> Hi, I use karmic to test launchpad, after register a new branch and launch in shell the command "bzr push lp://dev/...", I launch command "make sync_branches" and in a branch page I have this error "ImportError: No module named enum". I've installed python enum module but nothing changes...
<maxb> It is a known issue arising from a conflict with the system-installed lazr.* packages
<maxb> We hope that gary will be able to finish fixing it on monday
<maxb> As a workaround, you may: sudo rm /usr/lib/python2.4/site-packages/lazr.*-nspkg.pth
<maxb> But be aware this is mutilating the install of those packages for the sake of launchpad
<malloc_test> maxb: thanks, if you go in Italy I offer you a beer ;-)
#launchpad-dev 2010-10-18
<mwhudson> thumper: is there a bug report for the fact that i can be requested to review a branch i can't see?
<thumper> mwhudson: probably
<lifeless> mwhudson: I don't remember one
<thumper> it rings a vague bell
<mwhudson> yeah, me too
<mwhudson> thumper: can't you actually fix https://bugs.edge.launchpad.net/launchpad-code/+bug/624009 now?
<_mup_> Bug #624009: Email should not be sent for new WIP merge proposals <code-review> <email> <Launchpad Bazaar Integration:In Progress by thumper> <https://launchpad.net/bugs/624009>
<thumper> yeah...
<thumper> it is still showing on our kanban board :)
<mwhudson> and my bug is there https://bugs.edge.launchpad.net/launchpad-code/+bug/330290
<_mup_> Bug #330290: Can request a review from someone who can't see the merge proposal <code-review> <confusing-ui> <privacy> <Launchpad Bazaar Integration:Triaged> <https://launchpad.net/bugs/330290>
<mwhudson> $ time ./bin/py -c pass
<mwhudson> real	0m0.610s
<mwhudson> that's quite slow...
<mwhudson> $ strace ./bin/py -c pass 2>&1 | grep -c stat
<mwhudson> 19347
<mwhudson> that's quite a lot of stats
 * mwhudson wonders if this has regressed again
<lifeless> buildout ftw
<lifeless> ?
<mwhudson> yeah, to some extent certainly
<lifeless> (by which I mean pth files, egg lookups etc)
<mwhudson> stat()ing /home/mwh/canonical/checkouts/trunk/eggs/RestrictedPython-3.5.1-py2.6.egg three times seems a bit silly
<lifeless> that stack needs to be tuned and optimised for everyones sske
<mwhudson> there is this pattern of stats for every egg afaics: http://pastebin.ubuntu.com/515317/
 * thumper has errands in town, back soonish
<wallyworld> lifeless: can you please look at https://pastebin.canonical.com/38714/
<wallyworld> do you agree with the approach?
<lifeless> mmm
<lifeless> a few concerns
<lifeless> 'config' is establish as an idiom meaning 'the current config'
<lifeless> so overloading that will confuse people
<lifeless> lets not do that
<wallyworld> so i just don't use the alias?
<lifeless> secondly, testrunner-appserver is hopefully deletable
<lifeless> see the current thread on the list
<lifeless> even if isn't deletable, it won't be usable because its still going to be static
<lifeless> that said, I'd expect that individual tests *already* are running in that config, aren't they ?
<wallyworld> not sure. i can check.
<lifeless> (if not, how are they managing to connect at all ?)
<wallyworld> one of the ones i'm looking at to start is lib/canonical/launchpad/doc/launchpadlib.txt
<lifeless> if the answer is 'a subprocess is run in that config' then the approach of instantiating a specific config to let us refer to the config the external process is using makes sense, but lets not make it a global
<lifeless> for tw oreasons
<lifeless> two reasons
<lifeless> firstly, I'm this || close to sending in a patch to dynamically generate configs in the test run, so any global like that will be stale
<lifeless> secondly we have way too many globals, lets reduce them.
<wallyworld> sure, i was trying to fit in with how stuff had been done previously
<lifeless> I appreciate that :)
<lifeless> and I'm glad -very glad- you're looking at this
<wallyworld> i can just create a config instance on the fly: config = CanonicalConfig('testrunner-appserver')
<lifeless> something like
<lifeless> appserver_config = BaseLayer.getAppserverConfig()
<lifeless> is probably whats needed
<lifeless> the hard coded string there would be a problem
<wallyworld> ok. my exposure to this stuff is still very limited :-)
<lifeless> mine too
<lifeless> I started looking at the config system code on sunday :)
<wallyworld> so the BaseLayer.getAppServerConfig() should return me the correct config instance?
<wallyworld> i started 30 minutes ago :-)
<lifeless> I think that that will work
<wallyworld> ok. thanks. i'll suck it and see
<wallyworld> there's quite a few hard coded urls to fix. that's why i wanted to ask first :-)
<thumper> wallyworld: want a chat?
<lifeless> wallyworld: yeah
<lifeless> wallyworld: what I'm suggesting will give us one place to change if we need to change
<lifeless> \o/ databasefixture has landed
<wallyworld> thumper: ok
<wallyworld> lifeless: that's what i was looking for too. i'll let you know how i get on. thanks for the input
<thumper> lifeless: compile failed?
<lifeless> thumper: did it?
<thumper> lifeless: aye
<thumper> lifeless: can you parse the error?
<lifeless> Could not locate /srv/buildbot/slaves/launchpad/lucid-devel/build/orig_sourcecode/eggs/setuptools-0.6c11-py2.6.egg/setuptools/package_index.py:155: UserWarning: Unbuilt egg for ClientCookie [unknown version] (/usr/lib/python2.6/dist-packages)
<lifeless> ops issue i think, machine was being upgraded or some such
<spm> hmm?
<mwhudson> lifeless: i presume the plan after edge and lpnet are running the same code is to remove the edge.launchpad.net domain?
<mwhudson> well, not remove, deprecate & redirect from
<thumper> wallyworld: https://dev.launchpad.net/Code/BranchRevisions
<lifeless> mwhudson: yes
<lifeless> spm: was any maintenance being done on that machine
<lifeless> spm: that could cause that crazy output
<spm> the lucid-devel slave?
<lifeless> spm: yes
<lifeless> wallyworld: I'm exposing the config for you as BaseLayer.appserver_config_name
<wallyworld> lifeless: ok. i'm just coding everything now. i can make my branch dependent on yours?
<lifeless> sure
<wallyworld> for now i'll use a string literal in the getAppServerConfig, just to get stuff working
<lifeless> sure
<wgrant> How often will qastaging's DB be clobbered?
<LPCIBot> Project db-devel build (78): STILL FAILING in 4 hr 4 min: https://hudson.wedontsleep.org/job/db-devel/78/
<spm> lifeless: is that a moderately recent error? because there hasn't been any work on the slaves in ~ 4 days ish. afaict/find
<lifeless> spm: today
<lifeless> wgrant: same time stagings is
<spm> lifeless: http://www.mail-archive.com/launchpad-dev@lists.launchpad.net/msg02726.html ??
<spm> ie. lmgtfy ;-)
<lifeless> spm: huh?! *today*
<spm> same error
<spm> we've seen this beofre. it's funky crap inside the test suite
<lifeless> so you're thinkinging dirt in the lp buildbot build area?
<spm> no. I'm thinking funky crap inside the test suite(s)
<lifeless> spm: the only change was mine, and it passed ec2.
<spm> lifeless: not sure. All I can say is that we've seen that error at least once before, back in March. it was weirdness inside the lp code/tests/includes/whatevers. I'd focus there.
<thumper> lifeless: what are we doing about the thread garbage in ec2 tests?
<wgrant> The current policy seems to be "lalalala I'm going to throw it at the wall until it sticks"...
<thumper> :(
<wgrant> Although Hudson's incessant whinging has almost convinced me to dig into it.
<lifeless> thumper: I'm going to talk with stub again
<thumper> ack
<lifeless> I was advocating disabling the check last week
<lifeless> I think it has to be downgraded to a warning for now, at most.
<lifeless> but I need data
<wgrant> The remaining failures are just Windmill, aren't they?
<lifeless> I think so
<rockstar> poolie, hi. Do feature flags also work with the API?
<poolie> what do you mean?
<poolie> can code called by the api consult flags?
<poolie> i think it will work, but you may be the first to do it
<rockstar> poolie, I mean the web api.  If I land my work, it's going to show up in the web api, isn't it.
<thumper> rockstar: yes, I think so
<rockstar> thumper, yeah, that's what I figured.
<rockstar> I'm not sure whether or not I care enough to try and fix it.  I mean, merge queues have been available through the API for a while.  :)
<rockstar> I guess I'll just need to be quick.
<poolie> rockstar: i think it should just work but it depends on to what extent the web api uses the standard publication stuff
<poolie> i would really appreciate hearing that either works, crashes, or silently fails
<thumper> rockstar: just be quick :)
<poolie> ... by way of a bug in the latter two cases
<rockstar> poolie, how do you specify in an interface that a property is only accessibly through a feature flag?
<wgrant> You can't.
<wgrant> You have to check the flag inside the method.
<mwhudson> i guess you can make a method error in some meaningful way if a flag is not in effect
<mwhudson> a property... dunno
<wgrant> (I believe the DistroSeriesDifference API stuff is currently protected by flags)
<poolie> right
<thumper> lifeless: do you think calls like https://lp-oops.canonical.com/oops.py/?oopsid=1751XMLP32 could block the xmlrpc server for the other calls?
<poolie> rockstar: i mean presumably you would get... an AttributeError... if it didn't exist at all?
<poolie> you could raise that? or Unauthorized, perhaps?
<lifeless> rockstar: you'd need to change the wadl processing to use a flag in that fashion
<lifeless> rockstar: as wgrant says, its not designed for static things like that, because it depends on the db
<lifeless> I'd just do your things, merge queues are classified as unsupported by thumpers fiat ;)
<rockstar> lifeless, yeah.
<lifeless> as long as what you land is working at any point, its ok
<rockstar> lifeless, yeah, that's the plan.
<thumper> lifeless: did you boot buildbot?
<lifeless> yes
<lifeless> hoping its nonsense
<lifeless> which it looks like it was
<bac> good morning rockstar + antipodes
<wgrant> Afternoon.
<bac> wgrant: did you get all three of your branches landed, after the annoying ec2 faliures?
<thumper> hi bac
<wgrant> bac: And other three after that, yeah.
<rockstar> bac, are you in Japan?
<wgrant> After a couple of runs.
<bac> rockstar: while i am big in japan i'm actually in vietnam
<lifeless> wallyworld: rev 11741 in lp:~lifeless/launchpad/uniqueconfig is pushing now
<lifeless> wallyworld: I'm not done, but the bits you need are done.
<wallyworld> lifeless: thanks. only 100+ more tests to fix :-)
<wallyworld> lifeless: i'm just assuming your rev with include the app config name - i've got methods for all the rest i need
<wallyworld> s/with/will
<rockstar> wallyworld, did you get your windmill issues fixed?
<wallyworld> rockstar: no :-( but i talked to adam from the windmill devs and he suggested a windmill proxy issue may be the cause
<wallyworld> i'm not 100% sure exactly what that means just yet
<wallyworld> all i know is YUI and windmill don't play nicely together
<rockstar> wallyworld, yeah.  Here's what I've found.  The same tests fail (relatively) consistently on my machine, but ec2 has failed 6 different runs with different tests failing each time with no changes from me.
<rockstar> wallyworld, I don't see how YUI and windmill wouldn't play well together.  windmill doesn't know YUI from Adam.
<wallyworld> rockstar: the same general failure mode? ie YUI breaks somehow?
<wallyworld> from what adam said, there's some proxying the windmill does
<wallyworld> s/the/that
<wallyworld> rockstar: adam talked about needing to blacklist a url to prevent windmill proxying it and gave this reference
<wallyworld> http://github.com/windmill/windmill/wiki/Preferences-File
<rockstar> wallyworld, hm, I was under the impression that the windmill proxy was important.
<wallyworld> i'm not sure - i haven't devoted enough brainspace to understanding the windmill architecure at this point
<wallyworld> i was hoping it would make more immediate sense to you :-)
<rockstar> wallyworld, it doesn't make much sense to me now, but it's also a bit late right now.  Maybe I can make some sense of it tomorrow.
<wallyworld> rockstar: ok. we'll talk then. have a good evening
<lifeless> wallyworld: it generates unique testrunner-appserver configs in BaseLayer.setUp
<lifeless> wallyworld: it doesn't allocate unique ports in the generated config [yet]
<lifeless> wallyworld: and really the whole config thing is backwards here, we want to start the server and readback the port as I described on the list
<lifeless> EFUTURE
<wallyworld> lifeless: np. i'll look at the branch to see what's there. atm, i'm fixing the test logic but the expected results still contain the hardcoded 8085. if the ports are generated, i'll have to come back and deal with that too
<lifeless> wallyworld: what do you mean by 'expected results' ?
<lifeless> wallyworld: the goal is to use different ports so we can run tests in parallel
<wallyworld> lifeless: yeah, i realise that. here's a snippet from on e of the doc tests
<wallyworld>     >>> browser.url
<wallyworld>     'http://launchpad.dev:8085/~stimpy'
<wallyworld> see how the expected url value is hard coded
<lifeless> ok
<wallyworld> i can deal with that. my first aim is to get the test logic sorted out, then come back around and fix this other stuff
<lifeless> ok
<lifeless> nuking the darn doctests would be one way ;)
<wallyworld> well, the unit tests do the same thing :-)
<lifeless> wallyworld: except assertions are easier there
<wallyworld> +1
<lifeless> perhaps browser could grow some support
<lifeless> like
<lifeless> vhost : launchpad.dev / bugs.launchpad.dev etc
<lifeless> urlpath : /~stimpy
<wallyworld> the windmill layer setup does this sort of thing
<lifeless> if browser had that
<lifeless> then you could write the above as
<lifeless>  >>> browser.vhost
<lifeless>   launchpad.dev
<lifeless>   >>> browser.urlpath
<lifeless>   /~stimpy
<lifeless> which would probably be easier than converting to unittest in the short term
<lifeless> or even have a normalise_url, or something.
<wallyworld> yes. doc tests are here for a while yet given how much is in them
<wallyworld> what i am playing with now is I have a method called getRootLaunchpadUrl(sitename='mainsite') which delgates to BaseLayer.xxx
<wallyworld> so with one line of doc in doc tests i can get the url i needed
<wallyworld> but i could look at adding that to the browser as you suggest
<lifeless> the main thing for doctests is that you need to be able to write the literal expected value
<lifeless> if you can do that, awesome
<lifeless> I'm just riffing on the approach is all
<wallyworld> np. it's always very good to discuss different ideas
<lifeless> https://code.edge.launchpad.net/~lifeless/launchpad/uniqueconfig/+merge/38689 \o/
<lifeless> jml: testtools upgrade broke bizarrely
<LPCIBot> Project devel build (124): STILL FAILING in 4 hr 9 min: https://hudson.wedontsleep.org/job/devel/124/
<lifeless> is there a config serialiser class ? for writing to the configs ?
<wgrant> spm: Around?
<spm> wgrant: ish...
<wgrant> spm: The reporter of https://bugs.edge.launchpad.net/ubuntu/+source/beryl-core/+bug/109550 needs your magical banhammer.
<_mup_> Bug #109550: Black/White/unrendered windows in Beryl  <beryl-core (Ubuntu):Invalid> <https://launchpad.net/bugs/109550>
<spm> ahh. a special person eh?
<wgrant> Spam of some variety.
<wgrant> Only to that bug, that I've seen.
<spm> awesome. ta.
<wgrant> StevenK: Ohloh seems to be less glacial now... might be done tomorrow.
<StevenK> Colour me suprised.
<StevenK> http://paste.ubuntu.com/515453/ :-(
<wgrant> StevenK: That can mean a missing export_as_webservice_entry()
<wgrant> I think.
<wgrant> What's your diff?
<wgrant> It can also happen if your thing doesn't get imported into canonical.launchpad.interfaces.*
<wgrant> But your diff will tell me everything.
<StevenK> https://code.edge.launchpad.net/~stevenk/launchpad/db-add-derivedistroseries-api/+merge/38189
<StevenK> wgrant: It's messy due to the MP being targetted at db-devel, but now I'm free to land on devel, so read with caution
<wgrant> StevenK: What's the real diff?
<StevenK> wgrant: The test changes and lib/lp/registry/{interfaces,model}/distroseries.py
<StevenK> Addition of deriveDistroSeries()
<wgrant> Ah.
<wgrant> StevenK: You probably need to patch your 'distribution' argument's interface.
<wgrant> Since IDistroSeries.distribution is an Interface for circular import reasons.
<spm> wgrant: fyi. account zotted; old comments zotted as well
<wgrant> You know about the _schema_circular_imports fun, right?
<wgrant> spm: Thanks.
<StevenK> wgrant: Oh, right, so I can copy_field, as long as I also deal with circular_imports?
<wgrant> I think so.
<wgrant> Anyway, I need to run.
<lifeless> \o/
<lifeless> dynamic db names working.
<lifeless> in tests as in.
<lifeless> lets see if it works for the librarian too
<lifeless> seems too, but something remade the ftest db. Hmm.
<lifeless> O M G
<lifeless> lp test start reads from dev/random
<jpds> lifeless: Does it support sharding?
<lifeless> jpds: it must. Its web scale.
<lifeless> :!bin/test -vvt test_pgsql --parallel
<lifeless> Tests running...
<lifeless> Ran 15 tests in 27.225s
<lifeless> ^- woo
<lifeless> StevenK: hey, what instance size are you using in hudson ?
<adeuring> good morning
<StevenK> lifeless: The largest
<StevenK> lifeless: If you're running out of space on /, fiddle things to use /mnt
<lifeless> StevenK: well, its just that I'm curious how many cores it will get
<StevenK> lifeless: Oh, right. Lemme check
<lifeless> StevenK: largest is ill defined
<lifeless> there are ;argest-cpu, largest-memory etc
<lifeless> m1.xlarge ?
<StevenK> lifeless: Picking on one of the running instances, it has 2GB of RAM and 4 QEMU cores.
<lifeless> c1.xlarge would be interesting to try
<lifeless> 8 cores
<StevenK> That is c1.xlarge
<lifeless> surely not
<StevenK> Yes
<lifeless> http://aws.amazon.com/ec2/instance-types/
<StevenK> It isn't using ec2
<lifeless> oh
<lifeless> thats right
<lifeless> it would be nicer if they matched up
<StevenK> File a ticket? :-)
<StevenK> But I daresay, it's a problem of resource management
<lifeless> StevenK: btw
<lifeless>  https://devpad.canonical.com/~lpqateam/qa_reports/deployment-stable.html
<lifeless> blames you for needing to QA :)
<StevenK> lifeless: Yes, I'm blocked on dogfood, I'm waiting for Julian and Tom.
<lifeless> ok
<lifeless> please file a bug / RT ticket explaining what we need to change to let this get qa'd on qastaging.
<lifeless> we /have/ to fix this pipeline
 * wgrant can hear people crying already.
<StevenK> lifeless: Shell access
<lifeless> StevenK: surely you can come up with a better answer
<lifeless> StevenK: like 'this is a shell script, so I can write a test driver for it that the losas can run on request'
<StevenK> lifeless: No, I can't. The bits are infrastructure and I need to reach in and rummage around.
<lifeless> StevenK: why?
<StevenK> lifeless: Because I'd like to confirm things look good at a database level before I kick off a script, etc, etc.
<StevenK> I'd like to be perfectly clear the right things are happening
<lifeless> StevenK: this sounds automatable
<wgrant> (also, most of the archive admin tools still require shell access)
<lifeless> but even if its not, we need to be able to qa on qastaging
<lifeless> this is a fundamental tenant of being able to do continuous deployment.
<persia> (wgrant: most of them can be run by waldo though, by someone else if given the arguments)
<lifeless> it needs to be written down
<lifeless> handed off to a losa
<lifeless> -or something-
<StevenK> lifeless: In other news, can you poke at http://people.canonical.com/~stevenk/windmill-thread-debug-r11734.subunit.gz
<lifeless> forbidden
<lifeless> mail it to me or something
 * StevenK curses
<StevenK> lifeless: Fixed, please retry
<lifeless> so what do you want me to see?
<StevenK> lifeless: I added some more debugging to the windmill teardown, there's an oddness with a Timer thread there
<lifeless> I'd say thats a threading subclass
<StevenK> lifeless: And I also got 'test process died with exit code 2, but no tests failed', which I don't believe.
<lifeless> used by windmill
<lifeless> I think that test process thing is damage in zope.testing's subunit support, which I put a patch up for months ago
<lifeless> benji landed it in the weekend I think
<StevenK> Oh, damn it
<StevenK> That revision is qa-ok, but Ursinha's script munged it
<StevenK> lifeless: When does that page get re-generated?
<lifeless> StevenK: 10 minutes or so
<lifeless> Ursinha-afk: can you run it any more frequently now?
<StevenK> lifeless: As in, 'in ten minutes' or 'every ten minutes' ?
<lifeless> StevenK: its in cron, I think its a 10 minute frequency
<lifeless> grr wtf OperationalError: FATAL:  database "launchpad_ftest_template" does not exist
<StevenK> Right, now that page doesn't blame me, excellent.
<wgrant> Does it blame me instead?
<StevenK> No, sinzui
<lifeless> sinzui
<wgrant> Ah.
<lifeless> however, anyone can qa
<lifeless> in principle
<StevenK> wgrant: We can change it so it always blames you
<lifeless> Bug 46581 is marked as not-ok.
<_mup_> Bug #46581: Change a poll type URL manually crashes <oops> <polls> <qa-needstesting> <Launchpad Registry:Fix Committed by sinzui> <https://launchpad.net/bugs/46581>
<lifeless> I gotta say, wtf is dropping the template db
<StevenK> Crash dump was written to: erl_crash.dump
<StevenK> Kernel pid terminated (application_controller) ({application_start_failure,kernel,{shutdown,{kernel,start,[normal,[]]}}})
<StevenK> Wheee, yay for readable logs, rabbitmq
<StevenK> -rw-r----- 1 rabbitmq rabbitmq 243K 2010-10-18 08:02 /var/lib/rabbitmq/erl_crash.dump
 * StevenK sobs
<LPCIBot> Project devel build (125): STILL FAILING in 3 hr 44 min: https://hudson.wedontsleep.org/job/devel/125/
<LPCIBot> * Launchpad Patch Queue Manager: [r=henninge][ui=henninge,
<LPCIBot> sinzui][bug=639703] Correctly report bug tracking status for project
<LPCIBot> groups.
<LPCIBot> * Launchpad Patch Queue Manager: [r=mwhudson][ui=none][no-qa] Cleanup some partially-migrated-from\n\ttest support code.
<LPCIBot> * Launchpad Patch Queue Manager: [r=mwhudson][ui=none][bug=660843][no-qa] Move the garbo scripts into
<LPCIBot> lp.scripts
<StevenK> Hm, I think I need to turn that off.
<lifeless> https://bugs.edge.launchpad.net/launchpad-foundations/+bug/662519 made me cry
<_mup_> Bug #662519: lp test startup exhausts dev/random <Launchpad Foundations:New> <https://launchpad.net/bugs/662519>
<lifeless> test run should be finished soon.
<lifeless> \o/
<StevenK> lifeless: I can kill that parallel-test build if you wish
<lifeless> nono
<lifeless> its nearly done
<StevenK> Even with the no _template db errors?
<lifeless> need to get the details and debug them
<lifeless> most tests are passing
<lifeless> its probably the tests that sampledata loading/dumping works, or some such
<lifeless> hmm https://hudson.wedontsleep.org/job/parallel-test/4/ took 2 hours
<lifeless> so it may have an hour to go
<bigjools> wgrant: how much testing have you done on bug 655690?
<_mup_> Bug #655690: release_files_needed shouldn't live on FTPArchiveHandler <qa-needstesting> <soyuz-publisher> <tech-debt> <Soyuz:Fix Committed by wgrant> <https://launchpad.net/bugs/655690>
<lifeless> Announcing Amazon SNS Management Console looks interesting
<wgrant> bigjools: I've tested both a-f and NMAF fairly thoroughly.
<wgrant> Plus the tests are a bit better now.
<jml> want to see a blast from the past? https://devpad.canonical.com/~jml/mirror-failure.png
<lifeless> meep
<mwhudson_> jml: wow
<jml> lifeless: fwiw, something else weird is going on with your branch. You should definitely not get an email like that for test failures.
<lifeless> jml: the long spew of info? i filed a bug, its having a root logging handler installed that uses std*
<jml> lifeless: no, not the root logger
<jml> lifeless: although that's another problem
<jml> lifeless: I changed the email so that it should start with a list of failing tests, and then have only the error info for those tests
<jml> lifeless: it sends out the full horrible dump only when something surprisingly bad happens.
<jml> but a quick scan of the full log doesn't reveal anything
<lifeless> jml: what full horrible dump ?
<jml> lifeless: this is what a failing branch email is supposed to look like: Tests started at approximately 2010-10-16 10:04:52.243410
<jml> Source: bzr+ssh://bazaar.launchpad.net/~wgrant/launchpad/better-publisher-index-tests r9886
<jml> Target: bzr+ssh://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel/ r11722
<jml> 10565 tests run in 3:58:20.128578, 0 failures, 1 errors
<jml> Tests with errors
<jml> -----------------
<jml>  lp.code.windmill.tests.test_branchmergeproposal_commitmessage.TestQueueStatus.test_inline_queue_status_setting
<jml> ======================================================================
<jml> ERROR: lp.code.windmill.tests.test_branchmergeproposal_commitmessage.TestQueueStatus.test_inline_queue_status_setting (subunit.RemotedTestCase)
<jml> ----------------------------------------------------------------------
<jml> _StringException: Text attachment: garbage
<jml> ------------
<jml> [<Thread(Thread-1110, started daemon 47989191493392)>]
<jml> ------------
<jml> **NOT** submitted to PQM:
<jml> [r=bac][ui=none][no-qa] Start de-duplicating the apt-ftparchive and native publishing tests, and add more thorough index generation tests.
<jml> (See the attached file for the complete log)
<jml> sorry
<jml> http://paste.ubuntu.com/515540/ â I meant to paste that
<wgrant> Severe chastisement.
<lifeless> jml: thats what mine looked like
<jml> lifeless: no, it didn't. the testtools -> devel email has stdout in it, for example
<lifeless> ah
<jml> lifeless: it does not have a list of tests with errors, for another example :)
<lifeless> so, I think its because the *submitting tree* is used to control stuff
<lifeless> not the target/source branch
<jml> I don't follow
<lifeless> I can't ec2 submit from my dev environment
<jml> oh, you mean it's using remote.py from your branch? yes, that would be it.
<jml> your local working tree, in fact
<lifeless> I have a very old tree I never touch, because it 'works'
<lifeless> its outside my dev vm
<lifeless> generally when I touch it, it breaks. so I dont
<jml> lifeless: ahh, ok.
<jml> lifeless: well the new pretty email is very pretty and helpful.
<jml> lifeless: but I can understand not wanting to mess about
<lifeless> jml: testr looks after me ;)
<jml> lifeless: I've updated run-test-improvements
<LPCIBot> Project parallel-test build (5): STILL FAILING in 2 hr 15 min: https://hudson.wedontsleep.org/job/parallel-test/5/
<lifeless> jml: how about run_with rather than run_test_with ? just a thought
<lifeless> jml: def _run_test_method(self, result):
<lifeless> jml: doesn't use result
<jml> lifeless: thanks.
<jml> lifeless: it never did.
<lifeless> jml: interesting
<jml> lifeless: also, _run_setup and _run_teardown
<bigjools> jml: helleau.  Did you by any chance get to look further at the buildd-manager branch?  I'm about to run it on dogfood again, when DF is working.
<jml> bigjools: no, not yet. how about I do that now :)
<bigjools> jml: that'd be smashing
<lifeless> nightish, all.
<wgrant> Night lifeless.
<jml> lifeless: g'night
<jml> bigjools: you're landing it on devel, I take it?
<bigjools> jml: that's the plan
<jml> bigjools: you're not doing getFile => deferred as part of this branch, right?
<bigjools> jml: no, it's a lot of work
<jml> bigjools: ok.
<jml> bigjools: have you filed a bug about it?
<bigjools> jml: no
<jml> bigjools: that might be a good idea. you've noted a few places where it needs to change in this branch.  A bug number will make grepping easier later :)
<wgrant> bigjools: Can you think of a good reason to keep the behaviour where we skip publication of series without a lucilleconfig? An uninitialised series can't have any publications, so there's nothing to be published, so it seems pointless.
<bigjools> jml: I don't care about grepping, the few XXXes that are there are the tip of the iceberg.  I have a list on my desk of what needs to change on the delightful piece of cardboard that we used :)
<wgrant> (plus it's all that's keeping me from dropping the column)
<bigjools> wgrant: it was only being skipped to stop the publisher blowing up when it tried to get one that was not there
<wgrant> bigjools: That's what I thought, thanks.
<jml> bigjools: the bus factor for that is dangerously high
<bigjools> jml: I know.
<bigjools> jml: but it's not like someone else couldn't do what I did
<bigjools> I can add XXXes though
<bigjools> BTW I can thoroughly recommend the Finca El Fany from Hasbean.  Apart from having a hilarious name, it tastes delicious.
<jml> :D
<wgrant> Hmm. What's archivepublisher.root set to in the 'ppa' prod config?
<jml> I'm on Sainsbury's coffee until I return from the states
<bigjools> wgrant: why?
<jml> the slave api needs to change
<bigjools> the Sainsbury's ground colombian is very nice
<wgrant> bigjools: I believe the code currently uses Ubuntu's lucilleconfig's temp directory (/srv/launchpad.net/ubuntu-archive/ubuntu-temp) for PPAs.
<bigjools> wgrant: for doing what for PPAs?
<wgrant> bigjools: One of my next few branches switches to ${archivepublisher.root}/ubuntu-temp instead.
<wgrant> bigjools: It puts Sources and Packages in there first.
<bigjools> oh ok, before the atomic-ish mv
<wgrant> Yep.
<bigjools> it's set to the directory that contains all the repos
<bigjools> there's also a private_root
<wgrant> It isn't in my old copy.
<wgrant> That's the PPA root.
<wgrant> Not archivepublisher.root.
<bigjools> oh, meh
<wgrant> I could change it, but we probably don't want the temp directory showing up under the Apache-served root...
<bigjools> wgrant: ppa does not override the values used in the ubuntu publisher
<wgrant> bigjools: Does it inherit from ftpmaster?
<bigjools> yes
<bigjools> well
<wgrant> Ah.
<bigjools> no
<bigjools> I can't read today
<wgrant> My copy of the configs is a year old, and I believe things have changed a bit since then.
<wgrant> But by my reading it will inherit schema-lazr.conf's archivepublisher.root, which is /var/tmp/archive, which is probably wrong.
<bigjools> yes it will use that
<bigjools> even dogfood's config sets it.  Ha.
<wgrant> :(
<wgrant> Hah, to /srv/launchpad.net/ppa. How odd.
<bigjools> not really
<wgrant> But the PPA config doesn't set it to that...
<bigjools> jml: what's up with the slave api, apart from the fact it's crap?
<jml> bigjools: potato programming
<jml> sendFile, sendFile, sendFile, sendFile, ...
<bigjools> yeah that blows
<jml> why not sendFiles?
<jml> not only is it slower but it makes the client-side harder to write.
<jml> I guess a good thing about this change is that now everything goes through BuilderSlave, so you could change the API for BuilderSlave to look like what you want the XML-RPC API to look like
<bigjools> indeed, it's a nice layer of abstraction
<wgrant> RecordingSlave is gone?
<wgrant> Yay.
<bigjools> totally gone
<bigjools> the new manager.py is about 30% of the size
<wgrant> I meant to read the diff, but it's sort of big.
<jml> yeah
<jml> wgrant: I'm skimming the diff and then going to look at the new manager.py & test_manager.py â the diffs for those are so big as to be useless
<bigjools> jml: there's one more bug I need to fix in the code - the failure counting assumes there's always a job on the builder, but the scan can fail before that happens
<bigjools> also I want to make it allow a few more failures on builders before killing them
<jml> bigjools: isn't that last one just changing a constant somewhere?
<bigjools> no
<bigjools> the current code just checks that one is bigger than the other
<bigjools> jobs hopping around builders failing need to be culled quickly.  builders  need a bit more leeway
<wgrant> I think we need to be more careful about that, though.
<wgrant> Since network issues can kill lots of builds that way.
<bigjools> yes, I know, which is why builders need more leeway
<deryck> Morning, all.
<Ursinha> lifeless, running every 5 minutes now
<LPCIBot> Project devel build (126): STILL FAILING in 3 hr 44 min: https://hudson.wedontsleep.org/job/devel/126/
<LPCIBot> Launchpad Patch Queue Manager: [r=lifeless][ui=none][bug=631301] Prune CodeImportEvents.
<wgrant> The PQM regex doesn't actually confirm that there's a commit message?
<LPCIBot> Project db-devel build (79): STILL FAILING in 4 hr 8 min: https://hudson.wedontsleep.org/job/db-devel/79/
<Ursinha> good morning launchpad
<jml> done
<wgrant> You finished Launchpad?
<jml> Yeah, the end boss is tough
<jml> actually, I finished reviewing the new buildd-manager code
<wgrant> !!
<jml> not as thoroughly as I'd review a normal branch, of course, but thoroughly enough.
<wgrant> I feel that the branch could have a better name.
<wgrant> It's a bit deceptive.
<jml> yeah
<jml> lazr.config.interfaces.ConfigErrors: ConfigErrors: schema-lazr.conf does not have a initialisedistroseries section.
<wgrant> jml: That was fixed in devel r11715
<wgrant> 11712, even.
<jml> ahh right.
<jml> hasn't made it to bigjools' branch then
<bigjools> I've not merged devel today
<bigjools> jml: pushed a new one up
<jml> ta
<bigjools> "make check" is still screwed. Grar.
<mars> bigjools, 'make check' is deprecated.  We will not be fixing it.
<bigjools> mars: wtf?
<bigjools> so how do I run tests locally?
<mars> bigjools, bin/test is the recommended way.
<bigjools> mars: provided running it with no args runs all the tests, then fine.
<bigjools> but I am intrigued as to why make check hangs half way through.
<mars> bin/test runs everything.  IIRC there was no obvious reason behind the test hang.
<mars> especially since bin/test had no issue
<bigjools> indeed
<jml> mars: why not delete it now?
<mars> jml, the make check target?
<jml> mars: yes
<mars> it is still used by automated systems
<bigjools> what does ec2/buildbot use?
<mars> same
<mars> 'make check'
<bigjools> hmmm then I am quite concerned
<mars> ?
<mars> about what?
<bigjools> why does it work for those and not locally, basically
<bigjools> jml: I don't understand why you think some of the comments are obsolete in this branch.
<jml> bigjools: hmm. gimme a sec and I'll explain why.
<mars> jml, I should not have used the word 'deprecated'.  'make check' is not deprecated, but its use is not recommended for developer systems.  It is only intended for automated ones.
<jml> bigjools: e.g.
<jml> >     def _startCycle(self):
<jml> >         # Same as _startCycle but the next cycle is not scheduled.  This
<jml> >         # is so tests can initiate a single scan.
<bigjools> jml: that one is ok
<bigjools> jml: it's the "# Trap known exceptions..." one
<bigjools> anyway I like lots of comments.  Comments are rarely bad unless they're misleading.
<jml> bigjools: ahh right. What I mean there is that though the words are correct, it's incomplete, and the traceback/no-traceback thing isn't very interesting
<bigjools> I think it is :)
<bigjools> bitter experience of coming back to my own code 1 year later
<jml> bigjools: in that case, fix the grammar :)
<jml> bigjools: "Trap all errors. If it's a known error, log it; if it's not known, log it and include a traceback."
<jml> bigjools: but the main point of the review comment is that _scanFailed does much more now.
<bigjools> I love seeing a well placed semicolon.  I'm still searching. :)
<bigjools> indeed, I'll spruce it up
<jml> jelmer: is there a PPA somewhere with a version of testr with streaming result support?
<jelmer> jml: Not that I'm aware of. Rob has done all of the testrepository packaging so far. AFAIK there are only packages in Debian and Ubuntu's main archive.
<jelmer> It might be nice to set up a recipe build for testr.
<jml> jelmer: yes
<cody-somerville> jml, What you ever used twill? If so, whats your opinion of it?
<cody-somerville> err
<cody-somerville> *Have
<jml> deryck: hello
<deryck> hi jml.  on call
<jml> deryck: can we have a call sometime this week to talk about UDS?
<sinzui> jml, mumble?
<jml> sinzui: yep
<jml> sinzui: waiting
<jml> cody-somerville: never used twill.
<jml> sinzui: hello?
<gary_poster> deryck: I decided to push bug 659697 back to you.  Maybe I'm being dense about our options, but right now I don't see a clear foundational solution, and even if it were, I think the degree we care about this use case is also an application-level decision.  Tell me what you think when you get around to it. :-)
<_mup_> Bug #659697: [Wish] On-the-fly decompression of compressed attachments <Launchpad Bugs:New> <https://launchpad.net/bugs/659697>
<deryck> jml, sure, I'd love to chat.  Early as possible this week actually.  Meant to ping already.
<deryck> jml, can you calendar me a time that works for you?
<deryck> gary_poster, I was wondering at first if it was apache settings to get content-type and content-encoding right so the brower would do the work....
<jml> deryck: sure thing
<deryck> gary_poster, but having poked briefly at it, I guess the change has to happen in the librarian server itself.
<deryck> jml, thanks!
<gary_poster> deryck: right, and if I understood correctly, the proposal was to always unzip, and then let the server do gzip encoding if the browser supports it
<deryck> gary_poster, right.  As I understand it, too. :)
<deryck> gary_poster, so I'm happy for bugs to own it, but like you, I doubt we'll make it a priority anytime soon.
<deryck> I didn't know with the librarian being a shared resource who should own it either.
<deryck> the whole "own" thing is silly, I know.
<gary_poster> which...seems questionable, from the perspective of someone who feels fuzzy about the use cases.  My concerns would be how many browsers don't accept gzip, and how many people would *want* their browsers to display an arbitrarily-sized attachment rather than downloading it...
<gary_poster> (sorry, that was a follow on from my first statement)
<deryck> right, I agree with that, too.  Maybe we shouldn't do it.
<deryck> aribitrary files are much different that static resources of css or js which other sites gzip and decompress on the fly.
<gary_poster> right
<gary_poster> but so, even if you kicked something off to us for implementation, my concerns were whether we *should* do it from an application/usability perspective.  Which seemed to be Not Me. ;-)
<deryck> heh
<deryck> gary_poster, I didn't think about the arbitrary size issue before.  You having said that makes me agree.
<gary_poster> ok cool deryck :-)
<deryck> I'm not even sure the alternate link idea is worth the effort either.  How much trouble is it letting the browser dialog guide you?
<LPCIBot> Project devel build (127): STILL FAILING in 3 hr 49 min: https://hudson.wedontsleep.org/job/devel/127/
<LPCIBot> * Launchpad Patch Queue Manager: [r=jtv,lifeless][ui=none][bug=652280]
<LPCIBot> * Launchpad Patch Queue Manager: [r=gmb][ui=None][bug=644196] Allow derivers to create a derived
<LPCIBot> distroseries via the API.
<gmb> Dear LPCIBot: Please stop setting off my highlight.
<jtv> And mine.
<maxb> lpcibot sould send notices, not messages
<jml> and we should have a landing system that doesn't require reviewer nicks in the first line of the commit message
<LPCIBot> Project db-devel build (80): STILL FAILING in 3 hr 44 min: https://hudson.wedontsleep.org/job/db-devel/80/
<LPCIBot> Launchpad Patch Queue Manager: [rs=buildbot-poller] automatic merge from stable. Revisions: 11735,
<LPCIBot> 11736, 11737, 11738 included.
<jml> g'night all
<bigjools> night from me too
<sinzui> jcsackett, how many hours do you estimate it would take you to update that JoinNotAllowed raises a 403 over the API https://bugs.edge.launchpad.net/launchpad-registry/+bug/244527
<_mup_> Bug #244527: JoinNotAllowed should cause a 400 response when raised on the API layer <api> <Launchpad Registry:Triaged> <https://launchpad.net/bugs/244527>
<jcsackett> sinzui: the exception just needs to be moved into lp.registry.errors and have a webservice code assigned to it.
<sinzui> jcsackett, 2 hours?
<jcsackett> assuming i don't get into import hell and the tests are easy, probably 1-2, yes.
<sinzui> okay. thanks
<lifeless> moin
<lifeless> Ursinha: awesome
<mtaylor> moin lifeless
<lifeless> hi mtaylor
<LPCIBot> Project devel build (128): STILL FAILING in 3 hr 35 min: https://hudson.wedontsleep.org/job/devel/128/
<LPCIBot> * Launchpad Patch Queue Manager: [r=jelmer][ui=none][no-qa] Extract a few refactorings and tests from
<LPCIBot> the abandoned check-in-wadl branch.
<LPCIBot> * Launchpad Patch Queue Manager: [r=gary][ui=none][no-qa] update lazr.restful to the latest released
<LPCIBot> version (0.14.0)
<LPCIBot> * Launchpad Patch Queue Manager: [r=michael.nelson, stevenk,
<LPCIBot> julian.edwards][ui=none][bug=653720] Commit after changing build
<LPCIBot> status in the upload processor.
<lifeless> bac: hi
<gary_poster> Is this kind of thing expected when you try to view MPs on staging?  If not, is it indicative of staging's librarian having fallen over?  That's my impression of the traceback a skim of the pertinent code: https://lp-oops.canonical.com/oops.py/?oopsid=OOPS-1752S202
<lifeless> mmm
<lifeless> no, its a 404
<lifeless> it means 'the librarian is running but couldn't get your shit'
<lifeless> now the staging librarian is /meant/ to fallback, at least for public stuff, to the main librarian.
<gary_poster> oh, just saw reply
<jam> abentley: ping
<jam> mwhudson: ping about lp-service
<abentley> jam: pong
<mwhudson> jam: ping
<mwhudson> jam: ah crap, i didn't reply in the end did i?
<mwhudson> jam: basically i think it looks ok
<jam> mwhudson: I updated the test suite, I'm not sure if it is fully tasteful, but the test suite passes for me
<jam> abentley: want to skype? I responded to your email, but it may be faster to iterate vocally
<mwhudson> jam: using fixed paths to communicate between subprocesses is (sadly) how it's done in lp tests currently
<jam> mwhudson: le sigh
<abentley> jam, sure.  I don't see your reply yet.
<jam> abentley: just sent, may take a couple mins
<mwhudson> jam: lifeless is fixing that right now though :-)
<jam> yeah, I've been reading the discussion on the parallel running, etc.
<mwhudson> jam: he also suggested that you might want to make your helper class subclass Fixture
<mwhudson> to make the transition away from layers that smidgeon easier
<jam> mwhudson: I'd be happy to if I understood Fixture. Can you give any pointers there?
<mwhudson> jam: http://pypi.python.org/pypi/fixtures/0.3.2
<flacoste> lifeless: hi
<mwhudson> jam: oh yeah, at the atexit calls are a bit crazy
<lifeless> flacoste: hi
<flacoste> lifeless: skype?
<lifeless> yes, its hanging there o connecting
<mwhudson> jam: i posted another comment on the mp
<wgrant> abentley: The chroot *does not provide security*.
<wallyworld_> morning
<abentley> wgrant: elmo appears to believe otherwise, because he didn't have a problem with it being possible to run that code inside a chroot on a non-VM machine.
<wgrant> elmo: We install build-deps as root. Are the buildd kernels somehow protected from usual chroot escapes?
<rockstar> mars, gary_poster, FYI, I've seen your emails, and will make sure you have a reply in the morning.  Is that okay?
<gary_poster> rockstar: absolutely.  thank you
<gary_poster> rockstar: we're intending to have hands ready to work on this on Wednesday, so you know.
<LPCIBot> Project db-devel build (81): STILL FAILING in 4 hr 7 min: https://hudson.wedontsleep.org/job/db-devel/81/
<mwhudson> Conflict adding file lib/canonical/launchpad/apidoc.  Moved existing file to lib/canonical/launchpad/apidoc.moved.
<mwhudson> what *again*?
<benji> mwhudson: that's my fault; a fix will be comitted momentarily
<mwhudson> benji: heh, i don't know if it's worth fixing once the deed is done
<benji> there are other bits that *are* worth fixing :/
<mwhudson> hey guess what
<mwhudson> there's a blueprints pagetest that's misnamed and not run by default (i think)
<mwhudson> surprise surprise it fails
<mwhudson> ah maybe not
<thumper> testfix mode :-(
* thumper changed the topic of #launchpad-dev to: Launchpad Development Channel | Week 1 of 10.11 | PQM is open | firefighting: - | https:/â/âdev.launchpad.net/â | Get the code: https:/â/âdev.launchpad.net/âGetting
* lifeless changed the topic of #launchpad-dev to: Performance Tuesday | Launchpad Development Channel | Week 1 of 10.11 | PQM is open | firefighting: - | https:/â/âdev.launchpad.net/â | Get the code: https:/â/âdev.launchpad.net/âGetting
<LPCIBot> Project devel build (129): STILL FAILING in 3 hr 34 min: https://hudson.wedontsleep.org/job/devel/129/
<LPCIBot> Launchpad Patch Queue Manager: [r=edwin-grubbs][ui=none][bug=612754] remove the trailing whitespace
<LPCIBot> as
<LPCIBot> part of the newline canonicalization when checking the GPG signature of
<LPCIBot> incoming email
#launchpad-dev 2010-10-19
<lvh> hi
<lvh> I'm trying to configure my lp instance for remote access, and I'm in the /etc/hosts file
<lvh> is this rule supposed to remain unchanged?  127.0.0.77      vostok.dev archive.vostok.dev
<lifeless> have you looked on the wiki
<lifeless> Running/VM
<lifeless> I think
<lifeless> covers some sutff
<lvh> I hadn't, because I'm running this on a physical machine :-)
<lifeless> has the same issues though
<lifeless> note that a dev environment isn't suitable for production use (in case thats what you're trying to do)
<lvh> I'm experimenting, but that was the eventual plan, yes
<lifeless> hosts files are local - you'll need actual DNS, ssl certificates, front end servers etc
<lvh> Unless I use ssh forwarding, I still get unstyled main pages that say: Lost something?
<lvh> Thereâs no page with this address in Launchpad.
<lvh> (For the front page.)
<lifeless> you'll need a production config, lazr schema, slave db etc too
<lifeless> we don't really document how to run production LP - its not something that was envisioned when we open sourced it (note too that you /must/ rebrand it - the look n feel is not opened)
<lifeless> you may want to record the things you learn on the wiki somewhere.
<lifeless> I don't know why you'd get Lost something pages, unless you're bypassing your front-end apache
<lvh> I'm not trying to kick any shins, I'm fine just using ye olde bzr + roundup...
<lifeless> sorry, I can't parse that
<lvh> (Is there a canonical acronym for LAMP, but for developing software instead of serving webapps?)
<lvh> lifeless: I'm not trying to piss people off by running my own instances of lp that aren't for developing lp.
<lvh> lifeless: If that's something you don't want people doing, that's fine -- unfortunate, since that means I have to set up like twenty things instead of one, but I'll survive :-)
<lifeless> its fine to run your own instance, thats clearly something the AGPL permits; What you can't do is use the launchpad branching - the stock icons, widget set - because thats (IIRC) trademarked.
<lifeless> lvh: I am curious why you want to run your own instance rather than use launchpad.net (but its not really my business :))
<persia> Doesn't that just prohibit use to the public?
<persia> (but ask counsel for detailed use restrictions on trademarks)
<lifeless> persia: https://dev.launchpad.net/LaunchpadLicense
<lvh> lifeless: IANAL but I did the relevant parts of a law degree
<lvh> (Enough to know nobody really understands the AGPL ;-))
<lifeless> I wasn't part of the open sourcing process; its a bit neither-fish-nor-fowl at the moment.
<lifeless> it would be nice to move the theming into a dedicated tree and just say 'whats in tip, is fully agpl, go free'.
<lifeless> but there's some implications for test and development
<lvh> lifeless: Mostly because I'm researching development stacks at the moment
<lvh> lifeless: for closed-source stuff
<lifeless> lvh: sure. Are you aware that launchpad.net supports closed-source stuff ?
<lvh> lifeless: Yes
<lifeless> great
<lvh> lifeless: I think their pricing system is wrong ;-)
<lifeless> anyhow, I hope I've helped with the issue you're seeing
<lifeless> lvh: have you given that feedback to mrevell ?
<persia> lifeless, Makes sense.  I simply don't know enough of the relevant precedent to know if it's enforceable in cases where it's undiscoverable.  Given that it's an explcit restriction on license, rather than just a trademark claim, I'm no longer tempted to believe it's a public/private thing (which is the common trademark model)
<lvh> lifeless: Yeah.
<lifeless> lvh: ok, cool
<lvh> lifeless: Something along the lines of "I see your point, but it works for some people, and we're not changing it anyway"
<lifeless> mmm
<lifeless> nothing is unchangable
<lvh> lifeless: (I believe you should limit *people*, not *projects*)
<lvh> Or at least not primarily projects.
<lifeless> (because?)
<lvh> (And about a week after I blogged about that, Atlassian went out and did it with Bitbucket)
<lvh> lifeless: Well, practical example
<lvh> lifeless: Small interdependent pieces of code: good, big monolithic pieces of code: bad, right?
<lifeless> bitbucket does closed source by user, not product ?
<lvh> lifeless: bitbucket lets you make infinite repos
<lvh> closed or open
<lifeless> yeah, I can see the argument.
<lifeless> it makes sense to me.
<lvh> lifeless: however, if you want to *share* those repositories, that's when it costs you
<lvh> for 5 or below, it's 0, etc
<lifeless> afterall users are what incur costs, not small bits of metadata.
<lvh> right
<lvh> plus everything else pretty much bills you per user
<lvh> the cutoff point is pretty high too
<lvh> 250/yr, let's say 20/month
<lvh> that gets you 25 users at bitbucket, which isn't a trivial two-bit operation anymore
<lvh> and they get infinite projects
<lvh> (I realise my comparison is flawed because bitbucket doesn't have merge proposals and their bugtracker is useless, but you get the basic idea)
<lifeless> yeah
<lifeless> we're primarily here for open source though
<lifeless> somewhat different concerns
<lvh> well, that's true, but
<lvh> the closed source stuff is so I *can* work on open source stuff for free
<lvh> I still gotta eat at some point in time
<lifeless> sure
<lvh> (it actually says that explicitly in the notary ... not sure what the correct english term is. deed? document-that-starts-a-company. One of the goals of the comapny is to contribute to open source projects)
<lifeless> cool
<lifeless> articles of incorporation
<lifeless> constitution
<lifeless> depends on the country
<lvh> yeah, I'm from belgium, country-ish things are vague and fleeting
<lifeless> hmm, the quake a wee bit back was a 5.0
<lifeless> http://www.geonet.org.nz/earthquake/drums/
<poolie> <persia> Doesn't that just prohibit use to the public? (but ask counsel for detailed use restrictions on trademarks)
<poolie> this is kind of fud-ish
<poolie> i know it's not meant maliciously
<persia> poolie, It's structurally built into things: it's published that anyone providing legal advice who is not so licensed (where I live, where you live, and many other places) can be (successfully) sued by any counsel without such comments.
<poolie> well, don't provide legal advice then
<poolie> if i say "how do i set up gpg" and you tell me "ask counsel for detailed use restrictions" it may be technically true but it's not very welcoming
<poolie> do you see what i mean?
<persia> That's kinda different.  I may be overcareful when it comes to disclaimers when talking about licenses, trademarks, patents, etc., but I don't think it's better to not talk about them.
<poolie> i think it's better not to say something vaguely worrying
<persia> Which is the worrying bit?
<poolie> the "ask counsel" bit
<persia> I guess.  Similar to IANAL.  Anyway, off-topic, and taking to /query
<poolie> yeah, i guess it is just "ianal"
<poolie> to me, it's a more alarming way of saying it, but i see what you mean
<mwhudson> lifeless: hey, did you figure out your testtools error?
<lifeless> no
<lifeless> no idea at all
<mwhudson> lifeless: i mean "Subject: [Launchpad-dev] Fwd: Test results: testtools => devel: FAILURE"
<mwhudson> lifeless: i just had a WAG
<lifeless> \o/
<mwhudson> basically <storm expr> == anything is true?
<mwhudson> so perhaps some change in testtools now tests types, or something?
<mwhudson> i.e. the tests were already bong
<lifeless> could be
<lifeless> I'll look more closely; thanks.
<mwhudson> lifeless: hmm, doesn't look likely actually
<mwhudson> but it's still worth checking i guess
<mwhudson> lifeless: i was right, the test is broken
<mwhudson> lifeless: and the reason your branch exposes it is that some change in testtools flips the order of the comparison
<mwhudson> (Pdb) p operator.eq(matchee, matcher.expected)
<mwhudson> False
<mwhudson> (Pdb) p operator.eq(matcher.expected, matchee)
<mwhudson> <storm.expr.Eq object at 0xcc17d90>
<mwhudson> \o/ ?
<lifeless> hah
<wgrant> Heh.
<mwhudson>         self.assertEqual(UTC_NOW, branch.next_mirror_time)
<mwhudson> sigh
<mwhudson> should be
<mwhudson> self.assertSqlAttributeEqualsDate(branch, 'next_mirror_time', UTC_NOW)
<mwhudson> i guess
<mwhudson> lifeless: i guess you can fix that in your branch?
<lifeless> otp
<mwhudson> k
<mwhudson> the other failure is probably something similar
<thumper> are we still importing the with statement from the future?
<lifeless> no need
<thumper> so we are 2.6 everywhere now
<thumper> ?
<lifeless> yes
<lifeless> no
<thumper> w00t
<lifeless> I lied
<thumper> was there an email?
<lifeless> future still
<lifeless> yes
<wgrant> There were a few emails, all contradicting each other.
<lifeless> thread with mars and me
<thumper> so...
<lifeless> from __future__ is still needed
<thumper> still future
<thumper> ok
<wgrant> The PQM chroot?
<wgrant> Or is there something else?
<LPCIBot> Project db-devel build (82): STILL FAILING in 3 hr 44 min: https://hudson.wedontsleep.org/job/db-devel/82/
<LPCIBot> * Launchpad Patch Queue Manager: [r=abentley][ui=None] Add database patch for branch merge queues
<LPCIBot> * Launchpad Patch Queue Manager: [r=rockstar][ui=none][bug=662664] Ensure recipe builds run only under
<LPCIBot> Xen
<lifeless> mwhudson: thanks for looking into that
<mwhudson> lifeless: s'ok -- i hate mysteries :)
<thumper> wild stab in the dark, but I thought "bin/test -m lp.code.windmill" would only run the code windmill tests
 * thumper thinks
<thumper> actually I think there is a bug layer one there
<thumper> canonical.testing.layers.LayerIsolationError: Test didn't reset the socket default timeout.
<thumper> anyone seen that running windmill tests?
<lifeless> http://i.imgur.com/IOBPq.jpg
<thumper> I wonder how many page loads were needed to get the right ad :)
<mwhudson> :)
<thumper> ERROR DISABLED: Test left new live threads: [<Thread(Thread-3076, started daemon 140448791578384)>] ?!?
<wallyworld_> thumper: bin/test -vvt code.windmill.tests has been working for me
<wallyworld_> but yours should have worked too i guess
<wallyworld_> i'm also seeing the live threads stuff
<lifeless> bombs away
<wgrant> An interesting proposal.
<lifeless> 3 minutes, you're running slow
<wgrant> Shh.
<lifeless> ;)
<lifeless> wgrant: any more feedback than that ?
<wgrant> lifeless: I think it's a good idea to try it.
<wgrant> As long as the scope is clear.
<wgrant> But I haven't really got time to think about it too much at the moment.
<mars> wgrant, if you are talking about lifeless' reviews experiment: the scope is 'anything that requires craftsmanship', which basically means even a single line of code: SQL, Python, or otherwise.
<lifeless> mars: hi
<lifeless> mars: I think you're projecting
<lifeless> mars: I realise my examples can be categorised that way, but its not what I meant.
<lifeless> s/project/something/
<mars> lifeless, yes, but as wgrant pointed out, drawing the line can be difficult.  I just pointed out a minimum point at which the line may be drawn.  Hopefully discussion will take us beyond that.
 * mars signs off for the night - bye!
<rockstar> wallyworld_, I'm tempted to just say "forget the test, just land the code"
 * persia suspects that is a recipe for summoning for certain well-meaning over-knowledgeable folk
<rockstar> persia, then I punt to them.  :)
<lifeless> rockstar: whats the question ?
<LPCIBot> Project devel build (130): STILL FAILING in 3 hr 56 min: https://hudson.wedontsleep.org/job/devel/130/
<wallyworld_> rockstar: i'd still like to know what's going on
<wallyworld_> lifeless: the issue is that some ajax stuff works fine but fails under windmill
<lifeless> ugh
<lifeless> that sucks
<wallyworld_> yep. basically Y.io(...) is just plain failing
<lifeless> wallyworld_: btw
<lifeless> bug 663068
<_mup_> Bug #663068: Remove hard coded test urls <launchpad-foundations> <Launchpad itself:New for wallyworld> <https://launchpad.net/bugs/663068>
<lifeless> the tag is unusual
<lifeless> normally bugs are put on the launchpad-foundations *project* with separate tags ;)
<wallyworld_> bugger. i stuffed that one up. sorry
<lifeless> no worries
<lifeless> I've fixed
<wallyworld_> thanks
<lifeless> but I thought something might have got lost in translation
<lifeless> so letting you know
<lifeless> bug 663068
<_mup_> Bug #663068: Remove hard coded test urls <paralleltest> <Launchpad Foundations:New for wallyworld> <https://launchpad.net/bugs/663068>
<lifeless> see^
 * wallyworld_ files mental note
<lifeless> I think it would be nice to have all bugs on the launchpad project
<lifeless> but thats not the current practice ;)
<wallyworld_> well, it would make it easier perhaps
<lifeless> mwhudson: looks interesting vis a vis our discussion earlier - http://pypi.python.org/pypi/withrestart/0.2.7
<lifeless> not sure its tasteful though ;)
<adeuring> good morning
<lifeless> jelmer: hi
<lifeless> jelmer: the thing you CP'd. what rev of trunk is it? (by CPing you've kind of blocked all deployments till we catch up, now that we're on the new process)
<lifeless> jelmer: I want to assess what we should do now
<lifeless> hmmm
<lifeless> 9pm
<lifeless> time to write some code
<lifeless> [one of those days]
<jml> good morning
<bigjools> morning
<lifeless> hmmm, leaked config dirs
<jml> lifeless: any traction on your testtools => devel fail?
<lifeless> some
<lifeless> its a storm expr being used in the test
<lifeless> so its dependent on the magic ordering of ==
<jml> lifeless: oh wow.
<lifeless> we need to rewrite those tests, shallowly, or thoroughly
<jml> lifeless: ok
<lifeless> mwhudson dug into it, and phrased it as broken.
<lifeless> :)
<jml> lifeless: I retract any offer of help that my enquiry might have implied :P
<lifeless> hah!
<lifeless> jml: did you see https://bugs.edge.launchpad.net/launchpad-foundations/+bug/662519 ?
<_mup_> Bug #662519: lp test startup exhausts dev/random <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/662519>
<jml> no
<jml> but I love that title :)
<lifeless> it will add some hilarity to your day
<jml> of all of the problems that I would have guessed we had...
<lifeless> jml: well, the thing about surprises
<bigjools> wgrant: hello
<wgrant> bigjools: Hi.
<bigjools> wgrant: evening.  I'm trying to test your fix for bug 629921
<_mup_> Bug #629921: Archive:+packages with empty name search does like '%%' search. <pg83> <qa-needstesting> <timeout> <Soyuz:Fix Committed by wgrant> <https://launchpad.net/bugs/629921>
<wgrant> bigjools: Navigate to the page ++oops++, filter to Superseded packages, verify that the query isn't insane.
<bigjools> wgrant: it's timing out
<wgrant> ... hah.
<bigjools> I don't need a ++oops++ :)
<wgrant> Declivitous?
<bigjools> yes
<wgrant> It sounds negative.
<bigjools> it means "sloping down steeply"
<wgrant> Ah.
<bigjools> we're half way through the development
<wgrant> Optimistic!
<bigjools> not really
<wgrant> bigjools: I have the first three lucilleconfig-elimination branches proposed, if you'd like to eyeball them.
<wgrant> I still need to work out what to do about the germanium temproot issue, though.
<bigjools> wgrant: I don't have the time right now - I can look probably Thursday if you can sit on it a while
<wgrant> bigjools: Sure.
<wgrant> No urgency.
<bigjools> yeah :)
<bigjools> I am doing QA on your other publisher change at the moment
<wgrant> Ah, thanks. Sorry for leaving that to you, but I can't really do it myself yet...
<bigjools> indeed
<bigjools> right now you do increase my workload quite a lot :)
<jml> https://code.edge.launchpad.net/~jml/launchpad/codebrowse-port-config/+merge/38812 â need a review
<jml> it's part of getting qastaging up
<lifeless> +1
<jml> lifeless: heh heh
<jml> lifeless: thanks
<jml> lifeless: can you please rubber stamp for landing tools.
<lifeless> I did
<bigjools> wgrant: the next thing you might want to look at that would be most beneficial if you're feeling like doing more work is to do that query speedup in the domination
<bigjools> the PPA publishing cycle is slow :/
<wgrant> bigjools: Yeah, I need to do that. But I want to try to fix the actual query.
<bigjools> wgrant: fix?
<wgrant> bigjools: To not use a temp table.
<jml> lifeless: ta
<bigjools> wgrant: there's nothing wrong with that
<wgrant> bigjools: A big part of the publisher slowness (at least locally) is the regular cache purging.
<wgrant> Removing that cuts a primary run from >1min to <10s.
<bigjools> are you talking about the storm cache?
<wgrant> (every (distroarchseries, pocket) domination clears the Storm cache twice)
<bigjools> right
<bigjools> the first question is - why does it need to flush it
<wgrant> I'm not sure hwo dumb our cache eviction strategy is.
<wgrant> It may not evict at all.
<bigjools> I'm sure lifeless will have something useful to contribute :^)
<lifeless> beware the tides, wait oh what ?
<lifeless> storm cache depends on the cache thats installed
<lifeless> we have a custom one
<lifeless> I don't recall the details about when its installed
<lifeless> how does the sql temp table relate to storm cache flushing?
<bigjools> I no approximately nothing
<bigjools> jeez
<bigjools> know*
<wgrant> lifeless: I don't think it should relate, besides being in roughly the same piece of code.
<bigjools> lifeless: this is something that cprov may have talked to you about in the past when we started using storem
<lifeless> ok, so I was confused ;)
<lifeless> bigjools: which this do you refer to ?
<lifeless> wgrant: so are we talking storm performance or the query ?
<bigjools> lifeless: caching on  the publisher
<wgrant> The publisher drops caches.
<wgrant> Frequently.
<lifeless> ok, as a correctness thing I guess? you're changing values in the db
<bigjools> lifeless: see judgeAndDominate() inside lib/lp/archivepublisher/domination.py
<bigjools> which calls clear_cache()
<wgrant> That code isn't *too* painful any more.
<wgrant> So you won't go insane from reading it.
<wgrant> I don't think it's correctness.
<wgrant> We create a temp table, read in a list of publications from it, then use Storm operations on those.
<bigjools> lifeless: I've no idea why it needs to flush_database_updates() then clear_current_connection_cache() then gc.collect()
<bigjools> I have a vague recollection of it blowing memory into small pieces
<wgrant> bigjools: Because it wants to take 20 minutes :D :D :D
<bigjools> yeah
<wgrant> I don't see why it would be dealing with a particularly huge number of objects.
<lifeless> the conection cache so cccc is store.invalidate
<wgrant> At worst it should be a few thousand during an autosync run.
<lifeless> blah late night typing
<lifeless> so you could do this in a more direct form
<lifeless> drop the store object
<lifeless> put a new one in its place
<lifeless> wgrant: I agree - but - numbers will help.
<lifeless> lets get peak memory utilisation logged (sys.utimes or whatever)
<bigjools> it is logged
<lifeless> great
<lifeless> hows it look at the moment?
<bigjools> actually - maybe it's not logged for domination
<bigjools> ah it's deathrow that does it
<bigjools> it uses process_in_batchers
<lifeless> bah
<bigjools> sigh
<lifeless> librarian hates
<bigjools>  process_in_batches
<lifeless> psycopg2.OperationalError: FATAL:  database "launchpad_ftest" does not exist
<wgrant> bigjools: Doesn't lots of publisher stuff use that?
<wgrant> Or is PublishingTunableLoop a separate thing?
<wgrant> There are a few things like that.
<bigjools> wgrant: just deathrow and ftparchive it seems
<wgrant> Ah, yeah.
<bigjools> when overriding, for the latter
<wgrant> Not for generating file lists too?
<bigjools> ah yes
<wgrant> Yeah.
<wgrant> I've been in that code a bit lately...
<bigjools> :)
<bigjools> lucky you
<wgrant> Hey, deleting crappy code makes a good break from finishing my last two projects evaar.
<bigjools> I am running the ubuntu publisher on DF at the moment, when it finishes in about an hour or so we'll see if your other fix worked
<bigjools> if the publisher code is a break, I'd hate to see what your projects look like
<wgrant> bigjools: The file list generation really takes most of that time?
<wgrant> Not a-f itself?
<wgrant> I do not understand :(
<bigjools> wgrant: domination is SLOW
<wgrant> bigjools: The three minutes it takes on production is also slow. But I guess your definition is different.
<bigjools> dogfood is spethial
<lifeless> jml: hey, whats 'stnadard' to log something in twistd
<bigjools> 2010-10-19 09:38:42 DEBUG   Sorting packages...
<bigjools> 2010-10-19 09:43:39 DEBUG   Dominating packages...
<bigjools> wgrant: ^
<wgrant> bigjools: Ah, so only twice production's slowness.
<wgrant> Not so bad.
<jml> lifeless: log.msg(), where log = twisted.python.log
<lifeless> jml: nothing gets logged
<lifeless> jml: I guess the log only shows after the tac is parsed?
<jml> lifeless: that sounds vaguely familiar
<jml> lifeless: but I don't know for sure.
<lifeless> I wonder how I can easily make this easier to debug
<jml> lifeless: also, our tachandler nonsense might interfere with early logging
<bigjools> wgrant: filelist generation is the slowest part on dogfood it seems.  Probably production too.
<wgrant> bigjools: Better than before cprov fixed it a year ago...
<wgrant> bigjools: Prod file list calculation seems to take just a couple of minutes.
<wgrant> Then a-f takes ~15 minutes.
<wgrant> Debian recently parallelised a-f calls.
<LPCIBot> Project db-devel build (83): STILL FAILING in 3 hr 54 min: https://hudson.wedontsleep.org/job/db-devel/83/
<deryck> Morning, all
<jml> bigjools: btw, now that the first cut of testtools twisted support has landed, I'm blocking on the mega-branch for converting all our trial tests & deleting the twisted layers
<bigjools> jml: sweet
<bigjools> jml: are you going to make the change so I can merge it?
<jml> bigjools: no, I'm going to wait until your branch lands before I try to land on trunk
<bigjools> ok
<jml> bigjools: Launchpad doesn't have the testools twisted support yet, it's only landed in testtools trunk.
<wgrant> bigjools: Has dogfood blessed me yet?
<bigjools> wgrant: hahaha no
<wgrant> :(
<bigjools> 2010-10-19 09:50:16 DEBUG   Calculating binary filelist.
<bigjools> 2010-10-19 11:16:49 DEBUG   Iteration 9 (size 10000.0): 518.693 seconds
<jml> bigjools: I'm working on the existing twisted tests in LP in a branch now, working out the kinks with my perfect-in-theory twisted support code
<bigjools> 2010-10-19 11:25:25 DEBUG   Iteration 10 (size 10000.0): 516.320 seconds
<wgrant> ...
<wgrant> That is utterly pathetic.
<bigjools> dogfood is SLOW
<bigjools> jml: coolio
<wgrant> Maybe qastaging will be better.
<lifeless> wgrant: qastaging is on the staging machine, for now.
<lifeless> same db server, same appserver
<wgrant> Well, I don't know how fast asuka is these days.
<lifeless> 450 milliparsecs
<wgrant> But it can't be quite mawson-slow.
<bigjools> lifeless: less than twelve parsecs, surely.  :)
<lifeless> indeed
<lifeless> hhmm, nearly 1am
<lifeless> psql -l | grep launchpad_ftest_[\[:digit:\]] | awk '{ print $1 }' | xargs -n1 dropdb
<lifeless> ^ folk will need that till we shake the bugs out of the parallel stuff :<
<jml> lifeless: bah, kiwis are nocturnal anyway
 * jml is off to lunch & errands
<lifeless> I'm amazed some of this stuff ever worked
<lifeless> I hate globals.
<lifeless> there, I said it.
<persia> Needed to be said, really.
<lifeless> lib/canonical/lp/__init__.py takes the database name found at process startup and caches it indefinitely.
<persia> *choke*
<lifeless> that interacts badly with changing it in the test runner.
<wgrant> There are a lot of similar pieces of evil.
<lifeless> wgrant: check this out
<wgrant> And then you check their history and find they were added in early 2005 and probably haven't even been looked at since.
<lifeless> sqlbase.connect_string
<lifeless> config.rw_main_master
<lifeless> then lp.dbname
<lifeless> which is always set...
<lifeless> to dbconfig.main_master
<wgrant> lifeless: There used to be a non-main store.
<lifeless> yes
<lifeless> but its circular
<lifeless> scripts options stash it there
<lifeless> and then its read back
<wgrant> Ah.
<lifeless> so a dynamic config system becomes a static variable
<lifeless> facepalm
<wgrant> I'm glad you've found rationale sufficient to justify destruction of this madness.
<wgrant> Albeit at 1am.
<wgrant> bigjools: https://edge.launchpad.net/ubuntu/+source/icecc/0.9.6-1/+build/2000412 is FAILEDTOUPLOAD but has no upload log.
<wgrant> Is this meant to happen?
<wgrant> ('tis breaking scripts)
<bigjools> wgrant: yes - jelmer fixed stuff that was stuck in UPLOADING
<lifeless> \o/ appserver running up against launchpad_ftet_22437
<wgrant> bigjools: Ah, so the old stuff got set to FAILEDTOUPLOAD without a log?
 * wgrant makes the scripts cope.
<bigjools> yes - there was no log available.
<wgrant> :(
<bigjools> it was only 200 or so
<bigjools> annoying, but unavoidable because of the bug
<wgrant> Yup.
 * bigjools puts the finishing touches to the new buildd-manager and rejoices
<bigjools> assuming jml likes my changes of course :)
<lifeless> jml: when you return
<wgrant> What does it actually change?
<lifeless> my databasefixture branch has a couple of somewhat nontrivial alterations added to it
<wgrant> Makes dispatching properly asynchronous?
<lifeless> jml: ^ - would appreciate another glance please.
<bigjools> wgrant: yes
<bigjools> wgrant: I still need to make file fetching asynchronous but that's for another branch and day
<bigjools> the most important thing is that the code is now vastly simplified
<wgrant> Right.
<bigjools> I'm particularly pleased that all the error handling is done in one place now
<gmb> mars: Did you land that test helper for setting / unsetting feature flags?
<gmb> ISTR you mentioning it in an email thread ages ago but I can't find any details.
<LPCIBot> Project devel build (131): STILL FAILING in 3 hr 51 min: https://hudson.wedontsleep.org/job/devel/131/
<cr3> hi folks, is there a convention for separating names consisting of more than one word in launchpad urls? lets say I have a new target called "foo bar", should the url be "foo-bar", "foo_bar", "foobar" or is the convention to find whatever way to just have one word?
<jml> lifeless: will do.
<jml> bigjools: I'll take a look after looking at lifeless's branch
<bigjools> jml: I found some test failures caused by the most recent revision, ping me before you look as I'm fixing them now
<jml> bigjools: will do.
<bigjools> jml: ok it was a trivial piece of pebkac.  However I've got a weird error when it tears down the layers, have you got any idea what's wrong here? http://pastebin.ubuntu.com/516258/
<jml> bigjools: something is screwing with the layers. otp. things to try include checking for recent changes to devel that aren't in stable.
<bigjools> righto
<jml> (or bad revisions you've merged in from devel)
<bigjools> oh the shame
<mars> is the use of the rubber-stamp 'rs' review documented anywhere?
<jml> mars: if you can't find it, then the answer is "functionally, no"
<mars> jml, I did not even know what it meant until Deryck mentioned it in his listmail.  I thought it was just some funny typo people used from time to time.
<jml> bigjools: maybe r11734 is a culprit?
 * jml otp again
<bigjools> jml: I'd lay a wager on it
<jam> abentley: I think you have some private lp branches available. I ran your script, and only see 4.3k branches
<abentley> jam: quite possible.  As a member of the code team, I have read/write access to everything.
<LPCIBot> Project parallel-test build (6): STILL FAILING in 2 hr 7 min: https://hudson.wedontsleep.org/job/parallel-test/6/
<bigjools> jml: latest devel made it go away.  The branch is ready for your perusal.
<gmb> mars: Did you land that test helper for setting / unsetting feature flags?
<gmb> ISTR you mentioning it in an email thread ages ago but I can't find any details.
<gmb> I'm writing tests at the moment and the whole ignore = getFeatureStore().add(...) dance is horrendous.
<mars> gmb, no, the branch has two small failing tests in it.  I have to make time to fix them
<mars> gmb, :)
<gmb> mars: Which branch is it? I can try to take a look at the failures if you'd like.
<mars> gmb, sure!  lp:~mars/launchpad/add-profiling-feature-flag
<mars> gmb, the failing tests are: bin/test -cv lp.services.profile
<mars> gmb, just two simple ones IIRC
<mars> gmb, I changed the way that the profiling instruments know that they should be turned on and measuring the request.  The old code had some tests for 'instruments on' and 'instruments off', and they broke when I changed the way that the on/off switch works.
<gmb> mars: Okay, I'll take a look.
<mars> gmb, the new code for the flags helper and fixture is in lp.services.features.helpers
<gmb> Cool.
<mars> make that lp.services.features.testing
<abentley> jam, I've updated https://dev.launchpad.net/Code/BranchRevisions but it doesn't list you as an email recipient.
<jam> abentley: thanks. I had added "BranchRevisions" but maybe it needs a full match rather than a subset
<abentley> jam, anyhow, I'd really appreciate your feedback on it.
<jml> bigjools: are you intending anything more for this branch?
<bigjools> jml: nothing planned
<jml> bigjools: reviewed
<bigjools> jml: thanks, will look shortly
<jam> abentley: I'll have some more interesting numbers for you in a couple minutes
<jam> I ran the script and grabbed the revisions for all those branches
<abentley> jam, cool.  Hard data is good.
<jam> so at least for the ~4k branches I can see, I can tell you how the db will hold it
<jam> abentley: so the import is done, how would you like the numbers?
<jml> huh
<jml> how did that happen
<abentley> jam,  maybe the best place is the BranchRevision page?
<jml> bigjools: fwiw, I'm going to be off IRC all tomorrow
<jam> abentley: sure
<jam> abentley: updated, and you weren't listed as notified
<jam> (neither was I, but I'm hoping that is because it is me)
<abentley> jam, perhaps I should subscribe :-)
<abentley> jam, to calculate the BranchRevisions, take the distance-to-null of the branch tips and sum them.
<jam> abentley: it is the full ancestry
<jam> not the mainline
<jam> gdfo would be a decent approximation, but still low
<abentley> jam, my bad.
<jam> I'm just going to do a pass across the data, find the ancestry of each rev
<jam> and then sum those
<jam> dang, I have to do set operations because 2 parents don't "add" cleanly. but I'll get there
<abentley> jam, so it looks like the ratio of branches to dotted_revno_rows improves as the number of branches increases.
<jam> abentley: possibly. I think it depends a lot of the branches themselves
<jam> A long-lived branch will be 'bigger'
<jam> and can certainly skew the results more when there are fewer total branches
<abentley> jam, true.
<abentley> I get 410:1 with merged, 1030:1 with abandoned, 1219:1 with experimental.
<abentley> jam, it would be interesting to sort by the revision date.  That should give a rough idea how it scales over time.
<abentley> jam, I'd like to hear whether you think my algorithms under "use case analysis" make sense.
<jam> abentley: so by my numbers, BranchRevision for those 4k branches is 328M rows
<jam> your logic seems ok
<jam> I'm not sure that walking 'backwards' using mainline_parent is the best thing to do
<jam> specifically, as a branch changes over-time more entries will end up in the mainline_parent table
<jam> we'll need a way to remove old entries, etc.
<jam> that table wasn't very big, so I wasn't worried about it accumulating cruft
<abentley> jam, walking backwards when implementing a ParentsProvider, for example?
<jam> Well I say "backwards" I should be saying "forwards" (in the opposite direction of normal)
<jam> the ParentsProvider seems good
<abentley> jam, okay.
<jam> the 'most relevant branch' is tricky
<abentley> jam, yeah.  We might be looking at changing it so we cache the most relevant branch on the revision, and only need to calculate it if the branch goes away.
<jam> abentley: I think that is a good start, though AIUI it can change
<jam> specifically, if I push up a merge of your branch before you push up the branch itself
<abentley> jam, true.
<abentley> jam, the whole thing is a bit questionable.  The most important thing we need it for is calculating revision karma, which is only done once.
<jam> if you are willing to start with "Branches owned by the user in this given project" then you can start with the known tips and walk backwards
<abentley> jam, I have to meet someone for  lunch now.  TTYL.
<jam> later
<rockstar> Someone from foundations, I could really use your help.
<rockstar> Er, help figuring out what's going on with the librarian.
<mars> gary_poster, ^ ?
<jml> rockstar: what's the problem?
<rockstar> jml, http://pastebin.ubuntu.com/516392/ - that's usually what I expect when the librarian is still up.
<rockstar> But I don't see it in ps, and bin/kill-test-services appears to be broken.
<jml> rockstar: that message seems to indicate that the librarian is *not* running.
<jml> "Librarian has been killed or has hung." and then later "[Errno socket error] [Errno 111] Connection refused" â¦ looking at the layer code, that's coming from a failed urlopen(foo).read()
<rockstar> jml, yeah, we don't seem to be very good at error messages.
<sinzui> flacoste, mumble?
<rockstar> jml, nevermind.  It seems fixing bin/kill-test-services and then running it has fixed it.
<rockstar> jml, meaning that there was probably crap left over from the last librarian instance.
<jml> rockstar: I think there's a bug filed somewhere about it leaving its pid file around after dying
<rockstar> jml, ah, okay.
<rockstar> jml, I think that bug's been around for a while, and we've been using bin/kill-test-services to workaround it.
<rockstar> But when that script is broken, then it's a little harder to sort out.  :)
<LPCIBot> Project devel build (132): STILL FAILING in 3 hr 54 min: https://hudson.wedontsleep.org/job/devel/132/
<cr3> hi folks, sorry for asking again, is there a convention for separating names consisting of more than one word in launchpad urls? lets say I have a new target called "foo bar", should the url be "foo-bar", "foo_bar", "foobar" or is the convention to find whatever way to just have one word?
<rockstar> cr3, we mostly use foo-bar
<deryck> All my ec2 mails complain of failure when it's windmill errors due to threads.  Should I open a bug about this?
<jml> deryck: probably yes, but I'd also check StevenK's recent emails about windmill threading errors on the list
<deryck> jml, ok, will do.
<lamont> jml: so if someone claims something has landed on launchpad/db-stable (which I'm gathering == launchpad/staging?), then is it fair to assert that it should also exist on lp:launchpad/devel?
<jml> lamont: no, it's not.
<lamont> that explains why it fails that assertion
<lamont> jml: historically, launchpad-buildd has developed/released directly from /devel, since it's not really part of launchpad, etc, etc.  I'm wondering what else would break in breaking from that tradition, or if I should go have the person who landed his branch on /staging also land it on /devel
<jml> lamont: branches that require db changes land on db-develâdb-stable, after we've rolled out a db update, db-stable gets merged into devel.
<lamont> ah, interesting.  given that we tested this without db changes, I expect that he should have landed it on /devel instead then, yes?
<jml> lamont: probably
<lamont> or additionally, or whatever
<jml> lamont: but I couldn't say for sure
<lamont> can I propose someone elses branch for merging?
<jml> lamont: you can indeed
<cr3> rockstar: thanks, might you have an example I could use as precedent?
 * lamont goes to dig around a bit
<jml> lamont: fwiw, splitting buildd into its own branch & project is one of my many Launchpad someday/maybes.
<rockstar> cr3, not off the top of my head, no. Sorry.
<jml> anyway, I have to go now. have a great evening all.
<lamont> https://code.edge.launchpad.net/~abentley/launchpad/detect-xen/+merge/38867 <-- rockstar, wanna approve that beasty?
<rockstar> lamont, I reviewed that already. It looks like abentley landed it into db-devel (which is lp:launchpad)
<lamont> right.  and I need it landed on /devel
<rockstar> lamont, why?
<lamont> because that's where launchpad-buildd releases from, and since it's an as-released thing when it merges, merge conflicts absolutely suck, since it means that the source in lp is wrong (relative to what's actually running)
<lamont> rockstar: given that the two branches appear to be divergent, it's less work for me to just keep releasing from /devel rather than migrating the releases over to db-stable
<rockstar> lamont, okay.
<lamont> esp since the release process comes down to me changing debian/changelog and telling bigjools/whoever to "hey please land this, it's live, kthx"
<rockstar> lamont, I'm going to go on the assumption that abentley landed it in db-devel on accident, and approve this.  If this assumption is wrong, is quite likely that abentley will thump me.
<rockstar> (I just want you to know the risk I'm taking)
<lamont> rockstar: understood.  tell him I made you do it.
<lamont> 'twas tested without db changes, so I'm gonna assert that it doesn't need to be on db-stable
<rockstar> lamont, I know abentley well enough to know he won't accept excuses.  :)
<lamont> of course, the descriptions of the branches make even less sense, since db-stable claims to be "development" and /devel makes no such claim
<lamont> rockstar: my fall back is to just tell him to land it there and let him wait another day to get his code rolled out.
<lamont> > Please land your branch - after the merge pain of times past, I've learned
<lamont> > not to deploy from anywhere other than mainline launchpad/devel.
<lamont> The branch has landed on db-stable.
<rockstar> lamont, devel is to db-devel as stable is to db-stable
<lamont> so stable is for the existing db, and devel/db-devel is for the "we need a db upgrade" stuff?
<rockstar> lamont, no, things get landed into devel, and when buildbot blesses them, they get pulled into stable.  db-devel then merges stable, and when things get blessed by buildbot, they get pulled into db-stable
<rockstar> lamont, so, if the buildd code ever gets tests, you'll want to at least deploy from stable, since you know all the tests pass there.
<lamont> ah, yeah. tests would be cool
<lamont> we've been doing the "develop and test on dogfood, smack it into devel" process.
<lamont> sadly, sometimes devel moves during that process, and then I cry a little
<rockstar> abentley, could you please review https://code.edge.launchpad.net/~rockstar/launchpad/fix-kill-test-services/+merge/38870
<lamont> rockstar: and good to know, thanks.  I've mostly been flying blind wrt branches
<rockstar> abentley, I can chat whenever you can.
<lifeless> morning
<LPCIBot> Project db-devel build (84): STILL FAILING in 3 hr 57 min: https://hudson.wedontsleep.org/job/db-devel/84/
<lifeless> deryck: ping
<deryck> hi lifeless
<lifeless> henninge: hi
<lifeless> deryck: will come back to you :)
<lifeless> henninge: I haven't checked all my mail yet, but as we have only a brief overlap
<lifeless> henninge: ah, echannel
<deryck> lifeless, ack :-)  I'll be here awhile longer.
<mwhudson> lifeless: re http://pypi.python.org/pypi/withrestart/0.2.7, yeah, i've done something like that too, but as it's not part of the normal protocol in python programs it's kinda useless
<lifeless> mwhudson: yeah
<mwhudson> i gave a talk at .... accu? about this once
 * mwhudson hunts for slides
<mwhudson> http://www.nhplace.com/kent/Papers/Condition-Handling-2001.html <- the paper i was talking about
<lvh> is there a reasonable way of uninstalling laucnhpad after having been installed with rocketfuel-setup
<mwhudson> i guess reading the script and undoing what it does
<mwhudson> it's not _that_ bad for system pollution is it?  some /etc/hosts changes, an apache site, a ppa + some packages installed
<abentley> rockstar: chat?
<rockstar> abentley, yes please!
<lifeless> lvh: rm -rf / ?
<jam> abentley: would you be interested in a copy of the sqlite db? It is about 33MB after bzip2
<lvh> mwhudson: Yeah, no, it's not very terrible
<abentley> jam: sure.
<lvh> mwhudson: I was just hoping there was a rocketfuel-unsetup
<mwhudson> lvh: nope
<abentley> jam: I don't actually understand what to do to make history-db work locally.
<rockstar> lvh, launchpad is like the Hotel California. You can never leave.
<lvh> Yaay.
<lvh> at least I'm glad I didn't do it on my development box then
<jam> abentley: #1 install it as a bzr plugin ("bzr branch lp:bzr-history-db ~/.bazaar/plugins/history_db)
<lifeless> deryck: ok, hey
<deryck> lifeless, hey hey
<lifeless> I'm wondering if you'd be up for a voice chat
<lifeless> deryck: ^
<deryck> lifeless, I don't mind a voice chat.  but having to work and watch my youngest while Wendy picks up my oldest.
<deryck> so it's loud here. ;-)  Working with headphones and music while I eye the little on watching Sponge Bob :-)
<lifeless> if that means some interrupts during a call, its fine by me
<deryck> lifeless, ok.  it's kind of loud, too, and I can't leave the room.
<deryck> but I'm game if you are.
<lifeless> lets give it a shot
<lifeless> you have skype?
<rockstar> lvh, for context, I run launchpad in a chroot.
<lifeless> ah yes, you do :)
<deryck> lifeless, yup.  firing it up now.
<lvh> rockstar: I ran it in a VM but it was dead slow
<lvh> rockstar: turns out the reason why wasn't really related to it being ran in a VM
<lvh> so I ran it on a spare physical machine
<abentley> rockstar: https://dev.launchpad.net/Code/BranchRevisions
<mwhudson> wow, https://dev.launchpad.net/Code/BranchRevisions suggests that half of the rowns in the BranchRevision table are from Launchpad
<mbarnett> wgrant: ping.
<gary_poster> hey sinzui, is there a way to add members to teams if the member AJAX thing is timing out too much?
<sinzui> gary_poster, yes, the underlying view
<jkakar> Hi. :)
<jkakar> I continue to be basically unable to use any popup dialog in Launchpad.  They all timeout for me, almost all the time.
<jkakar> I've tried edge and not-edge, same result.
<jkakar> I've tried Firefox and Chrome, same result.
<jkakar> Hopefully all my attempts are creating timeout-juice that will bring these issues up on some report or another. :)
<sinzui> gary_poster, append +addmember to the team url
<jkakar> As a workaround I've been using /+request-review on merge proposals and the bug task disclosure triangle thing on bug reports.
<jkakar> sinzui: Ah, thanks!
<gary_poster> wunderbar sinzui, thank you
<jkakar> sinzui: I should have looked at the status bar, d'oh.  Anyway, it works, thanks a lot.
<sinzui> jkakar, gary_poster. EdwinGrubbs is doing SQL analysis right now to fix the issue
<jkakar> sinzui: Wicked, thanks!
<gary_poster> yay sinzui, thank you :-)
<gary_poster> and yay EdwinGrubbs, I should say
<sinzui> jkakar, gary_poster, last week I used an API script to set a user.
 * jkakar hugs sinzui
<gary_poster> :-)
<thumper> morning
<EdwinGrubbs> sinzui: on that topic, do you know why there is a Person.account and an EmailAddress.account foreign key?
<sinzui> EdwinGrubbs, : gary_poster:  historical baggage. I believe we intend to remove both
<gary_poster> EdwinGrubbs: step 5 of https://dev.launchpad.net/LEP/OpenIdRoadmap
<EdwinGrubbs> sinzui: if those are both removed, how can I tell if a person is active? Do I just check that Person.merged is false?
<sinzui> EdwinGrubbs, that is one reason why person.account still exists
<gary_poster> EdwinGrubbs: maybe I misunderstand the question, but part of # 5 is "Remove all Account and https://dev.launchpad.net/EmailAddress records not linked to a Person, reducing the number of Account records on the system by 90%."
<gary_poster> So I would think that the answer is that, when we drop the EmailAddress.account, it will no longer be an issue.
<sinzui> EdwinGrubbs, 18 months ago we wanted to separate account and person and host SSO. 6 months ago it was decided that we do not want SS0 and account separation cause too many bugs
<gary_poster> If I'm wrong, we should probably clarify that in that wiki page, and the associated bug
<EdwinGrubbs> sinzui, gary_poster: so for the time being can I assume that I want to check whether an person is active by looking at Person.account.status as opposed to Person.email.account.status, which the vocab is using now.
<sinzui> EdwinGrubbs, excellent question!
<EdwinGrubbs> I'm waiting for an excellent answer.
<sinzui> Edwin. before Account email address status=4 was the way we knew a user was active. SSO wrongly lets users with any email status to login
<gary_poster> um
<sinzui> EdwinGrubbs, ^ So account is the only way to check, but the who authentication system is untrust worthy see the daily oopses and weep
<gary_poster> sinzui, is this one of the balls I have dropped?  If so, please take this opportunity to kick me in an appropriate direction
<gary_poster> appropriately helpful to getting something fixed, is what I had in mind :-)
<sinzui> EdwinGrubbs, It is *still* our policy that all users must have a preferred email address and we want that check to suffice. We have dozens of checks in the code that require the email address.
<sinzui> gary_poster, yes, the ball dropped
<sinzui> EdwinGrubbs, Most cases where we pick out a person is to send notification...email. We know the user must have a preferred email. The mailing code does not check account. it knows not to send email to any user without a preferred email address
<rockstar> wallyworld__, thumper, stand up?
<thumper> rockstar: wallyworld__ won't be up yet
<thumper> rockstar: did you want to talk to me?
<gary_poster> sinzui, not clear yet, but I have call.  will ping you later to make sure I understand
<sinzui> gary_poster, I said this in the user question I answered two weeks ago for you. No user should be permitted to authenticate if there is no email address
<sinzui> The None error in +editemails and openid url errors that we see every day in oopses are when authentication falls over
<EdwinGrubbs> sinzui: when a person is deactivated, is the preferred email removed as well as the account being deactivated?
<sinzui> EdwinGrubbs, yes
<sinzui> Edwin ResetPassword used to fix the address. That does not happen any more.
<EdwinGrubbs> sinzui: ok, so it should be safe for me to check if a person has a preferred email to determine whether it is a valid person in the vocabulary. It would only cause problems for users that have active accounts but no preferred email address.
<sinzui> EdwinGrubbs, I think another way of saying that is that the other users are broken and there is not point in pretending that they belong in ValidPersonVocabulary
<EdwinGrubbs> ok
<sinzui> EdwinGrubbs, I assume teams are handled differently
<sinzui> EdwinGrubbs, is this issue about ValidPersonVocabulary?
 * sinzui wonders if that object really exists
<lifeless> gary_poster: hi
<gary_poster> lifeless on call, ping
<gary_poster> I mean will ping you
<lifeless> gary_poster: wondering if you wanted a catchup
<lifeless> yeah sure
<lifeless> I'll be here ;)
<sinzui> EdwinGrubbs, ValidPersonVocabulary must only contain users with preferred email addresses. Deactivated and Suspended users have NEW addresses. Unactivated users have NEW addresses too.
<wgrant> mbarnett: Hi.
<mbarnett> wgrant: heya.  i have a question for you, but probably better in pm.
<rockstar> thumper, I was talking with abentley, but my mental clock was an hour ahead.
<thumper> ok
<deryck> Ok, I'm out now.  Later on, everyone.
<abentley> jam, yes, I had already done step 1.
<lifeless> StevenK: it would be awesome to gather diskio and cpu use
<lifeless> I wonder if there is a plugin
<jam> abentley: after that you can do something like "bzr history-db-create --db foo.db -d branch"
<poolie> lifeless: hi there
<poolie> hi thumper, abentley, deryck
<poolie> and jam
<abentley> poolie: hi.
<thumper> hi poolie
<jam> hi poolie
<poolie> lifeless: it occurred to me this morning that filing RTs for things we don't intend to do soon is a kind of lean waste...
<StevenK> lifeless: Over the life of the test run? ENOIDEA
<abentley> jam, ISTM that we would get the best performance from mainline_parent_range if we added to existing ranges where possible.  Do you agree?
<jam> abentley: if you add to an existing one, what about the old tip?
<jam> the code as written, does try to always fill out full groups
<jam> so if it would chain a 3 + 70 it creates a new 73 length group
 * StevenK goes back for a lie down, his headache creeping back
<abentley> jam, the old tip is still part of the range, but no longer the tip.
<lifeless> poolie: well, we want to do it now
<lifeless> poolie: we're resource constrained
<lifeless> poolie: so I agree, and disagree.
<jam> abentley: right, but you can't find the range if it is in the middle (well, not easily, or without indexing all of them)
<poolie> i think keeping track of things you want but don't have time to start is good
<abentley> jam, you convert the revision-id to a revision primary key, you look up the primary key in the mainline_parent table.
<poolie> on the whole i think starting them just a little bit is normally bad, unless it's going to facilitate back-burner percolation
<EdwinGrubbs> sinzui: yes, this is about the ValidPersonOrTeamVocabulary and the email address is ignored for teams.
<sinzui> EdwinGrubbs, I assume is uses merged is not null
<abentley> jam, you could even use the revision primary key as the mainline_parent primary key.
<abentley> jam, because in this scenario each revision appears in only one revision range.
<abentley> jam, and THAT implies that we could combine the revision table with the mainline_parent table.
<thumper> mwhudson: know what's happening with https://code.edge.launchpad.net/~jameinel/launchpad/lp-service/+merge/37531 ?
<EdwinGrubbs> sinzui: it checks that merged is null. Not null would mean it is merged.
<jam> abentley: the mainline parent table intentionally de-normalizes groups into 100-ranges. I don't really see how you can put that in the revision table
<mwhudson> thumper: it's bouncing off ec2
<thumper> ah
<sinzui> EdwinGrubbs, I guess I know enough to write such a vocab :)
<abentley> jam, this would mean that recent groups would have 1-100 revisions, but they would expand until full.
<EdwinGrubbs> sinzui: can you run these queries. Hopefully, this will be the last set. http://pastebin.ubuntu.com/516472/
<jam> abentley: I had originally designed it so that any branch tip would find itself in the mainline_parent_range table. you're probably right that it could be done differently
<jam> I think the index would be much bigger
<abentley> jam, you're probably right about the index being bigger, but I don't usually worry about that.
<sinzui> EdwinGrubbs, https://pastebin.canonical.com/38825/
<abentley> jam: btw, I successfully ran bzr history-db-create
<wallyworld__> morning
<abentley> jam, at least I now understand your objection to the "relevant branch" algorithm better.
<abentley> wallyworld__: morning
<wallyworld__> abentley: ready for standup whenever you guys are
<abentley> thumper: ready for standup?
<thumper> abentley: aye
<sinzui> \o/ I traced an oops to a spam attack.
<lifeless> :)
<LPCIBot> Project devel build (133): STILL FAILING in 3 hr 31 min: https://hudson.wedontsleep.org/job/devel/133/
<LPCIBot> * Launchpad Patch Queue Manager: [r=allenap][ui=none][bug=653122] BugSubscriptionSubscribeSelfView is
<LPCIBot> now a LaunchpadFormView.
<LPCIBot> * Launchpad Patch Queue Manager: [r=bac][ui=none][no-qa] Only set LPCONFIG if it wasn't set in the
<LPCIBot> environment.
<LPCIBot> * Launchpad Patch Queue Manager: [r=lifeless][ui=none][no-qa] Codebrowse host and port are now
<LPCIBot> configured with codebrowse.listen_host and codebrowse.port.
<gary_poster> thumper: about to get on phone again, but would love your thoughts on https://bugs.edge.launchpad.net/launchpad-foundations/+bug/662912 .  Is this a known issue--am I flailing uselessly? :-)
<gary_poster> rockstar: any chance to get a reply to mars' mail before EoD?
<gary_poster> lifeless: off phone, ready when you are
<_mup_> Bug #662912: staging librarian is broken for merge proposals <Launchpad Bazaar Integration:New> <Launchpad Foundations:Triaged by gary> <https://launchpad.net/bugs/662912>
<rockstar> gary_poster, uh, I sent it this morning.  Did you not get it?
<gary_poster> rockstar, no
<thumper> gary_poster: I'll take a look
<gary_poster> thanks thumper
<rockstar> thumper, okay, I'll send again.
<rockstar> wallyworld__, so, I didn't want to ask on the standup for fear of getting off on a tangent, but did you say your Y.io call worked in windmill when you used a post instead of get?
<flacoste> thumper: call?
<thumper> flacoste: yep
<wallyworld__> rockstar: i'm still gathering evidence
<wallyworld__> rockstar: it *appears* at this point that it *may* make a difference
<rockstar> wallyworld__, I frakkin' hate windmill.
<wallyworld__> rockstar:  i put it aside to work on the windmill issue - i'll let you know as soon as i make some more progress on it
<wallyworld__> i should say - the *other* windmill issue ;-(
<rockstar> wallyworld__, what's the other windmill issue?  I heard you talking about something with setUp.
<wallyworld__> rockstar: yes - some seemingly innocuous refactoring has broken one and only one windmill test
<rockstar> wallyworld__, wanna skype about it?  I'm curious.
<wallyworld__> rockstar: ok
 * mwhudson finds another way one of the tests that broke lifeless' testtools branch is broken
<mwhudson>         future_time = datetime.now(pytz.UTC) - timedelta(days=1)
<lifeless> rockstar: what do you hate ?
<mwhudson> spot the mistake!
<lifeless> mwhudson: -
<beuno> or rather, the delta needs an actual delta?
<lifeless> no
<lifeless> thats one day in the past
<mwhudson> yeah, such an innocent little character
 * beuno sits back down
<jcsackett> mwhudson: was that passing for some cases? i would think a day in the past when you expect a day in the future would screw everything up.
<lifeless> jcsackett: storm :)
<mwhudson> jcsackett: yes, but then the test was written in a way that couldn't fail
<mwhudson> which was the first thing i fixed :)
<jcsackett> mwhudson: ah, fun times. :-)
<jcsackett> lifeless: sadly, i haven't had enough experience with storm where just referencing it explains away things. :-)
<jcsackett> though i
<mwhudson> jcsackett: if X is a storm SQL expression, X == anything is another storm SQL expression
<jcsackett> i've had this exchange often enough to accept it as gospel. :-P
<jcsackett> aaah.
<jcsackett> dig.
<mwhudson> which is true when evaluated in a boolean context
<mwhudson> aka "operator overloading is bad"
<wgrant> aka wgrant broke Soyuz :/
<jcsackett> mwhudson: okay, and thus everything was passing.
<mwhudson> yeah
<mwhudson> jcsackett: lifeless has a branch which has the sideffect of swapping the order the comparison is done in in assertEquals
<mwhudson> this found a few cases where == was not symmetric...
<lifeless> maybe we should check both directions for Equals
<lifeless> what do you think
<lifeless> a == b and b ==a
<lifeless> a <b and b>=a
<lifeless> etc
<lifeless> might be a little surprising
<lifeless> well, with the boolean logic done right
<mwhudson> i'd be careful of mixing ordering in
<jcsackett> lifeless: at least one fork of rspec (the ruby testing system) does just that. catches some surprising things when people start monkeypatching.
<mwhudson> but checking a == b and b == a and a != b and b != a are all consistent might be good
<mwhudson> ah hah, zope proxies
<lifeless> say its not so
<mwhudson> bug 331919
<_mup_> Bug #331919: delegates() should provide __eq__  and __ne__ operators. <tech-debt> <lazr.delegates:Triaged> <https://launchpad.net/bugs/331919>
<mwhudson> lifeless: https://code.edge.launchpad.net/~mwhudson/launchpad/testtools/+merge/38896
<lifeless> wow, fixtures is getting 30-40 downloads over the course of a few adys
<lifeless> http://pypi.python.org/pypi/fixtures/0.3.1
<lifeless> mwhudson: lp reckons there are conflicts
<mwhudson> lifeless: odd, i guess i'll merge trunk
<wallyworld__> rockstar: for when you get back from class - got the windmill test working, had to move login from test method to setup phase
<LPCIBot> Project db-devel build (85): STILL FAILING in 3 hr 39 min: https://hudson.wedontsleep.org/job/db-devel/85/
<LPCIBot> Launchpad Patch Queue Manager: [r=allenap][ui=none][bug=617699] Add getter/setter property for
<LPCIBot> linking distribution/source_package to BugTrackerComponents
<mwhudson> lifeless: the conflicts must be in your branch though :p
<lifeless> probably ;)
<mwhudson> yeah, a Contains matcher was added
<mwhudson> which should be swiftly shuffled off to testtools too i guess
<lifeless> hah
<lifeless> yes
<mwhudson> anyways, pushing the merged branch now
<lifeless> sweet. bombs away
<lifeless> I think this is starting to look pretty nice as a test:
<lifeless> http://pastebin.com/tq43kxhW
<wgrant> I think you could use some more context managers.
<lifeless> wgrant: don't be a hater ;)
<lifeless> mars: BaseWindmillLayer looks like it tries to disable the new thread check anyhow
<mars> lifeless, ? looking
<lifeless> look for disable_thread_check
<lifeless> in layers.py
<mars> yep
<mars> already had it open
<mars> reading
<mars> hmm
<mars> lifeless, could it be it is working, but subunit is eating the "ERROR DISABLED" message?
<mars> lifeless, or it could be that the disabling is not working any more
<lifeless> yes
<mars> lifeless, is this functionality tested?
<lifeless> if error disabled is being 'print' output, subunit will a) munch it and b) get corrupted
<lifeless> mars: I think it has some tests, yes. Dunno how many
<mars> lifeless, ./testing/tests/test_layers.py has a 'test_slow_thread' test
<mars> no test verification though - I guess it relies on the testrunner running the test and blowing up if it doesn't work
<wgrant> test_slow_thread is one that often has a LayerIsolationError.
<lifeless> mmm, I suspect all these parallel test branches of mine will land as a clump :(
<mars> ok, frustrating - if the 'wait for live threads to exit' should be working.  It gives /lots/ of time for them to shut down
<lifeless> mars: how long
<mars> 10 seconds
<mars> 100 * 0.1 seconds
<lifeless> mars: stuff dealing with tcp can timeout after 5 minutes
<mars> so_linger - yeah :(
<lifeless> (or more, but 300 seconds is the common one)
<mars> thought of that
<mars> that would make sense with StevenK's stale request hypothesis
<mars> you know, if it is always roughly the same windmill test that has issues
<mars> then we should be able to pick apart the test and see what is generating the request
<mars> it would not be the first time an stale thread issue was solved this way
<mars> bugs had the same problem
<lifeless> mars: fwiw gary has agreed to have the check disabled entirely until the windmill issue is addressed
<lifeless> I need to file a bug and make the change.
<mars> bug 383615
<_mup_> Bug #383615: Spurious test failure in emailaddress.txt. <spurious-test-failure> <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/383615>
<mars> ack, no
<mars> wrong bug
<mars> bug 516781
<_mup_> Bug #516781: Intermittant failure of test_inline_subscriber <qa-ok> <Launchpad Bugs:Fix Released by deryck> <https://launchpad.net/bugs/516781>
<jam> mwhudson: Pushed up a new revision which should fix those things, I still get an atexit warning, but I think that is the existing code.
<mwhudson> jam: cool, i'll throw it at ec2 again
<lifeless> the atexit is transitional
<lifeless> while we fix layers
#launchpad-dev 2010-10-20
<LPCIBot> Project parallel-test build (7): STILL FAILING in 2 hr 9 min: https://hudson.wedontsleep.org/job/parallel-test/7/
<mars> wgrant, did I hear you say you could reproduce the windmill threading error locally?
<mars> or something similar
<lifeless> mars: bug 461760
<_mup_> Bug #461760: Shouldn't check for left-over threads between Windmill tests <story-windmill-layer> <Launchpad Foundations:Fix Released by bjornt> <https://launchpad.net/bugs/461760>
<lifeless> q
<mars> lifeless, thaks
<mars> thanks
<abentley> lifeless: when I have claimed a review, feel free to review it yourself, but please don't set the status "approved" without letting me get my 2 cents in.
<lifeless> abentley: ok, can do
<lifeless> I thought you had EOD'd
<lifeless> and curtis had addressed your point
<abentley> lifeless: thanks.
<abentley> lifeless: I have EOD'd.  Curtis should have also EOD'd by now.  I would have reviewed it tomorrow, before he clocks in.
<lifeless> did you have other unaddressed concerns?
<abentley> lifeless: I'm a little worried by what he wrote about the diff being a lot bigger due to alphabetical sorting.
<abentley> lifeless: It's a principle thing.  If I claim a review, it means I intend to do it.
<wgrant> mars: I have had several EC2 runs fail with a LayerIsolationError after test_slow_threads.
<wgrant> mars: It manifests itself as a "make check returned an error code, but no tests failed" sort of thing, which you have to dig through the subunit log for.
<lifeless> wgrant: mars: I can trigger that by just running it; OTOH I'm futzing with the infrastructure in my branch, so its not necessarily representative
<lifeless> abentley: I've acked your request; I'm sure you were doing the review.
<abentley> lifeless: I don't follow.
<lifeless> abentley: on the thing you're worried about, I've noticed many imports in lib/canonical were not normalised by edwins script
<lifeless> abentley: I've said 'ok, I won't set the status on reviews you've claimed'
<wgrant> mars: Now I think about it, my log in bug #661931 might well be that issue.
<_mup_> Bug #661931: make check sometimes fails on EC2 without test failures <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/661931>
 * wgrant checks.
<wgrant> Indeed it is.
<abentley> lifeless: you went on to discuss the proposal as if I was concerned about the specific state of the proposal, so I wanted to make it clear that the specific circumstances of the proposal weren't my motivation.
<lifeless> abentley: ok; I understood you to be making a generic request.
<lifeless> I was also interested if in the specific case I had missed something important to the review.
<rockstar> wallyworld__, awesome.
<lifeless> e.g. 'could I, if I had been the original reviewer, have missed something important'
<wallyworld__> rockstar: i don't know why it fails but the change brings the tests in line with how others have been written. it seems the flakiness on *my* system has been solved at least
<rockstar> lifeless, I hate windmill's random failures.
<rockstar> wallyworld__, yeah, the flakiness may or may not be fixed for other systems.
<abentley> lifeless: Aside from the "this is much bigger than before" thing, I'm not aware of anything more to check.
<wallyworld__> rockstar: i'm running it through ec2 to see what happens. consider it an experiment :-) i'll let you know how it goes
<lifeless> grrr zcml
<LPCIBot> Project devel build (134): STILL FAILING in 3 hr 32 min: https://hudson.wedontsleep.org/job/devel/134/
<LPCIBot> * Launchpad Patch Queue Manager: [r=allenap,
<LPCIBot> bac][ui=none][no-qa] extract some refactorings from the abandoned
<LPCIBot> check-in-wadl branch
<LPCIBot> * Launchpad Patch Queue Manager: [r=adeuring][ui=none][bug=126516] Refactor doctest for bugtask status
<LPCIBot> changes into separate unit test and documentation.
<LPCIBot> * Launchpad Patch Queue Manager: [r=rockstar][ui=none][incr][bug=663436] Automatically determine (and
<LPCIBot> actually enforce) the expiration date of an OAuth request token,
<LPCIBot> freeing up the expiration_date field to be used for other purposes.
<LPCIBot> * Launchpad Patch Queue Manager: [r=gmb][ui=none][bug=636060] Adds latest upstream release version to
<LPCIBot> the sourcepackage upstream portlet.
<LPCIBot> * Launchpad Patch Queue Manager: [r=rockstar][ui=rockstar][bug=652126] Moves statistic information on
<LPCIBot> the product branches view into the main page and out of the
<LPCIBot> sidebar, where it didn't really fit.
<lifeless> gary_poster: / benji: if either of you are still around
<lifeless> I'd love some comment/tip on bug https://bugs.edge.launchpad.net/launchpad-foundations/+bug/663620
<_mup_> Bug #663620: generate overrides writes to global state <paralleltest> <Launchpad Foundations:New> <https://launchpad.net/bugs/663620>
<lifeless> wgrant: aroud ?
<lifeless> hmm
<lifeless> file:///var/launchpad/sourcecode/pygettextpo/ needs an upgrade to 2a
<mwhudson> on ec2?
<lifeless> yeah
<lifeless> also
<lifeless> Updating bzr-loom to revision 48
<lifeless> Doing on-the-fly conversion from RemoteRepositoryFormat(_network_name='Bazaar repository format 2a (needs bzr 1.16 or later)\n') to RepositoryFormatKnitPack1().
<lifeless> This may take some time. Upgrade the repositories to the same format for better performance.
<lifeless> starting upgrade of file:///var/launchpad/sourcecode/bzr-loom/
<lifeless> Updating pygettextpo to revision 24
<lifeless> Doing on-the-fly conversion from RemoteRepositoryFormat(_network_name='Bazaar repository format 2a (needs bzr 1.16 or later)\n') to RepositoryFormatKnitPack1().
<lifeless> This may take some time. Upgrade the repositories to the same format for better performance.
<lifeless> starting upgrade of file:///var/launchpad/sourcecode/pygettextpo/
<mwhudson> making a new image with bin/ec2 update-image should work for that
<LPCIBot> Project db-devel build (86): STILL FAILING in 3 hr 34 min: https://hudson.wedontsleep.org/job/db-devel/86/
<LPCIBot> Launchpad Patch Queue Manager: [rs=buildbot-poller] automatic merge from stable. Revisions: 11747
<LPCIBot> included.
<wgrant> lifeless: Hi.
<lifeless> wgrant: hi, I was going to ask if you'd looked into a particular subsystem
<lifeless> but I found the damage
<lifeless> (tight coupling)
<wgrant> What's broken?
<lifeless> if you run a unit test that uses TestMailer in a config with an instance name other than testrunner
<lifeless> then it uses a real mailer
<wgrant> Hah.
<wgrant> lifeless: How are you going to deal with tests that touch the FS?
<wgrant> Identify all the relevant config values, push a new config overlay, and hope that everything uses the config?
<lifeless> wgrant: BaseLayer.config.add_section(...)
<lifeless> wgrant: + grep for the existing literal to find things that don't use the config
<lifeless> wgrant: its working so far :)
<lifeless> wgrant: want to help?
<wgrant> lifeless: The publisher currently stores its path config in the DB, but that will be fixed soon.
<lifeless> the publisher layer/fixture can just update that config
<lifeless> depend on the DatabaseLayer (it should anyhow0
<wgrant> True.
<lifeless> my databasefixture branch is sufficient for you to experiment with that
<lifeless> wgrant: when do you start?
<wgrant> lifeless: With Soyuz?
<lifeless> yeah, whatever that means :)
<wgrant> Heh.
<wgrant> Dec 13
<lifeless> cool
<mars> lifeless, have a moment for a question about testing?
<lifeless> of course
<lifeless> irc or voice
<mars> irc is fine
<lifeless> ... :)
<mars> lifeless, so - layers are bad, test suite setup and teardown are bad.  How then do I implement a fixture that must run for the duration of a test suite?  Like, for example, rewriting the BaseWindmillLayer.
<lifeless> ok
<lifeless> so the layers core implementation has some problematic aspects
<lifeless> we can get to a tolerable place without replacing it.
<mwhudson> fixtures + testresources!
<lifeless> I would do this.
<lifeless> Write a Fixture
<lifeless> have a layer which does
<lifeless> cls._fixture = MyFixture()
<lifeless> cls._fixture.setUp()
<lifeless> ...
<lifeless> cls._fixture.cleanUp()
<lifeless> at the relevant places
<lifeless> you can see this sort of thing emerging in my parallel test branches
<lifeless> e.g. lp:~lifeless/launchpad/databasefixture
<lifeless> the reset method on a fixture is what you'd call in testSetup
<mars> ah
<mars> suites can't do that, can they?
<mars> testSetup
<mars> you would have to use layers
<lifeless> testresources can do it
<lifeless> but I wouldn't try and mix n match
<lifeless> testresources is a complete replacement for layers
<lifeless> for bonus points, you could write an adapter from Fixture to Layer as a currying helper
<mars> ok, so that makes sense to me - that covers what we have, what is available, and why we do what we do now
<mars> cool
<lifeless> would you like me to describe this in a more permanent fashion somewhere?
<lifeless> I wasn't really going to bother until tackling removal of layers was a feasible do-it-soon kindof project
<lifeless> but if you're liking what you're seeing of fixtures, perhaps its worth writing it up earlier
<mars> lifeless, a message to the dev list would be permanent enough IMHO. And anyone else interested in test engineering will see it there.
<mars> lifeless, you explained it quite concisely
<lifeless> heh :)
<mars> heck, you could copy-n-paste from 22:55 through 23:00 and be done with it :)
<lifeless> the [dis]advantange of a more permanent record is that folk may have less context and desire more exposition
<mars> can't help that - people are going to start asking "what are fixtures?" and "why do layers suck?" at some point anyway.
<mars> what other question might people ask if they read such a message?
<lifeless> does it make coffee?
<lifeless> what does it look like in anger ?
<mars> that's a good one
<lifeless> one thing that fixtures doesn't do well yet is arbitrary object graphs - the delegation to a child fixture can't handle anything other than linear lists (setUp doesn't cleanly skip if already setUp)
<lifeless> totally fixable, I just need to port that logic from testresources
<mwhudson> lifeless: aaaaaargh, since you created your testtools branch it looks like other branches have landed that import them from lp.testing.matchers :/
<mwhudson> s/them/Startswith/
<mwhudson> oh maybe only one
<lifeless> mwhudson: perhaps I should have left a forwarder in place
<lifeless> mwhudson: I hope that folk will get in the habit of contributing straight to testtools and rolling a temp build of it for LP
<mwhudson> hard to say
<lifeless> ditto storm.
<lifeless> and other deps
<mwhudson> lifeless: btw, you could consider assigning this bug to yourself: https://bugs.edge.launchpad.net/launchpad-foundations/+bug/107371
<_mup_> Bug #107371: Make the test suite able to be run in parallel on a single machine <build-infrastructure> <feature> <test-system> <Launchpad Foundations:Triaged> <https://launchpad.net/bugs/107371>
<lifeless> mwhudson: heh, I wouldn't want to stop other folk working on it :)
<mwhudson> heh heh
<jam> mwhudson: I see that the test failed again, but those tests look unrelated to my work. And they are *all* windmill tests.
<mwhudson> jam: :(
<jam> do you want to just try again?
<mwhudson> i'
<mwhudson> i'
<mwhudson> bah
<mwhudson> i'm not sure i WANT to
<mwhudson> but i will try :-)
<jam> mwhudson: well, if you think they will fail again, you don't have to
<jam> I just wonder about windmill, etc
<lifeless> mwhudson: trivial changes to the databasefixture branch (I forgot to tell its tests that the librarianlayer now implied db access working)
<lifeless> mwhudson: do you care?
<mwhudson> lifeless: i'll take a very quick peek i guess
<mwhudson> oh, it was jml who reviewed this
<lifeless> mwhudson: oh yeah >< :)
<mwhudson> lifeless: i guess the comment in what is now testUploadsSucceed is a bit wrong?
<lifeless> garh, I thought I updated it
<mwhudson> maybe you did
<mwhudson> i was looking at the individual diffs
<LPCIBot> Project parallel-test build (8): STILL FAILING in 1 hr 32 min: https://hudson.wedontsleep.org/job/parallel-test/8/
<LPCIBot> * Robert Collins: Merge unique config improvements.
<LPCIBot> * Robert Collins: Fix config key.
<lifeless> no, I didn't
<lifeless> fixing
<mwhudson> lifeless: i think the branch is fine now
<lifeless> thanks for looking
<lifeless> ok,good news
<lifeless> stubs landing was broken, so we can fix windmill easily.
<lifeless> is there a bug open for the issue with windmill ?
<lifeless> anyone that wants ec2 to behave better, please review https://code.edge.launchpad.net/~lifeless/launchpad/threads/+merge/38908 :)
<StevenK> \o/
<StevenK> lifeless: Then reply to 'Failures on Hudson' on the -dev list?
<lifeless> StevenK: well, after pqm takes it
<StevenK> Ah, it's a work-around
<StevenK> .. somewhat
<lifeless> no
<lifeless> its the fix
<lifeless> the disable-check code path is currently still checking
<StevenK> It is that one-line?
<lifeless> yes
<StevenK> Oh bloody hell, rs=everyone ?
<lifeless> you can try it yourself
<lifeless> bin/test -vvt g.tests.test_layer
<lifeless> with and without
 * StevenK prods thumper towards that MP
<lifeless> jtv: ^
<StevenK> lifeless: Oh, just to troll for one second: When branching using bzr, when it says Estimate: x/y, it's not fair if both x and y increase at the same rate
<jtv> lifeless: ?
<lifeless> blame tree structures
<lifeless> jtv: one line patch to fix all the windmill thread remaining failures
<jtv> oh great thanks
<jtv> Need a review?
<StevenK> I've added my conditional stamp to it
<lifeless> https://code.edge.launchpad.net/~lifeless/launchpad/threads/+merge/38908
 * jtv looks
<wgrant> Does this mean that Hudson will magically stop spamming the channel?
<lifeless> wgrant: hopefully
<StevenK> No, it will now say it built sucessfully
<StevenK> Hopefully
<wgrant> Heh.
<StevenK> wgrant: You didn't reply to the thread on -dev about it? :-P
 * lifeless polls the page looking for jtv's stamp
<wgrant> StevenK: No. I'm glad it spams when it fails.
 * StevenK idly wonders how efficent it is for lifeless to loop in select()
<wgrant> StevenK: I will also invoke EBUSY in my defence.
<jtv> My stamp is there now, but I couldn't set a review type.
<lifeless> tossed straight at ec2
<StevenK> Awww, not even pqm?
<lifeless> sorry, pqm :)
<StevenK> Which still says nothing doing
<lifeless> ah we're in testfix again
<lifeless> fortunately this *is* a testfix.
<StevenK> But has lp.translations.windmill.tests.test_pofile_translate.POFileTranslatorAndReviewerWorkingMode.test_pofile_reviewer_mode actually been fixed?
<lifeless> that fix might now be, but the regular thread leak one will be :)
<LPCIBot> Project devel build (135): STILL FAILING in 3 hr 32 min: https://hudson.wedontsleep.org/job/devel/135/
<LPCIBot> * Launchpad Patch Queue Manager: [r=lifeless][ui=none][no-qa] Add support for logging SQL bind values
<LPCIBot> * Launchpad Patch Queue Manager: [r=mwhudson][ui=none][bug=624009][incr] Add a
<LPCIBot> BranchMergeProposalNeedsReviewEvent but keep the functionality
<LPCIBot> the same.
<LPCIBot> * Launchpad Patch Queue Manager: [r=mwhudson][ui=none][bug=624009][incr] Rename
<LPCIBot> MergeProposalCreatedJob to MergeProposalNeedsReviewEmailJob.
<LPCIBot> * Launchpad Patch Queue Manager: [r=rockstar][ui=none][bug=638637] Stop loading the entire ancestry of
<LPCIBot> a branch during the scan
<LPCIBot> * Launchpad Patch Queue Manager: [r=adeuring][ui=none][no-qa] Change permission name for
<LPCIBot> EditDistributionSourcePackage to launchpad.BugSupervisor.
<lifeless> wallyworld__:     def __call__(self):
<lifeless> +
<lifeless> +        result = []
<lifeless> is a bit unusual, it separates the function definition and body visually, which makes it harder to read.
<wallyworld__> lifeless: ECONTEXT
<lifeless> I realise you're experimenting
<lifeless> improved-broken-link-handling
<wallyworld__> i'm just writing up the description. looks like the email went out as sonn as a changed it from "onhold"
<lifeless> yes, it does that :)
<lifeless> tim is improving it
<wallyworld__> which class was that code from again?
<lifeless> fairly early in the diff
<lifeless> I've closed it already though - sorry
<wallyworld__>  lifeless: oh, you just want some white space added?
<lifeless> wallyworld__: removed
<lifeless> the blank line
<lifeless> this is very surface, I'm just mentioning cause it stood out in a casual glance
<lifeless> I'm happy to discuss the deeper stuff you're doing if you like (but not right now, am naffed)
<wallyworld__> lifeless: it's not in the diff i'm seeing, nor in the code i have locally unless i've missed something. i'll triple check again. whoever picks up the review can check as well
<lifeless> wallyworld__: the diff mailed out may have been stale
<wallyworld__> lifeless: could be. go have a beer or something. i'm sure it's past beer o'clock over there :-)
<LPCIBot> Project parallel-test build (9): STILL FAILING in 1 hr 27 min: https://hudson.wedontsleep.org/job/parallel-test/9/
<LPCIBot> * Robert Collins: With the Librarian depending on the database, having the layer is enough to be useful.
<LPCIBot> * Robert Collins: Update Librarian test expectations.
<LPCIBot> * Robert Collins: Import BaseLayer at the right place.
<wgrant> I like that branch.
<wgrant> It doesn't hide the author behind PQM.
<StevenK> wgrant: parallel-test looks at branch of lifeless', so they're all going to be him
<wgrant> Yeah, I know.
<wgrant> But it's better than
<wgrant> PQM
<wgrant> PQM
<wgrant> PQM
<wgrant> PQM
<wgrant> PQM
<rockstar> wgrant, Tarmac sets the author properly, and then sets committer as "Tarmac"
<StevenK> rockstar: Does Tarmac work faster than PQM? ;_)
<rockstar> StevenK, what's slow about PQM?
 * rockstar answers a question with a question
<StevenK> rockstar: 30 minutes for an answer back from PQM with "Okay, your branch is merged"
<wgrant> rockstar: That sounds like it might not be completely insane.
<rockstar> StevenK, yeah, but that's mostly a function of the pre-commit hook it runs.
<wgrant> Wasn't buildbot introduced as part of the 5-minute-PQM effort?
<wgrant> Did you just give up after reaching half-hour-PQM?
<rockstar> wgrant, yeah, PQM still wants to build a tree and such.  It spends most of its time cleaning up after itself, actually.
<spm> hrm. it used to be, when I started, that PQM+the entire LP test suite would happen in 90 minutes. it was when that creeped to ~ 2-2.5 hours, that 5 min pqm was introduced.
<spm> so I'm not sure the fault lies entirely with pqm here; rather what it's being asked to do, is what's taking so long.
<StevenK> spm: I just like digging at pqm since lifeless wrote it
<spm> roger roger
<spm> I just have to (roundabout) defend him every now and then. afterall, it's easier to stab someone in the back, if you're standing behind them. :-P
<rockstar> StevenK, AIUI, lifeless didn't write it, he just performs CPR on it every once in a while.  I think PQM pre-dates baz.
<lifeless> StevenK: I didn't write it :)
<rockstar> Frankly, I'm surprised that bzr is the only DVCS with something like robotic landings.
<LPCIBot> Project parallel-test build (10): STILL FAILING in 1 hr 27 min: https://hudson.wedontsleep.org/job/parallel-test/10/
<LPCIBot> Robert Collins: Update missed comment.
<adeuring> good morning
<gmb> Aaaaaargh.
<wgrant> gmb's back!
<gmb> :)
<gmb> lifeless: Is there some special dance that I have to do to use feature flags in pagetests or has no-one been stupid enough to try doing that yet?
<lifeless> gmb: should work fine
<gmb> Hmm.
<lifeless> be sure to cargo cult something appropriate
<gmb> Yeah. I'm working on it :)
<lifeless> mars has a branch adding a context manager
<gmb> lifeless: Yes, I've got that branch merged. I'll keep prodding things until it works.
<lifeless> have you added a rule
<gmb> At least I know I'm not (entirely) mad.
<lifeless> lib/canonical/launchpad/doc/librarian.txt is an example that I know works.
<lifeless> or
<gmb> Aha, lemme look...
<lifeless> lib/canonical/launchpad/doc/timeout.txt
<gmb> lifeless: Yep. Got it working now, thanks.
<lifeless> gmb: awesome
<lifeless> gmb: if you want feedback on your use, just shout
<bigjools> morning
<wgrant> Evening.
 * thumper waves
 * bigjools shivers
 * thumper gives bigjools the chills
<bigjools> yes, baby
<thumper> db-devel failing with lp.translations.windmill.tests.test_serieslanguages.LanguagesSeriesTest.test_serieslanguages_table
<wgrant> StevenK: Why would anything except the publisher care about how they're published?
<StevenK> wgrant: In relation to my comment?
<wgrant> StevenK: Yes.
<StevenK> wgrant: Since I can think of at least two other places that would benefit from it
<mrevell> Morning
<wgrant> Oh?
<StevenK> wgrant: IDS for one
<wgrant> Hm, I don't see why that would care.
<wgrant> Why would it care?
<StevenK> Because it copies from PRIMARY and DEBUG
<wgrant> But that's just because they're PRIMARY and DEBUG.
<wgrant> Not because they're a-f (which is also copy archives).
<StevenK> wgrant: If you add one there, the publisher can use and IDS can as well
<wgrant> StevenK: But IDS doesn't want this one.
<wgrant> It wants (PRIMARY, DEBUG), not (PRIMARY, COPY[, soon DEBUG])
<StevenK> Oh, duh
 * StevenK should learn to read
<wgrant> Heh.
<StevenK> wgrant: Oh, and I tried Evolution now that I'm on Maverick. It still hates me.
<wgrant> StevenK: Thanks.
<wgrant> StevenK: :(
<StevenK> However, Thunderbird has gotten scary
<wgrant> I haven't used it since 2.0.0.0 was fresh.
<StevenK> You type in your e-mail address and it guesses and probes for the IMAP/POP3/SMTP servers it should use.
<wgrant> Ah.
<StevenK> How do I get the test suite to print out all SQL that gets run?
<lifeless> LP_DEBUG_SQL
 * StevenK drowns in it
<LPCIBot> Project devel build (136): STILL FAILING in 3 hr 58 min: https://hudson.wedontsleep.org/job/devel/136/
<StevenK> lifeless: When did your windmill thread fix land?
<StevenK> lifeless: Ah ha. #136 contained your fix, but six windmill tests still failed
<lifeless> StevenK: with thread errors?
<StevenK> lifeless: Yes
<lifeless> frell
<StevenK> https://hudson.wedontsleep.org/job/devel/lastFailedBuild/testReport/junit/lp.bugs.windmill.tests.test_official_bug_tags_management/TestOfficialBugTags/test_official_bug_tags_management_2/ for example
<lifeless> well
<lifeless> theres a few possibilities
<lifeless> the layer may not be setting the flag correctly
<lifeless> something else may be wrong
<lifeless> https://hudson.wedontsleep.org/job/devel/lastFailedBuild/consoleFull
<lifeless> search for test_official_bug_tags_management
<lifeless> raise WindmillTestClientException(result['result'])
<lifeless> the _2 is just reporting that there are new threads
<lifeless> its not erroring *because* of them
<lifeless> however this is an issue
<lifeless> test: canonical.testing.tests.test_layers.TestThreadWaiting.test_slow_thread
<StevenK> So subunit is masking the failures?
<lifeless> successful: canonical.testing.tests.test_layers.TestThreadWaiting.test_slow_thread
<lifeless> error: canonical.testing.tests.test_layers.TestThreadWaiting.test_slow_thread [ multipart
<lifeless> StevenK: huh, no.
<lifeless> https://hudson.wedontsleep.org/job/devel/136/testReport/junit/lp.bugs.windmill.tests.test_official_bug_tags_management/TestOfficialBugTags/test_official_bug_tags_management/
<lifeless> thats the failure
<bigjools> this broken person-picker is really getting on my tits now
<StevenK> I see, it appears twice
<lifeless> the subunit fragment I just pasted is a broken api usage in the test environment
<lifeless> which is also an issue
<StevenK> lifeless: However, it is the only one that does appear twice
<lifeless> I'm not sure where to file that one; jml may know
<lifeless> successful: lp.bugs.windmill.tests.test_bug_also_affects_new_upstream.TestBugAlsoAffects.test_bug_also_affects_picker
<lifeless> test: lp.bugs.windmill.tests.test_bug_also_affects_new_upstream.TestBugAlsoAffects.test_bug_also_affects_picker
<lifeless> tags: zope:threads
<lifeless> error: lp.bugs.windmill.tests.test_bug_also_affects_new_upstream.TestBugAlsoAffects.test_bug_also_affects_picker [ multipart
<lifeless> same issue
<lifeless> its a bug in the zope testrunner output glue
<lifeless> s/bug/questionable behaviour/
<StevenK> Ah
<lifeless> see how it reports the same test twice
<lifeless> once successful
<StevenK> The problem is that subunit/hudson is tripping over it
<lifeless> once error with the tag zope:threads
<lifeless> yes, needs a fix for sure
<lifeless> also
<lifeless> \o/ reading-back the ports from the librarian
<lifeless> night all
<jtv> gary_poster: do we have any concrete plans for getting that storm is_empty fix into zc.buildout?
 * jtv wonders vaguely if he's asking the right person
<jtv> â¦and concludes yes, probably
<henninge> Does anybody know what's wrong with the icon alignment on (some) Launchpad pages? It seems to be related to the "link" formatter.
<henninge> http://people.canonical.com/~henninge/screenshots/icon_misalignment_chromium.png
<henninge> It's slightly less in Firefox but still visible.
<henninge> http://people.canonical.com/~henninge/screenshots/icon_misalignment_firefox.png
<wgrant> It's been a problem since CSS sprites were introduced.
<henninge> But this is worse now. Could it be related to the Ubuntu font I am using now?
<henninge> Looking at bac's screenshot, I suspect that: http://people.canonical.com/~bac/branchvis/forbidden-with-teams.png
<henninge> wgrant: also, the branch icons for "bug" and "merge proposal" now come out on top of each other.
<henninge> but that may be a result of the higher line height introduced by the misalignment.
<wgrant> henninge: Branch emblems have always been a problem in WebKit. Or does it show up in Gecko now too?
<jcsackett> henninge: i just checked a few pages in safari, they appear slightly unaligned, as in bac's screenshot, not as bad as in the screenshots you posted.
<wgrant> I think the sprite alignment problem is just because the text height is larger.
<jcsackett> safari is definitely *not* using the ubuntu font. :-P
<wgrant> So the sprite appears at the top.
<henninge> wgrant: so it is related to the new font.
<wgrant> Related, but not specific to.
<henninge> jcsackett: do you have the new Ubuntu font installed?
<jcsackett> henninge: no, and i'm away from my development machine right now. i can install it and let you know if i see an increased problem later today, when i'm home.
<jcsackett> probably a bit after 9AM EST?
<henninge> jcsackett: yes, that's fine.
<jcsackett> henninge: okay, i'll let you know then.
<henninge> jcsackett: are you on EST? You are up early!
<henninge> Although it's EDT, actually.
<henninge> ;-)
<jcsackett> :-P
<jcsackett> henninge: i usually get up at five to walk my dogs and hit the gym.
<henninge> good man! I am glad when I make 6 o'clock ...
<jcsackett> it took a fair bit of retraining. i used to be a sleep 10 guy. :-)
<jcsackett> sleep till ten, rather.
<henninge> luckily I have a family that won't allow me to do that ...
 * gmb lunches. 
<deryck> Morning, all.
<jcsackett> henninge: was able to look at the site with the font; it does look a bit worse.
<jcsackett> but the alignment does seem off across browsers, even without the font. just not as bad as with.
<salgado> mrevell, when you have a moment, can you have a look at https://code.edge.launchpad.net/~leonardr/launchpad/temporary-integration/+merge/38836 ?
<mrevell> certainly salgado
<allenap> deryck: Robert Collins gets two entries for Launchpad Suite on https://edge.launchpad.net/malone/+subscribe. Have you seen that before?
 * deryck looks
<deryck> allenap, no, never seen that.  Odd.
<allenap> deryck: That could be a problem with new code. I'll investigate.
<deryck> allenap, ok, thanks
<abentley> bigjools, chat?
<mars> rockstar, ping, the mail you sent yesterday still has not made it through to Gary and I.  Strange.
<rockstar> mars, bugger. That means my SMTP server has done its weird "fall over but don't tell anyone" thing that I thought I fixed months ago.
<jkakar> Am experiencing more (I think) timeout related issues... can't download a 276kb image from a bug.
<jkakar> I appreciate the rationale for ratcheting down timeouts, but it's been *really* annoying.
<jkakar> I think it might be too extreme.  Maybe I just have bad luck and most people don't notice.
<deryck> jkakar, hi.  what bug and attachment?
<jkakar> deryck: https://bugs.edge.launchpad.net/landscape/+bug/663786
<jkakar> deryck: The first attachment, called 'Design'.
<deryck> jkakar, I can't see it because it's private.  I think this is the private attachments can't be downloaded bug, not timeout related.
 * deryck looks for a bug #
<jkakar> deryck: Ah, okay.  I was able to download the other attachment on the same bug, though.
<deryck> yeah, it's an inconsistent condition.
<jkakar> deryck: Ah, fun times. :)
<deryck> jkakar, yeah, you could call it that :-)
<jkakar> Anyway, sorry for jumping to conclusions about timeouts, but been experiencing them a lot for the last few weeks. :/
<deryck> jkakar, I subscribed you to Bug #638054, since it's private.  But I think that's what you might be experiencing.
<deryck> jkakar, and yeah, timeouts are also an issue.  In some cases, it's the transition to postgresql 8.4.
<jkakar> deryck: Cool, thanks a lot and thanks for listening. :)
<deryck> jkakar, np! :-)
<mars> henninge, looks like the lucid-db-devel build failure is in translations, so in your court?  https://lpbuildbot.canonical.com/builders/lucid_db_lp/builds/334
<mars> henninge, can't remember if that is a known issue - someone was working on it last night IIRC?
<mars> so it may be fixed in devel
<not-abentley> bigjools: chat?
<danilos> mars, I can't remember anything related to that being changed in literally ages, so it's really weird
<bigjools> abentley: 'sup?
<mars> danilos, ok, not fun
<abentley> bigjools: several things.  buildd install issues and a big build queue.
<abentley> bigjools: can we chat on mumble or skype?
<bigjools> abentley: I'm on mumble
<mars> danilos, maybe it was intermittent?
<abentley> bigjools: I'm in the Soyuz channel.  Where are you?
<danilos> mars, yeah :( I'd even say we haven't changed anything related since the last Epic, and it's the first time I see it
<mars> danilos, ok, I'll force a rebuild then, and we'll see
<mars> danilos, ah, same failure in lucid_lp, eh?
<mars> danilos, https://lpbuildbot.canonical.com/builders/lucid_lp/builds/268
<mars> danilos, and the following test on that same builder passed just fine
<bigjools> abentley: https://lpstats.canonical.com/graphs/BuildersActiveVirtual/20101019/20101021/
<danilos> mars, :( I'll take a look
<sinzui> allenap, a I have said in many emails. registry just models user. authentication is launchpad foundation
<allenap> sinzui: Sorry. I assume you've re-targeted that bug, but I'll try to remember for future bugs.
<sinzui> I have
<allenap> sinzui: I saw the bit about merged accounts and jumped to a decision.
<sinzui> I am powerless to help with openid issues
<danilos> mars, the failing tests are completely different
<sinzui> allenap, openid login to a wiki is the issue.
<allenap> sinzui: I was hasty :(
<mars> danilos, I forced the build anyway, we'll see what happens.
<jcsackett> can someone help me out with some weird issues with a method over launchpadlib?
<jcsackett> test that brings up the issue is here: https://pastebin.canonical.com/38848/
<jcsackett> pdb session showing odd thing happening inside test is here: https://pastebin.canonical.com/38849/
<sinzui> jcsackett, I am confused about the issue here.
<jcsackett> sinzui: person.join is never actually called in the codebase--i'm getting a return of the raw json dump of the test person as a return.
<sinzui> jcsackett, methods() do update the instance (they are transactional)
<sinzui> leonardr, ^ ca you help jcsackett with his lplib test?
<leonardr> i'll look
<jcsackett> leonardr: feel free to interrogate me a lot; this is easier to describe than pastebin an illustrative fashion.
<leonardr> ok
<sinzui> jcsackett, you use lp_save() when you change a property (eg, commit). You have not changed a property. I do not think it is needed
<jcsackett> sinzui: yeah, i wasn't sure. i was experimenting a bit there.
<jcsackett> the same scenario occurs regardless of lp_save, so i agree it's unnecessary.
<sinzui> jcsackett, regardless of the join behaviour, does the test pass?
<jcsackett> sinzui: no, the test is failing, because person.join doesn't seem to actually get called.
<jcsackett> so no membership is created. or in the case where i'm trying to catch an exception, the exception isn't raised.
<jcsackett> this being lp.registry.model.person.join, not the NamedOperation.
<henninge> sinzui: found another one of these: https://edge.launchpad.net/~timstoaster-deactivatedaccount
<leonardr> jcsackett: my advice based on zero information is to re-run the test with 'httplib2.debuglevel = 1' and show my the http requests and responses
<sinzui> henninge, that is because login/reset is broke
<henninge> sinzui: he is appaerently active because he just joined a team.
<henninge> yeah, I know.
<henninge> Can we not fix that, though?
<sinzui> henninge, he is active because her has a preferred email address
<henninge> I mean, the account.
<sinzui> henninge, the rename rule did not run. that is the only thing wrong. the user does not seem to care
<sinzui> https://edge.launchpad.net/~timstoaster-deactivatedaccount/+review
<henninge> sinzui: Yes, I just found that.
<leonardr> jcsackett: also, tell me lazr.restfulclient.__version__
<henninge> sinzui: is it harmless to just rename it?
<leonardr> and see if this starts working if you access test_team.name before invoking me.join()
<sinzui> henninge, yes, and it should have been renamed
<henninge> sinzui: ok, I'll  do that and inform the user.
<sinzui> henninge, if there was something like a ppa, the rename field would not appear
<henninge> ah, good to know
<henninge> sinzui: hm, appearently I cannot do it.
<sinzui> henninge, this is the E'FING hidden location bug that bac reported
<sinzui> The user can fix it, but we cannot
<jcsackett> leonardr: requests/responses are here https://pastebin.canonical.com/38851/
<jcsackett> version is 0.10.0
<jcsackett> calling test_team.name didn't have any effect.
<leonardr> ok, i guess that's good
<jcsackett> leonardr: i see a number of 303s around calls involving me; should i try getting the test person out of people['test-person'] instead of using .me?
<leonardr> jcsackett: yes, that's the problem. clients aren't allowed to re-submit post information if they get a 303
<jcsackett> leonardr: trying that now.
 * jcsackett wishes he had thought of httplib2.debuglevel before.
<leonardr> it is the general purpose tool
<jcsackett> leonardr: that worked, thanks for your help. i'll remember debuglevel next time. :-)
<leonardr> great
<leonardr> jcsackett: you ran into https://bugs.edge.launchpad.net/launchpadlib/+bug/504297
<_mup_> Bug #504297: calling named operation on launchpad.me <launchpadlib :Triaged> <https://launchpad.net/bugs/504297>
<jcsackett> leonardr: yup, that's exactly it.
<jam> mwhudson: when you wake up. we were down from 4 failing windmill tests to only 1 failing windmill test \o/ for reproducibility. I guess we have an open bug about the windmill tests being unstable? Can we submit anyway?
<jam> abentley: I forgot to mention yesterday, but the db is at chinstrap:~jameinel/test.db.bz2 if you want it.
<abentley> jam, cool.
<lifeless> lamont: hi
<lifeless> matsubara: commented
<matsubara> thanks lifeless
<mars> jam, do you have a copy of the test failure?
<jam> mars: when there was 1 failure, or when there were 4 ? :)
<jam> lp.code.windmill.tests.test_branchmergeproposal_review.TestReviewCommenting.test_merge_proposal_replying
<jam> was the single failure
<jam> _StringException: Text attachment: garbage
<mars> jam, I mean, what was the failure you were seeing?  Ah, 'garbage threads', Thread-1234, etc.
<lamont> morning lifeless
<jam> mars: yeah, something like that
<jam> I can forward you the email if you want
<mars> jam, please do, and I'll tell you if it is intermittent or not
<jam> mars: well, I'm pretty sure it is intermittent. As we submitted, and 4 windmill tests fails, we resubmitted without changes, and only 1 failed
<jam> (I guess a fix could have landed in the meantime)
<jam> mars: I sent you the single failure, and the 4-failure emails
<mars> jam, lifeless had a fix, may or may not have worked for that.  In the meantime we are saying "run ec2 test, if it looks good, then bzr lp-land"
<mars> thanks
<jam> mars: so ignore the failures and land anyway?
<mars> jam, from what I see, yes
<mars> that is the intermittent failure that everyone has been getting
<lifeless> matsubara: the windmill threads thing is fixed
<lifeless> bah
<lifeless> mars: ^
<matsubara> :-)
<lifeless> mars: we shouldn't be saying that any more
<mars> lifeless, proven fixed?
<lifeless> yes, hudson shows errors still because of a defect in zope.testrunner
<lifeless> it reports *any* new threads as an 'error:' at the subunit level.
<mars> lifeless, and when it comes to fixes for stuff like this, you have to deal with fallout for a week or two after, until everyone is only working on fresh branches of devel.
<lifeless> mars: buildbot won't report it as an error
<lifeless> mars: no, ec2 merges devel, so everyone gets it immediately
<mars> ah, right
<mars> jam, so as lifeless says, you can ec2 land it again if you want, it should pass
<mars> but since you already ran it by, you can probably just lp-land it
<mars> no need to run the suite again
<lifeless> lamont: ping
<mtaylor> sinzui: I just added information to an incomplete bug report - should _I_ pop the status back to new, or will just adding info be good enough?
<sinzui> I get emails. changing the status mean I need to change it back if I still do not see a problem
<mtaylor> ok. cool
<sinzui> mtaylor, I can see the sorting is working as designed. The common problem here is that this page seems to be designed for no one. Each person sees, tries to change it, then makes other people irrate.
<mtaylor> sinzui: really? what is causing cherry to show up first?
<sinzui> mtaylor, *who* needs to use this drizzle page
<mars> rockstar, the mail went through.  Thanks!
<sinzui> mtaylor, The only legitimate use case I have found for this page is for packagers who need the latest package to create an installer
<rockstar> mars, awesome.
<mtaylor> sinzui: well... I _was_ using it in the debian watch file as per the answer in the question about using launchpad via uscan
<mtaylor> sinzui: right. I was using it as part of packaging
<sinzui> mtaylor, why do you not release from the focus of development?
 * sinzui is search for more evidence that +downloads is so bad is needs to be shot, not fixed
<mtaylor> sinzui: oh - well, usually we do - I think we moved dev focus to freemont prematurely
<mtaylor> sinzui: well.... actually, I take that back
<mtaylor> sinzui: "focus of developement" seems to be slightly more vague than I'd like to it be ... focus of development sets lp:drizzle to something ... which is what our "trunk" branch is - which is where crazier things go, whereas elliot is in beta and thus is the thing we're working on solidifying and releasing
<sinzui> understood
<mtaylor> sinzui: I have no personal need for +download myself, as long as there is a sensible thing to put in debian/watch :)
<sinzui> this is a similar pattern to backporting and releasing to supported series, but +download and the latest release rules do not know how to negotiate that arrangement
<mtaylor> sure. nor would I expect them to
<mtaylor> but the latest download link on the main project page seems to have gotten it right
<mtaylor> why is it that the cherry release comes up as the most recent on +download ?
<sinzui> 50% of people think it is wrong :(
<mtaylor> and/or why is that 'as designed'
<mtaylor> haha
<jcsackett> so has anyone gotten an ec2 email back that says every test failed but it's all text attachment: garbage? or am i just especially unlucky today?
<sinzui> I honestly this this is unfixable. There are many user and usecases at play here and one very mediocre solution
<mtaylor> I think I'm tending to agree with you
<sinzui> debian/watch though is a very important use case and I will look into how we can ensure this is supported
<mtaylor> sinzui: well, for now it's working for me to use http://launchpad.net/drizzle rather than http://launchpad.net/drizzle/+download
<mtaylor> but - if 50% of the people think that http://launchpad.net/drizzle is showing the right thing...
<mtaylor> s/right/wrong/
<sinzui> I think we need to ensure know one has to guess at which url to use with debian watch. The portlet on the project page could change in a few weeks and there is no test says that debian watch needs the highest version number
<mtaylor> gotcha
<mtaylor> speaking of - I'd still like to understand what criteria would cause something other than drizzle7-2010.10.01.tar.gz to show up as the latest download... so that I can perhaps manage my meta data differently
<sinzui> well, maybe that is what we want and we can do that on the download page. Add a portlet that shows the highest version number release and ensure there is a url for it too
<mtaylor> (sorry if I'm being a pest about it, I just can't understand why a release from april would match "latest" under any set of criteria)
<sinzui> Since there are no release from trunk, the series are sorted by number high to low, but aphabetically low to high.
<sinzui> elliot comes after cherry
<mtaylor> ah
<mtaylor> ok.  so the series are not sorted by release date
<sinzui> We could change the rules to always sort alphabetically. I am not sure how many people would notice. I think most have adopted numbers when they know they are developing for a long time because Lp does that better
<mtaylor> but why not use the explicit release date?
<sinzui> We do not sort series be release date because people add releases to old series often. The sort we use addressed the last group of users who noted that the listing was unsorted
<mtaylor> heh
<mtaylor> well, I would personally prefer this page not exist rather than be sorted alphabetically reversed
<sinzui> the product-release-finder is also a factor. it add releases to series that may be ancient
<leonardr> salgado (or someone), i'm having problems getting testopenid.dev to work on my dev machine. i think this is the first time i've ever actually had to use testopenid.dev. can you help me?
<sinzui> mtaylor, your issue is a dup of bug 231797, but I am updating that bug to explain the issues with the project index page and download page
<_mup_> Bug #231797: no sensible way to use debian/watch files with launchpad hosted tarballs <Launchpad Registry:Triaged> <devscripts (Ubuntu):Invalid> <https://launchpad.net/bugs/231797>
<mtaylor> cool
<sinzui> oh, I see you have already comment on the bug
<leonardr> salgado (or whoever), my specific error seems to be the lack of a session cookie
<leonardr> salgado: got it working, nm
<Ursinha> lifeless, hello hello
<lifeless> hiya
<Ursinha> lifeless, so, bad-commit-xxxxx tag is added by hand, right?
<lifeless> yes
<lifeless> for now; if you want to add a scrpit to do it, awesome
<Ursinha> lifeless, I'm asking because there's a wiki page saying it's done by the script: https://dev.launchpad.net/QAForContinuousRollouts
<Ursinha> so I got confused
<lifeless> meep
<lifeless> well its *read* by the script
<lifeless> but the script can't assess good/bad
<lifeless> it can only read our decisions
<Ursinha> right, I thought so
<Ursinha> lifeless, bdmurray landed a fix for bug 347218 but didn't mention the rollback tag
<_mup_> Bug #347218: Allow bug supervisors to make tags official <bad-commit-11701> <qa-bad> <story-define-official-tags> <Launchpad Bugs:Fix Committed by brian-murray> <https://launchpad.net/bugs/347218>
<Ursinha> I've added the bad-commit tag
<lifeless> Ursinha: cool
<lifeless> Ursinha: so we need some way to say 'commit X fixes bad commit Y' other than [rollback]
<lifeless> and
<lifeless> Ursinha: so we need some way to say 'commit X fixes bad commit Y' other than by committing a new revision
<lifeless> these are separate but could be fixed in the same way
<Ursinha> I guess the tag
<Ursinha> if you add the tag, and mentions the bug in both commits
<Ursinha> hmm
<lifeless> mm
<lifeless> so I'm saying 'lets assume people fix by mistake'
<Ursinha> lifeless, how would the second work?
<lifeless> or thereabouts
<lifeless> well, one way would be:
<lifeless> bad-commit=xxxx,yyyyy
<lifeless> where xxxx is the bad commit
<lifeless> and yyyy is the fix for it
<lifeless> so the script cannot deploy xxx without yyyy
<Ursinha> where are we placing this?
<Ursinha> bug tag?
<Ursinha> lifeless, ^
<lifeless> otp
<Ursinha> sure
<gary_poster> rockstar, do you have a few min for a tarmac phone call?
<rockstar> gary_poster, indeed I do.
<gary_poster> awesome, thanks rockstar.  mumble or skype?
<gary_poster> I'm on both atm
<rockstar> gary_poster, skype please.
<abentley> jam: skype?
<Ursinha> lifeless, also, could you comment on bug 623239 later, please?
<_mup_> Bug #623239: please don't squash qa-untestable tags <qa-tagger:Incomplete> <https://launchpad.net/bugs/623239>
<jam> abentley: sure, let me go grab a drink, brb
<jam> abentley: back
<jam> calling
<jam> well, would if you were online :)
<rockstar> gary_poster, also, if yous guys have questions, there's also #tarmac
<gary_poster> rockstar: ah!  good to know, thanks
<lifeless> Ursinha: hi
<lifeless> I is back
<lifeless> Ursinha: I'm saying we could use the same bug tag
<lifeless> bad-commit=12345,12350
<lifeless> meaning broken in 12345 fixed in 12350
<Ursinha> right
<lifeless> commented on the bug, but perhaps we need some realtime discussion
<mwhudson> morning
<Ursinha> lifeless, want to mumble?
<lifeless> I could skype if you like
<lifeless> mumble works very badly here
<Ursinha> sure, a minute
<lifeless> I'm rbtcollins
<Ursinha> calling
<mwhudson> i wonder why mumble is so terrible from nz
<mwhudson> lag?
<thumper> mwhudson: I think it is telecom voip traffic shaping
<thumper> mwhudson: yet to confirm though
<mwhudson> thumper: i'
<mwhudson> thumper: i'm not on telecom though
<thumper> mwhudson: who are you through?
<mwhudson> thumper: voda at home, snap in the office i think
<mwhudson> thumper: the international bandwidth is all the same though
<mwhudson> i guess?
<thumper> mwhudson: how's the office thing working out?
<thumper> rockstar: skype?
<rockstar> mwhudson, you have an office, huh?
<mwhudson> thumper: well
<rockstar> thumper, sure.
<mwhudson> rockstar: yeah, i'm renting some space about 50m from where lca was
<mwhudson> decided some more work/life separation was in order
<mwhudson> also being in town regularly is nice
<jml> lifeless: when are we going to just split canonical.buildd into its own tree
<rockstar> mwhudson, yeah, I tried that, only to realize that I am socially "handicapped."
<lifeless> jml: when someone gets a tuit
<lifeless> oh yay
<lifeless> jml: bug 664107
<mwhudson> rockstar: i'm not sure quite what you mean, but there are some pretty socially handicapped people in this office too
<jml> lifeless: well, I guess we escalate that to Canonical Legal
<lifeless> 'punt'
<jml> lifeless: and hope that the amateur discussion is at least entertaining
<lifeless> :)
<mwhudson> rockstar: hey, when am i going to be able to click "merge this" on a mp?
<cr3> can someone recommend a good example of project packaging eggs? I'm not sure where to ask but perhaps launchpad folks might know
<lifeless> gary_poster: I'd like a few cycles to talk zcml if you can spare them today
<cr3> gary_poster: ^^^ you might know?
<lifeless> cr3: what do you mean?
<gary_poster> lifeless, sure thing.  about to go another call, then will ping
<lifeless> thansk
<lifeless> jml: also
<lifeless> jml: I have a zope testing question I *know* you know hte answer too
<gary_poster> cr3: I know a fair amount about packaging Python things as eggs, not sure what your question is.  Feel fee to ping me later if you like, but swamped atm, and others can probably answer
<cr3> lifeless: if a project creates eggs as its build process, they probably need to be packaged as part of the project package itself. I was hoping to find a good exmaple of a project that does something like that
<lifeless> jml: on hudson, thread leaks show up as errors with the same test id as the test that passes
<lifeless> jml: how do we change that to a skip, or something similar.
<cr3> gary_poster: ah, I meant "debian packaging" as opposed to "packaging eggs" :)
<jml> lifeless: hmm.
<lifeless> jml: or should StevenK filter them with subunit-filter
<lifeless> jml: this isn't in our logic - our logic is 'fixed' now.
<jml> lifeless: these are errors raised during testTearDown, right?
<lifeless> no
<LPCIBot> Project devel build (137): STILL FAILING in 3 hr 58 min: https://hudson.wedontsleep.org/job/devel/137/
<lifeless> there are two, overlapping, thread checks.
<jml> oh, zope's and our own custom one
<lifeless> 1) in BaseLayer. Fixed (the disable_thead_check flag was miscoded)
<lifeless> 2) in zope.
<gary_poster> cr3 ah gotcha.  Played with it.  assume you saw https://wiki.ubuntu.com/PackagingGuide/Python .  now really on call
<lifeless> it took 3 months for my last change to zope.testing to get in; I'm not stoked about the idea of trying again :(
<cr3> gary_poster: thanks, I didn't know about that guide, I'll certainly have a look
<thumper> jml: hi
<jml> thumper: hello
<jml> lifeless: hm.
<thumper> jml: I want you to look at the recipe stuff and let me know what needs to be done to say "we're done"
<jml> thumper: I already took a good look today
<lifeless> cr3: eggs and python packages are a little odd :)
<jml> thumper: writing it up tomorrow. who should I send it to?
<lifeless> cr3: the python packaging standard unpacks them on disk.
<cr3> how is ez_setup.py generated or where is it copied from? I copied mine from the distribute package but I'm not sure that's right
<thumper> jml: at least flacoste, me, abentley, rockstar, and wallyworld
<jml> thumper: will do.
<lifeless> if that what I think it is, its 'download unsigned code from the net and run it'
<thumper> jml: maybe the internal list?
<lifeless> so I would avoid it like the plague.
<jml> thumper: sure
<cr3> lifeless: it seems to be quite prevalent in lazr packages for example
<jml> thumper: btw, heard about open build service's "Source services"
<thumper> jml: yeah, saw the email
<thumper> not looked at it though
<lifeless> cr3: no comment ;)
<cr3> lifeless: at least the md5sum is checked, for what it's worth
<wgrant> cr3: Against what?
<wgrant> An HTTP page?
<lifeless> wgrant: yes, if they MITM you, they have to change *two* http requests.
<jml> thumper: I had a play today
<wgrant> lifeless: Awesome!
<jml> thumper: there's a lot I admire about OBS â I think they've got a great product.
<jml> thumper: but in this area we are still well ahead, I think.
<jml> thumper: particularly if we smarten up the UI
<thumper> jml: so lets get some good designers to focus on it...
<jml> lifeless: http://paste.ubuntu.com/517034/ that's the event. the error is tagged, so subunit-filter ought to be able to filter it, pending a new feature :)
<wgrant> Recipe builds would be really useful if they supported non-native packages, and customisable changelogs.
<cr3> wgrant: the downloaded setuptools egg
<lifeless> jml: Yeah, I'm wondering what we /want/ though
<jml> lifeless: oh. I reckon we want to filter out based on tags, probably.
<lifeless> ok, I'll file a bug on subunit to permit that
<jml> lifeless: or disable zope's checking
<jml> but it's pretty... embedded
<lifeless> jml: shrug, We're within a few months of dropping the zope testrunner entirely.
<jml> lifeless: frabjous day etc
<lifeless> let me just ask the bandersnatch for that lovely vorpal sword
 * thumper restarts server
<jml> I am going to bed. Happy Thursday, future folk.
<jam> abentley: I rememebered what I was going to say. The new DVCS "Veracity" has gdfo in its data structures (calling it 'generation')
<abentley> jam, ah.  Cool.
<jam> abentley: they have some fairly interesting C code for finding LCA, etc, in graphs fairly well documented
<jam> I think it comes from whatever their earlier VCS was
<jam> they also seem to do something where "merge" will merge multiple tips in one go
<LPCIBot> Project db-devel build (87): STILL FAILING in 3 hr 51 min: https://hudson.wedontsleep.org/job/db-devel/87/
<flacoste> thumper: udd meeting is starting in #ubuntu-meeting
<thumper> :(
<thumper> was just about to go and make another coffee
<wallyworld> abentley: rockstar: looks like thumper has a meeting now. so our standup will be delayed
<thumper> hopefully not for too long
<rockstar> wallyworld, want to have a chat about merge queues?
<wallyworld> rockstar: ok
<mwhudson> hm, is the disable thread check fix in db-devel yet?  if so, that hudson failure is a bit :(
<lifeless> mwhudson: hudson lies
<mwhudson> heh
<lifeless> mwhudson: because the zope->subunit glue reports leaked threads as errors
<mwhudson> how so?
<mwhudson> oh right
<lifeless> mwhudson: we need to filter them out, its separate to the disable_thread_check flag
<mwhudson> ok
<jcsackett> back
<mwhudson> grar, the same issue bounced testtools
<lifeless> mwhudson: from ec2?
<mwhudson> lifeless: yeah
<lifeless> :<
<mwhudson> oh no
<mwhudson> well
<mwhudson> that didn't have your fix in the branch tested
<mwhudson> same with lp-service
<lifeless> mwhudson: StevenK: https://bugs.edge.launchpad.net/subunit/+bug/664171
<_mup_> Bug #664171: support filtering on tags <subunit:New> <https://launchpad.net/bugs/664171>
<sinzui> EdwinGrubbs, I think your change broke some menus. This kind of check now fails: self.user.inTeam(self.context)
<lifeless> sinzui: if it did, can you mark the bug bad-commit=xxx to stop it deploying? thanks.
<sinzui> EdwinGrubbs, the cases look like where we know a user is a member of himself
<EdwinGrubbs> sinzui: yes, I fixed that error already and landed a branch for it. Unfortunately, I couldn't get anyone to restart lucid_lp without restarting everything.
<lifeless> EdwinGrubbs: did you tag it bad-commit ?
<EdwinGrubbs> lifeless: no, I've never heard of bad-commit before.
<lifeless> EdwinGrubbs: errata: its been announced on the list repeatedly as part of the RFWTAD workflow.
<lifeless> I'm just grabbing the wiki page about it for you
<lifeless> https://dev.launchpad.net/QAProcessContinuousRollouts
<wgrant> thumper: So WIP MPs will be usable from the next rollout?
<thumper> wgrant: hopefully even before that (from edge)
<thumper> wgrant: or whatever we are using then...
<wgrant> thumper: Hm, aren't they sent by a script?
<thumper> wgrant: yes...
<thumper> so?
<thumper> the job is created by the appserver
<wgrant> Or is it the same job, but now repurposed?
<thumper> same job
<wgrant> Ahh.
<thumper> just not triggered on wip
<thumper> and triggered on wip -> needs review
<rockstar> wallyworld, you keep cutting out.  I can't understand you.
<wallyworld> rockstar: i can hear you
<lifeless> thumper: wgrant: remember edge deployments are stopping
<lifeless> like asap
<thumper> lifeless: where does qastaging email go?
<lifeless> thumper: same story as staging I hope
<thumper> don't hope...
<thumper> make sure :-)
<lifeless> thumper: well, i can't tell, can I ?
<lifeless> thumper: you can try, trigger something on qastaging
<lifeless> if you get mail, we need to change stuff
<lifeless> http://qastaging.launchpad.net/
<thumper> oops 1754QS12 on that
<lifeless> https://qastaging.launchpad.net/ worked for me
<lifeless> I think it may have been paged out
<lifeless> :>
<wallyworld> rockstar: you ended the call?
<rockstar> wallyworld, headset is gone.
<wallyworld> bollocks
<thumper> whatever we have behind qastaging seems to be timing out a lot
<lifeless> its the same as staging
<lifeless> same boxes for everything
<rockstar> abentley, stand up?
<abentley> rockstar: sure.
<abentley> rockstar: who's hosting?
<jam> mwhudson: any chance you can lp-land my patch? mars at least approved of doing it. Or you can ec2land it, as they claimed the windmill stuff is fixed in devel
<rockstar> wallyworld, https://code.edge.launchpad.net/~rockstar/launchpad/merge-queues-model/+merge/38762
<mwhudson> jam: i've just sent it off to ec2 again
<jam> mwhudson: thanks
<mwhudson> jam: this has been a pretty long drawn out process, i hope your next launchpad branch goes more easily :)
<thumper> Loading results failed. for a person search on edge
<thumper> has this been fixed or looked at?
<lifeless> its the person search
<lifeless> edwin has a fix in progress
<mars> thumper, I was writing a bug for that same problem - it breaks merge proposals too, you can't request new reviewers
<mwhudson> usually middle click will short circuit the js
<mars> thumper, bug 664214
<_mup_> Bug #664214: "Request another review" search returns "Loading results failed" <Launchpad Bazaar Integration:New> <https://launchpad.net/bugs/664214>
<mars> mwhudson, oops, doesn't work for MPs
<mars> thumper, you are right, the "Subscribe someone else" link on bug reports has stopped working too
<mars> I'll file a new bug about it
 * thumper nods
 * thumper afk for lunch meetup
#launchpad-dev 2010-10-21
<mars> thumper (and sinzui), but 664220
<mars> bug 664220
<_mup_> Bug #664220: Person search is broken on edge <Launchpad Registry:New> <https://launchpad.net/bugs/664220>
<lifeless> mars: bug 664220
<_mup_> Bug #664220: Person search is broken on edge <Launchpad Registry:New> <https://launchpad.net/bugs/664220>
<lifeless> bah
<lifeless> mars: bug 664220
<_mup_> Bug #664220: Person search is broken on edge <Launchpad Registry:New> <https://launchpad.net/bugs/664220>
<lifeless> bah.
<lifeless> I've duped it anyhow
<mars> lifeless, thanks.  dupe search did not turn up that bug
<mars> neither did google
<lifeless> mars: bugs.lp.net/launchpad-project/+bugs?field.tag=timeout
<lifeless> I always hit that up and eyeball these days
<LPCIBot> Project devel build (138): STILL FAILING in 3 hr 31 min: https://hudson.wedontsleep.org/job/devel/138/
<LPCIBot> * Launchpad Patch Queue Manager: [r=gmb][ui=none][bug=347218] Allow bug supervisor to set
<LPCIBot> official_bug_tags for a distribution.
<LPCIBot> * Launchpad Patch Queue Manager: [r=lifeless][ui=none][no-qa] Fix multiline glob imports in doctests.
<LPCIBot> * Launchpad Patch Queue Manager: [r=mars][ui=none][bug=648476] Adds the "Add member" link to the block
<LPCIBot> that displays in the member listing when there are no members;
<LPCIBot> team owners should still have access to this link, regardless of
<LPCIBot> membership status.
<LPCIBot> * Launchpad Patch Queue Manager: [r=abentley][ui=none][no-qa] Fix bin/kill-test-services to work
<LPCIBot> properly again
<LPCIBot> * Launchpad Patch Queue Manager: [r=sinzui][ui=none][bug=655802] Fixed timeout when AJAX picker
<LPCIBot> searched person or team vocabularies.
<LPCIBot> * Launchpad Patch Queue Manager: [r=gmb][ui=sinzui,
<LPCIBot> salgado][bug=652232] Move links into sidebar on code.lp.net/$person
<LPCIBot> page.
<wallyworld> rockstar: ping - quick question
<rockstar> wallyworld, whazzup?
<rockstar> Oh balls.  Just realized it's time for the AsiaPac reviewer's meeting.
<wallyworld> the registrant attribute on IBranchMergeQueue.....
<wallyworld> says "The user that registered the branch"
<wallyworld> that isn't correct is it?
<rockstar> wallyworld, no, it's not.  I apparently can't always finish my thoughts.
<wallyworld> looks like a cut'n'paste :-)
<wallyworld> btw, i thought the reviewers' meeting time had changed?
<lifeless> bdmurray: https://devpad.canonical.com/~lpqateam/qa_reports/deployment-stable.html
<bdmurray> lifeless: so, it really is okay just incomplete rather
<bdmurray> lifeless: it only allows it for a project and I've another branch that allows it for a distribution
<lifeless> bdmurray: please mark it as qa-ok then
<lifeless> cause its blocking deployments
<lifeless> we need to iterate on our toolchain to allow more precision
<LPCIBot> Project db-devel build (88): STILL FAILING in 3 hr 35 min: https://hudson.wedontsleep.org/job/db-devel/88/
<LPCIBot> Launchpad Patch Queue Manager: [rs=buildbot-poller] automatic merge from stable. Revisions: 11748,
<LPCIBot> 11749, 11750, 11751, 11752, 11753, 11754, 11755, 11756, 11757,
<LPCIBot> 11758, 11759, 11760, 11761 included.
<lifeless> oh hai.
<rockstar> Dammit PQM...
<thumper> rockstar: whazzup?
<lifeless> rockstar: thats more buildbot :)
<rockstar> thumper, All lines of log output:"[['Connection closed: Unexpected end of message. Please check connectivity and permissions, and report a bug if problems persist. ']]"
<thumper> :(
<rockstar> lifeless, ^^ That error is not buildbot...
<lifeless> rockstar: the irc message is hudson
<lifeless> the size of the change is buildbot
<rockstar> lifeless, yeah, I can /ignore LPCIBot.  :)
<lifeless> the failure root cause is probably the subunit bug I linked here this morning
<lifeless> that it doesn't support filtering the bogus errors from zope.testing
<rockstar> So this is great.  I submitted two pipes of a pipeline to pqm.  pqm pukes on the first one, but seems to gobble up the second one.  All the changes are landed, but now the bug the first pipe was linked to isn't going to find the QA-bot.
<StevenK> Awww
<StevenK> rockstar: This is why I feed branches at PQM piece-meal
<lifeless> rockstar: by pukes
<lifeless> rockstar: do you mean 'we were in testfix' ?
<lifeless> rockstar: if you used --fixes lp:xxx, then the qabot should hopefully find it anyway.
<rockstar> lifeless, no, see my message to thumper above.
<lifeless> oh, I see
<lifeless> so codehosting was down
<rockstar> StevenK, I did feed them piece-meal, in that I sent them both separately.
<lifeless> spm: would like to know if we have a log of what went on for rockstar above
<rockstar> lifeless, yeah, it's not necessarily pqm's fault, but this camel is really tired of straw...
<spm> hm?
<StevenK> rockstar: Sorry, what I mean is submit ; wait for the e-mail until it says sucess and then submit the next one
<lifeless> rockstar: would tarmac handle that any better?
<rockstar> lifeless, not really "better" per se.  It won't merge the branch if the dependent branch hasn't been merged yet.
<lifeless> (not that its pqm vs tarmac, this is all about operational polish)
<lifeless> rockstar: thats intersting for me, because I often want to land all the branches at once.
<lifeless> rockstar: is there a knob for controlling htat
<rockstar> lifeless, no, there isn't.
<lifeless> spm: 13:48 < rockstar> thumper, All lines of log output:"[['Connection closed: Unexpected end of message. Please check connectivity and permissions, and report a bug if problems persist. ']]"
<lifeless> spm: usually means ssh handshaking didn't.
<spm> mmm
<lifeless> indeed
<rockstar> lifeless, I'm sure we can, but I think we can probably teach launchpad to have a better knob for controlling it.
 * rockstar quits for the night instead of getting his grump on
<lifeless> gnight
<lifeless> hmm, need the fire on I think.
<lifeless> brb
<wgrant> Shouldn't a UI designer be doing UI reviews?
<wgrant> There was a proposal that we would be able to forward things to the design team.
<wgrant> But that never happened.
<wgrant> And it doesn't help overall UI consistency.
<wgrant> Which Launchpad is sort of completely lacking.
<lifeless> its not completely lacking
<lifeless> I think thats a bit harsh
<wgrant> True, it is better than it used to be.
<wallyworld> thumper: you around?
<thumper> yep
<wallyworld> wanna have that skype call?
<wallyworld> thumper: ^^^^^ - i also had a question about that +branch bug
<thumper> wallyworld: sure
<thumper> wallyworld: skype doesn't like me
<wallyworld> thumper:  so it's not the only one then?
 * lifeless hates on doctests some more
<thumper> I can hear you
<wallyworld> thumper: can you hear me?
<lifeless> ok
<lifeless> this is weird
<lifeless> ah no, its not. Its subunit
<lifeless> jml: I think we're going to have to change the zope behaviour here.
<lifeless> jml: unless you want to filter in the ec2land code
<lifeless> mwhudson: hey
<mwhudson> lifeless: hello
<lifeless> my uniqueconfig branch bounced
<lifeless> I have a fix but its arguable
<mwhudson> gar, lp-service just did too :(
<lifeless> I want to see if you'd care
<mwhudson> ffs
<mwhudson> lifeless: link me up
<lifeless> lp:~lifeless/launchpad/uniqueconfig 11748
<lifeless> mwhudson: this isn't the thread thing
<mwhudson> ah ok
<lifeless> I'm going to have a poke at that now
 * mwhudson looks
<lifeless> the error was testrunner_12345
<lifeless> instead of testrunner
<mwhudson> lifeless: it's not very nice, but i think it's good enoguh
<lifeless> thanks
<lifeless> Spelling it out long ways was incomprehensible
<lifeless> not to mention redundant
<lifeless> mwhudson: ok, threading thing upcoming
<lifeless> mwhudson: https://code.edge.launchpad.net/~lifeless/launchpad/threads/+merge/39009
<mwhudson> lifeless: oh, before i do the second clicky, did you make a branch of zope.testing for this based on lp:~mars/zope.testing/3.9.4-p1 ?
<lifeless> no
<lifeless> we don't want this upstream
<lifeless> so that would be pointless
<mwhudson> well, it might make life ever so slightly easier if someone wants to make a 3.9.4-p3 version for whatever reason
<lifeless> tar xzf ...
<lifeless> (no really)
<lifeless> I'll probably get tag filtering done on the flight
<lifeless> and we can back this out then
<mwhudson> that makes it a bit harder for the next hacker to use version control
<mwhudson> i've done stuff like that and then been freaked out when bzr diff didn't work :)
<lifeless> the thing is that this is already out of vc
<lifeless> if we were using branchs ala config-manager
<lifeless> I'd totally agree
<lifeless> if you like I can do it
<lifeless> it just seems odd
<lifeless> particularly for a short term (hopefully) hack
<mwhudson> i would rather you did, but don't want to make too big a fuss
<lifeless> compromise
<mwhudson> and you know the thing about short term hacks... (although i agree in this case it probably will be)
<lifeless> if after uds I haven't done the real deal
<lifeless> I'll do it
<mwhudson> k
<wgrant> mwhudson: lucilleconfig was added as a temporary hack until the schema was sorted out.
<wgrant> 6 years and a few days ago.
<wgrant> It is still there.
<lifeless> wgrant: NOT HELPING
<mwhudson> lifeless: approved
<wgrant> NO TEMPORARY HACKS.
<lifeless> mwhudson: thanks
<lifeless> mwhudson: now, should I ec2land it :P
<mwhudson> eh
<mwhudson> probably i guess
<lifeless> yeah, I was trolling
<jam> mwhudson: it seems windmill still doesn't like me
<mwhudson> jam: yeah, i gave up and just lp-landed it
<jam> r
<jam> mwhudson: thank you
<lifeless> I â¥ addDetail
<jam> lifeless: you â¥ unicode, too, I see
<jtv> lifeless: damn, how do I type that?
 * jtv gotta have
<LPCIBot> Project devel build (139): STILL FAILING in 3 hr 38 min: https://hudson.wedontsleep.org/job/devel/139/
<LPCIBot> * Launchpad Patch Queue Manager: [r=edwin-grubbs][ui=salgado,
<LPCIBot> sinzui][bug=652156] Updates the projectgroup branches page to show a
<LPCIBot> message consistent with the other codehosting_usage messages
<LPCIBot> when there are no branches, instead of an empty table.
<LPCIBot> * Launchpad Patch Queue Manager: [r=jtv][ui=none][bug=659050,
<LPCIBot> 663075] Only load the link bug and subscription ++form++ when the
<LPCIBot> user is logged in.
<LPCIBot> * Launchpad Patch Queue Manager: [r=mwhudson][ui=none][bug=624009] Only send the initial review email
<LPCIBot> when the merge proposal needs review.
<lifeless> statik: oh hai :)
<lifeless> \o/
<lifeless> \o/
<lifeless> \o/
<lifeless> \o/
<lifeless> \o/
<lifeless> \o/
<lifeless> A
<wgrant> Oh?
<wgrant> That bad?
<lifeless> that good
<lifeless> totally ephemeral librarian
<wgrant> Nice!
<wgrant> It even finds its own ports to lurk on?
<lifeless> yes
<lifeless> and root dir
<lifeless> it also shows pretty clearly that we have a leak on the librarian :)
<lifeless> not in, of
<lifeless> lp:~lifeless/launchpad/librarian if you're interested
<lifeless> thumper: trade you reviews?
<LPCIBot> Project db-devel build (89): STILL FAILING in 3 hr 58 min: https://hudson.wedontsleep.org/job/db-devel/89/
<thumper> lifeless: I'm done for today...
<LPCIBot> Project devel build (140): STILL FAILING in 3 hr 29 min: https://hudson.wedontsleep.org/job/devel/140/
<LPCIBot> * Launchpad Patch Queue Manager: [r=mwhudson][ui=none][bug=660264] Implement LaunchpadForkingService to speed up bzr+ssh connection times.
<LPCIBot> * Launchpad Patch Queue Manager: [r=edwin-grubbs][ui=none][bug=663828] Split ReadyService into its own file for easier maintenance of TacTestSetup.
<adeuring> good moring
<mrevell> Hello
<jml> lifeless: the ec2 land code is complex enough already... and I've seen that you've changed testrunner, so it's all good
<wgrant> bigjools: How long has p-dr been broken?
<wgrant> It should have died within a day of the Intrepid purge, if that was it.
<wgrant> And I can't think what else it could have been.
<wgrant> :/
<LPCIBot> Project db-devel build (90): STILL FAILING in 3 hr 33 min: https://hudson.wedontsleep.org/job/db-devel/90/
<LPCIBot> Launchpad Patch Queue Manager: [rs=buildbot-poller] automatic merge from stable. Revisions: 11762,
<LPCIBot> 11763, 11764 included.
<bigjools> wgrant: I don't know.  A Long Time.
<wgrant> Ah, good.
<bigjools> wgrant: can you remember if there was another bug that left buildds idle?
<bigjools> not the aborted/aborting one that I fixed
<bigjools> artigas/hooker are not happy
<wgrant> bigjools: What's their status?
<bigjools> idle
<wgrant> The slave too?
<bigjools> lots of these in their logs:
<bigjools> 2010/10/21 10:19 +0100 [HTTPChannel,202,91.189.90.177] 91.189.90.177 - - [21/Oct/2010:09:19:53 +0000] "POST /rpc HTTP/1.0" 200 222 "-" "xmlrpclib.py/1.0.1 (by www.pythonware.com)"
<wgrant> artigas is sparc, hooker ia64, right?
<bigjools> yarp
<jml> bigjools: that just means "Successful XMLRPC request"
<bigjools> quite
<wgrant> bigjools: Calling the status() method says BuilderStatus.IDLE?
<bigjools> who knows, that's not something that's easy to find out
<jml> bigjools: fwiw, it wouldn't be too hard to get the actual methods called in the log.
<bigjools> we restarted the buildds and they're still idle
<wgrant> bigjools: There are no errors in the log about the directory already existing?
<wgrant> And it's not the b-m thing where it ignores new builders until it's restarted?
<bigjools> nope
<wgrant> Well, I'd be hijacking cesium for a quick python -c "import xmlrpclib; xmlrpclib.ServerProxy('http://artigas.buildd:8221/rpc').status()"
<bigjools> yeah I might do that
<bigjools> wgrant: it's IDLE
<bigjools> as expected
<wgrant> bigjools: What does b-m say about it?
<bigjools> zip
<wgrant> I'm suspecting the usual b-m bug.
<wgrant> Except that these builders aren't new.
<bigjools> I'm not sure that is a bug
<lifeless> win 67
<jpds> lifeless: Â£ or $ ?
<wgrant> bigjools: Isn't it?
<bigjools> wgrant: I had another instance yesterday of the b-m throwing lots of connection errors, but no builder had been added, so my original assessment was probably wrong
<bigjools> this is something different again
<wgrant> Yay
<wgrant> Maybe it will mysteriously vanish with builderslave-resume.
<bigjools> best branch name evar
 * bigjools considers renaming it to buildd-manager-apocalypse
<bigjools> jml: how can we get the methods in the log, BTW?
<jml> bigjools: off the top of my head, I don't know. it's a matter of finding out what does the dispatch to xmlrpc_FOO and whacking a log in there.
<jml> or just putting log statements in the actual xmlrpc_FOO methods
<bigjools> yeah that was what I would do, I thought you had a "clever" way :)
<jml> one of these days I'll write a decorator that logs function calls and return values
<wgrant> I don't understand why people like the GitHub "Fork" thing... it makes no sense :/
<jml> it makes a little bit of sense
<jml> I wish Launchpad had server-side branching
<wgrant> Why?
<StevenK> Because it's a shedload quicker for Launchpad to branch Launchpad than it is for you too
<jml> StevenK: well, that's only true if you don't ever intend to work on your branch
<jml> although "branch to local" then "push to LP" is probably slower than "branch on server", "branch to local"
<wgrant> Modulo bzr bugs it shouldn't be significantly slower.
<jml> wgrant: sometimes it's more a way of thinking than anything else, I guess.
<wgrant> (I know that in practice it is)
<jml> you know, I'm not grateful enough for the fact that I'm not an IP lawyer
<wgrant> jml: The Apple thing?
<jml> wgrant: well, that's the trigger, but it's also every damn thing about IP law
<wgrant> Heh, yes.
<wgrant> It is messy stuff.
<jml> I just want to make stuff, pay artists for stuff I like and get to use the stuff I buy.
<wgrant> And not have Apple steal your product names? :)
<lifeless> jml: any word from legal?
<jml> lifeless: no, none yet.
<LPCIBot> Project devel build (141): STILL FAILING in 3 hr 30 min: https://hudson.wedontsleep.org/job/devel/141/
<LPCIBot> Launchpad Patch Queue Manager: [r=abentley][ui=none][bug=114766] Restrict the ability to nominate
<LPCIBot> bugs for a release to bug supervisors, owners or drivers.
<wgrant> :(
 * StevenK notes that hudson has gotten quicker
<jml> StevenK: oh is the time in the IRC message the time for the build?
<wgrant> It took me a couple of days to work that out.
<lifeless> jml: you have mail. I'd appreciate you taking the time to read it thoughtfully ;)
<lifeless> jml: you'll understand why I specifically mention this when you see its size!
<jml> lifeless: ok :)
<lifeless> jml: I think we should filter in ec2test, but I'll do prep to make it easy, in subunit.
<jml> lifeless: well, lib/devscripts/ec2test/remote.py is the relevant place.
<jml> lifeless: also, importantly, lib/devscripts/ec2test/tests/test_remote.py
<lifeless> I need an incremental eyeball
<lifeless> rev 11773 of lp:~lifeless/launchpad/threads
<jml> lifeless: ok.
<lifeless> this should, finally, fix the test environment headaches
<lifeless> with threads
<jml> anything to distract me from email
<jml> +            new_threads = new_live_threads()
<jml> lifeless: that's the only interesting change in that revision
<lifeless> thats it
<lifeless> if you look at the commit message, or the context, it should make more sense
<jml> lifeless: well, that's a perfectly syntactically formed line of Python :)
<lifeless> thank you
<jml> lifeless: fetching the branch now to get context
<lifeless> I spent hours on it
<lifeless> jml: loggerhead can show that ;)
<jml> lifeless: loggerhead is too slow.
<lifeless> hah, ouch,
<wgrant> Loggerhead still times out for me on the first request for any LP branch.
<wgrant> But apart from that it's not too slow.
<jml> wgrant: it's not significantly faster than me fetching the branch and using my editor.
<jml> wgrant: especially given the way it always times out on the first request for any LP branch
<jml> lifeless: +1
<deryck> Morning, all.
<wgrant> jml: Ah, I guess it's different since you live next to the DC.
<StevenK> jml: Yes, 'in 3 hours 3 minutes' is how long it took
<lifeless> jml: I've thrown it straight at PQM
<jml> StevenK: I read it as "will still be failing in 3 hours 3 minutes"
<StevenK> jml: Patches welcome
<lifeless> jml: if it doesn't get through you probably want to resubmit it yourself: '[r=mwhudson][ui=none][bug=663644][no-qa] Switch to a patched zope.testing that does not emit subunit errors on leaked threads, for windmill test sanity.'
<StevenK> \o/ \o/
<lifeless> (I think we may be in testfix)
<StevenK> lifeless: Was the patch to zope.testing horrible?
<lifeless> no its tiny
<lifeless> we'll get a longer term fix later
<lifeless> hmm, shoving testfix on that
<jml> lifeless: would you like to borrow some of my apostrophes?
<lifeless> because it does, thought a different one.
<lifeless> jml: prhaps
<lifeless> jml: you're turning into mpt
<lifeless> (not a bad thing)
<jml> lifeless: my penmanship is vastly inferior.
<lifeless> but your list of ways to improve things are equivalent
<bigjools> given enough pens and enough jmls, the penmanship could be magnificent
<lifeless> or the universe may end
<bigjools> think of all the hair
<lifeless> StevenK: whats the thread topic you started about threading errors in hudson ?
 * bigjools goes back to fixing p-d-r
<lifeless> found it I think
<StevenK> lifeless: Failures on Hudson
<wgrant> bigjools: Which files are screwed?
<bigjools> querying that as you type
<wgrant> Ah, so that kind of fixing.
<bigjools> I'm not sure what could have caused this
<wgrant> The Intrepid purge could easily have, if the timing was just right...
<bigjools> other than over-zealous librarian file collection
<wgrant> But that seemed unlikely.
<bigjools> we probably marked something expired when it should not have been
 * bigjools would love reference counts on LFAs
<wgrant> 01:17 <wgrant> So, this query isn't perfect. It violates the PPA blacklists and stay of execution. But that should all be long-dead for intrepid.
<bigjools> hmm
<bigjools> wgrant: they are all gutsy files
<wgrant> bigjools: What's datesuperseded/dateremoved?
<wgrant> Sources or binaries?
<bigjools> only checkiungbinaries
<bigjools> sigh
<bigjools> only checking binaries
<wgrant> Er, yes, of course.
<bigjools> all 90 are superseded at the same time: 2010-05-28 14:12:50.17775
<wgrant> Not Deleted?
<bigjools> so if I update these to all have dateremoved set then it should fix p-d-r
<lifeless> bigjools: LFAs are not really intended to be shared with other objects
<lifeless> bigjools: does soyuz share them ?
<wgrant> But it would also leave cruft all over the disk.
<bigjools> lifeless: it shares them across publications
<lifeless> (LFAs are the reference counts for content objects)
<lifeless> gnight
<bigjools> wgrant: point
<bigjools> nn lifeless
<bigjools> it's only 90, I can collect filenames and do it manually
<wgrant> bigjools: What's the expiry date on the LFA?
<bigjools> 2010-01-07
<bigjools> !
<wgrant> ... pardon?
 * bigjools scratches  head
<wgrant> Is datepublished set?
<bigjools> yes
<wgrant> :(
<bigjools> I suspect we set expires back a long way to force GC action
<wgrant> Hmm.
<bigjools> that date is close to our sprint in NZ
<wgrant> It is, yes.
<wgrant> And I found a bug in that query soon after.
<wgrant> So it's fine. Delete them manually.
<wgrant> (the January query included in the EXCEPT clause only those binaries outside (feisty, gutsy), ignoring the possibility that (feisty, gutsy) pubs could still be alive outside the primary archive)
<bigjools> yeah
<bigjools> people backporting no doubt
<bigjools> I wonder if we could do what  lifeless suggests
<bigjools> would make life easier
<wgrant> Not easily.
<wgrant> Without redesigning most of the data model in unobvious ways.
<wgrant> (that date does match: the paste of the query is from the 2010/01/14, and it backdated expiry by a week)
<bigjools> well if it gives us gains elsewhere it's worth it
<wgrant> By "unobvious" I mean that it's not obviously possibly at all.
<bigjools> it's always possible
<wgrant> Not without making everything even uglier.
<bigjools> so pessimistic already at such a tender age ...
<wgrant> Shh.
<bigjools> :)
<bigjools> it took me years to get this jaded
<wgrant> Maybe we should expire Jaunty without backdating before it gets OMGCRITICAL, since I managed to find a bug in the query the day after both in January and this time...
<wgrant> But 'twas too late :(
<bigjools> yeah
<jml> bigjools: do we know how many people have uploaded to a PPA?
<jml> bigjools: also, do we know how many of them are Ubuntu devs?
<bigjools> jml: I'm sure that info can be mined
<jml> bigjools: I guess I don't know how to find "uploader" in the databes
<jml> database
<bigjools> jml: packageupload.signing_key
<jml> bigjools: ta
<bigjools> jml: you know how to write the query?
<jml> bigjools: not yet, but I'll figure it out soon enough, I think.
<bigjools> jml: ok - you need to join to spph and its archive and make sure purpose=2
<jml> bigjools: ahh of course, to rule out uploads to non-ppa
<wgrant> Or just go straight from the PU to the archive.
<bigjools> no
<bigjools> well - depends what he wants
<bigjools> do you need copies to PPAs too?
<wgrant> PU->archive gets uploads to PPAs, while skipping copies from primary to PPAs.
<bigjools> "Launchpad. A home for your apps."
<jml> uploads are fine for this, I reckon.
<bigjools> nice
<wgrant> bigjools: Yes :(
<wgrant> Someone didn't even bother Googling, apparently...
 * bigjools wonders if Launchpad is a TM in the USA
<jml> I've got ~4k people as having uploaded to a PPA. Does that sound right?
<jml> http://paste.ubuntu.com/517436/
<wgrant> Looks good to me.
<bigjools> jml: seems reasonablew
<jml> ta.
<wgrant> Is there some trick to getting Skype from Partner to not hang when signing in?
<jml> wgrant: don't know. I have no idea where my skype comes from, tbh.
<jml> sinzui: you'll let me know when the bridging-the-gap registry work is ready for review, right?
<sinzui> yes
<sinzui> I still do not see the last branches landed :(
<jml> sinzui: cool.
<gmb> Can anyone remind me how to trigger run an action in a view from a unit test? I'm tying myself in knots here.
<jml> gmb: "run an action"?
<gmb> jml: Not sure what the right language is here. I have an initialized view. I pass it form data with a LaunchpadTestRequest... how do I get its "continue" action to be called? Or do I have to call it directly and pass data into it (which seems odd)?
<jml> gmb: I don't know. In the past I've done direct calls. Maybe looking at the view base classes might give you a clue as to what to do?
 * jml is now otp
<salgado> gmb, doc/launchpadformharness.txt
<gmb> salgado: Thanks!
<gmb> mars: ping
 * bigjools back in 30m
<gmb> mars: Around?
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project devel build (142): FIXED in 4 hr 1 min: https://hudson.wedontsleep.org/job/devel/142/
<wgrant> !!!!!
<bigjools> lol
<bigjools> wgrant: errr it's 3am there!
<wgrant> Yeah, but we need to deliver our final project to the client in a couple of days.
<wgrant> Normal sleeping patterns do not apply.
<bigjools> sinzui: do you think that lp/registry/interfaces/distroseriesdifference.py should be in lp/soyuz ?  I think it's something that can only exist with Soyuz enabled on a distro.
<sinzui>  bigjools yes, and I think we should look at distroseries in general and ask if other aspects are soyuz
 * sinzui was thinking of moving pocket next month
<bigjools> sinzui: yes, I want to eventually rip loads of stuff out of distroseries
<sinzui> bigjools, I want to add and is_lts flag to distroseries. I do not think derivatives care, but there may be some rule I do not know about
<sinzui> bigjools, I also think it is time to ask if version is really required. Debian's experimental series will never be released as versions
<bigjools> sinzui: derivatives don't currently care, no.  What's it going to be for?
<bigjools> not sure we can get rid of version, although I'd like to
<sinzui> yes, I think version is required by lots of things
<sinzui> We cannot display "Ubuntu 10.04 LTS" in our pages because we cannot put LTS in the version
<bigjools> LTS is Ubuntu-specific
<sinzui> I think so too
<bigjools> why can't you put LTS in the version?
<sinzui> I Do not think "10.10 LTS" validates
<jelmer> IIRC we try to parse distroseries version strings with the debian version string parser, for sorting purposes.
<sinzui> bigjools, it has to pass sane_version validation which is a subset of debversion
<bigjools> :(
<bigjools> sinzui: is it only sorting that we need that for?
<bigjools> is it set in stone that version is a debversion-style string?
<jelmer> sinzui: Would a separate string field with a subtitle for a version string perhaps be an option?
<sinzui> bigjools, re-reading my comment  DistributionMirror and buildd require it to be a debversion. The db constraint is weaker
<sinzui> jelmer, it would if we wanted to support other distros.
<jelmer> sinzui: but don't we already do so ? we have debian, gentoo, nexenta, etc registered
<sinzui> No
<sinzui> We only support ubuntu
<jml> actually, while you're talking about this
<sinzui> We have lots of registry distros. Only Ubuntu works. Debian kind of works because we have partial publishing history, no other distro works. The do not have packages or mirrors, they cannot have a bug tracker or be translated
<jml> https://dev.launchpad.net/RegistrySimplifications
<sinzui> I love the spec
<jml> although I think it's based on db tables rather than interfaces and so misses little globules of fun like __str__ and named_version
<jelmer> sinzui: But isn't the intention that Distribution and DistroSeries are generic and not Ubuntu-specific? None of the existing fields appear to be specific to Ubuntu.
<bigjools> I have run out of brain juice for one day.
<sinzui> jml: Sure there are anachronisms in the spec, but I think it really is what we need to do. The language we use in the UI really frustrates users
<sinzui> jelmer, yes, but the generic interpretation has proven to be useless for most use cases
<bigjools> good night everyone
<sinzui> jelmer, users cannot use distro object to run a project like Ubuntu. to make a derivative, you need to other objects
<jml> bigjools: g'night
<sinzui> jelmer, most users want a distroseries to manage packages, and they do not get them. Debian is faulty because you cannot report a bug in a Debian package that is not already in Ubuntu
<sinzui> source package names are debian names too. We cannot support Fedora or Gentoo: ack != ack-grep != text/ack
<jelmer> sinzui, the distroseries in Debian are used for e.g. packaging branches though. and it would be useful to be able to mark a bug as existing in Debian too, without it existing in Ubuntu.
<jelmer> sinzui: I see it's not entirely useful at the moment, but it seems a pity to me to make it more Ubuntu-specific now.
<sinzui> jelmer, I am not advocating making distros Ubuntu specific. I am pointing out that as the Launchpad's primary registry admin and answer contact, user are very disappointed. Generic is not good enough for them and no community is will to extend Lp to support their distro
<jelmer> sinzui: I don't see how in this particular case a text tag ("LTS") that is more generic versus a is_lts boolean would be less useful for users.
<jelmer> sinzui: Anyway, I didn't mean to derail this discussion.
<sinzui> jelmer, I think a bool allows pages and tooltip to show what LTS means
<lifeless> moin
<jml> hi
<jelmer> sinzui: Ah, hmm, I hadn't considered that.
<lifeless> oh awesome, we got hudson to pass
<jml> lifeless: awesome indeed.
<jml> g'night all
<lifeless> leonardr: ping
<leonardr> hi lifeless
<lifeless> I have a branch, which appears to break xx-wadl
<lifeless> I was wondering if I could forward the failure to you, and we can try to think of what it might be together
<leonardr> lifeless, sure
<lifeless> its sending now
<lifeless> please excuse the slightly crappy layout, I have't updated my submit-to-lp-tree yet
<deryck> bryceh, hey, can you mark bug 617699 qa-ok, since as I understand from the standup the qa is done?
<_mup_> Bug #617699: Export bug_tracker APIs for upstream components <qa-needstesting> <story-bugzilla-component-link> <Launchpad Bugs:Fix Committed by bryce> <https://launchpad.net/bugs/617699>
<deryck> and slide the card across, too, please.
<leonardr> lifeless: the error is this "_StringException: <unprintable _StringException object>" thing?
<leonardr> what happens when you run that test (or -vvt webservice) locally?
<lifeless> leonardr: look at the gz file
<lifeless> search for xx-wadl
<leonardr> ok
<lifeless> its got nonutf8 text or a utf8 reencoded blad of some sort, in the traceback
<lifeless>     print webservice.get(
<lifeless>         '/', 'application/vd.sun.wadl+xml', api_version='devel').body
<lifeless> Differences (ndiff with -expected +actual):
<lifeless>     - Some fake WADL.
<lifeless>     + <?xml version="1.0"?>
<lifeless> ...
 * leonardr having difficulty getting the gzip log out of evolution
<lifeless> right click save as ?
<leonardr> the context button for the attachment is depressing, and then popping back up, without offering me any actions or doing anything
<leonardr> ok, taking a look now
<lifeless> leonardr: ok, I think I see
<lifeless> its a literal 'testrunner' string that I missed.
<leonardr> lifeless: are you expecting the huge failures where you get real wadl instead of the string 'some fake wadl'?
<lifeless> no, which is why I asked for a hand :)
<lifeless> but staring at it with fresh eyes has helped me
<lifeless> I presume you wrote this ?
<lifeless> if so I'd like to give a little feedback
<leonardr> sure
<lifeless> literal string values (not keys) should set of alarm bells
<lifeless> try and find the object in use instead
<lifeless> e.g. http://pastebin.com/7GJFXNAS
<leonardr> oh, for testrunner. i thought you meant for 'some fake wadl'
<leonardr> sure
<leonardr> lifeless: yes, fixing the 'testrunner' problem will solve the failure
<leonardr> the fake cached wadl was being written to the wrong place
<lifeless> I'll throw it back at ec2
<lifeless> hopefully it will go through :)
 * leonardr crosses fingers
<lifeless> statik: around?
<lifeless> Ursinha: so, we're live with RFWTAD
<lifeless> Ursinha: how does it feel ?
<Ursinha> lifeless, I saw the email
<Ursinha> lifeless, we don't have edge, what do we have?
<lifeless> in what regard?
<lifeless> Ursinha: https://devpad.canonical.com/~lpqateam/qa_reports/deployment-stable.html seems odd
<Ursinha> lifeless, yeah, I'm checking that
<lifeless> Ursinha: both lpnet and edge are running 11704
<lifeless> qastaging is running tip of stable
<Ursinha> lifeless, I used to check edge as landed stuff from stable and staging for landed stuff from db-stable
<Ursinha> s/as/for/
<Ursinha> we don't have edge now, so...
<lifeless> use qastaging
<Ursinha> lifeless, we have both then
<Ursinha> cool
<lifeless> we're not pushing to production-stable now either
<lifeless> so we can take that config out
<lifeless> (I think thats why its showing older revs as a starting point)
<Ursinha> lifeless, exactly
<lifeless> and we can change the db-stable reference branch from production-stable to stable
<Ursinha> lifeless, where can I check for the latest deployed revision then
<lifeless> (because we're now going to deploy from stable)
<Ursinha> ah, stable
<Ursinha> hm, no
<Ursinha> from stable
<Ursinha> to?
<lifeless> lets start over :)
<Ursinha> hehe
<lifeless> we have updates to do for the db-stable and the stable report
<lifeless> lets do stable first
<lifeless> candidates to deploy are in the branch .../stable
<lifeless> deployed revisions are found by querying lpnet
<Ursinha> lpnet, right
<lifeless> what we need to do is
<lifeless> in get_edge_revision
<Ursinha> not querying production-stable branch anymore, but... scraping lpnet page?
<lifeless> s/edge/lpnet
<lifeless> yes
<Ursinha> ah, cool
<lifeless> and change edge.launchpad.net -> launchpad.net
<Ursinha> lifeless, mind filing a bug for that? I'll get to it then, might be fast
<lifeless> what else do we need to change
<lifeless> we need to not check the merged branch
<lifeless> because we're doing a push deployment
<lifeless> mmm, thats it
<lifeless> I think ?
<Ursinha> I guess so
<lifeless> Ursinha: it should be trivial, I have filed a bug for reference purposes
<lifeless> https://bugs.edge.launchpad.net/qa-tagger/+bug/664671
<_mup_> Bug #664671: deployment of stable has changed <qa-tagger:New> <https://launchpad.net/bugs/664671>
<Ursinha> lifeless, perfect
<Ursinha> thanks
<lifeless> jkakar: hey
<jkakar> lifeless: Hi!
<lifeless> abentley is having storm pains
<lifeless> I wonder if you can spare a few minutes to eyeball some symptoms
<jkakar> lifeless: Is it about the bug he posted?
<lifeless> possibly
<jkakar> lifeless: Anyway, yes, am happy to help but I need 5 minutes.  Will ping in a minute.
<jkakar> Or 5. :)
<lifeless> last I saw it was on lp-code, I suggested he take it to storm
<lifeless> sure thing
<lifeless> jkakar: yes, when you return -  https://bugs.edge.launchpad.net/storm/+bug/659316
<_mup_> Bug #659316: KeyError in storm from test <Launchpad Bazaar Integration:Triaged> <Storm:New> <https://launchpad.net/bugs/659316>
<lifeless> abentley: one thing you might try is the 0.18 of storm
<abentley> lifeless: I'll look into that.
<lifeless> its either released or just about to be
<lifeless> I know that we have a blocking bug which is blocking its release, but if that branch works with that specific test, we can focus on getting 0.18 released and in LP
<jkakar> lifeless, abentley: Okay am back... was ordering a new laptop battery to be delivered to the hotel during UDS.  Anyway, 0.18 isn't out yet, gary_poster is working on it, trunk will become 0.18.
<jkakar> abentley: Can you reproduce this outside of Launchpad code?
<jkakar> I don't have Launchpad code here and don't really want it to eat my hosts/apache/etc. :/
<abentley> jkakar: no, this is as simple as I've been able to make it.
<lifeless> jkakar: vm's ftw
<lifeless> jkakar: https://dev.launchpad.net/Running/VirtualMachine
<lifeless> jkakar: I like my hosts file too
<abentley> jkakar: so far, it's still using a lot of Launchpad foo like login, logout, and AFAICT, they're required to exercise the problem.
<jkakar> abentley: Okay.
<abentley> jkakar: I could set up a shared screen session, though I don't remember how ATM.
<jkakar> abentley: *That* would be helpful.
<jkakar> abentley: That plus Skype might do the trick.
<abentley> jkakar: okay, what SSH public key should I use?
<jkakar> abentley: The one on Launchpad, please.
<abentley> jkakar: you should be able to ssh to abentley@99.245.196.243 now.
<jkakar> abentley: It's asking me for your password.
<abentley> jkakar: my bad.  Try again?
<jkakar> abentley: Am in... stumpy!
<jkakar> abentley: Skype?
<abentley> jkakar: Yep.  What's your id?
<jkakar> abentley: jkakar
<benji> bzr uncommit is beautiful
<lifeless> thanks :)
<lifeless> [on behalf of the whole team, which I'm not in anymore ;P]
<benji> :)
<jkakar> abentley: https://bugs.edge.launchpad.net/storm/+bug/244769
<_mup_> Bug #244769: Reference == Int works to build join condition, but Int == Reference fails <Storm:New> <https://launchpad.net/bugs/244769>
<gary_poster> jkakar, lifeless: is this a bug to hold 0.18 on?
<gary_poster> or abentley
<abentley> gary_poster: I think no.
<gary_poster> ok thanks abentley
<rockstar> thumper, when you're around, I'd like to chat with you.
<thumper> rockstar: ack
<thumper> rockstar: I need to talk Maia to school shortly
<thumper> she slept in
<rockstar> thumper, my experience with Maia is that she talks you to school.
<thumper> rockstar: uh ha
<thumper> rockstar: we had a school concert last night
<thumper> and she didn't get to bed until 10pm
<thumper> she woke up about 15 minutes ago, and is just having breakfast
<jkakar> abentley: I've logged out, you can remove my SSH key.
<abentley> jkakar: cool.
<jkakar> lifeless: We didn't find the issue in Storm, but I think I may have enough to build a test case.
<jkakar> lifeless: We did find the right place to put a Store.flush call to fix the issue.
<lifeless> awesome
<jkakar> It's a very strange situation that I *think* I might kinda understand, but we'll see.  There's a lot going on in the code we reviewed to try to understand the issue.
<jkakar> Also, folks, if you have Storm issues feel free to ping, you don't have to wait for lifeless to ask me to help you. :)
<jkakar> If I can't help you figure out an issue we can at least file a bug and try to generate a test case.  We're friendly on #storm, come talk to us when you run into problems instead of spinning in frustration.
<deryck> I need a review.  Anyone game for reviewing a 143 line diff.
<thumper> rockstar: back
<rockstar> thumper, cool.  Lemme go into the skype room.
<wallyworld> morning
<rockstar> wallyworld, abentley, thumper, now fer the standup?
<wallyworld> rockstar: yep
<thumper> ack
<wallyworld> thumper: abentley rockstar: my sound dies when i did an upgrade it appears :-(. i'll try and fix
<wallyworld> abentley: i'm back
<wallyworld> rockstar: i have to do the school drop off this morning, can you ping me after your class?
<rockstar> wallyworld, sure.  It'll be in 90-ish minutes.
<wallyworld> ok. speak then
<rockstar> wallyworld, do you have to do it now.  My skype computer's time was off.
<wallyworld> rockstar: ?
<rockstar> wallyworld, I have about 10 minutes before I have to go.  I thought I was going to be late.
<wallyworld> ok. we can chat now
<rockstar> wallyworld, actually, let's talk after.  We shouldn't need to hurry.
<wallyworld> rockstar: ack
<wallyworld> lifeless: is it just me, or did a change to tachandler yesterday break the library layer startup when running tests?
<lifeless> wallyworld: just you, I hope
<lifeless> I mean, its possible I naffed it up, but hudson reckoned its ok
<wallyworld> lifeless: in TacTestSetup, i had to comment out self._waitForDaemonStartup()
<wallyworld> the refactoring, for me, seems to have broken that
<lifeless> well thats trivially looking for LOG_MAGIC
<wallyworld> yes, and it complains about it somehow - i'd have to check again to see the exact error
<lifeless> please
<wallyworld> ok. give me a minute
<wallyworld> lifeless: wtf, it worked just now. i switched to the relevant bzr branch and re-ran the same tests a yesterday. go figure
 * wallyworld duck out to do school dropoff
<Ursinha> lifeless, actually it's not just renaming get_edge_revision, because edge was a qa environment
<Ursinha> I need to get the qastaging revno instead
<Ursinha> and for checking which revision has landed, duplicate that scraping for lpnet instead of checking prod-stable branch
<Ursinha> landed in prod, that is
<lifeless> Ursinha: so the qastaging revno is the stable tip, more or less
<Ursinha> lifeless, yes
<lifeless> anyhow, whatever works is fine by me
<Ursinha> lifeless, I'll do what I just told you, that should work appropriately
<lifeless> cool
<lifeless> gmb: can you add your new flag to https://dev.launchpad.net/LEP/FeatureFlags
#launchpad-dev 2010-10-22
<lifeless> afk - hospital visit
<rockstar> wallyworld, hi
<wallyworld> rockstar: hello, just have to deal with a little domestic issue. can i ping you shortly?
<rockstar> wallyworld, sure.
<wallyworld> thnks
<rockstar> wallyworld, although there is a Michigan game on tonight, so I might be a bit distracted...  ;)
<wallyworld> rockstar: skype?
<thumper> rockstar: http://pastebin.ubuntu.com/517782/
<thumper> devel is not very happy right now
 * thumper tries windmill tests locally
<wallyworld> thumper: what was the failure message?
<thumper> wallyworld: full errors here - https://lpbuildbot.canonical.com/builders/lucid_lp/builds/274/steps/shell_6/logs/summary
<rockstar> thumper, yeah, I think all windmill tests should be disabled.
<rockstar> wallyworld, sure.
<thumper> rockstar: and replaced with what?
<rockstar> thumper, well, with jstestdriver, but for now, just not replace it.
<rockstar> thumper, basically, windmill tests aren't telling us what they're supposed to be telling us.
<rockstar> I mean, what good is a test that randomly decides it's broken?  Might as well have no tests at all.
<mwhudson> it's a mean thing to say, but the status of the windmill tests makes me glad not to be a fulltime lp developer any more :/
<rockstar> mwhudson, yeah.  :(
<beuno> thumper, we're using jstestdriver in ubuntu one and landscape
<thumper> beuno: and how's that working for you?
<beuno> hopefully, that knowledge will go back with rockstar and he can migrate LP, if it makes sense
<beuno> thumper, they are reliable tests, that I can tell you  :)
 * thumper wants reliable tests
<beuno> also, there's a nice way of TDDing with javascript
<beuno> which I have been meaning to send an email about for months
<beuno> my new plan is to talk to rockstar, and let him be the messenger and take all the credit
<rockstar> beuno, :)
<rockstar> Yeah, the fact that _GMAIL_ gets tested with jstestdriver indicates to me that it's very robust.
<beuno> they moved away from browsers clicking stuff
<beuno> it didn't scale and they spent more time chasing false positives than actual problems
 * thumper votes to slaughter windmill with prejudice
<thumper> beuno: I don't want to have six months of windmill pain before we get something reliable
<thumper> beuno: please send email :-)
<beuno> thumper, I will have rockstar locked up in a third world country for a week, I'll make sure someone does so
<thumper> beuno: what? Florida?
<beuno> more country
<beuno> buenos aires
<rockstar> thumper, yeah, this is where I'm stuck as well.  We have no resources to maintain windmill, but we also have no resources to switch.
<rockstar> I would make it a personal vendetta if branch merge queues weren't already my personal vendetta.  :)
<thumper> lifeless: what are your thoughts on killing windmill and instating jstestdriver?
<thumper> lifeless: we need to do something
<thumper> lifeless: because what we have right now is a pile of smelly doggy do-do
<rockstar> thumper, I think it's just a matter of getting jstestdriver in sourcedeps, creating a layer, and then migrating the windmill tests.
<thumper> no disrespect meant to dogs
<lifeless> thumper: I've no opinion
<lifeless> I hear terrible things about all the browser test drivers
<thumper> lifeless: no opinion given that windmill is causing most of our test failures?
<lifeless> given the heavyweight nature of these things, I'd like us, should we do it, to do it full on
<thumper> I can't imagine that you have no opinion, sorry
<lifeless> thumper: causing or showing
<lifeless> the windmill threads stuff is now sorted
<lifeless> see - hudson went green
<thumper> lifeless: but buildbot is broke
<lifeless> devel or db-devel ?
<thumper> devel
<thumper> well, both actually
<lifeless> WindmillTestClientException: {u'suite_name': u'Inline bug page subscribers test', u'result': False, u'starttime': u'2010-9-22T1:26:51.618Z', u'params': {u'id': u'subscribers-links', u'timeout': u'20000'}, u'output': None, u'endtime': u'2010-9-22T1:27:11.722Z', u'method': u'waits.forElement'}
<lifeless> looks like a legitimate error to my naive eye
<thumper> I ran the test locally from devel and it was fine
<lifeless> busy machine ?
<thumper> so, no... I don't think it is legitimate
<thumper> even a busy machine shouldn't take 20 seconds to do that
<lifeless> so, if we're getting spurious failures, we should fix them
<lifeless> *that* I have an opinion on
<lifeless> I simply don't know enough about the windmill design / implementation to have an opinion on *it*
<rockstar> lifeless, thumper, I think having mindshare in javascript testing with the rest of the web properties at Canonical is of value as well.
<thumper> rockstar: agreed
<thumper> my question is how to transition
<rockstar> lifeless, windmill will fail tests on random I/O issues.
<thumper> should we just say to each team: do it, and do it now! ?
<lifeless> rockstar: it can be nice to share war stories, but unless we make lp -massively- simpler to work with, I doubt that there will be much direct cross pollination
<thumper> rockstar: how random?
<rockstar> lifeless, the big issue is that the way to fix the spurious test failures is to fix it in windmill.
<lifeless> is that particularly hard?
<lifeless> Understand that I have -no- loyalty to windmill
<thumper> it is if you don't know where the problem is
<rockstar> lifeless, it is in that no one has any understanding of windmill.
<lifeless> As I say, I simply don't know enough about its guts to comment
<lifeless> rockstar: do -we- have that understanding of jstestdriver?
 * thumper forces a build just to see if they are transient
<rockstar> lifeless, well, maybe not anyone on Launchpad, but Landscape, lazr-js, and U1 all use it.
<rockstar> lifeless, I know enough to write tests in it, since lazr-js uses it, and I use lazr-js.
<lifeless> rockstar: by which I mean the LP team; most of the teams at canonical are pretty flat chat - we shouldn't *expect* that they can fix issues we may encounter
<lifeless> anyhow, as I say
<lifeless> I don't have an opinion other than: if we're going to do it, do it as a dedicated focus.
<lifeless> have two different test drivers around at once for firefox scares me
<lifeless> rockstar: one thing that would make a compelling difference to me is parallel testing
<rockstar> lifeless, actually, I think we just need to kill windmill.  It'll remove test coverage, but like I said, if you can't rely on the tests, are they really tests?
<lifeless> rockstar: mmm
<lifeless> I can understand the feeling
<lifeless> but false positives are not false negatives
<lifeless> AIUI the issue we have is false positives, not false negatives.
<lifeless> Which means our tests fail spuriously, but they don't pass when they shouldn't.
<rockstar> lifeless, yes, but test failures drastically reduce our productivity and velocity.
<lifeless> they do
<lifeless> and we should fix that
<lifeless> risk assessment
<lifeless> lets say that the windmill tests cover 35% of our js code
<lifeless> and that 10% of js landings hit valid windmill failures
<lifeless> and that thats 5 landings a month
<lifeless> that would say that 65% of js could be broken without safeguard, and that of the 355 coverage we have we catch a bug every two months
<lifeless> and that a bug a month gets through to be caught later (e.g. by users)
<rockstar> I can't think of a single thing that would be unusable without javascript.
<lifeless> rockstar: when it breaks, people get unhappy
<lifeless> it can break in many ways
<lifeless> I don't know what those figures would be
<lifeless> but I can't really agree *or* disagree about the relative merits of the test coverage we have vs the velocity cost that it has without some close observation - or figures /like/ those.
<lifeless> I agree its a problem
<lifeless> and that we should fix it
<lifeless> and that that will help our velocity a lot - its why I've fixed the new threads issue with windmill
<rockstar> lifeless, did you fix it, or just not consider it a test failure?
<lifeless> rockstar: did you see my email about this? I detailed most fo it
<lifeless> one thing I didn't mention in my email was that WindmillLayer *already* had an *explicit* code path to disable the thread checks: it wasn't working.
<wallyworld> lifeless: rockstar: i have tried to land branches 5 or 6 times this week and have had "failures" due to windmill
<wallyworld> i have just now "solved" a different windmill issue which has consumed waaaay to many hours of my time
<wallyworld> the sooner it goes bye bye, the better IMHO
<wallyworld> we don't have 10000s of windmill tests. surely we can introduce jstestdriver, write new tests using that, and migrate windmill ones as drive bys or whatever as time permits
<beuno> I smell a joint sprint
<beuno> lazr-js: the revenge of the tests
<beuno> also, bed
 * beuno waves
<lifeless> wallyworld: yes, threads were fixed lastnight
<wallyworld> beuno: /me smells a big, steaming pile of excrement and it's called windmill :-(
<wallyworld> \o/
<wallyworld> lifeless: it's not just the threads issue. there other nasty things at work too
<lifeless> sure
<wallyworld> if you add up my time, thumper's time, rockstar etc etc, and the future time we will surely waste and the scalability problems with the browser driven approach; the common tools/mindshare issue - surely a case can be made to swtich
<lifeless> sure
<lifeless> I'm not disputing that
<rockstar> I don't think anyone is saying "Windmill is great."  I think we're just proposing different solutions.
<wallyworld> i don't have much hair left to pull out :-)
<lifeless> james_w: https://code.edge.launchpad.net/~james-w/launchpad/more-matchers/+merge/32057
<lifeless> james_w: can we land? no?
 * StevenK wonders how to easily get failing tests out of a subunit stream
<lifeless> subunit-filter | subunit-ls
<bac> lifeless: i talked to james_w about some approved-but-not-landed branches a couple of weeks ago.  he indicated there was no hold up except his lack of time.
<bac> thanks for following up.
<lifeless> it may be worth just doing the changes and landing them
<lifeless> StevenK: did that help?
<StevenK> lifeless: Hm, did which?
<lifeless> a16:53  * StevenK wonders how to easily get failing tests out of a subunit stream
<lifeless> 16:54 < lifeless> subunit-filter | subunit-ls
<StevenK> lifeless: Yes, it did, thanks.
<lifeless> k
<lifeless> jml: http://bzr.bz
 * StevenK grumbles at failures in buildbot
<lifeless> StevenK: hows hudson
<StevenK> From my scrollback:
<StevenK> [02:56] < LPCIBot> Yippie, build fixed!
<StevenK> [02:56] < LPCIBot> Project devel build (142): FIXED in 4 hr 1 min:
<StevenK>                    https://hudson.wedontsleep.org/job/devel/142/
<StevenK> But since devel is failing, stable isn't getting anything and therefore db-devel isn't
<lifeless> yeah
<lifeless> stuckage
<wallyworld> \o/ i finally got a branch through ec2 without spurious windmill failures!
<lifeless> jml: cool
<lifeless> bah
<lifeless> wallyworld: cool
<lifeless> wallyworld: so regarding 'add js, remove windmill lazily'
<lifeless> wallyworld: I don't see this as a good strategy
<lifeless> wallyworld: because the problem isn't with *new* windmill tests, its primarily with windmill itself / existing tests.
<lifeless> wallyworld: I could see 'add jstestdriver, sprint (virtual or otherwise) on migrating'
<wallyworld> lifeless: the problems this week for me were with a new test
<lifeless> wallyworld: but adding a new thing which has its own failure modes and leaving the old one around indefinitely just means we're paying two sets of overhead
<wallyworld> i don't think we should have *both* windmill and jstestdrive - i just want one framework that is fit for purpose
<lifeless> right, me too
<wallyworld> if that means we introduce jstestdriver and allow a bit of slack to migrate existing windmill tests, so be it :-)
<lifeless> mmm
<lifeless> that worries me
<wallyworld> well, we can't let it become technical debt which is never repaid
<lifeless> previous attempts at that ... are still incomplete.
<lifeless> I certainly won't stop anyone doing this
<wgrant> Leaving technical debt around is the Launchpad Wayâ¢ :(
<lifeless> I just think its illadvised
<wallyworld> yes, so we would need buy in from the teams that they see it as a very good thing so would be motivated to help get in and fix it
<wallyworld> wgrant: oh no, not just the launchpad way. on my last job, there were 4 1/2 gui frameworks in use
<wgrant> wallyworld: That is... impressive.
<wallyworld> and i do mean 1/2, cause one of them was only ever 1/2 finished
<wgrant> ...
<wallyworld> what a noob i am - what does ... mean? you want more info?
<lifeless> wallyworld: its a verbal pause
<wgrant> Depressed silence, I guess.
<wallyworld> lifeless: there's not *that* many windmill tests to migrate. it would only take each of us taking on one or two test classes each, maybe 4 hours work, and it would be done
<lifeless> wallyworld: if the story were 'windmill is ok, want new shiny', I'd be like 'ok rad'
<lifeless> wallyworld: great! organise it!
<wallyworld> ok. i'll get some hard facts first. i'll talk to ohers about jstestdrive so i understand the scope a bit better
<lifeless> wallyworld: my specific concerns are:
<lifeless>  - another half finished migration
<lifeless>  - not resolving the problems
<wallyworld> +1
<lifeless>  - and adding in a new set
<lifeless> these are all variations on a theme
<wallyworld> yes, and at this stage, to my mind at least, we are in the "we have a problem, lets consider options" stage
<wallyworld> where option 1 is still keep windmill but fix it
<wallyworld> but given the obstacles to that and the other drivers for something better, imho we need to take the next step of scoping option 2
<wallyworld> once the cost is understood, we can make an informed decision
<wallyworld> and we also should include as part of it all migrating one or two tests to dip our toes in the water and fully understand hpw it would play out
<lifeless> sure, but on a branch :)
<wallyworld> huh? why would it be any other way?
<lifeless> wallyworld: I've seen some mad shit in my time
<lifeless> wallyworld: I didn't mean to insult
<wallyworld> lifeless: no problem, i didn't take it as an insult. just swalllowing the bait :-)
<thumper> hah
<thumper> all those windmill failures were transient
<thumper> due to our spurious test failure rules
<thumper> we should disable windmill
<StevenK> s/isable/elete/
<wgrant> I thought that was meant to be fixed :(
<wgrant> Like three times now.
<wgrant> In the last two days.
<StevenK> wgrant: It was mostly impacting hudson
 * thumper shrugs
<thumper> but I'm please buildbot is green
<StevenK> Same, I can land stuff now
<StevenK> Except PQM still thinks we are in testfix. Bleh
 * StevenK kicks it
<StevenK> Or won't PQM come out of testfix until db-devel is fixed?
<lifeless> it should have come out by now
 * StevenK tries to land again
<StevenK> Still fails with testfix
<adeuring> good morning
<StevenK> lifeless: I guess I'm right, since pqm is still in testfix
<jml> lifeless: StevenK: there's a short, indefinite period of time between submitting a testfix and coming out of testfix iirc
<wgrant> I recall something about it polling every 15 minutes.
<wgrant> But I could well be wrong.
<jml> dev.launchpad.net/Trunk/Glue iirc
<wgrant> "(Builbot's built-in "Force Build" button doesn't work for us for rather boring reasons).
<wgrant> "
<wgrant> That sounds very obsolete.
<StevenK> jml: The devel build was sucessful about an hour ago. That isn't enough to drag us out of testfix?
<jml> wgrant: it doesn't work for us
<jml> StevenK: probably
<jml> wgrant: we use a different, custom button
<wgrant> jml: Ahh, I see.
<StevenK> jml: So I should submit a branch as a testfix?
<jml> StevenK: I don't know.
<StevenK> Why does our PQM configuration have to be such a black hole?
<mrevell> Morning
<wgrant> StevenK: It's a rockstar conspiracy.
<wgrant> StevenK: To make us use Tarmac and merge queues.
<jml> mrevell: morning
<lifeless> StevenK: its in a branch, just need it pushed to lp (private)
<lifeless> StevenK: but this stuff is in BB anyway
<StevenK> lifeless: It still doesn't explain if buildbot needs both devel and db-devel built sucessfully before we are out of testfix or if we need to be pushed out of testfix manually
<lifeless> I was told that when bb starts a build it takes us out of testfix
<lifeless> that may have been a misstatement
<lifeless> so why aren't we building db-devel ?
<StevenK> Hell, I don't even know if devel => stable has happened
<lifeless> bzr pull
<lifeless> or missing
<jml> transparency!
<lifeless> or whatever, can tell you that
<jml> it's not even a river in Egypt
<lifeless> http://yoshinorimatsunobu.blogspot.com/2010/10/using-mysql-as-nosql-story-for.html
<jml> lifeless: that looks interesting. unfortunately, I have to go and have a driving lesson.
<lifeless> jml: what are you learning to drive?
<jml> lifeless: automobiles
<StevenK> lifeless: stable is missing 11 revs -- 11765 to 11775
<StevenK> jml: Manual or automatic?
<jml> manual.
<jml> good bye.
<lifeless> 11775 passed lpbuildbot
<lifeless> so it should have pulled them
<StevenK> jml: Enjoy!
<lifeless> https://lpbuildbot.canonical.com/builders/lucid_lp/builds/275
<StevenK> lifeless: Does prase run the poller? If so, could the lucid upgrade killed it?
<lifeless> I didn't think praÃ© was lucid yet
<StevenK> It was upgraded yesterday
<lifeless> ok
 * lifeless is suprised pqm kept working
<lifeless> StevenK: thanks for noticing this
<StevenK> Well, it hasn't clicked off testfix mode, so maybe it isn't working?
 * lifeless boggles at launchpad-users
<lifeless> (lucid on prae did break lp buildbot polling)
<lifeless> appears fixed now
<bigjools> there's nothing like a stable dev environment
<wgrant> Even Hudson is happy now. Things have to be going better :)
<lifeless> grah
<lifeless> my ec2 test runs are not sending failure mails
<lifeless> or success mails
<lifeless> grah grah
 * lifeless sends it to pqm
<lifeless> nothing like breaking it 16 hours before I fly
<wallyworld> lifeless: i'm having the same issue, but only for one particular branch
<bigjools> lifeless: I need to book some time with you and discuss the scalability of PPAs.  The increasing number of them is putting some strain on us.
<lifeless> are you @ UDS ?
<bigjools> yes
<lifeless> lets do it there?
<bigjools> sounds good
<bigjools> thanks
<lifeless> kk
<bigjools> process-death-row is taking 1h17m :/
<wgrant> !
<wgrant> What's it doing?
<bigjools> scanning a bazilion PPAs
<wgrant> That sounds like a stupid bug, not a real scalability issue.
<bigjools> possibly
<bigjools> but we also need to address the publishing cycle
 * bigjools brb
<wgrant> Yes, but that's also stupid bugs.
<wgrant> I think we should make the code not suck before we decide we have a serious scalability issue.
<wgrant> For starters it could not iterate over EVERY PPA EVER.
<wgrant> And then issue a probably unoptimised query.
<wgrant> And then commit around 20000 times.
<wgrant> I bet you could remove the archive-driven approach and reduce it to two queries to identify all the publications that need work.
<wgrant> And that would take a few seconds.
<wgrant> Not 4500.
<wgrant> Or we could fix the data model slightly and make dateremoved work sanely and then trivially query the archives that require consideration.
<lifeless> +1
<lifeless> but devil will be in the details
<bigjools> always
<bigjools> but the scalability issue remains nonetheless
<bigjools> lifeless: can the feature flag work cope with changes to existing pages that I don't want to release, but I do want to land?
<wgrant> They do.
<bigjools> I guess I need to read about FFs then
<bigjools> urg, this is going to be painful
<wgrant> It's not just a matter of guarding a couple of sections of the page?
<bigjools> StevenK: given that all your IDS stuff has landed, I am thinking that my changes to +addseries don't need to be behind a feature flag
<bigjools> wgrant: see https://dev.launchpad.net/LEP/DerivativeDistributions and you tell me :)
<bigjools> a lot of the existing stuff was cleaned up as well as new bits
<bigjools> but like I said, I think it can just land.
<bigjools> wgrant: actually I'd like your opinion on something
<wgrant> Hmmm.
<bigjools> Distroseries has a parent_series which we can use to work out if something is derived or not (if the parent_series's distro is different)
<bigjools> however this doesn't help for us making Ubuntu derived from Debian
<wgrant> The distroseries parent thing in that spec is screwed.
<wgrant> Will not work at all.
<wgrant> You need two parents.
<bigjools> such a delicate way of putting it
<wgrant> Heh, sorry.
<bigjools> the mockup was reviewed many times by many people
<wgrant> It certainly can't work for Debian->Ubuntu. And I don't see how it can work for OEM.
<wgrant> Unless OEM wants to restart from scratch every time.
<bigjools> explain?
<wgrant> An Ubuntu series' parent is the previous Ubuntu series.
<wgrant> But it needs derivation features from a Debian series.
<wgrant> For the OEM case, s/Ubuntu/OEM/, s/Debian/Ubuntu/
<bigjools> yes
<bigjools> this we are aware of
<wgrant> Unless you want to fork, you need to inherit from within your own distro then use the new derivation UI from some other distro.
<wgrant> The proposed model does not support that.
<bigjools> which model?
<wgrant> I guess the spec may not actually mean parent when it says parent.
<bigjools> there are 2 types of parent, it's a little ambiguous
<bigjools> but the "parent" in the mockup refers to the derived parent
<wgrant> Ahh.
<wgrant> And that's separate from the current model's parent?
<bigjools> well, don't confuse what's in the model currently with what's on the mockup
<bigjools> we're using the mockup to drive the model
<bigjools> the model may have to change
<bigjools> it may not
<bigjools> I am pondering whether to add a "derived_from" column to distroseries
<wgrant> bigjools: I'm not sure how best to term them.
<wgrant> We do need a new column. That much is certain.
<bigjools> I think you are right.
<StevenK> You didn't say that to me last night when I said I think we do.
<StevenK> :-P
<bigjools> StevenK: I didn't say much at all did I? :)
 * bigjools brb
<wgrant> derived_from seems fairly bad, but I can't think of anything better.
<wgrant> It's ambiguous :(
<deryck> Morning, all.
<bigjools> wgrant: derived_from_series ?
<bigjools> howdy deryck
<wgrant> bigjools: But it's derived from both.
<bigjools> wgrant: both what?
<wgrant> bigjools: Natty is derived from both Maverick and Sid.
<wgrant> So calling one 'parent' and one 'derived_from_series' isn't awfully unconfusing.
<bigjools> wgrant: not really, it's derived from something way back, and its parent_series is maverick
<wgrant> Well, it's still derived from Maverick, even if that's not what Soyuz calls it.
<bigjools> wgrant: we are using derivation explicitly to mean "copied from a different distro"
<bigjools> natty is not derived from maverick, it's just a continuation of the series
<bigjools> consider them point releases if you like
<wgrant> In the Soyuz sense of the term, sure.
<wgrant> derived_from_series is fine, but something that doesn't use an ambiguous word would be better.
<bigjools> what's ambiguous for you?
<wgrant> bigjools: Now I have been informed of the limited Soyuz-specific definition of the word 'derivation', nothing.
<wgrant> But anyone else reading the schema is likely to be terribly confused.
<bigjools> wgrant: we have to make these definitions for that very reason
<bigjools> the whole project is called "Derived Distributions" for gawd sake :)
<wgrant> True.
<bigjools> as long as the meanings are well documented and not terribly confusing I think it's ok
<wgrant> I am thinking of some Registry dev coming along and wondering WTF is the difference between parent and derived_from_series.
<wgrant> But then again this may be the two hours of sleep talking, so you could just ignore me.
<bigjools> heh
<bigjools> The interface docstrings should make it very clear, I hope.
<wgrant> Point.
<wgrant> So, with that clarified, I have retracted my long-held view that the spec is insane.
<wgrant> yay.
<bigjools> heh
<wgrant> I was operating under the assumption that "parent" meant "parent".
<bigjools> it does mean, however, that the spec is not clear enough
<bigjools> I may clear up the ambiguity with s/parent/derived parent/ or something.
<wgrant> I think that would be good.
<bigjools> I am still a little worried about making this work for Ununtu though
<wgrant> I may take a more detailed look at that spec tomorrow.
<bigjools> Ubuntu, even
<wgrant> And work out how it would work for Ubuntu.
<bigjools> the Linaro workflow will be quite different
<wgrant> Howso?
<wgrant> I know the Ubuntu one, but not really the Linaro one.
<wgrant> Does the Linaro one exist yet?
<bigjools> I doubt they'll derive once, and keep releasing new series
<wgrant> They'll re-derive each series, then copy some stuff up?
<bigjools> we are also only making the differences stuff work through packagesets
<bigjools> I suspect so, but someone like james_w will help clarify that
<jml> that reminds me
<jml> I've been meaning to ask how visible packagesets are in the UI and how customizable they are
<jml> let me rephrase
<jml> if we wanted to start displaying aggregated bug information for Ubuntu & other distros, with packages grouped by an expert according to some heirarchy, what structures do we already have?
<bigjools> jml: packagesets are only exposed via the API
<wgrant> They're also exposed on +queue.
<wgrant> But that's it.l
<bigjools> "exposed"
<wgrant> Heh.
 * bigjools -> fud
<bigjools> have we got any pages where pop-up help is working?
<mars> gmb, ping, I have a MP for the feature flag work, 'Work in Progress' until I add something Martin P. wanted in it.  With that, I'll put it up for review
<bigjools> sinzui: I'm starting to redesign the distroseries:+addseries page to work with derived distros.  I've got a few questions about how to arrange the form fields if you have a moment?
<gmb> mars: Okay, thanks.
<LPCIBot> Project devel build (143): SUCCESS in 3 hr 47 min: https://hudson.wedontsleep.org/job/devel/143/
<sinzui> bigjools, yes I can talk. Sorry about the delay
<bigjools> sinzui: hi
<sinzui> bigjools, mumble?
<bigjools> sinzui: here is fine so others can pitch in if they want
<bigjools> sinzui: I'm trying to decide on how to model the parent distribution so it meets Ubuntu's needs and the Linaro needs
<bigjools> sinzui: there's effectively 2 parents now
<sinzui> bigjools, the form currently suggests every series of every distro. Do you want to restrict it to Ubuntu, or Ubuntu supported/development series
<bigjools> sinzui: the one we derived from and the one we "branch" from
<bigjools> that is part of what I'm fixing yeah
<bigjools> so what I need to work out is what to present on the form and how to model it
<bigjools> sinzui: https://dev.launchpad.net/LEP/DerivativeDistributions
<bigjools> sinzui: we don't need to present the parent series on the form if the distro already has a series, we implicitly branch from the previous series
<sinzui> bigjools, Right, that is true for Ubuntu, but in the case of a partner, they may choose a previous series or want to "rebase" from an Ubuntu series?
<sinzui> bigjools, I image a lot of partners may only want to work with LTS series
<bigjools> sinzui: that's fine, it would be the initial derivation
<bigjools> any time a 2nd, 3rd, 4th series is added we don't support it being "derived" from anything other than the previous series
<sinzui> So that means partners will be creating multiple distros, possibly for the same client?
 * persia would very much expect that to happen
 * sinzui ponders each OEM customer sku having a distro with only one serie
<sinzui> s
<bigjools> sinzui: that's entirely possible
<bigjools> and I suspect the majority case
<sinzui> bigjools, I suspect that OEM hopes that each customer will have one or two distros, and each sku is a series. Bugs can be seen to affect multiple series
<bigjools> james_w: around?
<james_w> yes
<bigjools> sinzui: so my current thinking is to change that "parent series" drop down so that it only appears when truly deriving, and also make it only present Ubuntu series.
<bigjools> james_w: hi - we're just chatting about how the new distro series form will look
<bigjools> james_w: can you check the immediate scrollback and see what you think?
<james_w> bigjools, what you are saying sounds about right to me
<jml> mars: there's a conflict in your feature flags branch. if you want to fix it I'll be happy to review
<james_w> though I feel I'm going to be overlooking some of the subtleties
<sinzui> bigjools, 1. I agree. 2. there is a bug stating that parent series can be done in cases when the distro is not based on Ubuntu (fedora or fedora derivatives). If we do this change. We should mark the bug as wont fix...distros are Ubuntu or possibly Debian based only
<bigjools> james_w: that's also my worry
<sinzui> bigjools, This later assertion is true now. We only support Debian/Ubuntu sourcepackagenames
<bigjools> james_w: sinzui: what this means is that we can't support ubuntu deriving from debian for now
<bigjools> for a start debian doesn't have packagesets
<bigjools> and the difference calculations are based around those
<sinzui> We also do not have a complete set of Debian sourcepackagenames
<bigjools> right
<mars> jml, sure.  I was just running rf-get to fix it
<james_w> that should be something that is fixed somehow IMO
<jml> mars, cool
<sinzui> So with this change we can be honest and say Launchpad support Ubuntu
<james_w> only allow deriving from Ubuntu> what about deriving from derivatives?
<bigjools> good point
<james_w> If Ubuntu can't derive from Debian then I would be worried about whether the feature is usable
<bigjools> it will be usable for the more simple cases
<sinzui> james_w, in the case of Debian sourcepackagenames, we wait for the next sync from Debian to get the new names
<james_w> I don't think not having all the sourcepackagenames is a big issue is it?
<james_w> at the next sync new packages will be imported in to Debian, and then show up as eligible for syncing
<james_w> bigjools, what in particular makes Debian/Ubuntu complicated? Just packagesets and sourcepackagenames?
<bigjools> sinzui, james_w: I think what we need for ubuntu is to make the +addseries form take another parameter that lets you specify which of the parent distro's series you are syncing from
<sinzui> +1
<bigjools> james_w: it's because of the fact that ubuntu actually derived from debian a long time ago
<james_w> before you said that wouldn't be needed for Debian/Ubuntu, you would just add a relationship to warty, and it would work. Has that changed?
<bigjools> james_w: it's not need to specify the parent series in the same distro, because we only support the immediately preceding one
<mars> jml, fix pushed
<jml> mars: ta
<bigjools> but it would be useful to specify to series in the distro we derived from
<bigjools> s/to/the/
<james_w> right, and you said you weren't doing that before?
<bigjools> right
<james_w> parent series: where packages are copied from at i-f-p time?
<james_w> derived series: where packages can be synced from later?
<bigjools> yep - that's always series-1
<james_w> is parent series used for anything else than copying packages at the start?
<james_w> ordering?
<jml> mars: are we good to start using Python 2.6 features yet?
<bigjools> yep - it's most useful for difference comparing
<jml> mars: last I heard we still had one more thing to fix
<bigjools> err
<bigjools> let me re-state
<bigjools> parent series does not need to be specified, ever, we only support the previous one
<bigjools> derived series is what you want to compare and sync against
<james_w> ok
<mars> jml, we were just discussing that.  Do you remember what the 'one more' thing was?  praesodium was the big one, and it looks like it is upgraded!
<james_w> so, two things come to mind: what changes at the edge case when you create a first series
<sinzui> bigjools, thanks! that explanation of how initialization works is helpful
<jml> mars: no, I don't. I'd have to search my mail, same as you :) But I think it was just praes.
<james_w> and what is the difference between the parent and the derived series for most of the life of the series?
<mars> jml, pity, should have talked to lifeless and mthaddon yesterday.  They're probably getting ready to fly now.  Maybe we could hammer it down on Monday.
<james_w> edge case> there is no parent series that can be inferred. At that point do you want to be able to specify a parent series from another distro?
<bigjools> james_w: parent is NULL for the first series
<jml> mars: that would be good.
<sinzui> james_w, right which is what happens with a new distro. Series are not require BTW. our nasty fedora and gentoo records do not have series
<jml> mars: I only ask because of the namedtuple use in your branch.
<mars> yeah, that was optimistic (same as my earlier Py2.6 listmail)
<bigjools> james_w: its presence on the database isn't really useful
<mars> jml, I can strip it out if we really want to get this landed
<jml> mars: fair enough.
<james_w> bigjools, ok, so if I create jamesbuntu, which is to be an Ubuntu derivative, my first series won't have a parent, and so be empty to start?
 * sinzui uses "nasty" to mean it looks operational in Lp, but it is not and users are very confused
<bigjools> james_w: yes, but the new form will allow you to specify where you want it initialized from
<james_w> I'm wondering if parent/derived is a useful distinction, and whether it is really a list of parents.
<bigjools> so derived_series will be natty, or whatever
<bigjools> james_w: it's entirely useless AFAIAC
<bigjools> sinzui: is parent_series used in registry for anything useful?
<sinzui> bigjools, no
<james_w> so, jamesbuntu maverick parents: [maverick], jamesbuntu natty parents: [jamesbuntu-maverick, natty]?
<sinzui> bigjools, I think it was a field to remind someone what to use when running the initialization scripts
<bigjools> heh
<bigjools> james_w: I don't understand what you're trying to express there
<james_w> bigjools, just trying to work out if parents can be a list without loss of generality
<bigjools> james_w: there's only one useful parent, that's the series you're syncing from/comparing with
<james_w> because it seems to me that it would make some of the questions we are asking entirely redundant
<bigjools> I think so
<james_w> bigjools, why can't that be more than one series? Comparing nattty with maverick could also be useful.
<james_w> seeing what are SRU candidates, noting which SRUs need to be merged in to natty...
<bigjools> james_w: I see it could be useful, I'm not sure if what we've done so far will work like that :/
<jml> mars: From 'test_fixture_deletes_existing_values', I gather that using a second FeatureFixture in a test reverts whatever changes were made by first one.
<jml> mars: Is this correct?
<mars> jml, yes.  Fixtures have a subtle trait of nuking the database before setting anything new
<mars> jml, err, feature flags have that trait
<james_w> ISTM that a parent/derived distinction is entirely useless, raises lots of odd questions, and also limits what you can do, so I want to confirm that and advocate for removing it if possible
<jml> mars: so this is a property of feature flags, rather than the new helper?
<mars> yes
<james_w> bigjools, any particular reason, or just that the assumption is baked in from very early on in Launchpad's development?
<jml> mars: hmm. ok. I guess it's out-of-scope for your branch. I find that kind of surprising, and am not sure it's a desirable behaviour.
<bigjools> james_w: baked in from the previous discussions around https://dev.launchpad.net/LEP/DerivativeDistributions
<mars> jml, lib/lp/services/features/rulesource.py:104: store.execute('DELETE FROM FeatureFlag')
<mars> jml, in setAllRules()
<jml> mars: thanks.
<james_w> bigjools, damn
<james_w> I should have fought for this louder and sooner
<bigjools> james_w: the UI stuff is all based around the diffs from the thing you derived from
<bigjools> and that is all mostly done
<james_w> bigjools, well, I presume the code could in theory diff any two series?
<bigjools> james_w: in theory yes
 * jml butts in
<james_w> (diffing arbitrary series would have its uses too)
<sinzui> bigjools, how does this affect a project like batlix. It is a derived distro (a remix) It cannot current use Lp to build itself. Baltix's first series is a derivative of Debian, later series are derivatives of Ubuntu
<jml> also, the code isn't probably not going to get any easier to change
 * bigjools sees jml's butt
<bigjools> sinzui: it would not be any different from the other distros I think
<james_w> jml, any particular insights, or are you just waving your butt around? :-)
<jml> just general ones, like "I'd rather we do something good even if that involves re-doing work"
<bigjools> define good
<jml> let me catch up with the actual content of the discussion though :)
<sinzui> bigjools, batlix is an IDerivativeDistro. Will the rules change when create a distro or a series to distinguish between Ubuntu, a partner, a community that remixes Ubuntu, a community that wants a distro that is not based on Ubuntu
<bigjools> sinzui: I've no idea what IDerivativeDistro is!
<sinzui> Read the __init__ in the distro model
<bigjools> sinzui: initially, we're only letting certain people use this stuff since it has massive performance implications
<sinzui> It basically means it is crippled. No soyuz, not mirrors, no packages
<bigjools> ok
<sinzui> bigjools, IDD was created during 3.0 to ensure the distro pages works for ubuntu, but did not break when used with batlix and gentoo
<bigjools> right
<bigjools> sinzui: how is the decision made if it's IDerivativeDistro or not?
<sinzui> bigjools, self == lpceleb.ubuntu
<bigjools> sinzui:  /o\
<sinzui> bigjools, I think there is an XXX saying this will have to change one day
<bigjools> ok
<sinzui> bigjools, there is exactly 1 place that need to change
<bigjools> that's good, I guess :)
<sinzui> however, I think we need to ask if we are going from two tiers of service to three tiers
<bigjools> I certainly have three tears right now
<sinzui> does baltix get packages now? can satanic-ubuntu use Lp to make it's remixes?
<jml> two elephants and a cymbal fall off a cliff
<jml> ok. caught up now I think.
<jml> wow, this sounds an awful lot like bzr branches
<jml> why don't we do it that way?
<jml> (list of parents, left-hand parent is special)
<sinzui> bigjools, I described how users are remixing ubuntu in a blog post last month I think. They need to use project groups, projects, and teams. I propose that we rename the baltix class of project to IRemixDistribution so that there is no confusion in the code
<james_w> +1
<bigjools> seems fair
<jml> I also think we should keep aiming to supporting the concept of Ubuntu as a Debian derivative, and Launchpad as fully functional for Debian derivatives
<persia> So, semantically, I'd like to preserve the idea that "Derivative" implies rebuilds of stuff and "Remix" implies just adding random stuff.
<jml> but I'm still not clear on exactly what the cost of that is relevant to the derivative distro LEP work
<jml> (I know there's other registry bugs, but one thing at a time)
<sinzui> jml, So batlic gets archives, builds, and mirrors?
<jml> sinzui: baltix? why not?
<bigjools> persia: does a new Ubuntu series fit outside of those two for you?
<jml> sinzui: maybe not mirrors though. Are we doing mirrors for linaro?
<persia> bigjools, Yes, very much so.
<bigjools> persia: great, that's how I see it too
<persia> The words that are important to be are "Release", "Flavour", "Remix", Derivative".
<sinzui> jml, I think the real blockers are archives, builds, and translations. The mirror script is not broken by the mirrors that were wrongly registered
<persia> s/be/me/
 * sinzui saw no reason to remove mirrors since nothing was visibly broken
<persia> "Release" is each Ubuntu series, "Flavour" is something with packagesets, "Remix" is a "Release"+ a PPA or equivalent (or maybe not).  "Derivative" rebuilds masses of stuff, changes lots of things, etc.
<bigjools> I like that
<bigjools> sinzui: this is something we can use in the UI
<sinzui> jml: I am cautiously +1 on permitting any community to have a derivative. Packages and bug fixes are usable by the Ubuntu community
<jml> sinzui: well, I think there are two distinct questions: should Launchpad be capable of such? should we allow such as a matter of policy?
<sinzui> persia, I agree with your definitions
<bigjools> sinzui: we need to be careful, we can't let just anyone start creating derivatives that generate huge numbers of database rows
<jml> sinzui: I don't really have a strong opinion about the second question, but I will answer a firm "yes" to the first.
<persia> bigjools, Note that "remix" sometimes means specific things in terms of the Ubuntu trademark license, for remixes of Ubuntu: you might want to seek wider clarification before using it in the UI.
<jml> and indeed you'll want to do user testing to get input from non-persias.
<sinzui> bigjools, yes, that is what I am cautious about.
<bigjools> we can let them register the distro itself as a particular type of course.
<bigjools> perhaps we need a new type of series where we can flag it as being managed by LP/Soyuz
<jml> bigjools: perhaps it's just a matter of whether it has an associated archive or not
<bigjools> jml: maybe, I'd need to think through the consequences
<bigjools> because the new derived stuff will need to create an archive somehow for certain derived distros
<bigjools> (given permissions)
<jml> bigjools: sure, I'm not strongly advocating that particular solution, just that we also should think carefully about adding new types.
<bigjools> agreed
<bigjools> but sometimes it's good to be explicit
<jml> or in this case, new types of types
<jml> james_w: I'm sorry, I've lost the thread of conversation where you were talking about fighting for a particular change earlier.
<james_w> jml, you mean you aren't sure which change I was talking about in particular?
<jml> james_w: yes, and I'm also not sure where the conversation about that change went after that :)
<james_w> jml, we paused for your butt :-)
<jml> james_w: a wise move.
<james_w> jml, I was referring to having a list of parents, though looking back I was never as explicit about it as I believed
<bigjools> sinzui: I want to remove "title" as a DB column
<bigjools> it should be generated I think
<jml> james_w: ok yeah, that sounds like a great idea
<jml> bigjools: why can't we have a list of parents?
<sinzui> bigjools, +1
<mars> jml, thanks for the review feedback
<jml> mars: my pleasuer.
<jml> mars: except, correctly spelled.
<bigjools> jml: it never came up in the initial discussions and noodles has now finished the main differences UI page
<sinzui> bigjools, I was contemplating the removal of .title from views and templates in December. We may need to do that sooner :)
<bigjools> which deals with differences with one parent
<jml> and now we're going to have to dig a Â£13bn pound tunnel underneath London to change it?
<bigjools> sinzui: I removed it from my mockup a while ago :)
<james_w> well, there's two slightly different things, having >1 derived parent, and making the parent series less of a special case
<james_w> the former definitely came up
<sinzui> bigjools, I would like an lts field. it can be a bool, but to support other distros, it should be a text field. We want to show users "Ubuntu 10.04 LTS"
<bigjools> the parent series is not used anywhere
<james_w> as for the UI, I presume it would be feasible to do comparison against one parent at a time?
<james_w> n-way diff of archives is hard enough to think about, let alone do a UI for
<bigjools> exactly :)
<jml> james_w: right.
<bigjools> we could do *something* as a comparison against more parents
<mars> Anyone else noticed that testr has started dumping "configs/testrunner_12345/" directories into the configs dir?
<jml> I thought we had to have something like a three-way comparison to handle the MoM case?
<bigjools> jml: I don't think it would require a Jubilee Line extension
<mars> It makes --strict bzr commits really annoying
<james_w> plus, Mark specifically said he was interested in multiple derived parents, so delivering that would be plus.
<bigjools> we do a 3-way right now in the UI
<jml> bigjools: ok, so maybe we could find the funding somehow. :)
<jml> (incidentally, crossrail is making my life slightly more inconvenient)
<bigjools> jml: right :)
<james_w> jml, yeah, we need 3 way, but going beyond THIS, OTHER and BASE is mighty complicated
<jml> sure.
<jml> james_w: I guess there too we can look to bzr for precedent
<bigjools> I think that we could add something in to do arbitrary parents
<bigjools> it just needs loads more data in the DB to be generated
<james_w> though 2-way diff with multiple series wouldn't be too hard
<jml> james_w: insofar as it allows N parents, and blesses the left-most, and basically only has UI for 2 parents
<james_w> as in listing the versions that each has
<jml> bigjools: perhaps there are ways around that.
<james_w> jml, right, allowing for diffs against any of the parents on demand, but not trying to show all of the diffs in "log -p" etc.
<bigjools> jml: not with the current schema
<bigjools> we're about to start work on the backend that works out what the differences are between series
<jml> bigjools: schemas can be changed
<bigjools> jml: anything can be changed, how much time have you got?
<jml> bigjools: for this project, quite a bit.
<bigjools> jml: well I have at least one interested party breathing down my neck :)
<jml> bigjools: I'm not worried about that so much.
 * james_w -> lunch
<jml> bigjools: we have an SLA, and if we can deliver useful things incrementally so much the better. But I don't think we need to aim lower if the only motivation is a deadline.
<bigjools> jml: I don't think that's ever been the case.
<jml> bigjools: cool :)
<cody-somerville> +1 :)
 * jml retreats into shell to do end-of-week stuff
<bigjools> good night people, see you at UDS
<LPCIBot> Project devel build (144): SUCCESS in 3 hr 28 min: https://hudson.wedontsleep.org/job/devel/144/
<LPCIBot> * Launchpad Patch Queue Manager: [r=allenap][ui=none][bug=664327] API:
<LPCIBot> IHasTranslationImports.getTranslationImportQueueEntries.
<LPCIBot> * Launchpad Patch Queue Manager: [r=maris, henninge][ui=salgado,
<LPCIBot> sinzui][bug=663436] Move most of the OAuth doctests into unit tests.
<LPCIBot> Make it possible to request a temporary integration of one's
<LPCIBot> desktop into the Launchpad web service.
<LPCIBot> * Launchpad Patch Queue Manager: [r=lifeless][ui=none][bug=664380] Add the PPA name to the Subject of
<LPCIBot> the reject message sent.
<LPCIBot> * Launchpad Patch Queue Manager: [r=allenap][ui=none][bug=608621] Fix the InitialiseDistroSeriesJob
<LPCIBot> cronscript to actually work.
<lifeless> moin
<jml> g'night all
<LPCIBot> Project db-devel build (91): STILL FAILING in 3 hr 58 min: https://hudson.wedontsleep.org/job/db-devel/91/
<deryck> mars, ping (when you're back).
<dobey> what's the normal procedure for getting a ~vcs-imports branch updated to point to the right place? file a 'question' on lp?
<persia> Yes
<dobey> ok, thanks
<mars> Hi deryck, what's up?
<bryceh> hmm, I've just merged db-devel trunk into my branch and now make schema fails...
<bryceh> http://pastebin.ubuntu.com/518226/
<gary_poster> bryceh: cd download-cache; bzr up; cd ..
 * gary_poster needs to find time to get rid of the download-cache
<bryceh> gary_poster, thanks
<gary_poster> np
<deryck> rockstar, ping
<rockstar> deryck, pong
<deryck> rockstar, yo dude.  Did you get those Windmill tests working?  I never saw bugs.
<rockstar> deryck, I still have yet to get that branch landed.
<rockstar> deryck, for context, windmill is preventing MANY code branches from landing right now.
<deryck> rockstar, so what can we do to move it forward?  I guess we can talk about it at UDS.  gmb isn't working on it next week anyway.
<rockstar> deryck, I'll keep trying.  I think we should talk seriously about it at UDS.
<rockstar> Mostly, I think we should just toss windmill.
<deryck> well, we could disable tests for now.  tossing windmill seems ambitious to get a branch landed. ;)
<rockstar> deryck, yeah, I disabled those tests, but every time I ec2 test it, it disappears in the black hole of ec2.
<deryck> rockstar, ah, ok.
<lifeless> something new is wrong with ec2test
<lifeless> I saw that too, but my change landed and passed both buildbot and hudson ok
<lamont> 2010-10-22 14:32:50+0000 [-]   File "/srv/launchpad.net/codelines/soyuz-production-rev-9886/lib/lp/buildmaster/model/builder.py", line 333, in updateBuilderStatus
<lamont> 2010-10-22 14:32:50+0000 [-]     MAX_EINTR_RETRIES, _update_builder_status, builder, logger=logger)
<lamont> 2010-10-22 14:32:50+0000 [-]   File "/srv/launchpad.net/codelines/soyuz-production-rev-9886/lib/lp/services/osutils.py", line 78, in until_no_eintr
<lamont> 2010-10-22 14:32:50+0000 [-]     return function(*args, **kwargs)
<lamont> wgrant: ^^  I SAID MAX_EINTR_RETRIES was a bad idea....
<rockstar> abentley, wow.  It's time to stand up, huh?  Mumble?
<abentley> rockstar: sure.
<lifeless> mars: hi
<lifeless> mars: I'd still like a config setting to completely turn off profiling
<mars> lifeless, could you please add that comment to the MP, "Needs Fixing"?
<LPCIBot> Project devel build (145): SUCCESS in 3 hr 27 min: https://hudson.wedontsleep.org/job/devel/145/
<LPCIBot> * Launchpad Patch Queue Manager: [r=mwhudson][ui=none][no-qa] Fix glob imports in python tests.
<LPCIBot> * Launchpad Patch Queue Manager: [r=edwin-grubbs][ui=none][bug=244527] Moves JoinNotAllowed to
<LPCIBot> registry.errors. This registers it on the webservice so that it
<LPCIBot> reports HTTP 400 BAD REQUEST across the API instead of a server error.
<LPCIBot> * Launchpad Patch Queue Manager: [r=deryck][ui=none][no-qa] Unit tests for BugTaskSet.search() and
<LPCIBot> BugTaskSet.searchBugIds()
<LPCIBot> * Launchpad Patch Queue Manager: [r=allenap,
<LPCIBot> danilo][ui=none][bug=608631] Users now can use the [nnbsp] tag to
<LPCIBot> input a narrow no-break space in Rosetta. This tag
<LPCIBot> representation is also a workaround for some Browsers (eg. in
<LPCIBot> Rekonq NNBSP is displayed as a zero-width character).
<LPCIBot> Project parallel-test build (11): STILL FAILING in 2 hr 26 min: https://hudson.wedontsleep.org/job/parallel-test/11/
<LPCIBot> Project db-devel build (92): STILL FAILING in 3 hr 30 min: https://hudson.wedontsleep.org/job/db-devel/92/
<LPCIBot> * Launchpad Patch Queue Manager: [r=lifeless][ui=none][no-qa] Fix failing tests.
<LPCIBot> * Launchpad Patch Queue Manager: [rs=buildbot-poller] automatic merge from stable. Revisions: 11776
<LPCIBot> included.
<wgrant> lamont: What happened?
#launchpad-dev 2010-10-23
<LPCIBot> Project devel build (146): FAILURE in 3 hr 29 min: https://hudson.wedontsleep.org/job/devel/146/
<LPCIBot> * Launchpad Patch Queue Manager: [r=bac][ui=none][bug=327688] Add a parameter for created_since to
<LPCIBot> searchTasks so that bug tasks created after a specific date can
<LPCIBot> be searched for using the API.
<LPCIBot> * Launchpad Patch Queue Manager: [r=jelmer][ui=none][bug=664096][incr] Lock FIXRELEASED bugtask status
<LPCIBot> so that only project maintainers and bug supervisors can
<LPCIBot> transition away from that status.
<wgrant> :(
<lifeless> airports!
<wgrant> Fun fun.
<wgrant> Your flight hasn't been cancelled or delayed by several hours yet?
<lifeless> I hope not
<lifeless> 25 minutes till boarding to akl
<lifeless> then 2 hours transfer
<lifeless> then transfer in la
<wgrant> Urrrgh LAX.
<lifeless> then then transfer in dfw
<wgrant> Would you like some more transfers?
<lifeless> then mco
<lifeless> wgrant: if it comes with fries
<persia> That can be arranged
<lifeless> and with that, fuck you all
<lifeless> :)
<wgrant> Heh.
<lifeless> -> queue
<LPCIBot> Yippie, build fixed!
<LPCIBot> Project db-devel build (93): FIXED in 3 hr 56 min: https://hudson.wedontsleep.org/job/db-devel/93/
<jcsackett> "and with that, fuck you all" should be the new "performance tuesday" subject line.
<wgrant> Heh.
<wgrant> I wonder if devel will pass this time.
<wgrant> I can't reproduce the failure.
<lamont> wgrant: ivy fell over hard in some way, caused exceptions in buildd-manager instead of just silently getting killed
<lamont> after I disabled ivy, and re-restarted buildd-manager, all is happy again
<wgrant> lamont: Yay.
<wgrant> lamont: Maybe the b-m rewrite will fix it.
<wgrant> That's apparently being merged soon.
<wgrant> lamont: Also, where are all the builders?
<lamont> oh gah
<lamont> one more thing to fix before I sleep
<lamont> wgrant: what do you mean where are all the builders?
<lamont> we're down maybe 3 each for i386/amd64
<lamont> don't scare me like that. :-p
<wgrant> Hm, I thought there were a few more than three missing.
<lamont> well... I'd shifted several over to amd64 last night
<wgrant> Ah.
<wgrant> The queues are oddly long at the moment.
<wgrant> Considering that amd64 had caught up a couple of hours ago.
<lamont> buildd-manager was down for several hours
<wgrant> Yeah, but amd64 (but not i386) caught up after that .
<wgrant> I guess this lot is just the dailies.
<lamont> that's prolly much of it.  I also did a mass giveback this morning
<LPCIBot> Project devel build (147): STILL FAILING in 3 hr 30 min: https://hudson.wedontsleep.org/job/devel/147/
<LPCIBot> * Launchpad Patch Queue Manager: [r=gmb][ui=none][no-qa] more unit tests for BugTaskSet.search()
<LPCIBot> * Launchpad Patch Queue Manager: [r=adeuring][ui=none][no-qa] Makes is_security_proxied_or_harmless()
<LPCIBot> check set,
<LPCIBot> frozenset and mapping types for proxied objects. Previously objects of
<LPCIBot> these types would be considered okay.
<wgrant> So, I just reinstalled and ran make schema, then started installing a buildd VM.
<wgrant> The VM Ubuntu installation finished before make schema.
<LPCIBot> Project db-devel build (94): SUCCESS in 3 hr 56 min: https://hudson.wedontsleep.org/job/db-devel/94/
<wgrant> No LOSAs around, I suppose?
<wgrant> edge has been a bit upset for a few hours.
<wgrant> Returning truncated responses sometimes, 502s others.
<wgrant> Last time this happened, the appservers were overloaded because half of them failed to upgrade properly.
<LPCIBot> Project devel build (148): STILL FAILING in 3 hr 54 min: https://hudson.wedontsleep.org/job/devel/148/
<LPCIBot> Project db-devel build (95): SUCCESS in 3 hr 37 min: https://hudson.wedontsleep.org/job/db-devel/95/
<LPCIBot> Launchpad Patch Queue Manager: [rs=buildbot-poller] automatic merge from stable. Revisions: 11785,
<LPCIBot> 11786, 11787, 11788 included.
<wgrant> Intriguing.
<bac> hi wgrant
<wgrant> Evening bac.
<bac> hey so your report about edge is likely causing the buildbot failure with 503, no?
<wgrant> I can't see buildbot, but I would expect more of a 502 than a 503.
<wgrant> But a 503 is possible.
<bac> wgrant: getting 503's trying to update-sourcecode.
<bac> it makes it through a few packages and then dies.
<wgrant> Sounds relevant.
<wgrant> Can you see load graphs of some kind for the edge appservers?
<bac> wgrant: i'll look for graphs
<jml> bac: getting 503s from lp alias lookup is not uncommon
<jml> don't know what it means though, other than "server not working so well"
<bac> jml: i've seen pages not loading completely too, and wgrant is seeing 502s
<jml> uh
<jml> bac: ahh, ok. that is more serious.
<wgrant> jml: It's doing the same thing as it was a couple of months back when some appservers failed to upgrade.
<wgrant> Truncating some responses after exactly 16KiB.
<bac> jml: yeah, i've been trying to decide if it meets 'critical'
<wgrant> And other times returning 502s.
<bac> wgrant: yes, i've seen that
<bac> 16K truncation
<jml> bac: what's down, exactly?
<bac> jml: hard to say.  xmlrpc.edge has spurious failures and edge web apps are failing as described above
<jml> failing, but not consistently?
<bac> certainly degraded experience
<bac> jml: yes.  pages are truncated.  reloads generally succeed
<bac> buildbot is consistently dying
<bac> wgrant, what else have you seen?
<jml> hmm.
<jml> I wish https://lpstats.canonical.com/graphs/AppServer5XXsEdge/ had percentage of 5XXs
<bac> jml: i would say we are here:
<bac> launchpad.net API/web UI degraded, unplanned with very limited customer impact
<bac> 1 working day
<jml> bac: fair enough.
<bac> jml: what is the y axis on that graph?
<wgrant> bac: Does edge count as launchpad.net?
<jml> wgrant: not really. the rules are pretty fuzzy.
<wgrant> Yay for deleting edge, then.
<jml> bac: could you send an email to the losas and canonical-launchpad so that spm can check this out first thing on his Monday?
<bac> wgrant: and are we sure it only affects edge or is edge all *we* see?
<wgrant> bac: Good question.
<bac> jml: i will
<jml> bac: thanks.
<jml> bac: I think it's 5XXs per second
<wgrant> My failing scripts are both using edge.
<jml> wgrant: yeah. bzrlib using edge by default is also a pain.
<jml> but it was either that or fix Python.
<wgrant> I thought that got fixed at some point.
<bac> jml: those numbers seem high to be failures/sec...but i guess that is plausible
<jml> wgrant: I don't think so, but icbw.
#launchpad-dev 2010-10-24
<lifeless> grah
<lifeless> Traceback (most recent call last):
<lifeless>  File "fti.py", line 714, in <module>
<lifeless>    sys.exit(main())
<lifeless>  File "fti.py", line 664, in main
<lifeless>    con = connect(lp.dbuser)
<lifeless>  File "/var/launchpad/test/lib/canonical/database/sqlbase.py", line 796, in connect
<lifeless>    con = psycopg2.connect(connect_string(user, dbname))
<lifeless> psycopg2.OperationalError: FATAL:  database "launchpad_dev" does not exist
<lifeless> make[1]: *** [create] Error 1
<lifeless> Failed to create database or load sampledata.
<lifeless> make: *** [check] Error 1
<lifeless> does Launchpad.log_with need the second parameter
<lifeless> login_with
<jcsackett> lifeless: is the second param the cachedir path?
<jcsackett> i think all the params are required.
<lifeless> jcsackett: no, its instance apparently
<lifeless> default is, of all things, staging
<lifeless> speaking of staging, its help is terrible.
<jcsackett> lifeless: huh. good to know.
<lifeless> where should folk create accounts for staging ?
<jcsackett> https://login.staging.launchpad.net/+new_account
<jcsackett> lifeless, you should have a staging account already; my understanding is that periodically account info from the lp sso is moved to the staging.lp sso.
<jcsackett> i can't tell you where i gleaned that info though, or even if it is correct.
<lifeless> jcsackett: its not for me
<lifeless> jcsackett: its for the help text on stging
<lifeless> ./lib/lp/registry/help/home-page-staging-help.html
<jcsackett> lifeless: ah, i see.
<lifeless> I'm touching that file
<lifeless> and it seems bogus
<jcsackett> yeah, so, i should say that one *can* make an account via that link. i have no idea if they *should*.
<lifeless> so I wanted to update it while I was here.
<jcsackett> lifeless sounds good.
<lifeless> but it seems nonobvious :)
<lifeless> so perhaps you could file a bug ?
<jcsackett> lifeless: does it need a bug if you're updating the docs now? or have i misinterpreted something?
<lifeless> I don't know what to update them with
<lifeless> we don't know what we should be saying
<lifeless> I'm removing references to 'edge'
<lifeless> and I don't want to block on figuring out the right answer to an opportunisitic fix.
<jcsackett> lifeless: fair. i'll throw in a bug saying we need better staging documentation.
<lifeless> reference that file perhaps ;)
#launchpad-dev 2011-10-17
<lifeless> desperately seeking susan^Wreviewer
<lifeless> checkwatches.base needs expurgation of its oops stuff.
<lifeless> wgrant: StevenK: I need a hand, make jsbuild is failing, and I have no idea why ;)
<lifeless> I'm guessing one of you ran into that recently
<lifeless> wallyworld: or ^
<wallyworld> lifeless: what's the issue?
<lifeless> no rule to make target 'lib/canonical/launchpad/icing/yui/yui/yui.js' needed by .../launchpad.js
<wallyworld> hmmm. haven't seen anything like that in a while. last time that sort of thing happened, a make clean got it going again
<lifeless> trying that, thanks
<wallyworld> it sounds like the yui symlinks got messed up
<lifeless> mwhudson: can we nuke vostok-archive ?
<mwhudson> lifeless: yeah, i reckon so
<mwhudson> nuke all of vostok if it's getting in the way any i guess
<lifeless> wallyworld: thank, that works (which means we have a bug in our dep rules)
<lifeless> mwhudson: it just seems stubby to me
<lifeless> mwhudson: as in, an abandoned experiment
<mwhudson> yep
<mwhudson> (hey at least i managed to clean up the publisher code in the rest of lp a bit when i added it ...)
<lifeless> \o/
<lifeless> and I'm back to nuking getLastOops calls
<wallyworld> what are those calls being replaced with?
<lifeless> self.oopses[-1]
<wallyworld> ok
<lifeless> or direct subscription in doctests
<lifeless> why ?
<lifeless> I mean, if you're hacking in that area, we can collaborate
<lifeless> I will broadcast to the list once I've sorted it all out
<wallyworld> i was just curious
<lifeless> kk :)
<lifeless> so getLastOopsReport is not isolated between tests
<lifeless> so test A can write an oops test B sees
<lifeless> and we've had this happening
<lifeless> its also not threadsafe
<lifeless> (test thread A can write a report test thread B thinks is its)
<wallyworld> yeah, i recall some issues from a while ago in this area, can't remember the details
<rsalveti> hey, don't know if someone reported this already, but it seems launchpad is blocking new packages to be both published and built today
<rsalveti> https://launchpad.net/~linaro-maintainers/+archive/overlay/+packages
<rsalveti> I copied a few packages from other series today, and also pushed new ones, but they are all waiting for hours already
<StevenK> Yes, we're debugging an issue with the PPA publisher
<rsalveti> StevenK: ok, great
<rsalveti> yeah, the issue is just at the publisher, just saw that the packages I pushed today were all built fine
<rsalveti> but they're now all locked at the publisher
<lifeless> hah
<lifeless> mailman doc tests are not being run
<StevenK> I thought that was by design?
<lifeless> for our monkey patches? no
<lifeless> wow
<lifeless> test_uploadProcessor is full of pain
<wallyworld> lifeless: i'm having a weird feature flag issue that has got me stumped, can you spare a minute to help a poor lost soul?
<lifeless> sure
<wallyworld> if i uncomment the commented out line, it works: https://pastebin.canonical.com/54429/
<wallyworld> so something is messing with the feature flag infrastructure
<wallyworld> to add a bit of content - i am expecting delete_allowed to be true
<lifeless> and whats it set to in the test ?
<wallyworld> flags = {u"disclosure.delete_bugtask.enabled": u"on"}
<wallyworld> there are other tests which all work
<wallyworld> but for this test, the flag cannot be found
<lifeless> the rule :)
<lifeless> the flag is always found, but may evaluation to None
<lifeless> what does getFeatureFlag(
<lifeless>             'disclosure.delete_bugtask.enabled')
<lifeless> evaluate to ?
<lifeless> oh, and are you sure its reaching your code?
<wgrant> I assume the authorization cache is being populated by the owner check.
<wallyworld> none i think, i'll check
<lifeless> zope security caches
 * lifeless bites back a biting commentary on using caches to solve architectural problems
<lifeless> (and yes, I'm aware of CPU layer-N caches)
<wallyworld> here's a test which works for example: i have a security adaptor and it is getting to that point ok
<wallyworld> oops
<wallyworld> paste error
<wallyworld> https://pastebin.canonical.com/54430/
<wallyworld> and to answer your question, getFeatureFlag('xxxx') evaluates to None
<StevenK> I didn't think flags worked inside the security adapter.
<wallyworld> inside the security adaptor, unless i uncomment that line
<wallyworld> StevenK: they appear to work, at least for the tests i have which pass :-)
<wgrant> StevenK: No reason they shouldn't.
<wgrant> Apart from caching issues like this :)
<wgrant> We'll see if they're relevant soon...
<wallyworld> not sure if it's a caching issue per se
<wgrant> It very probably is.
<lifeless> wallyworld: I'd like to see the following: a print / pdb session in the security adapter showing the feature controller and the evaluation of the flag, and if using print, prints before and after the call so we can eliminate other entries into the codepath
<StevenK> wgrant: Where is your obsolete-distroseries fix at?
<wgrant> StevenK: I should finish that off today, good point.
<wallyworld> lifeless: any particular attributes of the feature controller?
<lifeless> nope
 * wallyworld starts gather data
<wgrant> mwhudson: *cough* beautifulsoup on codeimport creation forms.
<mwhudson> wgrant: heh heh heh
<mwhudson> is that still there?
<wgrant> I dare not check.
<wgrant> lib/lp/code/browser/codeimport.py:from BeautifulSoup import BeautifulSoup
<wgrant> lib/lp/code/browser/codeimport.py:        soup = BeautifulSoup(self.widgets['rcs_type']())
<wgrant> Yes
<mwhudson> wgrant: beatifulsoup > re.compile(r'(?<=class=["\'])(.*)(?=["\'])') though
<wgrant> True.
<mwhudson> wgrant: i think the beautifulsoup thing falls into my bucket of "automatic form generation is a crock of **** and you shouldn't use it ever"
<wgrant> mwhudson: Somewhat like ORMs that lazy-load, automatic form generation makes small things easy but inevitably screws you over completely in expensive ways.
<wallyworld> lifeless: here's some printed data. the act of putting in the code to print the data made the test pass. https://pastebin.canonical.com/54431/
<lifeless> 15:52 < lifeless> nope
<lifeless> 15:53  * wallyworld starts gather data
<lifeless> before my adsl stopped
<wallyworld> lifeless: here's some printed data. the act of putting in the code to print the data made the test pass. https://pastebin.canonical.com/54431/
 * StevenK starts a fund to buy lifeless some better Internets
<lifeless> take out a hit on telecom
<StevenK> Haha
<wallyworld> the first printout is just before the check_permission call
<lifeless> wallyworld: I wanted the controller itself :)
<wallyworld> ah
<lifeless> wallyworld: so I could see if it was falling back to a different object
 * wallyworld tries again
<lifeless> e.g. due to the participation / interaction being futzed or something weird
<StevenK> lifeless: My DSL has been connected since the end of Aug
<lifeless> interesting data point that the debug fixed it
<StevenK> So 45 days or so
<StevenK> lifeless: I don't recall you having issues when you were in Epping
<lifeless> StevenK: indeed
<lifeless> StevenK: thus, telecom.
<lifeless> I will ring and rant soon
<StevenK> Do you hold out much hope?
<wallyworld> lifeless: before and inside the call, the feature controller is <lp.services.features.flags.FeatureController object at 0xf7d4b10>
<wallyworld> it fails without all the other print statements
<wallyworld> i'll see if i can find which print statement makes it work
<lifeless> StevenK: if we can id the issue, yes
<lifeless> thats mainly dependent on getting through first level 'technical' support
<StevenK> Oh, absolutely
<lifeless> which the rant is all about
<wallyworld> lifeless: so calling features.getAllFlags() before the check_permission call makes it work (as well as bug.default_bugtask)
<wallyworld> and not calling either of those 2 things makes it fail
<wgrant> jtv: There's a regression fix for cocoplum that I'd like to deploy tonight, and it's stuck behind your translations-export-to-branch fix. Are you likely to have QA for that done in the next 4 or so hours?
<jtv> wgrant: I think I will, but it won't be much sooner.
<wgrant> OK. I may have to cowboy it anyway, since there's an existing possibly unclobberable cowboy there.
<wallyworld> lifeless: narrowed it down to feature_controller.rule_source.getAllRulesAsDict() - so it seems the StormFeatureRuleSource() content is getting clobbered somehow?
<lifeless> I wonder if you can't trigger the first feature rule lookup from within a security adapter because they don't nest? [speculation]
<wallyworld> not sure
<wallyworld> but it all seems rather fragile at the moment
<lifeless> wgrant: what do you think about my speculation ?
<wgrant> lifeless: What don't nest?
<lifeless> wgrant: I'm trying to come up with a story that would explain wallyworlds symptoms
<wallyworld> weird thing is that bug.default_bugtask also makes it work
<lifeless> you probably need to step through with pdb
<wallyworld> yeah, started to do that. lots of api calls to look at
<lifeless> zope is phat
<wallyworld> at least it's not something dumb i'm doing wrong (hopefully)
<wallyworld> seems like a genuine problem with the infrastructure
<wgrant> lifeless: Are you sure the mailman test bug is actually a bug? We deliberately don't run MailmanLayer by default, because it's crap.
<lifeless> wgrant: we have tests that exist; they should be run, or not exist.
<lifeless> wgrant: otherwise they -will- bitrot and -will- just accumulate debt
<StevenK> Delete them, then
<lifeless> I found a bug that they were not being run, but it was fix released years ago indicating they were meant to run again.
<lifeless> StevenK: not without chatting to curtis I think
<StevenK> I wish I had more knowledge so we could delete lib/mailman
<wgrant> s/I/LP/ s/more knowledge/some architecture/
<StevenK> Harsh
<lifeless> 6 uses of getLastOops
<lifeless> mwhudson: hey, around ?
<lifeless> mwhudson: have a quickie on codehosting
<mwhudson> lifeless: yep
<lifeless> mwhudson: make_error_utility appends the pid to the oops prefix
<wgrant> Haahahah
<lifeless> is this for any reason *other* than oops sucking at concurrency
<wgrant> I think we established that there isn't.
<mwhudson> lifeless: no, i'm pretty sure that's only to avoid races
<wgrant> It's there to avoid concurrency issues, and to confuse the shit out of everyone.
<lifeless> mwhudson: as the processes are ephemeral, I'm checking theres not need to keep that
<lifeless> mwhudson: great, deleted.
<lifeless> wgrant: if you want confusing
<lifeless> grep for setOopsToken and note the EMAIL usage
<lifeless> mwhudson: I'm assuming pullerworker is the same ?
<wgrant> Huh. Handy.
<mwhudson> lifeless: yes
<lifeless> wgrant: do you happen to know if thats semantic or just crazy
<wgrant> lifeless: I assume crazy.
<wgrant> But don't know for sure.
<wgrant> Delete it and see if matsubara complains? :)
<lifeless> aieee @ available_oops_prefixes
<wgrant> Delete
<lifeless> oh man
<lifeless> I hope its not punning that with concurrency limiting
<wgrant> lol
<wgrant> lifeless: I think you may have projectegg set badly in python-oops-tools
<wgrant> It currently tries to use oops-tools.settings as the settings module.
<wgrant> Which obviously isn't going to work :)
<wallyworld_> lifeless: i've found the problem. the setAllRules() method on StormFeatureRuleSource needs a store.flush(). Or else the rules passed into the fixture setup are not written to tge db because check_permission() has a @block_implicit_flushes decorator
<wgrant> Hahaha
<wallyworld_> and those other things which trigger the test tp pass must have done so because they caused a flush
<lifeless> wgrant: thats fixed by my branch
<lifeless> wgrant: you're welcome to review it if you want
<wgrant> omg
* wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 262
<StevenK> wgrant: Hm?
<wgrant> The number went down :)
<wgrant> Significantly.
<StevenK> By 7
<wgrant> It was actually 272 this morning.
<StevenK> Oh, so 10
<mwhudson> 26 days to go!?
<wgrant> Heh
<StevenK> mwhudson: You tell funny jokes
<StevenK> I think I have to file one, anyway
<StevenK> I can't see DSP:+questions in our bugs
<mwhudson> wgrant: is it the ppa publisher that's backed up?
<wgrant> The queries are quick, so it's unlikely to time out much.
<wgrant> mwhudson: Was, but yes.
<wgrant> mwhudson: It's been fixed for a few hours.
<wgrant> And now even has scriptmonitor running on it.
<mwhudson> wgrant: ok, how often does it run when it's not backed up?
<wgrant> Every 5 minutes.
<wgrant> But often only every 10.
<wgrant> Because it's crap.
 * mwhudson has a vague memory of */20
<mwhudson> hah
<mwhudson> ok
<wgrant> It was */20 long ago
<lifeless> wgrant: you've been spelunking a lot; whats the fastest way to tell if script X has an oops config
<lifeless> wgrant: you've been spelunking a lot; whats the fastest way to tell if script X has an oops config
<wgrant> lifeless: Somewhat disturbingly, despite porting dozens of scripts around to LaunchpadScript and rewriting its internals, I've not run into that bit of code.
<lifeless> I want to check the setOopsToken('EMAIL') thing is safe when gone, if you see what i mean
<wgrant> Oh, that's lovely.
<wgrant> Scripts normally just call globalErrorUtility.configure('something') themselves.
<lifeless> +214
<lifeless> -687
<lifeless> 1874 lines of diff
<lifeless> and we're not done yet
<lifeless> + it would be freakishly hard to make this separate branches
<lifeless> I pity the fool^Wreviewer
<lifeless> StevenK: oops -> critical
<lifeless> StevenK: also, we don't use confirmed :)
<lifeless> poolie: hi, can we talk about your pending writes branch briefly ?
<poolie> sure!
<poolie> here, or phone?
<lifeless> either, whats your pref ?
<poolie> let's start here
<lifeless> so, the bug (as I read it) is that when someone pushes twice in quick succession, we don't update the merge diff properly
<lifeless> there are a few different orders to the race condition
<lifeless> sometimes we generate the error and update proplerl
<poolie> i think that's how you reach it yes
<lifeless> y
<poolie> yes it seems so
<lifeless> sometimes we generate the error and don't update properly
<poolie> that may be possible
<lifeless> I'm worried that you're papering over the issue
<poolie> i can understand that concern
<poolie> however
<poolie> i think there are really two bugs
<poolie> 1- "sometimes mp diffs are not generated if the branch is repeatedly written to"
<poolie> 2- "launchpad sends pointless spam"
<poolie> i'm trying to fix 2
<poolie> i'm not sure if 1 actually exists
<lifeless> I'm sure it does
<lifeless> per the analysis in comment #2
<poolie> i thought that perhaps the completion of the second write would cause a new job to be generated
<poolie> perhaps there is some ordering where that doesn't happen
<lifeless> can only have one job outstanding for the branch
<poolie> at any rate i don't see how leaving bug 2 open helps us fix bug 1
<poolie> at the moment we don't even log when this occurs!
<lifeless> so if the first job hasn't finished erroring before the second job is created, the second job isn't made and the first job just fails.
<lifeless> poolie: we don't generate an OOPS ?
<poolie> no
<lifeless> ok
<poolie> it is telling only the users who can't do anything about it
<poolie> unless the idea is to annoy them (me) into fixing the whole bug :)
<poolie> which is a valid, though risky, strategy
<lifeless> so, I think bug 1, which is the bug your branch purports to be about, is about fixing the race condition
<_mup_> Bug #1: Microsoft has a majority market share <iso-testing> <ubuntu> <Clubdistro:Confirmed> <Computer Science Ubuntu:Confirmed for compscibuntu-bugs> <dylan.NET.Reflection:Invalid> <dylan.NET:Invalid> <EasyPeasy Overview:Invalid by ramvi> <GenOS:In Progress by gen-os> <GNOME Screensaver:Won't Fix> <Ichthux:Invalid by raphink> <JAK LINUX:Invalid> <LibreOffice:In Progress by bjoern-michaelsen> <Linux:New> <Linux Mint:In Progress> <The Linux OS P
<lifeless> bah, 1-
<lifeless> :)
<poolie> my mp is only about suppressing the mail
<poolie> i get annoyed by the mail but i never see a missing diff
<lifeless> And also, issue 1- is the only one where the user has no control over the situation
<poolie> because they can delete/filter the mail?
<lifeless> poolie: because they can push content into the branch, or delete the mp if they had done something crazy
<lifeless> I admire your desire to stop sending spam, but I don't think, except for case 1-, that these branch mails -are- spam
<lifeless> and case 1- has an analysis of the race condition, just needs coding
<poolie> lots of people seem to disagree :)
<lifeless> poolie: not on that bug
<poolie> :(
<lifeless> poolie: in general, 'lp sends too much mail', sure : but telling you something you requested fails is useful
<poolie> it doesn't tell you what failed
<poolie> as james said "I don't really know what it means, which merge proposal it is referring to, or what
<poolie> I can do about it, so I don't know why I got an email about it."
<poolie> lp really should not be sending that
<poolie> it's different to bugmail
<lifeless> so, the other bug, which I've unduped, is about the lack of context
<poolie> which one?
<lifeless> fixing that will address some of james_w's mystery around the mail
<lifeless> bug 640882
<_mup_> Bug #640882: " Launchpad error while generating the diff for a merge proposal" mails don't indicate branch <code-review> <email> <lp-code> <Launchpad itself:Triaged> < https://launchpad.net/bugs/640882 >
<poolie> ok
<poolie> i'll just drop it
<poolie> i'm sad because i was trying to make this a little less crap and i feel like it's being held hostage to fixing the whole thing
<poolie> not sending pointless mail to people is a step forward
<poolie> recording when something goes wrong is a step forward
<lifeless> I'd love to see improvements here, I don't think masking the issue is one; fixing the issue (which should be ~ as simple as self.suspend(5 minutes) or something) would be
<lifeless> poolie: I agree that not sending pointless mail is a step forward; and recording when it goes wrong is a step forward.
<poolie> how is this masking it?
<lifeless> poolie: My understanding was that you were going to squelch the email for this case, and that that was the sum of the branch
<poolie> and i was going to log that it failed
<poolie> ok, if just doing self.suspend(5 minutes) is enough, i'll try that
<lifeless> I'm handwaving
<mrevell> Hi
<lifeless> poolie: jobs have a defer-for-a-bit system, I don't know the details.
<poolie> i don't feel you and aaron are taking into account the actual user data here
<lifeless> poolie: I think for the pending-writes case, logging and not emailing is fine; I agree with Aaron that the other cases are different enough not to change.
<poolie> nobody is saying "i'm glad lp told me about this" or "that explains why my thing had no diff"
<lifeless> poolie: I think fixing the issue, logging and not emailing is even better
<lifeless> I feel like you are saying 'not getting email is more important than the system working'
<lifeless> I know thats not what you mean
<lifeless> but it kindof feels that way
<poolie> mm
<lifeless> I think you mean 'not sending email in this case is better even if its not fixed', and I've acked that - twice I think - above
<lifeless> pending writes shouldn't be categorised as a user error
<poolie> mm
<poolie> i get more annoyance from lp spam than i do from diffs being missing
<poolie> in that sense it's more important
<poolie> and, generally, there are always going to be some errors, and i think handling them gracefully is important
<lifeless> I get annoyance from devs having to spend time tracking down, *again*, a self inflicted case of user confusion
<lifeless> :)
<poolie> why 'self inflicted'?
<lifeless> (self inflicted by us developers)
<poolie> oh i see
<lifeless> poolie: because we created a system with a race condition, classified it as user error, and tada
<poolie> yep
<poolie> and, i think, did not look at the actual mail that was sent
<lifeless> this needs two changes: unclassify it as user error, and fix the race condition
<poolie> yep
<lifeless> and yes, the lack of context in the mail is the icing on the cake
<poolie> if the race is as simple as just rescheduling the job i can do that
<lifeless> I think that if the branch has pending writes, the job should just wait for it
<lifeless> indefinitely
<poolie> i guess the 'lack of context' bug can then apply to other mail sent about branches, if any
<lifeless> poolie: there are, IIRC, 3 other cases for MP's where the same template is used for the email
<poolie> i agree, though i think the "users need to trust whether lp is working" argument applies equally there
<lifeless> poolie: cases which this bugfix won't impact
<lifeless> poolie: well, pushing up an empty branch and proposing it for merge, *is* a user error
<lifeless> if you get one of those mails, I think its helpful (if it told you the branch :))
<poolie> yes, bug 640882 will be irrelevant to the specific case it complains about, but relevant to things like empty branches
<_mup_> Bug #640882: " Launchpad error while generating the diff for a merge proposal" mails don't indicate branch <code-review> <email> <lp-code> <Launchpad itself:Triaged> < https://launchpad.net/bugs/640882 >
<poolie> ok
<poolie> so
<poolie> this would have been a lot easier if someone had just said "why don't you just call self.suspend and that will probably fix it"
<poolie> in the first place
<lifeless> poolie: that would have been nice
<lifeless> understand I'm handwaving, bug there is something like that there :)
<lifeless> and I'll be happy (tomorrow) to go spelunking with you looking for it
<poolie> it's probably fairly obvious on the base class
<poolie> so then no oops, just deferral
<poolie> and maybe a log message
<lifeless> yah
<jtv> wgrant: Q/A for those codehosting translations-export bugs is done.  Go ahead.
<wgrant> jtv: Thanks.
<poolie> lifeless: most of the bugs have all the issues of "no context" and "shouldn't get this mail anyhow" tangled together
<poolie> please don't undupe them all
<lifeless> poolie: there were two that were previously a unit, and you'd moved to the other bug, I was just restoring the,
<lifeless> poolie: (i.e. I've no more tweaking planned on these bugs)
<StevenK> lifeless: The bug I filed is not the cause of the OOPS -- that is already filed.
<StevenK> lifeless: I used High since the bug is *shown* in the OOPS, but isn't the cause.
<lifeless> StevenK: ah, that wasn't clear to me. Sorry for creating noise.
<StevenK> lifeless: Should I set it back to High, then?
<lifeless> StevenK: up to you; lazy evaluation and timeouts can be nonobvious - we may well have timeouts due to that bug anyhow
<lifeless> (it is a timeout isn't it ?)
<StevenK> lifeless: Yes, but the timeout is due to the direction=backward madness
<lifeless> ah right, a clear cause :)
<StevenK> jtv: Bug 375013 is not marked OK, but 812500 is
<_mup_> Bug #375013: Cannot commit directly to a stacked branch <commit> <launchpad> <lp-translations> <qa-ok> <rodeo2011> <stacking> <Bazaar:Fix Released by jameinel> <bzr-builder:Fix Released by jelmer> <Launchpad itself:Fix Committed by jtv> < https://launchpad.net/bugs/375013 >
<jtv> StevenK: I guess one of my changes didn't come through.  Hang on.
<poolie> lifeless: i think the other thing here is https://bugs.launchpad.net/launchpad/+bug/483945
<_mup_> Bug #483945: No way to ask Launchpad to refresh a stale diff <code-review> <lp-code> <mp-preview-diff> <openstack> <trivial> <Launchpad itself:Triaged> < https://launchpad.net/bugs/483945 >
<poolie> to give people a way tor ecover
<lifeless> poolie: that would be nice
<jtv> StevenK: actually, it did come through.  So the deployment report simply hasn't picked it up yet.
<jtv> StevenK: the one that's not marked OK yet is the one I updated last, IIRC.  Here's hoping this is not a problem with multiple bugtasks.
<StevenK> Can I get a review? https://code.launchpad.net/~stevenk/launchpad/dsp-questions-statement-death/+merge/79519
<adeuring> goood mornin
<StevenK> adeuring: Hi, I know it's not your OCR day, but would you mind a small (+22/-5) review? https://code.launchpad.net/~stevenk/launchpad/dsp-questions-statement-death/+merge/79519
<lifeless> wgrant: hi
<lifeless> wgrant: https://code.launchpad.net/~lifeless/python-oops-tools/amqp/+merge/79505
<lifeless> adeuring: StevenK: I've reviewed that branch
<lifeless> wgrant: baaah
<StevenK> Oh, have you?
<lifeless> StevenK: https://code.launchpad.net/~lifeless/python-oops-tools/amqp/+merge/79505
 * StevenK refreshes
<StevenK> lifeless: Haha
<StevenK> lifeless: Fine, I'll look at your MP then. :-P
<StevenK> lifeless: If you look at the OOPS attached to the bug it did 1,200 SPN queries
<adeuring> StevenK: anyway, a nice branch!
<lifeless> StevenK: yes, crazy shit man
<StevenK> adeuring: :-)
<StevenK> lifeless: r=me
<lifeless> thanks!
<stub> How do I set the submit branch?
<StevenK> bzr push --remember
<wgrant> That's the push branch.
<StevenK> Bah
<wgrant> wgrant@lucid-test-lp:~/launchpad/lp-branches/bug-876171$ grep -A1 lp-branches.$ ~/.bazaar/locations.conf
<wgrant> [/home/wgrant/launchpad/lp-branches]
<wgrant> submit_branch = bzr+ssh://bazaar.launchpad.net/~launchpad-pqm/launchpad/devel
<lifeless> the default submit branch is the parent branch isn't it ?
<wgrant> I think that applies in some places but not others.
<stub> I gotta weird one from an old branch and nothing in locations.conf
<wgrant> stub: .bzr/branch/branch.conf?
<stub>   submit branch: bzr+ssh://bazaar.launchpad.net/%2Bbranch/losa-db-scripts/
 * stub looks
 * StevenK peers at the e-mail he just got from PQM
<stub> yer - in there.
<wgrant> StevenK: Did lifeless set the wrong default reviewer?
<StevenK> PQMException: 'Failed to verify signature: gpgv exited with error code 2
<wgrant> Yeah.
<StevenK> I don't think I caused that
 * wgrant looks.
<wgrant> You sent email to it.
 * wgrant fixes.
<StevenK> Oh, the project is broken.
<wgrant> The branch, specifically.
<StevenK> Right
<nigelb> Morning everyone
<nigelb> On second thoughts, evening :)
<rvba> lifeless: Hi Rob, may I email you with the details of a test isolation failure I'm fighting with?  Maybe you'll have an idea on what's going on.
<lifeless> sure
<rvba> Thanks :)
<rvba> This is on a branch that is really not urgent.
<lifeless> \o/ \o/ \o/ \o/ \o/ \o/ \o/ \o/ \o/ \o/ \o/ \o/ \o/ \o/ \o/ \o/ \o/ \o/ \o/ \o/ \o/ \o/ \o/ \o/
<wgrant> Oh?
<lifeless> (I have a loopback test grabbing oopses off of amqp into TestCase.oopses
<wgrant> oh, nice.
<lifeless> the end is in sight
<lifeless> rvba: I will look tomorrow
<rvba> lifeless: Thank you.
<lifeless> rvba: if you want to look in the interim, I'd question the state that rabbit is in when the test starts
<rvba> lifeless: hum, right, I'll have a look.
<lifeless> rvba: I haven't looked at your failures yet but
<lifeless> I note that 'session' is a thread local
<lifeless> rvba: and they are generally trouble
<lifeless> rvba: in particular, I don't see anythin in RabbitSession to handle rabbit going down
<nigelb> bigjools: lol @ cricket :D
<lifeless> rvba: currently, as I read the code, the first reset of rabbit will break all LP appservers trying to use rabbit
<lifeless> bigjools: ^ you might like to file a bug on that, for fixing before we go love.
<lifeless> *live*
<bigjools> nigelb: meh
<lifeless> bigjools: (or tell me I'm wrong :))
<nigelb> muhahaha
<rvba> lifeless: that's the reason why we wanted to isolation rabbit's failures as much as possible.
<rvba> s/isolation/isolate/
<bigjools> nigelb: it's probably the dodgy food upsetting them
<lifeless> rvba: sure, but we need to handle things like rabbit being reset
<lifeless> rvba: the problem is that RabbitSession assumes it *never* goes away, but the Unreliable variant assumes that *any error doesn't matter*
<lifeless> rvba: there is a middle ground
<lifeless> where an EPIPE is handled by reconnecting, but other errors cause a failure
<rvba> lifeless: true, but since we don't know where this middle ground is we wanted to use the Unreliable variant and watch what was happening (hence the agressive oops logging).
<rvba> lifeless: that's the purpose of the branch in question.
<lifeless> ok, so I can give a -little- guidance (I'm a novice here too :P) about the middle ground
<lifeless> there are two broad cases I see
<lifeless> firstly if rabbit goes away and comes back, the first transaction *after* it comes back it should start working
<lifeless> this is testable (see the oops_amqp tests for inspiration)
<lifeless> secondly, if rabbit goes away and comes back, messages queue for send-at-end during the current transaction should also be sent
<lifeless> (also testable :P) - only if it goes and stays gone, should we be going into zomg mode
<lifeless> rvba: I think aggressive logging and oopsing is great as well
<lifeless> rvba: these two things just seem like high frequency cases we can anticipate
<rvba> lifeless: makes sense.
<lifeless> wgrant: btw the rabbit-management package should really put the cli on sbin
<rvba> lifeless: I'll file bugs about these two cases.
<lifeless> rvba: thanks!
<rvba> lifeless: thank *you* ;)
<lifeless> rvba: I have a minor tweak to rabbit.py in lp:~lifeless/launchpad-useoops - dunno when the branch will land, but thought a headsup might be useful
<rvba> Okay, I'll have a look.
<lifeless> bah
<lifeless> lp:~lifeless/launchpad/useoops
<wgrant> lifeless: Probably, yeah.
<lifeless> for now, -> bed
<wgrant> lifeless: File a bug against the PPA oh wait :)
<wgrant> Night.
<rvba> Night lifeless.
<lifeless> tomorrow I shall wire up subprocesses sending to amqp
<lifeless> call sync() on the 7 uses of getLastOopsReport
<lifeless> and purge purge purge purge purge purge
<mrevell> Hey henninge, how's it going?
<henninge> Hi mrevell!
<henninge> mrevell: Going well, thanks!
<mrevell> Good to hear :)
<henninge> mrevell: I have been looking for a place to find out who was hired to replace me. Wasn't there a squad listing on the dev or help wiki?
<mrevell> henninge, That person hasn't joined yet.
<henninge> ah
<henninge> hard to replace me, I know ...
<henninge> :-P
<mrevell> henninge, As for the page... let me look
<mrevell> heh, true true :)
<mrevell> henninge, There's this: https://dev.launchpad.net/Squads
<henninge> oh, that simple
<henninge> stupid moin, searching for "squads" does not yield that page.
<henninge> ah, case-sensitive search
<henninge> mrevell: thanks! ;)
<mrevell> henninge, Heh. We really need to sort out the dev wiki. We have a new Usability and Communications Specialist joining soon who will help us with that.
<henninge> cool
* benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: benji | Critical bugs: 262
<deryck> Morning, everyone.
<abentley> benji: could you please review https://code.launchpad.net/~abentley/launchpad/mustache-bugs/+merge/79441 ?
<benji> abentley: gladly
<bigjools> anyone else notice that DatabaseLayer is massively slower to start up these days?
<deryck> adeuring, ping for standup.
<adeuring> deryck: ok, soory
<abentley> deryck: https://code.launchpad.net/~abentley/launchpad/mustache-bugs/+merge/79441
<deryck> abentley, got it, thanks.
<benji> abentley: the branch looks good; I did note one small thing I think you'll want to change
<abentley> benji: what's that?
<benji> abentley: I suspect you want to remove the "<em>Server-side mustache</em>" and "<em>Client-side mustache</em>" bits as well as making the client/server rendering conditional on the feature flag.
<abentley> benji:   The client-server rendering is already conditional on the feature flag.  I don't want to remove the multiple copies yet, because I need to be able to QA all the renderings.
<abentley> benji: See line 450 of the patch for where the rendering is made conditional.
<benji> abentley: right, but it renders both versions; does the "normal" version get rendered if the feature flag is off?
<abentley> benji: Yes, the normal version gets rendered regardless of the feature flag status.
<benji> if so, I guess the feature flag is (at least at this moment) more about being able to do in-production QA and not about being able to really turn the feature on and off
<abentley> benji: Right, the feature is only for the team developing it at the moment.
<benji> abentley: k; I might have made that conditional on a query string instead of a feature flag, but I've heard that some people aren't me ;)
<abentley> benji: This is part of an actual feature that is planned to take 4-6 weeks to complete, and the flag will prevent work on that feature from affecting normal users while we are working on it.
<sinzui> gary_poster, bigjools: can either of you allocate someone to fix bug 870130
<_mup_> Bug #870130: shortlist error requesting recipe build <easy> <oops> <recipe> <Launchpad itself:Triaged> < https://launchpad.net/bugs/870130 >
<jelmer> hi benji, can I add another branch to your queue?
<benji> jelmer: certainly
<bigjools> sinzui: I can
<gary_poster> bigjools, if someone on your side could grab it...thank you bigjools :-) .
<bigjools> :)
<gmb> Can someone who knows something about the lazr.restfulclient .deb confirm or invalidate this bug: https://bugs.launchpad.net/lazr.restfulclient/+bug/876445
<_mup_> Bug #876445: missing lazr.uri dependency in the debian/ubuntu package <lazr.restfulclient:New> < https://launchpad.net/bugs/876445 >
<gmb> (Or tell me how to)
<benji> gmb: I don't know anything about the .deb but if it doesn't include a dependency on lazr.uri, then it should
<gmb> benji: Agreed, but I don't know where to look to check. I could grab the source package I suppose...
<gmb> benji: And whaddaya know, it doesn't have that dependency.
<jelmer> benji: Thanks - the MP is @ https://code.launchpad.net/~jelmer/launchpad/stacked-code-imports-newer-bzr/+merge/79538
<rvba> benji: Hi, could you please review this js branch? https://code.launchpad.net/~rvb/launchpad/bug-869221-archindepwidget/+merge/79561
<benji> rvba: sure
<gmb> benji: See, I like this arrangement, other people get to do the thinking and I look like I've done some actual work.
<gmb> Thanks :)
<rvba> benji: Thank you.
<benji> gmb: I do too, I just have to type and other people do the work
<gmb> benji: It's symbiotic development at its best.
<benji> gmb: I'm reminded of this xkcd: http://xkcd.com/722/
<gmb> Heh :)
<sinzui> jcsackett, do you have time to mumble in the new purple channel
<jcsackett> sure.
 * jcsackett goes to find the new purple channel.
<gary_poster> sinzui, hi.  Thank you for the html5browser review and help.  On a slightly related subject, Francis wanted me to offer my services to your squad to help you with the new yuixhr tests, if that is of value.  They should be of particular interest to feature squads.  I'm happy to help however you think might be appropriate, including phone calls and/or more/better documentation.
<gary_poster> Once the html5browser changes are live, I can (try to re-) land some simplifications and improvements to the yuixhr usage.  After that would probably be the right time to really dig in.
<gary_poster> (That's all I had to say; just making the offer.)
<sinzui> gary_poster, otp
<gary_poster> cool
<nigelb> HAHA
<nigelb> sinzui: You guys did take the name I suggested!
<nigelb> (I suggested Teal Assasins) :P
<sinzui> nigelb, Then my thanks go to you and wallyworld_ who contributed the colour
<nigelb> hehe
<nigelb> I'm still laughing uncontrollably :P
<flacoste> sinzui: shouldn,t it be Aubergine Assassins ;-)
<sinzui> gary_poster, bigjools: We have a spam project in the review list given to ~registry. The registrant must be suspended too. There may be other spams projects I did not see in my commercial view.
<gary_poster> sinzui, bigjools, I'll take it
<nigelb> flacoste: heh
<bigjools> Dammit I wanted to rename to purple!
<gary_poster> I'll start my CHR run in a few, and include that in it
<sinzui> flacoste, I pondered that. I didn't want people thinking we we will fix every Canonical stakeholder issue
<sinzui> We can fix community bugs too
<nigelb> heh
<nigelb> flacoste: if they were named aubergine, you could sacrifice that squad to stakeholders :P
<abentley> bigjools: You can rename to aubergine.
<bigjools> I shall think of better
<nigelb> cyan?
<flacoste> Pink!
<sinzui> bigjools, Teal is available now
<bigjools> flacoste: oooo get you
<bigjools> sinzui: lol
<bigjools> Taupe Twits
<nigelb> bigjools: *cough* cricket :P
<nigelb> bigjools: You can still take "Men in blue"
<bigjools> nigelb: how many games did you win over here all summer, again?
<nigelb> haha
<bigjools> ;)
<mrevell> Night all
<lifeless> morning
<lifeless> sinzui: you might like to rename your lp team
<lifeless> from 'green'
<sinzui> lifeless, I renamed the teal team before the annoucement
<sinzui> is there a green team?
<lifeless> sinzui: there is a wiki link
<lifeless> I saw it in one of the pages you edited
<sinzui> I will hunt those down
<lifeless> flacoste: yo
<flacoste> hi lifeless
<flacoste> lifeless: do you want to talk now?
<lifeless> that would be cool :)
 * deryck goes offline for long late lunch, back later
<micahg> benji: got a minute to chat about bug 876594?  I think this has the potential for ill will and is a regression in this case
<_mup_> Bug #876594: rejected builds for synced packages send mail to Debian maintainer <derivation> <Launchpad itself:Triaged> < https://launchpad.net/bugs/876594 >
<benji> micahg: sure, I'm glad to discuss it.  (Thanks for letting me know it's a regression, I'll update the bug.)
<micahg> benji: I think it is, cjwatson could probably verify
<cjwatson> micahg: sounds like it, yes.  We should not be mailing Debian maintainers for activity in Ubuntu just because their name was on the package we synced
<cjwatson> (to see why not, imagine the full set of Debian derivatives)
<micahg> benji: yeah ^^, so what you did was fine, thanks
<benji> micahg: my pleasure
<micahg> cjwatson: thanks
* benji changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 26
<huwshimi> Morning all
* lifeless changed the topic of #launchpad-dev to: Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 26
* StevenK changed the topic of #launchpad-dev to: Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: - | Critical bugs: 273
<elmo> are you guys ever going to implement the critical/high split?  the number in /topic is depressing me and I'm not even on the LP team
<Beret> heh
<StevenK> elmo: I hope so. Everytime I look through the Critical list, I get depressed.
* sinzui changed the topic of #launchpad-dev to: Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: - | Critical bugtasks: 273
<lifeless> elmo: what critical/high split ?
<elmo> lifeless: I thought there was discussion on splitting the 273 criticals into critical/high, with critical being reserved for the genuinely holy-crap-wtf crticials
<lifeless> elmo: there is discussion, there isn't resolution
<lifeless> elmo: we're analysing *why* we are generating so many new criticals
<lifeless> elmo: thats a crucial step in addressing the issue, because we're fairly good at closing them.
<elmo> lifeless: *shrug* k - just seems crazy to me - you de facto can't have 273 criticals, some of those must be more critical than others
<elmo> i.e. the ones you're working on now and next
<elmo> rather - you de facto can't have 273 criticals which are actually critical by the definition most people understand 'critical' to mean
<lifeless> elmo: so we have a policy that says things that are broken are critical, plus things stakeholders have escalted are critical
<lifeless> -all- the criticals meet one of those definitions
<lifeless> yes its a problem, yes folk can't be working on 273 things in parallel, but the root issue here is that we are breaking things faster than we're fixing them
<lifeless> if we shuffle all our highs to medium/low criticals to high and then pick a subset of brokenness to treat as critical, the problem doesn't get any better
<elmo> I'm not saying that's not an issue
<elmo> but I think it's orthogonal to the issue I'm unsolicitedly whining about
<elmo> you could still figure out the breaking faster than fixing thing, if stuff got shuffled, e.g. by treating escalations as high
<lifeless> politically, we want to work on escalations before we've fixed all thats broken
<lifeless> (and escalation are ~5% of maintenance work atm, IIRC)
<elmo> ok, sure
<elmo> I guess what I'm saying is - all the other stuff aside, having non-now/next items in critical strikes me as insane, and doing it over such a long period of time has to be demotiviational
<elmo> but *shrug*, that's just MO
<elmo> I'll shut up and go back to my own criticals ;-)
<lifeless> so, what is non-now/next in the critical bucket ?
<elmo> I'm talking non-now/next in the kanban sense
<elmo> at least AIUI
<elmo> which means you can't possibly have a capacity that would allow you to fit all 273 into now + next
<elmo> which means you've got X on the go and X that you'll be doing next
<elmo> and that X+X << 273
<lifeless> right, kanban wise if each engineer has a WIP limit of 2, now+next = 10*2*2 -> 40 items
<elmo> where did 10 come from?
<lifeless> but that aspect of kanban w.r.t. software development is primarily about directed-effort, this is category..
<lifeless> squad size 5, maintenance squad count 2
<elmo> ah
<lifeless> so squads * members * WIP * (now + next=2)
<StevenK> Er, we have 4 squads?
<lifeless> 2 on maintenance
<lifeless> feature squads are not working from the critical queue
<StevenK> [09:53] < lifeless> squad size 5, maintenance squad count 2
<lifeless> StevenK: yes?
<StevenK> Oh, the number of people in the squad
<lifeless> yes
 * StevenK goes back to breaking the archive.
<jelmer> hah, there is my punishment for deprecating Branch.revision_history
<jelmer> lots of things in Launchpad seem to rely on it
<wallyworld_> thumper: that utf-8 branch mail patch is deployed. can you let me know if you see any issues come up
<lifeless> jelmer: also pqm
<thumper> wallyworld_: awesome, thanks
<lifeless> jelmer: would love a patch for that too
<wallyworld_> thumper: np. and thanks for not mentioning the rugby :-(
<thumper> wallyworld_: np
<lifeless> elmo: when we restart rabbit in the dc, how long does it take to come back up?
<lifeless> elmo: also, when we restart a server of the type its going to be on, how long would it be down for?
<elmo> lifeless: I start to get worried if a DL380 takes > 100s to come back
<lifeless> elmo: I'm writing some code that I'd like to be resilient to rabbit getting maintained, but to not assume it will always come good eventually
<lifeless> elmo: so I'd like to have a timeout on its retry-connections duration, after which it will bail out and exit
<lifeless> elmo: does this sound reasonable /
<lifeless> (its the suck OOPSes from AMQP and push them into the oops-tools database code)
<elmo> lifeless: restarting rabbit on a random staging box took ~4s
<lifeless> ok, so ignoring truely exception things like box rebuilds, where we'd reconfigure to a different rabbit anyway, 2 minutes as a timeout seems sufficient ?
<lifeless> elmo: and does this approach sound ok from your ops perspective, or would you rather just fail-hard, or never-fail policies ?
<elmo> lifeless: is this oops, or in general?
<lifeless> this specific code is in oops_amqp so yes, oops specific
<lifeless> for the LP appservers we have clear transaction boundaries we can retry on, and disable rabbit if its not up at the start of the transaction
<lifeless> I guess the txlongpoll service has/needs something similar to this as well - i haven't audit its code.
<lifeless> wgrant: what does the txlongpoll twistd daemon do if rabbit is away / goes away ?
<elmo> lifeless: does this mean that an app server could stall for up to 2m trying to talk to rabbit?
<lifeless> elmo: no
<lifeless> elmo: for oopses, the sender fails fast
<lifeless> elmo: this is for the slave, the oops-tools side of it
<elmo> hmm
<wgrant> lifeless: Not sure.
<lifeless> elmo: which has nothing tickling it to say 'try again now' because its only interface is rabbit pushing stuff to it
<elmo> lifeless: so the thing trying to consume off the queue?
<lifeless> yes
<elmo> ok
<elmo> how would a 'never-fail' policy work?
<elmo> infinite retry?
<lifeless> yes
<lifeless> which makes me nervous :)
<elmo> ok, I think I prefer the 2m-timeout or fail-hard
<elmo> I don't mind which
<elmo> I guess 2m-timeout is a little more forgiving
<lifeless> just means a little more self healing
<lifeless> one less zomg chase after a reboot of rabbit
<elmo> self-healing would be nice
<elmo> yeah
<elmo> because apparently I just broke U1 staging
<lifeless> \o/
<elmo> with my test restart of rabbit
<lifeless> its bug filing time :)
<elmo> so, if we could avoid that same brain damage in LP, I'd appreciate it
<lifeless> yup
<lifeless> we filed bugs about this for the new stuff last night, when I read the current code
<lifeless> as I say, I haven't checked txlongpoll yet
<lifeless> its probably ok, if its written naively, it will only connect when a js session connects, and not share channels/connections, which scales poorly but would make it robust around this
#launchpad-dev 2011-10-18
<lifeless> whoever is trying to and
<lifeless> *land*
<lifeless> PQMException: 'Failed to verify signature: gpgv exited with error code 2'
<lifeless> please be signing, or update your key with losa
<StevenK> Or it could be OOPS tools branches have PQM subscribed again
 * lifeless checks
<lifeless> nope
<lifeless> https://code.launchpad.net/~launchpad-pqm/python-oops-tools/trunk
<lifeless> besides, this isn't the super-fast-busy-loop that normally creates :)
<StevenK> Yes, wgrant had that one fixed last night
<lifeless> what was subscribed /
<wgrant> launchpad-pqm
<wgrant> The branch owner.
<lifeless> bahhh
<lifeless> ugh wikipedia
<lifeless> why do I click on links.
* wgrant changed the topic of #launchpad-dev to: Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: wgrant | Critical bugtasks: 263
<wgrant> StevenK: https://code.launchpad.net/~wgrant/launchpad/bug-876171/+merge/79639
* leguin.freenode.net changed the topic of #launchpad-dev to: Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: - | Critical bugtasks: 273
<StevenK> wgrant: IDistroSeries._getAll{Sources,Binaries} do not take a status argument
<StevenK> Oh, sorry, misreading
<wgrant> StevenK: I would hope not.
<wgrant> Right.
<StevenK> There's a .find() hiding in those calls
<wgrant> I am working around our lack of a PublicationCollection,.
<StevenK> wgrant: Ignoring my misunderstanding -- r=me
<wgrant> wallyworld_: Around?
<wallyworld_> wgrant: yep
<wgrant> wallyworld_: Bug #876533
<_mup_> Bug #876533: changing the values in Comboboxes on my code page does not update the branch-list shown <regression> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/876533 >
 * wallyworld_ still waiting for lp
<wallyworld_> wgrant: ah, ok. it works with the ff on
<wallyworld_> but not off
<wallyworld_> i'll fix it
<wgrant> Thanks.
<wallyworld_> wgrant: wanna review it, and i'll get it in the pipeline https://code.launchpad.net/~wallyworld/launchpad/broken-branch-listing-876533/+merge/79640
<wgrant> Oh god, Julian's domination change has landed :(
<wgrant> It will be months before we can deploy!
<wallyworld_> what's the domination change?
<StevenK> Utter madness
<wgrant> wallyworld_: Shouldn't that be unconditonal?
<wallyworld_> wgrant: nope
<StevenK> Don't dominate arch-indep binaries if any sources build them
<wgrant> Or does the post_refresh_hook fire on load too?
<wallyworld_> if the ff is there, the setup is done in the js
<wallyworld_> that's why i didn't notice it
<wallyworld_> because it works fine for us
<wgrant>                     post_refresh_hook: hookUpFilterSubmission
<wallyworld_> yep
<wgrant> That's the only call I can see.
<wgrant> So either post_refresh_hook is a lie, or it's buggy.
<wallyworld_> wgrant: nope, that bit is behind the feature flag
<wallyworld_> so we need to provide a call when the ff is off
<wgrant> I realise.
<wallyworld_> that's the basis of the bug
<wgrant> But does post_refresh_hook fire on page load?
<wallyworld_> no, only if the ff is on
<wgrant> It looks like it should be broken when the ff is on as well.
<wgrant> That's what I mean.
<wallyworld_> otherwise that bit of javascript is not executed
<wgrant> Does post_refresh_hook fire on page load?
<wgrant> Assuming it's configured.
<wallyworld_> yes
<wgrant> Ahhhh
<wgrant> How confusing.
<wallyworld_> and it works fine then
<wgrant> OK.
<wallyworld_> it will be better when the ff is taken away and it's the same for everyone
<wgrant> Right, it's just very confusing that post_refresh_hook is executed on load too.
<wgrant> Approved.
<wallyworld_> thanks.  post_refresh_hook is not executed though unless ff is on
<lifeless> anyone up for an oops_amqp revu ?
<wgrant> wallyworld_: Clearly.
<wgrant> But it's confusing that it doesn't just apply to refreshes.
<wallyworld_> perhaps. the js module calls the hookup when it is done loading and also on refresh
<wallyworld_> ie whenever the html form changes
<lifeless> https://code.launchpad.net/~lifeless/python-oops-amqp/0.0.2/+merge/79637
<lifeless> ^ hint, hint
<wallyworld_> lifeless: i'll have a look but am  working get a web service test to run properly. i'll need a bit of time to ramp up on the ampq stuff. can i do it a bit later?
<lifeless> wallyworld_: well, I want low turnaround so that I can use it in the next project along
<wallyworld_> ah, right
<wallyworld_> i'll look now
<lifeless> wallyworld_: tuesday wgrant is on OCR
<lifeless> wgrant: ^ ;)
<wgrant> Sorry, just developing a test case to revert Julian's revision.
<lifeless> wallyworld_: you're working on a critical regression, I think you should focus on that :)
<wallyworld_> lifeless: already fixed
<wallyworld_> and in ec2
<lifeless> ah cool
<lifeless> in which case, thank you!
<wallyworld_> just a few lines of tales
<StevenK> wgrant: You want to revert it?
<wgrant> StevenK: It's far easier to QA if it's reverted. Plus I think it's buggy.
<lifeless> wgrant: julian didn't dogfoodinate it before landing ?
<wgrant> He did.
<lifeless> unity needs a cat-safe mode
<lifeless> 37 screenshots later..
<mwhudson> :)
<lifeless> all with the same second timestamp
<lifeless> argh, this use-oops-twisted branch is becoming a branch of doom
<lifeless> fix one test failure, 30 more pop out
<wgrant> StevenK: http://paste.ubuntu.com/711550/
<wgrant> StevenK: That test now fails.
<wgrant> Passes at devel-3
<wgrant> Think that's revert-worthy?
<StevenK> wgrant: The source is not superseded?
<StevenK> IE, 1.0's source is not
<wgrant> Ah, sorry. The binary isn't.
<wgrant> Arch-indep can no longer supersede arch-specific, and vice-versa.
<StevenK> Hmmmm
<wgrant> I think that's pretty clearly undesirable.
<StevenK> I think you should add that test and then revert Julian's branch
<StevenK> Or vice-versa
<wgrant> Other way around, but yes.
<wgrant> Thanks.
<nigelb> StevenK: \o/
<nigelb> Purple Assasins!
<nigelb> I lol'd
<lifeless> wallyworld_: so for clarity, you're doing that review, or wgrant is ?
<wallyworld_> lifeless: sorry, i assumed wgrant was
<wgrant> I'll be done with the revert+test in a few minutes.
<wgrant> Can do it then.;
<lifeless> wallyworld_: I think we tied ourselves up in knots :)
<lifeless> wallyworld_: wgrant: ok, wgrant it is
<wallyworld_> ouch
<lifeless> bwah I missed one more pending change. ><
<lifeless> right, pushed. *now* we're cooking. Vit gahs
 * wgrant glares disapprovingly at lifeless' implicit relative imports.
<wgrant> lifeless: Is r10 relevant to anything at all?
<lifeless> wgrant: very
<wgrant> Oh?
<lifeless> wgrant: fallback when amqp is down is a little cronscript that reads from the datedir repo and sends to amqp
<wgrant> It seems entirely unrelated to the rest of the branch.
<lifeless> the branches theme is 'stuff I learnt while hacking elsewhere'
<wgrant> That's slightly unpleasant.
<wgrant> But OK.
<wgrant> __init__ no longer sets self.stopping. Deliberate?
<lifeless> yeah
<lifeless> calling run_forever twice should work
<wgrant> Also, are you sure I can't send a message with a body of None?
<wgrant> You probably can't, but you never know.
<lifeless> moderately
<lifeless> can make it a random nonce / guard it if you like
<lifeless> I'm not sure if its needed or not
<lifeless> I've seen no reference to optional message bodies in amqp
<wgrant> lifeless: Won't the retry logic not work if it goes away more than two minutes after the connection was established?
<lifeless> grah, you are right
<lifeless> I will fix
<wgrant> Thanks.
<wgrant> lifeless: Reviewed, with a few more comments.
 * wgrant unsubscribes canonical-launchpad-branches from review mail on python-oops-amqp
<wgrant> Since PQM is unhappy.
<lifeless> wgrant: you know that refactoring that (mild) duplicate keeps the line count in each test as long due to PEP8 line wrapping rules...
<wgrant> lifeless: But it's going to be much clearer.
<StevenK> wgrant: The revert and test have landed?
<wgrant> StevenK: yes.
<StevenK> Excellent.
<lifeless> wgrant: actually, its a rabbit hole into that testtools 'function' object has no attribute getDetails
<lifeless> wgrant: so I want to defer it; I've added a fixture but using it -> death spiral
<wgrant> lifeless: at least turn it into a helper method.
<wgrant> On the test class.
<lifeless> wgrant: so I think that that hurts clarity
<lifeless> most of those tests have two distinct actors
<lifeless> the channel and the queue
<wgrant> All the present boilerplate certainly doesn't aid clarity.
<lifeless> right
<lifeless> so I wrote a fixture to get a channel
<lifeless> and something is wrong and its going to be a yak shaving session to figure it out
<lifeless> which needs doing(but not now)
<lifeless> hmm, I have a possible handle, poking
<lifeless> wgrant: +70 lines -48 lines
<lifeless> wgrant: (Which is why this was below my tolerance threshold)
<wgrant> How!?
<lifeless> pastebin.com/bTKHC5wN
<wgrant> You could collapse the channel = channel_fixture.channel into the previous statement.
<lifeless> +22 from the helper, docs, _all_ entry
<wgrant> And I was imagining the QueueFixture would go into the same helper method.
<wgrant> Since it's also shared between every test.
<wgrant> Possibly even the publisher as well.
<lifeless> way too much hidden in that
<lifeless> however collapsing the assignment is good
<wgrant> Hm? Why can't the test just start with a queue name and publisher as instance variables?
<lifeless> I've done that plenty before
<lifeless> so I'm going to play the ETIRED card for sympathy :P
<nigelb> Sympathy card, well played.
<wgrant> So, all the tests need a channel and a queue, and there's nothing test-specific about that setup.
<wgrant> Pull it out into setUp(). Problem solved.
<wallyworld_> why do i get a 'AttributeError: 'thread._local' object has no attribute 'interaction' error when i try and use 'with person_logged_in(xxx)'? i've tried a few things but can't seem to fix it
<lifeless> wgrant: Expensive things being in setUp is a mispattern
<wallyworld_> surely it matter not that there's no interaction before i try and log in
<lifeless> wgrant: they are out-of-site, out-of-mind
<mwhudson> wallyworld_: full traceback?
<lifeless> wgrant: I much prefer seeing whats going on
<wallyworld_> Traceback (most recent call last):
<wallyworld_>   File "/home/ian/projects/lp-branches/devel-sandbox/lib/lp/bugs/model/tests/test_bugtask.py", line 2390, in test_delete_bugtask
<wallyworld_>     with person_logged_in(db_bug.owner):
<wallyworld_> AttributeError: 'thread._local' object has no attribute 'interaction'
<wgrant> lifeless: Put them in a fixtures class attribute, or something, then?
<wgrant> This is currently really cluttering boilerplate.
<lifeless> +48-36 now, which is much better
<wgrant> I want it killed.
<wgrant> Somehow.
<lifeless> wgrant: its not
<mwhudson> wallyworld_: bet it's the security check on bug.owner :/
<lifeless> wgrant: yes its the same, but they are the essential actors in these tests
<lifeless> wgrant: which is different to boilerplate that isn't an essential actor, and which I have great joy encapsulating and hiding
<wgrant> It's like 12 consecutive lines that's shared between every test.
<wgrant> They are essential actors for this *class of test*.
<lifeless> 3, 5 if you could the publisher (which has different params sometimes)
<wallyworld_> mwhudson: you mean i have to remove the proxy?
<wallyworld_> mwhudson: i've used with person_logged_in(bugtask.owner) ok
<mwhudson> wallyworld_: i don]'
<mwhudson> wallyworld_: i don't know, it's just a guess :-)
 * wallyworld_ tries it
<lifeless> wgrant: I've pushed up rev 12
<wallyworld_> mwhudson: that got me one step closer thank you. it would have been nice if the error said what was wrong though
<wallyworld_> as in authorised attribute access or whatever
<mwhudson> wallyworld_: note that if you inherit from TestCaseWithFactory, you are logged in anonymously in setUp
<StevenK> Like Zope is going to be nice.
<mwhudson> handling security proxied objects without an interaction is like some metaphor involving explosives
<wallyworld_> mwhudson: yes, but before the bit that was failing i was doing a web service call and other stuff so the initial login was gone i think
<mwhudson> ah
<mwhudson> that's a bit naughty
<mwhudson> you could try restoreInteraction()
<mwhudson> that always seemed a bit magical
<StevenK> Ah, webservice tests are special
<wallyworld_> ok. the test case extends WebServiceTestCase. i have followed what others have done i think
<lifeless> s/c/th/
<StevenK> You can't mix factory calls with webservice calls
<mwhudson> wallyworld_: without wanting to be rude
<wallyworld_> you need to make actory calls to set up the objects
<mwhudson> wallyworld_: the "others" might not have understood what was going on :-p
<lifeless> so, webservice calls wipe the interaction
<StevenK> Create or grab *everything* from the factory/zope, logout() and then deal with only the webservice
<lifeless> I think someone has a patch up to fix that
<wallyworld_> mwhudson: clearly i don't quite either :-)
<lifeless> StevenK: or login again
<wallyworld_> and then if i want to look at the state of the db after making the ws calls, i need to logout and login again?
<StevenK> I prefer my way. Less chance of blowing off both legs
<mwhudson> wallyworld_: yeah, i have an email i need to write about that, come to think of it
<StevenK> wallyworld_: Yes
<lifeless> wgrant: what do you think?
<wgrant> lifeless: Oh, the diff is updated?
<wallyworld_> mwhudson: StevenK: thanks for the help.
<lifeless> wgrant: yes, or loggerhead :)
<wgrant> 401	+ queue = self.useFixture(
<wgrant> 402	+ QueueFixture(channel, self.getUniqueString))
<wgrant> Does that really not fit on one line?
<lifeless> wgrant: bah, I missed one - all the others were shuffled by channel_fixture
<lifeless> s/1/3/
<lifeless> no, really just the one
<mwhudson> wallyworld_: where is the code that makes a webservice request?
<wallyworld_> mwhudson: it's local. i'll push up something in a sec
<wgrant> lifeless: You could also s/connection_factory/conn_factory/ in the test and get the ChannelFixture stuff down to a single line.
<wallyworld_> as it turns out, calloing logout before webservice_for_person doesn't work
<StevenK> What traceback do you get?
<lifeless> wgrant: when we feel the urge to do that, its a hint that PEP8 is wrong :)
<wallyworld_> 'no local interaction', no real traceback
<wgrant> PEP 8 is not relevant here. Every coding standard ever is.
<wgrant> It's not an indentation problem, it's a line length problem.
<lifeless> precisely
<lifeless> anyhow, yes I could do that
<lifeless> however, the focus right now is on this patch
<wgrant> It looks more reasonable now.
<StevenK> wallyworld_: If you're doing webservice_for_person(bugtask.owner) that wants you logged in
<StevenK> You need to grab out the person beforehand
<StevenK> owner = bugtask.owner ; logout(); webservice_for_person(owner)
<wgrant> lifeless: Approved.
<StevenK> That's what I meant by *everything*
<lifeless> wgrant: thanks
 * StevenK heads out for about 0
<StevenK> Sigh
<StevenK> s/\(0\)/3\1/
 * wgrant is tempted to make LP mailing lists strip text/html parts.
<lifeless> s/.*/bye/ 'Fixed'
<wallyworld_> mwhudson: so this gets passed the ws stuff but the delete doesn't 'stick' so the last assert fails https://pastebin.canonical.com/54477/
<mwhudson> wallyworld_: i hope those commit() calls aren't really required
<wallyworld_> mwhudson: the first one is
<mwhudson> wallyworld_: i dunno what's going on though, maybe turn on statement logging in postgres?
<wallyworld_> or else the ws cannt see the newly created bug
<mwhudson> uh really?
<mwhudson> delightful
<wallyworld_> mwhudson: yeah, doing the sql logging now. nb the normal unit tests for this all work
<wallyworld_> i've tried removing the first commit and it blows up with a 'not found' error or similar
<mwhudson> oh right, webservicecaller makes an actual http request
<mwhudson> well, not quite
<mwhudson> wallyworld_: i'm in well over my head now
<wallyworld_> mwhudson: np. thanks for your input
 * wallyworld_ goes to get flat tyre fixed
<StevenK> The one on your car, or your belly?
<wallyworld_> ha
<wallyworld_> smart-ass
<wallyworld_> the new one on my car that went flat overnight :-(
<nigelb> wallyworld_: clearly you need to get it exchanged ;)
<stub> lifeless: So a quick scan of your oops/ampq work worries me a bit due to the amount of crap you need to use Rabbit sanely. I'll have a better scan later, but maybe ZeroMQ has less of an implementation burden than I thought (service registry etc.)
<lifeless> can you expand on what you mean ?
<stub> lifeless: Reconnection logic, EPIPE handling, local queuing - that sort of stuff. That seems flaky with the Rabbit libs making us do the work. In ZeroMQ we get that for free but need to implement some sort of exchange (such as a service registry to allow peer-peer, or a cookbook broker).
<lifeless> stub: could you do an 'approve' vote on https://code.launchpad.net/~lifeless/python-oops-tools/amqp/+merge/79644 - I think tarmac is not setup for self-review yet
<stub> done
<lifeless> fingers crossed
<lifeless> stub: so, we're very close to done with a working rabbit; I think you have a pretty solid point though
<wgrant> How is 0mq better on those fronts?
<wgrant> I don't see how it can do local queueing.
<lifeless> wgrant: you give it a disk path and it spools to disk if the endpoints are unavailable
<lifeless> wgrant: up to a threshold you set, at which point it either errors or discards (again, you set the policy)
<stub> So messages are retransmitted when it gets back up. But the local storage is not durable, so you lose it if the process terminates.
<lifeless> wgrant: only applies to pubsub obviously
<stub> But it is free
<wgrant> Right, we need durability across process restarts.
<stub> There are a lot of 'we needs'. What I'm wondering is if the stuff we need to implement for rabbit is going to be more work or less reliable than the stuff we need to implement for an alternative like ZeroMQ.
<stub> I've got rose coloured glasses on for ZeroMQ though, as I've not used it in anger and haven't gone over the ampq-oops work seriously to see how much of it we would have needed to do with ZeroMQ too.
<lifeless> stub: disk swaps don't reset on proces death do they ?
<lifeless> ah, it does
<lifeless> 'The SWAP file is not recoverable, so if a publisher dies and restarts, it will lose data that was in its transmit buffers, and that was in the network I/O buffers.'
<lifeless> stub: so, catching EPIPE is pretty shallow; it would be nice not to. We'd still need (for our use case) offline persistent queuing
<stub> I think it has to do with the runtime state (next time you restart, your connections might be completely different and incompatible), and maybe security (if your spool is durable, you need to worry about who can read it rather than keep it in a private temporary file)
<lifeless> makes sense to me
<lifeless> stub: OTOH I think its well encapsulated in oops-amqp (I'm about to cleanup the stuff in my useoops lp branch to use the 0.0.2 improvements)
<lifeless> stub: also once we are on -a- mq, I think its quite easy to transition, message-pair at a time
<stub> I suspect we will want to move some of this encapsulation to lazr.amqp or something for reuse. lazr.ipc? lazr.messaging?
<stub> Yer. I doubt I could argue a switch to ZeroMQ at this point. The Rabbit APIs bug me though. Weird terminology and promote ugly code.
<lifeless> yeah. There is some stuff in lp.servicces.messsaging, but its a little to keen to be zope integrated to be reusable (plus some of it doesn't carry its own weight)
<lifeless> I so want the compose-key-combo for theta
<lifeless> erm, not theta
<lifeless> the greek letter whose name escapes me that they use
<wgrant> Ã¸?
<lifeless> yes
<wgrant> Ã
<wgrant> Alt+/ O
<lifeless> no
<wgrant> s/Alt/Compose/
<lifeless> Ã¸ woo
<lifeless> yeah, its my left windows key
<wgrant> I use RAlt, since LSuper is for Unity.
<stub> I refuse to perpetuate that annoyance and promote ZeroMQ :-)
<lifeless> Ã¸Ã¸Ã¸
<wgrant> Ã¸MQ does look reasonably nice, but I'm a bit concerned that they promote it with a video from a PHP conference.
<wgrant> Oh.
<wgrant> #is has a better one.
<wgrant> â
<wgrant> Although it's not monospaced :/
<stub> It is a tool though for writing sane PHP since your PHP can easily become a thin layer talking to a middle layer written in something else :-)
<lifeless> wgrant: compose for that ?
<wgrant> lifeless: No clue.
<stub> ÃMQ is fine
<wgrant> I wonder if xkeyboard-config knows.
<wgrant> âMQ fails in a monospace font.
<wgrant> Need a smallish space..
<stub> ÃMQ looks better than âMQ in my non-monospace font too.
<wgrant> We really need to get rid of embedded JS.
<huwshimi> wgrant: You mean all the inline JS? We certainly do need to do that
<wgrant> Yes.
<huwshimi> wgrant: We really need a global dom ready and a place for all the js to exists
<wgrant> It's hideous and buggy and blah.
<wgrant> And TAL makes it even worse.
<huwshimi> sure is
<huwshimi> wgrant: Fixing that up should have a small speed improvement too
<wgrant> Indeed.
<poolie> lifeless previously said that .suspend on a bmp diff job would cause it to retry later
<poolie> but is that true
<poolie> it looks a bit more like 'suspend' means paused indefinitely
<adeuring> good morning
<jelmer> hi Abel
<wgrant> poolie: The solution should be clear.
<wgrant> Just schedule a BranchMergeProposalUpdatePreviewDiffUnsuspensionJob!
<wgrant> Sorry, BranchMergeProposalUpdatePreviewDiffJobUnsuspensionJob
<poolie> :)
<poolie> i think i can add it to the list of retriable exceptions
<mrevell> Hallo
<poolie> hi there
<poolie> mrevell: https://dev.launchpad.net/LEP/SSH_OAuth
<wallyworld_> so why does the web service interface hate me so? i have a LaunchpadWebServiceCaller instance, i have logged in, but when i try invoking a delete method exposed via  @export_destructor_operation,  i get a 405  and only GET PUT PATCH POST allowed
<wallyworld_> and it swallows the error so i have to hack in debugging to see the response
<lifeless> its not personal
<lifeless> it hates everyone equally
<wallyworld_> :-) so how so i get it to allow me to invoke a http delete?
<wallyworld_> i see for example there's a story test for deleting branches using a similar bit of code but there's no exposed branch delete method that i can see, so i'm not sure how that works
<rvba> adeuring: Hi, I'd like to ask you another question about lazr.batchnavigator if you have 5 minutes.
<adeuring> rvba: sure
<rvba> This time it's a *real* question.
<rvba> I'm working on bug 872086.
<_mup_> Bug #872086: Distribution:+ppas and Distribution:+questions issue unlimited queries when memo=0&direction=backwards <regression> <timeout> <Launchpad itself:Triaged> <Storm:In Progress by rvb> < https://launchpad.net/bugs/872086 >
<adeuring> rvba: well, your last one wass quite real  too ;)
<rvba> I've done the Storm (trivial) fix but I wonder if I should try to also add a small fix to the batchnavigator like I first envisioned.
 * adeuring is reading the bug report...
<rvba> My idea was to try to improve how the batchnavigator deals with empty slices (like res[x:x]) to avoid issuing a query.
<rvba> But, reading the code in src/lazr/batchnavigator/z3batching/batch.py I see that the way it works, slicing with [start:end+1] to get cleverly get the information about whether or not there are other elements, will make this improvement rather "clumsy" ...
<rvba> adeuring: The storm fix will fix the problem so my question to you is: do you think I should try to improve the navigator or not? (after looking at the code I confess I'm not sure it's worth it).
<adeuring> rvba: in general, it is of course a good idea to "short-cut" queries where you know that the result will be empty, like sequence[n:n]. The problem with StormRangeFactory is that it does not have any knowledge about the possible size of the result set: The SQL OFFSET clause is gone, so you don't have anything that would map to [N:N]. Or am I missing something?
<rvba> adeuring: right, that's why IÂ was thinking that the fix could be in src/lazr/batchnavigator/z3batching/batch.py.
<rvba> adeuring: But I'm just reading this code for the first time so I might be wrong.
<adeuring> rvba: this code is a bit messy: It mixes the old batching parameters, like start, with the new ones -- no realy clear separation
<adeuring> rvba: The class is supposed to rely _only_ on memo values and on a requested batch size
<lifeless> adeuring: rvba is talking about the List range factory
<lifeless> adeuring: I think.
<adeuring> lifeless: ah, right!
<adeuring> rvba: ok, in that case, tweaking ListRangeFactory is of course reasonable
 * adeuring needs more coffee
<rvba> Right, ListRangeFactory, I confess this code messes with my brain a little ;)
<lifeless> thats entirely my fault
<lifeless> I took some confused code and made it more capable
<lifeless> -> and more confused
<rvba> hehe
<rvba> Okay, I'll see what I can do with ListRangeFactory, thanks lifeless, adeuring.
* gmb changed the topic of #launchpad-dev to: Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: gmb | Critical bugtasks: 273
<jtv> gmb: hi there.  Could I grovel you for a review?  https://code.launchpad.net/~jtv/launchpad/bug-873421/+merge/79662
<gmb> jtv: If you missed a preposition in that sentence, sure. If not, I don't know what being grovelled by you actually entails...
<jtv> Just verbing my way around the language.
<jtv> It involves me grovelling.
<nigelb> I think the grovelling comes if you refuse to review :P
<gmb> jtv: That's fine, then. I didn't know if it was the Launchpad version of Thailand's annual scheduled coup.
<nigelb> lol
<gmb> jtv: Approved with one comment, discardable as appropriate.
<gmb> s/discard/disregard/g
<jtv> gmb: thanks.  Perhaps you were thinking of âshovelling someoneâ rather than âgrovelling someoneâ earlier.
<gmb> Heh.
 * gmb -> adding tea to brain; bbiab
<jtv> Making an interface attribute doNotSnapshot does not affect our definition of API compatibility, right?  ISTR some programs could theoretically break because of it, but we considered such programs buggy.
<gmb> jtv: Right, it shouldn't make a difference to API compatibility.
<jtv> Thanks.
<deryck> Morning, all.
<abentley> deryck: morning.
<lifeless> bigjools: allenap: what would cause txlongpoll to timeout on 'make rune'
<lifeless> s/rune/run/
<allenap> lifeless: Blimey, let me see...
<lifeless> 'Exception: Timeout waiting for txlongpoll to start.'
<lifeless> plus a trace back - > total output
<lifeless> presumably the fixture has a log in getDetails but we're not printing that
<bigjools> mmm it tries to connect on the frontend port
<bigjools> and won't continue until it connects
<bigjools> or times out
<lifeless> ok; is that something I would need to fiddle, add to hosts, ... ?
<bigjools> so I expect txlongpoll is failing to start up
<bigjools> can you run it standalone?
<lifeless> I don't know. how do I check?
<bigjools> run the binary!
<bigjools> well, "executable"
<lifeless> help me out here, its 0230 :)
<bigjools> bin/twistd-for-txlongpoll
<lifeless> [only awake due to unhappy baby]
<allenap> lifeless: I suggest adding a try...except: print $fixture_details in TxLongPollServer.setUp() (in the LP code base).
<lifeless> that returns a twistd options page
<bigjools> bin/twistd-for-txlongpoll  amqp-longpoll -f 9999 -u guest -a guest
<lifeless> ah! thanks
<bigjools> unless the port that the fixture is using is conflicting with something...
<lifeless> assert fallback is None or callable(fallback)
<lifeless> I'm guessing I dont' have your fixed txlongpoll
<lifeless> in this branch
<lifeless> yah, trunk has 0.2.8->0.2.9
<lifeless> thanks!
<lifeless> allenap: bigjools: does 'make run' run its own rabbit up ?
* flacoste changed the topic of #launchpad-dev to: Launchpad should be engineered like an AK-47 | Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: gmb | Critical bugtasks: 273
<allenap> lifeless: Yes, should do.
<lifeless> allenap: yes, I see
<jcsackett> wallyworld: you still actually around?
<wallyworld> jcsackett: sadly yes
<wallyworld> but off to bed soon
 * rvba takes a look at the world clock.
<jcsackett> rvba: he's up very late. :-P
<rvba> Very very late indeed.
<lifeless> pish
<jcsackett> lifeless: your habits can't be used to judge anyone else, for various reasons. :-P
<jcsackett> although you're up late by most standards too. :-)
<nigelb> Actually, with lifeless its hard to judge if he's up late or awake eearly.
<jcsackett> hm. that's *very* true.
<lifeless> \o/ \o/ \o/ \o/ \o/ \o/
<lifeless> instant-oops-delivery end to end manual test is go.
<lifeless> code.launchpad.dev/++oops++ -> 'received OOPS-...' on the amqp2disk console (run with -v); paste that into oops-tools web UI and tada immediately visible.
<jcsackett> nice!
<lifeless> gnight
<rvba> nn
<flacoste> rvba, allenap: i'm trying to find the cause of bug 830676, can you help me navigate the involved templates?
<_mup_> Bug #830676: Regression in PPA AJAX build status indicators <critical-analysis> <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/830676 >
<allenap> flacoste: Do you mind if I leave you in rvba's hands? I'm meant to be on leave :)
<rvba> flacoste: sure.
<flacoste> allenap: please be on leave!
<rvba> Let me have a look.
<flacoste> rvba: i'm utterly unfamiliar with how this is all hooked up together
<flacoste> my guess is that the change is done by doUpdate in archive-packages.pt
<flacoste> but i don't see where doUpdate is called from
<flacoste> or even if this is the right place to look
<rvba> looking
<flacoste> hmm, although i think this is for the expander part, not the portlet that the bug report talks about
 * rvba uploads a test package to a ppa to see the ajax request.
<flacoste> rvba: ok, i think this is related to lp.soyuz.dynamic_dom_updater
<rvba> flacoste: sounds like the soyuz way indeed.
<flacoste> rvba: do you know where the domupdater is hooked to a specific row in the build status column?
<flacoste> ah, update_build_status.js
<rvba> flacoste: The key is to provide the domUpdateFunction.
<flacoste> rvba: right, i'm looking at how this is all hooked up, like everytime you disturb someone to ask questions, you start making progress yourself!
<rvba> flacoste: so that's good, let's continue ;)
<flacoste> rvba: thanks for helping, i'll continue to investigate and bug you again if i get stuck :-)
<flacoste> rvba: i need to hop on a call
<rvba> k
<abentley> gmb: flacoste thinks my work on reordering bugs via ajax may overlap with your work on loading comments incrementally via ajax.  Can we chat?
<gmb> abentley: Certainly. I'll be free to talk in ~30mins if that's okay.
<abentley> gmb: works for me.
<gmb> abentley: Cool. I'll ping when I'm free.
<gmb> abentley: I'm free whenever you are. What works for you, mumble or skype?
<abentley> gmb: let's mumble in Orange 101
<gmb> abentley: Okidoke. 1 sec...
<elmo> a lot of the launchpad buildds are going down  for power work
<jtv> gmb: still ocr'ing?  If so, https://code.launchpad.net/~jtv/launchpad/bug-870130/+merge/79691
<gmb> jtv: Certainly; bear with me a little while and I'll take a look.
<jtv> I often feel bad about hitting the same reviewer twice in one day, though nowhere near as much as for reviewing the same developer twice in one day.
<jtv> It's a good thing I enjoy watching them suffer, or I'd be depressed.
<jtv> Thanks gmb
<abentley> deryck: We have form fields with ids such as "field.searchtext".  I'm having trouble retrieving them with YUI.one.  I think they may be invalid identifiers: http://www.w3.org/TR/CSS2/syndata.html
<deryck> abentley, they are invalid.
<deryck> abentley, there's some selector work around, but it escapes me now.  let me search.
<abentley> deryck: okay.
<abentley> deryck: [id=field.searchtext] seems to work.
<deryck> abentley, ah right.
<deryck> abentley, I was thinking of this construction:  Y.one(Y.DOM.byId('field.searchtext'))
<abentley> deryck: Y.one accepts a DOM node?  Weird.
<abentley> deryck: also, we seem to have a bunch of ids that are not unique, e.g. field.status.  Should I just give up and jam this stuff into the IJSONCache instead?
<deryck> abentley, yeah.  turns it into the Y.Node form.
<deryck> hmmm, thinking about that....
<deryck> abentley, I think I would stick it in the cache, yes.  Just to be careful and specific.
<abentley> deryck: Okay, I'll do that.
* gmb changed the topic of #launchpad-dev to: Launchpad should be engineered like an AK-47 | Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: - | Critical bugtasks: 273
<gmb> jtv: Approved.
<jtv> thanks gmb
<gmb> np
<gmb> And with that...
 * gmb -> afk for the evening
<jtv> Then next continent over is giving up?  Classic cue to stop working.
<Ursinha> lol
<jtv> hey Ursinha!
<Ursinha> hey jtv!
<Ursinha> :)
<nigelb> jtv: heh.
<flacoste> abentley, deryck: we should fix the the non-unique ids at some point
<abentley> flacoste: and make them valid ids, too.
<jtv> Arrrgh buildbot failure.  Something about bzr and twisted, I think.
<dobey> haha. launchpad /is/ engineered like an AK-47 no? jams all the time, and lubricated with vodka?
<jtv> dobey: no, that's just barry.
<jtv> At least, I hear he jams a lot.  Not 100% sure about the vodka.
<jtv> jelmer: do you have any idea what might have earned us the ongoing buildbot failure?  https://lpbuildbot.canonical.com/builders/lucid_lp/builds/1476/steps/shell_6/logs/summary
<jtv> Most of the failing tests are related to something I touched the other day, but not the first one.  So I'm hoping that first one is closer to the root cause. :)
<jelmer> jtv: hmm
<jelmer> jtv: that's an odd one. I'm not aware of any other changes in this area. How long has this been failing?
<abentley> deryck: Having trouble finding the spot where the search params are defined.  Any suggestions?
<jtv> jelmer: only just now, I think
<jtv> It looks like it might be transient, maybe timing-sensitive.
<deryck> abentley, are you looking for lp.bugs.interfaces.bugtask.BugTaskSearchParams ?
<jtv> allenap: nothing to do with your fix to un-hose our librarian tests?  https://lpbuildbot.canonical.com/builders/lucid_lp/builds/1476/steps/shell_6/logs/summary
<abentley> deryck: No, I
<abentley> deryck: 'm trying to figure out how it gets populated, so I can do it in the right place.
<deryck> abentley, lp.bugs.browser.bugtask.BugTaskSearchListingView.buildSearchParams ?
<abentley> deryck: Yes, but how does that get called?
<abentley> deryck: it gets supplied with extra_params, and I don't know how to get them.'
<deryck> abentley, same file, look for search --> searchUnbatched --> buildSearchParams.  Or are you asking how extra_params gets supplied from that top function?
<abentley> deryck: Yeah, I still need to know how extra_params get supplied to search().
<deryck> abentley, IIRC, I don't think that's used much.
<deryck> abentley, I'm looking around.
<abentley> deryck: I backtraced to eggs/zope.tales-3.4.0-py2.6.egg/zope/tales/expressions.py(211)_eval()
<deryck> abentley, I don't see it actually used anywhere.  You have some error, I take it, where you see extra_params being passed in?
<abentley> I don't know if extra_params is used anywhere, but I thought it was how the extra search parameters were getting into the page.
<deryck> abentley, so I think buildSearchParams is just doing the work. and data gets populated with the self._validate call.  after that it's Zope magic to me.
<flacoste> gary_poster: WebserviceTestCase uses the AppServerLayer, wouldn't it be possible to use wsgi.intercept instead and run in the LaunchpadLayer?
<flacoste> or even DatabaseLayer
<flacoste> DatbaseFunctionalLayer
<flacoste> and let users who need the librarian to upgrade their layer
<flacoste> gary_poster: i remember you adding support to use launchpadlib with wsgi.intercept for tests
<flacoste> but it seems the default testcase base class for that (and that is getting significant traction by devs) isn't taking advantage of that
<gary_poster> flacoste on call, off soon
<gary_poster> flacoste, I did add wsgi.intercept for launchpadlib, yeah
<gary_poster> lemme see where that was
<gary_poster> flacoste, it is in the FunctionalLayer
<gary_poster> So in DatabaseFunctionalLayer
<gary_poster> LaunchpadFunctionalLayer
<flacoste> gary_poster: so in theory, we should be able to change WebserviceTestCase to remove the layer attribute
<flacoste> or change it to layer = DatabaseFunctionalLayer
<flacoste> and everything should be honky dory?
<flacoste> gary_poster: ready on skype whenever you are
<gary_poster> flacoste, or any of the non-appserver layers, yeah.  it expects to answer on "localhost:80" and "api.launchpad.dev:80"
<gary_poster> ok
<benji> Does anyone want to review an exciting and well-written MP?
 * benji pretends as if he's not exaggerating.
<flacoste> benji: if this is the stakeholders' bug i'm thinking it is, i can have a shot at it
<benji> flacoste: it probably is: https://code.launchpad.net/~benji/launchpad/bug-828572/+merge/79733
<flacoste> benji: ok, i'm on it
<benji> cool
<flacoste> benji: looks very good, but i have two questions
<flacoste> 1) is there anything we should remove since the garbo migration is complete?
<flacoste> (code)
<flacoste> 2) since any() in this case is all about status items, couldn't we use search_value_to_where_condition() instead of using recursion and joining with ' OR '?
<benji> flacoste: 1) probably; I was thinking the same thing but I was temted to wait until I can ask Brad when he gets back, but honestly it's very, very likely that we should just remove the code from garbo
<flacoste> yes, if the transition is complete, we should remove it
<benji> flacoste:  2) I don't see an easy way because we have the odd-man-out of having a special case for INCOMPLETE; we could do it but the code would get quite a bit more complex
<benji> flacoste: that was odd, my internet connection dropped on me, did you see my response to 2?
<flacoste> benji: i did, you basically mean that we'd need to support any() in search_value_to_where_condition
<flacoste> benji: which it seems to support :-)
<flacoste> hmm
<flacoste> actually
<flacoste> no
<flacoste> it supports a single any
<flacoste> ok, so that should be fine then
<benji> flacoste: no (although that may be true), I mean that since the user can provide something like any(NEW, INCOMPLETE) we would have to generate something like "status IN (1, 2, 3)", where 2 and 3 are the enum values for INCOMPLETE_WITH_RESPONSE and INCOMPLETE_WITHOUT_RESPONSE
<flacoste> right
<flacoste> that's what i understood
<flacoste> benji: how about:
<flacoste> if INCOMPLETE in status.query_values:
<flacoste>    status = any([value for value in status.query_values if value != INCOMPLETE] + [INCOMPLETE_WITH, INCOMPLETE_WIHTOUT)
<benji> flacoste: I assume you mean for that to be inside the "if zope_isinstance(status, any)" clause.
<flacoste> yep
<flacoste> benji: feel free to use it or something like that if you feel it simplifies
<flacoste> your mp is approved anyway :-)
<benji> flacoste: thanks; I'm experimenting with a variation that might result in equally straight-forward code while also generating the slightly nice SQL you're looking for
<flacoste> that'd be awesome
<abentley> wallyworld_: flacoste has pointed out that your work on batchnavigator and mine on dynamic bug listings may overlap.  Can we chat?
<wallyworld_> abentley: sure
<abentley> wallyworld_: skype or mumple?
<wallyworld_> either. let's try mumble
<benji> flacoste: it turned out quite nicely, thanks for the prod in that direction: http://paste.ubuntu.com/712484/
<flacoste> benji: indeed!
<flacoste> that's nice
<wallyworld_> wgrant: https://code.launchpad.net/~wallyworld/launchpad/delete-bugtasks-1324/+merge/79541
<lifeless> flacoste: wow you're churning out critical-analysis tags now :)
<lifeless> sinzui: ping
<lifeless> sinzui: I want to talk failfan with you
<lifeless> sinzui: at a mutually convenient time
<lifeless> sinzui: also, StevenK wgrant and I were talking about private teams & socialisation, without you :( [you were on public holiday I think :))] - I'd like to confirm the conclusions with you
<wgrant> lifeless: He's out at the moment. Was planning to talk about private teams in 1.5 or 2 hours or so.
<wgrant> failfan?
<lifeless> mailman
<lifeless> but I wanted to say fail
<lifeless> wgrant: ok, well ping me (perhaps on skype) - I'm not here today, but meh... will swap things around on thurs/fri
<wgrant> Ah!
<lifeless> I was working at 4am...
<wgrant> ... bad lifeless
<lifeless> but working amqp oopses
<lifeless> \o/
<cjwatson> wgrant: so I've just been talking with slangasek about getting cron.germinate optimisation onto my official to-do list for 12.04, which will hopefully happen
<cjwatson> wgrant: it looks to me that its runtime is currently 9-10 minutes, 8 flavours * 5 architectures, and I would be surprised if I couldn't trim at least three-quarters of that
<wgrant> Yeah, it should be pretty cuttable.
<wgrant> That's excellent news.
<cjwatson> wgrant: that would mean that quite a few, but not all, publisher runs would fit within 30 minutes (looking at a sample of today)
<cjwatson> what do you think the threshold might be for switching to */30 runs?
<wgrant> And we can cut it down further if we split partner out...
<cjwatson> also parallelising ls-lR against other stuff, perhaps
 * wgrant finds a recent log.
<wgrant> I really should get it bumped up to DEBUG.
<wgrant> It's a bit quiet at the moment.
<cjwatson> ls-lR runtime isn't logged at the moment, no
<cjwatson> I had to watch it in action.  I didn't time that step specifically but it took a couple of minutes.
<cjwatson> actually from 23:27:00 to I think something around 23:30:40
<wgrant> I had a detailed timeline in a Tomboy note a year or so ago. I wonder if that's still around, or if that was in the batch that U1 deleted...
<wgrant> Ah, there it is.
<wgrant> I have ls-lR down as taking 2.5-3 minutes.
<wgrant> Germinate 7, but we have more flavours now.
<cjwatson> and one extra arch
<wgrant> I think this may have been before the powerpc/sparc elimination, but I don't quite recall.
<cjwatson> ah, could be
<cjwatson> (powerpc hasn't been eliminated though, maybe you mean lpia)
<cjwatson> (or hppa)
<wgrant> I meant ia64
<wgrant> ia64/sparc
<cjwatson> all these doorstops
<StevenK> Haha
<wgrant> I count sparc and powerpc as real archs, so I forgot about ia64.
<cjwatson> I haven't measured (I know ...) but it strikes me that ls-lR is I/O-bound while at least cron.germinate (and possibly other finalize.d type stuff) really aren't
<wgrant> That would indeed make sense.
<cjwatson> and cocoplum has four CPUs ...
<cjwatson> Is splitting out partner a near-future kind of thing?
<cjwatson> or merely priority High?
<wgrant> Near future doesn't even come from Critical importance.
<wgrant> it comes from someone doing it in violation of policy :)
<cjwatson> well, LP queue ordering policy doesn't apply to external contributors so no violation involved :-)  but I don't think I'd know where to start
<wgrant> I'm not sure that partner actually takes more than a few seconds, though, so it might not be worth it now that it's all in Python.
<mwhudson> data data data
<cjwatson> mm, I don't see much in the log I can clearly attribute to it
<wgrant> Previously partner had ~a minute of LP process startup time. Now it's all in one process, so it's almost free.
<wgrant> I'm getting data :)
#launchpad-dev 2011-10-19
<wgrant> Ah, ls-lR is done in Python, not with run-parts, so we could parallelise that without too much work.
<cjwatson> For values of "done in Python" including "Python that calls horrible self.executeShell", yes
<wgrant> Well, yes, but the stuff under run-parts is opaque to us.
<wgrant> So this is probably better :)
 * cjwatson wishes this at least used the subprocess module, but there you go
<wgrant> Oh, it uses os.system!?
<wgrant> Ew.
<wgrant> This is brand new code, too.
<wgrant> Ah
<wgrant>         This won't just load an external program and run it; the command
<wgrant>         line goes through the full shell treatment including variable
<wgrant>         substitutions, output redirections, and so on.
<wgrant> Not the best of justifications, but at least it's something.
<cjwatson> The only real justification I can see is that it logs something you can copy and paste as shell, but given how little of Launchpad that applies to I don't think that's much of an excuse
<cjwatson> It would probably be better for the caller to just do it itself using subprocess and kill that abstraction
<wgrant> Indeed.
<cjwatson> Anyway; the ideal management-level justification for me would be if this work let us run the publisher every 30 minutes, or at least brought us substantively closer to that
<wgrant> Yep.
<cjwatson> (Since the driver for this is Kate and Rick pointing out that the image build pipeline is much longer than it ought to be)
<wgrant> I think we'd get some runs in if we added a :32 run right now. But not all of them.
<cjwatson> You would have got at most three extra runs from that today
<wgrant> There we go, much better logging now.
 * wgrant watches.
<cjwatson> Well, I suppose the publisher might have been a bit quicker with less to do, so maybe slightly more
<wgrant> We'll have some better numbers in 20 minutes :)
<cjwatson> Yep, definitely more useful again now
<poolie> hi all
<poolie> lifeless: i posted a conceptual bmp diff spam suppression patch
<poolie> - is it approximately right?
<poolie> - should i bother persisting? i hate giving up, but sometimes things turn out not to be drive by
<StevenK> Oh, damn it all. The PersonFormatterAPI calls is_valid_person everytime
<lifeless> poolie: an actual fix no?
 * StevenK waits for failfan to build
<poolie> yes, this (tries to) make the job retry a bit later
<poolie> so an actual fix along the lines we discussed
<poolie> https://code.launchpad.net/~mbp/launchpad/612171-diff-generation-spam/+merge/79351
<lifeless> thats grewat
<lifeless> I'm not really here today
<cjwatson> ah yes, there we go, 3:01 for ls-lR
<lifeless> but if you haven't had a review tomorrow, I will look at it
<poolie> ok
<lifeless> (I'm taking leave to help w/cynthia wednesdays, + was up @ 4am finishing amqp stuff
<poolie> it's not passing its tests and it probably needs some new ones
<poolie> that's good; that's bad :)-
<StevenK> I'm OCR, but poolie has invited me out to lunch and I have to leave in 5 minutes.
<lifeless> wgrant: so what time is this call ?
<poolie> yeah i was just going to remind you
<poolie> hooray for lunch
<poolie> mwhudson: could you have a look at the diff in the last comment of https://code.launchpad.net/~mbp/launchpad/612171-diff-generation-spam/+merge/79351
 * StevenK looks up address
<poolie> st leonards station
<wgrant> lifeless: When he gets back :)
<poolie> walk out of the station (there's only one gate), look right, there it is
<StevenK> poolie: Right -- I was about to ask you. The address looked familiar so I was going to ask "That's the building the station is in, isn't it?"
<poolie> yep
<mwhudson> poolie: the diff doesn't trigger any particular thoughts in me
<poolie> ok :)
<mwhudson> poolie: i don't know if anything actually respects max_retries, but i'm very out of date there
<poolie> ok
<poolie> i might leave it for today
<poolie> rebooting, biab
<poolie> thanks anyhow
<sinzui> hi lifeless. Do you want to put mailman out of our misery?
<wgrant> It has been slow lately :(
<wgrant> mhonarc is not happy with ubuntu-x-swat.
<wgrant> I think we should disable it for that archive.
<lifeless> sinzui: I think we should do a detailed plan for front-ending it
<lifeless> sinzui: I think its a small project, done well, but I don't know enough of the ins and outs to predict WTF's
<sinzui> lifeless, you mean mhonarc?
<lifeless> lits.
<lifeless> lists.
<sinzui> lifeless, I can post my insane hack to the lp-dev list for everyone to ponder alternates. I think mustache make my insane json suggesting more palatable.
<lifeless> whats your insane hack ?
<lifeless> sinzui: I think there are multiple things to solve at once; we can call it 'refactoring to make it easy to do the intended work' :)
<lifeless> sinzui: slow archiving, poor branding and integration, poor privacy experience, private list archives being inaccessible
<sinzui> yes, those are all things that I thought could be solved by making mhonarc a json-encoded message server. I think we might also ask though is do we want a simpler mechanism to store and retrieve mbox message data? Messages could be places in a db, or we build a mbox server from other tools
<lifeless> AIUI generating the mboxes is cheap because you write to the end
<wgrant> I'd imagined that mailman would call an archive that injects the message into some kind of DB.
<wgrant> That isn't mbox.
<lifeless> wgrant: thats what mhonarc is, isn't it ?
<lifeless> wgrant: (in an SOA model)
<wgrant> Well, some kind of DB that doesn't then slowly generate static pages.
<lifeless> right, thats what the change to json is about
<lifeless> curtis and I have discussed this before
<wgrant> So it would generate static JSON instead?
<lifeless> I think the time is right to do it, rather than layering headaches on headaches, for lists. privacy
<lifeless> wgrant: yes, and stop the crazy rewriting of big lists
<sinzui> wgrant, yes, static fast data that the the Lp app can parse and linkify user and bugs
<wgrant> How do you plan to avoid rewriting but still generate static JSON?
<sinzui> but putting messages in a db is better for at least  two use cases. Searching, and deleting/redacting messages
<wgrant> And cheapness of updating.
<sinzui> mhonarc has an excellent mechanism to resurrect and present confidential data to everyone
<wgrant> Indeed.
<sinzui> wgrant, I wasn't planning to avoid rewriting. mhonarc does a lot of re writing now because we have the indexes set to reverse. Once a list has more than a page of messages, adding a message causes a rewrite of all indexes and most message :)
<wgrant> Yes.
<wgrant> If it's in a DB with a service serving dynamic JSON, that problem goes away.
<sinzui> I was thinking of using natural indexes and letting lp reverse the MM index data
<sinzui> But my hack is predicated on keeping mhonarc, and writing embarrassing PERL.
<lifeless> wgrant: so, there are two orthogonal things
<sinzui> I would be happier keeping messages in something I could confidently hack and search.
<lifeless> wgrant: a) better interface for LP - LP templates, json backend etc.
<lifeless> wgrant: b) better archiver implementation - cassandra etc etc
<lifeless> wgrant: a) with mhonarc generating forward indices rather than reverse, + json -> will solve nearly all our performance and ops issues
<lifeless> wgrant: giving us time to do b), or even advocate for someone else to do it for us :)
<lifeless> sinzui: I'm very keen to contain scope
<lifeless> sinzui: I think search should be done by inserting into solr-or-similar and querying that from LP
<sinzui> lifeless, importing and exporting data has been a blocker for list adoption. Users cannot easily migrate to Lp, nor can we give them the data for analysis. I could not answer statiks questions about monthly list volume for example
<lifeless> sinzui: I presume mhonarc makes that hard?
<sinzui> It's db is a perl hash :)
<lifeless> sinzui: aka yes ;)
<lifeless> sinzui: do you agree that its better to do two projects than one: sort out the interface, then sort out the now-contained evil ?
<sinzui> lifeless, yes
<lifeless> sinzui: cool
<lifeless> sinzui: do you think sorting out the interface [switch to LP templates + jason backend, *perhaps* do something about mbox [e.g. store in librarian]] is a better way to tackle the privacy banner than e.g. copying templates onto the mhonarc install ?
<lifeless> (better meaning cheaper-over-all-including-cost-of-maintenance)
<sinzui> I think making mhonarc serve JSON via hack and teaching LP to retrieve the data provides a lot of options. Importing/exporting mboxes might be easier in Lp. I am just not certain. export is easier, but import is still hard because we need to register all the email addresses
<sinzui> lifeless, wgrant: my hack idea assumes I can always make proper JSON to describe the message. I think that means hacking in perl :)
<sinzui> I am sure someone has done that, but It also feels like the time I was asked to write a multiple-part post decoded in vb because ASP did not support it, when every modern web server tool kit does
<lifeless> sinzui: did you have minions^Wsquad members then ?
<sinzui> odd when I last work with perl I had minions.
<sinzui> lifeless, I think the answer might be to extend https://launchpad.net/mmm-archive-manager to do the serialisation task to minimise mhonarc's part in our victory
<lifeless> sinzui: perhaps that should be part of launchpad-project? [or is it personal? I dunno...]
<lifeless> sinzui: anyhow, extending that makes as much sense as anything to me
<sinzui> I wrote it on my own time and I believe it works with Ubuntu list archives too
<lifeless> I have no objection to how its done ... and don't know enough to seriously compare implementation approachs at this level
<lifeless> sinzui: before you halt() tonight, I wanted to talk about private teams
<sinzui> yes. I asked wgranta about that. I want to talk about it
<lifeless> when is good?
<lifeless> sinzui:
<sinzui> now if you like
<lifeless> cool!
<lifeless> skynet ?
<sinzui> skype?
<lifeless> yes
<lifeless> <- tired, so the pathetic sense of humour is rolled out :>
<sinzui> lifeless, are you ready to talk?
<StevenK> sinzui: Are you still around?
<StevenK> wgrant: Ahh, just thinking about it, does your obsolete hackage allow us to kill bug 740584?
<_mup_> Bug #740584: Build lacks a corresponding source publication <oops> <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/740584 >
* jtv changed the topic of #launchpad-dev to: Launchpad should be engineered like an AK-47 | Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: jtv | Critical bugtasks: 273
<jtv> wgrant: I'm off for food, but meanwhile, would appreciate your thoughts on this!  http://paste.ubuntu.com/712840/
<wgrant> jtv-eat: So, when I said that it should be refactored, it was meant to just be a first step.
<wgrant> But it was instead treated as the final step.
<wgrant> What we want to here is not defined.
<wgrant> And probably not possible in the current model.
<wgrant> s/to/to do/
<StevenK> wgrant: Ahh, just thinking about it, does your obsolete hackage allow us to kill bug 740584?
<_mup_> Bug #740584: Build lacks a corresponding source publication <oops> <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/740584 >
<wgrant> StevenK: No.
<StevenK> Will that destroy most of the problematic packages?
<wgrant> It will remove some files from disk.
<wgrant> It doesn't affect the builds not having publications, however.
<jtv-eat> wgrant: thanks for looking into this.  Does your response mean that you found nothing wrong with my guesses that the problem is in notify() and that the unwanted recipient is the maintainer?
<wgrant> jtv-eat: I think that treating this as an isolated problem is probably missing a better solution
<wgrant> jtv-eat: We need to think about who actually wants failure notifications.
<wgrant> And then work out how to implement that.
<jtv-eat> I think it'll need some restructuring before we can do _anything_ with it.
<wgrant> Right.
<wgrant> As I said, StevenK's refactoring was meant to be a preliminary step in the DD build notification work.
<wgrant> But... well, we saw what happened :)
<jtv-eat> The current code seems fairly effective when it comes to obscuring the requirements.
<wgrant> It used to be far worse.
<wgrant> But yes.
 * jtv-eat really goes off to eat
<poolie> does anyone know off hand how to ask lp if my credentials are still valid?
<poolie> i guess just do any request
<poolie> ... but not all of them seem to actually check it
<wgrant> poolie: Some may be cached?
<poolie> no, apparently 'GET /1.0/' doesn't check the oauth tokens
<poolie> s/tokens/stuff
<poolie> is it a bug?
<poolie> https://bugs.launchpad.net/launchpad/+bug/877913 - not urgent
<_mup_> Bug #877913: GET api.launchpad.net/1.0/ doesn't check OAuth validity <api> <oauth> <Launchpad itself:Triaged> < https://launchpad.net/bugs/877913 >
<poolie> so the problem behind bug 877856 is partly that lp.distributions['debian'] does not cache the resulting object
<_mup_> Bug #877856: makes many pointless object reference api calls <Ubuntu Distributed Development:In Progress by mbp> < https://launchpad.net/bugs/877856 >
<poolie> seems like it could ; i don't know why not
<wgrant> wallyworld_: Do you happen to recall when the JS API client started returning Entrys instead of plain JSON dicts?
<wgrant> It caused bug #830676
<_mup_> Bug #830676: PPA AJAX build status indicators don't update properly <buildd-manager> <critical-analysis> <ppa> <regression> <Launchpad itself:Triaged> < https://launchpad.net/bugs/830676 >
<wallyworld_> wgrant: not sure. it may have been before i first worked with any of that stuff, since i only recall it being Entry
<wgrant> wallyworld_: Let me just check the build history of our excellent, thorough JavaScript test suite.
<wallyworld_> wgrant: is this doing a patch from the client?
<wgrant> revno: 11451 [merge]
<wgrant> timestamp: Thu 2010-08-26 16:32:27 +0100
<wgrant> message:
<wgrant>   [r=jtv][ui=none][bug=308198][for=foxxtrot] Make Collection contain
<wgrant>         proper Entry's.
<wgrant> Surely it hasn't been broken for a year...
<wallyworld_> that does seem right though, since as i said, it was always Entry when I looked at it
<wallyworld_> and i started in 08 last year
<poolie> wgrant i was wondering if we should have some kind of semaphore to limit outstanding requests from jubany to lp
<poolie> so we can ramp up the concurrent threads when they're busy doing non-lp work
<wgrant> poolie: That would probably be the right way to do things.
<wgrant> poolie: api.launchpad.net/1.0/ is served by Apache, at least for some Accept headers.
<poolie> ah, for performance
<poolie> so it's almost wontfix?
<wgrant> eg. the WADL is served at that URL with some obscene Content-Type, and that's direct from Apache.
<wgrant> I think so.
<poolie> yeah, insane Vary header
<poolie> and it's zope not apache that's checking oauth
<wgrant> Yep
<wgrant> wallyworld_: Ah, found it.
<wallyworld_> do tell
<wgrant> At least I think.
<wgrant> 13303.10.6
<wgrant> Just over a month before the bug.
 * wgrant tests.
<wallyworld_> at least that's not a year
<wgrant> Heh
<wgrant> Yep, that was it.
<wgrant> jtv-eat: Could you please review https://code.launchpad.net/~wgrant/launchpad/bug-830676/+merge/79772 upon your return?
<poolie> perhaps i should actually change lplib to optionally return slightly stale objects from cache
<poolie> not a lot of point having a cache otherwise
<lifeless> poolie: lplib objects are not cachable beyond deploys anyhow
<lifeless> I think caching beyond the freshness lifetime would likely lead to bugs - and a cache is plenty useful even if stale objects are never returned :)
<poolie> to avoid transferring the body?
<poolie> hm, what do you mean by 'freshness lifetime'?
<wgrant> I thought we must-revalidate
<poolie> if you mean the http lifetime, then they do not seem to last even that long
<poolie> wgrant, we don't actually send must-revalidate, but the behaviour is as if the application is always asking for it
<wgrant> Ah
<poolie> basically what i'm saying is: i don't know if all applications want must-revalidate always on
<poolie> perhaps if they fetched an object a second ago they'd be happy to get it back from cache
<poolie> "publication.distro_series.name.lower()" does a round trip
<poolie> headdesk
<poolie> every time you call it
<wgrant> Ah, and jubany is in the DC, so instead of being hilariously slow it just DoSes us.
<wgrant> lolz.
<poolie> to get back the string 'sid' or something of course
<wgrant> Yep
<poolie> ok this is much better now
<wgrant> Excellent.
<adeuring> good morning
<danilos> jtv-eat, hi, I wonder if you could take a look at https://code.launchpad.net/~danilo/launchpad/bug-817398/+merge/79571 when you are done eating, you'll probably like the branch (or at least, what's it fixing :))
<wgrant> danilos: Heh, and so the hackish launchpad.Edit grows a third instance...
<danilos> wgrant, yeah, I first started off with removeSecurityProxy in the buildd manager code, but that ain't any less hacky :) changing all the interfaces is even more fun, so I avoided that too
<danilos> jtv, hey-hey, I hope food hasn't killed you :)
<nigelb> heh
<jtv> danilos: hi, sorry, bit busy.  Gotta do wgrant's first.
<wgrant> Mine's pretty tiny :)
<danilos> I was just going to say how I've got another one in the queue for jtv
* danilos changed the topic of #launchpad-dev to: Launchpad should be engineered like an AK-47 | Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: danilos (lunching) | Critical bugtasks: 273
<nigelb> hah. Nice quote on topic :)
<bigjools> nigelb: although it could be misconstrued in *so* many ways
<bigjools> easy to shoot your foot... check
<bigjools> preferred choice of terrorists... check
<nigelb> bigjools: I was actually thinking about "easy to shoot your foot" myself :)
<bigjools> danilos: I have a short-ish branch for your pleasure when you get back from lunch, TIA.
* danilos changed the topic of #launchpad-dev to: Launchpad should be engineered like an AK-47 | Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: danilos (lunching) | Critical bugtasks: 273
<danilos> bigjools, "short-ish"? you mean just a bit over 600 lines? :)
* danilos changed the topic of #launchpad-dev to: Launchpad should be engineered like an AK-47 | Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: danilos | Critical bugtasks: 273
<danilos> bigjools, does "overriding" ddebs mean they will end up in the PPA directly? if so, will that affect allocated storage space?
<danilos> bigjools, fwiw, the branch is good as it is, i.e. it looks like it does the "overriding" of pocket/section/blah-blah well, and the tests "prove" it; the question above remains mostly for my own curiosity
<bigjools> danilos: hi, back from lunch myself
<bigjools> danilos: overriding means that the component/section/priority is set
<danilos> bigjools, so nothing but that? (otp, btw)
<bigjools> the ddebs need to be in lockstep with the debs like that, otherwise they won't get superseded properly
<bigjools> noep
<bigjools> nope, even
<danilos> ok, cool, thanks
<bigjools> np
<bigjools> thanks for the review
<danilos> yw
<deryck> Morning, all.
<adeuring> morning deryck
<adeuring> danilos: could you please review this MP: https://code.launchpad.net/~adeuring/launchpad/custom-bugs-listings-search-form-tweaks/+merge/79812 ?
<danilos> wallyworld_, hi, I reviewed https://code.launchpad.net/~wallyworld/launchpad/delete-bugtasks-1324/+merge/79541 and have a few questions in the MP; if you don't understand something, please let me know
<wallyworld_> danilos: thanks, will look
<danilos> adeuring, sure, would you like to return the favour by looking at https://code.launchpad.net/~danilo/launchpad/bug-877179/+merge/79781? :P (no diff either, that seems broken :/)
<adeuring> danilos: sure, I'll look
<danilos> adeuring, I'll also grant you the "last review Danilo will officially do as OCR" title, fwiw :)
<adeuring> ;)
* danilos changed the topic of #launchpad-dev to: Launchpad should be engineered like an AK-47 | Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: - | Critical bugtasks: 273
<wallyworld_> danilos: fair point about admin. i think that should be added. the store.flush() is definitely required for tests to ensure flags passed into the constructor are visible. for prod, i don't think it's needed but as you say, likely not an issue
<wallyworld_> the feature flag in the permission check - i'm not sure where else to put it. i'll need to think about it
<danilos> wallyworld_, well, generally one would put it around the code that is feature flagged; i.e. in delete method instead, instead of failing with Unauthorized, it could fail with IAmNotAMethodYouMustBeDrunk() exception or something :)
<wallyworld_> danilos: sure. i understood the requirements to be that if the fflag were not on, it was to be treated as 'unauthorised' hence the implementation
<danilos> ok, maybe a NotImplementedError, but I ain't sure really what's the "right thing" to do, I just dislike this ;) if you can't come up with anything nice, just leave it as is
<danilos> (though, I like the YouMustBeDrunk exception for this)
<danilos> wallyworld_, if those were the requirements, then fine, keep it as is (I don't like the requirements, but hey, life is tough)
<wallyworld_> :-) np. it's definitely a fair question. also, with the tests and the fflag stuff, the fflag will be going away (soon hopefully) hence i thought it ok to "double up" so to speak
<wallyworld_> i'll confirm the requirements in case i have misunderstood
<wallyworld_> which is not unprecedented :-)
<wallyworld_> i do like your suggested exception class :-)
<wallyworld_> i've had a few glasses of wine so am already half way there :-D
<danilos> wallyworld_, as for tests, sure, keep them as is, but don't forget to remove the feature flags then :)
<wallyworld_> will do :-) i'm about to remove all the feature flags for the picker enhancements \o/
<danilos> wallyworld_, cool, great to hear that!
<wallyworld_> :-)
<wallyworld_> i'll think about tests for conjoined bugtasks
<danilos> thanks
<danilos> wallyworld_, sorry for being such a PITA, but you got to know me already :)
<wallyworld_> a pre impl talk showed the code base doesn't really place any significance on them though
<wallyworld_> no, not a pita. i really appreciate the input
<danilos> wallyworld_, yeah, if so, then ignore me on that point, I just wanted some confirmation (you did mention the pre-imp with lifeless in the MP I believe)
<wallyworld_> shows you have taken the time to think about the code
<danilos> or that I am good at coming up with things which resemble arguments about code, yes
<wallyworld_> i talked with wgrant about the conjoined tasks in particular. he showed me how the gui/code now doesn;t care if a conjoined master/slave is there or  not
<danilos> wallyworld_, ah right, lifeless raised the issue, you talked it over with wgrant, which means you are pretty well covered :)
<wallyworld_> i would prefer to be challenged rather than a simple +1. that's the best way to avoid base code :-)
<wallyworld_> s/base/bad
<wallyworld_> yeah, if those guys make a mistake, it's pretty rare :-)
<wallyworld_> i'll add a note to the mp so that sinzui can see what we discussed when he looks at it
<bigjools> hey benji, I decided to sit down and fix that keyring corruption thing in launchpadlib, and blow me if it's not happening any more .... !
<bigjools> you can't make this stuff up
<benji> bigjools: heh, that's life for you
<bigjools> benji: I'm trying to work out exactly what the keyring was doing to the credential string in the first place. Maybe it was a bug in the kwallet and it's now fixed.  Hmph.
<mrevell> danhg is here and rockin' like Dokken
<danhg> I sure am
<bigjools> Dokken... not heard them for ages
 * bigjools welcomes danhg to the public channel too
<rvba> Hey danhg!
<nigelb> Hello! danhg
<james_w> welcome danhg
<deryck> allenap, ping
<allenap> deryck: pong
<deryck> allenap, hey man! :)  Did I hear correctly that you're looking into the ssh errors with test runs, that are supposed to be avoidable on Natty?
<allenap> deryck: Not ssh errors, but database errors. Are you getting ssh errors too? Gulp.
<allenap> deryck: The problem I'm working on is that some disconnections are not being detected as such and are bubbling up.
<deryck> allenap, oh, sorry, no.  I thought that's what they were.  abentley is blocked running tests locally with these errors, I believe.
<deryck> abentley, what allenap reports there is what you're seeing?
<allenap> deryck: I'm waiting on reviews from Storm people. I'll go and hussle for them now.
<deryck> allenap, thanks!  abentley is seeing this on Natty, too, so he's completely blocked landing some stuff.
<deryck> assuming we're talking about the same issue :)
<allenap> deryck: I've heard that the problem does not manifest in ec2, and mbp said tests run fine in a Lucid schroot. The latter only takes a few minutes to get set-up.
<allenap> So that's a workaround for now.
<deryck> ah, that's nice.
<adeuring> danilos: r=me
<deryck> abentley, ^^ Lucid schroot FTW? :)
<danilos> adeuring, thanks, I am still on yours (sorry for taking so long, had a call in the meantime)
<adeuring> danilos: np
<abentley> deryck, allenap: I am experiencing the SSL connection has been closed unexpectedly
<deryck> I knew there was an "ss" in there :)
<allenap> abentley: Yes, that's the badger.
<allenap> abentley, deryck: I'm sorry a fix has taken so long to be ready. Mainly it's been hard to replicate in tests, and I'm now blocked on reviews.
<abentley> deryck: I have natty and maverick vms.  I haven't been using schroots.
<abentley> deryck: I suspect that it was broken by recent changes to launchpad-dependencies.  I can try my maverick vm without upgrading, but having stale packages can itself be a problem.
<adeuring> danI just noticed that the cover letter was quite nonsensical. I added a hopefully better readable comment...
<danilos> adeuring, yeah, I kind of read it as random notes you were making for yourself, so other than figuring roughly what the branch is about, I didn't rely on them too much :)
<danilos> adeuring, anyway, r=me with a few superficial comments :)
<adeuring> danilos: thanks!
<danilos> adeuring, btw, you can edit the MP as well, fwiw :)
<danilos> adeuring, (no need to add a comment)
<adeuring> danilos: ah, right! next time, then ;)
<danilos> adeuring, and then nobody can tell ;)
<adeuring> ;)
<abentley> deryck: This branch has sorting, ajax and caching enabled: lp:~abentley/launchpad/cache-batches
<abentley> deryck: If you branch off it, please merge only it, not devel, to avoid criss-cross merges.
<danilos> wallyworld_, btw, I don't think the way you introduced security adapter is sufficient: in my reading, it modifies the launchpad.Admin privilege for BugTask to match the launchpad.Delete privilege, but does not really allow admins to do the delete
<danilos> wallyworld_, your test is not really testing what I thought it should (i.e. log in as admin@canonical.com and not someone holding a contextual launchpad.Admin permission [like target owner])
<danilos> wallyworld_, (or better yet, login_celebrity('admin'))
<abentley> deryck: maverick had bugs related to pgbouncer files being missing.  Trying a schroot now...
<deryck> abentley, ok, cool.
* abentley changed the topic of #launchpad-dev to: Launchpad should be engineered like an AK-47 | Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: abentley | Critical bugtasks: 273
<abentley> deryck: lucid schroot seems like a rather awful option, as it leaves me with the encrypted version of my home directory, and will have port conflicts with my system daemons.
<deryck> abentley, hmm, yeah, yuck.  I'm not a schrod user so don't have much to offer in advice.
<deryck> schroot even :)
<micahg> bigjools: about bug 877122, the API syncing was a new feature, but it used to be that all packages had debdiffs and now they don't, so wouldn't that be a regression?
<_mup_> Bug #877122: Packages synced through the API are not producing debdiffs <derivation> <Launchpad itself:Triaged by redsquad> < https://launchpad.net/bugs/877122 >
<bigjools> micahg: it's not a regression if this feature never had that :)  The old syncing did it of course.  I have scheduled it to be fixed soon tjough, don't worry
<bigjools> though*
<micahg> bigjools: ok, it is a regression in functionality for Ubuntu though, thanks for getting it scheduled
<dobey> hey guys, when do things fixed in launchpad stable, show up on production?
<maxb> dobey: http://lpqateam.canonical.com/qa-reports/deployment-stable.html
<dobey> ah, interesting. thanks maxb
<maxb> dobey: IIUC, once something goes green there, the remaining steps are for someone on the Launchpad team to ask for a deployment to happen, and for it to be actioned by losas - so fairly soon, subject to working days, etc.
<dobey> maxb: sure. just wondering when my bug will be fixed in production since jtv's branch landed :)
<lifeless> mpt: an observation: your 'what should happen' entries in bugs are very close to 'shoulds'
<mpt> lifeless, they are. It's the describing the problem first that's the important thing. :-)
<mpt> Maybe I could use "What I expected:" instead
<lifeless> mpt: Or 'one way I would not have been surprised:' :)
<lifeless> mpt: I guess my point is to open the solution space out
<mtaylor> I suppose folks know that launchpad login service is unhappy?
<mtaylor> org.openid4java.discovery.yadis.YadisException: 0x706: GET failed on https://login.launchpad.net/+openid : 503:HTTP/1.1 503 Service Unavailable
<lifeless> mtaylor: grah. Thats login.ubuntu.com. I'll go poke folk
<lifeless> mtaylor: fixed
<lifeless> mtaylor: sso is having some flakiness issues this week
<mtaylor> lifeless: ++
<lifeless> mtaylor: its escalated but not understood yet
<lifeless> (theories abound, proof is tricky)
<mtaylor> lifeless: distributed systems are hard
<mtaylor> lifeless: btw - it's still broken
<mtaylor> lifeless: https://jenkins.openstack.org
<lifeless> mbarnett: ^
<lifeless> mbarnett: ah you know, nvm
<lifeless> mtaylor: cascading trouble
<mtaylor> yay!
 * mtaylor loves trouble
 * mbarnett doesn't always break things, but when he does he breaks them all. 
* abentley changed the topic of #launchpad-dev to: Launchpad should be engineered like an AK-47 | Performance Tuesday! | https://dev.launchpad.net/ | On call reviewer: - | Critical bugtasks: 273
<thumper> hi people
<thumper> I'm looking for a "stock" presentation for "intro to launchpad"
<thumper> is there one around somewhere?
<thumper> We have a growing team, and some don't really understand launchpad, so I'm doing a presentation next week to cover the basics
<mwhudson> in unrelated news, launchpad's roles for managing (in various senses) projects make no sense to anyone
#launchpad-dev 2011-10-20
<lifeless> can has review ? https://code.launchpad.net/~lifeless/python-oops-tools/amqp/+merge/79749
<wgrant> lifeless: Don't want to use logging (timestamps!) rather than print?
<lifeless> wgrant: console debugging me
<lifeless> wgrant: I don't see a use case (today) for it live
<wgrant> eparse
<wgrant> Ah
<wgrant> Yeah, but.
<wgrant> OK.
<lifeless> s/me/meh/
<wgrant> Ah!
<wgrant> lifeless: https://code.launchpad.net/~wgrant/launchpad/bug-735998/+merge/79905
<lifeless> wgrant: btw I think the plan has changed over the last year
* wgrant changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: - | Critical bugtasks: 266
<lifeless> wgrant: prob with the table width changes
<wgrant> lifeless: Probably, yes.
<wgrant> This timeout is reasonably new.
<wgrant> And mostly affects #1.
<wgrant> lifeless: thanks.
<lifeless> np, thank you
<StevenK> stub: Hai! Could you look over https://code.launchpad.net/~stevenk/launchpad/db-packageupload-archive-index/+merge/79903 ?
<stub> à¸­à¸­
<wgrant> We really should have (archive, status), or perhaps (archive, distroseries, status).
<StevenK> As well?
<wgrant> Instead.
<StevenK> How that help the delete?
<wgrant> status doesn't, but it helps process-accepted.
<stub> StevenK: The immediate problem is deletion of rows? Those other indexes will be used for delete too.
<StevenK> stub: CASCADE deletion from Archive
<stub> We delete archives now?
<StevenK> We were going to make an exception in this case. 53 unused PPAs owned by open teams
<stub> We already have an index on (distroseries, status). Does that become redundant with (archive, distroseries, status). ie. when we are deleting by (distroseries, status), do we always include an archive too?
<stub> If it is one off, we can just delete the packageupload rows first, then the cascade delete from archive
<wgrant> There are no packageupload rows.
<wgrant> Or at least shouldn't be.
<wgrant> But there's no index to let it discover that within 2.5s.
<StevenK> Right
<stub> Which isn't a problem for a select
<wgrant> stub: Every query should specify an archive too. Except DistroSeries:+queue, which will do archive IN (1, 534).
<wgrant> But there shouldn't have been archiveless queries since 2007 :*(
<wgrant> :(
<wgrant> stub: Hm?
<wgrant> stub: We preselect the archive IDs.
<wgrant> Then try to delete them.
<stub> So we should drop the (distroseries status) index and add (archive, distroseries, status) which will help the immediate problem and make other stuff go faster too.
<wgrant> Which times out when looking for packageuploads to cascade to.
<wgrant> Yes.
<stub> What are the archive ids?
<StevenK> stub: The query is on RequestLogging in the usual place
<StevenK> Right, fixing the patch to drop and create an index
<wgrant> Will need to be two separate patches.
<wgrant> One will be run live with CONCURRENTLY.
<wgrant> Then drop the old index during downtime afterwards.
<StevenK> Why?
<wgrant> Because we can't create the index during downtime.
<wgrant> Too slow.
<stub> RequestLogging?
<stub> found it
<wgrant> https://wiki.canonical.com/InformationInfrastructure/OSA/RequestLogging/LP/SQL
<StevenK> -CREATE INDEX packageupload__archive__idx ON PackageUpload(archive);
<StevenK> +DROP INDEX packageupload__distroseries__status__idx;
<wgrant> Still need a patch to create the index.
<StevenK> Assuming the index is created beforehand
<wgrant> 92-1 will create
<wgrant> 92-2 will drop
<wgrant> 92-1 will be applied live with CONCURRENTLY
<stub> Yup
<StevenK> Changes pushing
<lifeless> StevenK: why are we dropping that index?
<StevenK> Because we're creating one on archive, distroseries, status instead
<wgrant> (distroseries, status) made sense until 2007.
<lifeless> uhm, isn't it ordered differently ?
<lifeless> I'm just flagging the 'btw this may break pages' aspect
<wgrant> If it does, the pages are buggy. We could perhaps delete in a few days after checking index usage numbers, though.
<lifeless> Give you once chance to guess what I think of that :)
<wgrant> Mmm.
<wgrant> I'd prefer to find the buggy pages :)
<StevenK> stub: Thanks for the +1, I'll lp-land it against db-devel
<wgrant> StevenK: Seen lifeless' comments?
<wgrant> Like 6 lines ago.
<StevenK> Yes.
<StevenK> wgrant: I'd rather we find the buggy pages too
<wgrant> Yes, but this could entirely break them.
<lifeless> given you guys are not on maintenance, and there is moderate risk triggering criticals...
<lifeless> stub: do we have index utilisation info now ?
<wgrant> It's likely that some queries use (distroseries, status) now.
<wgrant> So the stats now won't be too useful.
<wgrant> Because distroseries, status is reasonably selective where status is 0 or 1, which is the common case.
<lifeless> wgrant: the point being to get the new index up and see what moves across
<stub> lifeless: PG tracks hits on the indexes.
<lifeless> stub: ok cool
<stub> StevenK: I can create that index when the backups finish
<wgrant> lifeless: Oh, sorry, I thought you were asking if we could get stats right now.
<StevenK> stub: Excellent, thanks.
<StevenK> wgrant: So should I land this or not?
<wgrant> I'd land the first patch once the index is on prod, and the second once we know it's not being used significantly.
<stub> Nah... just keep them in the same branch and keep the mp open
<wgrant> Or that, if you don't care about having prod patched with something out-of-tree for days.
<stub> The second patch will go through staging too so get to test there for major perf regressions
<wgrant> Our trackrecord of catching that stuff on staging is... less than optimal.
<jtv> lifeless: are you reviewing?  If so, I have a very brief MP for you: https://code.launchpad.net/~jtv/launchpad/bug-878115/+merge/79919
* rvba changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | On call reviewer: rvba* (allenap) | Critical bugtasks: 266
<wgrant> jtv: Approved.
<jtv> Thanks.
<wgrant> jtv: security.cfg changes are applied during nodowntime updates.
<jtv> I thought PQM rejected any changes in database/schema for devel.
<wgrant> Only *-0.sql and fti.py, I believe.
<wgrant> And both of those restrictions are pointless now.
<wgrant> So I will drop the check entirely when nobody is looking.;
<jtv> Nope.  I ran into it a few times while fixing lint in python.
 * wgrant checks.
<wgrant> Yeah, *-0.sql and fti.py are banned.
<wgrant> Everything else is permitted.
<wgrant> [ -z "$(bzr status -S database/schema/ | grep -e '\(-0\.sql\|/fti.py\)$')" ]
<jtv> Maybe the lint I fixed was in fti.py then, but I think I touched more than one python file there.  Maybe the check's been updated already.
<wgrant> It was fti.py, yes.
<wgrant> The check hasn't been updated for a few months.
<wgrant> It used to be stricter.
<lifeless> wgrant: they aren't pointless; there is a report needed before we drop them - in deployment thingy
<jtv> OpenID, how many ways do I love thee?  Let me count the ways:
<jtv> â¦
<jtv> Session expired.  Please log in again.
<jtv> â¦
<wgrant> lifeless: Except that -0 patches don't exist any more.
<jtv> 404 no such page: /weird-openid-stuff-this-app-doesn't-handle
<nigelb> jtv: OAuth is even more awesome.
<wgrant> stub: Can I define a partial index in a CREATE TABLE, or must I do it afterwards?
<stub> wgrant: It needs to be done after.
<wgrant> :(
<wgrant> Thanks.
<lifeless> wgrant: stub: it does?
<lifeless> oh, depends on the definition of after
<lifeless> ... I mean 'same patch should be fine'
<stub> yes
<lifeless> stub: when would you like to catch up ?
<stub> but the sql syntax doesn't support it in the CREATE TABLE
<stub> lifeless: whenever :-)
<wgrant> Yeah, I meant just the syntax issue.
<jtv> That was clear from your question.
<jtv> However
<jtv> I have one for you:
<jtv> do you know how I find a particular recipe using launchpadlib?
<wgrant> I was wondering when you'd ask about that.
<lifeless> do you mean search, or obtain the object for a known recipe?
<jtv> Well, here we are.  :)
<jtv> lifeless: a particular recipe.
<wgrant> What lifeless said.
<wgrant> Ah, you know it?
<jtv> This one: https://code.staging.launchpad.net/~ubuntuone-hackers/+recipe/client-dailies
<wgrant> Ah, different issue.
<wgrant> I see.
<wgrant> I was hoping you were investigating the request-daily-builds failure.
<wgrant> However, lp.people['ubuntuone-hackers'].getRecipe(name='client-dailies') should work.
<jtv> Ah, thanks!
<wgrant> wallyworld_: We need to continue to permit nominations.
<wgrant> wallyworld_: Nominations do not transcend pillar boundaries.
<wgrant> (approving those nominations must be permitted, too)
<wgrant> But adding a new pillar to a bug, or transitioning the pillar of a task when there are series tasks, must be forbidden.
<wgrant> Series are not intended to be a privilege boundary, so such tasks shouldn't cause any issues with further disclosure implementation.
<wgrant> (well, they are a weak privilege boundary: driver privileges can be granted within a single series)
<wgrant> But they are irrelevant for visibility.
<adeuring> good morning
<mrevell> Hello
<jtv> wgrant: I'm trying to verify that bug 876594 is fixed on staging.  Are you privilege to run this, by any chance?  http://paste.ubuntu.com/713960/
<_mup_> Bug #876594: rejected builds for synced packages send mail to Debian maintainer <derivation> <regression> <Launchpad itself:In Progress by jtv> < https://launchpad.net/bugs/876594 >
<wgrant> I think that's the wrong bug.
<wgrant> But yes, I should be able to do that.
<jtv> How is the wrong bug?
<jtv> Oh
<jtv> It just is.
<jtv> Too many in progress.
<jtv> Hi bigjools
<jtv> The right bug is but 870130
<jtv> *bug 870130
<_mup_> Bug #870130: shortlist error requesting recipe build <easy> <oops> <qa-needstesting> <recipe> <Launchpad itself:Fix Committed by jtv> < https://launchpad.net/bugs/870130 >
<bigjools> o/
<nigelb> Morning bigjools
<jtv> wgrant: any luck?
<wgrant> jtv: >>> recipe.requestBuild(pocket='Release', distroseries=natty, archive=recipe.daily_build_archive)
<wgrant> <source_package_recipe_build at https://api.staging.launchpad.net/1.0/~ubuntuone/+archive/nightlies/+recipebuild/87799>
<jtv> wgrant: that sounds successful.
<jtv> Suggests that the bug is indeed fixed.
<wgrant> Yep.
<jtv> Thanks!
 * jtv goes for a quick relocate
 * wgrant adds bug #3 to the list to delete tasks from next week.
<_mup_> Bug #3: Custom information for each translation team <feature> <lp-translations> <Launchpad itself:Fix Released> <MTestZ:New> <Ubuntu:Invalid> <mono (Ubuntu):Invalid> < https://launchpad.net/bugs/3 >
<jtv> StevenK, wgrant: w.r.t. bug 876594 I feel somewhat tempted to make get_recipients even cleverer, but I'd like to resist.
<_mup_> Bug #876594: rejected builds for synced packages send mail to Debian maintainer <derivation> <regression> <Launchpad itself:In Progress by jtv> < https://launchpad.net/bugs/876594 >
<wgrant> jtv: Ignore implementation, define requirements :)
<jtv> Thank you.  I need some words of encouragement.
<wgrant> notifications got where they are today because of gradual papering-over of bugs.
<StevenK> Then they got a little bit better.
<StevenK> Well, I like to think so.
<wgrant> Let's see if we can work out what we actually want, and develop a consistent implementation :)
<jtv> Actually, in a way, developing an inconsistent implementation may be better.
<jtv> AFAICS we could just choose not to use notify() from the syncing code at all, and do something that's tailored to syncing.
<jtv> It doesn't solve the existing mess, but at least it takes a chunk out of it and lets us design something that makes sense for do_copy.
 * wgrant preemptively vetoes more duplicated implementations.
<wgrant> Unless you also schedule the removal of the old implementation.
<jtv> Doesn't make it a duplicate implementation.
<jtv> We're currently notifying on copy just as if it were a regular upload.
 * wgrant looks disapprovingly at direct copies, delayed copies, and PackageCopyJobs.
<jtv> I think we need a free rein to decide what's appropriate for copying, _not_ force "it must be the exact same code as upload notifications" onto the design a priori.
<wgrant> It could have different recipient determination code.
<wgrant> But the stuff to determine the content should be identical.
<wgrant> For values of "should" that tend towards "must"
<jtv> Pah.  That's content and I'm dealing with recipients.
<jtv> bigjools: I just reported bug 878795 for those annoying failures we're getting from the archive uploader.  I don't _think_ it's Critical, but not sure.
<_mup_> Bug #878795: InvalidEmailAddress in archive uploader <soyuz-upload> <Launchpad itself:Triaged> < https://launchpad.net/bugs/878795 >
<bigjools> jtv: dupe :)
<jtv> Oh, with what?
<bigjools> one that I filed earlier this week
 * bigjools looks
<bigjools> I wish +reportedbugs would default to newest first
<jtv> bigjools: find it, thanks.
<bigjools> bug 876348
<_mup_> Bug #876348: Soyuz upload processing bails out with an OOPS when encountering an invalid email address <oops> <soyuz-upload> <Launchpad itself:Triaged> < https://launchpad.net/bugs/876348 >
<jtv> Yeah.  Would you agree that it would make sense to treat this as an UploadError?
<bigjools> it;s not an oops for sure
<bigjools> need to reply to the uploader
<bigjools> the only time we need not reply is if the signature is invalid
<jtv> Does UploadError imply an oops?
<jtv> Oh, you mean to say the original bug description (mentioning oopses) wasn't entirely right?
<jtv> bigjools: it goes wrong in parseAddress, and its docstring says it will raise UploadError if parsing of the address fails for any reason.
<jtv> Will that DTRT?
<bigjools> jtv: the exception raising is correct, however the bit that generates oopses is crack
<jtv> bigjools: the exception raising that I'm proposing, or the exception raising that already happens?
<bigjools> there's a catch-all at the top level in uploadprocess.py somewhere, it's not very intelligent
<bigjools> already happens
<jtv> bigjools: I guess it's probably good though to be a bit specific about which exceptions are survivable.  I just attached a branch that makes it do what the docstring says; just one possible idea.
<bigjools> jtv: specific is always good
 * bigjools â lunch
<deryck> Morning, everyone.
<jtv> hi deryck
<jtv> bigjools: if you agree with the idea I attached to bug 876348, please let me know and I can put it up for review.  Otherwise, I'll treat it as just a suggestion.
<_mup_> Bug #876348: Soyuz upload processing bails out with an OOPS when encountering an invalid email address <oops> <soyuz-upload> <Launchpad itself:Triaged> < https://launchpad.net/bugs/876348 >
<bigjools> jtv: replied
<jtv> Thanks
<bigjools> jtv: alteratively to what I said, use reject("msg")
<flacoste> hi danhg!
<flacoste> welcome aboard!
* flacoste changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | Welcome danhg | On call reviewer: rvba* (allenap) | Critical bugtasks: 266
<jtv> bigjools: I guess I made a little thinko: âthis email address is invalid â there is nobody we can notify.â
<bigjools> jtv: wrong :)
<jtv> Evidently.
<bigjools> jtv: actually EarlyReturn is not ideal thinking about it
<bigjools> we just need to add a rejection and let processing continue so it collects all the errors
<bigjools> I really detest the upload processor
* jcsackett changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | Welcome danhg | On call reviewer: rvba* (allenap), jcsackett | Critical bugtasks: 266
<jtv> bigjools: I guess it's two problems: one, âmalformed email address and policy that says to create the userâ does something very different from âunknown email address and policy that says not to create the userâ or âmalformed email address and policy that says not to create the user.â  Two, it'd be nice to have more robust handling of these error conditions.
<jtv> Does that make sense?
<rvba> Hey jcsackett!
<jcsackett> morning, rvba. :-)
<bigjools> jtv: no
<bigjools> but then I am trying to do 2 things at once here
<jtv> OK â it's late here anyway, so we can leave this to ferment in the hindbrain for now.
<jcsackett> rvba: looking at your review right now, is there a test or anything that demonstrates using repr resolves the problem?
<gmb> rvba, would you be so kind as to review https://code.launchpad.net/~gmb/launchpad/weirdness-bug-867529/+merge/79972?
<rvba> gmb: sure.
<gmb> rvba: Thanks.
<flacoste> gmb, gary_poster: where is our lp  -> leankit integration code lives?
<gary_poster> flacoste on call.  danilo ^^
<gmb> flacoste: http://launchpad.net/lp2kanban IIRC. Unless you mean, where does it run, in which case danilos  would know.
<rvba> gmb: weird bug, I cannot reproduce it either ... the display does not seem to use floating elements except for "See full activity log
<rvba> "
<gmb> rvba: I know. It's weird. I can only reproduce it in Firefox and then only occasionally.
<rvba> The screenshot is indeed of a firefox window.
<rvba> I /think/ (by the looks of the tabs we barely see)
 * rvba hits F5
<rvba> again and again
<rvba> gmb: I've seen it!
<gmb> Heh
<rvba> But now it's fine again .... it was simply the browser 'lagging'.
<rvba> gmb: maybe your fix will force the browser to update the display more quickly.
<gmb> rvba: That's what I'm hoping. It seems to be some kind of lag in Firefox's rendering - it eventually sorts itself out.
<rvba> gmb: did you see it be 'broken' and stay that way?
<gmb> rvba: I saw it be broken for a short while, but scrolling up and down fixes it.
<gmb> well, in my experience.
<rvba> gmb: ok, same thing here.
<gmb> This is like trying to nail jelly to a wall.
<rvba> hehe
<rvba> jcsackett: hey, my official mentor is off today so maybe you will be ok to double check this tiny mp https://code.launchpad.net/~gmb/launchpad/weirdness-bug-867529/+merge/79972 ?
<jcsackett> sure, rvba.
<gmb> rvba: Merci beaucoup.
<rvba> jcsackett: btw I really wanted to make it up to you because you reviewed crazy branches last week and I was rather busy with a Critical bug ... so I'm the front line this week until I EOD ;)
<jcsackett> rvba: thanks. :-)
<rvba> gmb: welcome :)
<danilos> gmb, which reminded me, now you've got email about lp2kanban set-up on yellow@, so whoever wants to pick it up, go for it :) (it is running on my own server)
<flacoste> thanks gmb, i was interested in the code location
<gmb> danilos: Okay. I'm goingt to wait for someone other than me to pick it up, since running it on my server has been historically somewhat flaky.
<danilos> flacoste, if you need help in getting it going, you can still talk to me since I think I was the last person to shuffle the code around heavily :)
<flacoste> naba from product-strategy might be interested by it
<flacoste> danilos: ^^^
<flacoste> so i'll point him your way if he has any hard questions ;-)
<danilos> flacoste, sure, let them email me, but I hope config.ini and README should be good enough for starters as well
 * cjwatson grins at the blueprint person chooser saying "Pick me"
<jcsackett> gmb, rvba: this is the most irritating bug to try and see ever.
<gmb> Yup.
 * gary_poster goes for lunch
<jcsackett> gmb: i second rvba's approval, and have nothing to add.
<gmb> jcsackett: Thanks.
<rvba> jcsackett: Thanks for mentoring my review ;)
<jcsackett> you're quite welcome. :-)
<cjwatson> flacoste: would you be able to attend https://blueprints.launchpad.net/ubuntu/+spec/foundations-p-image-build-pipeline, or suggest somebody from LP who would be useful there?  It has moderate overspill into LP, although I think I've established that Ubuntu Engineering will have resources to actually do the work, so it's sort of more flagging if we're insane
<cjwatson> (One smallish piece of self-contained work; one rather larger job.)
<flacoste> cjwatson: sure, i'll attend
<flacoste> would probably make sense for bigjools to attend virtually as well
<cjwatson> flacoste: thanks, subscribed you
<cjwatson> ... and bigjools
 * bigjools prepares for blueprint spam
* rvba changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | Welcome danhg | On call reviewer: jcsackett | Critical bugtasks: 266
<sinzui> jcsackett, I appear to have been off IRC all day. DId you need me for anything?
<jcsackett> sinzui: nope, been fine, doing the review thing and chatting about mockups.
<sinzui> fab
<mrevell> Guten nacht alles
<abentley> deryck, allenap: I get "SSL connection has been closed unexpectedly" on Lucid, too.
<sinzui> jcsackett, do you believe this SQL is safe to run in production: https://pastebin.canonical.com/54678/
<jcsackett> sinzui: looks fine by me.
<sinzui> thank you. I may just fix the production issue today if the report has less than 50 teams
<deryck> abentley, dang.  Hopefully allenap's fix will work for you.
<abentley> deryck: cache-batches now respects the search parameters and has merged the latest stable.
<deryck> abentley, nice.  Thanks for the heads up/
<abentley> deryck: np
<abentley> deryck: it appears we're already running a fork of storm, so maybe allenap's changes should go in.
<deryck> abentley, yeah, maybe so.  no since waiting when we can roll our own. :)
<deryck> s/since/sense/
<abentley> deryck: wallyworld mentioned the idea of supporting infinite scrolling instead of batching.  I.e. when you're scrolling, it's loading batches via ajax.
<deryck> abentley, I think infinite scrolling makes sense when you have an infinite (or near infinite) list.  i.e. streams like twitter or news feedsâ¦..
<abentley> deryck: yes.
<deryck> abentley, I'm not sure it makes as much sense when the thing being viewed is a list or set of lists like bugs.  mentally it just fits better in a numbered, ordered set of lists.
<deryck> But I could certainly be wrong about this.  just my gut feeling about it.
<abentley> deryck: I dunno.  I think the main value of batching bugs is that you don't get overwhelmed by their numbers.  I find it a pain to click through to the next batch.
<deryck> yeah, maybe it doesn't matter.
<abentley> deryck: Anyhow, we don't have to use it for bugs, but any time we're doing batching, that's an alternative.
<deryck> abentley, certainly.
 * deryck reboots, brb
<lifeless> abentley: I think one bit of value in batch numbers is being able to say 'on page 5'... - but you could still do that with infinite scrolling
<lifeless> abentley: with a little care
<lifeless> abentley: (and its not a huge value)
<abentley> lifeless: Yeah, little pale-grey "page-break" markers or something.  They could have permalinks.
<abentley> deryck: Is there a guide to documenting our javascript methods/functions?
<deryck> abentley, at one time our style guide said the javadoc style that YUI uses.  No one really follows this, though.
<abentley> deryck: Okay.
<deryck> abentley, I personally don't like that style, so it's hard for me to endorse. :)  But I guess we should be consistent.
<deryck> abentley, we probably should raise it in some forum and get consensus on it.
<abentley> deryck: I'd like that.  I'm not very familiar with javadoc, so if we aren't really using it, I'll just document it as I see fit for now.
<deryck> abentley, that works.
<abentley> jcsackett_: could you please review https://code.launchpad.net/~abentley/launchpad/update-listings/+merge/79998?
<jcsackett_> certainly.
<abentley> jcsackett: thanks.  Also https://code.launchpad.net/~abentley/launchpad/cache-batches/+merge/80000 ?
<jcsackett> i'll grab it next, abentley. :-)
<abentley> jcsackett: It's a follow-on.
<jcsackett> dig.
<abentley> jcsackett: hey, look.  80000th merge proposal.
<jcsackett> oh, nice!
<abentley> thumper, rockstar: we just hit the 80000th merge proposal: https://code.launchpad.net/~abentley/launchpad/cache-batches/+merge/80000
<thumper> w00t
<thumper> abentley: do you have any bzr pipeline talks?
<thumper> anyone know of launchpad talks?
<abentley> thumper: No, I haven't given any talks on it.
<thumper> hmm...
<thumper> ok
<jcsackett> abentley: two questions on your first MP. 1) i am correct in assuming that when you call stringify on query in get_view_url, it handle the case of query being undefined (i.e. it converts undefined to an empty string?)
<jcsackett> 2) is the use of config.io_provider in update_listing just for testing purposes?
<abentley> jcsackett: I don't think the query string can be undefined, just an empty string.
<jcsackett> abentley: ah, fair. that does make sense. :-)
<abentley> jcsackett: it hasn't been an issue in local testing.
<abentley> jcsackett: config.io_provider is just for testing purposes.
<jcsackett> abentley: cool, thanks. r=me on the first MP then.
<abentley> jcsackett: thanks.
<jcsackett> abentley: and r=me on the second branch as well.
<abentley> jcsackett: thanks!
<jcsackett> you're welcome. :-)
<rockstar> abentley, congrats on 80000!
<StevenK> rockstar!!
<rockstar> StevenK!!
 * mwhudson knows what this is about, i think
<wgrant> lifeless: Your r12481 tests are spuriously broken in buildbot.
<wgrant> lifeless: I saw it break once in ec2 a couple of days back.
<wgrant> I can't work out why.
<wgrant> I can't even find the preloading that seems to be broken...
<lifeless> wgrant: win!
<lifeless> wgrant: :(
<wgrant> lifeless: Can you investigate that failure as a matter of urgency? It's broken both devel and db-devel builders now.
<wgrant> And I can't see where the preloading happens.
<lifeless> its been in since feb
<lifeless> so this is broken by some other change
<lifeless> wgrant: also I'm OTP
<wgrant> Yes, it's clearly broken by some other change.
* wallyworld_ changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | Welcome danhg | On call reviewer: wallyworld | Critical bugtasks: 266
#launchpad-dev 2011-10-21
<lifeless> wgrant: got a url to a test failure ?
<wgrant> https://lpbuildbot.canonical.com/builders/lucid_lp/builds/1484/steps/shell_6/logs/summary https://lpbuildbot.canonical.com/builders/lucid_db_lp/builds/1446/steps/shell_6/logs/summary
<wgrant> I first saw it in an ec2 run against devel r14174
<lifeless> it looks like a regression
<wgrant> In an untested JS-only branch of mine
<lifeless> the (158, 160, 'SQL-main-slave', 'SELECT Product.answers_usage, Product.blueprints_usage, Product.owner, Product.active, Product.autoupdate, Product.bug_reported_acknowledgement, Product.bug_reporting_guidelines, Product.bug_supervisor, Product.bugtracker, Product.date_next_suggest_packaging, Product.datecreated, Product.description, Product.development_focus, Product.displayname, Product.downloadurl, Product.driver, Product.enable_bug_expiration, 
<wgrant> But it is not reproducible locally, and a second ec2 run worked.
<wgrant> Yes.
<lifeless> bah
<lifeless> (244, 244, 'SQL-main-slave', 'SELECT Product.answers_usage, Product.blueprints_usage, Product.owner, Product.active, Product.autoupdate, Product.bug_reported_acknowledgement, Product.bug_reporting_guidelines, Product.bug_supervisor, Product.bugtracker, Product.date_next_suggest_packaging, Product.datecreated, Product.description, Product.development_focus, Product.displayname, Product.downloadurl, Product.driver, Product.enable_bug_expiration, Prod
<wgrant> That's clearly the regression.
<wgrant> I don't see how, though.
<lifeless> could be storm cache exhaustion ?
<lifeless> have you seen it happen locally?
<wgrant> No.
<wgrant> Never locally.
<wgrant> Locally it's way below the threshold.
<wgrant> By like 20
<wgrant> I had to reduce the limit to 20 or 30 or so to get it to fail locally.
<lifeless> that where X LIMIT 1 is odd
<lifeless> I have to pop out for a bit; doing a test run that generates backtraces on every query into the timeline would probably let this get diagnosed
<wgrant> Would it?
<wgrant> The revision that added the test speaks of testing for preloading of Product and Distribution.
<wgrant> Which suggests that there is preloading broken, not that there are extra product lookups.
<lifeless> that trace clearly shows the product being loaded twice
<lifeless> see the query starting at 159
<lifeless> 'product.id IN (...., 49, ....)'
<lifeless> and then the qyery at 244
<lifeless> 'product.id = 49 LIMIT 1'
<lifeless> same product
<lifeless> same store
<lifeless> wgrant: ^ I'm out for a bit sorry, but that smells like a storm cache issue to me; I think we should disable the silly size limits
<wgrant> Oh, so it does.
<wgrant> lifeless: You mean the custom dev limit of 100?
<wgrant> Whereas on prod it's 10000?
 * wgrant fixes qa-tagger to not use edge.
<lifeless> wgrant: yes
<lifeless> wgrant: dev should match prod, because devs want to reproduce behaviour on prod
<lifeless> separately, given we toss the entire cache away per request, we should stop using a fancy cache and just use a dict
<wgrant> lifeless: I imagine the motivation is similar to the batch size of 8.
<lifeless> batch size of 8 makes more sense
<lifeless> (but still is an issue for e.g. screen layout testing
<wgrant> Right.
<lifeless> anyhow, focus - the cache size is I think the issue here
<wgrant> So, fixing the 100 to 10000 is a matter of deleting a line from the dev config.
<wgrant> But do we want to identify the regression?
<lifeless> wow, wiredoo, wow
<lifeless> wgrant: well, if it is the cache, its not a regression
<lifeless> wgrant: prod won't be late-evaluating
<wgrant> Indeed, but something extra is getting into the cache.
<lifeless> test threads don't dump the cache
<lifeless> publication.py like 759
<lifeless> is the loop
<lifeless> and this is the kicker:
<lifeless>             if thread_name != 'MainThread' or name.endswith('-slave'):
<lifeless>                 store.reset()
<lifeless> not thread 0 or any slave, reset
<wgrant> Heh
<lifeless> and yes, thats production code
<lifeless> so, interestingly, this shold be getting dumped IFF the test runs through publication
<lifeless> ah, happens on endRequest, not begin
<mwhudson> that code did my head in the other week
<lifeless> mwhudson: of course, it exists to blow your mind
<lifeless> in fact, it should reset test stores
<lifeless> it only doesn't because the test fallout would be ginormous
<lifeless> aieee
<lifeless> http://wiki.postgresql.org/wiki/Loose_indexscan
<wgrant> Hah
<jtv> StevenK: did you come up with the term âblamerâ in the soyuz notification module?  I'm trying to make sense of it but not doing very well.
<wgrant> jtv: I believe it's more of the blamee.
<wgrant> The uploader or the copier, in general.
<jtv> Yes, I've been thinking it must be the blamee.  The code seems to assume pretty strongly that it's the signer.
<jtv> My clever scheme is to figure out whether a native English speaker came up with it.  If yes, I'm more willing to assume that there's a reason that I'm not seeing.
<wgrant> StevenK implemented it, but I don't recall who came up with it.
<wgrant> 'twasn'
<jtv> I'm separating the "figure out whom to email" code from the "compose and send" code.  There's only a few call sites, but one of them is in the package copier which I think is a very different use case.
<wgrant> t me.
<wgrant> The package copier is meant to use the person who requested the copy as the blamer.
<wgrant> I believe they receive notifications of the copy, and are used as the sender for the announcement message.
<jtv> Ironically, the maintainer and the person credited with the latest change are emailed _if_ the caller passes a "blamer."
<wgrant> That may be related to autosyncs.
<jtv> I'd sort of expect that to happen if you _don't_ pass one, as a fallback.
<wgrant> Which are unsigned, and should email nobody.
<jtv> Now, who should be notified in the case of a copy?
<wgrant> The person who requested it and an undefined set of other people.
<jtv> Just the person requesting it?
<wgrant> Except in the case of autosyncs, in which case an undefined set of people.
<jtv> I am asking you to help define that set.
<wgrant> That's one of the great mysteries of the universe, which we've never had a good answer to :/ But probably nobody, in the general case.
<wgrant> Build notifications are even less well defined :/
<spm> <wgrant> That's one of the great mysteries of the universe, <== the answer is 42. just sayin.
<jtv> Then how about we notify only the requester, then sit back and see who else wants to be in the loop?
<wgrant> Well, for one thing we don't know the requester for build notifications.
<wgrant> Which I believe are the problem here.
<jtv> Damn.  I have to go take of stuff.
<jtv> Especially drinking water.
<jtv> And seeing how my family is â their area got hit hard overnight and I think they lost telephone
* wallyworld_ changed the topic of #launchpad-dev to: https://dev.launchpad.net/ | Welcome danhg | On call reviewer: - | Critical bugtasks: 266
<nigelb> spm: heh
<adeuring> good morning
<rvba> Morning adeuring.
<adeuring> hi rvba!
<bigjools> morning
<mrevell> Good morning!
 * nigelb hugs wgrant 
<jtv> danilos: have fun in your new team!
<danilos> jtv, thanks :)
<jtv> Also, don't try sneaking out without finishing your Q/A because I would be able to get to it until Tuesday!
<StevenK> jml: Hai. I'd like to do a cherry pick release of Twisted so I can fix a Critical, but I can't figure out how you did it. The fix is already upstream, so perhaps they're close to a release?
<jml> StevenK: bigjools did one recently, iirc.
<jml> StevenK: and I could have sworn I wrote up the process a couple of years ago into a mailing list message.
<jml> StevenK: I have to be afk now, otherwise I'd help more.
<jml> StevenK: Twisted's looking for volunteers for an 11.1 release.
<bigjools> StevenK: I can  help
<StevenK> bigjools: I've been distracted, but I want to head to bed soon, so would you mind sending me a mail with follow-the-bouncing-ball?
<danhg> Hey gmb, thanks for the email
<danhg> gmb:
<danhg> hmmm...
<gmb> Hi danhg :)
<danhg> ah, it works!
<danhg> It didn't 'look' like it had
<flacoste> bigjools: so the fix for bug 34086 is live?
<_mup_> Bug #34086: removal of arch-all packages while there are arch-specific packages dependent on it results in uninstallable binaries <bad-commit-14159> <escalated> <feature> <lp-soyuz> <qa-ok> <rls-mgr-p-tracking> <soyuz-publish> <Launchpad itself:Fix Committed by julian-edwards> <Ubuntu:Invalid> < https://launchpad.net/bugs/34086 >
<bigjools> flacoste: no, it's still fix-committed
<flacoste> bigjools: 14171 was deployed earlier today
<bigjools> I didn't want to push a release out to cocoplum on a Friday
<bigjools> hmmm
<bigjools> flacoste: cocoplum is not in the FDT set
<flacoste> you mean the NDT set
<flacoste> but ok
<flacoste> thanks
<bigjools> I do!
<bigjools> flacoste: I told infinity it would most likely be on Monday anyway
<flacoste> great
<SpamapS> assuming this was a temporary problem: https://launchpadlibrarian.net/83224566/buildlog.txt.gz
<SpamapS> bzr: ERROR: xmlrpc protocol error connecting to https://xmlrpc.edge.launchpad.net/bazaar/: 502 Bad Gateway
<abentley> allenap: Where are we at with this storm bug you've fixed?
<rvba> abentley: allenap is off today.
<abentley> deryck: ^^
<deryck> ah bummer bummer
<deryck> bigjools, do you know where allenap was at with the Storm bug?
<deryck> hmmm, nearly call time for me. nm.
<bigjools> deryck: trying to get it reviewed - there was some back and forth.  I told him to just throw it at download-cache for now to unblock us but I dunno if he finished that
<bigjools> you could grab his branchj
<deryck> bigjools, ah great.  That's what I was going to suggest.
<deryck> abentley, can you grab his branch and roll up a tar ball for download-cache?
<abentley> bigjools: This is lp:~allenap/storm/oneiric-admin-shutdown-bug-871596 right?
<bigjools> yes
<abentley> deryck: It's tricky, because allenap's branch is against a newer version of storm than our current, so merging it brings in those changes.
<deryck> abentley, ah, right.  And we need to maintain our fork as well, for the with stuff.
<abentley> deryck: Right.  So I guess I can merge current storm into our fork, then merge allenap's stuff.
<deryck> abentley, yeah, that sounds right to me.
<deryck> mrevell, so I see the problem now in the image you sent me.  Was hard to spot at first.  The number and title part are further to the right from the status and importance than what I see.
<deryck> mrevell, sound right?
<mrevell> deryck, Yeah, that's it
<deryck> mrevell, gotcha.  that's easy.  I just gave each section a % width.  Easy enough not to do that.  The middle one needs to expand but not the first or last.
<deryck> mrevell, there are actually sections there, even though it's not a table.
<mrevell> Yeah, I like them, they solve the scannability problem.
<deryck> mrevell, yeah, I think so, too.
<sinzui> gary_poster, benji, bac: help! launchpadlib does not know who I am. I switched computers, and uses deja-dup to restore my user data to the new machine. I have deauthorised all my scripts in lp, deleted .launchpadlib cache, and deleted all the launchpad.net password in my keyring
<benji> sinzui: what error are you getting?  are you getting the authorize-this-script web page?
<sinzui> benji, I an never prompted to re-authorise. The script runs then when I need lp.edit or lp.moderate, I get permission denied
<benji> sinzui: did you delete all of .launchpadlib or just the cache directory?
<sinzui> there is nothing in the directory now
<benji> k
<sinzui> I logged and and back in thinking the keyring in the session had cached
<benji> sinzui: is your script using login_with or login?
<sinzui> well good question
<sinzui> benji, lpscript uses Launchpad.login_with(app_name, system_name, version='devel')
 * sinzui tries his test script
<sinzui> benji, my test script fails in the same way with Launchpad.login_with(
<sinzui>         'test', service_root='https://api.launchpad.net', version='devel'
<sinzui> benji, I did deauthorise all scripts since they were all tied to my previous computer
<sinzui> I really expect a prompt to reauthorise
<benji> sinzui: I suspect there's a deauthorized credential lurking in your keyring, that's the only scenario I can see at the moment that would cause these symptoms
<sinzui> yeah, that is why I got brutal and removed all Lp passwords.
<sinzui> benji, since these were system-wide for my computer, is Lp the real authorisor? could ubuntu/login.ubuntu.com be the actual authorising party?
<benji> sinzui: I don't understand the question, but regardless, I think the answer is "yes".
<sinzui> oh, I could rename the script which might cause an authorisation prompt
<benji> I don't think that'll work.  At least, I'd be dissapointed if it did.  The intent was that once a machine is authorized, all scripts are authorized.
<sinzui> does lp-lib piggy back on firefox? I think I same an authorisation request a new days ago that opened firefox, but that is not my preferred browser
<benji> sinzui: it uses python's webbrowser module (which is awful)
<sinzui> I renamed my moz dir, but that did not help
<flacoste> danilos: since it's your last LP day, thanks for everything and good luck with Linaro!
<bigjools> danilos: ditto from me
<flacoste> sinzui: did you cleaned-up seahorse?
<sinzui> I have been, but I believe I have a lot more to purge
<danilos> flacoste, bigjools: thanks both, it was a joyful ride! :)
<flacoste> sinzui: lplib tokens are very descriptively called 'network password' :-(
<sinzui> oh!
<sinzui> !!
<sinzui> there are a hell pf a lot of those
<flacoste> if you click on them, you should see that it relates to one of the LP hosts
<flacoste> danilos: and i'll have my illuminati cards in Orlando to kick your butt for leaving ;-)
<sinzui> interesting
<danilos> flacoste, hehe, you know that whoever is a rookie at the game will win (ok, or Martin :)
<flacoste> danilos: and of course buy you a beer afterwards to thank you for all of the good things you did
<danilos> flacoste, thanks, I owe you plenty of beers myself!
<sinzui> flacoste, deleting them did indeed prompt to reauthorise. But I still got the lp.moderate auth error in my script
<sinzui> I was also shown firefox, which is not my browser
<flacoste> i blame ubuntu for the later part :-)
<flacoste> sinzui: what's the lp.moderate error?
<sinzui> 401 (<Product at 0x145c8950>, 'project_reviewed', 'launchpad.Moderate')
<sinzui> flacoste, I do not think the script knows I am sinzui in ~registry
<sinzui> !
<sinzui> flacoste, I see the problem
<sinzui> flacoste, it is firefox
<sinzui> flacoste, It is not my browser. It is my daughter. She is only authorised to to choose television programs in my house
<flacoste> !
<flacoste> sinzui: you are on your daughter computer?
<sinzui> No, She does qa for me when I need a normal user
<sinzui> hence my remark that it is not my browser.
<sinzui> This is an easy fix in the browser, but I want launchpadlib to use the browser my computer is configured to use
<flacoste> sinzui: how did you configure the browser?
<flacoste> sinzui: iirc, we are using the python browser which should use either x-www-browser or www-browser depening on the DISPLAY env
<flacoste> does x-www-browser invokes your correct browser?
<sinzui> From Ubuntu's menu, System Settings > System Info, Default Applications
<sinzui> That controls the web browser shown in the dash
<sinzui> and it sets the xdg data
<flacoste> right
<flacoste> but that doesn't integrate with the Debian alternatives system
<flacoste> update-alternatives --display x-www-browser
<flacoste> what does that show?
<flacoste> use that to set your browser and you'll be fine
<flacoste> i'd advocate that changing default applications that doesn't update the alternatives is not a launchpadlib bug
<sinzui> flacoste, thank you very much. I reconfigured x-www-browser and chose the auto option and the system default browser now always the chosen browser
<flacoste> great!
<flacoste> now food
<sinzui> This looks like a bug in  x-www-browser where it chooses a specific
<jml> StevenK: http://people.canonical.com/~therve/Twisted/11.1.0pre1/ ; http://twistedmatrix.com/pipermail/twisted-python/2011-October/024686.html
 * bigjools gets the buildd-manager to cancel a build mid-build <evil cackle>
<abentley> deryck: how's it going?
<deryck> abentley, not too bad.  a little slower than I'd like.  But making progress.
<abentley> deryck: just a heads-up, I've reached the point where I see that what I have is better represented as an object with several methods, rather than a collection of functions.  So update_listing will become ListingNavigator.change_ordering.
<deryck> abentley, ah, interesting.
<deryck> abentley, presumably this still just needs the order by string from the order by widget to update the page?
<abentley> deryck: right.  It will allow me to write unit tests that don't have to mutate window.*
<deryck> abentley, ok, nice.
<abentley> deryck: Because that connection will be made by a factory method.
<sinzui> jcsackett, do you have time to mumble?
<jcsackett> sinzui: sure.
<sinzui> jcsackett, part1 https://help.ubuntu.com/community/MacBookAir4-2
<sinzui> jcsackett, part2 http://sourceforge.net/projects/gptfdisk/
<sinzui> jcsackett, part3 http://ubuntuforums.org/showpost.php?p=11215214&postcount=185
<flacoste> critical analysis draft finally completed!
<flacoste> sent to the list!
<flacoste> and i'm done for the day
<flacoste> enjoy your week-end
<wallyworld> sinzui: you still around?
<cjwatson> flacoste: illuminati> subscribe!  haven't played that in years
#launchpad-dev 2011-10-22
<lifeless> the tl;dr is interesting itself
<wgrant> It's got a typo, though.
<wgrant> Says 68% are legacy, but it's really 58%.
#launchpad-dev 2011-10-23
<lifeless> moin
<jelmer> g'morning lifeless
<lifeless> http://codepad.org/ADlGOI2N
<lifeless> flacoste: when I open your google docs document, google gives me a new empty doc instead
<lifeless> flacoste: I am logged in as the correct @canonical user
<wgrant> lifeless: I get a new empty doc when I'm not logged in as a canonical.com user...
<lifeless> wgrant: yes, but actually I can access the doc, I think the url is slightly wonky or something
<wgrant> Ah. Well, I don't know my canonical.com password. I should probably sort that out.
 * jelmer waves to lifeless, wgrant 
<wgrant> Morning jelmer.
<lifeless> wgrant: played with amqp2disk yet ?
<wgrant> lifeless: Not as yet.
<wgrant> lifeless: Where did I see the description of how to run it? MP or email?
<lifeless> email
<wgrant> That's what I thought, but I can't see it...
<lifeless> upcoming chansges to oopses
<wgrant> Oh, thanks, Thunderbird.
<wgrant> Your reply on the 19th is hiding between the two replies from the 5th.
<lifeless> \o/
<wgrant> Is the oops-tools branch merged yet?
<wgrant> Yes.
<wgrant> Ugly OOPS IDs, but damn, it works.
<wgrant> Very, very nice.
<wgrant> Is it on production yet? :)
<lifeless> not yet
<lifeless> I have to write the re-queue code for datedir-repo
<lifeless> and get my useoops branch passing all tests
<lifeless> another week at least I think
<wgrant> 2252 line diff on the LP branch...
<lifeless> nearly all of which is dealing with debt
<lifeless> yes, its a doozy.
<wgrant> Hmm, so the oops refix is not included in the metadata?
<wgrant> prefix
<wgrant> Ah, it is.
<wgrant> Just not shown in oops-tools yet.
<poolie> hi all
<wgrant> Hi poolie.
<lifeless> wgrant: reporter
<lifeless> wgrant: and yeah, its just not in the template
<poolie> huwshimi, hi, can you read https://code.launchpad.net/~mbp/launchpad/bug-width/+merge/80161 for me?
<huwshimi> poolie: sure
<poolie> lifeless, i looked into getting psql to show the query plan over the weekend
<poolie> the good news is that you can turn on logging such that it does send it
<poolie> and one would hope that increasing logging doesn't change behaviour beyond just the overhead of generating and sending the text
<poolie> the bad news is that it's in a barely readable very detailed form
#launchpad-dev 2012-10-15
<wgrant> https://oops.canonical.com/oops.py/?oopsid=OOPS-493eb7067f568cde87767956e4f8ce24 is impressive
<wgrant> 1061 repetitions of a query to check if the current user is affected by a bug
<StevenK> It wants to be REALLY sure
<wgrant>         # Loop over dupes.
<wgrant>         for dupe in self.duplicates:
<wgrant>             if dupe._getAffectedUser(user) is not None:
<wgrant>                 dupe.markUserAffected(user, affected)
<StevenK> Oh
<wgrant> That seems pretty pointless...
<wgrant> I guess it might be helpful to unaffect dupes when unaffecting the master
<StevenK> wgrant: I've fixed the test failures in destroy-old-bugactivity, I'll be landing it shortly.
<wgrant> StevenK: Great
<wgrant> Now go away :)
<StevenK> Haha
<nigelb> what? haha.
<StevenK> nigelb: I'm on holidays this week.
<StevenK> So wgrant is telling me to stop caring about work from last week. :-)
<nigelb> StevenK: Aha. yes, go away.
<StevenK> But it hasn't landed yet!
<nigelb> Where's a freenode staff when you need one.
<StevenK> Oh, don't even joke about that.
<nigelb> Waitaminute
<StevenK> They've k-lined me twice 'by accident'
<nigelb> I'm sure your SO can arrange that ;)
<StevenK> Hah
<wallyworld_> wgrant: does it make sense that when auto confirming a bug, any duplicates of that bug should be confirmed also, without the need for affected user checks against each individual dup?
<wgrant> wallyworld_: No. In fact it's a bug (probably already reported) that duplicates get autoconfirmed at all
<wgrant> wallyworld_: Launchpad supports autoconfirmation only provisionally and begrudgingly; don't feel like you have to go too far out of your way to make it smooth
<wallyworld_> wgrant: ok, so if i do a bulk update for affects user, i can ditch the need for any recursive call processing
<wgrant> Right
<wallyworld_> what about heat?
<wallyworld_> can dupe heat be set to master heat?
<wgrant> No
<wallyworld_> i may need to to a stored proc which bulk updates heat
<wallyworld_> instead of just one bug id at a time
<wgrant> Nah
<wgrant> See the garbo job
<wallyworld_> oh, ok. didn't realise there was one
<wgrant> You can do eg. some_result_set_of_bugs.set(heat=SQL('calculate_bug_heat(Bug.id)')
<wgrant> And it'll construct a single UPDATE
<wallyworld_> ok, nice
<wallyworld_> thanks
<wgrant> Calling the function a thousand times isn't a problem, as (particularly after I last took an axe to it) it's quite fast. The problem is making a bazillion roundtrip queries.
<wallyworld_> yes
<wgrant> A quick fix to this also wouldn't go near the heat issue at all
<wgrant> Since it's very unlikely that the user has clicked the affectsme button on more than a couple of dupes
<wgrant> So it won't be triggered more than a couple of times a request
<wgrant> But batchifying more code is always good, so why not do the comprehensive fix, I guess :)
<wallyworld_> well, i'm lookin at ditching the recursive call and bulk updating all dupes in the tope level affectsMe all
<wgrant> Note that it's not quite like that
<wgrant> It sets the affectsme status of any dupes that already have an affectsme status
<wallyworld_> sure, i'm handling that
<wgrant> Great
<wgrant> Just confirming
<wallyworld_> np :-)
<wallyworld_> but it means ditching the confirm of dupes, which you say above is ok
<wallyworld_> and i'll do a single update for heat
<wgrant> So yeah, I'd just find all the dupes that have a BugAffectsPerson for the user, set them, set their users_affected_count to AutoReload (or amybe not, it might mean we don't care), and then bulk update the heat
<wallyworld_> an invalidate and flush will do the same thing?
<wallyworld_> as setting to AutoReload
<wgrant> Yeah, but it's excessive
<wgrant> So it's nice to avoid it
<wallyworld_> it's used already
<wgrant> Yes
<wallyworld_> so i'll see if i can ditch it maybe
<wgrant> It's already excessive!
<wgrant> Right
<wallyworld_> not sure
<wgrant> To some extent we really don't care that the dupe counts are off by one
<wgrant> (for the current transaction)
<wallyworld_> ok, i may see what ec2 comes back with perhaps
<wgrant> But if we do care, then just set the affected columns to AutoReload, as it's much cheaper than invalidating the entire object
<wallyworld_> not sure if this will help the other bug 925937
<_mup_> Bug #925937: Does this bug affect you: Timeout error popup <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/925937 >
<wallyworld_> maybe
<wallyworld_> since auto confirm for dupes is not going to be dome now
<wgrant> wallyworld_: No, sadly
<wgrant> wallyworld_: That's autoconfirming the main bug
<wallyworld_> oh well
<nigelb> Oh. Now I realize wallyworld_ is Ian Booth.
<nigelb> Throws me off every time I read your real name :)
<wallyworld_> i like to be different
<nigelb> Heh
<wallyworld_> ianb would be such a boring nick
<nigelb> boothi sounds more fun ;)
<wallyworld_> there's a few other words that suit better, but they're not for a family channel
<nigelb> hahaha
<wallyworld_> right, off to pick up kid from school
<spm> wallyworld_: you could go for ixb
<wallyworld_> spm: i could, but it's too late now
<spm> it's never too late
<spm> you could have ixb_formally_known_as_wallyworld
<wgrant> wallyworld's his legal name? :)
<wallyworld_> i'm too lazy to change it
<wgrant> Hmm
<wgrant> SQLObjectResultSet.__bool__ irks me
<wgrant> Er, __nonzero__
<wgrant> That's the one
<wgrant> Still evil
<StevenK> SQLObject needs to be destroyed
<nigelb> Your internet access needs to be destroyed. Until after vacations.
<wgrant> Yep
<StevenK> But I needs it for Steam!
<StevenK> Anyway, I lose Internets on Wednesday
<nigelb> You need to take a vacation on some island with poor internet connectivity. But, you already live in Australia.
<StevenK> So yeah, mission accomplished?
<adeuring> good morning
<czajkowski> purple you guys were busy yesterday
<czajkowski> *today
<StevenK> We're always busy
<czajkowski> my inbox says very busy :)
* benji changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: benji | Firefighting: - | Critical bugs: ~260
<olli> gm Alice
<olli> seems you are well informed, hadn't heard about Sam until I saw your mail
<olli> wrong channel
<abentley> rick_h_: On https://qastaging.launchpad.net/bzrtools/+edit I see a green flash when I change information type.  I believe that's supposed to mean that the change has been applied to our server.
<rick_h_> abentley: looking
<rick_h_> abentley: ah, gotcha. will file a bug/card
<abentley> rick_h_: Cool.
<rick_h_> do we have a tag we're using for privateproject work?
<rick_h_> ah, see private-projects
<rick_h_> abentley: I also looked into that bug from friday and filed that bug along side this one
<rick_h_> where without JS you saw all of the radio buttons for all info types vs the 3 valid options
<abentley> rick_h_: Thanks.  I experienced that bug with JS enabled, though.
<rick_h_> abentley: hmm, well I figured there mus thave been a JS error in your local env since I didn't see it on qastaging but you mentioned you had it with js disabled
<rick_h_> abentley: I guess if you can repeat it with JS enabled let me know and we can look if something in the JS to build the widget failed since you shouldn't see radio buttons
<abentley> rick_h_: We should also fix JS errors that affect local dev environments.
<rick_h_> abentley: agree, but I don't know what your JS error was which is why I'm saying if you hit it please let me know more details
<abentley> rick_h_: Reproduced.
<rick_h_> abentley: awesome
<abentley> rick_h_: The console shows Error: extend failed, verify dependencies
<rick_h_> ugh...ok so this is on a project/+edit you're getting this in your local dev env?
<abentley> rick_h_: Yes, https://launchpad.dev/firefox/+edit
<rick_h_> abentley: ok, loading up a clean trunk dev env to peek at
<abentley> rick_h_: I am using firefox, btw.
<rick_h_> abentley: ok, checking. Not seeing it in chrome
<abentley> rick_h_: I'm seeing the radio buttons in chrome.
<rick_h_> abentley: ok, trying to see if I can replicate. make clean/make run and such.
<rick_h_> abentley: the error you hit is generally because there's a missing new SomeObject when creating an instance
<rick_h_> or a missing dep in the list of requires for some module
<abentley> rick_h_: This is on line 10 of https://launchpad.dev/+combo/?yui/yui/yui-min.js&lp/meta.js&yui/loader/loader-min.js
<rick_h_> abentley: does your line 10 look anything like: https://pastebin.canonical.com/76544/
<rick_h_> abentley: have you updated your feature flags locally? Are you running yui 3.5?
<rick_h_> I wonder if it's broken for you since I used Y.View which is yui 3.5 and you might still be set in your dev env for 3.3
<rick_h_> abentley: that would explain why it works in qastaging/production
<abentley> rick_h_: No, it looks like https://pastebin.canonical.com/76545/
<rick_h_> abentley: yea, I bet you're running the old YUI locally still via either lack of make clean/clean-buildout or via forced feature flag setting
<abentley> rick_h_: One should never have to run make clean.  I can confirm I'm still running 3.3
<rick_h_> abentley: ok, I guess that's a failing on my end then. I tend to run make clean whenever I bzr switch so didn't realize this
<abentley> rick_h_: Yeah, I only run make clean as a last resort.
<rick_h_> abentley: so the make commands around JS build check for the existing of the build/js/yui and build/js/lp and they existed
<abentley> Not when I switch or anything.
<rick_h_> and only rebuild around them missing so I guess we can look at a more robust or always rebuilding them
<abentley> rick_h_: So I guess 3.3 is obsolete now, and we should remove support for it.
<rick_h_> abentley: yes, it's on the JS clean up todo. SteveK talked about working on some of it, removing some of the jsbuild bits and such
<abentley> rick_h_: Now that I've "make clean"ed, I'm seeing the JS widget, not the radio buttons.
<rick_h_> granduating combo build up to full status and cleaning up the yui-deps.py
<rick_h_> abentley: cool, good to hear
<rick_h_> abentley: I'm working on trying to get InformationType to show up for new blueprints UI locally. I've enabled the feature flag, there's nothing in +configure-blueprints about adjusting the information types, but I don't get the field in the list as I do on qastaging/production.
<rick_h_> abentley: is there another step I'm missing to enable the field?
<abentley> rick_h_: OTP
<rick_h_> abentley: rgr
<sinzui> jcsackett, do you have time to hangout?
<abentley> rick_h_: Have you enabled the blueprint-specific flag?
<abentley> rick_h_: i.e. blueprints.information_type.enabled ?
<rick_h_> abentley: yes
<rick_h_> blueprints.information_type.enabled	default	1	on
<abentley> rick_h_: Have you enabled private blueprints under +sharing?
<rick_h_> abentley: ah I think I see. Since this new project is public and doesn't have a commercial subscription it can't immediately set a non-public blueprint?
<abentley> rick_h_: Indirectly.  If your specification sharing policy is "Public", you can't set non-public, so you don't get the field.  Non-public policies should only be available if you have a commercial subscription.
<rick_h_> abentley: got it
<rick_h_> abentley: yea, had to get a commercial subscription to get the edit UI for the +sharing to change the policy
<rick_h_> abentley: thanks
<abentley> rick_h_: np
<adeuring> abentley, deryck, rick_h_: sorry, I just saw another EC2 test failure that I missed earlier today; fixed now, but the new EC2 test is starting just now
<deryck> adeuring, no worries.  thanks for the heads-up.
<rick_h_> adeuring: cool thanks. Yea I'm just holding off until your stuff hits pqm and I'll merge/submit to ec2 EOD/tomorrow morning
<adeuring> rick_h_: thanks. I'll ping you once my stuff lands.
<rick_h_> adeuring: ty much
<rick_h_> sinzui: ping
<sinzui> hi rick_h_
<jml> which squad is doing maintenance at the moment?
<czajkowski> jml: purple
<jml> czajkowski: ta
<czajkowski> jml: why.. what have you broken :)
<jml> czajkowski: time.
<lifeless> \o/
<czajkowski> lifeless: :D
<sinzui> jml, you broke time?
<jml> sinzui: time is out of joint.
<rick_h_> sinzui: wanted to check on if you filed a bug for the JS inline stuff around https://bugs.launchpad.net/launchpad/+bug/1052551 The inline stuff was in the NullChoice side of the widget so wasn't sure if the bug was valid and wanted to check it out
<_mup_> Bug #1052551: Specification sharing policy is not honored by the edit information_type widget <css> <information-type> <javascript> <private-projects> <specifications> <ui> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1052551 >
<sinzui> rick_h_, why can't we change the info type of this blueprint. We own it and the project allows public :https://blueprints.qastaging.launchpad.net/launchpad-help-moin-theme/+spec/testing-proprietary-1
<rick_h_> sinzui: yea, there's bug in the JS/css setup in the .pt
<rick_h_> I've got that fixed, but was looking at the comment you made about the choiceedit.js doing manual style setup on 'inline'
<rick_h_> sinzui: https://code.launchpad.net/~rharding/launchpad/info_portlet_1052551/+merge/129706 for the .pt and setup issues I'm fixing currently
<sinzui> ah, I reported that as a separate issue.
<rick_h_> sinzui: right, so tring to find that separate issue atm to look at it and my search-fu is failing me
<sinzui> rick_h_,  this is definitely separate: https://bugs.launchpad.net/launchpad/+bug/1064555
<_mup_> Bug #1064555: choicedit assumes icons are inline and tampers with the style.display <disclosure> <information-type> <javascript> <private-projects> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1064555 >
<rick_h_> wondered if you had it handy and if you had realized the setStule stuff was in the NullChoice widget and if it still applies
<sinzui> rick_h_, all feature bugs should be tagged private-projects, that will narrow the search
<rick_h_> sinzui: yea, so the cause was manually adding display:inline to the .pt file
<rick_h_> not the JS in choiceedit
<rick_h_> ok cool, I'll reply to this one as well as I wrap things up here
<rick_h_> sinzui: note line #47 of my diff in my MP I'm working on
<rick_h_> sinzui: cool thanks for the link that helps
<sinzui> rick_h_, I would exepct the choicesource bug to appear more frequently. It may affect your feature, but it definitely pre-exists a lot of feature work by about 3 years
<rick_h_> sinzui: right, but in this case it wasn't a bug in choicesource, but in the JS manually run in specification-index.pt and boostrap html in blueprint-portlet-privacy.pt
<sinzui> rick_h_, oh, so maybe the error I saw about manipulating style.display is actually from the template...which may mean we have dead code in choice source
<rick_h_> sinzui: right, the code in choicesource you grepped pertains to a NullChoiceSource widget
<rick_h_> so maybe there's still a bug there, but that's why we don't see it much as it's not used as often
<sinzui> rick_h_, yes, I think you have fixed the real bug I experienced
<rick_h_> sinzui: ok cool. Thanks for the link to the other bug. Want to make sure I catch all the parts here
<sinzui> rick_h_, if you can edit, and after the edit the text "Edit" does not appear, you have fixed my use case. I will remove the tags from the other bug. I guess it is tech-debt
<rick_h_> sinzui: yea, the only thing now is that edit always shows even when there's only one value "Public" so I'll probably fix this bug but then file a bug about hiding that all together when there's only the one choice
<sinzui> ah
<sinzui> rick_h_, toggleClass('hidden') was the intent then...and choice source does not do it.
<rick_h_> sinzui: gotcha. Yea, so I'll file a new bug about that then with this as the specific use case
<sinzui> rick_h_, so maybe the choice source should know to hide EDITICON when the length of choices are 1?
<sinzui> How do proprietary only bugs and branch portlets solve this?
<rick_h_> sinzui: yea, that's what I would expect.
<rick_h_> sinzui: I'm going to check out how branches solve this. It's not jumping out at me and I want to get this fix in asap.
<sinzui> rick_h_, This is not your problem...or at least this is not about blueprints.
<sinzui> rick_h_, https://code.qastaging.launchpad.net/~launchpad/launchpad-help-moin-theme/moin_launchpad shows a choice of 1
<rick_h_> sinzui: agreed which is why it'll be a new bug for now. I've got to wrap this up and move onto some more private project stuff
<rick_h_> sinzui: ah, good to know then. Thanks
<sinzui> okay
<rick_h_> benji: https://code.launchpad.net/~rharding/launchpad/info_portlet_1052551/+merge/129706 for your eyeballs if you get a sec please
<benji> rick_h_: sure
<abentley> benji: could you please review https://code.launchpad.net/~abentley/launchpad/workitems-for-private-blueprints/+merge/129736 ?
<benji> abentley: sure
<abentley> benji: Thanks.
<benji> my pleasure
<abentley> rick_h_: Are you using my getProductPrivacyFilter for fixing bug #1063272 ?
<_mup_> Bug #1063272: Cannot view +related-projects because of private-project <403> <private-projects> <Launchpad itself:In Progress by rharding> < https://launchpad.net/bugs/1063272 >
<adeuring> rick_h_: my branch landed finally
<beuno> wgrant, ping
<beuno> wgrant, looking for a reviewer for a python-oops-tools branch: https://code.launchpad.net/~beuno/python-oops-tools/show-newest-oopses/+merge/129752
<czajkowski> beuno: on call reviewer is benji
<beuno> czajkowski, this may be a tad specific to throw in the general review queue
* benji changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: ~260
<beuno> benji, thanks
<beuno> benji, I'll reply in the merge proposal but the spaces after the comma are pep-8 complaints
<benji> beuno: right, but the commas themseleves are what stood out to me
<beuno> benji, ah, yes, there's a bunch of XXXs warning that it needs refactoring
<beuno> didn't look into it
<benji> beuno: I figured a code formatting tool or search/replace was used and the strange comma-after-a-single-item-list threw it off
<beuno> right
<beuno> no, all the crazy is hand-crafted!
 * benji starts selling t-shirts with "all the crazy is hand-crafted!" on them.
<wgrant> beuno: I'm happy to review that, but any Launchpad reviewer can also do it
<wgrant> As benji suggests :)
<beuno> wgrant, cool, thanks
<wgrant> abentley: Do you have a branch in progress to stormify Person.specifications? I have a branch that needs it, so if not then I'll do it today.
<abentley> wgrant: Yes, I do.
<abentley> wgrant: ~abentley/launchpad/user-blueprints
<wgrant> abentley: Thanks
<wgrant> Although it's mysteriously empty
<abentley> wgrant: I just pushed it now.
<wgrant> Ah
<wgrant> abentley: Is it likely to land in the next day or so?
<abentley> wgrant: I'm just in the middle of adding unit tests for the existing behaviour.
<wgrant> Right, it certainly needed it :/
<abentley> wgrant: Yes, ideally it will land tomorrow.
<wgrant> Great, thanks.
<abentley> wgrant: What is your branch doing?
<wgrant> abentley: Fixing the +specworkload timeouts, which needs that query to be reworded and run over multiple people
<abentley> wgrant: Okay.  So where I'd naively put  "Specification.drafter = user.id", you
<abentley> want to do "Specification.drafter.is_in(user_ids)" etc?
<beuno> benji, addresed and pushed
<wgrant> abentley: Right, I'll need to extract it out to separate function/method
<abentley> wgrant: I should have *something* in the next couple of hours, but likely won't finish 'till tomorrow.
<james_w> beuno, doesn't the db have the information on newest oopses?
<beuno> james_w, I don't actually know  :)
<wgrant> It does :)
<beuno> wasn't sure if the oopses get processed and stored in PG as they come in
<wgrant> I haven't actually looked at the branch yet
<wgrant> You can find the latest with a simple Django ORM query
<beuno> that would of been much simpler
<james_w> benji, they do
<james_w> beuno I mean
<james_w> and much more efficient :-)
<beuno> indeed
<beuno> ok
<beuno> so I need to change everything!
<wgrant> Well, "everything" == "one 10 line method", but yes :)
<james_w> well, not everything :-)
<beuno> no, EVERYTHING
<beuno> or not
<beuno> test should still pass, so forced TDD!
<beuno> but, at this stage I
<beuno> I'll add what I was going to in my follow up branch
<beuno> relative time since the oops and some data about it
<wgrant> I want to add a similar thing to show the latest OOPSes for a pageid on a particular instance.
<wgrant> But haven't got around to it yet
<abentley> wgrant: Initial implementation is pushed.  Lots more to go.
<wgrant> abentley: Lovely. Thanks
<rick_h_> abentley: didn't get that far before EOD to tell yet
<abentley> rick_h_: I expect that it will want to use it, so I was leaving it alone until the other branch lands.
<rick_h_> abentley: abel's branch?
<abentley> rick_h_: No, mine.
<rick_h_> abentley: I just merge his stuff and going to get my stuff waiting on that to ec2 for the night
<rick_h_> abentley: ah ok
<rick_h_> abentley: thanks for the heads up. I'll take a peek then at your stuff you've got in the wings
<wgrant> win 43
<rick_h_> 44
#launchpad-dev 2012-10-16
<wallyworld_> wgrant: there's a couple of criticals for timeouts doing the most simple updates to the bug table eg update bug set description = "blah" where id = 234. that took 8s. we should be able to close these as a one off you think?
<wgrant> wallyworld_: Not for writes, no
<wgrant> wallyworld_: Long trivial reads are The Glitchâ¢, but writes tend to be locks
<wgrant> Which are a real issue that warrants investigation
<wallyworld_> difficult given how infrequently it seems to happen
<wgrant> Well
<wgrant> We can put in place monitoring
<wgrant> To track down long invasive locks
<wgrant> But indeed.
<wallyworld_> and it seems sporadic - 3 bugs, 3 different single inserts, 3 different tables
<wgrant> Which bugs?
<wallyworld_> bug 1059853
<_mup_> Bug #1059853: BugTask:+editstatus timeout blocked on bug update setting heat <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1059853 >
<wgrant> Often we'll have multiple OOPSes
<wallyworld_> bug 1048984
<wgrant> Which suggest it's not just The Glitch
<_mup_> Bug #1048984: BugTask:+edit timeout <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1048984 >
<wgrant> But obviously writes *can* be affected by The Glitch too
<wallyworld_> bug 1028105
<_mup_> Bug #1028105: BugTask:+editstatus slow inserting BugMessage <bugs> <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1028105 >
<wgrant> So, all of those could conceivably be long Person locks
<wallyworld_> how is updating a bug description affected by a person lock?
<wgrant> Ah, true, the description change couldn't be affected
<wgrant> But the BugMessage one probably is
<wgrant> Due to the FK to person
<wgrant> If you're inserting a new row, or changing the value of the owner column, it will conflict with a write lock on the referenced person row
<wallyworld_> do we know what holds long write locks on Person?
<wgrant> But the UPDATE Bug SET date_last_updated=%s, description=%s WHERE Bug.id = %s; can indeed not be affected by that
<wgrant> It must be either a glitch or a lock on the bug row itself
<wgrant> All three could be explained by a lock on the bug row.
<wallyworld_> sure. finding out what is the root cause will be difficult
<adeuring> good morning
* frankban changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: frankban | Firefighting: - | Critical bugs: ~260
* rick_h_ changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: frankban, rick_h | Firefighting: - | Critical bugs: ~260
<sinzui> jcsackett, do you have time to hangout
<jcsackett> sinzui: sure, give me a moment to locate my phone.
<abentley> sinzui: Do you know whether the ProductReleaseFinder should consider non-public products?
<rick_h_> abentley: r=me but didn't mark approve since you mentioned some stray test failures in the stand up
<abentley> rick_h_: Thanks.  I've nailed down most of the test failures now, just got ProductReleaseFinder to worry about.
<rick_h_> abentley: k
<sinzui> abentley, I am confident that the answer is no
<abentley> sinzui: Thanks.
<sinzui> abentley, I don't think they should be permitted to have releases...but that can be debated. they should not be permitted to have a release_file_glob.
<adeuring> frankban, rick_h_: could one of you please review this MP: https://code.launchpad.net/~adeuring/launchpad/milestone-sec-adapter/+merge/129907 ?
<rick_h_> adeuring: sure thing, will take a look
<rick_h_> adeuring: is the QA on the other branch all ok?
<abentley> rick_h_: I've pushed r16113, which has no known test failures.
<adeuring> rick_h_: just marked it as OK
<rick_h_> adeuring: awesome thanks
<adeuring> rick_h_: ...and I noticed a bunch of merge conflicts in the new branch...
<rick_h_> adeuring: in the milestone one?
<adeuring> rick_h_: yes
<rick_h_> adeuring: ok cool, will work on getting the deploy done while you peek at that then
<adeuring> rick_h_: sounds good
<adeuring> rick_h_: I created a new branch & MP: https://code.launchpad.net/~adeuring/launchpad/milestone-sec-adapter/+merge/129917 (was easier create a new branch than to update a somewhat outdated one...)
<rick_h_> adeuring: rgr
<rick_h_> adeuring: is +            self.assertRaises(Unauthorized, getattr, obj, name)
<rick_h_> oops
<rick_h_> adeuring: is Authorzized a german thing? At first I thought it was a typo but it's consistantly repeated
<adeuring> rick_h_: nah, that's an oridnary typo ;)
<rick_h_> ah ok, well you're darn consistant with it and got me wondering :)
<mgz> is you're going to tyop, it's best to do it consistently!
<adeuring> I'll fix it
<rick_h_> adeuring: np, noting them in the comments
<rick_h_> mgz: exactly! Get them thinking
<rick_h_> adeuring: r=me with some typo and a question
<adeuring> rick_h_: thanks!
<rick_h_> thanks for the comments to help me along!
<rick_h_> adeuring: on the check vs the db constraint, I just bring it up as a general case of check what you want to check directly in case things come along later and you get more than you meant to at a later date
<adeuring> rick_h_: could you explain this a bit more?
<rick_h_> adeuring: so right now there's two cases, product nad distribution.
<adeuring> right
<rick_h_> later, let's say someone adds distroseries (not that it would in this case)
<rick_h_> your check is against product, so now two things would match, distribution and the newly added distroseries
<rick_h_> because you were only checking that product is None
<rick_h_> in this case, I guess it's unlikely to be expanded to other items, but as a general rule of thumb, if you want to make sure it's not a distribution I'd word it as self.distribution is not None
<rick_h_> vs the indirect product is None
<adeuring> rick_h_: ok, in that case we would anyway need to distinguish the cases for distribtuion and distroseries. The current check for "prodct is not None" has in this case the advantage that it prvents a failure for the case that distribution _and_ product are none because distroseries is not none
<rick_h_> adeuring: right, but you'd hit a failure and then make sure you take the new condition into account vs silently passing successfully
<rick_h_> adeuring: sorry, it's just a general comment and nothing in particular in this case.
<adeuring> rick_h_: ok ;) and ISTM that we have different kinds of failure in mind ;)
* frankban changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: rick_h | Firefighting: - | Critical bugs: ~260
<abentley> rick_h_: I'm going on lunch, but could you please review https://code.launchpad.net/~abentley/launchpad/user-blueprints-tests/+merge/129944 and https://code.launchpad.net/~abentley/launchpad/user-blueprints/+merge/129945 ?  They are two steps in the same work.
<rick_h_> abentley: loading up
<timrc> Question: we have a user using syncSources() to copy over binary packages from a source PPA to a target PPA.  The issue we are encountering is that if syncSources() runs and all arches are not already built in the source PPA, they apparently get built in the target PPA.  When the arch finishes building in the source PPA syncSources apparently will not overwrite what was built in the target PPA
<timrc> In our case it's imperative that the package is identical between the two PPAs
<timrc> He's currently having to add additional logic to his script to ensure syncSources is only called once all arches are built
<timrc> Is this a bug in LP or expected behavior?
<maxb> timrc: Somewhat expected. You may be interested in lp-promote-ppa in lp:hydrazine
<maxb> (Pre written cli script for this use case)
<timrc> maxb, I'll check it out.  Thanks for the tip
<abentley> rick_h_: private-product-listings is on devel now, so you should be able to merge and use it.
<rick_h_> abentley: awesome thanks
* rick_h_ changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: ~260
<abentley> wgrant: ~abentley/launchpad/user-blueprints is in EC2.
<cjwatson> timrc: By the way, is there a reason you're still using syncSources (which will almost certainly time out lots) and not copyPackages?  Not that it will IIRC make a difference to this bug, but still.
<cjwatson> s/bug/behaviour/
<abentley> deryck: There's no OCR.  Could you please review https://code.launchpad.net/~abentley/launchpad/user-blueprints-privacy/+merge/129967 ?
<maxb> cjwatson: syncSources generally works just fine for a single source. In that scenario, it is highly desirable to receive synchronous feedback
<timrc> cjwatson, Not sure why this particular user chose syncSources over copyPackages
<maxb> For a script writer's usage, syncSources is better
<maxb> Unless you're trying to sync something like the linux kernel, for which LP really is going to time out
<cjwatson> Why are you using syncSources rather than syncSource for a single source? :-)
<timrc> we use syncSource for a single, small, package in PES to copy a keyring package into PPAs that are created through one of our interfaces.  Works well enough.  I could see the concern if we were copying a lot of packages, though.
<cjwatson> syncSource I can more or less understand.  copyPackage and copyPackages I can certainly understand.  syncSources not so much.
<timrc> syncSource(s) / copyPackage(s) seem like poor choices for name when taken together
<timrc> s/name/naming/
<deryck> abentley, sure, I can review.
<abentley> deryck: Thanks.
<deryck> np
<abentley> deryck: rick_h_ already did the hard bits.
<cjwatson> timrc: Yes, they grew organically rather than being designed.  Once in place, of course, good APIs don't change.
<cjwatson> (Not designed as a whole, I mean.  Of course they were designed individually.)
<timrc> I really wish copyPackage/syncPackage had a flag that could be specified that would fail the operation if not all arches were yet built for the package in question
<deryck> abentley, yeah, that wasn't bad at all.  easy peasy.  r=me.
<timrc> it does seem like a bug to me if what you're trying to express is "copy but don't rebuild this package" only to do just that
<timrc> cjwatson, they change, hence versioning...
<timrc> seems like we decided to stop at 1.0 oh and default to devel, though
<abentley> deryck: thanks.  We're over our WIP limit on QA, so I'll hold off landing for now.
<cjwatson> timrc: Yeah, but in reality it would be very inconvenient for little real benefit
<cjwatson> timrc: Ubuntu uses copyPackage(s) *extensively*
<deryck> abentley, abel's should have moved across, sorry.  I updated the board.
<abentley> deryck: Cool.
<timrc> cjwatson, Yeah and our use of PPAs is not exactly what they were intended for, so I'm willing to cut some slack :D
<abentley> sinzui: around?
<sinzui> I am
<abentley> sinzui: Could you do me a favour and create a proprietary project on qastaging so I can test my fix for https://qastaging.launchpad.net/projects/+review-licenses ?
<sinzui> okay
<sinzui> abentley, reload. I created https://qastaging.launchpad.net/aaron-curtis-test
<abentley> sinzui: I see nothing new, but it's not broken.  Win.  Thanks.
<sinzui> yep
<wgrant> sinzui, abentley: That project doesn't look private
<wgrant> All its sharing polices are public
<wgrant> "Information Type: Public"
<abentley> sinzui: Yes, I can see the project under /projects, for example.
<sinzui> oh, I made a commercial project, not a public one.
<sinzui> I suck
 * sinzui tries again
<wgrant> It's Proprietary now
<abentley> sinzui: I want a proprietary one.
<sinzui> I will do both
<abentley> Strangely, I seem to be able to navigate to https://qastaging.launchpad.net/aaron-curtis-test but I don't see it listed at https://qastaging.launchpad.net/projects
<wgrant> As I mentioned in the review, it's because Product.userCanView lets ~registry see private projects
<wgrant> Which doesn't really make sense
<sinzui> I did it right the first time https://qastaging.launchpad.net/aaron-curtis-test-2
<abentley> sinzui: Okay, I don't see either one listed, but I can navigate to both.  I'll call qa-ok on this code, but I'll make sure we have a bug for this registry issue.
<sinzui> I see it as a CA
 * sinzui is happy
<abentley> sinzui: But you would also see it as product owner.  You should be able to see BzrTools as CA.
<sinzui> abentley, I see your private tests :)
#launchpad-dev 2012-10-17
<wallyworld> stub: hi, i'm getting this error from a query containing an IN expr with strings: "ProgrammingError: operator does not exist: text = bytea". i can copy and paste the sql from the traceback into a sql editor and it works fine. any clues?
<wallyworld> the sql statement contains a call to trim if it matters
<wallyworld> SELECT trim(leading 'pending-' from CommercialSubscription.sales_system_id) FROM CommercialSubscription WHERE (trim(leading 'pending-' from CommercialSubscription.sales_system_id)) IN ('foo', 'bar')
<stub> wallyworld: I think you are sending in a raw string instead of unicode, and it is being cast to a binary blob (BYTEA)
<wallyworld> i'm using storm
<stub> wallyworld: And your logging doesn't notice, because to it a string is a string
<wallyworld> Store.of(self).using(CommercialSubscription).find(SQL(voucher_expr), SQL(voucher_expr).is_in(voucher_ids))
<wallyworld> voucher_ids is a list of strings
<wgrant> Right, that's a unicode vs str issue
<stub> Are you sure about that?
<wallyworld> i tried casting to unicode
<wallyworld> let me try again
<stub> ok.
<wgrant> sales_system_id needs a unicode
<stub> surprised storm didn't pick it up - it is normally very picky
<wallyworld> i could have sworn i did that, i'll recheck
<wgrant> stub: Not when you use SQL()
<stub> Oh right, foot gun
<wgrant> Yeah
<wgrant> I'm not sure that using pending- as a prefix here is sensible
<wgrant> Rather than adding a new column, for example
<wallyworld> wgrant: so using SQL() breaks it?
<wgrant> wallyworld: No
<wgrant> It depends how you're building the string that you give to it
<wallyworld> ah, hang on, it worked that time
<stub> SQL() is a way of bypassing in built Storm checks
<wallyworld> i used SQL() because i wanted an expr with trim()
<wgrant> Hm
<stub> Sure, it has uses. Just have to be more careful since you are saying 'this is valid SQL, trust me'.
<wgrant> It's more likely that voucher_ids is a sequence of strs, actually
<wgrant> As the rest of the types should be inferred.
<wallyworld> i must have messed up the unicode cast first time
<wgrant> voucher_ids needs to be a sequence of unicodes, since the DB column is Unicode.
<wallyworld> i cast the voucher_ids to unicode
<wallyworld> i must have had fat fingers the first time
<adeuring> good morning
<rick_h_> adeuring: ping, got a sec?
<adeuring> rick_h_: sure
<rick_h_> adeuring: I'm looking at https://bugs.launchpad.net/launchpad/+bug/1063272
<_mup_> Bug #1063272: Cannot view +related-projects because of private-project <403> <private-projects> <Launchpad itself:In Progress by rharding> < https://launchpad.net/bugs/1063272 >
<rick_h_> and from the bug report, it appears the fix could just be a simple check for a PUBLIC information type
<rick_h_> is there any reason to deal with things like checking your access via access policy and such to this you think?
<adeuring> rick_h_: yes: i think the user himself should be able to see  a products he can access
<rick_h_> I've not really used these pages before so my thought is that it's really only useful to other people and only showing PUBLIC types shold work
<rick_h_> ok
<adeuring> after all, that's an easy way to navigate to the a product
<adeuring> rick_h_: but for a quick fix, returning only public prodcts is foine, I think
<rick_h_> ok, :P don't let me cheat
<rick_h_> adeuring: well it's just done in raw sql atm with a series of unioned joins so will be a bit messier
<adeuring> yeah, I can imagine ;)
<abentley> wgrant: The new Person.specifications has landed.
* abentley changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: abentley | Firefighting: - | Critical bugs: ~260
<abentley> abel: Do you have time for some reviews?  I'm the OCR, so I can't ask the OCR.
<sinzui> jcsackett, I lowered the priority of the gmail attachment bug and remove you from it because gmail is forcing CRLF every 76 characters when it does it's base64 encoding. I don't think this is a critical issue.
<jcsackett> sinzui: dig.
<adeuring> abentley, deryck, rick_h_: my branch with the security adapter for milestones causes again test failures for a recently added  test:  test_listings_consider_spec_visibility . Could you give me a few hours to try to get my branch landed (I'd like to avoid "race conditions" like those we had for the adapter for products...)
<rick_h_> adeuring: yep, thanks for the heads up
<deryck> adeuring, I can wait for my stuff.  still doing calls, so not able to fix my own tests yet.
<abentley> adeuring: Okay.
<adeuring> rick_h_, abentley, deryck: Thanks! The alternative is of course to merge this brnach: lp:~adeuring/launchpad/milestone-sec-adapter (though now I'm getting "connection closed" errors when i try to push the most recnet version...)
<cjwatson> abentley: would appreciate thoughts on lp:~cjwatson/launchpad/pcj-queue-ordering, particularly on whether you agree with my performance handwaving
<abentley> cjwatson: Okay, give me a couple minutes.
<abentley> cjwatson: Can you add "Not(Job.id.is_in(seen))" to the find, and so skip the loops?
<adeuring> abentley, deryck, rick_h_: I started another EC2 test  a few minutes ago
<deryck> adeuring, cool, thanks for the heads-up.
<cjwatson> abentley: Oh, is_in doesn't have to take a Select or similar?
<abentley> cjwatson: No, it takes values.
<cjwatson> Neat.
<cjwatson> I'll give that a try.
<abentley> cjwatson: I don't fundamentally see away around changing the number of DB queries to scaling O(n) with next().
<abentley> s/away/a way
<cjwatson> Me neither.
<cjwatson> Unfortunately I don't have subsecond profiling data on how long those queries usually take at the moment.
<cjwatson> At least it's in a script, so worst case it goes slow not times out ...
<cjwatson> abentley: Not(Job.id.is_in(seen)) causes one of the tests to enter an infinite loop in the kind of way I'd expect if that test was entirely missing.
<cjwatson> (i.e. a test which doesn't actually process the jobs, just iterates over them all.)
<abentley> cjwatson: Can I see the diff?
<cjwatson> Sure.  http://paste.ubuntu.com/1285251/ - the failing test is test_iterReady_orders_by_copy_policy.
<cjwatson> Oh, wait, PCJ.id != Job.id perhaps
<abentley> cjwatson: Yes, but seen and the is_in are both in terms of Job.id.
<cjwatson> No, seen is in terms of PCJ.id
<cjwatson> seen.add(job.id) -> seen.add(job.job_id) fixes it
<abentley> cjwatson: great.
<cjwatson> Pushed.
<abentley> cjwatson: grr, diff still not updated.  I think this is reasonable.  You're aware you're making it possible for interactive jobs to starve mass jobs, right?
<cjwatson> Yes.
<abentley> cjwatson: The And() in the find statement is superfluous-- find accepts multiple clauses and ANDs them automatically.
<cjwatson> The volume of interactive jobs is a couple of orders of magnitude below capacity, and hardly ever goes over maybe a couple of dozen at a time.
<cjwatson> Thanks, removing the And.
<abentley> cjwatson: I would probably leave out the else after "if job is None", since everything following will be the remainder of the loop.
<cjwatson> Fair enough
<abentley> cjwatson: As for LOC, are you saying this work is part of an arc that will ultimately remove code?
<cjwatson> Yes.
<cjwatson> Let me double-check numbers quickly.
<abentley> Cool.
<abentley> cjwatson: r=me.
<cjwatson> The arc is, er, not exactly sure at this point but I think a few hundred so far; this should be near the end of it.  I have 1465 lines queued for deletion.
<abentley> cjwatson: if you did care about starving mass jobs, I'd suggest splitting interactive and mass into separate lists and yielding from each in turn in a ratio.  (Say 10 interactive for every 1 mass).
<cjwatson> Right, thanks.  My preliminary analysis a week or two back suggested it wouldn't be necessary; the queue is empty or very close most of the time, except when we're auto-syncing from Debian.
<cjwatson> (Which will be in a week or two, so I want to make sure PS doesn't come and shout at me around then ...)
<abentley> :-)
<cjwatson> In fact we'll hopefully (!) be doubling our copy job load by staging everything in -proposed first, if we get the infrastructure for that working in time.
<cjwatson> Incidentally, recent LoC progress has been pretty fantastic.  http://people.canonical.com/~cjwatson/tmp/loc-cum.png
<czajkowski> cjwatson: you're on the TB right?
<czajkowski> cjwatson: a mail was sent to it earlier today, if it hasn't been moderated could you please moderate it . thanks
<cjwatson> czajkowski: Yes.  Done.
<czajkowski> cjwatson: thank you
<sinzui> jcsackett, I think I found the strip() that lost the last line when making the attachment
<sinzui> jcsackett, I would like you thoughts about the comment I added to https://bugs.launchpad.net/launchpad/+bug/898227
<_mup_> Bug #898227: Newlines are removed from the end of bug attachments submitted by email <Launchpad itself:Triaged> < https://launchpad.net/bugs/898227 >
<abentley> deryck: I think that disabling privacy checks should be a separate feature flag, to avoid changing the meaning of existing tests, and to reduce risks on production (where we'd never turn it on).
<deryck> abentley, I agree.
<abentley> rick_h_: Could you please review https://code.launchpad.net/~abentley/launchpad/flag-enables-privacy-checks/+merge/130185 ?
<rick_h_> abentley: loading
<abentley> deryck: I'm the OCR.  Could you please review https://code.launchpad.net/~abentley/launchpad/product-specifications-tests/+merge/130000 https://code.launchpad.net/~abentley/launchpad/product-specifications-storm/+merge/130002 and https://code.launchpad.net/~abentley/launchpad/product-specifications-privacy/+merge/130003 ?
<deryck> abentley, sure.  That is 3 MPs, correct?
<abentley> deryck: Yes, 1. add tests 2. stormify 3. add privacy.
<deryck> abentley, ah, gotchas.  ok, jumping on them now.
<abentley> deryck: Thanks.
<deryck> np!
<rick_h_> abentley: r=me
<abentley> rick_h_: Thanks!
<deryck> abentley, so for the first branchâ¦. should there be some tests removed too?  If this is copied, should the other place die?  Or these are slightly different?
<abentley> deryck: The behaviour is different.  You can list the specifications of an inactive product.  You just can't see them elsewhere (like off a Person).
<abentley> deryck: After Stormifying, you can't list the specifications of an inactive product.  I wasn't sure whether that would cause test failures, but apparently not.  I don't know what the behaviour should be.
<deryck> abentley, I don't follow, sorry.  The tests are no good then, i.e. you don't get the failure you expect?  or you mean something else?
<abentley> deryck: The existing tests of Products don't care whether it's possible to list the specifications of a disabled product.  I don't know whether we care.
<deryck> abentley, ah.
<abentley> deryck: If we do care, then we should add tests to ensure we get whatever behaviour we want.
<deryck> abentley, right, I follow now.  Was just thinking myself if we care.
<deryck> abentley, I don't think we care.  no need for new tests.
<abentley> deryck: Okay.
<jcsackett> sinzui: what file is the block you posted from? i think you're right, but i don't recall seeing that chunk in the handler.
<jcsackett> (it also doesn't -- to me, anyway -- explain why there was no reproducing it in a test).
<sinzui> jcsackett, I am not convinced. It is in messages/model/messaging.py
<sinzui> I have a test with the real message from the librarian, but it does not fail
<deryck> abentley, all these look fine.  good work.  (r=me)*3
<abentley> deryck: Thanks.
<jcsackett> sinzui: yeah--that's the problem. i couldn't seem to get any sort of similar failure.
<sinzui> jcsackett, I found the real message that is stored in the librarian. I am feeing it into a test that created the inbox copy, then sends it through the bugs mail handler. the base64 message always has the proper new lines
<jcsackett> oh, that's just maddening.
<jcsackett> and yet; in a dire hope the bug was gone i emailed an attach to the bug from my gmail account, and proved it's still an issue.
<sinzui> I can see that block will do odd things, but I just don't see the real example step into the block
<sinzui> I saw
<jcsackett> sinzui: do we know all the callsites? could it be mutated at some other point?
<sinzui> jcsackett, I guess the real new news is that we can be sure the message was not corrupted when we took it from the inbox and put it into the librarian. I think the bug/message/bugcomment rules mess with it
<jcsackett> sinzui: must be.
<sinzui> jcsackett, message does mutate. It does forced decoding
<abentley> deryck: These three branches take us over the WIP limit.
<deryck> abentley, ok, thanks for the heads up.  Most of this is my fault.
<abentley> np
<deryck> abentley, if I don't get my stuff out of the way here shortly, I'll up the WIP limit.
<sinzui> The real message https://pastebin.canonical.com/76759/
<sinzui> ^ jcsackett note that there are two encoding which are fine, but I think Message can do something wrong
<jcsackett> sinzui: good to know that the raw message at least is stored properly.
<jcsackett> sinzui: how did you check that, btw?
<sinzui> I looked at the message ids for the attachments to the bug in staging's db...
<sinzui> ...then guess that message.id - 1 would show me what incoming.py got
<sinzui> incoming creates a random file name and and sequenced id. I typed the url in to my browser to pull it out of the librarian
<jcsackett> aaah. clever.
<sinzui> jcsackett, also, before the suspected block, there is `content = part.get_payload(decode=True)`
<sinzui> This is correct when I step though, but I am using that latest python 2.7
<jcsackett> the suspected block is the bit you posted, yes?
<sinzui> yes
<jcsackett> sinzui: are we on 2.6 on production?
<sinzui> correct
<jcsackett> huh.
<deryck> abentley, I have my fix for that DecoratedBranch stuff now.  So I can submit to ec2 now.
<sinzui> jcsackett, \o/ http://bugs.python.org/issue7143
<jcsackett> sinzui: w00t!
<jcsackett> sinzui: so, we can't reproduce it in tests because we're using a fixed version. so we need to upgrade to fix it in prod.
<sinzui> jcsackett, which we plan to do
<jcsackett> sinzui: so hurrah, this will be fixed then.
<sinzui> The bug is not critical, but I will tag it as python-upgrade so that we know to close it when we complete the task
<jcsackett> sinzui: cool.
<adeuring> abentley, deryck, rick_h_: my branch landed
<deryck> adeuring, awesome!
<abentley> sinzui: I'm having trouble reproducing the milestones side of bug #1063291
<_mup_> Bug #1063291: Project groups are broken by private projects <milestones> <private-projects> <projectgroups> <Launchpad itself:In Progress by abentley> < https://launchpad.net/bugs/1063291 >
<sinzui> hmm
<abentley> sinzui: This passes: http://pastebin.ubuntu.com/1285813/
<sinzui> abentley, did I have urls in the original bug?
<abentley> sinzui: No.
 * sinzui looks on qastaging again
<sinzui> abentley, still looking. anon users cannot see https://qastaging.launchpad.net/launchpad-project
<abentley> sinzui: Yes, I've got a fix for the private product part of that.
<sinzui> I thought so
<abentley> sinzui: There's proprietary blueprints on that-- that's probably the reason.
<sinzui> I am getting timeouts creating a milestone on a private project that matches a milestone on a public project, so I am being patient
<sinzui> abentley, non-priv user gets a 403 on qastaging for https://qastaging.launchpad.net/launchpad-project/+milestone/3.1
<sinzui> she can see https://qastaging.launchpad.net/launchpad-project/+milestones, but not the milestone itself
<abentley> sinzui: So I can rephrase as "If the project has a milestone with a bug or blueprint targeted, then I cannot see the  the groups's own milestone page"?
<sinzui> yes.
<abentley> sinzui: Thanks.
<sinzui> abentley, I think your blueprint speculation is correct. The page loads for the no-priv user and it has just a private bug targeted https://qastaging.launchpad.net/launchpad-project/+milestone/3.2
<sinzui> The 3.1 test has a proprietary blueprint from a previous test
<abentley> sinzui: Cool.
<sinzui> wgrant, https://bugs.launchpad.net/launchpad/+bug/1059853
<_mup_> Bug #1059853: BugTask:+editstatus timeout blocked on bug update setting heat <timeout> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1059853 >
#launchpad-dev 2012-10-18
<wallyworld> wgrant: i've created a master bug for that suspected lock timeout issue, and duped the 3 examples against that. i think those 3 should now be marked as High rather than Critical
<wgrant> wallyworld: Well, if you duped them then it doesn't matter
<wallyworld> the bug count still has not reduced - how often is it updated?
<wgrant> wallyworld: On webnumbr.com? Hourly
<wallyworld> no, in the lp portal
<wgrant> You can see the last update time down the bottom
<wgrant> Usually 7-10 past
<wgrant> LP portal?
<wgrant> Oh, the portlet?
<wallyworld> yeah, that
<wgrant> That drops immediately
<wgrant> It says 245 for me, which is roughly correct given the privacy skew
<nigelb> How do I commit a merge? I seem to have forgotten bzr.
<wallyworld> i could have sworn that it was 244 before i duped
<wgrant> nigelb: Same as any other change
<wgrant> bzr commit
<wallyworld> and it was 244 after
<wallyworld> maybe new bugs were filed in the time it took me
<wallyworld> or i misremembered the number
<wgrant> It was 251 yesterday
<wgrant> So it's decreased roughly the right amount
<wgrant> I know that dupes aren't included in that count, cause I rewrote it 3 months ago
<wallyworld> i was definately 24x before
<wallyworld> anyway, doesn't really matter, was just curious
<nigelb> Hrm. The history doesn't look right.
<wgrant> nigelb: Oh?
<nigelb> wgrant: Shouldn't it show me as the commiter only and the branch author as author?
<wgrant> nigelb: Remember that 'bzr log' supresses sub-revisions of merges by default. If you give it -n0 it'll expand merges.
<wgrant> nigelb: No
<wgrant> nigelb: Only if you use --author
<nigelb> ahh.
<nigelb> Am I supposed to do that so that history isn't skewed?
<nigelb> well, skewed is the wrong word.
<nigelb> I meant. History wrongly credits me.
<wgrant> No
<wgrant> You're credited for the merge, 'cause you did the merge :)
<wgrant> You can use --author if you wan
<wgrant> t
<nigelb> ah.
<wgrant> But the original author is still credited for all the revisions under the merge
<nigelb> launchpad will show the right author?
<wgrant> nigelb: No
<nigelb> :(
* wgrant changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: abentley | Firefighting: - | Critical bugs: ~240
* wgrant changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: ~240
<wgrant> We're now back to the start of Sep 2011 :)
<cjwatson> Has everyone stopped using 'ec2 land' or something?
<cjwatson> Yesterday I tried twice to land a branch and got unrelated test failures both times
<cjwatson> lp.soyuz.browser.tests.test_archive_webservice.TestArchiveWebservice.test_getAllPermissions_constant_query_count (OK, that's pretty sensitive anyway, but damn sure the PCJ model and tests shouldn't affect it), and pgbouncer syntax errors in lp.services.webapp.tests.test_dbpolicy.TestFastDowntimeRollout.test_master_slave_fast_downtime_rollout and ...
<cjwatson> ... lp.services.webapp.tests.test_dbpolicy.TestFastDowntimeRollout.test_slave_only_fast_downtime_rollout (WTF)
<cjwatson> Should I just lp-land and hope?
<stub> cjwatson: The pgbouncer syntax error is interesting - it would mean you are using an old version of pgbouncer.
<stub> cjwatson: I have used ec2 recently just fine
<cjwatson> It was complaining about the DISABLE command
<cjwatson> And I didn't do anything special in ec2, just the defaults
<stub> cjwatson: Perhaps you have an old ec2 image overriding the correct one?
<stub> DISABLE command wgrant added to me. We are currently running a fork of pgbouncer until it gets merged upstream, packaged etc.
<stub> And this is enforced in the -dependencies package
<stub> c/to me/for me
<cjwatson> Don't suppose the PPA and CAT are out of sync or something?
<stub> There seem to be lots and lots of CAT archives. I do know we are running the fork on staging and production, and I'm pretty certain it was rebuilt with the same version number
<cjwatson> Oh, haha, the second run got "Connection failed" when trying to install pgbouncer, but because we're using --force-yes it ignored that
<cjwatson> I WARNED YOU
<stub> cjwatson: If the tests pass locally, lp-land will be fine btw.
<cjwatson> Yeah, I haven't run the whole suite but as I say I'm certain the PCJ model won't affect the archive webservice tests.  Will lp-land.
<wgrant> cjwatson: When was that?
<stub> and the fdt tests should certainly pass locally for you. query count tests are certainly fragile though, and can be broken in unexpected ways
<wgrant> It was probably while haetae was off the network
<wgrant> (and yes, most people have stopped using ec2 for 90% of stuff)
<wgrant> wallyworld: Hm, your projectgroup changes earlier in the week have caused some performance regressions
<stub> cjwatson: bzr lp-land doesn't run tests, it will go straight to buildbot to choke on.
<wgrant> wallyworld: eg. HasAnnouncementsView.announcements materialises an unbounded set of announcements, causing eg. https://launchpad.net/+announcements/+announcements to time out
<cjwatson> wgrant: around 19:45 UTC
<wgrant> cjwatson: Right, that's around when stuff was broken
<cjwatson> I'm vaguely tempted to make that query count test use like ten additions in the second run instead of one, and loosen the query count requirement just slightly
<cjwatson> To account for minor auth differences
<wgrant> Yeah
<wallyworld> wgrant: i'll take a look
<adeuring> jcsackett: could you please review this MP: https://code.launchpad.net/~adeuring/launchpad/productseries-sec-adapter/+merge/130305 ?
<adeuring> deryck: could you please take a look at this mp: https://code.launchpad.net/~adeuring/launchpad/productseries-sec-adapter/+merge/130305
<deryck> adeuring, indeed
<adeuring> thanks!
<deryck> adeuring, my two branches that I'm trying to land add information_type to ProductSeries and Milestone via adaptation.  The approach conflicts with your approach here.
<deryck> adeuring, mostly that the information_type property is not needed this way.
<adeuring> deryck: ok. So, I'll merge your branch?
<deryck> adeuring, yeah, I think so.  let me get the url for youâ¦.
<deryck> adeuring, lp:~deryck/launchpad/series-information-type
<adeuring> deryck: ack
<deryck> adeuring, that is built on the milestone work.  so lots of changes, but they were reviewed separately.
<abentley> deryck: We do have some examples of Union in the codebase.  It's only used about 6 times, though.
<deryck> abentley, ah, cool.  Thanks for checking on that.
* jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: jcsackett | Firefighting: - | Critical bugs: ~240
<deryck> and yeah, I generally don't like using Union if it can be avoided.
<rick_h_> cool, maybe I'll try to look at it real quick
<jcsackett> adeuring: looking at yours now.
<rick_h_> I hate having these rules defined multiple times
<rick_h_> seems like a bug just waiting to get missed in some future tweak
<wgrant> deryck, abentley: I'm a little concerned that we're seeing performance regressions across all views that respect blueprint privacy
<wgrant> Because you're pushing new complex multi-level joins into these queries
<wgrant> We avoided this in the bug and branch implementations by denormalising the access data
<wgrant> The blueprint case is identical to the branch one
<wgrant> So the branch implementation can be reused
<deryck> wgrant, yeah, we have plans to focus just on performance after we hit beta.  and are saving denormalizing work until then.
<deryck> wgrant, well, not *just* on performance.  but performance is one focus for us after beta.
<deryck> wgrant, but thanks for the heads up about the branch implementation.
<wgrant> It's a little worrying that we're knowingly regressing performance when it's roughly a day of work to fix it last week
<deryck> how seriously are we regressing?  Are pages timing out?
<deryck> adeuring, I'll wait on you to merge me before I review further, since it's worth reviewing the branch in the context of my work.
<wgrant> We have a timeout exception for +specworkload which is no longer sufficient
<adeuring> deryck: sure, nearly done.
<wgrant> Anything that executes more than one specification-related query is likely to be a little sad now
<deryck> wgrant, I don't see any timeout for +specworkload on lpnet.  Am I just overlooking it?
<deryck> wgrant, timeout flag change, I mean.
<wgrant> Huh, maybe it was removed and this just pushed it further over the edge of the default
<deryck> wgrant, I understand your concerns, and appreciate that you wouldn't have approached it the way we are.  but I'm going to stick with the plan, and we can turn to performance stuff next week.
<wgrant> But, in general, this is a very cheap change to make beforehand -- we have the code to do it, we know exactly the performance characteristics of both methods, and it's around a day of work to do
<wgrant> OK
<wgrant> I cannot agree with that, but OK :)
<deryck> wgrant, I understand.  but hitting our milestones is very important to me.  and we don't have a day to spare right now.
<wgrant> I understand that milestones are important
<wgrant> But that makes about 9 critical regressions from this feature so far
<wgrant> It's a bit worrying
<deryck> wgrant, thanks for understanding.  but they're only worrying to me if they remain unfixed.  we're not releasing with those regressions.
<deryck> wgrant, not final release I mean.
<wgrant> They're critical because they're actively degrading Launchpad for non-beta users
<deryck> wgrant, but we consciously allowed space for regressions along the way if we could move faster.
<deryck> wgrant, hmmm, let me look at bugs then, I'm not aware of some of these, I bet.
<wgrant> Ah, some of them are actually fixed now
<wgrant> But there's still a number of private blueprint listing issues, plus the performance regression
<wgrant> Hm, and one of them is incorrectly critical, since private projects aren't enabled on prod yet
<adeuring> deryck: using IInforamtionType(productseries) leads to a permission problem: http://paste.ubuntu.com/1286974/
<wgrant> Ah, nevermind, Aaron marked that one critical, so it's fine
<wgrant> So not 9 any more, but still a few
<wgrant> (that were predicted)
<adeuring> deryck: I could use removeSecurityProxy(series).product but I don't like to do this outside tests...
<deryck> wgrant, sure, but it's not as if we're not being sensitive to this.  we knew this could happen, but it has allowed us to move faster.  and in the most egregious cases, we've fixed them immediately...
<deryck> wgrant, and we're committed to fix them all.
<adeuring> erm, I mean IIformationType(removeSecProxy(series))
<deryck> wgrant, so I respect that you would approach this differently, but please don't act as if we're being cavalier and not thinking about what we're doing here.
<deryck> wgrant, we're making hard choices to try to finish this a month from now.  and sometimes we do let a critical bug through.  but we will fix them all.
<deryck> adeuring, looking now...
<deryck> adeuring, isn't that the XXX from abentley that you removed.  where he went info_type = productseries.product.information_type because adaptation didn't work.
<adeuring> deryck: I think that was about another issue with the adaptation
<adeuring> deryck: the permission problem is new
<deryck> adeuring, ah right.  seems like I had something similar with milestone and a change in zcml to allow IInformationType fixed it.  but I could be recalling wrong.
<adeuring> deryck: here, we would have to allow public access to productseries.product...
<adeuring> well, let's, that _might_ be possible...
<deryck> adeuring, hmm, are we sure the test is setup right?  maybe the unauthorized is accurate.  if you can view the product series you should be able to view to product.
<deryck> adeuring, or maybe I'm misunderstanding what's happening.
<adeuring> deryck: ah, right, we can login as the "right" user, that should fix the issue
<abentley> adeuring: I'm not sure whether or not that's a symptom of same problem, because in my bug, adaptation did something evil to the Checker.
<adeuring> abentley: well, is IInformationType(obj) supposed to provide all attributes of obj itself?
<abentley> adeuring: No.
<adeuring> abentley: sorry, mis-read your bug report.
<adeuring> abentley, deryck: but then we two issues with IInfoType(obj)
<adeuring> deryck: sees that my approach to simply implement InfoType in ProdcutSeries might be more reliable
<deryck> adeuring, so it is easier/more reliable on one hand, but the view/ui work is easier with adaptation.  very little work to handle this with adaptation.
<deryck> adeuring, are there other issues?  I didn't follow you a few lines back, or changing the test is sufficient?
<adeuring> deryck: no, test works fine with a "with_person_logged_in()", but I am a bit concerned about the bug abentleyreported
<deryck> adeuring, agreed.  but good lord, we have enough tests that anything worrying should be caught, I think. :)  Let's stay with adaptation and move ahead.
<deryck> adeuring, and FWIW, I'm no fan of adaptation.  I think normal python properties are always better.  but in this case, it does help tie in to other parts of the system better.
<adeuring> deryck: ok. I've pushed a new revision
<deryck> adeuring, ok, cool, looking again.
<abentley> deryck: I'm getting test failures on stable r16164.  Can you reproduce them? http://pastebin.ubuntu.com/1287026/
<abentley> deryck: They also occurred on ec2, devel and my product-specifications-privacy branch.
<deryck> abentley, ok, doesn't sound good. let me seeâ¦.
<wgrant> abentley: Were these from ec2 about 20 hours ago?
<wgrant> You'll get them locally if you're running an old version of pgbouncer, and on ec2 if you started a run while ppa.launchpad.net was down 20ish hours ago
<abentley> wgrant: 15 hours ago or so.
<wgrant> abentley: Things were getting back to normal around then
<wgrant> But this error basically means you couldn't talk to ppa.launchpad.net at the start of the run
<wgrant> So I'd ignore it
<abentley> wgrant: Okay, thanks.
<wgrant> As for the local failure, apt-get upgrade should fix it
<wgrant> Unless you're running something newer than lucid
<abentley> wgrant: I'm running quantal, like a good little launchpad dev.
<wgrant> Ah
<abentley> wgrant: But I'd somehow missed restoring the launchpad ppa.
<wgrant> I guess Blue's work made it a bit more feasible to run on 2.7
<wgrant> abentley: I'm not sure if we have the patched pgbouncer for >lucid
<wgrant> Oh, we do
<abentley> wgrant: looks like.
<wgrant> 1.5.2-2+lp2~24~quantal1
<abentley> wgrant: And fixed.
<wgrant> Great
<deryck> abentley, so I don't see them locally, but I see wgrant helped sort it out.
<abentley> deryck: Yup.  Thanks.
<deryck> adeuring, sorry it's taking me a bit.  diff got huge with my branch. :)  Sorting through it all though.
<deryck> adeuring, I still have a couple tests to fix in mine, too.  So not sure if you want to wait on me and re-merge.  or take your chances in ec2, too. :)
<adeuring> deryck: right, there was a kind of a explosion in the diff size ;) And, right, I think I'll wait for your fixes
<deryck> adeuring, shouldn't take me too long after the review.
<adeuring> sounds good
<deryck> mine branch was actually two branches originally, too. :)
<deryck> adeuring, r=me.  good stuff, thanks!
<adeuring> deryck: thanks :)
<deryck> np!
<rick_h_> deryck[lunch]: ping when you get back
<abentley> deryck: chat?
<deryck> abentley, sure.
<deryck> abentley, meet you in the stand-up hangout.
<abentley> deryck: The comment is the output of our chat: http://pastebin.ubuntu.com/1287627/
<deryck> abentley, looking....
<deryck> abentley, yeah, that looks right to me.
<abentley> deryck: cool.
<cr3> is there a way to get an archive of a private mailing list on launchpad?
<maxb> WebOps have provided mboxes of public archives in the past - maybe file a question and see what they say?
<czajkowski> cr3: if you file an answer on LP
<czajkowski> we'll get web ops to look at it and see what it entails
<cr3> czajkowski: will do, thanks
<czajkowski> cr3: I#d say as a once off it'd be done but not on a regular basis
<cr3> czajkowski: one off would be much appreciated, and I'll share with my team just in case it might interest them as well
<czajkowski> nods
<czajkowski> and then next time archive mail rahter than delete mail
<czajkowski> mdz had a point about deleting mail :)
<cr3> czajkowski: I never delete my personal mail though, just mailing list email
<czajkowski> ah :/
<jpds> People delete emails?
<cr3> czajkowski: my process is partly based on the assumption that mailing lists use something like mailman where a tarball is always available
<cr3> czajkowski: and I'm usually not interested in mailing lists that don't use mailman anyways, but the irony is that launchpad actually does use mailman in the backend :)
<czajkowski> launchpad is unique :)
* jcsackett changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: ~240
<oalca_> anyone?
<oalca_> im wondering,
<oalca_> if somebody is working in a comment feature on diffs?
<oalca_> not in the comment section, but the green/red highlighted sections of the diff
<oalca__> i'm wondering, if somebody is working in a comment feature on diffs?, not in the comment section, but the green/red highlighted sections of the diff
<oalca__> no?
#launchpad-dev 2012-10-19
<wallyworld> wgrant: a quick one for you https://code.launchpad.net/~wallyworld/launchpad/translator-licence-checks-531720/+merge/130458
<wgrant> Already there :)
<wgrant> wallyworld: Does set_translations_relicensing agreement want to invalidate the cache?
<wallyworld> wgrant: i think it should
<wgrant> Oh
<wallyworld> so that next time get is called, it returns the new value
<wgrant> I missed the last hunk, nevermind
<wgrant> It's a bit sad that we have a property into a method into a cached property into a method
<wgrant> But it's the easiest way to do this
<wgrant> r=me
<wgrant> Thanks
<wallyworld> thanks, i didn't want to mess with t too much
<wallyworld> wgrant: hwdb - we have plans to remove soon right? if so, we should be able to reduce the priority of the hwdb criticals
<wgrant> wallyworld: I demoted the non-ZOP criticals (eg. regressions) a few weeks ago, but by ZOP the timeouts are still critical, though we have decided to ignore them
<wallyworld> sure, but if the feature is being removed....
<wallyworld> we don't really care
<wgrant> Right, but they're still noise
<wallyworld> noise in the critical bug listing, yes
<wgrant> And in the OOPS reports :)
<wallyworld> only till the feature is removed
<wgrant> Sure
<wgrant> Removing code is the best way of fixing criticals :)
<wallyworld> if we aren't going to fix, regardless of oops report noise, it's no using seeing them
<wallyworld> when did we say we can remove the code?
<wgrant> Probably in about 6 months
<wgrant> Anyway, you can demote them if you want
<wgrant> There's ~1 timeout and ~2 OOPSes
<wallyworld> i will do that i think
<adeuring> good mmorning
<RoelV> hi!
<RoelV> how do I make an existing project part of another project in launchpad?
<czajkowski> cjwatson: it feels like we only just compelted the opening of translations for Q!
<cjwatson> The wheel keeps on turning
<cjwatson> And to be fair it was a bit prolonged last time because it took ages for people to work out what to do, hence me providing some links up-front
<czajkowski> it's a slow cog in motion today
<shadeslayer> I seem to be getting this error : http://paste.kde.org/574226/ : when polling launchpad every 5 minutes
<shadeslayer> script : http://paste.kde.org/574232/
<shadeslayer> could someone suggest a polling interval that won't get my ip blocked?
<cjwatson> I don't know the reason for the error, but that error doesn't look like a blocked IP to me
<cjwatson> Your problem might well simply be that you're trying to use launchpadlib in a threaded program
<shadeslayer> oh
<cjwatson> Try rewriting it based on an event loop
<shadeslayer> yeah, I guess I'll have to do that
<jml> how do I get a launchpadlib object given the normal browser URL  for that?
<jml> load(), maybe
<jml> ah, but the URL needs to be transformed, somehow
<jml> <app> -> api, insert api version as prefix
<cjwatson> jml: or just strip the scheme and host off the front
<cjwatson> >>> lp.load('/ubuntu/+source/base-files')
<jml> cjwatson: oh, that works? neat. thank you.
<cjwatson> <distribution_source_package at https://api.launchpad.net/1.0/ubuntu/+source/base-files>
<flacoste> jelmer: thank you very much for all your work over the last 3 years! good luck with your future projects!
<nigelb> jelmer's leaving too? :(
<cjwatson> hm, I guess I can't land those two branches since we're in testfix mode
<rick_h_> abentley: r=me
<abentley> rick_h_: You mean abel?
<rick_h_> doh, bad tab completion, yea adeuring ^
<adeuring> rick_h_: thanks!
<cjwatson> adeuring: did you notice your buildbot failure?
<adeuring> cjwatson: no, thanks for the heads up!
<adeuring> cjwatson: looks like a flaky db connection, we had tehse failures too often... I#ll start a another buildbot run
<cjwatson> ok
<rick_h_> deryck: ping, got a sec?
<deryck> rick_h_, sure, what's up?
<rick_h_> deryck: want to chat and bring up another possible card for beta
<deryck> rick_h_, sure.  meet you in the standup hangout.
<jcsackett> sinzui: available to chat?
<sinzui> yes
* bac changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: bac | Firefighting: - | Critical bugs: ~240
<oalca> https://blueprints.launchpad.net/lp-dev-utils/+spec/comments-in-line-on-diffs
<oalca> watch this
<oalca> its what I believe a good blueprint
<jml> *sigh*
<czajkowski> oalca: thats a blueprint issue
<czajkowski> oalca: it'd be a bug if anything to be developed on , if you like please file a bug .
<czajkowski> oalca: https://bugs.launchpad.net/launchpad/+bug/780165  maybe this might be close to what you are looking at ?
<_mup_> Bug #780165: commenting on specific lines in a diff using the web UI is tedious <code-review> <Launchpad itself:Triaged> < https://launchpad.net/bugs/780165 >
<oalca> lets see
<oalca> <czajkowski> lets see
<cjwatson> oalca: I've generally found that it's not a good idea to register blueprints on projects you're not a developer on; it makes more sense for developers to decide whether something's big and complex enough to need that level of planning
<oalca> ok, im sorry for that, im kind of new on launchpad, I tought bugs where for fails on systems
<czajkowski> oalca: no worries have to learn somewhere.
<oalca> yesterday I was here, asking about this, but none answered, so I decide to do that
<abentley> bac: Could you please review https://code.launchpad.net/~abentley/launchpad/projectgroup-private-projects/+merge/130590 ?
<cjwatson> These two buildbot failures are both transient, right?
<cjwatson> The devel one certainly has nothing to do with my changes, so I'm retrying it
<cjwatson> But the db-devel one looks pretty transient too
<abentley> cjwatson: I agree.  Parallel-testing has unfortunately made buildbot a bit more flaky.  Still worth the trade-off, though :-)
<jml> *so* glad that happened, btw.
<abentley> jml: Yeah, it's great.
<cjwatson> OK, forced db-devel too
<cjwatson> And I definitely agree on the trade-off
<sinzui> jcsackett, I added a comment with out finding: https://bugs.launchpad.net/launchpad/+bug/403629
<_mup_> Bug #403629: Translation message link points to wrong message number <404> <lp-translations> <message-sharing> <Launchpad itself:Triaged by jcsackett> < https://launchpad.net/bugs/403629 >
<sinzui> ^ I think there is a second case were a POFile translation was accepted, but later changed. the submission remain, but it is not in the pofile
<lifeless> schpeeed!
<czajkowski> jelmer: good luck and we shall see you soon when you're in my neck of the woods!
<czajkowski> lifeless: ohhh hello there :D
<bac> sorry abentley, was on a call.  can i look at it after lunch?
<abentley> bac: sure.
<bac> abentley: ok, will do
<abentley> bac: thanks.
<abentley> deryck: Just added a card for bug #1068719 to the beta lane.
<_mup_> Bug #1068719: Person overview page breaks when assigned proprietary blueprints <oops> <private-projects> <Launchpad itself:Triaged> < https://launchpad.net/bugs/1068719 >
<deryck> abentley, I saw that.  thanks for adding it.
<deryck> abentley, do you think there's more investigation to do for the "blueprint listings related to product" or just drop that now, in favor of these other reported bugs/cards?
<abentley> deryck: I think we need to look for other possible issues there.
<deryck> abentley, ok, thanks
<rick_h_> abentley: ping, got a sec for eyeballs?
<rick_h_> abentley: doh nvm...dippy _owner for query :/
<rick_h_> sinzui: ping
<sinzui> hi rick_h_
<rick_h_> sinzui: I'm working on implementing this 'userCanBeDeactived' idea
<rick_h_> and I see lots of other examples of userCanXXXXX
<rick_h_> but none I can find are in a validator-like context. Where not only do I want a bool, but a textual message reason I can pass back to the UI
<rick_h_> sinzui: do you know of a case you can think of? I'm bzr grep'ing and failing.
<rick_h_> and I'm not totally cool with having a userCanBeDeactivated returning a tuple of (bool, [list,of,reasons])
<rick_h_> since it's a bit against the pattern in place for something named that way
<sinzui> rick_h_, lib/lp/registry/browser/peoplemerge.py illustrates what we do now to check for ppas and private branches
<rick_h_> looking, thanks
<sinzui> rick_h_, I see duplicate in between the validator in that module and PersonSet._merge
<sinzui> both are calling the same methods, but they handle them differently
<rick_h_> sinzui: yea, that's what I was trying to avoid. I hoped to really make the validator just call Person.userCanBeDeactivated and then that method would return true/false and messages
<sinzui> +1
<rick_h_> but it seemed against the current pattern, which appears to be true
<rick_h_> so wonder if I should just call it something different and move along
<sinzui> rick_h_, I think there needs to be complex method that raises an error with an informative message (that can be captured an safely displayed), and a method that captures any error and returns true. I think this is the common practice used by Zope fields to validator or sae if the data is valid
<rick_h_> sinzui: ah, ok cool. I'll add that second layer in
<abentley> sinzui: I've been thinking that maybe validation methods should yield (not raise) exceptions, so that you can capture all the validation errors.
<abentley> sinzui: Then methods that wanted to validate before making a change would simply raise the first exception, if any.
<rick_h_> abentley: yea, that'd be cool. I'm capting them into a list currently and then checking that return list.
<sinzui> That would solve the awkward validator in peoplemerge.py
<abentley> sinzui, rick_h_: Something like this: http://pastebin.ubuntu.com/1289982/
<sinzui> I think we prefer the look-before-you-leap style where canXXX returns early, other methods would collect them to show the user all the reasons
<abentley> deryck: chat?
<deryck> abentley, sure, give me 5 or 10 minutes to get free.
<bac> abentley: your branch looks good.  thanks.
<abentley> bac: Thanks for the review.
<deryck> abentley, I can chat now.  jumping into stand-up hangout.
<rick_h_> bac: got another branch for you if you've got time please? https://code.launchpad.net/~rharding/launchpad/related_projects_1063272/+merge/130414
<rick_h_> bac: tried to explain things as they can kind of come across as a bunch of disjoint changes
<bac> sure rick_h_
<rick_h_> bac: heading afk, if there's anything that comes up pop it in the MP and I'll get back to you.
<bac> rick_h_: ok, sorry i got distracted
* bac changed the topic of #launchpad-dev to: http://dev.launchpad.net/ | On call reviewer: - | Firefighting: - | Critical bugs: ~240
#launchpad-dev 2012-10-21
<jcsackett> anyone who followed the install launchpad in an lxc tutorial wind up with the lxc not starting b/c it couldn't mount /dev/shm because it's a "symbolic link to nowhere"?
<rick_h_> jcsackett: doh, let me know if you find an answer. I am running it that way right now but imagine I'll need to rebuild it on upgrade
<rick_h_> actually, should start that upgrade now I guess :/ so behind
<rick_h_> bah, so many PPAs to worry about
<jcsackett> rick_h_: indeed. i did a clean install b/c i needed to upgrade drives anyway. thought it was a good time to do the lxc thing but it seems to be fail.
<rick_h_> jcsackett: yea, I always do clean install so think that's safer for me anyway
<rick_h_> yea, I did the lxc thing, but I didn't want it in my home drive so I did some manual overrides
<rick_h_> actually wondering about doing it in a virtualbox server install this time
<rick_h_> like having LP dev on a diff setup
<jcsackett> fair.
<jcsackett> i think i'll have to go the usual way; wanted lxc but i need to be able to work tomorrow and don't have much time to muck around today.
<cjwatson> jcsackett: that sounds like /run erroneously not being (bind-)mounted
<cjwatson> sort of thing you might get with instructions from pre-er-precise-or-so I guess
<jcsackett> cjwatson: any thoughts on how to remedy it?
<cjwatson> no idea what instructions you're following
<cjwatson> jcsackett: I guess that in general it depends on whether the procedure in question is binding mountpoints from the host system, or constructing its own versions of important system filesystems internally
<jcsackett> cjwatson: i was just running the directions in https://dev.launchpad.net/Running/LXC figure i should become familiar with lxc and i basically know nothing about it at this time.
<lifeless> jcsackett: I haven't seen the symbolic link to nowhere happen before
<lifeless> jcsackett: I'd check for bugs in ubuntu/+source/lxc - closed ones in particular
<jcsackett> lifeless: thanks. i check through that, but found nothing too applicable. gary_poster pointed out there were some issues relating to dev/shm *after* starting an lxc, but nothing quite like this.
<jcsackett> there's some indication that the basic handling of dev/shm changed in quantal. i think that may be the failure point, though i'm unsure how to approach it.
#launchpad-dev 2013-10-14
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/spec-names-are-not-unique-vocab/+merge/190849
<wgrant> StevenK: Could be a while
<wgrant> but yay
<mwhudson> yay vocabularies
<jelmer> yay zope
<wa5qjh> Hey yall, can somebody tell me if that section between the USB connector and the Launchpad user section is anything besides  a USB to serial adapter,  or is it a sort of jtag or similar?
<cjwatson> You're still in the wrong channel, sorry :-)
<wa5qjh> weird!! was on a TI launchpad site and it referred me here ( I thought)
<wa5qjh> what command do you use on irc to exit just one channel ?
<cjwatson> I am REALLY not awake
<cjwatson> /part
<wgrant> Heh
<wa5qjh> Ah.  thanks!! odd word for that function :)  Tried bye and exit.  not what I wanted :)
<wa5qjh>  Thanks guys. you dont have any knowledge of an IRC channel for  the Texas Instruments Launchpad Development tool do you?
<cjwatson> Afraid not, never heard of it before you asked last night
<wa5qjh> OH, ok, well it was worth asking. what is  this channela ll about then ?
<cjwatson> The only one I find from Google is #43oh
<cjwatson> Dunno if that's close enough
<cjwatson> Development of https://launchpad.net/ which is a platform for collaborative free software development
<cjwatson> And the main infrastructure for the Ubuntu OS among others
<wa5qjh> Yeah, been there. still got it bookmarked, and one of a couple dozen windowes I have up in my browser at the moment.
<lifeless> wa5qjh: yeah, TI folk turn up here from time to time, univerally confused ;)
<lifeless> wa5qjh: it was very odd when TI released a product with the same name :)
<wa5qjh> Heh,  not too surprised,  That's why  I went ahead and asked if youd become aware of the right name of the ir irc if they have one.
<wa5qjh> so what are yall,  then ?
<lifeless> sorry no
<lifeless> wa5qjh: as cjwatson said : Development of https://launchpad.net/ which is a platform for collaborative free
<lifeless>                   software development
<lifeless> wa5qjh: And the main infrastructure for the Ubuntu OS among others
<wa5qjh> interesting!!  cause the other thing I've been looking for as a 1K "Tiny Forth"  nobody these days has a clue what that is.
<wa5qjh>  you ever heard of it ?
<lifeless> Forth ? yes.
<wa5qjh> I been playing with it since some time in Mid 80's but never really got proficient with it.
<wa5qjh>  But I'm basically bass-ackards anyhow so maybe that's why it facinates me so much.
<wa5qjh>  I tried  to install eclipse into my Ubuntu 11.04  but they didnt seem to want to supo my old unsupported 11.04rt in any way,  even with archives,
<wa5qjh> my od unsupported 11.04
<StevenK> Archives for 11.04 are around, but you should really really upgrade.
<wa5qjh> I wanted to when I managed to crash 11.04 last year. but when I tried to put 12.04 in,  it did not want to go in at all. and I would up having to p=map out about4GB of a 100MB partition just so I could get 11.04 back in.  that's an oversimplification of everything  but a digest of it.
<wa5qjh> I've also tried Kubuntu 11.10 as well. had it running till a blackout here finally took its toll.
<wa5qjh> one blackout too many.
#launchpad-dev 2013-10-15
<StevenK> wgrant: WTB reviews: https://code.launchpad.net/~stevenk/launchpad/spec-names-are-not-unique-vocab/+merge/190849 and https://code.launchpad.net/~stevenk/launchpad/unmute-tooltip/+merge/190857
<wgrant> StevenK: Done.
<StevenK> wgrant: Something about QA?
<wgrant> Oh, forgot about that one
<wgrant> Doing
<wgrant> StevenK: done
<StevenK> lifeless: So, bug 1050173, Resolution: accepted is now Fix Released? The code currently maps closed/accepted to Fix Commited, and closed/fixed to Fix Released.
<_mup_> Bug #1050173: bugwatches set to accepted closed in python.org roundup maps to 'fix committed' not 'fix released' <bugs> <bugtrackers> <bugwatch> <roundup> <Launchpad itself:Triaged> <https://launchpad.net/bugs/1050173>
<StevenK> lifeless: http://docs.python.org/devguide/triaging.html#resolution still mentions 'fixed'
<lifeless> StevenK: TTBOMK there is no 'fix released' concept in roundup
<lifeless> StevenK: those bugs will stay open forever if you try to model that
<lifeless> StevenK: http://bugs.python.org/issue1638033 + https://bugs.launchpad.net/python/+bug/96878 for instance
<_mup_> Bug #96878: Launchpad session cookie is accessible from Javascript <bugjam2010> <infrastructure> <lp-foundations> <qa-ok> <Launchpad itself:Fix Released by wgrant> <Python:Fix Committed> <https://launchpad.net/bugs/96878>
<lifeless> StevenK: 'accepted' is not listed in the devguide, but it's what bugs are set to in roundup AFAICT. Go figure.
<lifeless> StevenK: I would say both accepted and fixed should map to FR.
<StevenK> lifeless: I'd be curious to see what percentage of bugs are closed/accepted versus closed/fixed.
<lifeless> StevenK: public data, you can generate that :)
<StevenK> lifeless: Except that you can't search by Resolution: accepted :-)
<wgrant> Is accepted a resolution?
<StevenK> http://bugs.python.org/issue1638033 => Status: closed Resolution: accepted
<StevenK> I'm tempted to JFDI closed/accepted to Fix Released as lifeless says.
#launchpad-dev 2013-10-16
<StevenK> wgrant: So, something about Javascript?
<wgrant> StevenK: What about it?
<StevenK> wgrant: The JS branch I have up for review.
<wgrant> There aren't any
<StevenK> Oh bah, it's still WiP
<StevenK> https://code.launchpad.net/~stevenk/launchpad/inline-diff-comments-ui/+merge/184006
#launchpad-dev 2013-10-17
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/do-not-validate-bugwatch-on-delete/+merge/191543
<wgrant> StevenK: r=me
<StevenK> wgrant: Javascript, since I've marked the MP as Needs Review?
<StevenK> wgrant: What about your favourite topic, microservices and the releases of auditor{,fixture,client}?
<wgrant> StevenK: Have you tested the JS for XSSes?
<StevenK> I think it's okay in that respect.
<wgrant> You thought so last time too :P
<wgrant> But it looks much better this time
<StevenK> wgrant: https://code.launchpad.net/~stevenk/launchpad/use-transitionToStatus-createManyTasks/+merge/191554
<wgrant> StevenK: That's not clearly correct.
<wgrant> StevenK: It'll now create items in the activity log and send emails.
<StevenK> wgrant: I don't see anything in IBugTask.transitionToStatus() that notifies
<wgrant> ah, right, it's the terribly broken subscriber model that does that.
<StevenK> I can refactor out the reseting and setting attribute logic so that createManyTasks and transitionToStatus both call it.
<wgrant> How expensive is transitionToStatus?
<StevenK> It checks if the user can edit the status, switches the status and sets columns based on the status
<wgrant> So, createManyTasks is meant to be vaguely efficient
<StevenK> Sure
<wgrant> It currently isn't great, because it calls a separate method to update the target name cache
<wgrant> But transitionToStatus will make it a lot worse.
<StevenK> :-(
<StevenK> wgrant: But we're only updating a few columns of one row?
<StevenK> Or is it the horrible zope subscribers that are defeating us?
<wgrant> StevenK: Confused
<wgrant> StevenK: You're making various auth queries, then setting a few columns on a row, for several rows.
<StevenK> wgrant: So we refactor out the reset and setting logic out of transitionToStatus(), and then no auth queries.
#launchpad-dev 2013-10-18
<cjwatson> wgrant: Did https://code.launchpad.net/~cjwatson/launchpad/db-builder-version/+merge/190633 look OK?
<cjwatson> Aside from insane sampledata diff
<wgrant> cjwatson: Oh, sorry, missed that. The patch looks fine.
<StevenK> That was me using a raring machine to generate the current sampledata
<cjwatson> Oh, and re the JS failures on precise, did anyone happen to spot https://bugs.launchpad.net/launchpad/+bug/1090395 ?
<_mup_> Bug #1090395: YUITest failures because quantal's webkit disagrees with lucids <javascript> <python-upgrade> <spurious-test-failure> <Launchpad itself:Triaged> <https://launchpad.net/bugs/1090395>
<wgrant> Yeah
<cjwatson> StevenK: immutable> I guess that was a regression due to my change some time back.  Sorry and thanks.
<cjwatson> Err, whoops
<cjwatson> wgrant: I seem to have accidentally landed db-builder-version on devel
<wgrant> Yeah, just noticed that
<cjwatson> wgrant: Is the best recovery path to re-propose to db-devel and land there as well?
<wgrant> cjwatson: Nah, we have nothing urgent landing atm, will deploy it on Monday
<cjwatson> Or just wait for the auto-merge to db-devel
<cjwatson> ?
<wgrant> It'll land on db-devel in about 20 minutes :)
<wgrant> yeah
<cjwatson> OK.  Sorry about that
<wgrant> np
<wgrant> cjwatson: Doing that without adjusting some nagios checks might upset some sysadmins.
<wgrant> Also potentially ppa_reset
<cjwatson> ppa_reset should be OK - it greps for BuilderStatus.IDLE anywhere in the output, which is still there
<wgrant> Ah, good
<cjwatson> Are the nagios checks in bzr anywhere?
<wgrant> check_builder_status or something might be in puppet
 * wgrant looks
<wgrant> The ABORTED check will break
<cjwatson> Where's that?
<wgrant> In fact I guess the whole thing will crash because it uses status[0]
<wgrant> files/nagios/plugins/check_builder_status in memento
<cjwatson> ok, should be able to fix that
<cjwatson> thanks
#launchpad-dev 2014-10-15
<stub> https://code.launchpad.net/~stub/launchpad/trivial/+merge/238401
#launchpad-dev 2014-10-16
<bigjools> halp: bzr: ERROR: Server sent an unexpected error: ('error', 'GhostRevisionsHaveNoRevno', 'Could not determine revno for {3260} because its ancestry shows a ghost at {maas_lander-20141015231443-x15k09gg438itz2s}')
<bigjools> wgrant: know what's going on there?
<bigjools> that's the result of doing bzr checkout --revision 3260 --lightweight lp:maas/1.7 maas
<wgrant> bigjools: That's more of a bzr question. But there are some bugs around stacking there; I'm not sure a lightweight checkout of a remote stacked 2a branch works.
<bigjools> wgrant: it's worked up until I just made a new branch, as it's in our jenkins config
<bigjools> If I remove the lightweight it still fails
<bigjools> I'll delete the branch and try again, one sec
<wgrant> bigjools: It may be because the 1.7 branch doesn't include the revision that you're asking for.
<bigjools> it does
<wgrant> Sorry, the branch's repo.
<bigjools> wgrant: weird.  If I "bzr revno lp:maas/1.7" it returns the same error
<wgrant> bigjools: That's what I'd expect in this case.
<wgrant> bigjools: Did you just branch that straight from trunk?
<bigjools> yes
<wgrant> bzr probably doesn't handle the case where the requested (or in this case head) revision doesn't exist in the branch's repository.
<wgrant> I *suspect* that it'll magically start working if you make a commit on 1.7.
 * bigjools tries
<bigjools> wgrant: yeah fixed it.  How weird!
<bigjools> thanks
#launchpad-dev 2014-10-17
<stub> wgrant: https://code.launchpad.net/~stub/launchpad/trivial/+merge/238401
<stub> At which point I see a bug, as I'm counting a 404 as a fail
<wgrant> stub: Ah, just said that in the MP myself.
<stub> I've shuffled it by one line.
<stub> I opened a bug on the annoying need to sniff 404's
<stub> wgrant: yes, there is an off my one issue in the error message, although the weasel words mean it is technically correct (1000 attempts, 999 failures), as it doesn't claim the final attempt has succeeded...
<stub> wgrant: it could be better, but then I need to worry about try: finally: blocks around the twisted call backs which was too hard to quick and easy stats
<wgrant> Heh
<wgrant> Yeah, it was an "if you can be bothered".
<stub> wgrant: Tests are failing, and I don't know why
<stub> 2014-10-17 13:43:39+0530 [-] Logged OOPS id OOPS-92d609222ecc4fb473d60ce517edadb2: AssertionError: file id has exceeded filesystem db maximum in the buildbot summary, which is nuts
<stub> gah... unless file_id is a string
<stub> yeah, better. So my lp container does work now, and I had a real problem ;)
<wgrant> Heh
#launchpad-dev 2014-10-19
<buzzCBSP> hi folks. Im getting an error trying to setup a launchpad dev system for the first time. Im using a fresh Ubuntu server 14.04.1 install on virtualbox. On running rocketfuel-setup it errors (bad config fiile name in Makefile - which Ive temporarily fixed..probably needs to be pushed back to the main repository (with a bug report)...anyway...I now get the following error (mod_rewrite is installed and enabled):
<buzzCBSP> AH00526 syntax error on line 4 of /etc/apache2/sites-enabled/local-launchpad.conf
<buzzCBSP> Invalid command 'RewwriteLock'....[snip]
<buzzCBSP> Any pointers?
#launchpad-dev 2015-10-12
<wgrant> FAILURE: lp.services.webapp.tests.test_dbpolicy.TestFastDowntimeRollout.test_master_slave_fast_downtime_rollout
<wgrant> Possibly related to my test DB reconnection changes.
<wgrant> I guess we'll see if it happens again soon.
<blr> morning
<mapreri> meh at OPs poking bugs after 2 year somebody replied them... (lp #1065398)
<mup> Bug #1065398: Email address doesn't change with username <Launchpad itself:Invalid> <https://launchpad.net/bugs/1065398>
<cjwatson> wgrant: safer-ajax-strings will almost certainly fix bug 581748, yes, but I'll test that before landing.
<mup> Bug #581748: 10.10 milestone name corrupted in JS: 10.1 <javascript> <lp-registry> <milestones> <Launchpad itself:Triaged> <lazr.restful:Triaged> <https://launchpad.net/bugs/581748>
<cjwatson> wgrant: Do you have a reference to a good LP.client *integration* test anywhere (that isn't disabled)?
<wgrant> cjwatson: NFI
<cjwatson> It was worth a shot.
#launchpad-dev 2015-10-13
<cjwatson> wgrant: It does indeed fix that bug, anyway.
<wgrant> cjwatson: Yay.
<cjwatson> 2015-10-13 18:15:37.484 10525 WARNING Too long db name: launchpad_ftest_template_10065_e95fa37c_71a7_11e5_961a_00163e9546d5
<cjwatson> hmm
<cjwatson> Maybe a UUID was a bit ambitious
<cjwatson> Using uuid.hex would be enough to get us under the limit though
<jtv> cjwatson: really going for scalability there, eh?
<cjwatson> Totally!
<cjwatson> (PIDs were clashing in practice sometimes, esp with LXC ...)
<jtv> 16 bits is not a lot...
<jtv> Then again, I've worked with long, randomised pids and those are no fun either.  :)
<jtv> (Even if they are probably safer)
<jtv> But security was more important than convenience...  For a few weeks I had no independent access to food, bathrooms, the outside, or the internet while doing my work.
<jtv> You really get to hate those long PIDs through a slow, fixed-80-column Windows NT terminal emulator with acute line-break bugs after a while...
<jtv> I mean really _hate_.
<lifeless> wgrant: what happened to bug linking
<lifeless> wgrant: was it behind a feature flag, or never delivered?
<wgrant> lifeless: Never implemented.
<wgrant> The team was vaporised days after you left :)
<lifeless> ouch, thanks
<wgrant> lifeless: We're vaguely working on arbitrary object cross-references, though.
<lifeless> oh, that would be good
<lifeless> if meta
<wgrant> lifeless: question/bug/cve/spec links use generic infrastructure now, though the UI is not genericised yet.
<wgrant> Just changed over the last couple of weeks.
<lifeless> nice
<wgrant> The plan is to genericise the UI, and then basic bug linking becomes easy, and we can work from there.
<wgrant> lifeless: You probably recall the whole weirdness around multi-task bugs and bugwatches, though.
<wgrant> Some cases of buglinking are handled natively by tasks.
<wgrant> And external bugs can be linked to in a limited way.
<wgrant> But linking to other internal bugs is not currently possible.
<wgrant> And adding a third way is Funâ¢.
<lifeless> yes
<lifeless> federated 3 eva
#launchpad-dev 2015-10-14
<jtv> Hi wgrant, hi lifeless
<wgrant> Morning jtv!
<jtv> Not a very busy channel these days, is it?
<lifeless> o/jtv
<wgrant> cjwatson: On bmp-webhooks, what's the motivation behind *_git_ref?
<cjwatson> wgrant: I think it's cleaner to give a unified description of changes in any of the refs being used; a change in the repository alone isn't very meaningful, say
<wgrant> cjwatson: We don't issue ObjectModifiedEvents on supersede, do we?
<wgrant> Unless we do, they can't change at all.
<cjwatson> wgrant: We do now, I fixed that.
<wgrant> Oh.
<wgrant> Hm, that's inconvenient.
<cjwatson> (back in consistent-bmp-events or so)
<cjwatson> That said, supersede would be a "modify old, create new" pair
<wgrant> I'd prefer that we special-cased those fields to emit both when either changed.
<wgrant> Oh
<wgrant> Apart from git_repository.branches and snap.git_ref, git_ref is input-only.
<wgrant> BMP never exposes them on the API today
<cjwatson> We issue an OME to describe the state change to Superseded, is all.
<wgrant> Oh
<cjwatson> (Which means that now people actually get mail about that)
<wgrant> So the fields can't appear to change in an event?
<wgrant> That's what I suspected would be the case.
<cjwatson> Yeah, I wonder why these are in BMPDelta to begin with
<cjwatson> Maybe a better fix is to delete those *and* their *_branch partners from BMPDelta.delta_values (with a bit more hunting around to make sure of this) and move the webhook payload composition directly into lp.code.subscribers.branchmergeproposal since once it needs to be a superset of BMPDelta's list it doesn't derive very much benefit from being there
<wgrant> That's certainly my preference.
<wgrant> It seems very weird to have them there.
<cjwatson> I probably wouldn't have started down that path had *_branch not been in BMPDelta.delta_values to begin with.
<wgrant> Yep
<wgrant> It gives us more flexibility with the payload composition.
<wgrant> And makes it slightly less strange.
<wgrant> (I also don't really know why the new_values thing is separate... surely that's a property of the output format, not the delta... but anyway)
<cjwatson> Oh.  Helpful.  Blame leads me back to r5608.7.5, whose commit message is "Further updates".
<wgrant> Heh
<cjwatson> Trunk-only blame is r6060, which added MP notifications.
#launchpad-dev 2015-10-15
<cjwatson> wgrant: bmp-webhooks fixed.  We do have to keep those fields in BMPDelta so that they're part of the snapshot in ObjectModifiedEvents, but I did all the rest of it as discussed.
<wgrant> cjwatson: Hum, don't we have the entire new object in the OME?
<cjwatson> wgrant: New yes, but the only old fields we have are those that are explicitly snapshotted by the delta
<wgrant> cjwatson: Right, but if the fields can never change then why are the old copies necessary?
<wgrant> Like we don't snapshot bug.id
<wgrant> (Well OK maybe we do, but it wouldn't seem necessary)
<cjwatson> er, oh, reasonable point
<cjwatson> let me work out (not right now) why it failed tests without that
<wgrant> Oh it actually failed tests?
<wgrant> Odd
<wgrant> That is certainly something that deserves investigation.
<cjwatson> I think just because those fields were missing
<wgrant> Oh, boring.
<cjwatson> perhaps should special-case copying those fields from the new object so that webhook consumers don't need to hardcode the assumption that they won't change
<cjwatson> it is conceivable that at some point we might allow changing prerequisites without resubmitting, for instance
<wgrant> Indeed, it is possible.
<wgrant> Not a terrible idea.
<wgrant> But I still think that, until that time, that belongs in the event bits rather than the delta itself.
<cjwatson> how do you mean?
<cjwatson> outside the "old"/"new" elements of the payload, you mean?
<wgrant> Oh, no, just what you suggested.
<cjwatson> oh, ok
<wgrant> The fact that they might change in future isn't good reason to keep them in the internal delta object to confuse us.
<cjwatson> I'll do that tomorrow then, hopefully I won't get hideously sidetracked by another click vulnerability :P
<wgrant> But it is probably good enough reason to include them in the payload generator.
<wgrant> Heh, yeah.
<cjwatson> I made a little progress on the turnip CI job today, though still a fair way to go.  But importantly I figured out the virtualenv construction bug I'd been stuck on
<cjwatson> specifically that "find_links = /path/to/directory" in .pydistutils.cfg is *not* the same as "find_links = /path/to/directory/"
<cjwatson> rsync disease
<wgrant> I actually use a full file:// URL
<wgrant> I don't remember why, but there must be a reason.
<cjwatson> er, right, so do I
<wgrant> Though it too has a trailing slash.
<wgrant> Ah
<cjwatson> but file:///path/to/directory != file:///path/to/directory/
<wgrant> So probably the reason is that a raw path doesn't work, that'd do it.
<wgrant> Yeah, odd.
<cjwatson> the former stats the directory and then goes "oh, I clearly don't care about that"
<cjwatson> thanks, setuptools
<wgrant> ...
<wgrant> Helpful.
#launchpad-dev 2015-10-16
<sil2100> o/
<sil2100> Hey guys, I seem to have noticed a slight problem with LP and old source binaries
<sil2100> I wanted to get the binaries of apport 2.17.2-0ubuntu1.3 from vivid-updates (so 2 versions behind)
<sil2100> It seems that when doing getPublishedBinaries() on the source_package_publishing_history of that package version, I get 0 binaries
<sil2100> But it seems LP has the binaries as per: https://launchpad.net/ubuntu/+source/apport/2.17.2-0ubuntu1.3/+build/7791083
<sil2100> Does anyone know the reasons?
<sil2100> I would need to perform a copy-package of that version's binary somewhere, but I can't because of this
<cjwatson> sil2100 asked me the same question in /query, and I answered there
<cjwatson> 11:49 <cjwatson> It'll copy the binaries, it's just that the client isn't smart enough to know how to tell you what they are
<cjwatson> 11:49 <cjwatson> Those binaries aren't published any more because they've been superseded
<cjwatson> 11:49 <cjwatson> But that shouldn't stop the actual server-side copy code dealing with them, and copy-package will pass the right parameter for that anyway
<cjwatson> 11:50 <cjwatson> So just go ahead and use --include-binaries there and don't worry about the inconsistent output
<sil2100> \o/
<sil2100> cjwatson: it you say so, then I suppose I believe you - I was a bit worried as I didn't want the sources being re-built
<sil2100> Thanks :)
<cjwatson> Yeah, it was certainly worth checking.
<cjwatson> wgrant: bmp-webhooks fixed even harder.
#launchpad-dev 2015-10-18
<blr> morning
<wgrant> cjwatson: Hum, why did you choose to include the virtualenv in the tarball?
<wgrant> That seems pretty dodgy.
<lifeless> they aren't typically portable...
<cjwatson> I thought the deployment target would be similar enough to the build environment in this very specific case that that wouldn't be a problem
<lifeless> so the two specific things that cause issues
<lifeless> a) the path is hard coded
<wgrant> cjwatson: What about eg. libgit2 versions?
<wgrant> Does the mojo build container have the right one?
<wgrant> Or paths.
<cjwatson> We can make it have the right one!
<lifeless> b) so library bumps can cause havoc (e.g. even with Python x.y rebuilds)
<cjwatson> But maybe paths are an issue.
<wgrant> Absolute paths are hardcoded in virtualenvs for no obvious reason.
<lifeless> we broke virtualenvs in 2.7 on some ubuntu releases with that one
<wgrant> Eh, Python minor releases always break a lot of things.
<cjwatson> I also wanted to avoid having the virtualenv construction code be duplicated.  But perhaps we could ship an appropriate script in the tarball and use it in turnip's Makefile.
<wgrant> SRUing them is entirely insane, so not a huge issue.
<cjwatson> (Or ship the Makefile, which we probably already do.)
<lifeless> cjwatson: so, I'd suggest an LXC container if you want a big blob to ship around
<lifeless> cjwatson: (or similar)
<cjwatson> Er
<cjwatson> I don't think you're entirely familiar with the environment here
<lifeless> cjwatson: otherwise construct in-place is much better for virtualenvs
<lifeless> cjwatson: I'm surely stale :)
<lifeless> cjwatson: happy to be educated, or to butt out :)
<cjwatson> I think nesting an LXC container (at least in some cases it will be nesting) is probably overkill.  This is in the context of a Mojo deployment
<cjwatson> But I'm happy to convert to constructing in-place - I didn't realise virtualenvs were quite *that* fragile
<wgrant> They're entirely unportable for a combination of good and terrible reasons.
<cjwatson> (Well, convert back, but with better code for it.)
<wgrant> The long-term solution is to use wheels.
<wgrant> Having a separate virtualenv build script probably makes sense.
<cjwatson> lifeless: more specifically the context is lp:~cjwatson/turnip/virtualenv
<cjwatson> That gets us closer to using wheels - I just didn't do that as a first step
<cjwatson> It wouldn't take much to have a "make compile" equivalent on top of that branch.
<cjwatson> lifeless: (sorry, the above came across sharper than intended, I was just having an "argh, not another container" moment)
<lifeless> cjwatson: no worries
<lifeless> cjwatson: I've merged your testtools branch now (sorry for the delay) - and filed the bug on twisted
<lifeless> cjwatson: I'll cut a release after the tokyo summit I think, unless you're blocked ?
<cjwatson> lifeless: I saw, thanks!  Not blocked at the moment, because I need to yak-shave extensively before we can upgrade testtools
<cjwatson> (i.e. convert Launchpad to pip)
<cjwatson> And I think I want to build up experience with something a bit smaller like turnip before finishing off my embryonic work on that
<blr> cjwatson: well, not quite as terrifying as juju-fying lp. That's going to be interesting.
<cjwatson> blr: Yes, quite.  Though working on this I begin to see some of the shape that that could take.
<blr> right
<cjwatson> Once we're on pip I think I could reasonably figure out how to jujuify importds, err, given about two or three weeks
#launchpad-dev 2016-10-22
<voidconf> anyone around?
#launchpad-dev 2017-10-19
<cjwatson> wgrant: thanks for the huge pile of reviews
<cjwatson> wgrant: Could either I have PyPI access to lazr.config and lazr.delegates, or you do a release of both?
<cjwatson> wgrant: I think my remaining MPs involved in non-trivial dependency stacks are https://code.launchpad.net/~cjwatson/meta-lp-deps/frontend-dependencies/+merge/331760 and https://code.launchpad.net/~cjwatson/launchpad/optimise-bin-py/+merge/331863
<dbwells> Hello, can anyone here answer a few questions about LP translation features?
<cjwatson> It's my weakest area of LP, but maybe.
<dbwells> I am a dev for the Evergreen ILS project (library software), and we have been using LP for translations for quite a few years, but only for master.
<dbwells> We are now trying to offer translations for each supported series.
<dbwells> When we update a string the LP interface, it updates for all series, as expected.  However, the last edit stamp of the non-edited series does not change, and nothing gets exported into the auto-export branch.  Are we missing something?
<dbwells> (I don't know if the "last edit" stamp and the lack of export are related, but it seems they might be.)
<cjwatson> That does seem likely.
<cjwatson> It would not surprise me to find that we are somewhere failing to update date_changed on all the related pofiles or similar, but I'm reeeeally unfamiliar with this and it would need setting up some kind of test environment and doing some experimentation.
<cjwatson> Could you file a bug with details of what you know so far?
<cjwatson> Including exactly where you're seeing the last edit stamp (that will be helpful for grepping)
<dbwells> Sure, I can do that.
<cjwatson> ta
<dbwells> cjwatson: After some further searching, I think the bug is already reported here: https://bugs.launchpad.net/launchpad/+bug/987199
<mup> Bug #987199: Automatic translation exports fail to commit latest translations  <lp-translations> <translations-branch> <Launchpad itself:Triaged> <Ubuntu Translations:New> <https://launchpad.net/bugs/987199>
<dbwells> I have added a small update to that bug, but it is 5 years old already :(
<cjwatson> Ah yes.  That wasn't long before most of the team who really knew Translations well was reassigned
<cjwatson> I guess we'd sort of need the inverse of POFile.getTranslationRows
 * tsimonq2 should now have time to finish up that MP this weekend now that Artful is released...
